Add Azure Blob storage as a data source

[This topic is pre-release documentation and is subject to change.]

If you use Azure Storage as a data source for Dynamics 365 for Customer Insights, you must provide the connection keys to your storage account through the Azure Key Vault. This ensures security for your key.

Set up your Azure Key Vault

To use Azure Storage as a data source for Customer Insights, you can configure your Azure Key Vault by using the connection keys for your storage account.

  1. Set up Azure PowerShell with the instructions at How to install and configure Azure PowerShell.

  2. Create a PowerShell script file named SetupAzureKeyVault.ps1

  3. Copy the following script into the file, and update the fields at the beginning of the script as described in the table.

  4. Run the script from the PowerShell prompt.

  # ------------------------------------------------------------------- #
  $keyvaultName = '<provide-a-keyvault-name>'
  $secretName = '<name-the-secret-for-your-storage-account>'
  $resourceGroupName = '<specify-a-resource-group-name>'
  $location = '<Azure-region-of-your-storage-account>'
  $connectionString = '<provide-the-connection-string>'
  $tags = @{'AllowedACIHubs'='<HubName1>;<HubName2>'}
  # ------------------------------------------------------------------- #


  # Login to Azure account.
  Login-AzureRmAccount

  # Create new resource group if not exists.
  if(!(Get-AzureRmResourceGroup -Name $resourceGroupName -Location $location)){
    New-AzureRmResourceGroup -Name $resourceGroupName -Location $location
  }

  # Create new key vault if not exists.
  if(!(Get-AzureRmKeyVault -VaultName $keyvaultName -ResourceGroupName $resourceGroupName)){
    New-AzureRmKeyVault -VaultName $keyvaultName -ResourceGroupName $resourceGroupName -Location $location
  }

  # Convert connection string
  $secretValue = ConvertTo-SecureString $connectionString -AsPlainText -Force
  Set-AzureKeyVaultSecret -VaultName $keyvaultName -Name $secretName -SecretValue $secretValue -Tags $tags

  # Authorize application to access key vault.
  $servicePrincipal = '38c77d00-5fcb-4cce-9d93-af4738258e3c'
  Set-AzureRmKeyVaultAccessPolicy -VaultName $keyvaultName -ServicePrincipalName $servicePrincipal -PermissionsToKeys all -PermissionsToSecrets all

  # Get and display secret url.
  $secret = Get-AzureKeyVaultSecret -VaultName $keyvaultName -Name $secretName
  Write-Host "Connection key vault URL is "$secret.id"" 

The values for the fields at the beginning of the script are described in the following table.

Value Description
$keyvaultName Enter a name for a key vault. Azure Key Vault helps safeguard storage account keys.
$secretName Enter a name for the secret to be added to the key vault.
$resourceGroupName Enter a resource group name. If the resource group does not exist, it will be created.
$location Enter the region for your Azure storage account.
$connectionString Enter the Azure storage account key connection string.
$tags Enter the Customer Insights hubs to be used for data modeling.
  1. The script will ask for your Azure account credentials.

  2. Save the Key Vault URI printed at the end of the script.

Add an Azure Blob container as a data source

To create an Azure Blob container as a data source by using the Azure portal:

  1. On your Customer Insights hub blade, select Data Sources.

  2. On the All Data Sources blade, select Add.

  3. On the New Data Source blade, select Data Source Type.

  4. On the Data Source Type blade, select AzureBlob.

  5. Enter a Name for the data source.

  6. Paste the Connection KeyVault URL you saved in step 6 of "Set up your Azure Key Vault" section earlier in this topic.

  7. Select Create to provision the data source.

    Note

    After the data source has been provisioned, you must first finish the steps in Data modeling and come back to the data source to create file mappings between CSV files in the Azure Blob storage containers and data modeled entities (interactions).

    After finishing data modeling steps, return and continue with the following steps to complete the data source setup process.

  8. You'll now define how your source data files map to interactions. Right-click the data source you added, and then select Mappings.

  9. On the File Mappings blade, select New File Mapping.

  10. In the Source Folder dialog box, enter the folder in the Azure Storage account from where you want to pick up the CSV file.

    You can find this by opening your storage account, going to Overview > Blobs and noting the name of the storage container.

  11. Specify a Filename Pattern for the source file. This can include one wildcard character, typically at the end, for instance BankDeposit*.

  12. Choose a Target Interaction type for which you want to insert or update the interactions, from the rows in the specified CSV file.

  13. Choose an Error Management policy that you want to apply for this file mapping.

  14. Choose an Error Limit for each processing instance.

  15. Choose a Frequency for the polling interval.

  16. Choose a Polling Interval for this file mapping.

  17. Choose an On Complete policy.

  18. Choose Create to provision the file mapping.

  19. For your data source files, the content of the files need to map to the fields and attributes that are defined in the interactions. For each file mapping, right-click the file mapping, and then select Attribute Mappings.

  20. Select the Select Sample File field, or select the folder icon next to it, to upload a sample CSV file.

  21. Select Save.