Configure customer-managed-keys using C#
Azure Data Explorer encrypts all data in a storage account at rest. By default, data is encrypted with Microsoft-managed keys. For additional control over encryption keys, you can supply customer-managed keys to use for data encryption.
Customer-managed keys must be stored in an Azure Key Vault. You can create your own keys and store them in a key vault, or you can use an Azure Key Vault API to generate keys. The Azure Data Explorer cluster and the key vault must be in the same region, but they can be in different subscriptions. For a detailed explanation on customer-managed keys, see customer-managed keys with Azure Key Vault.
This article shows you how to configure customer-managed keys.
Configure Azure Key Vault
To configure customer-managed keys with Azure Data Explorer, you must set two properties on the key vault: Soft Delete and Do Not Purge. These properties aren't enabled by default. To enable these properties, perform Enabling soft-delete and Enabling Purge Protection in PowerShell or Azure CLI on a new or existing key vault. Only RSA keys of size 2048 are supported. For more information about keys, see Key Vault keys.
Note
Data encryption using customer managed keys is not supported on leader and follower clusters
Assign an identity to the cluster
To enable customer-managed keys for your cluster, first assign either a system-assigned or user-assigned managed identity to the cluster. You'll use this managed identity to grant the cluster permissions to access the key vault. To configure managed identities, see managed identities.
Create a new key vault
To create a new key vault using PowerShell, call New-AzKeyVault. The key vault that you use to store customer-managed keys for Azure Data Explorer encryption must have two key protection settings enabled, Soft Delete and Do Not Purge. Replace the placeholder values in brackets with your own values in example below.
$keyVault = New-AzKeyVault -Name <key-vault> `
-ResourceGroupName <resource_group> `
-Location <location> `
-EnableSoftDelete `
-EnablePurgeProtection
Configure the key vault access policy
Next, configure the access policy for the key vault so that the cluster has permissions to access it. In this step, you'll use either the system-assigned or user-assigned managed identity that you previously assigned to the cluster. To set the access policy for the key vault, call Set-AzKeyVaultAccessPolicy. Replace the placeholder values in brackets with your own values and use the variables defined in the previous examples.
For system assigned identity, use the cluster's principalId:
Set-AzKeyVaultAccessPolicy `
-VaultName $keyVault.VaultName `
-ObjectId $cluster.Identity.PrincipalId `
-PermissionsToKeys wrapkey,unwrapkey,get
For user assigned identity, use the identity's principalId:
Set-AzKeyVaultAccessPolicy `
-VaultName $keyVault.VaultName `
-ObjectId $userIdentity.Properties.PrincipalId `
-PermissionsToKeys wrapkey,unwrapkey,get
Create a new key
Next, create a new key in the key vault. To create a new key, call Add-AzKeyVaultKey. Replace the placeholder values in brackets with your own values and use the variables defined in the previous examples.
$key = Add-AzKeyVaultKey -VaultName $keyVault.VaultName -Name <key> -Destination 'Software'
Configure encryption with customer-managed keys
This section shows you how to configure customer-managed keys encryption using the Azure Data Explorer C# client.
Prerequisites
- Visual Studio 2019, download and use the free Visual Studio 2019 Community Edition. Enable Azure development during the Visual Studio setup.
- An Azure subscription. Create a free Azure account.
Install C# NuGet
Install the Azure Data Explorer (Kusto) NuGet package.
Install the Microsoft.IdentityModel.Clients.ActiveDirectory NuGet package for authentication.
Authentication
To run the examples in this article, create an Azure AD application and service principal that can access resources. You can add role assignment at the subscription scope and get the required Directory (tenant) ID, Application ID, and Client Secret.
Configure cluster
By default, Azure Data Explorer encryption uses Microsoft-managed keys. Configure your Azure Data Explorer cluster to use customer-managed keys and specify the key to associate with the cluster.
Update your cluster by using the following code:
var tenantId = "xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx";//Directory (tenant) ID var clientId = "xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx";//Application ID var clientSecret = "xxxxxxxxxxxxxx";//Client Secret var subscriptionId = "xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx"; var authenticationContext = new AuthenticationContext($"https://login.windows.net/{tenantId}"); var credential = new ClientCredential(clientId, clientSecret); var result = await authenticationContext.AcquireTokenAsync(resource: "https://management.core.windows.net/", clientCredential: credential); var credentials = new TokenCredentials(result.AccessToken, result.AccessTokenType); var kustoManagementClient = new KustoManagementClient(credentials) { SubscriptionId = subscriptionId }; var resourceGroupName = "testrg"; var clusterName = "mykustocluster"; var keyName = "myKey"; var keyVersion = "5b52b20e8d8a42e6bd7527211ae32654"; // Optional, leave as NULL for the latest version of the key. var keyVaultUri = "https://mykeyvault.vault.azure.net/"; var keyVaultIdentity = "/subscriptions/xxxxxxxx-xxxxx-xxxx-xxxx-xxxxxxxxx/resourcegroups/identityResourceGroupName/providers/Microsoft.ManagedIdentity/userAssignedIdentities/identityName"; // Use NULL if you want to use system assigned identity. var keyVaultProperties = new KeyVaultProperties(keyName, keyVaultUri, keyVersion, keyVaultIdentity); var clusterUpdate = new ClusterUpdate(keyVaultProperties: keyVaultProperties); await kustoManagementClient.Clusters.UpdateAsync(resourceGroupName, clusterName, clusterUpdate);Run the following command to check if your cluster was successfully updated:
kustoManagementClient.Clusters.Get(resourceGroupName, clusterName);If the result contains
ProvisioningStatewith theSucceededvalue, then your cluster was successfully updated.
Update the key version
When you create a new version of a key, you'll need to update the cluster to use the new version. First, call Get-AzKeyVaultKey to get the latest version of the key. Then update the cluster's key vault properties to use the new version of the key, as shown in Configure cluster.