Configure customer-managed keys for DBFS using PowerShell
This feature is available only in the Azure Databricks Premium Plan.
You can use PowerShell to configure your own encryption key to encrypt the DBFS root storage account. You must use Azure Key Vault to store the key.
For more information about customer-managed keys for DBFS, see Configure customer-managed keys for DBFS root.
Install the Azure Databricks PowerShell module
Prepare a new or existing Azure Databricks workspace for encryption
Replace the placeholder values in brackets with your own values. The
<workspace-name> is the resource name as displayed in the Azure portal.
Prepare encryption when you create a workspace:
$workSpace = New-AzDatabricksWorkspace -Name <workspace-name> -Location <workspace-location> -ResourceGroupName <resource-group> -Sku premium -PrepareEncryption
Prepare an existing workspace for encryption:
$workSpace = Update-AzDatabricksWorkspace -Name <workspace-name> -ResourceGroupName <resource-group> -PrepareEncryption
For more information about PowerShell cmdlets for Azure Databricks workspaces, see the Az.Databricks reference.
Create a new key vault
The key vault that you use to store customer-managed keys for default (root) DBFS must have two key protection settings enabled, Soft Delete and Purge Protection.
In version 2.0.0 and later of the
Az.KeyVault module, soft delete is enabled by default when you create a new key vault.
The following example creates a new key vault with the Soft Delete and Purge Protection properties enabled. Replace the placeholder values in brackets with your own values.
$keyVault = New-AzKeyVault -Name <key-vault> ` -ResourceGroupName <resource-group> ` -Location <location> ` -EnablePurgeProtection
To learn how to enable Soft Delete and Purge Protection on an existing key vault with PowerShell, see “Enabling soft-delete” and “Enabling Purge Protection” in How to use Key Vault soft-delete with PowerShell.
Configure the key vault access policy
Set the access policy for the key vault so that the Azure Databricks workspace has permission to access it, using Set-AzKeyVaultAccessPolicy.
Set-AzKeyVaultAccessPolicy ` -VaultName $keyVault.VaultName ` -ObjectId $workSpace.StorageAccountIdentity.PrincipalId ` -PermissionsToKeys wrapkey,unwrapkey,get
Create a new key
Create a new key in the key vault using the Add-AzKeyVaultKey cmdlet. Replace the placeholder values in brackets with your own values.
$key = Add-AzKeyVaultKey -VaultName $keyVault.VaultName -Name <key> -Destination 'Software'
DBFS root storage supports RSA and RSA-HSM keys of sizes 2048, 3072 and 4096. For more information about keys, see About Key Vault keys.
Configure DBFS encryption with customer-managed keys
Configure your Azure Databricks workspace to use the key you created in your Azure Key Vault. Replace the placeholder values in brackets with your own values.
Update-AzDatabricksWorkspace -ResourceGroupName <resource-group> ` -Name <workspace-name> -EncryptionKeySource Microsoft.Keyvault ` -EncryptionKeyName $key.Name ` -EncryptionKeyVersion $key.Version ` -EncryptionKeyVaultUri $keyVault.VaultUri
Disable customer-managed keys
When you disable customer-managed keys, your storage account is once again encrypted with Microsoft-managed keys.
Replace the placeholder values in brackets with your own values and use the variables defined in the previous steps.
Update-AzDatabricksWorkspace -Name <workspace-name> -ResourceGroupName <resource-group> -EncryptionKeySource Default
Submit and view feedback for