Enabling Azure Data Lake Storage Credential Passthrough

Azure Databricks supports a type of cluster configuration, called Azure Data Lake Storage credential passthrough, that allows users to authenticate to Azure Data Lake Storage from Azure Databricks clusters using the same Azure Active Directory identity that they use to log into Azure Databricks. When a cluster is enabled for Azure Data Lake Storage credential passthrough, commands run on that cluster can read and write data in Azure Data Lake Storage without requiring users to configure service principal credentials to access the storage. The credentials are set automatically, based on the user initiating the action.

When users run notebooks that access Azure Data Lake Storage using Azure Data Lake Storage credential passthrough, all of the data they access must be stored entirely in Azure Data Lake Storage. Azure Data Lake Storage credential passthrough does not support filesystems other than Azure Data Lake Storage.

As an administrator, you must make sure that user permissions are set so that the Azure Active Directory user who logs into Azure Databricks has read and write access to the proper Azure Data Lake Storage data.

For information about enabling clusters for Azure Data Lake Storage credential passthrough and for reading and writing data in Azure Data Lake Storage, see Authenticate to Azure Data Lake Storage using Azure Active Directory Credentials.

Note

It is safe to share Azure Data Lake Storage credential passthrough clusters with other users. Users are isolated from each other and are not be able to read or use each other’s credentials.