Connect to Azure Data Explorer from Azure Databricks
Azure Databricks is an Apache Spark-based analytics platform that's optimized for the Microsoft Azure platform. This article shows you how to use Azure Databricks to access data from Azure Data Explorer. There are several ways to authenticate with Azure Data Explorer, including a device login and an Azure Active Directory (Azure AD) app.
Create an Azure Databricks workspace. Under Azure Databricks Service, in the Pricing Tier drop-down list, select Premium. This selection enables you to use Azure Databricks secrets to store your credentials and reference them in notebooks and jobs.
Create a cluster in Azure Databricks with the default settings.
Install the Kusto Spark connector on your Azure Databricks cluster
To install the spark-kusto-connector on your Azure Databricks cluster:
- Go to your Azure Databricks workspace and create a library.
- Search for the spark-kusto-connector package on Maven Central, install the latest version, and attach to your cluster.
Connect to Azure Data Explorer by using a device authentication
Connect to Azure Data Explorer by using an Azure AD app
Create Azure AD app by provisioning an Azure AD application.
Grant access to your Azure AD app in your Azure Data Explorer database as follows:
.set database <DB Name> users ('aadapp=<AAD App ID>;<AAD Tenant ID>') 'AAD App to connect Spark to ADX
your database name
AAD App ID
your Azure AD app ID
AAD Tenant ID
your Azure AD tenant ID
Find your Azure AD tenant ID
To authenticate an application, Azure Data Explorer uses your Azure AD tenant ID. To find your tenant ID, use the following URL. Substitute your domain for YourDomain.
For example, if your domain is contoso.com, the URL is: https://login.windows.net/contoso.com/.well-known/openid-configuration/. Select this URL to see the results. The first line is as follows:
Your tenant ID is
Store and secure your Azure AD app ID and key (optional)
Store and secure your Azure AD app ID and key by using Azure Databricks secrets as follows:
Configure the secrets by using the following sample commands:
databricks secrets create-scope --scope adx
databricks secrets put --scope adx --key myaadappid
databricks secrets put --scope adx --key myaadappkey
databricks secrets list --scope adx
- Sample code.
- Update the placeholder values with your cluster name, database name, table name, Azure AD tenant ID, AAD App ID, and AAD App Key. If you are storing your credentials in databricks secrets store, update the code accordingly to retrieve values from dbutils.