This feature is in Public Preview.
To enable provisioning to Azure Databricks using Azure Active Directory (Azure AD) you must create an enterprise application for each Azure Databricks workspace.
The way provisioning is configured is entirely separate from configuring authentication and conditional access for Azure Databricks workspaces. Authentication for Azure Databricks is handled automatically by Azure Active Directory, using the OpenID Connect protocol flow. You configure conditional access, which lets you create rules to require multi-factor authentication or restrict logins to local networks, at the service level.
- Your Azure Databricks account must have the Azure Databricks Premium Plan.
- Your Azure Active Directory account must be a Premium edition account.
- You must be a global administrator for the Azure Active Directory account.
There are two ways to configure provisioning:
- You can create an enterprise application in the Azure portal, and use that application for provisioning.
- If you have an existing application, you can modify its code to automate SCIM provisioning using Microsoft Graph. This removes the need for a separate provisioning application in the Azure Portal.
In the following examples, replace
<workspace-url> with the workspace URL of your Azure Databricks deployment.
In this section:
- Create the enterprise application and connect it to the Azure Databricks SCIM API
- Assign users and groups to the application
Create the enterprise application and connect it to the Azure Databricks SCIM API
Generate a personal access token in Azure Databricks and copy it. You provide this token to Azure Active Directory in a subsequent step.
Generate this token as an Azure Databricks admin who is not managed by the Azure Active Directory enterprise application. If the Azure Databricks admin user who owns the personal access token is deprovisioned using Azure Active Directory, the SCIM provisioning application will be disabled.
In your Azure portal, go to Azure Active Directory > Enterprise Applications.
Click + New Application above the application list. Under Add from the gallery, search for and select Azure Databricks SCIM Provisioning Connector.
Enter a Name for the application and click Add. Use a name that will help administrators find it, like
Under the Manage menu, click Provisioning.
Set Provisioning Mode to Automatic.
Enter the SCIM API endpoint URL. Append
/api/2.0/preview/scimto your workspace URL:
<workspace-url>with the workspace URL of your Azure Databricks deployment. See Get workspace, cluster, notebook, model, and job identifiers.
Set Secret Token to the Azure Databricks personal access token that you generated in step 1.
Click Test Connection and wait for the message that confirms that the credentials are authorized to enable provisioning.
Optionally, enter a notification email to receive notifications of critical errors with SCIM provisioning.
Assign users and groups to the application
Go to Manage > Provisioning.
Under Settings, set Scope to Sync only assigned users and groups.
Databricks recommends this option, which syncs only users and groups assigned to the enterprise application.
Azure Active Directory does not support the automatic provisioning of nested groups to Azure Databricks. Azure Active Directory can only read and provision users that are immediate members of the explicitly assigned group. As a workaround, explicitly assign (or otherwise scope in) the groups that contain the users who need to be provisioned. For more information, see this FAQ.
To start synchronizing Azure Active Directory users and groups to Azure Databricks, click the Provisioning Status toggle.
Test your provisioning setup:
- Go to Manage > Users and groups.
- Add some users and groups. Click Add user, select the users and groups, and click the Assign button.
- Wait a few minutes and check that the users and groups exist in your Azure Databricks workspace.
In the future, users and groups that you add and assign are automatically provisioned when Azure Active Directory schedules the next sync.
Do not assign the Azure Databricks admin whose personal access token was used to configure the Azure Databricks SCIM Provisioning Connector application.
Automate SCIM provisioning using Microsoft Graph
Microsoft Graph includes authentication and authorization libraries that you can integrate into your application to automate provisioning of users and groups to Azure Databricks, instead of configuring a SCIM provisioning connector application.
- Follow the instructions for registering an application with Microsoft Graph. Make a note of the Application ID and the Tenant ID for the application
- Go to the applications’s Overview page. On that page:
- Configure a client secret for the application, and make a note of the secret.
- Grant the application these permissions:
- Ask an Azure Active Directory administrator to grant admin consent.
- Update your application’s code to add support for Microsoft Graph.
- Users and groups that existed in Azure Databricks prior to enabling provisioning exhibit the following behavior upon provisioning sync:
- Are merged if they also exist in Azure Active Directory
- Are ignored if they don’t exist in Azure Active Directory
- User permissions that are assigned individually and are duplicated through membership in a group remain after the group membership is removed for the user.
- Users removed from an Azure Databricks workspace directly, using the Azure Databricks Admin console:
- Lose access to that Azure Databricks workspace but may still have access to other Azure Databricks workspaces.
- Will not be synced again using Azure Active Directory provisioning, even if they remain in the enterprise application.
- The initial Azure Active Directory sync is triggered immediately after you enable provisioning. Subsequent syncs are triggered every 20-40 minutes, depending on the number of users and groups in the application. See Provisioning summary report in the Azure Active Directory documentation.
- You cannot update the username or email address of a Azure Databricks user.
adminsgroup is a reserved group in Azure Databricks and cannot be removed.
- Groups cannot be renamed in Azure Databricks; do not attempt to rename them in Azure Active Directory.
- You can use the Azure Databricks Groups API or the Groups UI to get a list of members of any Azure Databricks group.
Users and groups do not sync
- If you are using the Azure Databricks SCIM Provisioning Connector application: In the Azure Databricks admin console, verify that the Azure Databricks user whose personal access token is being used by the Azure Databricks SCIM Provisioning Connector application is still an admin user in Azure Databricks and that the token is still valid.
- Do not attempt to sync nested groups, which are not supported by Azure Active Directory automatic provisioning. For more information, see this FAQ.
After initial sync, the users and groups stop syncing
If you are using the Azure Databricks SCIM Provisioning Connector application: After the initial sync, Azure Active Directory does not sync immediately after you change user or group assignments. It schedules a sync with the application after a delay, based on the number of users and groups. To request an immediate sync, go to Manage > Provisioning for the enterprise application and select Clear current state and restart synchronization.
Azure Active Directory provisioning service IP range not accessible
The Azure Active Directory provisioning service operates under specific IP ranges. If you need to restrict network access, you must allow traffic from the IP addresses for
AzureActiveDirectory in this IP range file. For more information, see IP Ranges.