Manage service principals

Important

This feature is in Public Preview.

A service principal is an identity created for use with automated tools, running jobs, and applications. You can restrict a service principal’s access to resources using permissions, in the same way as an Azure Databricks user. You can add entitlements to a service principal. You can add a service principal to a group, including the admins group. Unlike an Azure Databricks user, a service principal is an API-only identity; it cannot be used to access the Azure Databricks UI.

For security reasons, Databricks recommends using service principals to give automated tools and scripts API-only access to Azure Databricks resources.

Create a service principal

To use service principals on Azure Databricks, an admin user must create a new Azure Active Directory (Azure AD) application and then add it to the Azure Databricks workspace to use as a service principal. Service principals cannot be created directly within Azure Databricks at this time. To create an Azure AD service principal, follow these instructions:

  1. Sign in to the Azure portal.

    Note

    The portal to use is different depending on whether your Azure AD application runs in the Azure public cloud or in a national or sovereign cloud. For more information, see National clouds.

  2. If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to provision the service principal.

  3. Search for and select Azure Active Directory.

  4. Within Manage, click App registrations > New registration.

  5. For Name, enter a name for the application.

  6. In the Supported account types section, select Accounts in this organizational directory only (Single tenant).

  7. Click Register.

  8. Within Manage, click Certificates & secrets.

  9. On the Client secrets tab, click New client secret.

    New client secret

  10. In the Add a client secret pane, for Description, enter a description for the client secret.

  11. For Expires, select an expiry time period for the client secret, and then click Add.

  12. Copy and store the client secret’s Value in a secure place, as this client secret is the password for your application.

  13. On the application page’s Overview page, in the Essentials section, copy the following values:

    • Application (client) ID
    • Directory (tenant) ID

    Azure registered app overview

Finally, add the Azure Active Directory application to the Azure Databricks workspace by using the Add service principal endpoint of the SCIM API 2.0 (ServicePrincipals) API.

Manage access tokens for a service principal

/token-management/on-behalf-of/tokens` operation in the token management REST API. An administrator can also list personal access tokens and delete them using the same API.

To authenticate a service principal to APIs on Azure Databricks, an administrator can create an Azure AD access token on behalf of the service principal. The Azure AD access token can be used in place of a user’s Databricks Access Token to call Databricks REST APIs. Create an Azure AD access token by following these instructions:

  1. Gather the following information:

    Parameter Description
    Tenant ID The Directory (tenant) ID for the application registered in Azure AD.
    Client ID The Application (client) ID for the application registered in Azure AD.
    Client secret The Value of the client secret for the application registered in Azure AD.
  2. Use the preceding information along with curl to get the Azure AD access token.

    curl -X POST -H 'Content-Type: application/x-www-form-urlencoded' \
    https://login.microsoftonline.com/<tenant-id>/oauth2/v2.0/token \
    -d 'client_id=<client-id>' \
    -d 'grant_type=client_credentials' \
    -d 'scope=2ff814a6-3304-4ab8-85cb-cd0e6f879c1d%2F.default' \
    -d 'client_secret=<client-secret>'
    

    Replace:

    • <tenant-id> with the registered application’s tenant ID.
    • <client-id> with the registered application’s client ID.
    • <client-secret> with the registered application’s client secret value.

    Do not change the value of the scope parameter. It represents the programmatic ID for Azure Databricks (2ff814a6-3304-4ab8-85cb-cd0e6f879c1d) along with the default scope (/.default, URL-encoded as %2f.default).

    For example:

    curl -X POST -H 'Content-Type: application/x-www-form-urlencoded' \
    https://login.microsoftonline.com/a1bc2d34-5e67-8f89-01ab-c2345d6c78de/oauth2/v2.0/token \
    -d 'client_id=12a34b56-789c-0d12-e3fa-b456789c0123' \
    -d 'grant_type=client_credentials' \
    -d 'scope=2ff814a6-3304-4ab8-85cb-cd0e6f879c1d%2F.default' \
    -d 'client_secret=abc1D~Ef...2ghIJKlM3'
    

    The Azure AD access token is in the access_token value within the output of the call.

Note

It’s not possible to create, list, or manage a token for a service principal from within the Azure Databricks UI.

Manage entitlements for a service principal

An entitlement is a property that allows a user, service principal, or group to interact with Azure Databricks in a specified way. In the following table, each entitlement’s UI and API name is shown.

Entitlement name (UI) Entitlement name (API) Default Description
Workspace access workspace-access Granted by default. When granted to a user or service principal, they can access the Data Science & Engineering workspace and Databricks Machine Learning.

Can’t be removed from workspace administrators.
Databricks SQL access databricks-sql-access Granted by default. When granted to a user or service principal, they can access Databricks SQL.
Allow unrestricted cluster creation allow-cluster-create Not granted to users or service principals by default. When granted to a user or service principal, they can create clusters. You can restrict access to existing clusters using cluster-level permissions.

Can’t be removed from admin users.
allow-instance-pool-create allow-instance-pool-create Can’t be granted to individual users or service principals. When granted to a group, its members can create instance pools.

Can’t be removed from workspace administrators.

To add or remove an entitlement for a service principal, use the SCIM API 2.0 (ServicePrincipals) API.