Cluster Policies API 2.0

Is this page helpful?

Any additional feedback?

Feedback will be sent to Microsoft: By pressing the submit button, your feedback will be used to improve Microsoft products and services. Privacy policy.

Important

This feature is in Public Preview.

A cluster policy limits the ability to create clusters based on a set of rules. The policy rules limit the attributes or attribute values available for cluster creation. Cluster policies have ACLs that limit their use to specific users and groups.

Only admin users can create, edit, and delete policies. Admin users also have access to all policies.

For requirements and limitations on cluster policies, see Manage cluster policies.

Important

To access Databricks REST APIs, you must authenticate.

Cluster Policies API

The Cluster Policies API allows you to create, list, and edit cluster policies. Creation and editing is available to admins only. Listing can be performed by any user and is limited to policies accessible by that user.

Important

The Cluster Policies API requires a policy JSON definition to be passed within a JSON request in stringified form. In most cases this requires escaping of the quote characters.

In this section:

• Get
• List
• Create
• Edit
• Delete
• Data structures

Get

Endpoint	HTTP Method
2.0/policies/clusters/get	GET

Return a policy specification given a policy ID.

Example

curl --netrc -X GET \
https://adb-1234567890123456.7.azuredatabricks.net/api/2.0/policies/clusters/get \
--data '{ "policy_id": "ABCD000000000000" }' \
| jq .

{
  "policy_id": "ABCD000000000000",
  "name": "Test policy",
  "definition": "{\"spark_conf.spark.databricks.cluster.profile\":{\"type\":\"forbidden\",\"hidden\":true}}",
  "created_at_timestamp": 1600000000000
}

Request structure

Field Name	Type	Description
policy_id	STRING	The policy ID about which to retrieve information.

Response structure

Field Name	Type	Description
policy_id	STRING	Canonical unique identifier for the cluster policy.
name	STRING	Cluster policy name. This must be unique. Length must be between 1 and 100 characters.
definition	STRING	Policy definition JSON document expressed in Databricks Policy Definition Language. The JSON document must be passed as a string and cannot be simply embedded in the requests.
created_at_timestamp	INT64	Creation time. The timestamp (in millisecond) when this cluster policy was created.

List

Endpoint	HTTP Method
2.0/policies/clusters/list	GET

Return a list of policies accessible by the requesting user.

Example

curl --netrc -X GET \
https://adb-1234567890123456.7.azuredatabricks.net/api/2.0/policies/clusters/list \
--data '{ "sort_order": "DESC", "sort_column": "POLICY_CREATION_TIME" }' \
| jq .

{
  "policies": [
    {
      "policy_id": "ABCD000000000001",
      "name": "Empty",
      "definition": "{}",
      "created_at_timestamp": 1600000000002
    },
    {
      "policy_id": "ABCD000000000000",
      "name": "Test policy",
      "definition": "{\"spark_conf.spark.databricks.cluster.profile\":{\"type\":\"forbidden\",\"hidden\":true}}",
      "created_at_timestamp": 1600000000000
    }
  ],
  "total_count": 2
}

Request structure

Field Name	Type	Description
sort_order	ListOrder	The order direction to list the policies in; either ASC or DESC. Defaults to DESC.
sort_column	PolicySortColumn	The ClusterPolicy attribute to sort by. Defaults to POLICY_CREATION_TIME.

Response structure

Field Name	Type	Description
policies	An array of Policy	List of policies.
total_count	INT64	The total number of policies.

Create

Endpoint	HTTP Method
2.0/policies/clusters/create	POST

Create a new policy with a given name and definition.

Example

curl --netrc -X POST \
https://adb-1234567890123456.7.azuredatabricks.net/api/2.0/policies/clusters/create \
--data @create-cluster-policy.json

create-cluster-policy.json:

{
  "name": "Test policy",
  "definition": "{\"spark_conf.spark.databricks.cluster.profile\":{\"type\":\"forbidden\",\"hidden\":true}}"
}

{ "policy_id": "ABCD000000000000" }

Request structure

Field Name	Type	Description
name	STRING	Cluster policy name. This must be unique. Length must be between 1 and 100 characters.
definition	STRING	Policy definition JSON document expressed in Databricks Policy Definition Language. You must pass the JSON document as a string; it cannot be simply embedded in the requests.

Response structure

Field Name	Type	Description
policy_id	STRING	Canonical unique identifier for the cluster policy.

Edit

Endpoint	HTTP Method
2.0/policies/clusters/edit	POST

Update an existing policy. This may make some clusters governed by this policy invalid. For such clusters the next cluster edit must provide a confirming configuration, but otherwise they can continue to run.

Example

curl --netrc -X POST \
https://adb-1234567890123456.7.azuredatabricks.net/api/2.0/policies/clusters/edit \
--data @edit-cluster-policy.json

edit-cluster-policy.json:

{
  "policy_id": "ABCD000000000000",
  "name": "Test policy",
  "definition": "{\"spark_conf.spark.databricks.cluster.profile\":{\"type\":\"forbidden\",\"hidden\":true}}"
}

{}

Request structure

Field Name	Type	Description
policy_id	STRING	The ID of the policy to update. This field is required.
name	STRING	Cluster policy name. This must be unique. Length must be between 1 and 100 characters.
definition	STRING	Policy definition JSON document expressed in Databricks Policy Definition Language. You must pass the JSON document as a string; it cannot be simply embedded in the requests.

Delete

Endpoint	HTTP Method
2.0/policies/clusters/delete	POST

Delete a policy. Clusters governed by this policy can still run, but cannot be edited.

Example

curl --netrc -X POST \
https://adb-1234567890123456.7.azuredatabricks.net/api/2.0/policies/clusters/delete \
--data '{ "policy_id": "ABCD000000000000" }'

{}

Request structure

Field Name	Type	Description
policy_id	STRING	The ID of the policy to delete. This field is required.

Data structures

In this section:

• Policy
• PolicySortColumn

Policy

A cluster policy entity.

Field Name	Type	Description
policy_id	STRING	Canonical unique identifier for the cluster policy.
name	STRING	Cluster policy name. This must be unique. Length must be between 1 and 100 characters.
definition	STRING	Policy definition JSON document expressed in Databricks Policy Definition Language. You must pass the JSON document as a string; it cannot be simply embedded in the requests.
creator_user_name	STRING	Creator user name. The field won’t be included in the response if the user has already been deleted.
created_at_timestamp	INT64	Creation time. The timestamp (in millisecond) when this cluster policy was created.

PolicySortColumn

The sort order for the ListPolices request.

Name	Description
POLICY_CREATION_TIME	Sort result list by policy creation type.
POLICY_NAME	Sort result list by policy name.

Cluster Policy Permissions API

The Cluster Policy Permissions API enables you to set permissions on a cluster policy. When you grant CAN_USE permission on a policy to a user, the user will be able to create new clusters based on it. A user does not need the cluster_create permission to create new clusters.

Only admin users can set permissions on cluster policies.

In this section:

• Get permissions
• Get permission levels
• Add or modify permissions
• Set or delete permissions
• Data structures

Get permissions

Endpoint	HTTP Method
2.0/preview/permissions/cluster-policies/<clusterPolicyId>	GET

Example

curl --netrc -X GET \
https://adb-1234567890123456.7.azuredatabricks.net/api/2.0/preview/permissions/cluster-policies/ABCD000000000000 \
| jq .

{
  "object_id": "/cluster-policies/ABCD000000000000",
  "object_type": "cluster-policy",
  "access_control_list": [
    {
      "user_name": "someone@example.com",
      "all_permissions": [
        {
          "permission_level": "CAN_USE",
          "inherited": false
        }
      ]
    },
    {
      "group_name": "admins",
      "all_permissions": [
        {
          "permission_level": "CAN_USE",
          "inherited": true,
          "inherited_from_object": [
            "/cluster-policies/cluster-policies"
          ]
        }
      ]
    }
  ]
}

Request structure

Field Name	Type	Description
clusterPolicyId	STRING	The policy about which to retrieve permissions. This field is required.

Response structure

A Clusters ACL.

Get permission levels

Endpoint	HTTP Method
2.0/preview/permissions/cluster-policies/<clusterPolicyId>/permissionLevels	GET

Example

curl --netrc -X GET \
https://adb-1234567890123456.7.azuredatabricks.net/api/2.0/preview/permissions/cluster-policies/ABCD000000000000/permissionLevels \
| jq .

{
  "permission_levels": [
    {
      "permission_level": "CAN_USE",
      "description": "Can use the policy"
    }
  ]
}

Request structure

Field Name	Type	Description
clusterPolicyId	STRING	The policy about which to retrieve permission levels. This field is required.

Response structure

An array of PermissionLevel with associated description.

Add or modify permissions

Endpoint	HTTP Method
2.0/preview/permissions/cluster-policies/<clusterPolicyId>	PATCH

Example

curl --netrc -X PATCH \
https://adb-1234567890123456.7.azuredatabricks.net/api/2.0/preview/permissions/cluster-policies/ABCD000000000000 \
--data @add-cluster-policy-permissions.json \
| jq .

add-cluster-policy-permissions.json:

{
  "access_control_list": [
    {
      "user_name": "someone-else@example.com",
      "permission_level": "CAN_USE"
    }
  ]
}

{
  "object_id": "/cluster-policies/ABCD000000000000",
  "object_type": "cluster-policy",
  "access_control_list": [
    {
      "user_name": "mary@example.com",
      "all_permissions": [
        {
          "permission_level": "CAN_USE",
          "inherited": false
        }
      ]
    },
    {
      "user_name": "someone-else@example.com",
      "all_permissions": [
        {
          "permission_level": "CAN_USE",
          "inherited": false
        }
      ]
    },
    {
      "group_name": "admins",
      "all_permissions": [
        {
          "permission_level": "CAN_USE",
          "inherited": true,
          "inherited_from_object": [
              "/cluster-policies/"
          ]
        }
      ]
    }
  ]
}

Request structure

Field Name	Type	Description
clusterPolicyId	STRING	The policy about which to modify permissions. This field is required.

Request body

Field Name	Type	Description
access_control_list	Array of AccessControl	An array of access control lists.

Response body

A Clusters ACL.

Set or delete permissions

A PUT request replaces all direct permissions on the cluster policy object. You can make delete requests by making a GET request to retrieve the current list of permissions followed by a PUT request removing entries to be deleted.

Endpoint	HTTP Method
2.0/preview/permissions/cluster-policies/<clusterPolicyId>	PUT

Example

curl --netrc -X PUT \
https://adb-1234567890123456.7.azuredatabricks.net/api/2.0/preview/permissions/cluster-policies/ABCD000000000000 \
--data @set-cluster-policy-permissions.json \
| jq .

set-cluster-policy-permissions.json:

{
  "access_control_list": [
    {
      "user_name": "someone@example.com",
      "permission_level": "CAN_USE"
    }
  ]
}

{
  "object_id": "/cluster-policies/ABCD000000000000",
  "object_type": "cluster-policy",
  "access_control_list": [
    {
      "user_name": "someone@example.com",
      "all_permissions": [
        {
          "permission_level": "CAN_USE",
          "inherited": false
        }
      ]
    },
    {
      "group_name": "admins",
      "all_permissions": [
        {
          "permission_level": "CAN_USE",
          "inherited": true,
          "inherited_from_object": [
              "/cluster-policies/"
          ]
        }
      ]
    }
  ]
}

Request structure

Field Name	Type	Description
clusterPolicyId	STRING	The policy about which to set permissions. This field is required.

Request body

Field Name	Type	Description
access_control_list	Array of AccessControlInput	An array of access controls.

Response body

A Clusters ACL.

Data structures

In this section:

• Clusters ACL
• AccessControl
• Permission
• AccessControlInput
• PermissionLevel

Clusters ACL

Attribute Name	Type	Description
object_id	STRING	The ID of the ACL object, for example, ../cluster-policies/<clusterPolicyId>.
object_type	STRING	The Databricks ACL object type, for example, cluster-policy.
access_control_list	Array of AccessControl	The access controls set on the ACL object.

AccessControl

Attribute Name	Type	Description
user_name, group_name, OR service_principal_name	STRING	Name of the user/group or application ID of the service principal that has permissions set on the ACL object.
all_permissions	Array of Permission	List of all permissions set on this ACL object for a specific principal. Includes both permissions directly set on this ACL object and permissions inherited from an ancestor ACL object.

Permission

Attribute Name	Type	Description
permission_level	STRING	The name of the permission level.
inherited	BOOLEAN	True when the ACL permission is not set directly but inherited from an ancestor ACL object. False if set directly on the ACL object.
inherited_from_object	List[STRING]	The list of parent ACL object IDs that contribute to inherited permission on an ACL object. This is defined only if inherited is true.

AccessControlInput

An item representing an ACL rule applied to the principal (user, group, or service principal).

Attribute Name	Type	Description
user_name, group_name, OR service_principal_name	STRING	Name of the user/group or application ID of the service principal that has permissions set on the ACL object.
permission_level	STRING	The name of the permission level.

PermissionLevel

Permission level that you can set on a cluster policy.

Permission Level	Description
CAN_USE	Allow user to create clusters based on the policy. The user does not need the cluster create permission.