Cluster Policies APIs

Important

This feature is in Public Preview.

A cluster policy limits the ability to create clusters based on a set of rules. The policy rules limit the attributes or attribute values available for cluster creation. Cluster policies have ACLs that limit their use to specific users and groups.

Only admin users can create, edit, and delete policies. Admin users also have access to all policies.

For requirements and limitations on cluster policies, see Manage cluster policies.

Important

To access Databricks REST APIs, you must authenticate.

Cluster Policies API

The Cluster Policies API allows you to create, list, and edit cluster policies. Creation and editing is available to admins only. Listing can be performed by any user and is limited to policies accessible by that user.

Important

The Cluster Policies API requires a policy JSON definition to be passed within a JSON request in stringified form. In most cases this requires escaping of the quote characters.

In this section:

Get

Endpoint HTTP Method
2.0/policies/clusters/get GET

Return a policy specification given a policy ID.

Request structure

Field Name Type Description
policy_id STRING The policy ID about which to retrieve information.

Response structure

Field Name Type Description
policy_id STRING Canonical unique identifier for the cluster policy.
name STRING Cluster policy name. This must be unique. Length must be between 1 and 100 characters.
definition STRING Policy definition JSON document expressed in Databricks Policy Definition Language. The JSON document must be passed as a string and cannot be simply embedded in the requests.
created_at_timestamp INT64 Creation time. The timestamp (in millisecond) when this cluster policy was created.

List

Endpoint HTTP Method
2.0/policies/clusters/list GET

Return a list of policies accessible by the requesting user.

Request structure

Field Name Type Description
sort_order ListOrder The order direction to list the policies in; either ASC or DESC. Defaults to DESC.
sort_column PolicySortColumn The ClusterPolicy attribute to sort by. Defaults to CREATION_TIME.

Response structure

Field Name Type Description
policies An array of Policy List of policies.
total_count INT64 The total number of policies.
Example
{
  "policies": [
    {
      "policy_id": "ABCD000000000000",
      "name": "Test policy",
      "definition": "{\"spark_conf.spark.databricks.cluster.profile\":{\"type\":\"forbidden\",\"hidden\":true}}",
      "created_at_timestamp": 1600000000000
    }
    {
      "policy_id": "ABCD000000000001",
      "name": "Empty",
      "definition": "{}",
      "created_at_timestamp": 1600000000002
    }
  ],
  "total_count": 1
}

Create

Endpoint HTTP Method
2.0/policies/clusters/create POST

Create a new policy with a given name and definition.

Request structure

Field Name Type Description
name STRING Cluster policy name. This must be unique. Length must be between 1 and 100 characters.
definition STRING Policy definition JSON document expressed in Databricks Policy Definition Language. You must pass the JSON document as a string; it cannot be simply embedded in the requests.
Example
{
  "name": "Example Policy",
  "definition": "{\"spark_version\":{\"type\":\"fixed\",\"value\":\"next-major-version-scala2.12\",\"hidden\":true}}"
}

Response structure

Field Name Type Description
policy_id STRING Canonical unique identifier for the cluster policy.
Example
{
  "policy_id": "ABCD000000000000",
}

Edit

Endpoint HTTP Method
2.0/policies/clusters/edit POST

Update an existing policy. This may make some clusters governed by this policy invalid. For such clusters the next cluster edit must provide a confirming configuration, but otherwise they can continue to run.

Request structure

Field Name Type Description
policy_id STRING The ID of the policy to update. This field is required.
name STRING Cluster policy name. This must be unique. Length must be between 1 and 100 characters.
definition STRING Policy definition JSON document expressed in Databricks Policy Definition Language. You must pass the JSON document as a string; it cannot be simply embedded in the requests.
Example
{
  "policy_id": "ABCD000000000000",
  "name": "Example Policy",
  "definition": "{\"spark_version\":{\"type\":\"fixed\",\"value\":\"next-major-version-scala2.12\",\"hidden\":true}}"
}

Delete

Endpoint HTTP Method
2.0/policies/clusters/delete POST

Delete a policy. Clusters governed by this policy can still run, but cannot be edited.

Request structure

Field Name Type Description
policy_id STRING The ID of the policy to delete. This field is required.
Example
{
  "policy_id": "ABCD000000000000"
}

Data structures

In this section:

Policy

A cluster policy entity.

Field Name Type Description
policy_id STRING Canonical unique identifier for the cluster policy.
name STRING Cluster policy name. This must be unique. Length must be between 1 and 100 characters.
definition STRING Policy definition JSON document expressed in Databricks Policy Definition Language. You must pass the JSON document as a string; it cannot be simply embedded in the requests.
creator_user_name STRING Creator user name. The field won’t be included in the response if the user has already been deleted.
created_at_timestamp INT64 Creation time. The timestamp (in millisecond) when this cluster policy was created.

PolicySortColumn

The sort order for the ListPolices request.

Name Description
POLICY_CREATION_TIME Sort result list by policy creation type.
POLICY_NAME Sort result list by policy name.

Cluster Policy Permissions API

The Cluster Policy Permissions API enables you to set permissions on a cluster policy. When you grant CAN_USE permission on a policy to a user, the user will be able to create new clusters based on it. A user does not need the cluster_create permission to create new clusters.

Only admin users can set permissions on cluster policies.

In the following endpoints, <basepath> = /api/2.0/preview.

In this section:

Get permissions

Endpoint HTTP Method
<basepath>/permissions/cluster-policies/<clusterPolicyId>/permissionLevels GET

Request structure

Field Name Type Description
clusterPolicyId STRING The policy about which to retrieve permissions. This field is required.

Response structure

A Clusters ACL.

Example response

{
  "object_id": "/cluster-policies/D55CAFDD8E00002B",
  "object_type": "cluster-policy",
  "access_control_list": [
    {
      "user_name": "user@mydomain.com",
      "all_permissions": [
        {
          "permission_level": "CAN_USE",
          "inherited": false
        },
      ]
    },
    {
      "group_name": "admins",
      "all_permissions": [
        {
          "permission_level": "CAN_USE",
          "inherited": true,
          "inherited_from_object": [
              "/cluster-policies/"
          ]
        }
      ]
    },
  ]
}

Add or modify permissions

Endpoint HTTP Method
<basepath>/permissions/cluster-policies/<clusterPolicyId> PATCH

Request structure

Field Name Type Description
clusterPolicyId STRING The policy about which to modify permissions. This field is required.

Request body

Field Name Type Description
access_control_list Array of AccessControl An array of access control lists.

Response body

Same as a GET call on <clusterPolicyId>, returns back modified permissions for cluster.

Set or delete permissions

A PUT request replaces all direct permissions on the cluster policy object. You can make delete requests by making a GET request to retrieve the current list of permissions followed by a PUT request removing entries to be deleted.

Endpoint HTTP Method
<basepath>/permissions/cluster-policies/<clusterPolicyId> PUT

Request structure

Field Name Type Description
clusterPolicyId STRING The policy about which to set permissions. This field is required.

Request body

Field Name Type Description
access_control_list Array of AccessControlInput An array of access controls.

Response body

Same as a GET call on <clusterPolicyId>, returns back modified permissions for cluster.

Data structures

In this section:

Clusters ACL

Attribute Name Type Description
object_id STRING The ID of the ACL object, for example, ../cluster-policies/<clusterPolicyId>.
object_type STRING The Databricks ACL object type, for example, cluster-policy.
access_control_list Array of AccessControl The access controls set on the ACL object.

AccessControl

Attribute Name Type Description
user_name OR group_name STRING Name of the principal (user or group) that has permissions set on the ACL object.
all_permissions Array of Permission List of all permissions set on this ACL object for a specific principal. Includes both permissions directly set on this ACL object and permissions inherited from an ancestor ACL object.

Permission

Attribute Name Type Description
permission_level STRING The name of the permission level.
inherited BOOLEAN True when the ACL permission is not set directly but inherited from an ancestor ACL object. False if set directly on the ACL object.
inherited_from_object List[STRING] The list of parent ACL object IDs that contribute to inherited permission on an ACL object. This is defined only if inherited is true.

AccessControlInput

An item representing an ACL rule applied to the principal (user or group).

Attribute Name Type Description
user_name OR group_name STRING Name of the principal (user or group) that has permissions set on the ACL object.
permission_level STRING The name of the permission level.

PermissionLevel

Permission level that you can set on a cluster policy.

Permission Level Description
CAN_USE Allow user to create clusters based on the policy. The user does not need the cluster create permission.