AzureDatabricksLinkedService Class

public class AzureDatabricksLinkedService
extends LinkedService

Azure Databricks linked service.

Constructor Summary

Constructor Description
AzureDatabricksLinkedService()

Creates an instance of AzureDatabricksLinkedService class.

Method Summary

Modifier and Type Method and Description
SecretBase getAccessToken()

Get the accessToken property: Access token for databricks REST API.

Object getAuthentication()

Get the authentication property: Required to specify MSI, if using Workspace resource id for databricks REST API.

CredentialReference getCredential()

Get the credential property: The credential reference containing authentication information.

Object getDomain()

Get the domain property: .azuredatabricks.net, domain name of your Databricks deployment.

Object getEncryptedCredential()

Get the encryptedCredential property: The encrypted credential used for authentication.

Object getExistingClusterId()

Get the existingClusterId property: The id of an existing interactive cluster that will be used for all runs of this activity.

Object getInstancePoolId()

Get the instancePoolId property: The id of an existing instance pool that will be used for all runs of this activity.

Map<String,Object> getNewClusterCustomTags()

Get the newClusterCustomTags property: Additional tags for cluster resources.

Object getNewClusterDriverNodeType()

Get the newClusterDriverNodeType property: The driver node type for the new job cluster.

Object getNewClusterEnableElasticDisk()

Get the newClusterEnableElasticDisk property: Enable the elastic disk on the new cluster.

Object getNewClusterInitScripts()

Get the newClusterInitScripts property: User-defined initialization scripts for the new cluster.

Object getNewClusterLogDestination()

Get the newClusterLogDestination property: Specify a location to deliver Spark driver, worker, and event logs.

Object getNewClusterNodeType()

Get the newClusterNodeType property: The node type of the new job cluster.

Object getNewClusterNumOfWorker()

Get the newClusterNumOfWorker property: If not using an existing interactive cluster, this specifies the number of worker nodes to use for the new job cluster or instance pool.

Map<String,Object> getNewClusterSparkConf()

Get the newClusterSparkConf property: A set of optional, user-specified Spark configuration key-value pairs.

Map<String,Object> getNewClusterSparkEnvVars()

Get the newClusterSparkEnvVars property: A set of optional, user-specified Spark environment variables key-value pairs.

Object getNewClusterVersion()

Get the newClusterVersion property: If not using an existing interactive cluster, this specifies the Spark version of a new job cluster or instance pool nodes created for each run of this activity.

Object getPolicyId()

Get the policyId property: The policy id for limiting the ability to configure clusters based on a user defined set of rules.

Object getWorkspaceResourceId()

Get the workspaceResourceId property: Workspace resource id for databricks REST API.

AzureDatabricksLinkedService setAccessToken(SecretBase accessToken)

Set the accessToken property: Access token for databricks REST API.

AzureDatabricksLinkedService setAnnotations(List<Object> annotations)

Set the annotations property: List of tags that can be used for describing the linked service.

AzureDatabricksLinkedService setAuthentication(Object authentication)

Set the authentication property: Required to specify MSI, if using Workspace resource id for databricks REST API.

AzureDatabricksLinkedService setConnectVia(IntegrationRuntimeReference connectVia)

Set the connectVia property: The integration runtime reference.

AzureDatabricksLinkedService setCredential(CredentialReference credential)

Set the credential property: The credential reference containing authentication information.

AzureDatabricksLinkedService setDescription(String description)

Set the description property: Linked service description.

AzureDatabricksLinkedService setDomain(Object domain)

Set the domain property: .azuredatabricks.net, domain name of your Databricks deployment.

AzureDatabricksLinkedService setEncryptedCredential(Object encryptedCredential)

Set the encryptedCredential property: The encrypted credential used for authentication.

AzureDatabricksLinkedService setExistingClusterId(Object existingClusterId)

Set the existingClusterId property: The id of an existing interactive cluster that will be used for all runs of this activity.

AzureDatabricksLinkedService setInstancePoolId(Object instancePoolId)

Set the instancePoolId property: The id of an existing instance pool that will be used for all runs of this activity.

AzureDatabricksLinkedService setNewClusterCustomTags(Map<String,Object> newClusterCustomTags)

Set the newClusterCustomTags property: Additional tags for cluster resources.

AzureDatabricksLinkedService setNewClusterDriverNodeType(Object newClusterDriverNodeType)

Set the newClusterDriverNodeType property: The driver node type for the new job cluster.

AzureDatabricksLinkedService setNewClusterEnableElasticDisk(Object newClusterEnableElasticDisk)

Set the newClusterEnableElasticDisk property: Enable the elastic disk on the new cluster.

AzureDatabricksLinkedService setNewClusterInitScripts(Object newClusterInitScripts)

Set the newClusterInitScripts property: User-defined initialization scripts for the new cluster.

AzureDatabricksLinkedService setNewClusterLogDestination(Object newClusterLogDestination)

Set the newClusterLogDestination property: Specify a location to deliver Spark driver, worker, and event logs.

AzureDatabricksLinkedService setNewClusterNodeType(Object newClusterNodeType)

Set the newClusterNodeType property: The node type of the new job cluster.

AzureDatabricksLinkedService setNewClusterNumOfWorker(Object newClusterNumOfWorker)

Set the newClusterNumOfWorker property: If not using an existing interactive cluster, this specifies the number of worker nodes to use for the new job cluster or instance pool.

AzureDatabricksLinkedService setNewClusterSparkConf(Map<String,Object> newClusterSparkConf)

Set the newClusterSparkConf property: A set of optional, user-specified Spark configuration key-value pairs.

AzureDatabricksLinkedService setNewClusterSparkEnvVars(Map<String,Object> newClusterSparkEnvVars)

Set the newClusterSparkEnvVars property: A set of optional, user-specified Spark environment variables key-value pairs.

AzureDatabricksLinkedService setNewClusterVersion(Object newClusterVersion)

Set the newClusterVersion property: If not using an existing interactive cluster, this specifies the Spark version of a new job cluster or instance pool nodes created for each run of this activity.

AzureDatabricksLinkedService setParameters(Map<String,ParameterSpecification> parameters)

Set the parameters property: Parameters for linked service.

AzureDatabricksLinkedService setPolicyId(Object policyId)

Set the policyId property: The policy id for limiting the ability to configure clusters based on a user defined set of rules.

AzureDatabricksLinkedService setWorkspaceResourceId(Object workspaceResourceId)

Set the workspaceResourceId property: Workspace resource id for databricks REST API.

Methods inherited from LinkedService

Methods inherited from java.lang.Object

Constructor Details

AzureDatabricksLinkedService

public AzureDatabricksLinkedService()

Creates an instance of AzureDatabricksLinkedService class.

Method Details

getAccessToken

public SecretBase getAccessToken()

Get the accessToken property: Access token for databricks REST API. Refer to https://docs.azuredatabricks.net/api/latest/authentication.html. Type: string (or Expression with resultType string).

Returns:

the accessToken value.

getAuthentication

public Object getAuthentication()

Get the authentication property: Required to specify MSI, if using Workspace resource id for databricks REST API. Type: string (or Expression with resultType string).

Returns:

the authentication value.

getCredential

public CredentialReference getCredential()

Get the credential property: The credential reference containing authentication information.

Returns:

the credential value.

getDomain

public Object getDomain()

Get the domain property: .azuredatabricks.net, domain name of your Databricks deployment. Type: string (or Expression with resultType string).

Returns:

the domain value.

getEncryptedCredential

public Object getEncryptedCredential()

Get the encryptedCredential property: The encrypted credential used for authentication. Credentials are encrypted using the integration runtime credential manager. Type: string (or Expression with resultType string).

Returns:

the encryptedCredential value.

getExistingClusterId

public Object getExistingClusterId()

Get the existingClusterId property: The id of an existing interactive cluster that will be used for all runs of this activity. Type: string (or Expression with resultType string).

Returns:

the existingClusterId value.

getInstancePoolId

public Object getInstancePoolId()

Get the instancePoolId property: The id of an existing instance pool that will be used for all runs of this activity. Type: string (or Expression with resultType string).

Returns:

the instancePoolId value.

getNewClusterCustomTags

public Map getNewClusterCustomTags()

Get the newClusterCustomTags property: Additional tags for cluster resources. This property is ignored in instance pool configurations.

Returns:

the newClusterCustomTags value.

getNewClusterDriverNodeType

public Object getNewClusterDriverNodeType()

Get the newClusterDriverNodeType property: The driver node type for the new job cluster. This property is ignored in instance pool configurations. Type: string (or Expression with resultType string).

Returns:

the newClusterDriverNodeType value.

getNewClusterEnableElasticDisk

public Object getNewClusterEnableElasticDisk()

Get the newClusterEnableElasticDisk property: Enable the elastic disk on the new cluster. This property is now ignored, and takes the default elastic disk behavior in Databricks (elastic disks are always enabled). Type: boolean (or Expression with resultType boolean).

Returns:

the newClusterEnableElasticDisk value.

getNewClusterInitScripts

public Object getNewClusterInitScripts()

Get the newClusterInitScripts property: User-defined initialization scripts for the new cluster. Type: array of strings (or Expression with resultType array of strings).

Returns:

the newClusterInitScripts value.

getNewClusterLogDestination

public Object getNewClusterLogDestination()

Get the newClusterLogDestination property: Specify a location to deliver Spark driver, worker, and event logs. Type: string (or Expression with resultType string).

Returns:

the newClusterLogDestination value.

getNewClusterNodeType

public Object getNewClusterNodeType()

Get the newClusterNodeType property: The node type of the new job cluster. This property is required if newClusterVersion is specified and instancePoolId is not specified. If instancePoolId is specified, this property is ignored. Type: string (or Expression with resultType string).

Returns:

the newClusterNodeType value.

getNewClusterNumOfWorker

public Object getNewClusterNumOfWorker()

Get the newClusterNumOfWorker property: If not using an existing interactive cluster, this specifies the number of worker nodes to use for the new job cluster or instance pool. For new job clusters, this a string-formatted Int32, like '1' means numOfWorker is 1 or '1:10' means auto-scale from 1 (min) to 10 (max). For instance pools, this is a string-formatted Int32, and can only specify a fixed number of worker nodes, such as '2'. Required if newClusterVersion is specified. Type: string (or Expression with resultType string).

Returns:

the newClusterNumOfWorker value.

getNewClusterSparkConf

public Map getNewClusterSparkConf()

Get the newClusterSparkConf property: A set of optional, user-specified Spark configuration key-value pairs.

Returns:

the newClusterSparkConf value.

getNewClusterSparkEnvVars

public Map getNewClusterSparkEnvVars()

Get the newClusterSparkEnvVars property: A set of optional, user-specified Spark environment variables key-value pairs.

Returns:

the newClusterSparkEnvVars value.

getNewClusterVersion

public Object getNewClusterVersion()

Get the newClusterVersion property: If not using an existing interactive cluster, this specifies the Spark version of a new job cluster or instance pool nodes created for each run of this activity. Required if instancePoolId is specified. Type: string (or Expression with resultType string).

Returns:

the newClusterVersion value.

getPolicyId

public Object getPolicyId()

Get the policyId property: The policy id for limiting the ability to configure clusters based on a user defined set of rules. Type: string (or Expression with resultType string).

Returns:

the policyId value.

getWorkspaceResourceId

public Object getWorkspaceResourceId()

Get the workspaceResourceId property: Workspace resource id for databricks REST API. Type: string (or Expression with resultType string).

Returns:

the workspaceResourceId value.

setAccessToken

public AzureDatabricksLinkedService setAccessToken(SecretBase accessToken)

Set the accessToken property: Access token for databricks REST API. Refer to https://docs.azuredatabricks.net/api/latest/authentication.html. Type: string (or Expression with resultType string).

Parameters:

accessToken - the accessToken value to set.

Returns:

the AzureDatabricksLinkedService object itself.

setAnnotations

public AzureDatabricksLinkedService setAnnotations(List annotations)

Set the annotations property: List of tags that can be used for describing the linked service.

Overrides:

AzureDatabricksLinkedService.setAnnotations(List<Object> annotations)

Parameters:

annotations

setAuthentication

public AzureDatabricksLinkedService setAuthentication(Object authentication)

Set the authentication property: Required to specify MSI, if using Workspace resource id for databricks REST API. Type: string (or Expression with resultType string).

Parameters:

authentication - the authentication value to set.

Returns:

the AzureDatabricksLinkedService object itself.

setConnectVia

public AzureDatabricksLinkedService setConnectVia(IntegrationRuntimeReference connectVia)

Set the connectVia property: The integration runtime reference.

Overrides:

AzureDatabricksLinkedService.setConnectVia(IntegrationRuntimeReference connectVia)

Parameters:

connectVia

setCredential

public AzureDatabricksLinkedService setCredential(CredentialReference credential)

Set the credential property: The credential reference containing authentication information.

Parameters:

credential - the credential value to set.

Returns:

the AzureDatabricksLinkedService object itself.

setDescription

public AzureDatabricksLinkedService setDescription(String description)

Set the description property: Linked service description.

Overrides:

AzureDatabricksLinkedService.setDescription(String description)

Parameters:

description

setDomain

public AzureDatabricksLinkedService setDomain(Object domain)

Set the domain property: .azuredatabricks.net, domain name of your Databricks deployment. Type: string (or Expression with resultType string).

Parameters:

domain - the domain value to set.

Returns:

the AzureDatabricksLinkedService object itself.

setEncryptedCredential

public AzureDatabricksLinkedService setEncryptedCredential(Object encryptedCredential)

Set the encryptedCredential property: The encrypted credential used for authentication. Credentials are encrypted using the integration runtime credential manager. Type: string (or Expression with resultType string).

Parameters:

encryptedCredential - the encryptedCredential value to set.

Returns:

the AzureDatabricksLinkedService object itself.

setExistingClusterId

public AzureDatabricksLinkedService setExistingClusterId(Object existingClusterId)

Set the existingClusterId property: The id of an existing interactive cluster that will be used for all runs of this activity. Type: string (or Expression with resultType string).

Parameters:

existingClusterId - the existingClusterId value to set.

Returns:

the AzureDatabricksLinkedService object itself.

setInstancePoolId

public AzureDatabricksLinkedService setInstancePoolId(Object instancePoolId)

Set the instancePoolId property: The id of an existing instance pool that will be used for all runs of this activity. Type: string (or Expression with resultType string).

Parameters:

instancePoolId - the instancePoolId value to set.

Returns:

the AzureDatabricksLinkedService object itself.

setNewClusterCustomTags

public AzureDatabricksLinkedService setNewClusterCustomTags(Map newClusterCustomTags)

Set the newClusterCustomTags property: Additional tags for cluster resources. This property is ignored in instance pool configurations.

Parameters:

newClusterCustomTags - the newClusterCustomTags value to set.

Returns:

the AzureDatabricksLinkedService object itself.

setNewClusterDriverNodeType

public AzureDatabricksLinkedService setNewClusterDriverNodeType(Object newClusterDriverNodeType)

Set the newClusterDriverNodeType property: The driver node type for the new job cluster. This property is ignored in instance pool configurations. Type: string (or Expression with resultType string).

Parameters:

newClusterDriverNodeType - the newClusterDriverNodeType value to set.

Returns:

the AzureDatabricksLinkedService object itself.

setNewClusterEnableElasticDisk

public AzureDatabricksLinkedService setNewClusterEnableElasticDisk(Object newClusterEnableElasticDisk)

Set the newClusterEnableElasticDisk property: Enable the elastic disk on the new cluster. This property is now ignored, and takes the default elastic disk behavior in Databricks (elastic disks are always enabled). Type: boolean (or Expression with resultType boolean).

Parameters:

newClusterEnableElasticDisk - the newClusterEnableElasticDisk value to set.

Returns:

the AzureDatabricksLinkedService object itself.

setNewClusterInitScripts

public AzureDatabricksLinkedService setNewClusterInitScripts(Object newClusterInitScripts)

Set the newClusterInitScripts property: User-defined initialization scripts for the new cluster. Type: array of strings (or Expression with resultType array of strings).

Parameters:

newClusterInitScripts - the newClusterInitScripts value to set.

Returns:

the AzureDatabricksLinkedService object itself.

setNewClusterLogDestination

public AzureDatabricksLinkedService setNewClusterLogDestination(Object newClusterLogDestination)

Set the newClusterLogDestination property: Specify a location to deliver Spark driver, worker, and event logs. Type: string (or Expression with resultType string).

Parameters:

newClusterLogDestination - the newClusterLogDestination value to set.

Returns:

the AzureDatabricksLinkedService object itself.

setNewClusterNodeType

public AzureDatabricksLinkedService setNewClusterNodeType(Object newClusterNodeType)

Set the newClusterNodeType property: The node type of the new job cluster. This property is required if newClusterVersion is specified and instancePoolId is not specified. If instancePoolId is specified, this property is ignored. Type: string (or Expression with resultType string).

Parameters:

newClusterNodeType - the newClusterNodeType value to set.

Returns:

the AzureDatabricksLinkedService object itself.

setNewClusterNumOfWorker

public AzureDatabricksLinkedService setNewClusterNumOfWorker(Object newClusterNumOfWorker)

Set the newClusterNumOfWorker property: If not using an existing interactive cluster, this specifies the number of worker nodes to use for the new job cluster or instance pool. For new job clusters, this a string-formatted Int32, like '1' means numOfWorker is 1 or '1:10' means auto-scale from 1 (min) to 10 (max). For instance pools, this is a string-formatted Int32, and can only specify a fixed number of worker nodes, such as '2'. Required if newClusterVersion is specified. Type: string (or Expression with resultType string).

Parameters:

newClusterNumOfWorker - the newClusterNumOfWorker value to set.

Returns:

the AzureDatabricksLinkedService object itself.

setNewClusterSparkConf

public AzureDatabricksLinkedService setNewClusterSparkConf(Map newClusterSparkConf)

Set the newClusterSparkConf property: A set of optional, user-specified Spark configuration key-value pairs.

Parameters:

newClusterSparkConf - the newClusterSparkConf value to set.

Returns:

the AzureDatabricksLinkedService object itself.

setNewClusterSparkEnvVars

public AzureDatabricksLinkedService setNewClusterSparkEnvVars(Map newClusterSparkEnvVars)

Set the newClusterSparkEnvVars property: A set of optional, user-specified Spark environment variables key-value pairs.

Parameters:

newClusterSparkEnvVars - the newClusterSparkEnvVars value to set.

Returns:

the AzureDatabricksLinkedService object itself.

setNewClusterVersion

public AzureDatabricksLinkedService setNewClusterVersion(Object newClusterVersion)

Set the newClusterVersion property: If not using an existing interactive cluster, this specifies the Spark version of a new job cluster or instance pool nodes created for each run of this activity. Required if instancePoolId is specified. Type: string (or Expression with resultType string).

Parameters:

newClusterVersion - the newClusterVersion value to set.

Returns:

the AzureDatabricksLinkedService object itself.

setParameters

public AzureDatabricksLinkedService setParameters(Map parameters)

Set the parameters property: Parameters for linked service.

Overrides:

AzureDatabricksLinkedService.setParameters(Map<String,ParameterSpecification> parameters)

Parameters:

parameters

setPolicyId

public AzureDatabricksLinkedService setPolicyId(Object policyId)

Set the policyId property: The policy id for limiting the ability to configure clusters based on a user defined set of rules. Type: string (or Expression with resultType string).

Parameters:

policyId - the policyId value to set.

Returns:

the AzureDatabricksLinkedService object itself.

setWorkspaceResourceId

public AzureDatabricksLinkedService setWorkspaceResourceId(Object workspaceResourceId)

Set the workspaceResourceId property: Workspace resource id for databricks REST API. Type: string (or Expression with resultType string).

Parameters:

workspaceResourceId - the workspaceResourceId value to set.

Returns:

the AzureDatabricksLinkedService object itself.

Applies to