az hdinsight

Manage HDInsight resources.

Commands

az hdinsight application Manage HDInsight applications.
az hdinsight application create Create an application for a HDInsight cluster.
az hdinsight application delete Deletes the specified application on the HDInsight cluster.
az hdinsight application list Lists all of the applications for the HDInsight cluster.
az hdinsight application show Gets properties of the specified application.
az hdinsight application wait Place the CLI in a waiting state until an operation is complete.
az hdinsight create Creates a new cluster.
az hdinsight delete Deletes the specified HDInsight cluster.
az hdinsight list List clusters in the resource group or subscription.
az hdinsight list-usage Lists the usages for the specified location.
az hdinsight monitor Manage Azure Monitor logs integration on the HDInsight cluster.
az hdinsight monitor disable Disables the Azure Monitor logs integration on the HDInsight cluster.
az hdinsight monitor enable Enables the Azure Monitor logs integration on the HDInsight cluster.
az hdinsight monitor show Gets the status of Azure Monitor logs integration on the HDInsight cluster.
az hdinsight resize Resizes the specified HDInsight cluster to the specified size.
az hdinsight rotate-disk-encryption-key Rotate disk encryption key of the specified HDInsight cluster.
az hdinsight script-action Manage HDInsight script actions.
az hdinsight script-action delete Deletes a specified persisted script action of the cluster.
az hdinsight script-action execute Executes script actions on the specified HDInsight cluster.
az hdinsight script-action list Lists script actions for the specified cluster.
az hdinsight script-action promote Promotes the specified ad-hoc script execution to a persisted script.
az hdinsight script-action show Gets the script execution detail for the given script execution ID.
az hdinsight show Gets the specified cluster.
az hdinsight update Patch HDInsight cluster with the specified parameters.
az hdinsight wait Place the CLI in a waiting state until an operation is complete.

az hdinsight create

Creates a new cluster.

az hdinsight create --name
--resource-group
--type
[--assign-identity]
[--cluster-admin-account]
[--cluster-admin-password]
[--cluster-configurations]
[--cluster-tier {Premium, Standard}]
[--cluster-users-group-dns]
[--component-version]
[--domain]
[--edgenode-size]
[--encryption-algorithm {RSA-OAEP, RSA-OAEP-256, RSA1_5}]
[--encryption-key-name]
[--encryption-key-version]
[--encryption-vault-uri]
[--esp {false, true}]
[--headnode-size]
[--http-password]
[--http-user]
[--ldaps-urls]
[--location]
[--no-validation-timeout {false, true}]
[--no-wait]
[--size]
[--ssh-password]
[--ssh-public-key]
[--ssh-user]
[--storage-account]
[--storage-account-key]
[--storage-account-managed-identity]
[--storage-default-container]
[--storage-default-filesystem]
[--subnet]
[--subscription]
[--tags]
[--version]
[--vnet-name]
[--workernode-data-disk-size]
[--workernode-data-disk-storage-account-type {premium_lrs, standard_lrs}]
[--workernode-data-disks-per-node]
[--workernode-size]
[--zookeepernode-size]

Examples

Create a cluster with an existing storage account.

az hdinsight create -t spark -g MyResourceGroup -n MyCluster \
                        -p "HttpPassword1234!" \
                        --storage-account MyStorageAccount

Create a cluster with Enterprise Security Package.

az hdinsight create --esp -t spark -g MyResourceGroup -n MyCluster \
                        -p "HttpPassword1234!" \
                        --storage-account MyStorageAccount \
                        --subnet "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/MyRG/providers/Microsoft.Network/virtualNetworks/MyVnet/subnets/subnet1" \
                        --domain "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/MyRG/providers/Microsoft.AAD/domainServices/MyDomain.onmicrosoft.com" \
                        --assign-identity "/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/MyMsiRG/providers/Microsoft.ManagedIdentity/userAssignedIdentities/MyMSI" \
                        --cluster-admin-account MyAdminAccount@MyDomain.onmicrosoft.com \
                        --cluster-users-group-dns MyGroup

Create a Kafka cluster with disk encryption. See https://docs.microsoft.com/en-us/azure/hdinsight/kafka/apache-kafka-byok.

az hdinsight create -t kafka -g MyResourceGroup -n MyCluster \
                        -p "HttpPassword1234!" --workernode-data-disks-per-node 2 \
                        --storage-account MyStorageAccount \
                        --encryption-key-name kafkaClusterKey \
                        --encryption-key-version 00000000000000000000000000000000 \
                        --encryption-vault-uri https://MyKeyVault.vault.azure.net \
                        --assign-identity MyMSI

Create a cluster with Azure Data Lake Storage Gen2

az hdinsight create -t spark -g MyResourceGroup -n MyCluster \
                        -p "HttpPassword1234!" \
                        --storage-account MyStorageAccount \
                        --storage-account-managed-identity MyMSI

Required Parameters

--name -n

The name of the cluster.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

--type -t

Type of HDInsight cluster, like: hadoop, interactivehive, hbase, kafka, storm, spark, rserver, mlservices. See also: https://docs.microsoft.com/en-us/azure/hdinsight/hdinsight-hadoop-provision-linux-clusters#cluster-types.

Optional Parameters

--assign-identity

The name or ID of user assigned identity.

--cluster-admin-account

The domain user account that will have admin privileges on the cluster. Required only when create cluster with Enterprise Security Package.

--cluster-admin-password

The domain admin password. Required only when create cluster with Enterprise Security Package.

--cluster-configurations

Extra configurations of various components, in JSON.

--cluster-tier

The tier of the cluster.

accepted values: Premium, Standard
--cluster-users-group-dns

A space-delimited list of Distinguished Names for cluster user groups. Required only when create cluster with Enterprise Security Package.

--component-version

The versions of various Hadoop components, in space-separated versions in 'component=version' format. Example: Spark=2.0 Hadoop=2.7.3 See also: https://docs.microsoft.com/en-us/azure/hdinsight/hdinsight-component-versioning#hadoop-components-available-with-different-hdinsight-versions.

--domain

The name or resource ID of the user's Azure Active Directory Domain Service. Required only when create cluster with Enterprise Security Package.

--edgenode-size
--encryption-algorithm

Algorithm identifier for encryption.

accepted values: RSA-OAEP, RSA-OAEP-256, RSA1_5
default value: RSA-OAEP
--encryption-key-name

Key name that is used for enabling disk encryption.

--encryption-key-version

Key version that is used for enabling disk encryption.

--encryption-vault-uri

Base key vault URI where the customers key is located eg. https://myvault.vault.azure.net.

--esp

Specify to create cluster with Enterprise Security Package.

accepted values: false, true
--headnode-size
--http-password -p

HTTP password for the cluster. Will prompt if not given.

--http-user -u

HTTP username for the cluster. Default: admin.

--ldaps-urls

A space-delimited list of LDAPS protocol URLs to communicate with the Active Directory. Required only when create cluster with Enterprise Security Package.

--location -l

Location. Values from: az account list-locations. You can configure the default location using az configure --defaults location=<location>.

--no-validation-timeout

Do not report timeout error during argument validation phase.

accepted values: false, true
--no-wait

Do not wait for the long-running operation to finish.

--size -s

The number of worker nodes in the cluster.

default value: 3
--ssh-password -P

SSH password for the cluster nodes. If none specified, uses the HTTP password.

--ssh-public-key -K

SSH public key for the cluster nodes.

--ssh-user -U

SSH username for the cluster nodes.

default value: sshuser
--storage-account

The name or ID of the storage account.

--storage-account-key

The storage account key. A key can be retrieved automatically if the user has access to the storage account.

--storage-account-managed-identity

User-assigned managed identity with access to the storage account filesystem. Only required when storage account type is Azure Data Lake Storage Gen2.

--storage-default-container

The storage container the cluster will use. Uses the cluster name if none was specified. (WASB only).

--storage-default-filesystem

The storage filesystem the cluster will use. (DFS only).

--subnet

The name or ID of subnet. If name is supplied, --vnet-name must be supplied.

--subscription

Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.

--tags

Space-separated tags in 'key[=value]' format. Use "" to clear existing tags.

--version -v
--vnet-name

The name of a virtual network.

--workernode-data-disk-size

The size of the data disk in GB, e.g. 1023.

--workernode-data-disk-storage-account-type

The type of storage account that will be used for the data disks: standard_lrs or premium_lrs.

accepted values: premium_lrs, standard_lrs
--workernode-data-disks-per-node

The number of data disks to use per worker node.

--workernode-size
--zookeepernode-size

az hdinsight delete

Deletes the specified HDInsight cluster.

az hdinsight delete --name
--resource-group
[--no-wait]
[--subscription]
[--yes]

Required Parameters

--name -n

The name of the cluster.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

Optional Parameters

--no-wait

Do not wait for the long-running operation to finish.

--subscription

Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.

--yes -y

Do not prompt for confirmation.

az hdinsight list

List clusters in the resource group or subscription.

az hdinsight list [--resource-group]
[--subscription]

Optional Parameters

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

--subscription

Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.

az hdinsight list-usage

Lists the usages for the specified location.

az hdinsight list-usage --location
[--subscription]

Required Parameters

--location -l

Location. Values from: az account list-locations. You can configure the default location using az configure --defaults location=<location>.

Optional Parameters

--subscription

Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.

az hdinsight resize

Resizes the specified HDInsight cluster to the specified size.

az hdinsight resize --name
--resource-group
[--no-wait]
[--subscription]
[--target-instance-count]

Required Parameters

--name -n

The name of the cluster.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

Optional Parameters

--no-wait

Do not wait for the long-running operation to finish.

--subscription

Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.

--target-instance-count

The target instance count for the operation.

az hdinsight rotate-disk-encryption-key

Rotate disk encryption key of the specified HDInsight cluster.

az hdinsight rotate-disk-encryption-key --encryption-key-name
--encryption-key-version
--encryption-vault-uri
--name
--resource-group
[--no-wait]
[--subscription]

Required Parameters

--encryption-key-name

Key name that is used for enabling disk encryption.

--encryption-key-version

Key version that is used for enabling disk encryption.

--encryption-vault-uri

Base key vault URI where the customers key is located eg. https://myvault.vault.azure.net.

--name -n

The name of the cluster.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

Optional Parameters

--no-wait

Do not wait for the long-running operation to finish.

--subscription

Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.

az hdinsight show

Gets the specified cluster.

az hdinsight show --name
--resource-group
[--subscription]

Required Parameters

--name -n

The name of the cluster.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

Optional Parameters

--subscription

Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.

az hdinsight update

Patch HDInsight cluster with the specified parameters.

az hdinsight update --name
--resource-group
[--no-wait]
[--subscription]
[--tags]

Required Parameters

--name -n

The name of the cluster.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

Optional Parameters

--no-wait

Do not wait for the long-running operation to finish.

--subscription

Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.

--tags

Space-separated tags in 'key[=value]' format. Use "" to clear existing tags.

az hdinsight wait

Place the CLI in a waiting state until an operation is complete.

az hdinsight wait --name
--resource-group
[--created]
[--custom]
[--deleted]
[--exists]
[--interval]
[--subscription]
[--timeout]
[--updated]

Required Parameters

--name -n

The name of the cluster.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

Optional Parameters

--created

Wait until created with 'provisioningState' at 'Succeeded'.

--custom

Wait until the condition satisfies a custom JMESPath query. E.g. provisioningState!='InProgress', instanceView.statuses[?code=='PowerState/running'].

--deleted

Wait until deleted.

--exists

Wait until the resource exists.

--interval

Polling interval in seconds.

default value: 30
--subscription

Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.

--timeout

Maximum wait in seconds.

default value: 3600
--updated

Wait until updated with provisioningState at 'Succeeded'.