The steps in this document walk-through creating a HDInsight 3.5 cluster using the Azure CLI 1.0.
Linux is the only operating system used on HDInsight version 3.4 or greater. For more information, see HDInsight retirement on Windows.
Billing for HDInsight clusters is prorated per minute, whether you are using them or not. Be sure to delete your cluster after you have finished using it. For more information, see How to delete an HDInsight cluster.
An Azure subscription. See Get Azure free trial.
Azure CLI. The steps in this document were last tested with Azure CLI version 0.10.1.
The steps in this document do not work with Azure CLI 2.0. Azure CLI 2.0 does not support creating an HDInsight cluster.
Log in to your Azure subscription
Follow the steps documented in Connect to an Azure subscription from the Azure Command-Line Interface (Azure CLI) and connect to your subscription using the login method.
Create a cluster
The following steps should be performed from a command-prompt, shell, or terminal session after installing and configuring the Azure CLI.
Use the following command to authenticate to your Azure subscription:
You are prompted to provide your name and password. If you have multiple Azure subscriptions, use
azure account set <subscriptionname>to set the subscription that the Azure CLI commands use.
Switch to Azure Resource Manager mode using the following command:
azure config mode arm
Create a resource group. This resource group contains the HDInsight cluster and associated storage account.
azure group create groupname location
Replace groupname with a unique name for the group.
Replace location with the geographic region that you want to create the group in.
For a list of valid locations, use the
azure location listcommand, and then use one of the locations from the Name column.
Create a storage account. This storage account is used as the default storage for the HDInsight cluster.
azure storage account create -g groupname --sku-name RAGRS -l location --kind Storage storagename
Replace groupname with the name of the group created in the previous step.
Replace location with the same location used in the previous step.
Replace storagename with a unique name for the storage account.
For more information on the parameters used in this command, use
azure storage account create -hto view help for this command.
Retrieve the key used to access the storage account.
azure storage account keys list -g groupname storagename
- Replace groupname with the resource group name.
Replace storagename with the name of the storage account.
In the data that is returned, save the key value for key1.
Create an HDInsight cluster.
azure hdinsight cluster create -g groupname -l location -y Linux --clusterType Hadoop --defaultStorageAccountName storagename.blob.core.windows.net --defaultStorageAccountKey storagekey --defaultStorageContainer clustername --workerNodeCount 2 --userName admin --password httppassword --sshUserName sshuser --sshPassword sshuserpassword clustername
Replace groupname with the resource group name.
Replace Hadoop with the cluster type that you wish to create. For example, Hadoop, HBase, Storm, or Spark.
HDInsight clusters come in various types, which correspond to the workload or technology that the cluster is tuned for. There is no supported method to create a cluster that combines multiple types, such as Storm and HBase on one cluster.
Replace location with the same location used in previous steps.
Replace storagename with the storage account name.
Replace storagekey with the key obtained in the previous step.
--defaultStorageContainerparameter, use the same name as you are using for the cluster.
Replace admin and httppassword with the name and password you wish to use when accessing the cluster through HTTPS.
Replace sshuser and sshuserpassword with the username and password you wish to use when accessing the cluster using SSH
This example creates a cluster with two worker notes. If you plan on more than 32 worker nodes (during cluster creation or by scaling the cluster,) then you must select a head node size with at least 8 cores and 14-GB RAM. You can set the head node size by using the
For more information on node sizes and associated costs, see HDInsight pricing.
It may take several minutes for the cluster creation process to finish. Usually around 15.
If you run into issues with creating HDInsight clusters, see access control requirements.
Now that you have successfully created an HDInsight cluster using the Azure CLI, use the following to learn how to work with your cluster: