New-AzureRmHDInsightCluster
Creates an Azure HDInsight cluster in the specified resource group for the current subscription.
Warning
The AzureRM PowerShell module has been officially deprecated as of February 29, 2024. Users are advised to migrate from AzureRM to the Az PowerShell module to ensure continued support and updates.
Although the AzureRM module may still function, it's no longer maintained or supported, placing any continued use at the user's discretion and risk. Please refer to our migration resources for guidance on transitioning to the Az module.
Syntax
New-AzureRmHDInsightCluster
[-Location] <String>
[-ResourceGroupName] <String>
[-ClusterName] <String>
[-ClusterSizeInNodes] <Int32>
[-HttpCredential] <PSCredential>
[[-DefaultStorageAccountName] <String>]
[[-DefaultStorageAccountKey] <String>]
[-DefaultStorageAccountType <StorageType>]
[-Config <AzureHDInsightConfig>]
[-OozieMetastore <AzureHDInsightMetastore>]
[-HiveMetastore <AzureHDInsightMetastore>]
[-AdditionalStorageAccounts <System.Collections.Generic.Dictionary`2[System.String,System.String]>]
[-Configurations <System.Collections.Generic.Dictionary`2[System.String,System.Collections.Generic.Dictionary`2[System.String,System.String]]>]
[-ScriptActions <System.Collections.Generic.Dictionary`2[Microsoft.Azure.Management.HDInsight.Models.ClusterNodeType,System.Collections.Generic.List`1[Microsoft.Azure.Commands.HDInsight.Models.Management.AzureHDInsightScriptAction]]>]
[-DefaultStorageContainer <String>]
[-DefaultStorageRootPath <String>]
[-Version <String>]
[-HeadNodeSize <String>]
[-WorkerNodeSize <String>]
[-EdgeNodeSize <String>]
[-ZookeeperNodeSize <String>]
[-ClusterType <String>]
[-ComponentVersion <System.Collections.Generic.Dictionary`2[System.String,System.String]>]
[-VirtualNetworkId <String>]
[-SubnetName <String>]
[-OSType <OSType>]
[-ClusterTier <Tier>]
[-SshCredential <PSCredential>]
[-SshPublicKey <String>]
[-RdpCredential <PSCredential>]
[-RdpAccessExpiry <DateTime>]
[-ObjectId <Guid>]
[-CertificatePassword <String>]
[-AadTenantId <Guid>]
[-SecurityProfile <AzureHDInsightSecurityProfile>]
[-DisksPerWorkerNode <Int32>]
[-DefaultProfile <IAzureContextContainer>]
[<CommonParameters>] New-AzureRmHDInsightCluster
[-Location] <String>
[-ResourceGroupName] <String>
[-ClusterName] <String>
[-ClusterSizeInNodes] <Int32>
[-HttpCredential] <PSCredential>
[[-DefaultStorageAccountName] <String>]
[[-DefaultStorageAccountKey] <String>]
[-DefaultStorageAccountType <StorageType>]
[-Config <AzureHDInsightConfig>]
[-OozieMetastore <AzureHDInsightMetastore>]
[-HiveMetastore <AzureHDInsightMetastore>]
[-AdditionalStorageAccounts <System.Collections.Generic.Dictionary`2[System.String,System.String]>]
[-Configurations <System.Collections.Generic.Dictionary`2[System.String,System.Collections.Generic.Dictionary`2[System.String,System.String]]>]
[-ScriptActions <System.Collections.Generic.Dictionary`2[Microsoft.Azure.Management.HDInsight.Models.ClusterNodeType,System.Collections.Generic.List`1[Microsoft.Azure.Commands.HDInsight.Models.Management.AzureHDInsightScriptAction]]>]
[-DefaultStorageContainer <String>]
[-DefaultStorageRootPath <String>]
[-Version <String>]
[-HeadNodeSize <String>]
[-WorkerNodeSize <String>]
[-EdgeNodeSize <String>]
[-ZookeeperNodeSize <String>]
[-ClusterType <String>]
[-ComponentVersion <System.Collections.Generic.Dictionary`2[System.String,System.String]>]
[-VirtualNetworkId <String>]
[-SubnetName <String>]
[-OSType <OSType>]
[-ClusterTier <Tier>]
[-SshCredential <PSCredential>]
[-SshPublicKey <String>]
[-RdpCredential <PSCredential>]
[-RdpAccessExpiry <DateTime>]
[-ObjectId <Guid>]
[-CertificateFilePath <String>]
[-CertificatePassword <String>]
[-AadTenantId <Guid>]
[-SecurityProfile <AzureHDInsightSecurityProfile>]
[-DisksPerWorkerNode <Int32>]
[-DefaultProfile <IAzureContextContainer>]
[<CommonParameters>] New-AzureRmHDInsightCluster
[-Location] <String>
[-ResourceGroupName] <String>
[-ClusterName] <String>
[-ClusterSizeInNodes] <Int32>
[-HttpCredential] <PSCredential>
[[-DefaultStorageAccountName] <String>]
[[-DefaultStorageAccountKey] <String>]
[-DefaultStorageAccountType <StorageType>]
[-Config <AzureHDInsightConfig>]
[-OozieMetastore <AzureHDInsightMetastore>]
[-HiveMetastore <AzureHDInsightMetastore>]
[-AdditionalStorageAccounts <System.Collections.Generic.Dictionary`2[System.String,System.String]>]
[-Configurations <System.Collections.Generic.Dictionary`2[System.String,System.Collections.Generic.Dictionary`2[System.String,System.String]]>]
[-ScriptActions <System.Collections.Generic.Dictionary`2[Microsoft.Azure.Management.HDInsight.Models.ClusterNodeType,System.Collections.Generic.List`1[Microsoft.Azure.Commands.HDInsight.Models.Management.AzureHDInsightScriptAction]]>]
[-DefaultStorageContainer <String>]
[-DefaultStorageRootPath <String>]
[-Version <String>]
[-HeadNodeSize <String>]
[-WorkerNodeSize <String>]
[-EdgeNodeSize <String>]
[-ZookeeperNodeSize <String>]
[-ClusterType <String>]
[-ComponentVersion <System.Collections.Generic.Dictionary`2[System.String,System.String]>]
[-VirtualNetworkId <String>]
[-SubnetName <String>]
[-OSType <OSType>]
[-ClusterTier <Tier>]
[-SshCredential <PSCredential>]
[-SshPublicKey <String>]
[-RdpCredential <PSCredential>]
[-RdpAccessExpiry <DateTime>]
[-ObjectId <Guid>]
[-CertificateFileContents <Byte[]>]
[-CertificatePassword <String>]
[-AadTenantId <Guid>]
[-SecurityProfile <AzureHDInsightSecurityProfile>]
[-DisksPerWorkerNode <Int32>]
[-DefaultProfile <IAzureContextContainer>]
[<CommonParameters>] Description
The New-AzureHDInsightCluster creates an Azure HDInsight cluster by using the specified parameters or by using a configuration object that is created by using the New-AzureRmHDInsightClusterConfig cmdlet.
Examples
Example 1: Create an Azure HDInsight cluster
PS C:\> # Primary storage account info
$storageAccountResourceGroupName = "Group"
$storageAccountName = "yourstorageacct001"
$storageAccountKey = Get-AzureStorageAccountKey `
-ResourceGroupName $storageAccountResourceGroupName `
-Name $storageAccountName | %{ $_.Key1 }
$storageContainer = "container002"
# Cluster configuration info
$location = "East US 2"
$clusterResourceGroupName = "Group"
$clusterName = "your-hadoop-002"
$clusterCreds = Get-Credential
# If the cluster's resource group doesn't exist yet, run:
# New-AzureRMResourceGroup -Name $clusterResourceGroupName -Location $location
# Create the cluster
New-AzureRmHDInsightCluster `
-ClusterType Hadoop `
-OSType Windows `
-ClusterSizeInNodes 4 `
-ResourceGroupName $clusterResourceGroupName `
-ClusterName $clusterName `
-HttpCredential $clusterCreds `
-Location $location `
-DefaultStorageAccountName "$storageAccountName.blob.core.contoso.net" `
-DefaultStorageAccountKey $storageAccountKey `
-DefaultStorageContainer $storageContainerThis command creates a cluster in the current subscription.
Parameters
-AadTenantId
Specifies the Microsoft Entra tenant ID that will be used when accessing Azure Data Lake Store.
| Type: | Guid |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-AdditionalStorageAccounts
Specifies the additional Azure Storage accounts for the cluster. You can alternatively use the Add-AzureRmHDInsightStorage cmdlet.
| Type: | Dictionary<TKey,TValue>[System.String,System.String] |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-CertificateFileContents
Specifies file contents of the certificate that will be used when accessing Azure Data Lake Store.
| Type: | Byte[] |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-CertificateFilePath
Specifies the file path to the certificate that will be used to authenticate as the Service Principal. The cluster will use this when accessing Azure Data Lake Store.
| Type: | String |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-CertificatePassword
Specifies the password for the certificate that will be used to authenticate as the Service Principal. The cluster will use this when accessing Azure Data Lake Store.
| Type: | String |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-ClusterName
Specifies the name of the cluster.
| Type: | String |
| Position: | 2 |
| Default value: | None |
| Required: | True |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-ClusterSizeInNodes
Specifies the number of Worker nodes for the cluster.
| Type: | Int32 |
| Position: | 3 |
| Default value: | None |
| Required: | True |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-ClusterTier
Specifies the HDInsight cluster tier. By default, this is Standard. The Premium tier can only be used with Linux clusters, and it enables the use of some new features.
| Type: | Tier |
| Accepted values: | Standard, Premium |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-ClusterType
Specifies the type of cluster to create. Options are: Hadoop, HBase, Storm, Spark
| Type: | String |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-ComponentVersion
| Type: | Dictionary<TKey,TValue>[System.String,System.String] |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-Config
Specifies the cluster object to be used to create the cluster. This object can be created by using the New-AzureRmHDInsightClusterConfig cmdlet.
| Type: | AzureHDInsightConfig |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | True |
| Accept wildcard characters: | False |
-Configurations
Specifies the configurations of this HDInsight cluster. You can alternatively use the Add-AzureRmHDInsightConfigValues cmdlet.
| Type: | Dictionary<TKey,TValue>[System.String,System.Collections.Generic.Dictionary`2[System.String,System.String]] |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-DefaultProfile
The credentials, account, tenant, and subscription used for communication with azure
| Type: | IAzureContextContainer |
| Aliases: | AzureRmContext, AzureCredential |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-DefaultStorageAccountKey
Specifies the account key for the default Azure Storage account that the HDInsight cluster will use. You can alternatively use the Set-AzureRmHDInsightDefaultStorage cmdlet.
| Type: | String |
| Position: | 6 |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-DefaultStorageAccountName
Specifies the name of the default Azure Storage account that the HDInsight cluster will use. You can alternatively use the Set-AzureRmHDInsightDefaultStorage cmdlet.
| Type: | String |
| Position: | 5 |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-DefaultStorageAccountType
Specifies the type of the default storage account that the HDInsight cluster will use. Possible values are AzureStorage and AzureDataLakeStore. Defaults to AzureStorage if not specified.
| Type: | Nullable<T>[StorageType] |
| Accepted values: | AzureStorage, AzureDataLakeStore |
| Position: | Named |
| Default value: | AzureStorage |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-DefaultStorageContainer
Specifies the name of the default container in the default Azure storage account that the HDInsight cluster will use. You can alternatively use the Set-AzureRmHDInsightDefaultStorage cmdlet.
| Type: | String |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-DefaultStorageRootPath
Specifies the path-prefix in the Data Lake Store Account that the HDInsight cluster will use as the default filesystem.
| Type: | String |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-DisksPerWorkerNode
Specifies the number of disks for worker node role in the cluster.
| Type: | Int32 |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-EdgeNodeSize
Specifies the size of the virtual machine for the edge node. Use Get-AzureRmVMSize for acceptable VM sizes, and see HDInsight's pricing page. This parameter is valid only for RServer clusters.
| Type: | String |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-HeadNodeSize
Specifies the size of the virtual machine for the Head node. Use Get-AzureRmVMSize for acceptable VM sizes, and see HDInsight's pricing page.
| Type: | String |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-HiveMetastore
Specifies the SQL Database to store Hive metadata. You can alternatively use the Add-AzureRmHDInsightMetastore cmdlet.
| Type: | AzureHDInsightMetastore |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-HttpCredential
Specifies the cluster login (HTTP) credentials for the cluster.
| Type: | PSCredential |
| Position: | 4 |
| Default value: | None |
| Required: | True |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-Location
Specifies the location for the cluster.
| Type: | String |
| Position: | 0 |
| Default value: | None |
| Required: | True |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-ObjectId
Specifies the Microsoft Entra object ID (a GUID) of the Microsoft Entra service principal that represents the cluster. The cluster will use this when accessing Azure Data Lake Store.
| Type: | Guid |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-OozieMetastore
Specifies the SQL Database to store Oozie metadata. You can alternatively use the Add-AzureRmHDInsightMetastore cmdlet.
| Type: | AzureHDInsightMetastore |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-OSType
Specifies the operating system for the cluster. Options are: Windows, Linux
| Type: | OSType |
| Accepted values: | Windows, Linux |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-RdpAccessExpiry
Specifies the expiration, as a DateTime object, for Remote Desktop Protocol (RDP) access to a cluster.
| Type: | DateTime |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-RdpCredential
Specifies the Remote Desktop (RDP) credentials for the cluster. This is only for Windows clusters.
| Type: | PSCredential |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-ResourceGroupName
Specifies the name of the resource group.
| Type: | String |
| Position: | 1 |
| Default value: | None |
| Required: | True |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-ScriptActions
Specifies the script actions to run on the cluster at the end of cluster creation. You can alternatively use Add-AzureRmHDInsightScriptAction.
| Type: | Dictionary<TKey,TValue>[Microsoft.Azure.Management.HDInsight.Models.ClusterNodeType,System.Collections.Generic.List`1[AzureHDInsightScriptAction]] |
| Accepted values: | HeadNode, WorkerNode, ZookeeperNode, EdgeNode |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-SecurityProfile
Specifies the security related properties used to create a secure cluster. You can alternatively use the Add-AzureRmHDInsightSecurityProfile cmdlet.
| Type: | AzureHDInsightSecurityProfile |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-SshCredential
Specifies the SSH credential to be used for SSH connections. This is only for Linux clusters.
| Type: | PSCredential |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-SshPublicKey
Specifies the public key to be used for SSH connections. This is only for Linux clusters.
| Type: | String |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-SubnetName
Specifies the name of a subnet within the chosen virtual network for the cluster.
| Type: | String |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-Version
Specifies the HDI version of the HDInsight cluster.
| Type: | String |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-VirtualNetworkId
Specifies the ID of the virtual network into which to provision the cluster.
| Type: | String |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-WorkerNodeSize
Specifies the size of the virtual machine for the Worker node. Use Get-AzureRmVMSize for acceptable VM sizes, and see HDInsight's pricing page.
| Type: | String |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-ZookeeperNodeSize
Specifies the size of the virtual machine for the Zookeeper node. Use Get-AzureRmVMSize for acceptable VM sizes, and see HDInsight's pricing page. This parameter is valid only for HBase or Storm clusters.
| Type: | String |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
Inputs
Parameters: Config (ByValue)
Outputs
Notes
Keywords: azure, azurerm, arm, resource, management, manager, hadoop, hdinsight, hd, insight