Add-AzureRmHDInsightConfigValues
Adds a Hadoop configuration value customization and/or a Hive shared library customization to a cluster configuration object.
Warning
The AzureRM PowerShell module has been officially deprecated as of February 29, 2024. Users are advised to migrate from AzureRM to the Az PowerShell module to ensure continued support and updates.
Although the AzureRM module may still function, it's no longer maintained or supported, placing any continued use at the user's discretion and risk. Please refer to our migration resources for guidance on transitioning to the Az module.
Syntax
Add-AzureRmHDInsightConfigValues
[-Config] <AzureHDInsightConfig>
[-Core <Hashtable>]
[-HiveSite <Hashtable>]
[-HiveEnv <Hashtable>]
[-OozieSite <Hashtable>]
[-OozieEnv <Hashtable>]
[-WebHCat <Hashtable>]
[-HBaseSite <Hashtable>]
[-HBaseEnv <Hashtable>]
[-Storm <Hashtable>]
[-Yarn <Hashtable>]
[-MapRed <Hashtable>]
[-Tez <Hashtable>]
[-Hdfs <Hashtable>]
[-RServer <Hashtable>]
[-SparkDefaults <Hashtable>]
[-SparkThriftConf <Hashtable>]
[-DefaultProfile <IAzureContextContainer>]
[<CommonParameters>] Add-AzureRmHDInsightConfigValues
[-Config] <AzureHDInsightConfig>
[-Core <Hashtable>]
[-HiveSite <Hashtable>]
[-HiveEnv <Hashtable>]
[-OozieSite <Hashtable>]
[-OozieEnv <Hashtable>]
[-WebHCat <Hashtable>]
[-HBaseSite <Hashtable>]
[-HBaseEnv <Hashtable>]
[-Storm <Hashtable>]
[-Yarn <Hashtable>]
[-MapRed <Hashtable>]
[-Tez <Hashtable>]
[-Hdfs <Hashtable>]
[-RServer <Hashtable>]
[-Spark2Defaults <Hashtable>]
[-Spark2ThriftConf <Hashtable>]
[-DefaultProfile <IAzureContextContainer>]
[<CommonParameters>] Description
The Add-AzureRmHDInsightConfigValues cmdlet adds a Hadoop configuration value customization, such as core-site.xml or hive-site.xml, and/or a Hive shared library customization to the HDInsight configuration object created by the New-AzureRmHDInsightClusterConfig cmdlet.
Examples
Example 1: Add a custom configuration value to the cluster configuration object
PS C:\># Primary storage account info
PS C:\> $storageAccountResourceGroupName = "Group"
PS C:\> $storageAccountName = "yourstorageacct001"
PS C:\> $storageAccountKey = (Get-AzureRmStorageAccountKey -ResourceGroupName $storageAccountResourceGroupName -Name $storageAccountName)[0].value
PS C:\> $storageContainer = "container001"
# Cluster configuration info
PS C:\> $location = "East US 2"
PS C:\> $clusterResourceGroupName = "Group"
PS C:\> $clusterName = "your-hadoop-001"
PS C:\> $clusterCreds = Get-Credential
# If the cluster's resource group doesn't exist yet, run:
# New-AzureRmResourceGroup -Name $clusterResourceGroupName -Location $location
# Config values
PS C:\> $coreConfigs = @{"io.file.buffer.size"="300000"}
PS C:\> $mapRedConfigs = @{"mapred.map.max.attempts"="2"}
# Create the cluster
PS C:\> New-AzureRmHDInsightClusterConfig `
| Add-AzureRmHDInsightConfigValues `
-Core $coreConfigs `
-MapRed $mapRedConfigs `
| New-AzureRmHDInsightCluster `
-ClusterType Hadoop `
-OSType Windows `
-ClusterSizeInNodes 4 `
-ResourceGroupName $clusterResourceGroupName `
-ClusterName $clusterName `
-HttpCredential $clusterCreds `
-Location $location `
-DefaultStorageAccountName "$storageAccountName.blob.core.windows.net" `
-DefaultStorageAccountKey $storageAccountKey `
-DefaultStorageContainer $storageAccountContainerThis command adds a Hadoop configuration value to the cluster named your-hadoop-001.
Parameters
-Config
Specifies the HDInsight cluster configuration object that this cmdlet modifies. This object is created by the New-AzureRmHDInsightClusterConfig cmdlet.
| Type: | AzureHDInsightConfig |
| Position: | 0 |
| Default value: | None |
| Required: | True |
| Accept pipeline input: | True |
| Accept wildcard characters: | False |
-Core
Specifies the Core Site configurations of this HDInsight cluster.
| Type: | Hashtable |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-DefaultProfile
The credentials, account, tenant, and subscription used for communication with azure
| Type: | IAzureContextContainer |
| Aliases: | AzureRmContext, AzureCredential |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-HBaseEnv
Specifies the HBase Env configurations of this HDInsight cluster.
| Type: | Hashtable |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-HBaseSite
Specifies the HBase Site configurations of this HDInsight cluster.
| Type: | Hashtable |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-Hdfs
Specifies the HDFS configurations of this HDInsight cluster.
| Type: | Hashtable |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-HiveEnv
Specifies the Hive Env configurations of this HDInsight cluster.
| Type: | Hashtable |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-HiveSite
Specifies the Hive Site configurations of this HDInsight cluster.
| Type: | Hashtable |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-MapRed
Specifies the MapRed Site configurations of this HDInsight cluster.
| Type: | Hashtable |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-OozieEnv
Specifies the Oozie Env configurations of this HDInsight cluster.
| Type: | Hashtable |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-OozieSite
Specifies the Oozie Site configurations of this HDInsight cluster.
| Type: | Hashtable |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-RServer
Specifies the RServer configurations. Valid only for RServer clusters.
| Type: | Hashtable |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-Spark2Defaults
Specifies the Spark2 Defaults configurations of this HDInsight cluster.
| Type: | Hashtable |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-Spark2ThriftConf
Specifies the Spark2 Thrift SparkConf configurations of this HDInsight cluster.
| Type: | Hashtable |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-SparkDefaults
Specifies the Spark Defaults configurations of this HDInsight cluster.
| Type: | Hashtable |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-SparkThriftConf
Specifies the Spark Thrift SparkConf configurations of this HDInsight cluster.
| Type: | Hashtable |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-Storm
Specifies the Storm Site configurations of this HDInsight cluster.
| Type: | Hashtable |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-Tez
Specifies the Tez Site configurations of this HDInsight cluster.
| Type: | Hashtable |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-WebHCat
Specifies the WebHCat Site configurations of this HDInsight cluster.
| Type: | Hashtable |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
-Yarn
Specifies the YARN Site configurations of this HDInsight cluster.
| Type: | Hashtable |
| Position: | Named |
| Default value: | None |
| Required: | False |
| Accept pipeline input: | False |
| Accept wildcard characters: | False |
Inputs
Parameters: Config (ByValue)