Add-​Azure​Rm​HD​Insight​Config​Values

Adds a Hadoop configuration value customization and/or a Hive shared library customization to a cluster configuration object.

Syntax

Add-AzureRmHDInsightConfigValues
   [-Config] <AzureHDInsightConfig>
   [-Core <Hashtable>]
   [-HiveSite <Hashtable>]
   [-HiveEnv <Hashtable>]
   [-OozieSite <Hashtable>]
   [-OozieEnv <Hashtable>]
   [-WebHCat <Hashtable>]
   [-HBaseSite <Hashtable>]
   [-HBaseEnv <Hashtable>]
   [-Storm <Hashtable>]
   [-Yarn <Hashtable>]
   [-MapRed <Hashtable>]
   [-Tez <Hashtable>]
   [-Hdfs <Hashtable>]
   [-RServer <Hashtable>]
   [-InformationAction <ActionPreference>]
   [-InformationVariable <String>]
   [<CommonParameters>]

Description

The Add-AzureRmHDInsightConfigValues cmdlet adds a Hadoop configuration value customization, such as core-site.xml or hive-site.xml, and/or a Hive shared library customization to the HDInsight configuration object created by the New-AzureRmHDInsightClusterConfig cmdlet.

Examples

Example 1: Add a custom configuration value to the cluster configuration object

PS C:\># Primary storage account info
PS C:\> $storageAccountResourceGroupName = "Group"
PS C:\> $storageAccountName = "yourstorageacct001"
PS C:\> $storageAccountKey = (Get-AzureRmStorageAccountKey -ResourceGroupName $storageAccountResourceGroupName -Name $storageAccountName)[0].value

PS C:\> $storageContainer = "container001"

# Cluster configuration info
PS C:\> $location = "East US 2"
PS C:\> $clusterResourceGroupName = "Group"
PS C:\> $clusterName = "your-hadoop-001"
PS C:\> $clusterCreds = Get-Credential

# If the cluster's resource group doesn't exist yet, run:
#   New-AzureRmResourceGroup -Name $clusterResourceGroupName -Location $location

# Config values
PS C:\> $coreConfigs = @{"io.file.buffer.size"="300000"}
PS C:\> $mapRedConfigs = @{"mapred.map.max.attempts"="2"}

# Create the cluster
PS C:\> New-AzureRmHDInsightClusterConfig `
            | Add-AzureRmHDInsightConfigValues `
                -Core $coreConfigs `
                -MapRed $mapRedConfigs `
            | New-AzureRmHDInsightCluster `
                -ClusterType Hadoop `
                -OSType Windows `
                -ClusterSizeInNodes 4 `
                -ResourceGroupName $clusterResourceGroupName `
                -ClusterName $clusterName `
                -HttpCredential $clusterCreds `
                -Location $location `
                -DefaultStorageAccountName "$storageAccountName.blob.core.windows.net" `
                -DefaultStorageAccountKey $storageAccountKey `
                -DefaultStorageContainer $storageAccountContainer

This command adds a Hadoop configuration value to the cluster named your-hadoop-001.

Required Parameters

-Config

Specifies the HDInsight cluster configuration object that this cmdlet modifies. This object is created by the New-AzureRmHDInsightClusterConfig cmdlet.

Type:AzureHDInsightConfig
Position:0
Default value:None
Accept pipeline input:True (ByValue)
Accept wildcard characters:False

Optional Parameters

-Core

Specifies the Core Site configurations of this HDInsight cluster.

Type:Hashtable
Position:Named
Default value:None
Accept pipeline input:False
Accept wildcard characters:False
-HBaseEnv

Specifies the HBase Env configurations of this HDInsight cluster.

Type:Hashtable
Position:Named
Default value:None
Accept pipeline input:False
Accept wildcard characters:False
-HBaseSite

Specifies the HBase Site configurations of this HDInsight cluster.

Type:Hashtable
Position:Named
Default value:None
Accept pipeline input:False
Accept wildcard characters:False
-Hdfs

Specifies the HDFS configurations of this HDInsight cluster.

Type:Hashtable
Position:Named
Default value:None
Accept pipeline input:False
Accept wildcard characters:False
-HiveEnv

Specifies the Hive Env configurations of this HDInsight cluster.

Type:Hashtable
Position:Named
Default value:None
Accept pipeline input:False
Accept wildcard characters:False
-HiveSite

Specifies the Hive Site configurations of this HDInsight cluster.

Type:Hashtable
Position:Named
Default value:None
Accept pipeline input:False
Accept wildcard characters:False
-InformationAction

Specifies how this cmdlet responds to an information event.The acceptable values for this parameter are:- Continue

  • Ignore
  • Inquire
  • SilentlyContinue
  • Stop
  • Suspend
Type:ActionPreference
Aliases:infa
Position:Named
Default value:None
Accept pipeline input:False
Accept wildcard characters:False
-InformationVariable

Specifies an information variable.

Type:String
Aliases:iv
Position:Named
Default value:None
Accept pipeline input:False
Accept wildcard characters:False
-MapRed

Specifies the MapRed Site configurations of this HDInsight cluster.

Type:Hashtable
Position:Named
Default value:None
Accept pipeline input:False
Accept wildcard characters:False
-OozieEnv

Specifies the Oozie Env configurations of this HDInsight cluster.

Type:Hashtable
Position:Named
Default value:None
Accept pipeline input:False
Accept wildcard characters:False
-OozieSite

Specifies the Oozie Site configurations of this HDInsight cluster.

Type:Hashtable
Position:Named
Default value:None
Accept pipeline input:False
Accept wildcard characters:False
-RServer

Specifies the RServer configurations. Valid only for RServer clusters.

Type:Hashtable
Position:Named
Default value:None
Accept pipeline input:False
Accept wildcard characters:False
-Storm

Specifies the Storm Site configurations of this HDInsight cluster.

Type:Hashtable
Position:Named
Default value:None
Accept pipeline input:False
Accept wildcard characters:False
-Tez

Specifies the Tez Site configurations of this HDInsight cluster.

Type:Hashtable
Position:Named
Default value:None
Accept pipeline input:False
Accept wildcard characters:False
-WebHCat

Specifies the WebHCat Site configurations of this HDInsight cluster.

Type:Hashtable
Position:Named
Default value:None
Accept pipeline input:False
Accept wildcard characters:False
-Yarn

Specifies the YARN Site configurations of this HDInsight cluster.

Type:Hashtable
Position:Named
Default value:None
Accept pipeline input:False
Accept wildcard characters:False