Edit

New-AzHDInsightStreamingMapReduceJobDefinition

  • Reference
Module:
Az.HDInsight

Creates a Streaming MapReduce job object.

Syntax

New-AzHDInsightStreamingMapReduceJobDefinition
   [-Arguments <String[]>]
   [-File <String>]
   [-Files <String[]>]
   [-StatusFolder <String>]
   [-CommandEnvironment <Hashtable>]
   [-Defines <Hashtable>]
   -InputPath <String>
   [-Mapper <String>]
   [-OutputPath <String>]
   [-Reducer <String>]
   [-DefaultProfile <IAzureContextContainer>]
   [<CommonParameters>]

Description

The New-AzHDInsightStreamingMapReduceJobDefinition cmdlet defines a Streaming MapReduce job object for use with an Azure HDInsight cluster.

Examples

Example 1: Create a Streaming MapReduce job definition

# Cluster info
$clusterName = "your-hadoop-001"
$clusterCreds = Get-Credential

# Streaming MapReduce job details
$statusFolder = "tempStatusFolder/"
$query = "SHOW TABLES"

New-AzHDInsightStreamingMapReduceJobDefinition -StatusFolder $statusFolder -File $query `
        | Start-AzHDInsightJob `
            -ClusterName $clusterName `
            -ClusterCredential $clusterCreds

This command creates a Streaming MapReduce job definition.

Parameters

-Arguments

Specifies an array of arguments for the job. The arguments are passed as command-line arguments to each task.

Type:String[]
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-CommandEnvironment

Specifies an array of command-line environment variables to set when a job runs on worker nodes.

Type:Hashtable
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-DefaultProfile

The credentials, account, tenant, and subscription used for communication with azure

Type:IAzureContextContainer
Aliases:AzContext, AzureRmContext, AzureCredential
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-Defines

Specifies Hadoop configuration values to set for when the job runs.

Type:Hashtable
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-File

Specifies the path to a file that contains a query to run. You can use this parameter instead of the Query parameter.

Type:String
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-Files

Specifies a collection of files that are associated with a Hive job.

Type:String[]
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-InputPath

Specifies the path to the input files.

Type:String
Position:Named
Default value:None
Required:True
Accept pipeline input:False
Accept wildcard characters:False

-Mapper

Specifies a Mapper file name.

Type:String
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-OutputPath

Specifies the path for the job output.

Type:String
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-Reducer

Specifies a Reducer file name.

Type:String
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-StatusFolder

Specifies the location of the folder that contains standard outputs and error outputs for a job.

Type:String
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

Inputs

None

Outputs

AzureHDInsightStreamingMapReduceJobDefinition