Invoke-​Azure​Rm​HD​Insight​Hive​Job

Submits a Hive query to an HDInsight cluster and retrieves query results in one operation.

Syntax

Invoke-AzureRmHDInsightHiveJob
      [-Arguments <String[]>]
      [-Files <String[]>]
      [-StatusFolder <String>]
      [-Defines <Hashtable>]
      [-File <String>]
      [-JobName <String>]
      [-Query <String>]
      [-RunAsFileJob]
      [-DefaultContainer <String>]
      [-DefaultStorageAccountName <String>]
      [-DefaultStorageAccountKey <String>]
      [-InformationAction <ActionPreference>]
      [-InformationVariable <String>]
      [<CommonParameters>]

Description

The Invoke-AzureRmHDInsightHiveJob cmdlet submits a Hive query to an Azure HDInsight cluster and retrieves query results in one operation. Use the Use-AzureRmHDInsightCluster cmdlet before calling Invoke-AzureRmHDInsightHiveJob to specify which cluster will be used for the query.

Examples

Example 1: Submit a Hive query to an Azure HDInsight cluster

PS C:\># Primary storage account info
PS C:\> $storageAccountResourceGroupName = "Group"
PS C:\> $storageAccountName = "yourstorageacct001"
PS C:\> $storageAccountKey = (Get-AzureRmStorageAccountKey -ResourceGroupName $storageAccountResourceGroupName -Name $storageAccountName)[0].value


PS C:\> $storageContainer = "container001"

# Cluster info
PS C:\> $clusterName = "your-hadoop-001"
PS C:\> $clusterCreds = Get-Credential

# Hive job details
PS C:\> $statusFolder = "tempStatusFolder/"
PS C:\> $query = "SHOW TABLES"

PS C:\> Use-AzureRmHDInsightCluster `
            -ClusterCredential $clusterCreds `
            -ClusterName $clusterName

PS C:\> Invoke-AzureRmHDInsightHiveJob -StatusFolder $statusFolder `
            -Query $query `
            -DefaultContainer $storageAccountContainer `
            -DefaultStorageAccountName "$storageAccountName.blob.core.windows.net" `
            -DefaultStorageAccountKey $storageAccountKey

This command submits the query SHOW TABLES to the cluster named your-hadoop-001.

Optional Parameters

-Arguments

Specifies an array of arguments for the job. The arguments are passed as command-line arguments to each task.

Type:String[]
Position:Named
Default value:None
Accept pipeline input:False
Accept wildcard characters:False
-DefaultContainer

Specifies the name of the default container in the default Azure Storage account that an HDInsight cluster uses.

Type:String
Position:Named
Default value:None
Accept pipeline input:False
Accept wildcard characters:False
-DefaultStorageAccountKey

Specifies the account key for the default storage account that the HDInsight cluster uses.

Type:String
Position:Named
Default value:None
Accept pipeline input:False
Accept wildcard characters:False
-DefaultStorageAccountName

Specifies the name of the default storage account that the HDInsight cluster uses.

Type:String
Position:Named
Default value:None
Accept pipeline input:False
Accept wildcard characters:False
-Defines

Specifies Hadoop configuration values to set when a job runs.

Type:Hashtable
Position:Named
Default value:None
Accept pipeline input:False
Accept wildcard characters:False
-File

Specifies the path to a file in Azure Storage that contains the query to run. You can use this parameter instead of the Query parameter.

Type:String
Position:Named
Default value:None
Accept pipeline input:False
Accept wildcard characters:False
-Files

Specifies a collection of files that are required for a Hive job.

Type:String[]
Position:Named
Default value:None
Accept pipeline input:False
Accept wildcard characters:False
-InformationAction

Specifies how this cmdlet responds to an information event.The acceptable values for this parameter are:- Continue

  • Ignore
  • Inquire
  • SilentlyContinue
  • Stop
  • Suspend
Type:ActionPreference
Aliases:infa
Position:Named
Default value:None
Accept pipeline input:False
Accept wildcard characters:False
-InformationVariable

Specifies an information variable.

Type:String
Aliases:iv
Position:Named
Default value:None
Accept pipeline input:False
Accept wildcard characters:False
-JobName

Specifies the name of a Hive job. If you do not specify this parameter, this cmdlet uses the default value: "Hive: <first 100 characters of Query>".

Type:String
Position:Named
Default value:None
Accept pipeline input:False
Accept wildcard characters:False
-Query

Specifies the Hive query.

Type:String
Position:Named
Default value:None
Accept pipeline input:False
Accept wildcard characters:False
-RunAsFileJob

Indicates that this cmdlet creates a file in the default Azure storage account in which to store a query. This cmdlet submits the job that references this file as a script to run.You can use this functionality to handle special characters such as percent sign (%) that would fail on a job submission through Templeton, because Templeton interprets a query with a percent sign as a URL parameter.

Type:SwitchParameter
Position:Named
Default value:None
Accept pipeline input:False
Accept wildcard characters:False
-StatusFolder

Specifies the location of the folder that contains standard outputs and error outputs for a job.

Type:String
Position:Named
Default value:None
Accept pipeline input:False
Accept wildcard characters:False