Quickstart: Run your first Batch job with the Azure CLI

The Azure CLI is used to create and manage Azure resources from the command line or in scripts. This quickstart shows how to use the Azure CLI to create a Batch account, a pool of compute nodes (virtual machines), and a job that runs tasks on the pool. Each sample task runs a basic command on one of the pool nodes. After completing this quickstart, you will understand the key concepts of the Batch service and be ready to try Batch with more realistic workloads at larger scale.

If you don't have an Azure subscription, create a free account before you begin.

Launch Azure Cloud Shell

The Azure Cloud Shell is a free interactive shell that you can use to run the steps in this article. It has common Azure tools preinstalled and configured to use with your account. Just click the Copy button to copy the code, paste it into the Cloud Shell, and then press enter to run it. There are a few ways to launch the Cloud Shell:

Click Try It in the upper right corner of a code block. Cloud Shell in this article
Open Cloud Shell in your browser. https://shell.azure.com/bash
Click the Cloud Shell button on the menu in the upper right of the Azure portal. Cloud Shell in the portal

If you choose to install and use the CLI locally, this quickstart requires that you are running the Azure CLI version 2.0.20 or later. Run az --version to find the version. If you need to install or upgrade, see Install Azure CLI 2.0.

Create a resource group

Create a resource group with the az group create command. An Azure resource group is a logical container into which Azure resources are deployed and managed.

The following example creates a resource group named myResourceGroup in the eastus2 location.

az group create \
    --name myResourceGroup \
    --location eastus2

Create a storage account

You can link an Azure general-purpose storage account with your Batch account. Although not required for this quickstart, the storage account is useful to deploy applications and store input and output data for most real-world workloads. Create a storage account in your resource group with the az storage account create command.

az storage account create \
    --resource-group myResourceGroup \
    --name mystorageaccount \
    --location eastus2 \
    --sku Standard_LRS

Create a Batch account

Create a Batch account with the az batch account create command. You need an account to create compute resources (pools of compute nodes) and Batch jobs.

The following example creates a Batch account named mybatchaccount in myResourceGroup, and links the storage account you created.

az batch account create \
    --name mybatchaccount \
    --storage-account mystorageaccount \
    --resource-group myResourceGroup \
    --location eastus2

To create and manage compute pools and jobs, you need to authenticate with Batch. Log in to the account with the az batch account login command. After you log in, your az batch commands use this account context.

az batch account login \
    --name mybatchaccount \
    --resource-group myResourceGroup \

Create a pool of compute nodes

Now that you have a Batch account, create a sample pool of Linux compute nodes using the az batch pool create command. The following example creates a pool named mypool of 2 size Standard_A1_v2 nodes running Ubuntu 16.04 LTS. The suggested node size offers a good balance of performance versus cost for this quick example.

az batch pool create \
    --id mypool --vm-size Standard_A1_v2 \
    --target-dedicated-nodes 2 \
    --image canonical:ubuntuserver:16.04.0-LTS \
    --node-agent-sku-id "batch.node.ubuntu 16.04" 

Batch creates the pool immediately, but it takes a few minutes to allocate and start the compute nodes. During this time, the pool is in the resizing state. To see the status of the pool, run the az batch pool show command. This command shows all the properties of the pool, and you can query for specific properties. The following command gets the allocation state of the pool:

az batch pool show --pool-id mypool \
    --query "allocationState"

Continue the following steps to create a job and tasks while the pool state is changing. The pool is ready to run tasks when the allocation state is steady and all the nodes are running.

Create a job

Now that you have a pool, create a job to run on it. A Batch job is a logical group for one or more tasks. A job includes settings common to the tasks, such as priority and the pool to run tasks on. Create a Batch job by using the az batch job create command. The following example creates a job myjob on the pool mypool. Initially the job has no tasks.

az batch job create \
    --id myjob \
    --pool-id mypool

Create tasks

Now use the az batch task create command to create some tasks to run in the job. In this example, you create four identical tasks. Each task runs a command-line to display the Batch environment variables on a compute node, and then waits 90 seconds. When you use Batch, this command line is where you specify your app or script. Batch provides a number of ways to deploy apps and scripts to compute nodes.

The following Bash script creates 4 parallel tasks (mytask1 to mytask4).

for i in {1..4}
   az batch task create \
    --task-id mytask$i \
    --job-id myjob \
    --command-line "/bin/bash -c 'printenv | grep AZ_BATCH; sleep 90s'"

The command output shows settings for each of the tasks. Batch distributes the tasks to the compute nodes.

View task status

After you create a task, Batch queues it to run on the pool. Once a node is available to run it, the task runs.

Use the az batch task show command to view the status of the Batch tasks. The following example shows details about mytask1 running on one of the pool nodes.

az batch task show \
    --job-id myjob \
    --task-id mytask1

The command output includes many details, but take note of the exitCode of the task command line and the nodeId. An exitCode of 0 indicates that the task command line completed successfully. The nodeId indicates the ID of the pool node on which the task ran.

View task output

To list the files created by a task on a compute node, use the az batch task file list command. The following command lists the files created by mytask1:

az batch task file list \
    --job-id myjob \
    --task-id mytask1 \
    --output table

Output is similar to the following:

Name        URL                                                                                         Is Directory      Content Length
----------  ------------------------------------------------------------------------------------------  --------------  ----------------
stdout.txt  https://mybatchaccount.eastus2.batch.azure.com/jobs/myjob/tasks/mytask1/files/stdout.txt  False                  695
certs       https://mybatchaccount.eastus2.batch.azure.com/jobs/myjob/tasks/mytask1/files/certs       True
wd          https://mybatchaccount.eastus2.batch.azure.com/jobs/myjob/tasks/mytask1/files/wd          True
stderr.txt  https://mybatchaccount.eastus2.batch.azure.com/jobs/myjob/tasks/mytask1/files/stderr.txt  False                     0

To download one of the output files to a local directory, use the az batch task file download command. In this example, task output is in stdout.txt.

az batch task file download \
    --job-id myjob \
    --task-id mytask1 \
    --file-path stdout.txt \
    --destination ./stdout.txt

You can view the contents of stdout.txt in a text editor. The contents show the Azure Batch environment variables that are set on the node. When you create your own Batch jobs, you can reference these environment variables in task command lines, and in the apps and scripts run by the command lines.


Clean up resources

If you want to continue with Batch tutorials and samples, use the Batch account and linked storage account created in this quickstart. There is no charge for the Batch account itself.

You are charged for pools while the nodes are running, even if no jobs are scheduled. When you no longer need a pool, delete it with the az batch pool delete command:

az batch pool delete --pool-id mypool

When no longer needed, you can use the az group delete command to remove the resource group, Batch account, pools, and all related resources. Delete the resources as follows:

az group delete --name myResourceGroup

Next steps

In this quickstart, you created a Batch account, a Batch pool, and a Batch job. The job ran sample tasks, and you viewed output created on one of the nodes. Now that you understand the key concepts of the Batch service, you are ready to try Batch with more realistic workloads at larger scale. To learn more about Azure Batch, continue to the Azure Batch tutorials.