Processing parallel workloads with Azure Batch is typically done programmatically by using one of the Batch APIs. Your client application or service can use the Batch APIs to communicate with the Batch service. With the Batch APIs, you can create and manage pools of compute nodes, either virtual machines or cloud services. You can then schedule jobs and tasks to run on those nodes.
You can efficiently process large-scale workloads for your organization, or provide a service front end to your customers so that they can run jobs and tasks--on demand, or on a schedule--on one, hundreds, or even thousands of nodes. You can also use Azure Batch as part of a larger workflow, managed by tools such as Azure Data Factory.
When you're ready to dig in to the Batch API for a more in-depth understanding of the features it provides, check out the Batch feature overview for developers.
Azure accounts for Batch development
When you develop Batch solutions, you'll use the following accounts in Microsoft Azure.
- Azure account and subscription - If you don't already have an Azure subscription, you can activate your MSDN subscriber benefit, or sign up for a free Azure account. When you create an account, a default subscription is created for you.
- Batch account - Azure Batch resources, including pools, compute nodes, jobs, and tasks, are associated with an Azure Batch account. When your application makes a request against the Batch service, it authenticates the request using the Azure Batch account name, the URL of the account, and an access key. You can create Batch account in the Azure portal.
- Storage account - Batch includes built-in support for working with files in Azure Storage. Nearly every Batch scenario uses Azure Blob storage for staging the programs that your tasks run and the data that they process, and for the storage of output data that they generate. To create a Storage account, see About Azure storage accounts.
Batch service APIs
Your applications and services can issue direct REST API calls or use one or more of the following client libraries to run and manage your Azure Batch workloads.
|API||API reference||Download||Tutorial||Code samples||More Info|
|Batch REST||MSDN||N/A||-||-||Supported Versions|
|Batch .NET||docs.microsoft.com||NuGet||Tutorial||GitHub||Release Notes|
Batch Management APIs
The Azure Resource Manager APIs for Batch provide programmatic access to Batch accounts. Using these APIs, you can programmatically manage Batch accounts, quotas, and application packages.
|API||API reference||Download||Tutorial||Code samples|
|Batch Resource Manager REST||docs.microsoft.com||N/A||-||GitHub|
|Batch Resource Manager .NET||docs.microsoft.com||NuGet||Tutorial||GitHub|
Batch command-line tools
These command-line tools provide the same functionality as the Batch service and Batch Management APIs:
- Batch PowerShell cmdlets: The Azure Batch cmdlets in the Azure PowerShell module enable you to manage Batch resources with PowerShell.
- Azure CLI: The Azure Command-Line Interface (Azure CLI) is a cross-platform toolset that provides shell commands for interacting with many Azure services, including the Batch service and Batch Management service. See Manage Batch resources with Azure CLI for more information about using the Azure CLI with Batch.
Other tools for application development
Here are some additional tools that may be helpful for building and debugging your Batch applications and services:
- Azure portal: You can create, monitor, and delete Batch pools, jobs, and tasks in the Azure portal's Batch blades. You can view the status information for these and other resources while you run your jobs, and even download files from the compute nodes in your pools. For example, you can download a failed task's
stderr.txtwhile troubleshooting. You can also download Remote Desktop (RDP) files that you can use to log in to compute nodes.
- Azure Batch Explorer: Batch Explorer provides similar Batch resource management functionality as the Azure portal, but in a standalone Windows Presentation Foundation (WPF) client application. One of the Batch .NET sample applications available on GitHub, you can build it with Visual Studio 2015 or newer and use it to browse and manage the resources in your Batch account while you develop and debug your Batch solutions. View job, pool, and task details, download files from compute nodes, and connect to nodes remotely by using Remote Desktop (RDP) files you can download with Batch Explorer.
- Microsoft Azure Storage Explorer: While not strictly an Azure Batch tool, the Storage Explorer is another valuable tool to have while you are developing and debugging your Batch solutions.
- To learn about logging events from your Batch application, see Log events for diagnostic evaluation and monitoring of Batch solutions. For a reference on events raised by the Batch service, see Batch Analytics.
- For information about environment variables for compute nodes, see Azure Batch compute node environment variables.
- Read the Batch feature overview for developers, essential information for anyone preparing to use Batch. The article contains more detailed information about Batch service resources like pools, nodes, jobs, and tasks, and the many API features that you can use while building your Batch application.
- Get started with the Azure Batch library for .NET to learn how to use C# and the Batch .NET library to execute a simple workload using a common Batch workflow. This article should be one of your first stops while learning how to use the Batch service. There is also a Python version of the tutorial.
- Download the code samples on GitHub to see how both C# and Python can interface with Batch to schedule and process sample workloads.
- Check out the Batch Learning Path to get an idea of the resources available to you as you learn to work with Batch.