Files and directories in Azure Batch
In Azure Batch, each task has a working directory under which it can create files and directories. This working directory can be used for storing the program that is run by the task, the data that it processes, and the output of the processing it performs. All files and directories of a task are owned by the task user.
The Batch service exposes a portion of the file system on a node as the root directory. This root directory is located on the temporary storage drive of the VM, not directly on the OS drive.
Tasks can access the root directory by referencing the
AZ_BATCH_NODE_ROOT_DIR environment variable. For more information about using environment variables, see Environment settings for tasks.
Root directory structure
The root directory contains the following directory structure:
applications: Contains information about the details of application packages installed on the compute node. Tasks can access this directory by referencing the
fsmounts: The directory contains any file systems that are mounted on a compute node. Tasks can access this directory by referencing the
AZ_BATCH_NODE_MOUNTS_DIRenvironment variable. For more information, see Mount a virtual file system on a Batch pool.
shared: This directory provides read/write access to all tasks that run on a node. Any task that runs on the node can create, read, update, and delete files in this directory. Tasks can access this directory by referencing the
startup: This directory is used by a start task as its working directory. All of the files that are downloaded to the node by the start task are stored here. The start task can create, read, update, and delete files under this directory. Tasks can access this directory by referencing the
volatile: This directory is for internal purposes. There's no guarantee that any files in this directory or that the directory itself will exist in the future.
workitems: This directory contains the directories for jobs and their tasks on the compute node.
Within the workitems directory, a Tasks directory is created for each task that runs on the node. This directory can be accessed by referencing the
Within each Tasks directory, the Batch service creates a working directory (
wd) whose unique path is specified by the
AZ_BATCH_TASK_WORKING_DIRenvironment variable. This directory provides read/write access to the task. The task can create, read, update, and delete files under this directory. This directory is retained based on the RetentionTime constraint that is specified for the task.
stderr.txtfiles are written to the Tasks folder during the execution of the task.
When a node is removed from the pool, all of the files that are stored on the node are removed.
- Learn about error handling and detection in Azure Batch.