az ml job
Note
This reference is part of the ml extension for the Azure CLI (version 2.15.0 or higher). The extension will automatically install the first time you run an az ml job command. Learn more about extensions.
Manage Azure ML jobs.
An Azure ML job executes a task against a specified compute target. You can configure jobs to scale out model training on Azure. Azure ML supports different job types with different capabilities. For example, the most basic job, a command job, executes a command in a Docker container and can be leveraged for single-node and distributed training. A sweep job executes a hyperparameter sweep over a specified search space for tuning a model's hyperparameters.
Jobs also enable systematic tracking for your ML experimentation and workflows. Once a job is created, Azure ML maintains a run record for the job that includes the metadata, any metrics, logs, and artifacts generated during the job, code that was executed, and the Azure ML environment used. All of your jobs' run records can be viewed in Azure ML studio.
Commands
| Name | Description | Type | Status |
|---|---|---|---|
| az ml job archive |
Archive a job. |
Extension | GA |
| az ml job cancel |
Cancel a job. |
Extension | GA |
| az ml job connect-ssh |
Set up ssh connection and sends the request to the SSH service running inside user's container through Tundra. |
Extension | GA |
| az ml job create |
Create a job. |
Extension | GA |
| az ml job download |
Download all job-related files. |
Extension | GA |
| az ml job list |
List jobs in a workspace. |
Extension | GA |
| az ml job restore |
Restore an archived job. |
Extension | GA |
| az ml job show |
Show details for a job. |
Extension | GA |
| az ml job show-services |
Show services of a job per node. |
Extension | GA |
| az ml job stream |
Stream job logs to the console. |
Extension | GA |
| az ml job update |
Update a job. |
Extension | GA |
| az ml job validate |
Validate a job. This command works for pipeline jobs only for now. |
Extension | GA |
az ml job archive
Archive a job.
Archiving a job will hide it by default from list queries (az ml job list). You can still continue to reference and use an archived job in your workflows. Only completed jobs can be archived.
az ml job archive --name
--resource-group
--workspace-nameRequired Parameters
Name of the job.
Name of resource group. You can configure the default group using az configure --defaults group=<name>.
Name of the Azure ML workspace. You can configure the default workspace using az configure --defaults workspace=<name>.
Global Parameters
Increase logging verbosity to show all debug logs.
Show this help message and exit.
Only show errors, suppressing warnings.
Output format.
JMESPath query string. See http://jmespath.org/ for more information and examples.
Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.
Increase logging verbosity. Use --debug for full debug logs.
az ml job cancel
Cancel a job.
az ml job cancel --name
--resource-group
--workspace-nameExamples
Cancel a job by name
az ml job cancel --name my-job-id --resource-group my-resource-group --workspace-name my-workspaceRequired Parameters
Name of the job.
Name of resource group. You can configure the default group using az configure --defaults group=<name>.
Name of the Azure ML workspace. You can configure the default workspace using az configure --defaults workspace=<name>.
Global Parameters
Increase logging verbosity to show all debug logs.
Show this help message and exit.
Only show errors, suppressing warnings.
Output format.
JMESPath query string. See http://jmespath.org/ for more information and examples.
Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.
Increase logging verbosity. Use --debug for full debug logs.
az ml job connect-ssh
Set up ssh connection and sends the request to the SSH service running inside user's container through Tundra.
az ml job connect-ssh --name
--resource-group
--workspace-name
[--node-index]
[--private-key-file-path]Examples
Set up ssh connection and sends the request to the SSH service.
az ml job connect-ssh --name my-job-id --node-index 0 --private-key-file-path "C:/Temp/.ssh/id_rsa" --resource-group my-resource-group --workspace-name my-workspaceRequired Parameters
Name of the job.
Name of resource group. You can configure the default group using az configure --defaults group=<name>.
Name of the Azure ML workspace. You can configure the default workspace using az configure --defaults workspace=<name>.
Optional Parameters
The index of the node to connect through ssh.
The path to the private key file file.
Global Parameters
Increase logging verbosity to show all debug logs.
Show this help message and exit.
Only show errors, suppressing warnings.
Output format.
JMESPath query string. See http://jmespath.org/ for more information and examples.
Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.
Increase logging verbosity. Use --debug for full debug logs.
az ml job create
Create a job.
To create a job, you will typically need to configure any code to be run, an environment encapsulating the dependencies, a compute target to execute the job on, and any additional job-specific settings. When a job is created, it is submitted for execution against the specified compute resource.
az ml job create --file
--resource-group
--workspace-name
[--name]
[--save-as]
[--set]
[--skip-validation]
[--stream]
[--web]Examples
Create a job from a YAML specification file
az ml job create --file job.yml --resource-group my-resource-group --workspace-name my-workspaceCreate a job from a YAML specification file and open the job's run details in the Azure ML studio portal
az ml job create --file job.yml --web --resource-group my-resource-group --workspace-name my-workspaceRequired Parameters
Local path to the YAML file containing the Azure ML job specification. The YAML reference docs for job can be found at: https://aka.ms/ml-cli-v2-job-command-yaml-reference, https://aka.ms/ml-cli-v2-job-sweep-yaml-reference, https://aka.ms/ml-cli-v2-job-pipeline-yaml-reference.
Name of resource group. You can configure the default group using az configure --defaults group=<name>.
Name of the Azure ML workspace. You can configure the default workspace using az configure --defaults workspace=<name>.
Optional Parameters
Name of the job.
File to which the created job's state in YAML format will be written.
Update an object by specifying a property path and value to set. Example: --set property1.property2=.
Skip validation in creating the resource. Note that dependent resources will not skip their validation in creating.
Indicates whether to stream the job's logs to the console.
Show the job's run details in Azure ML studio in a web browser.
Global Parameters
Increase logging verbosity to show all debug logs.
Show this help message and exit.
Only show errors, suppressing warnings.
Output format.
JMESPath query string. See http://jmespath.org/ for more information and examples.
Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.
Increase logging verbosity. Use --debug for full debug logs.
az ml job download
Download all job-related files.
The files will be downloaded in a folder named after the job's name.
az ml job download --name
--resource-group
--workspace-name
[--all]
[--download-path]
[--output-name]Examples
Download a job's logs and outputs to the current working directory
az ml job download --name my-job --resource-group my-resource-group --workspace-name my-workspaceRequired Parameters
Name of the job.
Name of resource group. You can configure the default group using az configure --defaults group=<name>.
Name of the Azure ML workspace. You can configure the default workspace using az configure --defaults workspace=<name>.
Optional Parameters
Download all the outputs of the job.
Path to download the job files to. If omitted, job files will be downloaded to the current directory.
The name of the user-defined output to download. This should correspond to a key in the outputs dictionary of a job. If omitted, the job's default artifact output files will be downloaded.
Global Parameters
Increase logging verbosity to show all debug logs.
Show this help message and exit.
Only show errors, suppressing warnings.
Output format.
JMESPath query string. See http://jmespath.org/ for more information and examples.
Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.
Increase logging verbosity. Use --debug for full debug logs.
az ml job list
List jobs in a workspace.
az ml job list --resource-group
--workspace-name
[--all-results {false, true}]
[--archived-only]
[--include-archived]
[--max-results]
[--parent-job-name]Examples
List all the jobs status in a workspace using --query argument to execute a JMESPath query on the results of commands.
az ml job list --query "[].{Name:name,Jobstatus:status}" --output table --resource-group my-resource-group --workspace-name my-workspaceRequired Parameters
Name of resource group. You can configure the default group using az configure --defaults group=<name>.
Name of the Azure ML workspace. You can configure the default workspace using az configure --defaults workspace=<name>.
Optional Parameters
Returns all results.
List archived jobs only.
List archived jobs and active jobs.
Max number of results to return. Default is 50.
Name of the parent job. Will list all jobs whose parent_job_name matches the given name.
Global Parameters
Increase logging verbosity to show all debug logs.
Show this help message and exit.
Only show errors, suppressing warnings.
Output format.
JMESPath query string. See http://jmespath.org/ for more information and examples.
Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.
Increase logging verbosity. Use --debug for full debug logs.
az ml job restore
Restore an archived job.
When an archived job is restored, it will no longer be hidden from list queries (az ml job list).
az ml job restore --name
--resource-group
--workspace-nameRequired Parameters
Name of the job.
Name of resource group. You can configure the default group using az configure --defaults group=<name>.
Name of the Azure ML workspace. You can configure the default workspace using az configure --defaults workspace=<name>.
Global Parameters
Increase logging verbosity to show all debug logs.
Show this help message and exit.
Only show errors, suppressing warnings.
Output format.
JMESPath query string. See http://jmespath.org/ for more information and examples.
Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.
Increase logging verbosity. Use --debug for full debug logs.
az ml job show
Show details for a job.
az ml job show --name
--resource-group
--workspace-name
[--web]Examples
Show the status of a job using --query argument to execute a JMESPath query on the results of commands.
az ml job show --name my-job-id --query "{Name:name,Jobstatus:status}" --output table --resource-group my-resource-group --workspace-name my-workspaceRequired Parameters
Name of the job.
Name of resource group. You can configure the default group using az configure --defaults group=<name>.
Name of the Azure ML workspace. You can configure the default workspace using az configure --defaults workspace=<name>.
Optional Parameters
Show the job's run details in Azure ML studio in a web browser.
Global Parameters
Increase logging verbosity to show all debug logs.
Show this help message and exit.
Only show errors, suppressing warnings.
Output format.
JMESPath query string. See http://jmespath.org/ for more information and examples.
Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.
Increase logging verbosity. Use --debug for full debug logs.
az ml job show-services
Show services of a job per node.
az ml job show-services --name
--resource-group
--workspace-name
[--node-index]Examples
Show the services of a job per node using --query argument to execute a JMESPath query on the results of commands.
az ml job show-services --name my-job-id --node-index 0 --query "{Name:name,Jobstatus:status}" --output table --resource-group my-resource-group --workspace-name my-workspaceRequired Parameters
Name of the job.
Name of resource group. You can configure the default group using az configure --defaults group=<name>.
Name of the Azure ML workspace. You can configure the default workspace using az configure --defaults workspace=<name>.
Optional Parameters
The index of the node for which the services has to be shown.
Global Parameters
Increase logging verbosity to show all debug logs.
Show this help message and exit.
Only show errors, suppressing warnings.
Output format.
JMESPath query string. See http://jmespath.org/ for more information and examples.
Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.
Increase logging verbosity. Use --debug for full debug logs.
az ml job stream
Stream job logs to the console.
az ml job stream --name
--resource-group
--workspace-nameRequired Parameters
Name of the job.
Name of resource group. You can configure the default group using az configure --defaults group=<name>.
Name of the Azure ML workspace. You can configure the default workspace using az configure --defaults workspace=<name>.
Global Parameters
Increase logging verbosity to show all debug logs.
Show this help message and exit.
Only show errors, suppressing warnings.
Output format.
JMESPath query string. See http://jmespath.org/ for more information and examples.
Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.
Increase logging verbosity. Use --debug for full debug logs.
az ml job update
Update a job.
Only the 'tags' and 'properties' properties can be updated.
az ml job update --name
--resource-group
--workspace-name
[--add]
[--force-string]
[--remove]
[--set]
[--web]Required Parameters
Name of the job.
Name of resource group. You can configure the default group using az configure --defaults group=<name>.
Name of the Azure ML workspace. You can configure the default workspace using az configure --defaults workspace=<name>.
Optional Parameters
Add an object to a list of objects by specifying a path and key value pairs. Example: --add property.listProperty <key=value, string or JSON string>.
When using 'set' or 'add', preserve string literals instead of attempting to convert to JSON.
Remove a property or an element from a list. Example: --remove property.list <indexToRemove> OR --remove propertyToRemove.
Update an object by specifying a property path and value to set. Example: --set property1.property2=<value>.
Show the job's run details in Azure ML studio in a web browser.
Global Parameters
Increase logging verbosity to show all debug logs.
Show this help message and exit.
Only show errors, suppressing warnings.
Output format.
JMESPath query string. See http://jmespath.org/ for more information and examples.
Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.
Increase logging verbosity. Use --debug for full debug logs.
az ml job validate
Validate a job. This command works for pipeline jobs only for now.
This command will validate a YAML specification file to check if it is valid for job creation, and return all issues that were found. Validation mainly includes local checking for schema, like missing fields, environment without version specified, code referred to an non-existent local path; it will also check for the existence of referenced compute targets in the target workspace. Validation result will be printed to the console, including both errors and warnings. Only errors will cause the validation to fail. A job passed validation will be able to be submitted. This command works for pipeline jobs only for now.
az ml job validate --file
--resource-group
--workspace-name
[--set]Examples
Validate a YAML specification file to check if it is valid for job creation.
az ml job validate --file job.yml --resource-group my-resource-group --workspace-name my-workspaceRequired Parameters
Local path to the YAML file containing the Azure ML job specification. The YAML reference docs for job can be found at: https://aka.ms/ml-cli-v2-job-pipeline-yaml-reference.
Name of resource group. You can configure the default group using az configure --defaults group=<name>.
Name of the Azure ML workspace. You can configure the default workspace using az configure --defaults workspace=<name>.
Optional Parameters
Update an object by specifying a property path and value to set. Example: --set property1.property2=.
Global Parameters
Increase logging verbosity to show all debug logs.
Show this help message and exit.
Only show errors, suppressing warnings.
Output format.
JMESPath query string. See http://jmespath.org/ for more information and examples.
Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.
Increase logging verbosity. Use --debug for full debug logs.
Feedback
Coming soon: Throughout 2024 we will be phasing out GitHub Issues as the feedback mechanism for content and replacing it with a new feedback system. For more information see: https://aka.ms/ContentUserFeedback.
Submit and view feedback for