az ml data

Note

This reference is part of the ml extension for Azure CLI and requires version 2.15.0 or higher. The extension will automatically install the first time you run an az ml data command. Learn more about extensions.

Manage Azure ML data assets.

Azure ML data assets are references to file(s) in your storage services or public URLs along with any corresponding metadata. They are not copies of your data. You can use these data assets to access relevant data during model training and mount or download the referenced data to your compute target.

Commands

az ml data create

Create a data asset.

az ml data delete

Delete a data asset.

az ml data list

List data assets in a workspace.

az ml data show

Shows details for a data asset.

az ml data update

Update a data asset.

az ml data create

Create a data asset.

Data assets can be defined from files on your local machine or as references to files in cloud storage. The created data asset will be tracked in the workspace under the specified name and version.

To create a data asset from file(s) on your local machine, specify the 'local_path' field in your YAML config. Azure ML will upload these file(s) to the blob container that backs the workspace's default datastore (named 'workspaceblobstore'). The created data asset will then point to that uploaded data.

To create a data asset that references file(s) in cloud storage, specify the 'datastore' that corresponds to the storage service and the 'path' to the file(s) in storage in your YAML config.

You can also create a data asset directly from a storage URL or public URL. To do so, specify the URL to the 'path' field in your YAML config.

az ml data create --resource-group
                  --workspace-name
                  [--datastore-name]
                  [--file]
                  [--name]
                  [--set]
                  [--version]

Examples

Create a data asset from a YAML specification file

az ml data create --file data.yml --resource-group my-resource-group --workspace-name my-workspace

Required Parameters

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

--workspace-name -w

Name of the Azure ML workspace. You can configure the default group using az configure --defaults workspace=<name>.

Optional Parameters

--datastore-name -z

Name of the datastore to upload the data to.

--file -f

Local path to the YAML file containing the Azure ML data specification.

--name -n

Name of the data asset.

--set

Update an object by specifying a property path and value to set. Example: --set property1.property2=.

--version -v

Version of the data asset.

az ml data delete

Delete a data asset.

az ml data delete --name
                  --resource-group
                  --version
                  --workspace-name

Required Parameters

--name -n

Name of the data asset.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

--version -v

Version of the data asset.

--workspace-name -w

Name of the Azure ML workspace. You can configure the default group using az configure --defaults workspace=<name>.

az ml data list

List data assets in a workspace.

az ml data list --resource-group
                --workspace-name
                [--max-results]
                [--name]

Examples

List all the data assets in a workspace

az ml data list --resource-group my-resource-group --workspace-name my-workspace

List all the data asset versions for the specified name in a workspace

az ml data list --name my-data --resource-group my-resource-group --workspace-name my-workspace

List all the data assets in a workspace using --query argument to execute a JMESPath query on the results of commands.

az ml data list --query "[].{Name:name}" --output table --resource-group my-resource-group --workspace-name my-workspace

Required Parameters

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

--workspace-name -w

Name of the Azure ML workspace. You can configure the default group using az configure --defaults workspace=<name>.

Optional Parameters

--max-results -r

Max number of results to return.

--name -n

Name of the data asset. If provided, all the data versions under this name will be returned.

az ml data show

Shows details for a data asset.

az ml data show --name
                --resource-group
                --workspace-name
                [--version]

Examples

Show details for a data asset with the specified name and version

az ml data show --name my-data --version 1 --resource-group my-resource-group --workspace-name my-workspace

Show details for the latest version of a data asset with the specified name

az ml data show --name my-data --resource-group my-resource-group --workspace-name my-workspace

Required Parameters

--name -n

Name of the data asset.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

--workspace-name -w

Name of the Azure ML workspace. You can configure the default group using az configure --defaults workspace=<name>.

Optional Parameters

--version -v

Version of the data asset. If omitted, the latest version is shown.

az ml data update

Update a data asset.

Only the 'description' and 'tags' properties can be updated.

az ml data update --resource-group
                  --workspace-name
                  [--add]
                  [--force-string]
                  [--name]
                  [--remove]
                  [--set]
                  [--version]

Required Parameters

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

--workspace-name -w

Name of the Azure ML workspace. You can configure the default group using az configure --defaults workspace=<name>.

Optional Parameters

--add

Add an object to a list of objects by specifying a path and key value pairs. Example: --add property.listProperty <key=value, string or JSON string>.

--force-string

When using 'set' or 'add', preserve string literals instead of attempting to convert to JSON.

--name -n

Name of the data asset.

--remove

Remove a property or an element from a list. Example: --remove property.list OR --remove propertyToRemove.

--set

Update an object by specifying a property path and value to set. Example: --set property1.property2=.

--version -v

Version of the data asset.