az ml datastore

Commands for managing and using datastores with the Azure ML Workspace.

Commands

az ml datastore attach-adls

Attach an ADLS datastore.

az ml datastore attach-adls-gen2

Attach a ADLS Gen2 datastore.

az ml datastore attach-blob

Attach a blob storage datastore.

az ml datastore attach-dbfs

Attach a Databricks File System datastore.

az ml datastore attach-file

Attach a file share datastore.

az ml datastore attach-mysqldb

Attach an Azure MySQL datastore.

az ml datastore attach-psqldb

Attach an Azure PostgreSQL datastore.

az ml datastore attach-sqldb

Attach an Azure SQL datastore.

az ml datastore detach

Detach a datastore by name.

az ml datastore download

Download files from a Datastore.

az ml datastore list

List datastores in the workspace.

az ml datastore set-default

Set the workspace default datastore by name.

az ml datastore show

Show a single datastore by name.

az ml datastore show-default

Show the workspace default datastore.

az ml datastore upload

Upload files to a Datastore.

az ml datastore attach-adls

Attach an ADLS datastore.

az ml datastore attach-adls --client-id
                            --client-secret
                            --name
                            --store-name
                            --tenant-id
                            [--adls-resource-group]
                            [--adls-subscription-id]
                            [--authority-url]
                            [--grant-workspace-msi-access]
                            [--include-secret]
                            [--output-metadata-file]
                            [--path]
                            [--resource-group]
                            [--resource-url]
                            [--subscription-id]
                            [--workspace-name]

Required Parameters

--client-id

The service principal's client/application ID.

--client-secret

The service principal's secret.

--name -n

The datastore name.

--store-name -c

The ADLS store name.

--tenant-id

The service principal Tenant ID.

Optional Parameters

--adls-resource-group

The resource group the ADLS store belongs to.

--adls-subscription-id

The ID of the subscription the ADLS store belongs to.

--authority-url

Authority url used to authenticate the user.

default value: <a href="https://login.microsoftonline.com">https://login.microsoftonline.com</a>
--grant-workspace-msi-access

Defaults to False. Set it to True to access data behind virtual network from Machine Learning Studio. This makes data access from Machine Learning Studio use workspace managed identity for authentication, You have to be Owner or User Access Administrator of the storage to opt-in. Ask your administrator to configure it for you if you do not have the required permission. Learn more 'https://docs.microsoft.com/azure/machine-learning/how-to-enable-studio-virtual-network.

--include-secret

Show the registered secret for the datastores.

--output-metadata-file -t

Provide an optional output file location for structured object output.

--path

Path to a root directory for run configuration files.

default value: .
--resource-group -g

Resource group name.

--resource-url

Determines what operations will be performed on the data lake store. Defaults to https://datalake.azure.net/ allowing for filesystem operations.

default value: <a href="https://datalake.azure.net/">https://datalake.azure.net/</a>
--subscription-id

Specifies the subscription Id.

--workspace-name -w

Workspace name.

az ml datastore attach-adls-gen2

Attach a ADLS Gen2 datastore.

az ml datastore attach-adls-gen2 --account-name
                                 --client-id
                                 --client-secret
                                 --file-system
                                 --name
                                 --tenant-id
                                 [--adlsgen2-account-resource-group]
                                 [--adlsgen2-account-subscription-id]
                                 [--authority-url]
                                 [--endpoint]
                                 [--grant-workspace-msi-access]
                                 [--include-secret]
                                 [--output-metadata-file]
                                 [--path]
                                 [--protocol]
                                 [--resource-group]
                                 [--resource-url]
                                 [--subscription-id]
                                 [--workspace-name]

Required Parameters

--account-name -a

The name of the storage account.

--client-id

The service principal's client/application ID.

--client-secret

The service principal's secret.

--file-system -c

The file system name of the ADLS Gen2.

--name -n

The datastore name.

--tenant-id

The service principal Tenant ID.

Optional Parameters

--adlsgen2-account-resource-group

The resource group of the ADLS Gen2 storage account.

--adlsgen2-account-subscription-id

The subscription ID of the ADLS Gen2 storage account.

--authority-url

Authority url used to authenticate the user.

default value: <a href="https://login.microsoftonline.com">https://login.microsoftonline.com</a>
--endpoint

The endpoint of the storage account. Defaults to core.windows.net.

default value: core.windows.net
--grant-workspace-msi-access

Defaults to False. Set it to True to access data behind virtual network from Machine Learning Studio. This makes data access from Machine Learning Studio use workspace managed identity for authentication, You have to be Owner or User Access Administrator of the storage to opt-in. Ask your administrator to configure it for you if you do not have the required permission. Learn more 'https://docs.microsoft.com/azure/machine-learning/how-to-enable-studio-virtual-network.

--include-secret

Show the registered secret for the datastores.

--output-metadata-file -t

Provide an optional output file location for structured object output.

--path

Path to a root directory for run configuration files.

default value: .
--protocol

Protocol to use to connect to the blob container. If not specified, defaults to https.

default value: https
--resource-group -g

Resource group name.

--resource-url

Determines what operations will be performed on the data lake store. Defaults to https://storage.azure.com/ allowing for filesystem operations.

default value: <a href="https://storage.azure.com/">https://storage.azure.com/</a>
--subscription-id

Specifies the subscription Id.

--workspace-name -w

Workspace name.

az ml datastore attach-blob

Attach a blob storage datastore.

az ml datastore attach-blob --account-name
                            --container-name
                            --name
                            [--account-key]
                            [--endpoint]
                            [--grant-workspace-msi-access]
                            [--include-secret]
                            [--output-metadata-file]
                            [--path]
                            [--protocol]
                            [--resource-group]
                            [--sas-token]
                            [--storage-account-resource-group]
                            [--storage-account-subscription-id]
                            [--subscription-id]
                            [--workspace-name]

Required Parameters

--account-name -a

The name of the storage account.

--container-name -c

The blob container name.

--name -n

The datastore name.

Optional Parameters

--account-key -k

The storage account key.

--endpoint

The endpoint of the storage account. Defaults to core.windows.net.

default value: core.windows.net
--grant-workspace-msi-access

Defaults to False. Set it to True to access data behind virtual network from Machine Learning Studio. This makes data access from Machine Learning Studio use workspace managed identity for authentication, You have to be Owner or User Access Administrator of the storage to opt-in. Ask your administrator to configure it for you if you do not have the required permission. Learn more 'https://docs.microsoft.com/azure/machine-learning/how-to-enable-studio-virtual-network.

--include-secret

Show the registered secret for the datastores.

--output-metadata-file -t

Provide an optional output file location for structured object output.

--path

Path to a root directory for run configuration files.

default value: .
--protocol

Protocol to use to connect to the blob container. If not specified, defaults to https.

default value: https
--resource-group -g

Resource group name.

--sas-token

A SAS token for the blob container.

--storage-account-resource-group

The resource group of the storage account.

--storage-account-subscription-id

The subscription ID of the storage account.

--subscription-id

Specifies the subscription Id.

--workspace-name -w

Workspace name.

az ml datastore attach-dbfs

Attach a Databricks File System datastore.

az ml datastore attach-dbfs --name
                            [--output-metadata-file]
                            [--path]
                            [--resource-group]
                            [--subscription-id]
                            [--workspace-name]

Required Parameters

--name -n

The datastore name.

Optional Parameters

--output-metadata-file -t

Provide an optional output file location for structured object output.

--path

Path to a root directory for run configuration files.

default value: .
--resource-group -g

Resource group name.

--subscription-id

Specifies the subscription Id.

--workspace-name -w

Workspace name.

az ml datastore attach-file

Attach a file share datastore.

az ml datastore attach-file --account-name
                            --name
                            --share-name
                            [--account-key]
                            [--endpoint]
                            [--include-secret]
                            [--output-metadata-file]
                            [--path]
                            [--protocol]
                            [--resource-group]
                            [--sas-token]
                            [--subscription-id]
                            [--workspace-name]

Required Parameters

--account-name -a

The name of the storage account.

--name -n

The datastore name.

--share-name -c

The file share name.

Optional Parameters

--account-key -k

The storage account key.

--endpoint

The endpoint of the storage account. Defaults to core.windows.net.

default value: core.windows.net
--include-secret

Show the registered secret for the datastores.

--output-metadata-file -t

Provide an optional output file location for structured object output.

--path

Path to a root directory for run configuration files.

default value: .
--protocol

Protocol to use to connect to the blob container. If not specified, defaults to https.

default value: https
--resource-group -g

Resource group name.

--sas-token

A SAS token for the blob container.

--subscription-id

Specifies the subscription Id.

--workspace-name -w

Workspace name.

az ml datastore attach-mysqldb

Attach an Azure MySQL datastore.

az ml datastore attach-mysqldb --database-name
                               --name
                               --password
                               --server-name
                               --user-id
                               [--endpoint]
                               [--include-secret]
                               [--output-metadata-file]
                               [--path]
                               [--port]
                               [--resource-group]
                               [--subscription-id]
                               [--workspace-name]

Required Parameters

--database-name -d

The database name.

--name -n

The datastore name.

--password -p

The password.

--server-name

The SQL/PostgreSQL/MySQL server name.

--user-id -u

The user ID.

Optional Parameters

--endpoint

The endpoint of the server. Defaults to mysql.database.azure.com.

default value: mysql.database.azure.com
--include-secret

Show the registered secret for the datastores.

--output-metadata-file -t

Provide an optional output file location for structured object output.

--path

Path to a root directory for run configuration files.

default value: .
--port

The port number.

default value: 5432
--resource-group -g

Resource group name.

--subscription-id

Specifies the subscription Id.

--workspace-name -w

Workspace name.

az ml datastore attach-psqldb

Attach an Azure PostgreSQL datastore.

az ml datastore attach-psqldb --database-name
                              --name
                              --password
                              --server-name
                              --user-id
                              [--endpoint]
                              [--enforce-ssl]
                              [--include-secret]
                              [--output-metadata-file]
                              [--path]
                              [--port]
                              [--resource-group]
                              [--subscription-id]
                              [--workspace-name]

Required Parameters

--database-name -d

The database name.

--name -n

The datastore name.

--password -p

The password.

--server-name

The SQL/PostgreSQL/MySQL server name.

--user-id -u

The user ID.

Optional Parameters

--endpoint

The endpoint of the server. Defaults to postgres.database.azure.com.

default value: postgres.database.azure.com
--enforce-ssl

This sets the ssl value of the server. Defaults to true if not set.

default value: 1
--include-secret

Show the registered secret for the datastores.

--output-metadata-file -t

Provide an optional output file location for structured object output.

--path

Path to a root directory for run configuration files.

default value: .
--port

The port number.

default value: 5432
--resource-group -g

Resource group name.

--subscription-id

Specifies the subscription Id.

--workspace-name -w

Workspace name.

az ml datastore attach-sqldb

Attach an Azure SQL datastore.

az ml datastore attach-sqldb --database-name
                             --name
                             --server-name
                             [--authority-url]
                             [--client-id]
                             [--client-secret]
                             [--endpoint]
                             [--grant-workspace-msi-access]
                             [--include-secret]
                             [--output-metadata-file]
                             [--password]
                             [--path]
                             [--resource-group]
                             [--resource-url]
                             [--sql-resource-group]
                             [--sql-subscription-id]
                             [--subscription-id]
                             [--tenant-id]
                             [--username]
                             [--workspace-name]

Required Parameters

--database-name -d

The database name.

--name -n

The datastore name.

--server-name

The SQL/PostgreSQL/MySQL server name.

Optional Parameters

--authority-url

Authority url used to authenticate the user.

default value: <a href="https://login.microsoftonline.com">https://login.microsoftonline.com</a>
--client-id

The service principal/application ID.

--client-secret

The service principal's secret.

--endpoint

The endpoint of the sql server. Defaults to database.windows.net.

default value: database.windows.net
--grant-workspace-msi-access

Defaults to False. Set it to True to access data behind virtual network from Machine Learning Studio. This makes data access from Machine Learning Studio use workspace managed identity for authentication, You have to be Owner or User Access Administrator of the storage to opt-in. Ask your administrator to configure it for you if you do not have the required permission. Learn more 'https://docs.microsoft.com/azure/machine-learning/how-to-enable-studio-virtual-network.

--include-secret

Show the registered secret for the datastores.

--output-metadata-file -t

Provide an optional output file location for structured object output.

--password

The password of the database user to access the database.

--path

Path to a root directory for run configuration files.

default value: .
--resource-group -g

Resource group name.

--resource-url

Determines what operations will be performed on the database.

default value: <a href="https://database.windows.net/">https://database.windows.net/</a>
--sql-resource-group

The resource group of the Azure Sql Server.

--sql-subscription-id

The subscription ID of the Azure Sql Server.

--subscription-id

Specifies the subscription Id.

--tenant-id

The service principal Tenant ID.

--username

The username of the database user to access the database.

--workspace-name -w

Workspace name.

az ml datastore detach

Detach a datastore by name.

az ml datastore detach --name
                       [--output-metadata-file]
                       [--path]
                       [--resource-group]
                       [--subscription-id]
                       [--workspace-name]

Required Parameters

--name -n

The datastore name.

Optional Parameters

--output-metadata-file -t

Provide an optional output file location for structured object output.

--path

Path to a root directory for run configuration files.

default value: .
--resource-group -g

Resource group name.

--subscription-id

Specifies the subscription Id.

--workspace-name -w

Workspace name.

az ml datastore download

Download files from a Datastore.

az ml datastore download --name
                         --target-path
                         [--hide-progress]
                         [--output-metadata-file]
                         [--overwrite]
                         [--path]
                         [--prefix]
                         [--resource-group]
                         [--subscription-id]
                         [--workspace-name]

Required Parameters

--name -n

The datastore name.

--target-path -d

Target path for the downloaded files.

Optional Parameters

--hide-progress

Whether to hide progress of operation.

--output-metadata-file -t

Provide an optional output file location for structured object output.

--overwrite

Overwrite target files if they exist.

--path

Path to a root directory for run configuration files.

default value: .
--prefix -p

Path filter for files to download. If none is provided, downloads everything.

--resource-group -g

Resource group name.

--subscription-id

Specifies the subscription Id.

--workspace-name -w

Workspace name.

az ml datastore list

List datastores in the workspace.

az ml datastore list [--include-secret]
                     [--output-metadata-file]
                     [--path]
                     [--resource-group]
                     [--subscription-id]
                     [--workspace-name]

Optional Parameters

--include-secret

Show the registered secret for the datastores.

--output-metadata-file -t

Provide an optional output file location for structured object output.

--path

Path to a root directory for run configuration files.

default value: .
--resource-group -g

Resource group name.

--subscription-id

Specifies the subscription Id.

--workspace-name -w

Workspace name.

az ml datastore set-default

Set the workspace default datastore by name.

az ml datastore set-default --name
                            [--output-metadata-file]
                            [--path]
                            [--resource-group]
                            [--subscription-id]
                            [--workspace-name]

Required Parameters

--name -n

The datastore name.

Optional Parameters

--output-metadata-file -t

Provide an optional output file location for structured object output.

--path

Path to a root directory for run configuration files.

default value: .
--resource-group -g

Resource group name.

--subscription-id

Specifies the subscription Id.

--workspace-name -w

Workspace name.

az ml datastore show

Show a single datastore by name.

az ml datastore show --name
                     [--include-secret]
                     [--output-metadata-file]
                     [--path]
                     [--resource-group]
                     [--subscription-id]
                     [--workspace-name]

Required Parameters

--name -n

The datastore name.

Optional Parameters

--include-secret

Show the registered secret for the datastores.

--output-metadata-file -t

Provide an optional output file location for structured object output.

--path

Path to a root directory for run configuration files.

default value: .
--resource-group -g

Resource group name.

--subscription-id

Specifies the subscription Id.

--workspace-name -w

Workspace name.

az ml datastore show-default

Show the workspace default datastore.

az ml datastore show-default [--include-secret]
                             [--output-metadata-file]
                             [--path]
                             [--resource-group]
                             [--subscription-id]
                             [--workspace-name]

Optional Parameters

--include-secret

Show the registered secret for the datastores.

--output-metadata-file -t

Provide an optional output file location for structured object output.

--path

Path to a root directory for run configuration files.

default value: .
--resource-group -g

Resource group name.

--subscription-id

Specifies the subscription Id.

--workspace-name -w

Workspace name.

az ml datastore upload

Upload files to a Datastore.

az ml datastore upload --name
                       --src-path
                       [--hide-progress]
                       [--output-metadata-file]
                       [--overwrite]
                       [--path]
                       [--resource-group]
                       [--subscription-id]
                       [--target-path]
                       [--workspace-name]

Required Parameters

--name -n

The datastore name.

--src-path -p

Path from which to upload data.

Optional Parameters

--hide-progress

Whether to hide progress of operation.

--output-metadata-file -t

Provide an optional output file location for structured object output.

--overwrite

Overwrite target files if they exist.

--path

Path to a root directory for run configuration files.

default value: .
--resource-group -g

Resource group name.

--subscription-id

Specifies the subscription Id.

--target-path -u

Path to upload data in the container. Uploads to the root by default.

--workspace-name -w

Workspace name.