Create a workspace for Azure Machine Learning with Azure CLI

In this article, you learn how to create an Azure Machine Learning workspace using the Azure CLI. The Azure CLI provides commands for managing Azure resources. The machine learning extension to the CLI provides commands for working with Azure Machine Learning resources.

Prerequisites

Connect the CLI to your Azure subscription

Important

If you are using the Azure Cloud Shell, you can skip this section. The cloud shell automatically authenticates you using the account you log into your Azure subscription.

There are several ways that you can authenticate to your Azure subscription from the CLI. The most simple is to interactively authenticate using a browser. To authenticate interactively, open a command line or terminal and use the following command:

az login

If the CLI can open your default browser, it will do so and load a sign-in page. Otherwise, you need to open a browser and follow the instructions on the command line. The instructions involve browsing to https://aka.ms/devicelogin and entering an authorization code.

Tip

After logging in, you see a list of subscriptions associated with your Azure account. The subscription information with isDefault: true is the currently activated subscription for Azure CLI commands. This subscription must be the same one that contains your Azure Machine Learning workspace. You can find the subscription ID from the Azure portal by visiting the overview page for your workspace. You can also use the SDK to get the subscription ID from the workspace object. For example, Workspace.from_config().subscription_id.

To select another subscription, use the az account set -s <subscription name or ID> command and specify the subscription name or ID to switch to. For more information about subscription selection, see Use multiple Azure Subscriptions.

For other methods of authenticating, see Sign in with Azure CLI.

Install the machine learning extension

To install the machine learning extension, use the following command:

az extension add -n azure-cli-ml

Create a workspace

The Azure Machine Learning workspace relies on the following Azure services or entities:

Important

If you do not specify an existing Azure service, one will be created automatically during workspace creation. You must always specify a resource group. When attaching your own storage account, make sure that it meets the following criteria:

  • The storage account is not a premium account (Premium_LRS and Premium_GRS)
  • Both Azure Blob and Azure File capabilities enabled
  • Hierarchical Namespace (ADLS Gen 2) is disabled

These requirements are only for the default storage account used by the workspace.

Service Parameter to specify an existing instance
Azure resource group -g <resource-group-name>
Azure Storage Account --storage-account <service-id>
Azure Application Insights --application-insights <service-id>
Azure Key Vault --keyvault <service-id>
Azure Container Registry --container-registry <service-id>

Create a resource group

The Azure Machine Learning workspace must be created inside a resource group. You can use an existing resource group or create a new one. To create a new resource group, use the following command. Replace <resource-group-name> with the name to use for this resource group. Replace <location> with the Azure region to use for this resource group:

Tip

You should select a region where Azure Machine Learning is available. For information, see Products available by region.

az group create --name <resource-group-name> --location <location>

The response from this command is similar to the following JSON:

{
  "id": "/subscriptions/<subscription-GUID>/resourceGroups/<resourcegroupname>",
  "location": "<location>",
  "managedBy": null,
  "name": "<resource-group-name>",
  "properties": {
    "provisioningState": "Succeeded"
  },
  "tags": null,
  "type": null
}

For more information on working with resource groups, see az group.

Automatically create required resources

To create a new workspace where the services are automatically created, use the following command:

az ml workspace create -w <workspace-name> -g <resource-group-name>

Note

The workspace name is case-insensitive.

The output of this command is similar to the following JSON:

{
  "applicationInsights": "/subscriptions/<service-GUID>/resourcegroups/<resource-group-name>/providers/microsoft.insights/components/<application-insight-name>",
  "containerRegistry": "/subscriptions/<service-GUID>/resourcegroups/<resource-group-name>/providers/microsoft.containerregistry/registries/<acr-name>",
  "creationTime": "2019-08-30T20:24:19.6984254+00:00",
  "description": "",
  "friendlyName": "<workspace-name>",
  "id": "/subscriptions/<service-GUID>/resourceGroups/<resource-group-name>/providers/Microsoft.MachineLearningServices/workspaces/<workspace-name>",
  "identityPrincipalId": "<GUID>",
  "identityTenantId": "<GUID>",
  "identityType": "SystemAssigned",
  "keyVault": "/subscriptions/<service-GUID>/resourcegroups/<resource-group-name>/providers/microsoft.keyvault/vaults/<key-vault-name>",
  "location": "<location>",
  "name": "<workspace-name>",
  "resourceGroup": "<resource-group-name>",
  "storageAccount": "/subscriptions/<service-GUID>/resourcegroups/<resource-group-name>/providers/microsoft.storage/storageaccounts/<storage-account-name>",
  "type": "Microsoft.MachineLearningServices/workspaces",
  "workspaceid": "<GUID>"
}

Virtual network and private endpoint

Important

Using an Azure Machine Learning workspace with private link is not available in the Azure Government regions or Azure China 21Vianet regions.

If you want to restrict access to your workspace to a virtual network, you can use the following parameters:

  • --pe-name: The name of the private endpoint that is created.
  • --pe-auto-approval: Whether private endpoint connections to the workspace should be automatically approved.
  • --pe-resource-group: The resource group to create the private endpoint in. Must be the same group that contains the virtual network.
  • --pe-vnet-name: The existing virtual network to create the private endpoint in.
  • --pe-subnet-name: The name of the subnet to create the private endpoint in. The default value is default.

For more information on using a private endpoint and virtual network with your workspace, see Virtual network isolation and privacy overview.

Customer-managed key and high business impact workspace

By default, metrics and metadata for the workspace is stored in an Azure Cosmos DB instance that Microsoft maintains. This data is encrypted using Microsoft-managed keys.

Instead of using the Microsoft-managed key, you can use the provide your own key. Doing so creates the Azure Cosmos DB instance that stores metrics and metadata in your Azure subscription. Use the --cmk-keyvault parameter to specify the Azure Key Vault that contains the key, and --resource-cmk-uri to specify the URL of the key within the vault.

Important

Before using the --cmk-keyvault and --resource-cmk-uri parameters, you must first perform the following actions:

  1. Authorize the Machine Learning App (in Identity and Access Management) with contributor permissions on your subscription.

  2. Follow the steps in Configure customer-managed keys to:

    • Register the Azure Cosmos DB provider
    • Create and configure an Azure Key Vault
    • Generate a key

    You do not need to manually create the Azure Cosmos DB instance, one will be created for you during workspace creation. This Azure Cosmos DB instance will be created in a separate resource group using a name based on this pattern: <your-resource-group-name>_<GUID>.

You cannot change this setting after workspace creation. If you delete the Azure Cosmos DB used by your workspace, you must also delete the workspace that is using it.

To limit the data that Microsoft collects on your workspace, use the --hbi-workspace parameter.

Important

Selecting high business impact can only be done when creating a workspace. You cannot change this setting after workspace creation.

For more information on customer-managed keys and high business impact workspace, see Enterprise security for Azure Machine Learning.

Use existing resources

To create a workspace that uses existing resources, you must provide the ID for the resources. Use the following commands to get the ID for the services:

Important

You don't have to specify all existing resources. You can specify one or more. For example, you can specify an existing storage account and the workspace will create the other resources.

  • Azure Storage Account: az storage account show --name <storage-account-name> --query "id"

    The response from this command is similar to the following text, and is the ID for your storage account:

    "/subscriptions/<service-GUID>/resourceGroups/<resource-group-name>/providers/Microsoft.Storage/storageAccounts/<storage-account-name>"

    Important

    If you want to use an existing Azure Storage account, it cannot be a premium account (Premium_LRS and Premium_GRS). It also cannot have a hierarchical namespace (used with Azure Data Lake Storage Gen2). Neither premium storage or hierarchical namespace are supported with the default storage account of the workspace. You can use premium storage or hierarchical namespace with non-default storage accounts.

  • Azure Application Insights:

    1. Install the application insights extension:

      az extension add -n application-insights
      
    2. Get the ID of your application insight service:

      az monitor app-insights component show --app <application-insight-name> -g <resource-group-name> --query "id"
      

      The response from this command is similar to the following text, and is the ID for your application insights service:

      "/subscriptions/<service-GUID>/resourceGroups/<resource-group-name>/providers/microsoft.insights/components/<application-insight-name>"

  • Azure Key Vault: az keyvault show --name <key-vault-name> --query "ID"

    The response from this command is similar to the following text, and is the ID for your key vault:

    "/subscriptions/<service-GUID>/resourceGroups/<resource-group-name>/providers/Microsoft.KeyVault/vaults/<key-vault-name>"

  • Azure Container Registry: az acr show --name <acr-name> -g <resource-group-name> --query "id"

    The response from this command is similar to the following text, and is the ID for the container registry:

    "/subscriptions/<service-GUID>/resourceGroups/<resource-group-name>/providers/Microsoft.ContainerRegistry/registries/<acr-name>"

    Important

    The container registry must have the the admin account enabled before it can be used with an Azure Machine Learning workspace.

Once you have the IDs for the resource(s) that you want to use with the workspace, use the base az workspace create -w <workspace-name> -g <resource-group-name> command and add the parameter(s) and ID(s) for the existing resources. For example, the following command creates a workspace that uses an existing container registry:

az ml workspace create -w <workspace-name> -g <resource-group-name> --container-registry "/subscriptions/<service-GUID>/resourceGroups/<resource-group-name>/providers/Microsoft.ContainerRegistry/registries/<acr-name>"

The output of this command is similar to the following JSON:

{
  "applicationInsights": "/subscriptions/<service-GUID>/resourcegroups/<resource-group-name>/providers/microsoft.insights/components/<application-insight-name>",
  "containerRegistry": "/subscriptions/<service-GUID>/resourcegroups/<resource-group-name>/providers/microsoft.containerregistry/registries/<acr-name>",
  "creationTime": "2019-08-30T20:24:19.6984254+00:00",
  "description": "",
  "friendlyName": "<workspace-name>",
  "id": "/subscriptions/<service-GUID>/resourceGroups/<resource-group-name>/providers/Microsoft.MachineLearningServices/workspaces/<workspace-name>",
  "identityPrincipalId": "<GUID>",
  "identityTenantId": "<GUID>",
  "identityType": "SystemAssigned",
  "keyVault": "/subscriptions/<service-GUID>/resourcegroups/<resource-group-name>/providers/microsoft.keyvault/vaults/<key-vault-name>",
  "location": "<location>",
  "name": "<workspace-name>",
  "resourceGroup": "<resource-group-name>",
  "storageAccount": "/subscriptions/<service-GUID>/resourcegroups/<resource-group-name>/providers/microsoft.storage/storageaccounts/<storage-account-name>",
  "type": "Microsoft.MachineLearningServices/workspaces",
  "workspaceid": "<GUID>"
}

List workspaces

To list all the workspaces for your Azure subscription, use the following command:

az ml workspace list

The output of this command is similar to the following JSON:

[
  {
    "resourceGroup": "myresourcegroup",
    "subscriptionId": "<subscription-id>",
    "workspaceName": "myml"
  },
  {
    "resourceGroup": "anotherresourcegroup",
    "subscriptionId": "<subscription-id>",
    "workspaceName": "anotherml"
  }
]

For more information, see the az ml workspace list documentation.

Get workspace information

To get information about a workspace, use the following command:

az ml workspace show -w <workspace-name> -g <resource-group-name>

The output of this command is similar to the following JSON:

{
  "applicationInsights": "/subscriptions/<service-GUID>/resourcegroups/<resource-group-name>/providers/microsoft.insights/components/application-insight-name>",
  "creationTime": "2019-08-30T18:55:03.1807976+00:00",
  "description": "",
  "friendlyName": "",
  "id": "/subscriptions/<service-GUID>/resourceGroups/<resource-group-name>/providers/Microsoft.MachineLearningServices/workspaces/<workspace-name>",
  "identityPrincipalId": "<GUID>",
  "identityTenantId": "<GUID>",
  "identityType": "SystemAssigned",
  "keyVault": "/subscriptions/<service-GUID>/resourcegroups/<resource-group-name>/providers/microsoft.keyvault/vaults/<key-vault-name>",
  "location": "<location>",
  "name": "<workspace-name>",
  "resourceGroup": "<resource-group-name>",
  "storageAccount": "/subscriptions/<service-GUID>/resourcegroups/<resource-group-name>/providers/microsoft.storage/storageaccounts/<storage-account-name>",
  "tags": {},
  "type": "Microsoft.MachineLearningServices/workspaces",
  "workspaceid": "<GUID>"
}

For more information, see the az ml workspace show documentation.

Update a workspace

To update a workspace, use the following command:

az ml workspace update -w <workspace-name> -g <resource-group-name>

The output of this command is similar to the following JSON:

{
  "applicationInsights": "/subscriptions/<service-GUID>/resourcegroups/<resource-group-name>/providers/microsoft.insights/components/application-insight-name>",
  "creationTime": "2019-08-30T18:55:03.1807976+00:00",
  "description": "",
  "friendlyName": "",
  "id": "/subscriptions/<service-GUID>/resourceGroups/<resource-group-name>/providers/Microsoft.MachineLearningServices/workspaces/<workspace-name>",
  "identityPrincipalId": "<GUID>",
  "identityTenantId": "<GUID>",
  "identityType": "SystemAssigned",
  "keyVault": "/subscriptions/<service-GUID>/resourcegroups/<resource-group-name>/providers/microsoft.keyvault/vaults/<key-vault-name>",
  "location": "<location>",
  "name": "<workspace-name>",
  "resourceGroup": "<resource-group-name>",
  "storageAccount": "/subscriptions/<service-GUID>/resourcegroups/<resource-group-name>/providers/microsoft.storage/storageaccounts/<storage-account-name>",
  "tags": {},
  "type": "Microsoft.MachineLearningServices/workspaces",
  "workspaceid": "<GUID>"
}

For more information, see the az ml workspace update documentation.

Share a workspace with another user

To share a workspace with another user on your subscription, use the following command:

az ml workspace share -w <workspace-name> -g <resource-group-name> --user <user> --role <role>

For more information on Azure role-based access control (Azure RBAC) with Azure Machine Learning, see Manage users and roles.

For more information, see the az ml workspace share documentation.

Sync keys for dependent resources

If you change access keys for one of the resources used by your workspace, it takes around an hour for the workspace to synchronize to the new key. To force the workspace to sync the new keys immediately, use the following command:

az ml workspace sync-keys -w <workspace-name> -g <resource-group-name>

For more information on changing keys, see Regenerate storage access keys.

For more information, see the az ml workspace sync-keys documentation.

Delete a workspace

To delete a workspace after it is no longer needed, use the following command:

az ml workspace delete -w <workspace-name> -g <resource-group-name>

Important

Deleting a workspace does not delete the application insight, storage account, key vault, or container registry used by the workspace.

You can also delete the resource group, which deletes the workspace and all other Azure resources in the resource group. To delete the resource group, use the following command:

az group delete -g <resource-group-name>

For more information, see the az ml workspace delete documentation.

Troubleshooting

Resource provider errors

When creating an Azure Machine Learning workspace, or a resource used by the workspace, you may receive an error similar to the following messages:

  • No registered resource provider found for location {location}
  • The subscription is not registered to use namespace {resource-provider-namespace}

Most resource providers are automatically registered, but not all. If you receive this message, you need to register the provider mentioned.

For information on registering resource providers, see Resolve errors for resource provider registration.

Moving the workspace

Warning

Moving your Azure Machine Learning workspace to a different subscription, or moving the owning subscription to a new tenant, is not supported. Doing so may cause errors.

Deleting the Azure Container Registry

The Azure Machine Learning workspace uses Azure Container Registry (ACR) for some operations. It will automatically create an ACR instance when it first needs one.

Warning

Once an Azure Container Registry has been created for a workspace, do not delete it. Doing so will break your Azure Machine Learning workspace.

Next steps

For more information on the Azure CLI extension for machine learning, see the az ml documentation.