Create and manage Azure Machine Learning workspaces

In this article, you'll create, view, and delete Azure Machine Learning workspaces for Azure Machine Learning, using the Azure portal or the SDK for Python

As your needs change or requirements for automation increase you can also create and delete workspaces using the CLI, or via the VS Code extension.

Prerequisites

Limitations

  • When creating a new workspace, you can either automatically create services needed by the workspace or use existing services. If you want to use existing services from a different Azure subscription than the workspace, you must register the Azure Machine Learning namespace in the subscription that contains those services. For example, creating a workspace in subscription A that uses a storage account from subscription B, the Azure Machine Learning namespace must be registered in subscription B before you can use the storage account with the workspace.

    The resource provider for Azure Machine Learning is Microsoft.MachineLearningService. For information on how to see if it is registered and how to register it, see the Azure resource providers and types article.

    Important

    This only applies to resources provided during workspace creation; Azure Storage Accounts, Azure Container Register, Azure Key Vault, and Application Insights.

Create a workspace

  • Default specification. By default, dependent resources as well as the resource group will be created automatically. This code creates a workspace named myworkspace and a resource group named myresourcegroup in eastus2.

    from azureml.core import Workspace
    
    ws = Workspace.create(name='myworkspace',
                   subscription_id='<azure-subscription-id>',
                   resource_group='myresourcegroup',
                   create_resource_group=True,
                   location='eastus2'
                   )
    

    Set create_resource_group to False if you have an existing Azure resource group that you want to use for the workspace.

  • Multiple tenants. If you have multiple accounts, add the tenant ID of the Azure Active Directory you wish to use. Find your tenant ID from the Azure portal under Azure Active Directory, External Identities.

    from azureml.core.authentication import InteractiveLoginAuthentication
    from azureml.core import Workspace
    
    interactive_auth = InteractiveLoginAuthentication(tenant_id="my-tenant-id")
    ws = Workspace.create(name='myworkspace',
                subscription_id='<azure-subscription-id>',
                resource_group='myresourcegroup',
                create_resource_group=True,
                location='eastus2',
                auth=interactive_auth
                )
    
  • Sovereign cloud. You'll need extra code to authenticate to Azure if you're working in a sovereign cloud.

    from azureml.core.authentication import InteractiveLoginAuthentication
    from azureml.core import Workspace
    
    interactive_auth = InteractiveLoginAuthentication(cloud="<cloud name>") # for example, cloud="AzureUSGovernment"
    ws = Workspace.create(name='myworkspace',
                subscription_id='<azure-subscription-id>',
                resource_group='myresourcegroup',
                create_resource_group=True,
                location='eastus2',
                auth=interactive_auth
                )
    
  • Use existing Azure resources. You can also create a workspace that uses existing Azure resources with the Azure resource ID format. Find the specific Azure resource IDs in the Azure portal or with the SDK. This example assumes that the resource group, storage account, key vault, App Insights and container registry already exist.

    import os
    from azureml.core import Workspace
    from azureml.core.authentication import ServicePrincipalAuthentication
    
    service_principal_password = os.environ.get("AZUREML_PASSWORD")
    
    service_principal_auth = ServicePrincipalAuthentication(
        tenant_id="<tenant-id>",
        username="<application-id>",
        password=service_principal_password)
    
                          auth=service_principal_auth,
                               subscription_id='<azure-subscription-id>',
                               resource_group='myresourcegroup',
                               create_resource_group=False,
                               location='eastus2',
                               friendly_name='My workspace',
                               storage_account='subscriptions/<azure-subscription-id>/resourcegroups/myresourcegroup/providers/microsoft.storage/storageaccounts/mystorageaccount',
                               key_vault='subscriptions/<azure-subscription-id>/resourcegroups/myresourcegroup/providers/microsoft.keyvault/vaults/mykeyvault',
                               app_insights='subscriptions/<azure-subscription-id>/resourcegroups/myresourcegroup/providers/microsoft.insights/components/myappinsights',
                               container_registry='subscriptions/<azure-subscription-id>/resourcegroups/myresourcegroup/providers/microsoft.containerregistry/registries/mycontainerregistry',
                               exist_ok=False)
    

For more information, see Workspace SDK reference.

If you have problems in accessing your subscription, see Set up authentication for Azure Machine Learning resources and workflows, as well as the Authentication in Azure Machine Learning notebook.

Networking

Important

For more information on using a private endpoint and virtual network with your workspace, see Network isolation and privacy.

The Azure Machine Learning Python SDK provides the PrivateEndpointConfig class, which can be used with Workspace.create() to create a workspace with a private endpoint. This class requires an existing virtual network.

Important

Using a private endpoint with Azure Machine Learning workspace is currently in public preview. This preview is provided without a service level agreement, and it's not recommended for production workloads. Certain features might not be supported or might have constrained capabilities. For more information, see Supplemental Terms of Use for Microsoft Azure Previews.

Multiple workspaces with private endpoint

When you create a private endpoint, a new Private DNS Zone named privatelink.api.azureml.ms is created. This contains a link to the virtual network. If you create multiple workspaces with private endpoints in the same resource group, only the virtual network for the first private endpoint may be added to the DNS zone. To add entries for the virtual networks used by the additional workspaces/private endpoints, use the following steps:

  1. In the Azure portal, select the resource group that contains the workspace. Then select the Private DNS Zone resource named privatelink.api.azureml.ms
  2. In the Settings, select Virtual network links.
  3. Select Add. From the Add virtual network link page, provide a unique Link name, and then select the Virtual network to be added. Select OK to add the network link.

For more information, see Azure Private Endpoint DNS configuration.

Vulnerability scanning

Azure Security Center provides unified security management and advanced threat protection across hybrid cloud workloads. You should allow Azure Security Center to scan your resources and follow its recommendations. For more, see Azure Container Registry image scanning by Security Center and Azure Kubernetes Services integration with Security Center.

Advanced

By default, metadata for the workspace is stored in an Azure Cosmos DB instance that Microsoft maintains. This data is encrypted using Microsoft-managed keys.

To limit the data that Microsoft collects on your workspace, select High business impact workspace in the portal, or set hbi_workspace=true in Python. For more information on this setting, see Encryption at rest.

Important

Selecting high business impact can only be done when creating a workspace. You cannot change this setting after workspace creation.

Use your own key

You can provide your own key for data encryption. Doing so creates the Azure Cosmos DB instance that stores metadata in your Azure subscription.

Important

The Cosmos DB instance is created in a Microsoft-managed resource group in your subscription, along with any resources it needs. This means that you are charged for this Cosmos DB instance. The managed resource group is named in the format <AML Workspace Resource Group Name><GUID>. If your Azure Machine Learning workspace uses a private endpoint, a virtual network is also created for the Cosmos DB instance. This VNet is used to secure communication between Cosmos DB and Azure Machine Learning.

  • Do not delete the resource group that contains this Cosmos DB instance, or any of the resources automatically created in this group. If you need to delete the resource group, Cosmos DB instance, etc., you must delete the Azure Machine Learning workspace that uses it. The resource group, Cosmos DB instance, and other automatically created resources are deleted when the associated workspace is deleted.
  • The default Request Units for this Cosmos DB account is set at 8000.
  • You cannot provide your own VNet for use with the Cosmos DB instance that is created. You also cannot modify the virtual network. For example, you cannot change the IP address range that it uses.

Use the following steps to provide your own key:

Important

Before following these steps, you must first perform the following actions:

  1. Authorize the Machine Learning App (in Identity and Access Management) with contributor permissions on your subscription.

  2. Follow the steps in Configure customer-managed keys to:

    • Register the Azure Cosmos DB provider
    • Create and configure an Azure Key Vault
    • Generate a key

    You do not need to manually create the Azure Cosmos DB instance, one will be created for you during workspace creation. This Azure Cosmos DB instance will be created in a separate resource group using a name based on this pattern: <your-workspace-resource-name>_<GUID>.

You cannot change this setting after workspace creation. If you delete the Azure Cosmos DB used by your workspace, you must also delete the workspace that is using it.

Use cmk_keyvault and resource_cmk_uri to specify the customer managed key.

from azureml.core import Workspace
   ws = Workspace.create(name='myworkspace',
               subscription_id='<azure-subscription-id>',
               resource_group='myresourcegroup',
               create_resource_group=True,
               location='eastus2'
               cmk_keyvault='subscriptions/<azure-subscription-id>/resourcegroups/myresourcegroup/providers/microsoft.keyvault/vaults/<keyvault-name>', 
               resource_cmk_uri='<key-identifier>'
               )

Download a configuration file

If you will be creating a compute instance, skip this step. The compute instance has already created a copy of this file for you.

If you plan to use code on your local environment that references this workspace (ws), write the configuration file:

ws.write_config()

Place the file into the directory structure with your Python scripts or Jupyter Notebooks. It can be in the same directory, a subdirectory named .azureml, or in a parent directory. When you create a compute instance, this file is added to the correct directory on the VM for you.

Connect to a workspace

In your Python code, you create a workspace object to connect to your workspace. This code will read the contents of the configuration file to find your workspace. You will get a prompt to sign in if you are not already authenticated.

from azureml.core import Workspace

ws = Workspace.from_config()
  • Multiple tenants. If you have multiple accounts, add the tenant ID of the Azure Active Directory you wish to use. Find your tenant ID from the Azure portal under Azure Active Directory, External Identities.

    from azureml.core.authentication import InteractiveLoginAuthentication
    from azureml.core import Workspace
    
    interactive_auth = InteractiveLoginAuthentication(tenant_id="my-tenant-id")
    ws = Workspace.from_config(auth=interactive_auth)
    
  • Sovereign cloud. You'll need extra code to authenticate to Azure if you're working in a sovereign cloud.

    from azureml.core.authentication import InteractiveLoginAuthentication
    from azureml.core import Workspace
    
    interactive_auth = InteractiveLoginAuthentication(cloud="<cloud name>") # for example, cloud="AzureUSGovernment"
    ws = Workspace.from_config(auth=interactive_auth)
    

If you have problems in accessing your subscription, see Set up authentication for Azure Machine Learning resources and workflows, as well as the Authentication in Azure Machine Learning notebook.

Find a workspace

See a list of all the workspaces you can use.

Find your subscriptions in the Subscriptions page in the Azure portal. Copy the ID and use it in the code below to see all workspaces available for that subscription.

from azureml.core import Workspace

Workspace.list('<subscription-id>')

Delete a workspace

When you no longer need a workspace, delete it.

Delete the workspace ws:

ws.delete(delete_dependent_resources=False, no_wait=False)

The default action is not to delete resources associated with the workspace, i.e., container registry, storage account, key vault, and application insights. Set delete_dependent_resources to True to delete these resources as well.

Clean up resources

Important

The resources that you created can be used as prerequisites to other Azure Machine Learning tutorials and how-to articles.

If you don't plan to use the resources that you created, delete them so you don't incur any charges:

  1. In the Azure portal, select Resource groups on the far left.

  2. From the list, select the resource group that you created.

  3. Select Delete resource group.

    Screenshot of the selections to delete a resource group in the Azure portal.

  4. Enter the resource group name. Then select Delete.

Troubleshooting

Resource provider errors

When creating an Azure Machine Learning workspace, or a resource used by the workspace, you may receive an error similar to the following messages:

  • No registered resource provider found for location {location}
  • The subscription is not registered to use namespace {resource-provider-namespace}

Most resource providers are automatically registered, but not all. If you receive this message, you need to register the provider mentioned.

For information on registering resource providers, see Resolve errors for resource provider registration.

Moving the workspace

Warning

Moving your Azure Machine Learning workspace to a different subscription, or moving the owning subscription to a new tenant, is not supported. Doing so may cause errors.

Deleting the Azure Container Registry

The Azure Machine Learning workspace uses Azure Container Registry (ACR) for some operations. It will automatically create an ACR instance when it first needs one.

Warning

Once an Azure Container Registry has been created for a workspace, do not delete it. Doing so will break your Azure Machine Learning workspace.

Examples

Examples of creating a workspace:

Next steps

Once you have a workspace, learn how to Train and deploy a model.