Manage Azure Machine Learning workspaces in the portal or with the Python SDK

In this article, you create, view, and delete Azure Machine Learning workspaces for Azure Machine Learning, using the Azure portal or the SDK for Python

As your needs change or requirements for automation increase you can also manage workspaces using the CLI, or via the VS Code extension.

Prerequisites

Limitations

  • When creating a new workspace, you can either automatically create services needed by the workspace or use existing services. If you want to use existing services from a different Azure subscription than the workspace, you must register the Azure Machine Learning namespace in the subscription that contains those services. For example, creating a workspace in subscription A that uses a storage account from subscription B, the Azure Machine Learning namespace must be registered in subscription B before you can use the storage account with the workspace.

    The resource provider for Azure Machine Learning is Microsoft.MachineLearningServices. For information on how to see if it is registered and how to register it, see the Azure resource providers and types article.

    Important

    This only applies to resources provided during workspace creation; Azure Storage Accounts, Azure Container Register, Azure Key Vault, and Application Insights.

  • By default, creating a workspace also creates an Azure Container Registry (ACR). Since ACR does not currently support unicode characters in resource group names, use a resource group that does not contain these characters.

  • Azure Machine Learning does not support hierarchical namespace (Azure Data Lake Storage Gen2 feature) for the workspace's default storage account.

Tip

An Azure Application Insights instance is created when you create the workspace. You can delete the Application Insights instance after cluster creation if you want. Deleting it limits the information gathered from the workspace, and may make it more difficult to troubleshoot problems. If you delete the Application Insights instance created by the workspace, you cannot re-create it without deleting and recreating the workspace.

For more information on using this Application Insights instance, see Monitor and collect data from Machine Learning web service endpoints.

Create a workspace

APPLIES TO: Python SDK azureml v1

  • Default specification. By default, dependent resources and the resource group will be created automatically. This code creates a workspace named myworkspace and a resource group named myresourcegroup in eastus2.

    from azureml.core import Workspace
    
    ws = Workspace.create(name='myworkspace',
                   subscription_id='<azure-subscription-id>',
                   resource_group='myresourcegroup',
                   create_resource_group=True,
                   location='eastus2'
                   )
    

    Set create_resource_group to False if you have an existing Azure resource group that you want to use for the workspace.

  • Multiple tenants. If you have multiple accounts, add the tenant ID of the Azure Active Directory you wish to use. Find your tenant ID from the Azure portal under Azure Active Directory, External Identities.

    from azureml.core.authentication import InteractiveLoginAuthentication
    from azureml.core import Workspace
    
    interactive_auth = InteractiveLoginAuthentication(tenant_id="my-tenant-id")
    ws = Workspace.create(name='myworkspace',
                subscription_id='<azure-subscription-id>',
                resource_group='myresourcegroup',
                create_resource_group=True,
                location='eastus2',
                auth=interactive_auth
                )
    
  • Sovereign cloud. You'll need extra code to authenticate to Azure if you're working in a sovereign cloud.

    from azureml.core.authentication import InteractiveLoginAuthentication
    from azureml.core import Workspace
    
    interactive_auth = InteractiveLoginAuthentication(cloud="<cloud name>") # for example, cloud="AzureUSGovernment"
    ws = Workspace.create(name='myworkspace',
                subscription_id='<azure-subscription-id>',
                resource_group='myresourcegroup',
                create_resource_group=True,
                location='eastus2',
                auth=interactive_auth
                )
    
  • Use existing Azure resources. You can also create a workspace that uses existing Azure resources with the Azure resource ID format. Find the specific Azure resource IDs in the Azure portal or with the SDK. This example assumes that the resource group, storage account, key vault, App Insights, and container registry already exist.

    import os
    from azureml.core import Workspace
    from azureml.core.authentication import ServicePrincipalAuthentication
    
    service_principal_password = os.environ.get("AZUREML_PASSWORD")
    
    service_principal_auth = ServicePrincipalAuthentication(
        tenant_id="<tenant-id>",
        username="<application-id>",
        password=service_principal_password)
    
                          auth=service_principal_auth,
                               subscription_id='<azure-subscription-id>',
                               resource_group='myresourcegroup',
                               create_resource_group=False,
                               location='eastus2',
                               friendly_name='My workspace',
                               storage_account='subscriptions/<azure-subscription-id>/resourcegroups/myresourcegroup/providers/microsoft.storage/storageaccounts/mystorageaccount',
                               key_vault='subscriptions/<azure-subscription-id>/resourcegroups/myresourcegroup/providers/microsoft.keyvault/vaults/mykeyvault',
                               app_insights='subscriptions/<azure-subscription-id>/resourcegroups/myresourcegroup/providers/microsoft.insights/components/myappinsights',
                               container_registry='subscriptions/<azure-subscription-id>/resourcegroups/myresourcegroup/providers/microsoft.containerregistry/registries/mycontainerregistry',
                               exist_ok=False)
    

For more information, see Workspace SDK reference.

If you have problems in accessing your subscription, see Set up authentication for Azure Machine Learning resources and workflows, as well as the Authentication in Azure Machine Learning notebook.

Networking

Important

For more information on using a private endpoint and virtual network with your workspace, see Network isolation and privacy.

The Azure Machine Learning Python SDK provides the PrivateEndpointConfig class, which can be used with Workspace.create() to create a workspace with a private endpoint. This class requires an existing virtual network.

Advanced

By default, metadata for the workspace is stored in an Azure Cosmos DB instance that Microsoft maintains. This data is encrypted using Microsoft-managed keys.

To limit the data that Microsoft collects on your workspace, select High business impact workspace in the portal, or set hbi_workspace=true in Python. For more information on this setting, see Encryption at rest.

Important

Selecting high business impact can only be done when creating a workspace. You cannot change this setting after workspace creation.

Use your own key

You can provide your own key for data encryption. Doing so creates the Azure Cosmos DB instance that stores metadata in your Azure subscription. For more information, see [Customer-managed keys](concept-customer-managed keys.md).

Use the following steps to provide your own key:

Important

Before following these steps, you must first perform the following actions:

Follow the steps in Configure customer-managed keys to:

  • Register the Azure Cosmos DB provider
  • Create and configure an Azure Key Vault
  • Generate a key

Use cmk_keyvault and resource_cmk_uri to specify the customer managed key.

from azureml.core import Workspace
   ws = Workspace.create(name='myworkspace',
               subscription_id='<azure-subscription-id>',
               resource_group='myresourcegroup',
               create_resource_group=True,
               location='eastus2'
               cmk_keyvault='subscriptions/<azure-subscription-id>/resourcegroups/myresourcegroup/providers/microsoft.keyvault/vaults/<keyvault-name>', 
               resource_cmk_uri='<key-identifier>'
               )

Download a configuration file

If you will be creating a compute instance, skip this step. The compute instance has already created a copy of this file for you.

If you plan to use code on your local environment that references this workspace (ws), write the configuration file:

ws.write_config()

Place the file into the directory structure with your Python scripts or Jupyter Notebooks. It can be in the same directory, a subdirectory named .azureml, or in a parent directory. When you create a compute instance, this file is added to the correct directory on the VM for you.

Connect to a workspace

APPLIES TO: Python SDK azureml v1

In your Python code, you create a workspace object to connect to your workspace. This code will read the contents of the configuration file to find your workspace. You will get a prompt to sign in if you are not already authenticated.

from azureml.core import Workspace

ws = Workspace.from_config()
  • Multiple tenants. If you have multiple accounts, add the tenant ID of the Azure Active Directory you wish to use. Find your tenant ID from the Azure portal under Azure Active Directory, External Identities.

    from azureml.core.authentication import InteractiveLoginAuthentication
    from azureml.core import Workspace
    
    interactive_auth = InteractiveLoginAuthentication(tenant_id="my-tenant-id")
    ws = Workspace.from_config(auth=interactive_auth)
    
  • Sovereign cloud. You'll need extra code to authenticate to Azure if you're working in a sovereign cloud.

    APPLIES TO: Python SDK azureml v1

    from azureml.core.authentication import InteractiveLoginAuthentication
    from azureml.core import Workspace
    
    interactive_auth = InteractiveLoginAuthentication(cloud="<cloud name>") # for example, cloud="AzureUSGovernment"
    ws = Workspace.from_config(auth=interactive_auth)
    

If you have problems in accessing your subscription, see Set up authentication for Azure Machine Learning resources and workflows, as well as the Authentication in Azure Machine Learning notebook.

Find a workspace

See a list of all the workspaces you can use.

APPLIES TO: Python SDK azureml v1

Find your subscriptions in the Subscriptions page in the Azure portal. Copy the ID and use it in the code below to see all workspaces available for that subscription.

from azureml.core import Workspace

Workspace.list('<subscription-id>')

The Workspace.list(..) method does not return the full workspace object. It includes only basic information about existing workspaces in the subscription. To get a full object for specific workspace, use Workspace.get(..).

Search for assets across a workspace (preview)

With the public preview search capability, you can search for machine learning assets such as jobs, models, components, environments, and data across all workspaces, resource groups, and subscriptions in your organization through a unified global view.

  1. Start from Azure Machine Learning studio.
  2. If a workspace is open, select either the Microsoft menu item or the Microsoft link in the breadcrumb at the top of the page.

Screenshot shows two ways to get to the home page - menu item or breadcrumb link.

Type search text into the global search bar on the top of the studio Microsoft page and hit enter to trigger a 'contains' search. A contains search scans across all metadata fields for the given asset and sorts results relevance.

Screenshot shows the top search bar.

You can use the asset quick links to navigate to search results for jobs, models, components, environments, and data assets that you created.

Also, you can change the scope of applicable subscriptions and workspaces via the 'Change' link in the search bar drop down.

Search-bar list

Select any number of filters to create more specific search queries. The following filters are supported:

  • Job:
  • Model:
  • Component:
  • Tags:
  • SubmittedBy:
  • Environment:
  • Data:

If an asset filter (job, model, component, environment, data) is present, results are scoped to those tabs. Other filters apply to all assets unless an asset filter is also present in the query. Similarly, free text search can be provided alongside filters, but are scoped to the tabs chosen by asset filters, if present.

Tip

  • Filters search for exact matches of text. Use free text queries for a contains search.
  • Quotations are required around values that include spaces or other special characters.
  • If duplicate filters are provided, only the first will be recognized in search results.
  • Input text of any language is supported but filter strings must match the provided options (ex. submittedBy:).
  • The tags filter can accept multiple key:value pairs separated by a comma (ex. tags:"key1:value1, key2:value2").

View search results

You can view your search results in the individual Jobs, Models, Components, Environments, and Data tabs. Select an asset to open its Details page in the context of the relevant workspace. Results from workspaces you don't have permissions to view are not displayed.

Results displayed after search

If you've used this feature in a previous update, a search result error may occur. Reselect your preferred workspaces in the Directory + Subscription + Workspace tab.

Important

Search results may be unexpected for multiword terms in other languages (ex. Chinese characters).

Delete a workspace

When you no longer need a workspace, delete it.

Warning

Once an Azure Machine Learning workspace has been deleted, it cannot be recovered.

If you accidentally deleted your workspace, you may still be able to retrieve your notebooks. For details, see Failover for business continuity and disaster recovery.

APPLIES TO: Python SDK azureml v1

Delete the workspace ws:

ws.delete(delete_dependent_resources=False, no_wait=False)

The default action is not to delete resources associated with the workspace, that is, container registry, storage account, key vault, and application insights. Set delete_dependent_resources to True to delete these resources as well.

Clean up resources

Important

The resources that you created can be used as prerequisites to other Azure Machine Learning tutorials and how-to articles.

If you don't plan to use any of the resources that you created, delete them so you don't incur any charges:

  1. In the Azure portal, select Resource groups on the far left.

  2. From the list, select the resource group that you created.

  3. Select Delete resource group.

    Screenshot of the selections to delete a resource group in the Azure portal.

  4. Enter the resource group name. Then select Delete.

Troubleshooting

  • Supported browsers in Azure Machine Learning studio: We recommend that you use the most up-to-date browser that's compatible with your operating system. The following browsers are supported:

    • Microsoft Edge (The new Microsoft Edge, latest version. Not Microsoft Edge legacy)
    • Safari (latest version, Mac only)
    • Chrome (latest version)
    • Firefox (latest version)
  • Azure portal:

    • If you go directly to your workspace from a share link from the SDK or the Azure portal, you can't view the standard Overview page that has subscription information in the extension. In this scenario, you also can't switch to another workspace. To view another workspace, go directly to Azure Machine Learning studio and search for the workspace name.
    • All assets (Data, Experiments, Computes, and so on) are available only in Azure Machine Learning studio. They're not available from the Azure portal.
    • Attempting to export a template for a workspace from the Azure portal may return an error similar to the following text: Could not get resource of the type <type>. Resources of this type will not be exported. As a workaround, use one of the templates provided at https://github.com/Azure/azure-quickstart-templates/tree/master/quickstarts/microsoft.machinelearningservices as the basis for your template.

Workspace diagnostics

You can run diagnostics on your workspace from Azure Machine Learning studio or the Python SDK. After diagnostics run, a list of any detected problems is returned. This list includes links to possible solutions. For more information, see How to use workspace diagnostics.

Resource provider errors

When creating an Azure Machine Learning workspace, or a resource used by the workspace, you may receive an error similar to the following messages:

  • No registered resource provider found for location {location}
  • The subscription is not registered to use namespace {resource-provider-namespace}

Most resource providers are automatically registered, but not all. If you receive this message, you need to register the provider mentioned.

The following table contains a list of the resource providers required by Azure Machine Learning:

Resource provider Why it's needed
Microsoft.MachineLearningServices Creating the Azure Machine Learning workspace.
Microsoft.Storage Azure Storage Account is used as the default storage for the workspace.
Microsoft.ContainerRegistry Azure Container Registry is used by the workspace to build Docker images.
Microsoft.KeyVault Azure Key Vault is used by the workspace to store secrets.
Microsoft.Notebooks/NotebookProxies Integrated notebooks on Azure Machine Learning compute instance.
Microsoft.ContainerService If you plan on deploying trained models to Azure Kubernetes Services.

If you plan on using a customer-managed key with Azure Machine Learning, then the following service providers must be registered:

Resource provider Why it's needed
Microsoft.DocumentDB/databaseAccounts Azure CosmosDB instance that logs metadata for the workspace.
Microsoft.Search/searchServices Azure Search provides indexing capabilities for the workspace.

For information on registering resource providers, see Resolve errors for resource provider registration.

Deleting the Azure Container Registry

The Azure Machine Learning workspace uses Azure Container Registry (ACR) for some operations. It will automatically create an ACR instance when it first needs one.

Warning

Once an Azure Container Registry has been created for a workspace, do not delete it. Doing so will break your Azure Machine Learning workspace.

Examples

Examples of creating a workspace:

Next steps

Once you have a workspace, learn how to Train and deploy a model.

To learn more about planning a workspace for your organization's requirements, see Organize and set up Azure Machine Learning.

To check for problems with your workspace, see How to use workspace diagnostics.

If you need to move a workspace to another Azure subscription, see How to move a workspace.

For information on how to keep your Azure ML up to date with the latest security updates, see Vulnerability management.