What is Azure Machine Learning designer (preview)?

APPLIES TO: noBasic edition yesEnterprise edition                       (Upgrade to Enterprise)

Azure Machine Learning designer lets you visually connect datasets and modules on an interactive canvas to create machine learning models. To learn how to get started with the designer, see Tutorial: Predict automobile price with the designer

Azure Machine Learning designer example

The designer uses your Azure Machine Learning workspace to organize shared resources such as:

Model training and deployment

The designer gives you a visual canvas to build, test, and deploy machine learning models. With the designer you can:

  • Drag-and-drop datasets and modules onto the canvas.
  • Connect the modules together to create a pipeline draft.
  • Submit a pipeline run using the compute resources in your Azure Machine Learning workspace.
  • Convert your training pipelines to inference pipelines.
  • Publish your pipelines to a REST pipeline endpoint to submit new pipeline runs with different parameters and datasets.
    • Publish a training pipeline to reuse a single pipeline to train multiple models while changing parameters and datasets.
    • Publish a batch inference pipeline to make predictions on new data by using a previously trained model.
  • Deploy a real-time inference pipeline to a real-time endpoint to make predictions on new data in real time.

Workflow diagram for training, batch inference, and real-time inference in the designer

Pipeline

A pipeline consists of datasets and analytical modules, which you connect together. Pipelines have many uses: you can make a pipeline that trains a single model, or one that trains multiple models. You can create a pipeline that makes predictions in real time or in batch, or make a pipeline that only cleans data. Pipelines let you reuse your work and organize your projects.

Pipeline draft

As you edit a pipeline in the designer, your progress is saved as a pipeline draft. You can edit a pipeline draft at any point by adding or removing modules, configuring compute targets, creating parameters, and so on.

A valid pipeline has these characteristics:

  • Datasets can only connect to modules.
  • Modules can only connect to either datasets or other modules.
  • All input ports for modules must have some connection to the data flow.
  • All required parameters for each module must be set.

When you're ready to run your pipeline draft, you submit a pipeline run.

Pipeline run

Each time you run a pipeline, the configuration of the pipeline and its results are stored in your workspace as a pipeline run. You can go back to any pipeline run to inspect it for troubleshooting or auditing purposes. Clone a pipeline run to create a new pipeline draft for you to edit.

Pipeline runs are grouped into experiments to organize run history. You can set the experiment for every pipeline run.

Datasets

A machine learning dataset makes it easy to access and work with your data. A number of sample datasets are included in the designer for you to experiment with. You can register more datasets as you need them.

Module

A module is an algorithm that you can perform on your data. The designer has a number of modules ranging from data ingress functions to training, scoring, and validation processes.

A module may have a set of parameters that you can use to configure the module's internal algorithms. When you select a module on the canvas, the module's parameters are displayed in the Properties pane to the right of the canvas. You can modify the parameters in that pane to tune your model. You can set the compute resources for individual modules in the designer.

Module properties

For some help navigating through the library of machine learning algorithms available, see Algorithm & module reference overview

Compute resources

Use compute resources from your workspace to run your pipeline and host your deployed models as real-time endpoints or pipeline endpoints (for batch inference). The supported compute targets are:

Compute target Training Deployment
Azure Machine Learning compute
Azure Kubernetes Service

Compute targets are attached to your Machine Learning workspace. You manage your compute targets in your workspace in Azure Machine Learning studio.

Deploy

To perform real-time inferencing, you must deploy a pipeline as a real-time endpoint. The real-time endpoint creates an interface between an external application and your scoring model. A call to a real-time endpoint returns prediction results to the application in real time. To make a call to a real-time endpoint, you pass the API key that was created when you deployed the endpoint. The endpoint is based on REST, a popular architecture choice for web programming projects.

Real-time endpoints must be deployed to an Azure Kubernetes Service cluster.

To learn how to deploy your model, see Tutorial: Deploy a machine learning model with the designer.

Publish

You can also publish a pipeline to a pipeline endpoint. Similar to a real-time endpoint, a pipeline endpoint lets you submit new pipeline runs from external applications using REST calls. However, you cannot send or receive data in real-time using a pipeline endpoint.

Published pipelines are flexible, they can be used to train or retrain models, perform batch inferencing, process new data, and much more. You can publish multiple pipelines to a single pipeline endpoint and specify which pipeline version to run.

A published pipeline runs on the compute resources you define in the pipeline draft for each module.

The designer creates the same PublishedPipeline object as the SDK.

Moving from the visual interface to the designer

The visual interface (preview) has been updated and is now Azure Machine Learning designer (preview). The designer has been rearchitected to use a pipeline-based backend that fully integrates with the other features of Azure Machine Learning.

As a result of these updates, some concepts and terms for the visual interface have been changed or renamed. See the table below for the most important conceptual changes.

Concept in the designer Previously in the visual interface
Pipeline draft Experiment
Real-time endpoint Web service

Migrating to the designer

You can convert existing visual interface experiments and web services to pipelines and real-time endpoints in the designer. Use the following steps to migrate your visual interface assets:

  1. Sign in to Azure Machine Learning studio.

  2. Upgrade your workspace to Enterprise edition.

    After upgrading, all of your visual interface experiments will convert to pipeline drafts in the designer.

    Note

    You don't need to upgrade to the Enterprise edition to convert visual interface web services to real-time endpoints.

  3. Go to the designer section of the workspace to view your list of pipeline drafts.

    Converted web services can be found by navigating to Endpoints > Real-time endpoints.

  4. Select a pipeline draft to open it.

    If there was an error during the conversion process, an error message will appear with instructions to resolve the issue.

Known issues

Below are known migration issues that need to be addressed manually:

  • Import Data or Export Data modules

    If you have an Import Data or Export Data module in the experiment, you need to update the data source to use a datastores. To learn how to create a datastore, see How to Access Data in Azure storage services. Your cloud storage account information have been added in the comments of the Import Data or Export Data module for your convenience.

Next steps