MLflow and Azure Machine Learning

APPLIES TO: Azure CLI ml extension v2 (current) Python SDK azure-ai-ml v2 (preview)

• v1
• v2 (current version)

MLflow is an open-source framework, designed to manage the complete machine learning lifecycle. Its ability to train and serve models on different platforms allows you to use a consistent set of tools regardless of where your experiments are running: locally on your computer, on a remote compute target, a virtual machine or an Azure Machine Learning compute instance.

Tip

Azure Machine Learning workspaces are MLflow-compatible, meaning that you can use Azure Machine Learning workspaces in the same way you use an MLflow Tracking Server. Such compatibility has the following advantages:

• You can use Azure Machine Learning workspaces as your tracking server for any experiment you are running with MLflow, regardless if they run on Azure Machine Learning or not. You only need to configure MLflow to point to the workspace where the tracking should happen.
• You can run any training routine that uses MLflow in Azure Machine Learning without changes. Model mangagement and model deployment capabilities are also supported.

MLflow can manage the complete machine learning lifecycle using four core capabilities:

• Tracking is a component of MLflow that logs and tracks your training job metrics, parameters and model artifacts; no matter your experiment's environment--locally on your computer, on a remote compute target, a virtual machine or an Azure Machine Learning compute instance.
• Model Registries is a component of MLflow that manage model's versions in a centralized repository.
• Model Deployments is a component of MLflow that deploys models registered using the MLflow format to different compute targets. Because of how MLflow models are stored, there's no need to provide scoring scripts for models in such format.
• Projects is a format for packaging data science code in a reusable and reproducible way, based primarily on conventions. It's supported on preview on Azure Machine Learning.

Tracking with MLflow

Azure Machine Learning uses MLflow Tracking for metric logging and artifact storage for your experiments, whether you created the experiment via the Azure Machine Learning Python SDK, Azure Machine Learning CLI or the Azure Machine Learning studio. We recommend using MLflow for tracking experiments. To get you started, see Log & view metrics and log files with MLflow.

Note

Unlike the Azure Machine Learning SDK v1, there's no logging functionality in the SDK v2 (preview), and it is recommended to use MLflow for logging and tracking.

With MLflow Tracking you can connect Azure Machine Learning as the backend of your MLflow experiments. The workspace provides a centralized, secure, and scalable location to store training metrics and models. This includes:

• Track ML experiments and models running locally or in the cloud with MLflow in Azure Machine Learning.
• Track Azure Databricks ML experiments with MLflow in Azure Machine Learning.
• Track Azure Synapse Analytics ML experiments with MLflow in Azure Machine Learning.

Important

• MLflow in R support is limited to tracking experiment's metrics, parameters and models on Azure Machine Learning jobs. RStudio or Jupyter Notebooks with R kernels are not supported. Model registries are not supported using the MLflow R SDK. As an alternative, use Azure ML CLI or Azure ML studio for model registration and management. View the following R example about using the MLflow tracking client with Azure Machine Learning.
• MLflow in Java support is limited to tracking experiment's metrics and parameters on Azure Machine Learning jobs. Artifacts and models can't be tracked using the MLflow Java SDK. View the following Java example about using the MLflow tracking client with the Azure Machine Learning.

To learn how to use MLflow to query experiments and runs in Azure Machine Learning, see Manage experiments and runs with MLflow

Model Registries with MLflow

Azure Machine Learning supports MLflow for model management. This represents a convenient way to support the entire model lifecycle for users familiar with the MLFlow client.

To learn more about how you can manage models using the MLflow API in Azure Machine Learning, view Manage models registries in Azure Machine Learning with MLflow.

Model Deployments of MLflow models

You can deploy MLflow models to Azure Machine Learning, so you can leverage and apply Azure Machine Learning's model management capabilities and no-code deployment offering. We support deploying MLflow models to both real-time and batch endpoints. You can use the azureml-mlflow MLflow plugin, the Azure ML CLI v2, and using the user interface in Azure Machine Learning studio.

Learn more at Deploy MLflow models to Azure Machine Learning.

Train MLflow projects (preview)

Important

This feature is currently in public preview. This preview version is provided without a service-level agreement, and it's not recommended for production workloads. Certain features might not be supported or might have constrained capabilities. For more information, see Supplemental Terms of Use for Microsoft Azure Previews.

You can submit training jobs to Azure Machine Learning using MLflow Projects (preview). You can submit jobs locally with Azure Machine Learning tracking or migrate your jobs to the cloud like via an Azure Machine Learning Compute.

Learn more at Train ML models with MLflow projects and Azure Machine Learning (preview).

MLflow SDK, Azure ML v2 and Azure ML Studio capabilities

The following table shows which operations are supported by each of the tools available in the ML lifecycle.

Feature	MLflow SDK	Azure ML v2 (CLI/SDK)	Azure ML Studio
Track and log metrics, parameters and models	✓
Retrieve metrics, parameters and models	✓1	2	✓
Submit training jobs with MLflow projects	✓
Submit training jobs with inputs and outputs		✓
Submit training pipelines		✓
Manage experiments runs	✓1	✓	✓
Manage MLflow models	✓3	✓	✓
Manage non-MLflow models		✓	✓
Deploy MLflow models to Azure Machine Learning	✓4	✓	✓
Deploy non-MLflow models to Azure Machine Learning		✓	✓

Note

• ¹ View Manage experiments and runs with MLflow for details.
• ² Only artifacts and models can be downloaded.
• ³ View Manage models registries in Azure Machine Learning with MLflow for details.
• ⁴ View Deploy MLflow models to Azure Machine Learning for details. Deployment of MLflow models to batch inference using the MLflow SDK is not possible by the moment.

Example notebooks

If you are getting started with MLflow in Azure Machine Learning, we recommend you to explore the notebooks examples about how to user MLflow:

• Training and tracking a classifier with MLflow: Demonstrates how to track experiments using MLflow, log models and combine multiple flavors into pipelines.
• Training and tracking a classifier with MLflow using Service Principal authentication: Demonstrate how to track experiments using MLflow from compute that is running outside Azure ML and how to authenticate against Azure ML services using a Service Principal.
• Hyper-parameters optimization using child runs with MLflow and HyperOpt optimizer: Demonstrate how to use child runs in MLflow to do hyper-parameter optimization for models using the popular library HyperOpt. It shows how to transfer metrics, params and artifacts from child runs to parent runs.
• Logging models instead of assets with MLflow: Demonstrates how to use the concept of models instead of artifacts with MLflow, including how to construct custom models.
• Manage experiments and runs with MLflow: Demonstrates how to query experiments, runs, metrics, parameters and artifacts from Azure ML using MLflow.
• Manage models registries with MLflow: Demonstrates how to manage models in registries using MLflow.
• No-code deployment with MLflow: Demonstrates how to deploy models in MLflow format to the different deployment target in Azure ML.
• Training models in Azure Databricks and deploying them on Azure ML with MLflow: Demonstrates how to train models in Azure Databricks and deploy them in Azure ML. It also includes how to handle cases where you also want to track the experiments with the MLflow instance in Azure Databricks.
• Migrating models with scoring scripts to MLflow format: Demonstrates how to migrate models with scoring scripts to no-code-deployment with MLflow.
• Using MLflow REST with Azure ML: Demonstrates how to work with MLflow REST API when connected to Azure ML.