Edit

Package and deploy models outside Azure Machine Learning (preview)

  • Article

You can deploy models outside of Azure Machine Learning for online serving by creating model packages (preview). Azure Machine Learning allows you to create a model package that collects all the dependencies required for deploying a machine learning model to a serving platform. You can move a model package across workspaces and even outside of Azure Machine Learning. To learn more about model packages, see Model packages for deployment (preview).

Important

This feature is currently in public preview. This preview version is provided without a service-level agreement, and we don't recommend it for production workloads. Certain features might not be supported or might have constrained capabilities.

For more information, see Supplemental Terms of Use for Microsoft Azure Previews.

In this article, you learn how package a model and deploy it to an Azure App Service.

Prerequisites

Before following the steps in this article, make sure you have the following prerequisites:

  • An Azure subscription. If you don't have an Azure subscription, create a free account before you begin. Try the free or paid version of Azure Machine Learning.

  • An Azure Machine Learning workspace. If you don't have one, use the steps in the How to manage workspaces article to create one.

    Note

    Private link enabled workspaces don't support packaging models for deployment outside of Azure Machine Learning.

  • Azure role-based access controls (Azure RBAC) are used to grant access to operations in Azure Machine Learning. To perform the steps in this article, your user account must be assigned the owner or contributor role for the Azure Machine Learning workspace, or a custom role. For more information, see Manage access to an Azure Machine Learning workspace.

Prepare your system

Follow these steps to prepare your system.

  1. The example in this article is based on code samples contained in the azureml-examples repository. To run the commands locally without having to copy/paste YAML and other files, first clone the repo and then change directories to the folder:

    git clone https://github.com/Azure/azureml-examples --depth 1
cd azureml-examples/cli

    This article uses the example in the folder endpoints/online/deploy-with-packages/mlflow-model.

  2. Connect to the Azure Machine Learning workspace where you'll do your work.

    az account set --subscription <subscription>
az configure --defaults workspace=<workspace> group=<resource-group> location=<location>

  3. Packages require the model to be registered in either your workspace or in an Azure Machine Learning registry. In this example, there's a local copy of the model in the repository, so you only need to publish the model to the registry in the workspace. You can skip this step if the model you're trying to deploy is already registered.

    MODEL_NAME='heart-classifier-mlflow'
MODEL_PATH='model'
az ml model create --name $MODEL_NAME --path $MODEL_PATH --type mlflow_model

Deploy a model package to the Azure App Service

In this section, you package the previously registered MLflow model and deploy it to the Azure App Service.

  1. Deploying a model outside of Azure Machine Learning requires creating a package specification. To create a package that's completely disconnected from Azure Machine Learning, specify the copy mode in the model configuration. The copy mode tells the package to copy the artifacts inside of the package. The following code shows how to specify the copy mode for the model configuration:

    Create a package YAML specification:

    package-external.yml

    $schema: http://azureml/sdk-2-0/ModelVersionPackage.json
target_environment: heart-classifier-mlflow-pkg
inferencing_server: 
    type: azureml_online
model_configuration:
    mode: copy

    Tip

    When you specify the model configuration using copy for the mode property, you guarantee that all the model artifacts are copied inside the generated docker image instead of downloaded from the Azure Machine Learning model registry, thereby allowing true portability outside of Azure Machine Learning. For a full specification about all the options when creating packages see Create a package specification.

  2. Start the package operation.

    az ml model package --name $MODEL_NAME --version $MODEL_VERSION --file package-external.yml

  3. The result of the package operation is an environment in Azure Machine Learning. The advantage of having this environment is that each environment has a corresponding docker image that you can use in an external deployment. Images are hosted in the Azure Container Registry. The following steps show how you get the name of the generated image:

    1. Go to the Azure Machine Learning studio.

    2. Select the Environments section.

    3. Select the Custom environments tab.

    4. Look for the environment named heart-classifier-mlflow-package, which is the name of the package you just created.

    5. Copy the value that's in the Azure container registry field.

    A screenshot showing the section where the Azure container registry image name is displayed in Azure Machine Learning studio.

  4. Now, deploy this package in an App Service.

    1. Go to the Azure portal and create a new App Service resource.

    2. In the creation wizard, select the subscription and resource group you're using.

    3. In the Instance details section, give the app a name.

    4. For Publish, select Docker container.

    5. For Operating System, select Linux.

      A screenshot showing how to configure the app service to deploy the generated docker container image.

    6. Configure the rest of the page as needed and select Next.

    7. Go to the Docker tab.

    8. For Options, select Single Container.

    9. For Image Source, select Azure Container Registry.

    10. Configure the Azure container registry options as follows:

      1. For Registry, select the Azure Container Registry associated with the Azure Machine Learning workspace.

      2. For Image, select the image that you found in step 3(e) of this tutorial.

      3. For Tag, select latest.

      A screenshot showing the section Docker of the wizard, where the docker image associated with the package is indicated.

    11. Configure the rest of the wizard as needed.

    12. Select Create. The model is now deployed in the App Service you created.

    13. The way you invoke and get predictions depends on the inference server you used. In this example, you used the Azure Machine Learning inferencing server, which creates predictions under the route /score. For more information about the input formats and features, see the details of the package azureml-inference-server-http.

    14. Prepare the request payload. The format for an MLflow model deployed with Azure Machine Learning inferencing server is as follows:

      sample-request.json

      {
    "input_data": {
        "columns": [
            "age", "sex", "cp", "trestbps", "chol", "fbs", "restecg", "thalach", "exang", "oldpeak", "slope", "ca", "thal"
        ],
        "index": [1],
        "data": [
            [1, 1, 4, 145, 233, 1, 2, 150, 0, 2.3, 3, 0, 2]
        ]
    }
}

    15. Test the model deployment to see if it works.

      cat -A sample-request.json | curl http://heart-classifier-mlflow-pkg.azurewebsites.net/score \
    --request POST \
    --header 'Content-Type: application/json' \
    --data-binary @-

Next step

Model packages for deployment (preview)