Create model packages (preview)

Grein
12/22/2023

Model package is a capability in Azure Machine Learning that allows you to collect all the dependencies required to deploy a machine learning model to a serving platform. Creating packages before deploying models provides robust and reliable deployment and a more efficient MLOps workflow. Packages can be moved across workspaces and even outside of Azure Machine Learning.

Important

This feature is currently in public preview. This preview version is provided without a service-level agreement, and we don't recommend it for production workloads. Certain features might not be supported or might have constrained capabilities.

For more information, see Supplemental Terms of Use for Microsoft Azure Previews.

In this article, you learn how to package a model for deployment.

Prerequisites

Before following the steps in this article, make sure you have the following prerequisites:

An Azure subscription. If you don't have an Azure subscription, create a free account before you begin. Try the free or paid version of Azure Machine Learning.
An Azure Machine Learning workspace. If you don't have one, use the steps in the How to manage workspacesarticle to create one.
Azure role-based access controls (Azure RBAC) are used to grant access to operations in Azure Machine Learning. To perform the steps in this article, your user account must be assigned the owner or contributor role for the Azure Machine Learning workspace, or a custom role. For more information, see Manage access to an Azure Machine Learning workspace.

About this example

In this example, you will learn how to package models in Azure Machine Learning.

Clone the repository

The example in this article is based on code samples contained in the azureml-examples repository. To run the commands locally without having to copy/paste YAML and other files, first clone the repo and then change directories to the folder:

Azure CLI
Python

git clone https://github.com/Azure/azureml-examples --depth 1
cd azureml-examples/cli

!git clone https://github.com/Azure/azureml-examples --depth 1
!cd azureml-examples/sdk/python

This section uses the example in the folder endpoints/online/deploy-packages/custom-model.

Connect to your workspace

Connect to the Azure Machine Learning workspace where you'll do your work.

Azure CLI
Python

az account set --subscription <subscription>
az configure --defaults workspace=<workspace> group=<resource-group> location=<location>

The workspace is the top-level resource for Azure Machine Learning, providing a centralized place to work with all the artifacts you create when you use Azure Machine Learning. In this section, you connect to the workspace in which you perform deployment tasks.

Import the required libraries:

from azure.identity import DefaultAzureCredential
from azure.ai.ml import MLClient
from azure.ai.ml.entities import (
    AzureMLOnlineInferencingServer,
    ModelPackage,
    CodeConfiguration,
    BaseEnvironment,
    ModelConfiguration,
)
from azure.ai.ml.entities import (
    ManagedOnlineEndpoint,
    ManagedOnlineDeployment,
    Environment,
    Model,
)

If you're running in a compute instance in Azure Machine Learning, create an MLClient as follows:

ml_client = MLClient.from_config(DefaultAzureCredential())

Otherwise, configure your workspace details and get a handle to the workspace:

subscription_id = "<subscription>"
resource_group = "<resource-group>"
workspace = "<workspace>"

ml_client = MLClient(DefaultAzureCredential(), subscription_id, resource_group, workspace)

Package a model

You can create model packages explicitly to allow you to control how the packaging operation is done. Use this workflow when:

You want to customize how the model package is created.
You want to deploy the model package outside Azure Machine Learning.
You want to use model packages in an MLOps workflow.

You can create model packages by specifying the:

Model to package: Each model package can contain only a single model. Azure Machine Learning doesn't support packaging of multiple models under the same model package.
Base environment: Environments are used to indicate the base image, and in Python packages dependencies your model need. For MLflow models, Azure Machine Learning automatically generates the base environment. For custom models, you need to specify it.
Serving technology: The inferencing stack used to run the model.

Register the model

Model packages require the model to be registered in either your workspace or in an Azure Machine Learning registry. In this example, you already have a local copy of the model in the repository, so you only need to publish the model to the registry in the workspace. You can skip this section if the model you're trying to deploy is already registered.

Azure CLI
Python

MODEL_NAME='sklearn-regression'
MODEL_PATH='model'
az ml model create --name $MODEL_NAME --path $MODEL_PATH --type custom_model

model_name = "sklearn-regression"
model_path = "model"

model = ml_client.models.create_or_update(Model(name=model_name, path=model_path))

Create the base environment

Base environments are used to indicate the base image and the model Python package dependencies. Our model requires the following packages to be used as indicated in the conda file:

conda.yaml

name: model-env
channels:
  - conda-forge
dependencies:
  - python=3.9
  - numpy=1.23.5
  - pip=23.0.1
  - scikit-learn=1.2.2
  - scipy=1.10.1
  - xgboost==1.3.3

Note

How is the base environment different from the environment you use for model deployment to online and batch endpoints? When you deploy models to endpoints, your environment needs to include the dependencies of the model and the Python packages that are required for managed online endpoints to work. This brings a manual process into the deployment, where you have to combine the requirements of your model with the requirements of the serving platform. On the other hand, use of model packages removes this friction, since the required packages for the inference server will automatically be injected into the model package at packaging time.

Create the environment as follows:

Azure CLI
Python

Create an environment definition:

sklearn-regression-env.yml

$schema: https://azuremlschemas.azureedge.net/latest/environment.schema.json
name: sklearn-regression-env
image: mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu22.04
conda_file: conda.yaml
description: An environment for models built with XGBoost and Scikit-learn.

Then create the environment:

az ml environment create -f environment/sklearn-regression-env.yml

base_environment = ml_client.environments.create_or_update(
    Environment(
        name=f"{model_name}-env",
        image="mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu22.04",
        conda_file="environment/conda.yml",
    )
)

Create a package specification

You can create model packages in Azure Machine Learning, using the Azure CLI or the Azure Machine Learning SDK for Python. The custom package specification supports the following attributes:

Azure CLI
Python

Attribute	Type	Description	Required
`target_environment`	`str`	The name of the package to create. The result of a package operation is an environment in Azure Machine Learning.	Yes
`base_environment_source`	`object`	The base image to use to create the package where dependencies for the model are specified.	Yes, unless model is MLflow.
`base_environment_source.type`	`str`	The type of the base image. Only using another environment as the base image is supported (`type: environment_asset`) is supported.
`base_environment_source.resource_id`	`str`	The resource ID of the base environment to use. Use format `azureml:<name>:<version>` or a long resource id.
`inferencing_server`	`object`	The inferencing server to use.	Yes
`inferencing_server.type`	`azureml_online` `custom`	Use `azureml_online` for the Azure Machine Learning inferencing server, or `custom` for a custom online server like TensorFlow serving or Torch Serve.	Yes
`inferencing_server.code_configuration`	`object`	The code configuration with the inference routine. It should contain at least one Python file with methods `init` and `run`.	Yes, unless model is MLflow.
`model_configuration`	`object`	The model configuration. Use this attribute to control how the model is packaged in the resulting image.	No
`model_configuration.mode`	`download` `copy`	Indicate how the model would be placed in the package. Possible values are `download` (default) and `copy`. Use `download` when you want the model to be downloaded from the model registry at deployment time. This option create smaller docker images since the model is not included on it. Use `copy` when you want to disconnect the image from Azure Machine Learning. Model will be copied inside of the docker image at package time. `copy` is not supported on private link-enabled workspaces.	No

Attribute	Type	Description	Required
`target_environment`	`str`	The name of the package to create. The result of a package operation is an environment in Azure Machine Learning.	Yes
`base_environment_source`	`BaseEnvironment`	The base image to use to create the package where dependencies for the model are specified.	Yes, unless model type is MLflow.
`base_environment_source.type`	`BaseEnvironmentType`	The type of the base image. Only using another environment (`EnvironmentAsset`) as the base image is supported.
`base_environment_source.resource_id`	`str`	The resource ID of the base environment to use. Use format `azureml:<name>:<version>` or a long-format resource id.
`inferencing_server`	`AzureMLOnlineInferencingServer` `CustomOnlineInferenceServer`	The inferencing server to use. Use `AzureMLOnlineInferencingServer` to Azure Machine Learning inferencing server, or `CustomOnlineInferenceServer` for a custom online server like TensorFlow serving, Torch Serve, etc. If set to `AzureMLInferencingServer` and the model type isn't Mlflow, a code configuration section should be specified, containing at least one Python file with methods `init` and `run`. If set to `CustomOnlineInferenceServer`, an online server configuration section should be specified.	Yes
`model_configuration`	`ModelConfiguration`	The model configuration. Use this attribute to control how the model is packaged in the resulting image.	No
`model_configuration.mode`	`ModelInputMode`	Specify how the model would be placed in the package. Possible values are `ModelInputMode.DOWNLOAD` (default) and `ModelInputMode.COPY`. Use `ModelInputMode.DOWNLOAD` when you want the model to be downloaded from the model registry at deployment time. This option create smaller docker images since the model is not included on it. Use `ModelInputMode.COPY` when you want to disconnect the image from Azure Machine Learning. Model will be copied inside of the docker image at package time. `ModelInputMode.COPY` is not supported on private link-enabled workspaces.	No

Create a package specification as follows:

Azure CLI
Python

package-moe.yml

$schema: http://azureml/sdk-2-0/ModelVersionPackage.json
base_environment_source:
    type: environment_asset
    resource_id: azureml:sklearn-regression-env:1
target_environment: sklearn-regression-online-pkg
inferencing_server: 
    type: azureml_online
    code_configuration:
      code: src
      scoring_script: score.py

To create a model package, create a package specification as follows:

pakage_config = ModelPackage(
    target_environment="sklearn-regression-online-pkg",
    base_environment_source=BaseEnvironment(
        type="asset",
        resource_id=f"azureml:{base_environment.name}:{base_environment.version}",
    ),
    inferencing_server=AzureMLOnlineInferencingServer(
        code_configuration=CodeConfiguration(code="src", scoring_script="score.py")
    ),
)

Start the model package operation:

Azure CLI
Python

az ml model package -n $MODEL_NAME -v $MODEL_VERSION --file package-moe.yml

model_package = ml_client.models.package(model_name, model.version, pakage_config)

The result of the package operation is an environment.

Package a model that has dependencies in private Python feeds

Model packages can resolve Python dependencies that are available in private feeds. To use this capability, you need to create a connection from your workspace to the feed and specify the PAT token configuration. The following Python code shows how you can configure the workspace where you're running the package operation.

from azure.ai.ml.entities import WorkspaceConnection
from azure.ai.ml.entities import PatTokenConfiguration

# fetching secrets from env var to secure access, these secrets can be set outside or source code
git_pat = os.environ["GIT_PAT"]

credentials = PatTokenConfiguration(pat=git_pat)

ws_connection = WorkspaceConnection(
    name="<workspace_connection_name>",
    target="<git_url>",
    type="git",
    credentials=credentials,
)

ml_client.connections.create_or_update(ws_connection)

Once the connection is created, build the model package as described in the section for Package a model. In the following example, the base environment of the package uses a private feed for the Python dependency bar, as specified in the following conda file:

conda.yml

name: foo
channels:
  - defaults
dependencies:
  - python
  - pip
  - pip:
    - --extra-index-url <python_feed_url>
    - bar

If you're using an MLflow model, model dependencies are indicated inside the model itself, and hence a base environment isn't needed. Instead, specify private feed dependencies when logging the model, as explained in Logging models with a custom signature, environment or samples.

Package a model that is hosted in a registry

Model packages provide a convenient way to collect dependencies before deployment. However, when models are hosted in registries, the deployment target is usually another workspace. When creating packages in this setup, use the target_environment property to specify the full location where you want the model package to be created, instead of just its name.

The following code creates a package of the t5-base model from a registry:

Connect to the registry where the model is located and the workspace in which you need the model package to be created:

Azure CLI
Python

az login

registry_client = MLClient(
    credential=DefaultAzureCredential(),
    subscription_id=ml_client.subscription_id,
    resource_group_name=ml_client.resource_group_name,
    workspace_reference=ml_client.workspace_name,
    registry_name=registry_name,
)

Get a reference to the model you want to package. In this case we are packaging the model t5-base from azureml registry.

Azure CLI
Python

MODEL_NAME="t5-base"
MODEL_VERSION=$(az ml model show --name $MODEL_NAME --label latest --registry-name azureml | jq .version -r)

model_name = "t5-base"
model = registry_client.models.get(name=model_name, label="latest")

Configure a package specification. Since the model we want to package is MLflow, base environment and scoring script is optional.

Azure CLI
Python

package.yml

$schema: http://azureml/sdk-2-0/ModelVersionPackage.json
target_environment: pkg-t5-base-online
inferencing_server: 
    type: azureml_online

import time

model_package_name = f"pkg-{model.name}-{model.version}"
model_package_version = str(int(time.time()))

package_config = ModelPackage(
    target_environment=model_package_name,
    target_environment_version=model_package_version,
    inferencing_server=AzureMLOnlineInferencingServer(),
)

Start the operation to create the model package:

Azure CLI
Python

az ml model package --name $MODEL_NAME \
                    --version $MODEL_VERSION \
                    --registry-name azureml \
                    --file package.yml

model_package = registry_client.models.package(
    model.name, model.version, package_config
)

The package is now created in the target workspace and ready to be deployed.

Package models to deploy outside of Azure Machine Learning

Model packages can be deployed outside of Azure Machine Learning if needed. To guarantee portability, you only need to ensure that the model configuration in your package has the mode set to copy so that the model itself is copied inside the generated docker image instead of referenced from the model registry in Azure Machine Learning.

The following code shows how to configure copy in a model package:

Azure CLI
Python

package-external.yml

$schema: http://azureml/sdk-2-0/ModelVersionPackage.json
base_environment_source:
    type: environment_asset
    resource_id: azureml:sklearn-regression-env:1
target_environment: sklearn-regression-docker-pkg
inferencing_server: 
    type: azureml_online
    code_configuration:
      code: src
      scoring_script: score.py
model_configuration:
  mode: copy

pakage_config = ModelPackage(
    target_environment="sklearn-regression-online-pkg",
    base_environment_source=BaseEnvironment(
        type="asset",
        resource_id=f"azureml:{base_environment.name}:{base_environment.version}",
    ),
    inferencing_server=AzureMLOnlineInferencingServer(
        code_configuration=CodeConfiguration(code="src", scoring_script="score.py")
    ),
    model_configuration=ModelConfiguration(mode="copy"),
)

Deila með