Deploy a model to Azure Container Instances

APPLIES TO: yesBasic edition yesEnterprise edition                    (Upgrade to Enterprise edition)

Learn how to use Azure Machine Learning to deploy a model as a web service on Azure Container Instances (ACI). Use Azure Container Instances if one of the following conditions is true:

  • You need to quickly deploy and validate your model. You do not need to create ACI containers ahead of time. They are created as part of the deployment process.
  • You are testing a model that is under development.

For information on quota and region availability for ACI, see Quotas and region availability for Azure Container Instances article.

Prerequisites

Deploy to ACI

To deploy a model to Azure Container Instances, create a deployment configuration that describes the compute resources needed. For example, number of cores and memory. You also need an inference configuration, which describes the environment needed to host the model and web service. For more information on creating the inference configuration, see How and where to deploy models.

Using the SDK

from azureml.core.webservice import AciWebservice, Webservice
from azureml.core.model import Model

deployment_config = AciWebservice.deploy_configuration(cpu_cores = 1, memory_gb = 1)
service = Model.deploy(ws, "aciservice", [model], inference_config, deployment_config)
service.wait_for_deployment(show_output = True)
print(service.state)

For more information on the classes, methods, and parameters used in this example, see the following reference documents:

Using the CLI

To deploy using the CLI, use the following command. Replace mymodel:1 with the name and version of the registered model. Replace myservice with the name to give this service:

az ml model deploy -m mymodel:1 -n myservice -ic inferenceconfig.json -dc deploymentconfig.json

The entries in the deploymentconfig.json document map to the parameters for AciWebservice.deploy_configuration. The following table describes the mapping between the entities in the JSON document and the parameters for the method:

JSON entity Method parameter Description
computeType NA The compute target. For ACI, the value must be ACI.
containerResourceRequirements NA Container for the CPU and memory entities.
  cpu cpu_cores The number of CPU cores to allocate. Defaults, 0.1
  memoryInGB memory_gb The amount of memory (in GB) to allocate for this web service. Default, 0.5
location location The Azure region to deploy this Webservice to. If not specified the Workspace location will be used. More details on available regions can be found here: ACI Regions
authEnabled auth_enabled Whether to enable auth for this Webservice. Defaults to False
sslEnabled ssl_enabled Whether to enable SSL for this Webservice. Defaults to False.
appInsightsEnabled enable_app_insights Whether to enable AppInsights for this Webservice. Defaults to False
sslCertificate ssl_cert_pem_file The cert file needed if SSL is enabled
sslKey ssl_key_pem_file The key file needed if SSL is enabled
cname ssl_cname The cname for if SSL is enabled
dnsNameLabel dns_name_label The dns name label for the scoring endpoint. If not specified a unique dns name label will be generated for the scoring endpoint.

The following JSON is an example deployment configuration for use with the CLI:

{
    "computeType": "aci",
    "containerResourceRequirements":
    {
        "cpu": 0.5,
        "memoryInGB": 1.0
    },
    "authEnabled": true,
    "sslEnabled": false,
    "appInsightsEnabled": false
}

For more information, see the az ml model deploy reference.

Using VS Code

See deploy your models with VS Code.

Important

You don't need to create an ACI container to test in advance. ACI containers are created as needed.

Update the web service

To update a web service, use the update method. You can update the web service to use a new model, a new entry script, or new dependencies that can be specified in an inference configuration. For more information, see the documentation for Webservice.update.

Important

When you create a new version of a model, you must manually update each service that you want to use it.

You can not use the SDK to update a web service published from the Azure Machine Learning designer.

Using the SDK

The following code shows how to use the SDK to update the model, environment, and entry script for a web service:

from azureml.core import Environment
from azureml.core.webservice import Webservice
from azureml.core.model import Model, InferenceConfig

# Register new model.
new_model = Model.register(model_path="outputs/sklearn_mnist_model.pkl",
                           model_name="sklearn_mnist",
                           tags={"key": "0.1"},
                           description="test",
                           workspace=ws)

# Use version 3 of the environment.
deploy_env = Environment.get(workspace=ws,name="myenv",version="3")
inference_config = InferenceConfig(entry_script="score.py",
                                   environment=deploy_env)

service_name = 'myservice'
# Retrieve existing service.
service = Webservice(name=service_name, workspace=ws)



# Update to new model(s).
service.update(models=[new_model], inference_config=inference_config)
print(service.state)
print(service.get_logs())

Using the CLI

You can also update a web service by using the ML CLI. The following example demonstrates registering a new model and then updating a web service to use the new model:

az ml model register -n sklearn_mnist  --asset-path outputs/sklearn_mnist_model.pkl  --experiment-name myexperiment --output-metadata-file modelinfo.json
az ml service update -n myservice --model-metadata-file modelinfo.json

Tip

In this example, a JSON document is used to pass the model information from the registration command into the update command.

To update the service to use a new entry script or environment, create an inference configuration file and specify it with the ic parameter.

For more information, see the az ml service update documentation.

Next steps