您现在访问的是微软AZURE全球版技术文档网站,若需要访问由世纪互联运营的MICROSOFT AZURE中国区技术文档网站,请访问 https://docs.azure.cn.

使用 Azure 机器学习部署模型Deploy models with Azure Machine Learning

适用于:是基本版是企业版               (升级到企业版APPLIES TO: yesBasic edition yesEnterprise edition                    (Upgrade to Enterprise edition)

了解如何将机器学习模型作为 Web 服务部署在 Azure 云或 Azure IoT Edge 设备中。Learn how to deploy your machine learning model as a web service in the Azure cloud or to Azure IoT Edge devices.

无论您在何处部署模型,工作流都是类似的:The workflow is similar no matter where you deploy your model:

  1. 注册模型。Register the model.
  2. 准备部署。Prepare to deploy. (指定资产、使用情况、计算目标。)(Specify assets, usage, compute target.)
  3. 将模型部署到计算目标。Deploy the model to the compute target.
  4. 测试已部署的模型(也称为“Web 服务”)。Test the deployed model, also called a web service.

有关部署工作流涉及的概念的详细信息,请参阅使用 Azure 机器学习管理、部署和监视模型For more information on the concepts involved in the deployment workflow, see Manage, deploy, and monitor models with Azure Machine Learning.

先决条件Prerequisites

连接到工作区Connect to your workspace

下面的代码演示如何使用缓存到本地开发环境中的信息连接到 Azure 机器学习工作区:The following code shows how to connect to an Azure Machine Learning workspace by using information cached to the local development environment:

  • 使用 SDKUsing the SDK

    from azureml.core import Workspace
    ws = Workspace.from_config(path=".file-path/ws_config.json")
    

    若要详细了解如何使用 SDK 连接到工作区,请参阅用于 Python 的 Azure 机器学习 SDK 文档。For more information on using the SDK to connect to a workspace, see the Azure Machine Learning SDK for Python documentation.

  • 使用 CLIUsing the CLI

    使用 CLI 时,请使用 -w--workspace-name 参数指定命令的工作区。When using the CLI, use the -w or --workspace-name parameter to specify the workspace for the command.

  • 使用 Visual Studio CodeUsing Visual Studio Code

    使用可视化工作室代码时,可以使用图形界面选择工作区。When you use Visual Studio Code, you select the workspace by using a graphical interface. 有关详细信息,请参阅在可视化工作室代码扩展文档中部署和管理模型For more information, see Deploy and manage models in the Visual Studio Code extension documentation.

注册您的模型Register your model

已注册的模型是构成模型的一个或多个文件的逻辑容器。A registered model is a logical container for one or more files that make up your model. 例如,如果有一个存储在多个文件中的模型,则可以在工作区中将这些文件注册为单个模型。For example, if you have a model that's stored in multiple files, you can register them as a single model in the workspace. 注册这些文件后,可以下载或部署已注册的模型,并接收注册的所有文件。After you register the files, you can then download or deploy the registered model and receive all the files that you registered.

提示

注册模型时,请提供云位置(来自训练运行)或本地目录的路径。When you register a model, you provide the path of either a cloud location (from a training run) or a local directory. 此路径仅用于在注册过程中查找要上传的文件。This path is just to locate the files for upload as part of the registration process. 它不需要与入口脚本中使用的路径匹配。It doesn't need to match the path used in the entry script. 有关详细信息,请参阅在入口脚本中查找模型文件For more information, see Locate model files in your entry script.

机器学习模型会注册到 Azure 机器学习工作区中。Machine learning models are registered in your Azure Machine Learning workspace. 模型可以来自 Azure 机器学习或其他位置。The model can come from Azure Machine Learning or from somewhere else. 注册模型时,可以选择提供有关模型的元数据。When registering a model, you can optionally provide metadata about the model. 然后tagsproperties可用于筛选模型的和字典。The tags and properties dictionaries that you apply to a model registration can then be used to filter models.

以下示例演示如何注册模型。The following examples demonstrate how to register a model.

在实验运行中注册模型Register a model from an experiment run

本节中的代码片段演示如何在训练运行中注册模型:The code snippets in this section demonstrate how to register a model from a training run:

重要

若要使用这些代码段,需要先执行一个训练运行,并且需要有权访问 Run 对象(SDK 示例)或运行 ID 值(CLI 示例)。To use these snippets, you need to have previously performed a training run and you need to have access to the Run object (SDK example) or the run ID value (CLI example). 若要详细了解如何训练模型,请参阅设置模型训练的计算目标For more information on training models, see Set up compute targets for model training.

  • 使用 SDKUsing the SDK

    使用 SDK 训练模型时,可以接收 Run 对象或 AutoMLRun 对象,具体取决于模型的训练方式。When you use the SDK to train a model, you can receive either a Run object or an AutoMLRun object, depending on how you trained the model. 每个对象都可用于注册通过实验运行创建的模型。Each object can be used to register a model created by an experiment run.

    • 通过 azureml.core.Run 对象注册模型:Register a model from an azureml.core.Run object:

      model = run.register_model(model_name='sklearn_mnist',
                                 tags={'area': 'mnist'},
                                 model_path='outputs/sklearn_mnist_model.pkl')
      print(model.name, model.id, model.version, sep='\t')
      

      model_path 参数表示模型的云位置。The model_path parameter refers to the cloud location of the model. 本示例使用的是单个文件的路径。In this example, the path of a single file is used. 若要在模型注册中包含多个文件,请将 model_path 设置为包含文件的文件夹的路径。To include multiple files in the model registration, set model_path to the path of a folder that contains the files. 有关详细信息,请参阅 Run.register_model 文档。For more information, see the Run.register_model documentation.

    • 通过 azureml.train.automl.run.AutoMLRun 对象注册模型:Register a model from an azureml.train.automl.run.AutoMLRun object:

          description = 'My AutoML Model'
          model = run.register_model(description = description,
                                     tags={'area': 'mnist'})
      
          print(run.model_id)
      

      在此示例中,未指定 metriciteration 参数,因此将注册具有最佳主要指标的迭代。In this example, the metric and iteration parameters aren't specified, so the iteration with the best primary metric will be registered. 不会使用模型名称,而是使用从运行返回的 model_id 值。The model_id value returned from the run is used instead of a model name.

      有关详细信息,请参阅AutoMLRun.register_model文档。For more information, see the AutoMLRun.register_model documentation.

  • 使用 CLIUsing the CLI

    az ml model register -n sklearn_mnist  --asset-path outputs/sklearn_mnist_model.pkl  --experiment-name myexperiment --run-id myrunid --tag area=mnist
    

    提示

    如果收到一条错误消息,指出未安装 ml 扩展,请使用以下命令进行安装:If you get an error message stating that the ml extension isn't installed, use the following command to install it:

    az extension add -n azure-cli-ml
    

    --asset-path 参数表示模型的云位置。The --asset-path parameter refers to the cloud location of the model. 本示例使用的是单个文件的路径。In this example, the path of a single file is used. 若要在模型注册中包含多个文件,请将 --asset-path 设置为包含文件的文件夹的路径。To include multiple files in the model registration, set --asset-path to the path of a folder that contains the files.

  • 使用 Visual Studio CodeUsing Visual Studio Code

    使用Visual Studio 代码扩展名使用任何模型文件或文件夹注册模型。Register models using any model files or folders by using the Visual Studio Code extension.

通过本地文件注册模型Register a model from a local file

可以通过提供模型的本地路径来注册模型。You can register a model by providing the local path of the model. 可以提供文件夹或单个文件的路径。You can provide the path of either a folder or a single file. 可以使用此方法来注册使用 Azure 机器学习训练并下载的模型。You can use this method to register models trained with Azure Machine Learning and then downloaded. 也可以使用此方法来注册在 Azure 机器学习之外训练的模型。You can also use this method to register models trained outside of Azure Machine Learning.

重要

应仅使用自己创建的或从受信任源获得的模型。You should use only models that you create or obtain from a trusted source. 应将序列化模型视为代码,因为在许多常用格式中都发现了安全漏洞。You should treat serialized models as code, because security vulnerabilities have been discovered in a number of popular formats. 此外,可能有人恶意将模型训练为提供有偏差或不准确的输出。Also, models might be intentionally trained with malicious intent to provide biased or inaccurate output.

  • 使用 SDK 和 ONNXUsing the SDK and ONNX

    import os
    import urllib.request
    from azureml.core.model import Model
    # Download model
    onnx_model_url = "https://www.cntk.ai/OnnxModels/mnist/opset_7/mnist.tar.gz"
    urllib.request.urlretrieve(onnx_model_url, filename="mnist.tar.gz")
    os.system('tar xvzf mnist.tar.gz')
    # Register model
    model = Model.register(workspace = ws,
                            model_path ="mnist/model.onnx",
                            model_name = "onnx_mnist",
                            tags = {"onnx": "demo"},
                            description = "MNIST image classification CNN from ONNX Model Zoo",)
    

    若要在模型注册中包含多个文件,请将 model_path 设置为包含文件的文件夹的路径。To include multiple files in the model registration, set model_path to the path of a folder that contains the files.

  • 使用 CLIUsing the CLI

    az ml model register -n onnx_mnist -p mnist/model.onnx
    

    若要在模型注册中包含多个文件,请将 -p 设置为包含文件的文件夹的路径。To include multiple files in the model registration, set -p to the path of a folder that contains the files.

时间估计:大约10秒。Time estimate: Approximately 10 seconds.

有关详细信息,请参阅关于模型类的文档。For more information, see the documentation for the Model class.

若要详细了解如何使用在 Azure 机器学习之外训练的模型,请参阅如何部署现有模型For more information on working with models trained outside Azure Machine Learning, see How to deploy an existing model.

单模型与多模型端点Single versus multi-model endpoints

Azure ML 支持在单个终结点后面部署单个或多个模型。Azure ML supports deploying single or multiple models behind a single endpoint.

多模型终结点使用共享容器承载多个模型。Multi-model endpoints use a shared container to host multiple models. 这有助于降低间接费用,提高利用率,并使您能够将模块链接到一起到集合中。This helps to reduce overhead costs, improves utilization, and enables you to chain modules together into ensembles. 在部署脚本中指定的模型将装载并在服务容器的磁盘上可用 - 您可以按需将它们加载到内存中,并根据评分时请求的特定模型进行评分。Models you specify in your deployment script are mounted and made available on the disk of the serving container - you can load them into memory on demand and score based on the specific model being requested at scoring time.

有关 E2E 示例(该示例演示如何在单个容器化终结点后面使用多个模型),请参阅此示例For an E2E example, which shows how to use multiple models behind a single containerized endpoint, see this example

准备部署Prepare to deploy

要将模型部署为服务,您需要以下组件:To deploy the model as a service, you need the following components:

  • 定义推理环境Define inference environment. 此环境封装运行模型进行推理所需的依赖项。This environment encapsulates the dependencies required to run your model for inference.
  • 定义计分代码Define scoring code. 此脚本接受请求、使用模型为请求评分并返回结果。This script accepts requests, scores the requests by using the model, and returns the results.
  • 定义推理配置Define inference configuration. 推理配置指定将模型作为服务运行所需的环境配置、条目脚本和其他组件。The inference configuration specifies the environment configuration, entry script, and other components needed to run the model as a service.

获得必要的组件后,可以分析部署模型以了解其 CPU 和内存要求后将创建的服务。Once you have the necessary components, you can profile the service that will be created as a result of deploying your model to understand its CPU and memory requirements.

1. 定义推理环境1. Define inference environment

推理配置描述如何设置包含模型的 Web 服务。An inference configuration describes how to set up the web-service containing your model. 此配置稍后在部署模型时使用。It's used later, when you deploy the model.

推理配置使用 Azure 机器学习环境来定义部署所需的软件依赖项。Inference configuration uses Azure Machine Learning environments to define the software dependencies needed for your deployment. 利用环境,你可以创建、管理和重复使用训练和部署所需的软件依赖项。Environments allow you to create, manage, and reuse the software dependencies required for training and deployment. 可以从自定义依赖项文件创建环境,或使用精心策划的 Azure 机器学习环境之一。You can create an environment from custom dependency files or use one of the curated Azure Machine Learning environments. 以下 YAML 是用于推理的 Conda 依赖项文件的一个示例。The following YAML is an example of a Conda dependencies file for inference. 请注意,必须指示 azureml 默认值,其中 veriver >= 1.0.45 作为点依赖项,因为它包含将模型托管为 Web 服务所需的功能。Note that you must indicate azureml-defaults with verion >= 1.0.45 as a pip dependency, because it contains the functionality needed to host the model as a web service. 如果要使用自动架构生成,则输入脚本还必须导入inference-schema包。If you want to use automatic schema generation, your entry script must also import the inference-schema packages.

name: project_environment
dependencies:
  - python=3.6.2
  - scikit-learn=0.20.0
  - pip:
      # You must list azureml-defaults as a pip dependency
    - azureml-defaults>=1.0.45
    - inference-schema[numpy-support]

重要

如果你的依赖项可通过 Conda 和 pip(通过 PyPi)使用,Microsoft 建议使用 Conda 版本,因为 Conda 包通常附带预生成的二进制文件,能让安装更可靠。If your dependency is available through both Conda and pip (from PyPi), Microsoft recommends using the Conda version, as Conda packages typically come with pre-built binaries that make installation more reliable.

有关详细信息,请参阅了解 Conda 和 PipFor more information, see Understanding Conda and Pip.

要检查依赖项是否可通过 Conda 可用,请使用conda search <package-name>命令,或使用 和https://anaconda.org/anaconda/repohttps://anaconda.org/conda-forge/repo上的包索引。To check if your dependency is available through Conda, use the conda search <package-name> command, or use the package indexes at https://anaconda.org/anaconda/repo and https://anaconda.org/conda-forge/repo.

您可以使用依赖项文件创建环境对象并将其保存到工作区以供将来使用:You can use the dependencies file to create an environment object and save it to your workspace for future use:

from azureml.core.environment import Environment
myenv = Environment.from_conda_specification(name = 'myenv',
                                             file_path = 'path-to-conda-specification-file'
myenv.register(workspace=ws)

2. 定义计分代码2. Define scoring code

入口脚本接收提交到已部署 Web 服务的数据,并将此数据传递给模型。The entry script receives data submitted to a deployed web service and passes it to the model. 然后,该脚本接收模型返回的响应,并将该响应返回给客户端。It then takes the response returned by the model and returns that to the client. 该脚本特定于你的模型**。The script is specific to your model. 它必须能够识别模型需要和返回的数据。It must understand the data that the model expects and returns.

该脚本包含两个用于加载和运行模型的函数:The script contains two functions that load and run the model:

  • init()通常,此函数将模型加载到全局对象中。init(): Typically, this function loads the model into a global object. 此函数只能在 Web 服务的 Docker 容器启动时运行一次。This function is run only once, when the Docker container for your web service is started.

  • run(input_data): 此函数使用模型预测基于输入数据的值。run(input_data): This function uses the model to predict a value based on the input data. 运行的输入和输出通常使用 JSON 进行序列化和反序列化。Inputs and outputs of the run typically use JSON for serialization and deserialization. 也可以处理原始二进制数据。You can also work with raw binary data. 可以先转换数据,然后再将数据发送到模型或返回给客户端。You can transform the data before sending it to the model or before returning it to the client.

在输入脚本中加载模型文件Load model files in your entry script

可以通过两种方法在入口脚本中查找模型:There are two ways to locate models in your entry script:

  • AZUREML_MODEL_DIR:包含到模型位置的路径的环境变量。AZUREML_MODEL_DIR: An environment variable containing the path to the model location.
  • Model.get_model_path:使用已注册的模型名称将路径返回到模型文件的 API。Model.get_model_path: An API that returns the path to model file using the registered model name.
AZUREML_MODEL_DIRAZUREML_MODEL_DIR

AZUREML_MODEL_DIR 是在服务部署过程中创建的环境变量。AZUREML_MODEL_DIR is an environment variable created during service deployment. 可以使用此环境变量来查找部署的模型的位置。You can use this environment variable to find the location of the deployed model(s).

下表描述了 AZUREML_MODEL_DIR 的值,它的值取决于部署的模型数:The following table describes the value of AZUREML_MODEL_DIR depending on the number of models deployed:

部署Deployment 环境变量值Environment variable value
单个模型Single model 包含模型的文件夹的路径。The path to the folder containing the model.
多个模型Multiple models 包含所有模型的文件夹的路径。The path to the folder containing all models. 各个模型按名称和版本放置在此文件夹中 ($MODEL_NAME/$VERSION)Models are located by name and version in this folder ($MODEL_NAME/$VERSION)

在模型注册和部署期间,模型被放置在AZUREML_MODEL_DIR路径中,并保留其原始文件名。During model registration and deployment, Models are placed in the AZUREML_MODEL_DIR path, and their original filenames are preserved.

要获取条目脚本中模型文件的路径,请将环境变量与要查找的文件路径合并。To get the path to a model file in your entry script, combine the environment variable with the file path you're looking for.

单个模型示例Single model example

# Example when the model is a file
model_path = os.path.join(os.getenv('AZUREML_MODEL_DIR'), 'sklearn_regression_model.pkl')

# Example when the model is a folder containing a file
file_path = os.path.join(os.getenv('AZUREML_MODEL_DIR'), 'my_model_folder', 'sklearn_regression_model.pkl')

多个模型示例Multiple model example

# Example when the model is a file, and the deployment contains multiple models
model_path = os.path.join(os.getenv('AZUREML_MODEL_DIR'), 'sklearn_model', '1', 'sklearn_regression_model.pkl')
get_model_pathget_model_path

注册模型时,请提供用于在注册表中管理该模型的模型名称。When you register a model, you provide a model name that's used for managing the model in the registry. 将此名称与 Model.get_model_path() 方法结合使用,以检索本地文件系统上一个或多个模型文件的路径。You use this name with the Model.get_model_path() method to retrieve the path of the model file or files on the local file system. 如果注册文件夹或文件集合,此 API 会返回包含这些文件的目录的路径。If you register a folder or a collection of files, this API returns the path of the directory that contains those files.

注册模型时,请为其指定一个名称。When you register a model, you give it a name. 该名称对应于模型的放置位置(本地位置或在服务部署过程中指定的位置)。The name corresponds to where the model is placed, either locally or during service deployment.

(可选)定义模型 Web 服务架构(Optional) Define model web service schema

若要为 Web 服务自动生成架构,请在一个已定义的类型对象的构造函数中提供输入和/或输出的示例。To automatically generate a schema for your web service, provide a sample of the input and/or output in the constructor for one of the defined type objects. 该类型和示例用于自动创建架构。The type and sample are used to automatically create the schema. Azure 机器学习随后会在部署过程中创建 Web 服务的 OpenAPI (Swagger) 规范。Azure Machine Learning then creates an OpenAPI (Swagger) specification for the web service during deployment.

当前支持以下类型:These types are currently supported:

  • pandas
  • numpy
  • pyspark
  • 标准 Python 对象Standard Python object

要使用架构生成,请在依赖项文件中包括inference-schema开源包。To use schema generation, include the open-source inference-schema package in your dependencies file. 有关此包的详细信息,请参阅https://github.com/Azure/InferenceSchemaFor more information on this package, see https://github.com/Azure/InferenceSchema. 定义 input_sampleoutput_sample 变量中的输入和输出示例格式,它们表示 Web 服务的请求和响应格式。Define the input and output sample formats in the input_sample and output_sample variables, which represent the request and response formats for the web service. run() 函数的输入和输出函数修饰器中使用这些示例。Use these samples in the input and output function decorators on the run() function. 以下 scikit-learn 示例使用架构生成功能。The following scikit-learn example uses schema generation.

入口脚本示例Example entry script

以下示例演示如何接受和返回 JSON 数据:The following example demonstrates how to accept and return JSON data:

#Example: scikit-learn and Swagger
import json
import numpy as np
import os
from sklearn.externals import joblib
from sklearn.linear_model import Ridge

from inference_schema.schema_decorators import input_schema, output_schema
from inference_schema.parameter_types.numpy_parameter_type import NumpyParameterType


def init():
    global model
    # AZUREML_MODEL_DIR is an environment variable created during deployment. Join this path with the filename of the model file.
    # It holds the path to the directory that contains the deployed model (./azureml-models/$MODEL_NAME/$VERSION).
    # If there are multiple models, this value is the path to the directory containing all deployed models (./azureml-models).
    # Alternatively: model_path = Model.get_model_path('sklearn_mnist')
    model_path = os.path.join(os.getenv('AZUREML_MODEL_DIR'), 'sklearn_mnist_model.pkl')
    # Deserialize the model file back into a sklearn model
    model = joblib.load(model_path)


input_sample = np.array([[10, 9, 8, 7, 6, 5, 4, 3, 2, 1]])
output_sample = np.array([3726.995])


@input_schema('data', NumpyParameterType(input_sample))
@output_schema(NumpyParameterType(output_sample))
def run(data):
    try:
        result = model.predict(data)
        # You can return any data type, as long as it is JSON serializable.
        return result.tolist()
    except Exception as e:
        error = str(e)
        return error

以下示例演示如何使用数据帧将输入数据定义为 <key: value> 字典。The following example demonstrates how to define the input data as a <key: value> dictionary by using a DataFrame. 此方法支持使用 Power BI 中已部署的 Web 服务。This method is supported for consuming the deployed web service from Power BI. 详细了解如何使用 Power BI 中的 Web 服务。)(Learn more about how to consume the web service from Power BI.)

import json
import pickle
import numpy as np
import pandas as pd
import azureml.train.automl
from sklearn.externals import joblib
from azureml.core.model import Model

from inference_schema.schema_decorators import input_schema, output_schema
from inference_schema.parameter_types.numpy_parameter_type import NumpyParameterType
from inference_schema.parameter_types.pandas_parameter_type import PandasParameterType


def init():
    global model
    # Replace filename if needed.
    model_path = os.path.join(os.getenv('AZUREML_MODEL_DIR'), 'model_file.pkl')
    # Deserialize the model file back into a sklearn model.
    model = joblib.load(model_path)


input_sample = pd.DataFrame(data=[{
    # This is a decimal type sample. Use the data type that reflects this column in your data.
    "input_name_1": 5.1,
    # This is a string type sample. Use the data type that reflects this column in your data.
    "input_name_2": "value2",
    # This is an integer type sample. Use the data type that reflects this column in your data.
    "input_name_3": 3
}])

# This is an integer type sample. Use the data type that reflects the expected result.
output_sample = np.array([0])


@input_schema('data', PandasParameterType(input_sample))
@output_schema(NumpyParameterType(output_sample))
def run(data):
    try:
        result = model.predict(data)
        # You can return any data type, as long as it is JSON serializable.
        return result.tolist()
    except Exception as e:
        error = str(e)
        return error

若要查看更多示例,请参阅以下脚本:For more examples, see the following scripts:

3. 定义推理配置3. Define inference configuration

下面的示例演示如何从工作区加载环境,并将其与推理配置结合使用:The following example demonstrates loading an environment from your workspace and then using it with the inference configuration:

from azureml.core.environment import Environment
from azureml.core.model import InferenceConfig


myenv = Environment.get(workspace=ws, name='myenv', version='1')
inference_config = InferenceConfig(entry_script='path-to-score.py',
                                   environment=myenv)

有关环境的详细信息,请参阅创建和管理用于训练和部署的环境For more information on environments, see Create and manage environments for training and deployment.

有关推理配置的详细信息,请参阅 InferenceConfig 类文档。For more information on inference configuration, see the InferenceConfig class documentation.

若要详细了解如何将自定义 Docker 映像与推理配置结合使用,请参阅如何使用自定义 Docker 映像部署模型For information on using a custom Docker image with an inference configuration, see How to deploy a model using a custom Docker image.

InferenceConfig 的 CLI 示例CLI example of InferenceConfig

inferenceconfig.json 文档中的条目映射到 InferenceConfig 类的参数。The entries in the inferenceconfig.json document map to the parameters for the InferenceConfig class. 下表描述了 JSON 文档中的实体与方法参数之间的映射:The following table describes the mapping between entities in the JSON document and the parameters for the method:

JSON 实体JSON entity 方法参数Method parameter 描述Description
entryScript entry_script 包含要为映像运行的代码的本地文件路径。Path to a local file that contains the code to run for the image.
sourceDirectory source_directory 可选。Optional. 包含创建映像的所有文件的文件夹的路径,从而轻松访问此文件夹或子文件夹中的任何文件。Path to folders that contain all files to create the image, which makes it easy to access any files within this folder or subfolder. 可以从本地计算机上载整个文件夹作为 Web 服务的依赖项。You can upload an entire folder from your local machine as dependencies for the Webservice. 注意:entry_script、conda_file和extra_docker_file_steps路径是source_directory路径的相对路径。Note: your entry_script, conda_file, and extra_docker_file_steps paths are relative paths to the source_directory path.
environment environment 可选。Optional. Azure 机器学习环境Azure Machine Learning environment.

您可以在推理配置文件中包括 Azure 机器学习环境的完整规范。You can include full specifications of an Azure Machine Learning environment in the inference configuration file. 如果工作区中不存在此环境,Azure 机器学习将创建它。If this environment doesn't exist in your workspace, Azure Machine Learning will create it. 否则,Azure 机器学习将在必要时更新环境。Otherwise, Azure Machine Learning will update the environment if necessary. 以下 JSON 就是一个示例:The following JSON is an example:

{
    "entryScript": "score.py",
    "environment": {
        "docker": {
            "arguments": [],
            "baseDockerfile": null,
            "baseImage": "mcr.microsoft.com/azureml/base:intelmpi2018.3-ubuntu16.04",
            "enabled": false,
            "sharedVolumes": true,
            "shmSize": null
        },
        "environmentVariables": {
            "EXAMPLE_ENV_VAR": "EXAMPLE_VALUE"
        },
        "name": "my-deploy-env",
        "python": {
            "baseCondaEnvironment": null,
            "condaDependencies": {
                "channels": [
                    "conda-forge"
                ],
                "dependencies": [
                    "python=3.6.2",
                    {
                        "pip": [
                            "azureml-defaults",
                            "azureml-telemetry",
                            "scikit-learn",
                            "inference-schema[numpy-support]"
                        ]
                    }
                ],
                "name": "project_environment"
            },
            "condaDependenciesFile": null,
            "interpreterPath": "python",
            "userManagedDependencies": false
        },
        "version": "1"
    }
}

还可以在分离的 CLI 参数中使用现有的 Azure 机器学习环境,并从推理配置文件中删除"环境"密钥。You can also use an existing Azure Machine Learning environment in separated CLI parameters and remove the "environment" key from the inference configuration file. 将 -e 用于环境名称,对于环境版本使用 -ev。Use -e for the environment name, and --ev for the environment version. 如果不指定 --ev,将使用最新版本。If you don't specify --ev, the latest version will be used. 下面是推理配置文件的示例:Here is an example of an inference configuration file:

{
    "entryScript": "score.py",
    "sourceDirectory": null
}

以下命令演示如何使用以前的推理配置文件(名为 myinferenceConfig.json)部署模型。The following command demonstrates how to deploy a model using the previous inference configuration file (named myInferenceConfig.json).

它还使用现有 Azure 机器学习环境的最新版本(称为 AzureML-最小)。It also uses the latest version of an existing Azure Machine Learning environment (named AzureML-Minimal).

az ml model deploy -m mymodel:1 --ic myInferenceConfig.json -e AzureML-Minimal --dc deploymentconfig.json

以下命令演示如何使用 CLI 部署模型:The following command demonstrates how to deploy a model by using the CLI:

az ml model deploy -n myservice -m mymodel:1 --ic inferenceconfig.json

此实例中的配置指定以下设置:In this example, the configuration specifies the following settings:

  • 该模型需要使用 Python。That the model requires Python.
  • 入口脚本,用于处理发送到部署的服务的 Web 请求The entry script, which is used to handle web requests sent to the deployed service.
  • 用于描述推理所需的 Python 包的 Conda 文件。The Conda file that describes the Python packages needed for inference.

若要详细了解如何将自定义 Docker 映像与推理配置结合使用,请参阅如何使用自定义 Docker 映像部署模型For information on using a custom Docker image with an inference configuration, see How to deploy a model using a custom Docker image.

4. (可选) 分析模型以确定资源利用率4. (Optional) Profile your model to determine resource utilization

注册模型并准备了部署模型所需的其他组件后,可以确定部署的服务所需的 CPU 和内存。Once you have registered your model and prepared the other components necessary for its deployment, you can determine the CPU and memory the deployed service will need. 分析测试运行模型的服务,并返回 CPU 使用率、内存使用情况和响应延迟等信息。Profiling tests the service that runs your model and returns information such as the CPU usage, memory usage, and response latency. 它还根据资源使用情况为 CPU 和内存提供建议。It also provides a recommendation for the CPU and memory based on resource usage.

为了分析您的模型,您需要:In order to profile your model, you will need:

  • 已注册的模型。A registered model.
  • 基于条目脚本和推理环境定义的推理配置。An inference configuration based on your entry script and inference environment definition.
  • 单列表格数据集,其中每行包含一个表示示例请求数据的字符串。A single column tabular dataset, where each row contains a string representing sample request data.

重要

此时,我们仅支持分析预期其请求数据为字符串的服务,例如:字符串序列化 json、文本、字符串序列化映像等。数据集的每一行(字符串)的内容将放入 HTTP 请求的正文中,并发送到封装模型以进行评分的服务。At this point we only support profiling of services that expect their request data to be a string, for example: string serialized json, text, string serialized image, etc. The content of each row of the dataset (string) will be put into the body of the HTTP request and sent to the service encapsulating the model for scoring.

下面是如何构造输入数据集来分析期望其传入请求数据包含序列化 json 的服务的示例。Below is an example of how you can construct an input dataset to profile a service that expects its incoming request data to contain serialized json. 在这种情况下,我们创建了一个基于相同请求数据内容的一百个实例的数据集。In this case, we created a dataset based one hundred instances of the same request data content. 在现实世界中,我们建议您使用包含各种输入的较大数据集,尤其是在模型资源使用情况/行为与输入相关的情况下。In real world scenarios we suggest that you use larger datasets containing various inputs, especially if your model resource usage/behavior is input dependent.

import json
from azureml.core import Datastore
from azureml.core.dataset import Dataset
from azureml.data import dataset_type_definitions

input_json = {'data': [[1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
                       [10, 9, 8, 7, 6, 5, 4, 3, 2, 1]]}
# create a string that can be utf-8 encoded and
# put in the body of the request
serialized_input_json = json.dumps(input_json)
dataset_content = []
for i in range(100):
    dataset_content.append(serialized_input_json)
dataset_content = '\n'.join(dataset_content)
file_name = 'sample_request_data.txt'
f = open(file_name, 'w')
f.write(dataset_content)
f.close()

# upload the txt file created above to the Datastore and create a dataset from it
data_store = Datastore.get_default(ws)
data_store.upload_files(['./' + file_name], target_path='sample_request_data')
datastore_path = [(data_store, 'sample_request_data' +'/' + file_name)]
sample_request_data = Dataset.Tabular.from_delimited_files(
    datastore_path, separator='\n',
    infer_column_types=True,
    header=dataset_type_definitions.PromoteHeadersBehavior.NO_HEADERS)
sample_request_data = sample_request_data.register(workspace=ws,
                                                   name='sample_request_data',
                                                   create_new_version=True)

准备好包含示例请求数据的数据集后,创建推理配置。Once you have the dataset containing sample request data ready, create an inference configuration. 推理配置基于score.py和环境定义。Inference configuration is based on the score.py and the environment definition. 以下示例演示如何创建推理配置和运行分析:The following example demonstrates how to create the inference configuration and run profiling:

from azureml.core.model import InferenceConfig, Model
from azureml.core.dataset import Dataset


model = Model(ws, id=model_id)
inference_config = InferenceConfig(entry_script='path-to-score.py',
                                   environment=myenv)
input_dataset = Dataset.get_by_name(workspace=ws, name='sample_request_data')
profile = Model.profile(ws,
            'unique_name',
            [model],
            inference_config,
            input_dataset=input_dataset)

profile.wait_for_completion(True)

# see the result
details = profile.get_details()

以下命令演示如何使用 CLI 分析模型:The following command demonstrates how to profile a model by using the CLI:

az ml model profile -g <resource-group-name> -w <workspace-name> --inference-config-file <path-to-inf-config.json> -m <model-id> --idi <input-dataset-id> -n <unique-name>

提示

要保留通过分析返回的信息,请使用模型的标记或属性。To persist the information returned by profiling, use tags or properties for the model. 使用标记或属性在模型注册表中存储与模型的数据。Using tags or properties stores the data with the model in the model registry. 以下示例演示添加了包含requestedCpurequestedMemoryInGb的信息的新标记:The following examples demonstrate adding a new tag containing the requestedCpu and requestedMemoryInGb information:

model.add_tags({'requestedCpu': details['requestedCpu'],
                'requestedMemoryInGb': details['requestedMemoryInGb']})
az ml model profile -g <resource-group-name> -w <workspace-name> --i <model-id> --add-tag requestedCpu=1 --add-tag requestedMemoryInGb=0.5

部署到目标Deploy to target

部署使用推理配置部署配置来部署模型。Deployment uses the inference configuration deployment configuration to deploy the models. 不管计算目标如何,部署过程都是类似的。The deployment process is similar regardless of the compute target. 部署到 AKS 的过程略有不同,因为必须提供对 AKS 群集的引用。Deploying to AKS is slightly different because you must provide a reference to the AKS cluster.

选择计算目标Choose a compute target

可以使用以下计算目标(或计算资源)来托管 Web 服务部署:You can use the following compute targets, or compute resources, to host your web service deployment:

计算目标Compute target 用于Used for GPU 支持GPU support FPGA 支持FPGA support 描述Description
本地 web 服务Local web service 测试/调试Testing/debugging     用于有限的测试和故障排除。Use for limited testing and troubleshooting. 硬件加速依赖于本地系统中库的使用情况。Hardware acceleration depends on use of libraries in the local system.
Azure 机器学习计算实例 web 服务Azure Machine Learning compute instance web service 测试/调试Testing/debugging     用于有限的测试和故障排除。Use for limited testing and troubleshooting.
Azure 库伯奈斯服务 (AKS)Azure Kubernetes Service (AKS) 实时推理Real-time inference (Web 服务部署)Yes (web service deployment) Yes 用于大规模生产部署。Use for high-scale production deployments. 提供所部署服务的快速响应时间和自动缩放。Provides fast response time and autoscaling of the deployed service. 不支持通过 Azure 机器学习 SDK 进行群集自动缩放。Cluster autoscaling isn't supported through the Azure Machine Learning SDK. 若要更改 AKS 群集中的节点,请在 Azure 门户中使用 AKS 群集的 UI。To change the nodes in the AKS cluster, use the UI for your AKS cluster in the Azure portal. AKS 是可用于设计器的唯一选项。AKS is the only option available for the designer.
Azure 容器实例Azure Container Instances 测试或开发Testing or development     用于需要小于 48 GB RAM 的基于 CPU 的小规模工作负载。Use for low-scale CPU-based workloads that require less than 48 GB of RAM.
Azure 机器学习计算群集Azure Machine Learning compute clusters (预览)批处理 推理(Preview) Batch inference (机器学习管道)Yes (machine learning pipeline)   对无服务器计算运行批量评分。Run batch scoring on serverless compute. 支持普通 VM 和低优先级 VM。Supports normal and low-priority VMs.
Azure FunctionsAzure Functions (预览)实时推理(Preview) Real-time inference      
Azure IoT EdgeAzure IoT Edge (预览)IoT 模块(Preview) IoT module     在 IoT 设备上部署和提供 ML 模型。Deploy and serve ML models on IoT devices.
Azure Data Box EdgeAzure Data Box Edge 通过 IoT EdgeVia IoT Edge   Yes 在 IoT 设备上部署和提供 ML 模型。Deploy and serve ML models on IoT devices.

备注

尽管计算目标(例如本地、Azure 机器学习计算实例和 Azure 机器学习计算群集)支持用于训练和试验的 GPU,但在__部署为 Web 服务__时,使用 GPU 进行推理仅在 Azure Kubernetes 服务中受支持。Although compute targets like local, Azure Machine Learning compute instance, and Azure Machine Learning compute clusters support GPU for training and experimentation, using GPU for inference when deployed as a web service is supported only on Azure Kubernetes Service.

只有在 Azure 机器学习计算上,才能__在通过机器学习管道评分时__使用 GPU 进行推理。Using a GPU for inference when scoring with a machine learning pipeline is supported only on Azure Machine Learning Compute.

定义部署配置Define your deployment configuration

在部署模型之前,必须定义部署配置。Before deploying your model, you must define the deployment configuration. 部署配置特定于将托管 Web 服务的计算目标**。The deployment configuration is specific to the compute target that will host the web service. 例如,在本地部署模型时,必须指定服务接受请求的端口。For example, when you deploy a model locally, you must specify the port where the service accepts requests. 该部署配置不属于入口脚本。The deployment configuration isn't part of your entry script. 它用于定义将托管模型和入口脚本的计算目标的特征。It's used to define the characteristics of the compute target that will host the model and entry script.

例如,如果没有与工作区关联的 Azure Kubernetes 服务 (AKS) 实例,则可能还需要创建计算资源。You might also need to create the compute resource, if, for example, you don't already have an Azure Kubernetes Service (AKS) instance associated with your workspace.

下表提供了为每个计算目标创建部署配置的示例:The following table provides an example of creating a deployment configuration for each compute target:

计算目标Compute target 部署配置示例Deployment configuration example
LocalLocal deployment_config = LocalWebservice.deploy_configuration(port=8890)
Azure 容器实例Azure Container Instances deployment_config = AciWebservice.deploy_configuration(cpu_cores = 1, memory_gb = 1)
Azure Kubernetes 服务Azure Kubernetes Service deployment_config = AksWebservice.deploy_configuration(cpu_cores = 1, memory_gb = 1)

可从 azureml.core.webservice 导入的本地、Azure 容器实例和 AKS Web 服务的类:The classes for local, Azure Container Instances, and AKS web services can be imported from azureml.core.webservice:

from azureml.core.webservice import AciWebservice, AksWebservice, LocalWebservice

使用 TLS 保护部署Securing deployments with TLS

有关如何保护 Web 服务部署的详细信息,请参阅启用 TLS 和部署For more information on how to secure a web service deployment, see Enable TLS and deploy.

本地部署Local deployment

若要在本地部署模型,需要在本地计算机上安装 Docker。To deploy a model locally, you need to have Docker installed on your local machine.

使用 SDKUsing the SDK

from azureml.core.webservice import LocalWebservice, Webservice

deployment_config = LocalWebservice.deploy_configuration(port=8890)
service = Model.deploy(ws, "myservice", [model], inference_config, deployment_config)
service.wait_for_deployment(show_output = True)
print(service.state)

有关详细信息,请参阅关于 LocalWebserviceModel.deploy()Webservice 的文档。For more information, see the documentation for LocalWebservice, Model.deploy(), and Webservice.

使用 CLIUsing the CLI

要使用 CLI 部署模型,请使用以下命令。To deploy a model by using the CLI, use the following command. mymodel:1 替换为注册的模型的名称和版本:Replace mymodel:1 with the name and version of the registered model:

az ml model deploy -m mymodel:1 --ic inferenceconfig.json --dc deploymentconfig.json

deploymentconfig.json 文档中的条目对应于 LocalWebservice.deploy_configuration 的参数。The entries in the deploymentconfig.json document map to the parameters for LocalWebservice.deploy_configuration. 下表描述了 JSON 文档中的实体与方法参数之间的映射:The following table describes the mapping between the entities in the JSON document and the parameters for the method:

JSON 实体JSON entity 方法参数Method parameter 描述Description
computeType NANA 计算目标。The compute target. 对于本地目标,值必须是 localFor local targets, the value must be local.
port port 用于公开服务的 HTTP 终结点的本地端口。The local port on which to expose the service's HTTP endpoint.

此 JSON 是用于 CLI 的部署配置示例:This JSON is an example deployment configuration for use with the CLI:

{
    "computeType": "local",
    "port": 32267
}

有关详细信息,请参阅 az ml 模型部署文档。For more information, see the az ml model deploy documentation.

了解服务状态Understanding service state

在模型部署期间,您可能会在服务状态完全部署时看到更改。During model deployment, you may see the service state change while it fully deploys.

下表描述了不同的服务状态:The following table describes the different service states:

Web 服务状态Webservice state 说明Description 最终状态?Final state?
过渡Transitioning 该服务正在部署中。The service is in the process of deployment. No
不正常Unhealthy 该服务已部署,但当前无法访问。The service has deployed but is currently unreachable. No
不可舍尔Unschedulable 由于缺乏资源,此时无法部署该服务。The service cannot be deployed at this time due to lack of resources. No
失败Failed 由于错误或崩溃,服务无法部署。The service has failed to deploy due to an error or crash. Yes
正常Healthy 该服务是正常运行的,并且终结点可用。The service is healthy and the endpoint is available. Yes

计算实例 Web 服务(开发/测试)Compute instance web service (dev/test)

请参阅将模型部署到 Azure 机器学习计算实例See Deploy a model to Azure Machine Learning compute instance.

Azure 容器实例(开发/测试)Azure Container Instances (dev/test)

请参阅部署到 Azure 容器实例See Deploy to Azure Container Instances.

Azure Kubernetes 服务(开发/测试和生产)Azure Kubernetes Service (dev/test and production)

请参阅部署到 Azure Kubernetes 服务See Deploy to Azure Kubernetes Service.

A/B 测试(受控推出)A/B Testing (controlled rollout)

有关详细信息,请参阅 ML 模型的受控推出See Controlled rollout of ML models for more information.

使用 Web 服务Consume web services

每个部署的 Web 服务都提供有一个 REST 终结点,因此可以使用任何编程语言创建客户端应用程序。Every deployed web service provides a REST endpoint, so you can create client applications in any programming language. 如果已为服务启用基于密钥的身份验证,则需要提供服务密钥,将其作为请求标头中的令牌。If you've enabled key-based authentication for your service, you need to provide a service key as a token in your request header. 如果已为服务启用基于令牌的身份验证,则需要提供 Azure 机器学习 JSON Web 令牌 (JWT),将其作为请求标头中的持有者令牌。If you've enabled token-based authentication for your service, you need to provide an Azure Machine Learning JSON Web Token (JWT) as a bearer token in your request header.

主要区别在于,密钥是静态的且能手动重新生成,而令牌需要在到期时刷新********。The primary difference is that keys are static and can be regenerated manually, and tokens need to be refreshed upon expiration. Azure 容器实例和 Azure Kubernetes 服务部署的 Web 服务支持基于密钥的身份验证,而基于令牌的身份验证仅能用于 Azure Kubernetes 服务部署****。Key-based auth is supported for Azure Container Instance and Azure Kubernetes Service deployed web-services, and token-based auth is only available for Azure Kubernetes Service deployments. 请参阅身份验证操作说明,了解更多信息和特定代码示例。See the how-to on authentication for more information and specific code samples.

提示

部署服务后,可以检索架构 JSON 文档。You can retrieve the schema JSON document after you deploy the service. 使用部署的 Web 服务中的 swagger_uri 属性(例如 service.swagger_uri)获取本地 Web 服务的 Swagger 文件的 URI。Use the swagger_uri property from the deployed web service (for example, service.swagger_uri) to get the URI to the local web service's Swagger file.

使用“请求-响应”Request-response consumption

下面是如何在 Python 中调用服务的示例:Here's an example of how to invoke your service in Python:

import requests
import json

headers = {'Content-Type': 'application/json'}

if service.auth_enabled:
    headers['Authorization'] = 'Bearer '+service.get_keys()[0]
elif service.token_auth_enabled:
    headers['Authorization'] = 'Bearer '+service.get_token()[0]

print(headers)

test_sample = json.dumps({'data': [
    [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
    [10, 9, 8, 7, 6, 5, 4, 3, 2, 1]
]})

response = requests.post(
    service.scoring_uri, data=test_sample, headers=headers)
print(response.status_code)
print(response.elapsed)
print(response.json())

有关详细信息,请参阅创建客户端应用程序以使用 Web 服务For more information, see Create client applications to consume web services.

Web 服务架构(OpenAPI 规范)Web service schema (OpenAPI specification)

如果在部署中使用了自动生成架构,则可以通过使用 swagger_uri 属性获取服务的 OpenAPI 规范的地址。If you used automatic schema generation with your deployment, you can get the address of the OpenAPI specification for the service by using the swagger_uri property. (例如.) print(service.swagger_uri)使用 GET 请求或在浏览器中打开 URI 以检索规范。(For example, print(service.swagger_uri).) Use a GET request or open the URI in a browser to retrieve the specification.

以下 JSON 文档是为部署生成的架构(OpenAPI 规范)示例:The following JSON document is an example of a schema (OpenAPI specification) generated for a deployment:

{
    "swagger": "2.0",
    "info": {
        "title": "myservice",
        "description": "API specification for Azure Machine Learning myservice",
        "version": "1.0"
    },
    "schemes": [
        "https"
    ],
    "consumes": [
        "application/json"
    ],
    "produces": [
        "application/json"
    ],
    "securityDefinitions": {
        "Bearer": {
            "type": "apiKey",
            "name": "Authorization",
            "in": "header",
            "description": "For example: Bearer abc123"
        }
    },
    "paths": {
        "/": {
            "get": {
                "operationId": "ServiceHealthCheck",
                "description": "Simple health check endpoint to ensure the service is up at any given point.",
                "responses": {
                    "200": {
                        "description": "If service is up and running, this response will be returned with the content 'Healthy'",
                        "schema": {
                            "type": "string"
                        },
                        "examples": {
                            "application/json": "Healthy"
                        }
                    },
                    "default": {
                        "description": "The service failed to execute due to an error.",
                        "schema": {
                            "$ref": "#/definitions/ErrorResponse"
                        }
                    }
                }
            }
        },
        "/score": {
            "post": {
                "operationId": "RunMLService",
                "description": "Run web service's model and get the prediction output",
                "security": [
                    {
                        "Bearer": []
                    }
                ],
                "parameters": [
                    {
                        "name": "serviceInputPayload",
                        "in": "body",
                        "description": "The input payload for executing the real-time machine learning service.",
                        "schema": {
                            "$ref": "#/definitions/ServiceInput"
                        }
                    }
                ],
                "responses": {
                    "200": {
                        "description": "The service processed the input correctly and provided a result prediction, if applicable.",
                        "schema": {
                            "$ref": "#/definitions/ServiceOutput"
                        }
                    },
                    "default": {
                        "description": "The service failed to execute due to an error.",
                        "schema": {
                            "$ref": "#/definitions/ErrorResponse"
                        }
                    }
                }
            }
        }
    },
    "definitions": {
        "ServiceInput": {
            "type": "object",
            "properties": {
                "data": {
                    "type": "array",
                    "items": {
                        "type": "array",
                        "items": {
                            "type": "integer",
                            "format": "int64"
                        }
                    }
                }
            },
            "example": {
                "data": [
                    [ 10, 9, 8, 7, 6, 5, 4, 3, 2, 1 ]
                ]
            }
        },
        "ServiceOutput": {
            "type": "array",
            "items": {
                "type": "number",
                "format": "double"
            },
            "example": [
                3726.995
            ]
        },
        "ErrorResponse": {
            "type": "object",
            "properties": {
                "status_code": {
                    "type": "integer",
                    "format": "int32"
                },
                "message": {
                    "type": "string"
                }
            }
        }
    }
}

有关详细信息,请参阅 OpenAPI 规范For more information, see OpenAPI specification.

若要了解可根据规范创建客户端库的实用工具,请参阅 swagger-codegenFor a utility that can create client libraries from the specification, see swagger-codegen.

批量推理Batch inference

Azure 机器学习计算目标由 Azure 机器学习创建和管理。Azure Machine Learning Compute targets are created and managed by Azure Machine Learning. 它们可用于 Azure 机器学习管道中的批量预测。They can be used for batch prediction from Azure Machine Learning pipelines.

若要查看使用 Azure 机器学习计算进行批量推理的演练,请参阅如何运行批量预测For a walkthrough of batch inference with Azure Machine Learning Compute, see How to run batch predictions.

IoT Edge 推理IoT Edge inference

对部署到边缘的支持处于预览阶段。Support for deploying to the edge is in preview. 有关详细信息,请参阅将 Azure 机器学习部署为 IoT Edge 模块For more information, see Deploy Azure Machine Learning as an IoT Edge module.

更新 Web 服务Update web services

若要更新 Web 服务,请使用 update 方法。To update a web service, use the update method. 你可以更新 Web 服务,以使用可以在推理配置中指定的新模型、新入口脚本或新依赖项。You can update the web service to use a new model, a new entry script, or new dependencies that can be specified in an inference configuration. 有关详细信息,请参阅 Webservice.update 的文档。For more information, see the documentation for Webservice.update.

重要

创建模型的新版本时,必须手动更新要使用的每个服务。When you create a new version of a model, you must manually update each service that you want to use it.

不能使用 SDK 更新从 Azure 机器学习设计器发布的 Web 服务。You can not use the SDK to update a web service published from the Azure Machine Learning designer.

使用 SDKUsing the SDK

下面的代码演示如何使用 SDK 更新 Web 服务的模型、环境和入口脚本:The following code shows how to use the SDK to update the model, environment, and entry script for a web service:

from azureml.core import Environment
from azureml.core.webservice import Webservice
from azureml.core.model import Model, InferenceConfig

# Register new model.
new_model = Model.register(model_path="outputs/sklearn_mnist_model.pkl",
                           model_name="sklearn_mnist",
                           tags={"key": "0.1"},
                           description="test",
                           workspace=ws)

# Use version 3 of the environment.
deploy_env = Environment.get(workspace=ws,name="myenv",version="3")
inference_config = InferenceConfig(entry_script="score.py",
                                   environment=deploy_env)

service_name = 'myservice'
# Retrieve existing service.
service = Webservice(name=service_name, workspace=ws)



# Update to new model(s).
service.update(models=[new_model], inference_config=inference_config)
print(service.state)
print(service.get_logs())

使用 CLIUsing the CLI

还可以使用 ML CLI 更新 Web 服务。You can also update a web service by using the ML CLI. 以下示例演示如何注册新模型,然后更新 Web 服务以使用新模型:The following example demonstrates registering a new model and then updating a web service to use the new model:

az ml model register -n sklearn_mnist  --asset-path outputs/sklearn_mnist_model.pkl  --experiment-name myexperiment --output-metadata-file modelinfo.json
az ml service update -n myservice --model-metadata-file modelinfo.json

提示

此示例使用 JSON 文档将模型信息从注册命令传递到更新命令。In this example, a JSON document is used to pass the model information from the registration command into the update command.

若要更新服务以使用新的入口脚本或环境,请创建推理配置文件并使用 ic 参数指定它。To update the service to use a new entry script or environment, create an inference configuration file and specify it with the ic parameter.

有关详细信息,请参阅 az ml 服务更新文档。For more information, see the az ml service update documentation.

持续部署模型Continuously deploy models

可以通过使用 Azure DevOps 的机器学习扩展来持续部署模型。You can continuously deploy models by using the Machine Learning extension for Azure DevOps. 如果在 Azure 机器学习工作区中注册了新的机器学习模型,则可以使用 Azure DevOps 的机器学习扩展来触发部署管道。You can use the Machine Learning extension for Azure DevOps to trigger a deployment pipeline when a new machine learning model is registered in an Azure Machine Learning workspace.

  1. 注册 Azure Pipelines,它能将应用程序持续集成和交付到任何平台或云。Sign up for Azure Pipelines, which makes continuous integration and delivery of your application to any platform or cloud possible. (请注意,Azure Pipelines 不同于机器学习管道。)(Note that Azure Pipelines isn't the same as Machine Learning pipelines.)

  2. 创建 Azure DevOps 项目。Create an Azure DevOps project.

  3. 安装 Azure Pipelines 的机器学习扩展Install the Machine Learning extension for Azure Pipelines.

  4. 使用服务连接设置与 Azure 机器学习工作区的服务主体连接,以便访问你的项目。Use service connections to set up a service principal connection to your Azure Machine Learning workspace so you can access your artifacts. 转到项目设置,选择“服务连接”,然后选择“Azure 资源管理器”********:Go to project settings, select Service connections, and then select Azure Resource Manager:

    选择 Azure 资源管理器Select Azure Resource Manager

  5. 在“范围级别”列表中选择“AzureMLWorkspace”,然后输入其余值********:In the Scope level list, select AzureMLWorkspace, and then enter the rest of the values:

    选择 AzureMLWorkspace

  6. 若要使用 Azure Pipelines 持续部署机器学习模型,请在管道下选择“发布”****。To continuously deploy your machine learning model by using Azure Pipelines, under pipelines, select release. 添加新项目,然后选择之前创建的“AzureML 模型”项目以及服务连接****。Add a new artifact, and then select the AzureML Model artifact and the service connection that you created earlier. 选择模型和版本以触发部署:Select the model and version to trigger a deployment:

    选择 AzureML 模型Select AzureML Model

  7. 对模型项目启用模型触发器。Enable the model trigger on your model artifact. 如果开启触发器,则每次在工作区中注册该模型的指定版本(即最新版本)时,都将触发 Azure DevOps 发布管道。When you turn on the trigger, every time the specified version (that is, the newest version) of that model is registered in your workspace, an Azure DevOps release pipeline is triggered.

    启用模型触发器Enable the model trigger

若要查看更多示例项目和示例,请参阅 GitHub 中的以下示例存储库:For more sample projects and examples, see these sample repos in GitHub:

下载模型Download a model

如果要下载模型以便在自己的执行环境中使用它,可以使用以下 SDK/CLI 命令:If you want to download your model to use it in your own execution environment, you can do so with the following SDK / CLI commands:

SDK:SDK:

model_path = Model(ws,'mymodel').download()

CLI:CLI:

az ml model download --model-id mymodel:1 --target-dir model_folder

(预览)无代码模型部署(Preview) No-code model deployment

无代码模型部署目前处于预览阶段,支持以下机器学习框架:No-code model deployment is currently in preview and supports the following machine learning frameworks:

Tensorflow SavedModel 格式Tensorflow SavedModel format

需要以 SavedModel 格式注册 Tensorflow 模型,才能进行无代码模型部署****。Tensorflow models need to be registered in SavedModel format to work with no-code model deployment.

请访问此链接以了解如何创建 SavedModel。Please see this link for information on how to create a SavedModel.

from azureml.core import Model

model = Model.register(workspace=ws,
                       model_name='flowers',                        # Name of the registered model in your workspace.
                       model_path='./flowers_model',                # Local Tensorflow SavedModel folder to upload and register as a model.
                       model_framework=Model.Framework.TENSORFLOW,  # Framework used to create the model.
                       model_framework_version='1.14.0',            # Version of Tensorflow used to create the model.
                       description='Flowers model')

service_name = 'tensorflow-flower-service'
service = Model.deploy(ws, service_name, [model])

ONNX 模型ONNX models

任何 ONNX 推理图都支持 ONNX 模型注册和部署。ONNX model registration and deployment is supported for any ONNX inference graph. 当前不支持预处理和后续处理步骤。Preprocess and postprocess steps are not currently supported.

下面是有关如何注册和部署 MNIST ONNX 模型的示例:Here is an example of how to register and deploy an MNIST ONNX model:

from azureml.core import Model

model = Model.register(workspace=ws,
                       model_name='mnist-sample',                  # Name of the registered model in your workspace.
                       model_path='mnist-model.onnx',              # Local ONNX model to upload and register as a model.
                       model_framework=Model.Framework.ONNX ,      # Framework used to create the model.
                       model_framework_version='1.3',              # Version of ONNX used to create the model.
                       description='Onnx MNIST model')

service_name = 'onnx-mnist-service'
service = Model.deploy(ws, service_name, [model])

如果您使用的是 Pytorch,则将模型从 PyTorch 导出到 ONNX时,有转换和限制的详细信息。If you're using Pytorch, Exporting models from PyTorch to ONNX has the details on conversion and limitations.

Scikit-learn 模型Scikit-learn models

所有内置 scikit-learn 模型类型都支持无代码模型部署。No code model deployment is supported for all built-in scikit-learn model types.

下面的示例演示如何在不使用额外代码的情况下注册和部署 sklearn 模型:Here is an example of how to register and deploy a sklearn model with no extra code:

from azureml.core import Model
from azureml.core.resource_configuration import ResourceConfiguration

model = Model.register(workspace=ws,
                       model_name='my-sklearn-model',                # Name of the registered model in your workspace.
                       model_path='./sklearn_regression_model.pkl',  # Local file to upload and register as a model.
                       model_framework=Model.Framework.SCIKITLEARN,  # Framework used to create the model.
                       model_framework_version='0.19.1',             # Version of scikit-learn used to create the model.
                       resource_configuration=ResourceConfiguration(cpu=1, memory_in_gb=0.5),
                       description='Ridge regression model to predict diabetes progression.',
                       tags={'area': 'diabetes', 'type': 'regression'})
                       
service_name = 'my-sklearn-service'
service = Model.deploy(ws, service_name, [model])

注: 默认情况下,支持predict_proba的模型将使用该方法。NOTE: Models which support predict_proba will use that method by default. 要重写此内容以使用预测,可以修改 POST 正文,如下所示:To override this to use predict you can modify the POST body as below:

import json


input_payload = json.dumps({
    'data': [
        [ 0.03807591,  0.05068012,  0.06169621, 0.02187235, -0.0442235,
         -0.03482076, -0.04340085, -0.00259226, 0.01990842, -0.01764613]
    ],
    'method': 'predict'  # If you have a classification model, the default behavior is to run 'predict_proba'.
})

output = service.run(input_payload)

print(output)

注: 这些依赖项包含在预构建的 sk学习推理容器中:NOTE: These dependencies are included in the prebuilt sklearn inference container:

    - azureml-defaults
    - inference-schema[numpy-support]
    - scikit-learn
    - numpy

包模型Package models

在某些情况下,可能需要在不部署模型的情况下创建 Docker 映像(例如要部署到 Azure 应用服务)。In some cases, you might want to create a Docker image without deploying the model (if, for example, you plan to deploy to Azure App Service). 或者,你可能希望下载映像并在本地 Docker 安装上运行它。Or you might want to download the image and run it on a local Docker installation. 甚至可能需要下载用于生成映像的文件、对其进行检查和修改并手动生成映像。You might even want to download the files used to build the image, inspect them, modify them, and build the image manually.

这些工作都可以通过打包模型来完成。Model packaging enables you to do these things. 此方法能对将模型作为 Web 服务托管所需的全部资产进行打包,让你能下载完整生成的 Docker 映像或生成该映像所需的文件。It packages all the assets needed to host a model as a web service and allows you to download either a fully built Docker image or the files needed to build one. 可以通过两种方式使用模型打包:There are two ways to use model packaging:

下载打包的模型: 下载包含模型和其他文件作为 Web 服务托管所需的 Docker 映像。Download a packaged model: Download a Docker image that contains the model and other files needed to host it as a web service.

生成 Docker 文件: 下载构建 Docker 映像所需的 Docker 文件、模型、条目脚本和其他资产。Generate a Dockerfile: Download the Dockerfile, model, entry script, and other assets needed to build a Docker image. 然后可以先检查这些文件或进行修改,再在本地生成映像。You can then inspect the files or make changes before you build the image locally.

这两个包都可用于获取本地 Docker 映像。Both packages can be used to get a local Docker image.

提示

创建包类似于部署模型。Creating a package is similar to deploying a model. 使用注册的模型和推理配置。You use a registered model and an inference configuration.

重要

若要下载完整生成的映像或要在本地生成映像,需要在开发环境中安装 DockerTo download a fully built image or build an image locally, you need to have Docker installed in your development environment.

下载已打包的模型Download a packaged model

下面的示例生成一个映像,该映像已在工作区的 Azure 容器注册表中注册:The following example builds an image, which is registered in the Azure container registry for your workspace:

package = Model.package(ws, [model], inference_config)
package.wait_for_creation(show_output=True)

创建包后,可以使用 package.pull() 将映像拉取到本地 Docker 环境。After you create a package, you can use package.pull() to pull the image to your local Docker environment. 此命令的输出将显示映像的名称。The output of this command will display the name of the image. 例如:For example:

Status: Downloaded newer image for myworkspacef78fd10.azurecr.io/package:20190822181338.Status: Downloaded newer image for myworkspacef78fd10.azurecr.io/package:20190822181338.

下载模型后,使用 docker images 命令列出本地映像:After you download the model, use the docker images command to list the local images:

REPOSITORY                               TAG                 IMAGE ID            CREATED             SIZE
myworkspacef78fd10.azurecr.io/package    20190822181338      7ff48015d5bd        4 minutes ago       1.43 GB

若要启动基于此映像的本地容器,请使用以下命令从 Shell 或命令行启动已命名容器。To start a local container based on this image, use the following command to start a named container from the shell or command line. 使用 docker images 命令返回的映像 ID 替换 <imageid> 值。Replace the <imageid> value with the image ID returned by the docker images command.

docker run -p 6789:5001 --name mycontainer <imageid>

此命令启动名为 myimage 的映像的最新版本。This command starts the latest version of the image named myimage. 它将本地端口 6789 映射到 Web 服务正在侦听的容器中的端口 (5001)。It maps local port 6789 to the port in the container on which the web service is listening (5001). 还将名称 mycontainer 分配给容器,使容器更易于停止。It also assigns the name mycontainer to the container, which makes the container easier to stop. 启动容器后,可以将请求提交到 http://localhost:6789/scoreAfter the container is started, you can submit requests to http://localhost:6789/score.

生成 Dockerfile 和依赖项Generate a Dockerfile and dependencies

下面的示例演示如何下载在本地生成映像所需的 Dockerfile、模型和其他资产。The following example shows how to download the Dockerfile, model, and other assets needed to build an image locally. generate_dockerfile=True 参数表示需要的是文件,而不是完整生成的映像。The generate_dockerfile=True parameter indicates that you want the files, not a fully built image.

package = Model.package(ws, [model], inference_config, generate_dockerfile=True)
package.wait_for_creation(show_output=True)
# Download the package.
package.save("./imagefiles")
# Get the Azure container registry that the model/Dockerfile uses.
acr=package.get_container_registry()
print("Address:", acr.address)
print("Username:", acr.username)
print("Password:", acr.password)

此代码将生成映像所需的文件下载到 imagefiles 目录。This code downloads the files needed to build the image to the imagefiles directory. 已保存的文件中包含的 Dockerfile 引用存储在 Azure 容器注册表中的基础映像。The Dockerfile included in the saved files references a base image stored in an Azure container registry. 在本地 Docker 安装上生成映像时,需要使用地址、用户名和密码完成注册表身份验证。When you build the image on your local Docker installation, you need to use the address, user name, and password to authenticate to the registry. 采取以下步骤,通过本地 Docker 安装生成映像:Use the following steps to build the image by using a local Docker installation:

  1. 在 Shell 或命令行会话中使用以下命令,使用 Azure 容器注册表对 Docker 进行身份验证。From a shell or command-line session, use the following command to authenticate Docker with the Azure container registry. <address><username><password> 替换为 package.get_container_registry() 检索到的值。Replace <address>, <username>, and <password> with the values retrieved by package.get_container_registry().

    docker login <address> -u <username> -p <password>
    
  2. 若要生成映像,请使用以下命令。To build the image, use the following command. <imagefiles> 替换为 package.save() 保存文件的目录的路径。Replace <imagefiles> with the path of the directory where package.save() saved the files.

    docker build --tag myimage <imagefiles>
    

    此命令将映像名设置为 myimageThis command sets the image name to myimage.

若要验证是否已生成映像,请使用 docker images 命令。To verify that the image is built, use the docker images command. 在列表中应该能看到 myimage 映像:You should see the myimage image in the list:

REPOSITORY      TAG                 IMAGE ID            CREATED             SIZE
<none>          <none>              2d5ee0bf3b3b        49 seconds ago      1.43 GB
myimage         latest              739f22498d64        3 minutes ago       1.43 GB

若要启动基于此映像的新容器,请使用以下命令:To start a new container based on this image, use the following command:

docker run -p 6789:5001 --name mycontainer myimage:latest

此命令启动名为 myimage 的映像的最新版本。This command starts the latest version of the image named myimage. 它将本地端口 6789 映射到 Web 服务正在侦听的容器中的端口 (5001)。It maps local port 6789 to the port in the container on which the web service is listening (5001). 还将名称 mycontainer 分配给容器,使容器更易于停止。It also assigns the name mycontainer to the container, which makes the container easier to stop. 启动容器后,可以将请求提交到 http://localhost:6789/scoreAfter the container is started, you can submit requests to http://localhost:6789/score.

测试本地容器的示例客户端Example client to test the local container

以下代码是可与容器结合使用的 Python 客户端示例:The following code is an example of a Python client that can be used with the container:

import requests
import json

# URL for the web service.
scoring_uri = 'http://localhost:6789/score'

# Two sets of data to score, so we get two results back.
data = {"data":
        [
            [ 1,2,3,4,5,6,7,8,9,10 ],
            [ 10,9,8,7,6,5,4,3,2,1 ]
        ]
        }
# Convert to JSON string.
input_data = json.dumps(data)

# Set the content type.
headers = {'Content-Type': 'application/json'}

# Make the request and display the response.
resp = requests.post(scoring_uri, input_data, headers=headers)
print(resp.text)

若要通过示例了解采用其他编程语言的客户端,请参阅使用部署为 Web 服务的模型For example clients in other programming languages, see Consume models deployed as web services.

停止 Docker 容器Stop the Docker container

若要停止容器,请在不同的 Shell 或命令行中使用以下命令:To stop the container, use the following command from a different shell or command line:

docker kill mycontainer

清理资源Clean up resources

若要删除已部署的 Web 服务,请使用 service.delete()To delete a deployed web service, use service.delete(). 若要删除已注册的模型,请使用 model.delete()To delete a registered model, use model.delete().

有关详细信息,请参阅关于 WebService.delete()Model.delete() 的文档。For more information, see the documentation for WebService.delete() and Model.delete().

高级条目脚本创作Advanced entry script authoring

Binary dataBinary data

如果模型接受二进制数据(如映像),则必须修改用于部署的 score.py 文件以接受原始 HTTP 请求。If your model accepts binary data, like an image, you must modify the score.py file used for your deployment to accept raw HTTP requests. 若要接受原始数据,请在入口脚本中使用 AMLRequest 类,并向 run() 函数添加 @rawhttp 修饰器。To accept raw data, use the AMLRequest class in your entry script and add the @rawhttp decorator to the run() function.

下面是接受二进制数据的 score.py 的示例:Here's an example of a score.py that accepts binary data:

from azureml.contrib.services.aml_request import AMLRequest, rawhttp
from azureml.contrib.services.aml_response import AMLResponse


def init():
    print("This is init()")


@rawhttp
def run(request):
    print("This is run()")
    print("Request: [{0}]".format(request))
    if request.method == 'GET':
        # For this example, just return the URL for GETs.
        respBody = str.encode(request.full_path)
        return AMLResponse(respBody, 200)
    elif request.method == 'POST':
        reqBody = request.get_data(False)
        # For a real-world solution, you would load the data from reqBody
        # and send it to the model. Then return the response.

        # For demonstration purposes, this example just returns the posted data as the response.
        return AMLResponse(reqBody, 200)
    else:
        return AMLResponse("bad request", 500)

重要

AMLRequest 类位于 azureml.contrib 命名空间中。The AMLRequest class is in the azureml.contrib namespace. 此命名空间中的实体会频繁更改,因为我们正在改进服务。Entities in this namespace change frequently as we work to improve the service. 此命名空间中的任何内容都应被视为预览版,Microsoft 并不完全支持这些内容。Anything in this namespace should be considered a preview that's not fully supported by Microsoft.

如果需要在本地开发环境中对此进行测试,可以使用以下命令安装这些组件:If you need to test this in your local development environment, you can install the components by using the following command:

pip install azureml-contrib-services

跨域资源共享 (CORS)Cross-origin resource sharing (CORS)

跨源资源共享是允许从其他域请求网页上的资源的一种方式。Cross-origin resource sharing is a way to allow resources on a webpage to be requested from another domain. CORS 通过 HTTP 标头工作,这些标头通过客户端请求发送并随服务响应返回。CORS works via HTTP headers sent with the client request and returned with the service response. 若要详细了解 CORS 和有效标头,请参阅维基百科上的跨域资源共享 (CORS)For more information on CORS and valid headers, see Cross-origin resource sharing in Wikipedia.

若要配置模型部署以支持 CORS,请在入口脚本中使用 AMLResponse 类。To configure your model deployment to support CORS, use the AMLResponse class in your entry script. 使用此类,可设置响应对象的标头。This class allows you to set the headers on the response object.

以下示例在入口脚本中设置响应的 Access-Control-Allow-Origin 标头:The following example sets the Access-Control-Allow-Origin header for the response from the entry script:

from azureml.contrib.services.aml_response import AMLResponse

def init():
    print("This is init()")

def run(request):
    print("This is run()")
    print("Request: [{0}]".format(request))
    if request.method == 'GET':
        # For this example, just return the URL for GETs.
        respBody = str.encode(request.full_path)
        return AMLResponse(respBody, 200)
    elif request.method == 'POST':
        reqBody = request.get_data(False)
        # For a real-world solution, you would load the data from reqBody
        # and send it to the model. Then return the response.

        # For demonstration purposes, this example
        # adds a header and returns the request body.
        resp = AMLResponse(reqBody, 200)
        resp.headers['Access-Control-Allow-Origin'] = "http://www.example.com"
        return resp
    else:
        return AMLResponse("bad request", 500)

重要

AMLResponse 类位于 azureml.contrib 命名空间中。The AMLResponse class is in the azureml.contrib namespace. 此命名空间中的实体会频繁更改,因为我们正在改进服务。Entities in this namespace change frequently as we work to improve the service. 此命名空间中的任何内容都应被视为预览版,Microsoft 并不完全支持这些内容。Anything in this namespace should be considered a preview that's not fully supported by Microsoft.

如果需要在本地开发环境中对此进行测试,可以使用以下命令安装这些组件:If you need to test this in your local development environment, you can install the components by using the following command:

pip install azureml-contrib-services

警告

Azure 机器学习将仅将开机自走自走和 GET 请求路由到运行评分服务的容器。Azure Machine Learning will route only POST and GET requests to the containers running the scoring service. 这可能导致错误,因为浏览器使用 OPTIONS 请求进行预检 CORS 请求。This can cause errors due to browsers using OPTIONS requests to pre-flight CORS requests.

后续步骤Next steps