将机器学习模型转换为 ONNX 与 WinMLToolsConvert ML models to ONNX with WinMLTools

WinMLTools可以将转换为 ONNX 使用不同的训练框架创建的机器学习模型。WinMLTools enables you to convert machine learning models created with different training frameworks into ONNX. 是的扩展ONNXMLToolsTF2ONNX将转换为 ONNX 模型,以用于 Windows 机器学习。It is an extension of ONNXMLTools and TF2ONNX to convert models to ONNX for use with Windows ML.

WinMLTools 目前支持以下框架从转换:WinMLTools currently supports conversion from the following frameworks:

  • Apple Core MLApple Core ML
  • KerasKeras
  • scikit-learnscikit-learn
  • lightgbmlightgbm
  • xgboostxgboost
  • libSVMlibSVM
  • TensorFlow (实验性)TensorFlow (experimental)

若要了解如何从其他 ML 框架导出,看一看ONNX 教程GitHub 上。To learn how to export from other ML frameworks, take a look at the ONNX tutorials on GitHub.

在本文中,我们将演示如何使用 WinMLTools 到:In this article, we demonstrate how to use WinMLTools to:

  • Core ML 模型转换为 ONNXConvert Core ML models into ONNX
  • 转换 scikit-了解到 ONNX 模型Convert scikit-learn models into ONNX
  • 转换为 ONNX TensorFlow 模型Convert TensorFlow models into ONNX
  • 应用后培训权重 ONNX 模型向量化Apply post-training weight quantization to ONNX models
  • 将转换为 16 位浮点模型点模型精度的浮点数Convert floating point models to 16-bit floating point precision models
  • 创建自定义 ONNX 运算符Create custom ONNX operators

备注

最新版本的 WinMLTools支持将转换为 ONNX 版本 1.2.2 及以下版本 1.3,分别由 ONNX opsets 7 和 8 指定。The latest version of WinMLTools supports conversion to ONNX versions 1.2.2 and 1.3, as specified respectively by ONNX opsets 7 and 8. Tool 的以前版本不具有对 ONNX 1.3 的支持。Previous versions of the tool do not have support for ONNX 1.3.

安装 WinMLToolsInstall WinMLTools

WinMLTools 是 Python 包 (winmltools) 支持 Python 2.7 和 3.6 版。WinMLTools is a Python package (winmltools) that supports Python versions 2.7 and 3.6. 如果你在开展数据科学项目,我们建议你安装 Python 科学发行版,如 Anaconda。If you are working on a data science project, we recommend installing a scientific Python distribution such as Anaconda.

备注

WinMLTools 当前不支持 Python 3.7。WinMLTools does not currently support Python 3.7.

遵循 WinMLTools标准 Python 包安装过程WinMLTools follows the standard Python package installation process. 在 Python 环境中,运行:From your Python environment, run:

pip install winmltools

WinMLTools 要依赖以下几项才能运行:WinMLTools has the following dependencies:

  • numpy v1.10.0+numpy v1.10.0+
  • protobuf v.3.6.0+protobuf v.3.6.0+
  • onnx v1.3.0+onnx v1.3.0+
  • onnxmltools v1.3.0+onnxmltools v1.3.0+
  • tf2onnx v0.3.2+tf2onnx v0.3.2+

若要更新的相关软件,运行pip命令与-U参数。To update the dependent packages, run the pip command with the -U argument.

pip install -U winmltools

有关不同转换器,需要安装不同的包。For different converters, you will have to install different packages.

有关libsvm,可以下载libsvm 滚轮从各种 web 源。For libsvm, you can download libsvm wheel from various web sources. 一个示例,请参阅加利福尼亚大学欧文分校的网站One example can be found at the University of California, Irvine's website.

有关coremltools,目前,Apple 将 Core ML 打包在 Windows 上的不分发。For coremltools, currently Apple does not distribute Core ML packaging on Windows. 你可以从源安装:You can install from source:

pip install git+https://github.com/apple/coremltools

请按照onnxmltools有关详细信息的 GitHub 上onnxmltools依赖项。Follow onnxmltools on GitHub for further information on onnxmltools dependencies.

可以使用的特定于包的文档上找到更多详细信息以使用 WinMLToolshelp函数。Additional details on how to use WinMLTools can be found on the package-specific documentation with the help function.

help(winmltools)

将核心 ML 模型转换Convert Core ML models

下面,我们假设一个示例 Core ML 模型文件的路径为 example.mlmodelHere, we assume that the path of an example Core ML model file is example.mlmodel.

from coremltools.models.utils import load_spec
# Load model file
model_coreml = load_spec('example.mlmodel')
from winmltools import convert_coreml
# Convert it!
# The automatic code generator (mlgen) uses the name parameter to generate class names.
model_onnx = convert_coreml(model_coreml, 7, name='ExampleModel')

备注

对 convert_coreml() 的调用中的第二个参数是 target_opset,和它引用默认命名空间中的运算符的版本号ai.onnxThe second parameter in the call to convert_coreml() is the target_opset, and it refers to the version number of the operators in the default namespace ai.onnx. 请参阅这些运算符更多详细信息此处See more details on these operators here. 此参数才 WinMLTools,使开发人员以面向不同 ONNX 版本 (当前支持版本 1.2.2 及以下版本 1.3) 的最新版本上提供。This parameter is only available on the latest version of WinMLTools, enabling developers to target different ONNX versions (currently versions 1.2.2 and 1.3 are supported). 要转换模型以运行与 Windows 10 2018 年 10 月更新,请使用target_opset7 (ONNX v1.2.2)。To convert models to run with the Windows 10 October 2018 update, use target_opset 7 (ONNX v1.2.2). 对于 Windows 10 版本大于 17763,WinML 接受具有模型target_opset7 和 8 (ONNX v.1.3)。For Windows 10 builds greater than 17763, WinML accepts models with target_opset 7 and 8 (ONNX v.1.3). 发行说明部分还包含 WinML 在不同版本中支持的最小值和最大 ONNX 版本。The Release Notes section also contains the min and max ONNX versions supported by WinML in different builds.

model_onnx 是 ONNX ModelProto对象。model_onnx is an ONNX ModelProto object. 我们可以用两种不同的格式保存它。We can save it in two different formats.

from winmltools.utils import save_model
# Save the produced ONNX model in binary format
save_model(model_onnx, 'example.onnx')
# Save the produced ONNX model in text format
from winmltools.utils import save_text
save_text(model_onnx, 'example.txt')

备注

核心 MLTools 是由 Apple 提供的 Python 包,但在 Windows 上不可用。Core MLTools is a Python package provided by Apple, but is not available on Windows. 如果你需要在 Windows 上安装该包,可直接从资源库安装:If you need to install the package on Windows, install the package directly from the repo:

pip install git+https://github.com/apple/coremltools

将核心机器学习模型与图像输入或输出转换Convert Core ML models with image inputs or outputs

由于缺少的 ONNX 中的映像类型,转换 Core ML 图像模型 (即,使用图像作为输入或输出的模型) 需要一些预处理和后处理步骤。Because of the lack of image types in ONNX, converting Core ML image models (that is, models using images as inputs or outputs) requires some pre-processing and post-processing steps.

预处理的目标是确保输入图像格式正确,可作为 ONNX 张量。The objective of pre-processing is to make sure the input image is properly formatted as an ONNX tensor. 假设 X 是 Core ML 中具有形状 [C, H, W] 的图像输入。Assume X is an image input with shape [C, H, W] in Core ML. 在 ONNX 中,变量 X 会是具有相同形状的浮点张量,X[0, :, :]/X[1, :, :]/X[2, :, :] 存储图像的红/绿/蓝通道。In ONNX, the variable X would be a floating-point tensor with the same shape and X[0, :, :]/X[1, :, :]/X[2, :, :] stores the image's red/green/blue channel. 对于核心机器学习中的灰度图像,其格式为 [1,H、 W]-在 ONNX tensors 因为我们只需一个通道。For grayscale images in Core ML, their format is [1, H, W]-tensors in ONNX because we only have one channel.

如果原始 Core ML 模型输出一个图像,请手动将 ONNX 的浮点输出张量重新转换为图像。If the original Core ML model outputs an image, manually convert ONNX's floating-point output tensors back into images. 以下是两个基本步骤。There are two basic steps. 第一步是将超过 255 的值截断为 255,并将所有负值更改为 0。The first step is to truncate values greater than 255 to 255 and change all negative values to 0. 第 2 步是将所有像素值取整(先增加 0.5,然后去掉小数部分)。The second step is to round all pixel values to integers (by adding 0.5 and then truncating the decimals). Core ML 模型中表示输出通道的顺序 (例如,RGB 或 BGR)。The output channel order (for example, RGB or BGR) is indicated in the Core ML model. 若要生成正确的图像输出,我们可能需要反转或打乱以恢复所需的格式。To generate proper image output, we may need to transpose or shuffle to recover the desired format.

我们将从 GitHub 下载的 Core ML 模型 FNS-Candy 视为演示 ONNX 和 Core ML 之间区别的一个具体转换示例。Here we consider a Core ML model, FNS-Candy, downloaded from GitHub, as a concrete conversion example to demonstrate the difference between ONNX and Core ML formats. 请注意,在 Python 环境中执行所有后续命令。Note that all the subsequent commands are executed in a Python environment.

首先,我们加载 Core ML 模型,然后检查其输入和输出格式。First, we load the Core ML model and examine its input and output formats.

from coremltools.models.utils import load_spec
model_coreml = load_spec('FNS-Candy.mlmodel')
model_coreml.description # Print the content of Core ML model description

屏幕输出:Screen output:

...
input {
    ...
      imageType {
      width: 720
      height: 720
      colorSpace: BGR
    ...
}
...
output {
    ...
      imageType {
      width: 720
      height: 720
      colorSpace: BGR
    ...
}
...

在这种情况下,输入和输出是 720 x 720 BGR 的映像。In this case, both the input and output are 720x720 BGR-images. 接下来我们要使用 WinMLTools 转换该 Core ML 模型。Our next step is to convert the Core ML model with WinMLTools.

# The automatic code generator (mlgen) uses the name parameter to generate class names.
from winmltools import convert_coreml
model_onnx = convert_coreml(model_coreml, 7, name='FNSCandy')

若要查看的模型输入和输出中 ONNX 格式另一种方法是使用以下命令:An alternative method to view the model input and output formats in ONNX is to use the following command:

model_onnx.graph.input # Print out the ONNX input tensor's information

屏幕输出:Screen output:

...
  tensor_type {
    elem_type: FLOAT
    shape {
      dim {
        dim_param: "None"
      }
      dim {
        dim_value: 3
      }
      dim {
        dim_value: 720
      }
      dim {
        dim_value: 720
      }
    }
  }
...

在 ONNX 中生成的输入(由 X 指定) 是一个 4D 张量。The produced input (denoted by X) in ONNX is a 4-D tensor. 后 3 个轴分别是 C、H 和 W 轴。The last 3 axes are C-, H-, and W-axes, respectively. 第一个维是“无”,这就是说此 ONNX 模型可应用于任意数量的图像。The first dimension is "None" which means that this ONNX model can be applied to any number of images. 要应用此模型来处理 2 个图像,第一幅图像对应于 X[0, :, :, :] ,而第二个图像对应于 X[1:,:,:]。To apply this model to process a batch of 2 images, the first image corresponds to X[0, :, :, :] while X[1, :, :, :] corresponds to the second image. 第一个图像的蓝/绿/红通道是 X[0, 0, :, :]/X[0, 1, :, :]/X[0, 2, :, :],第二个图像与此类似。The blue/green/red channels of the first image are X[0, 0, :, :]/X[0, 1, :, :]/X[0, 2, :, :], and similar for the second image.

model_onnx.graph.output # Print out the ONNX output tensor's information

屏幕输出:Screen output:

...
  tensor_type {
    elem_type: FLOAT
    shape {
      dim {
        dim_param: "None"
      }
      dim {
        dim_value: 3
      }
      dim {
        dim_value: 720
      }
      dim {
        dim_value: 720
      }
    }
  }
...

如你所见,所生成的格式与原始模型输入格式相同。As you can see, the produced format is identical to the original model input format. 但是,在此例中,它不是图像,因为像素值是整数而不是浮点数。However, in this case, it's not an image because the pixel values are integers, not floating-point numbers. 若要重新转换为图像,请将大于 255 的值截断为 255,将负值更改为 0,再对所有小数取整,使之变为整数。To convert back to an image, truncate values greater than 255 to 255, change negative values to 0, and round all decimals to integers.

转换 scikit-了解模型Convert scikit-learn models

下面的代码段在 scikit-learn 中训练线性支持向量机,并将该模型转换为 ONNX。The following code snippet trains a linear support vector machine in scikit-learn and converts the model into ONNX.

# First, we create a toy data set.
# The training matrix X contains three examples, with two features each.
# The label vector, y, stores the labels of all examples.
X = [[0.5, 1.], [-1., -1.5], [0., -2.]]
y = [1, -1, -1]

# Then, we create a linear classifier and train it.
from sklearn.svm import LinearSVC
linear_svc = LinearSVC()
linear_svc.fit(X, y)

# To convert scikit-learn models, we need to specify the input feature's name and type for our converter.
# The following line means we have a 2-D float feature vector, and its name is "input" in ONNX.
# The automatic code generator (mlgen) uses the name parameter to generate class names.
from winmltools import convert_sklearn
from winmltools.convert.common.data_types import FloatTensorType
linear_svc_onnx = convert_sklearn(linear_svc, 7, name='LinearSVC',
                                  initial_types=[('input', FloatTensorType([1, 2]))])

# Now, we save the ONNX model into binary format.
from winmltools.utils import save_model
save_model(linear_svc_onnx, 'linear_svc.onnx')

# If you'd like to load an ONNX binary file, our tool can also help.
from winmltools.utils import load_model
linear_svc_onnx = load_model('linear_svc.onnx')

# To see the produced ONNX model, we can print its contents or save it in text format.
print(linear_svc_onnx)
from winmltools.utils import save_text
save_text(linear_svc_onnx, 'linear_svc.txt')

# The conversion of linear regression is very similar. See the example below.
from sklearn.svm import LinearSVR
linear_svr = LinearSVR()
linear_svr.fit(X, y)
linear_svr_onnx = convert_sklearn(linear_svr, 7, name='LinearSVR',
                                  initial_types=[('input', FloatTensorType([1, 2]))])

像以前那样convert_sklearn采用 scikit-了解作为其第一个参数,模型和target_opset的第二个参数。As before convert_sklearn takes a scikit-learn model as its first argument, and the target_opset for the second argument. 用户可将 LinearSVC 替换为其他 scikit-learn 模型,如 RandomForestClassifierUsers can replace LinearSVC with other scikit-learn models such as RandomForestClassifier. 请注意, mlgen使用name参数来生成的类名和变量。Please note that mlgen uses the name parameter to generate class names and variables. 如果未提供 name,则会生成一个 GUID,它将不符合 C++/C# 等语言的变量命名约定。If name is not provided, then a GUID is generated, which will not comply with variable naming conventions for languages like C++/C#.

转换 scikit-了解管道Convert scikit-learn pipelines

接下来,我们将介绍 scikit-learn 管道可如何转换为 ONNX。Next, we show how scikit-learn pipelines can be converted into ONNX.

# First, we create a toy data set.
# Notice that the first example's last feature value, 300, is large.
X = [[0.5, 1., 300.], [-1., -1.5, -4.], [0., -2., -1.]]
y = [1, -1, -1]

# Then, we declare a linear classifier.
from sklearn.svm import LinearSVC
linear_svc = LinearSVC()

# One common trick to improve a linear model's performance is to normalize the input data.
from sklearn.preprocessing import Normalizer
normalizer = Normalizer()

# Here, we compose our scikit-learn pipeline.
# First, we apply our normalization.
# Then we feed the normalized data into the linear model.
from sklearn.pipeline import make_pipeline
pipeline = make_pipeline(normalizer, linear_svc)
pipeline.fit(X, y)

# Now, we convert the scikit-learn pipeline into ONNX format.
# Compared to the previous example, notice that the specified feature dimension becomes 3.
# The automatic code generator (mlgen) uses the name parameter to generate class names.
from winmltools import convert_sklearn
from winmltools.convert.common.data_types import FloatTensorType, Int64TensorType
pipeline_onnx = convert_sklearn(linear_svc, name='NormalizerLinearSVC',
                                input_features=[('input', FloatTensorType([1, 3]))])

# We can print the fresh ONNX model.
print(pipeline_onnx)

# We can also save the ONNX model into a binary file for later use.
from winmltools.utils import save_model
save_model(pipeline_onnx, 'pipeline.onnx')

# Our conversion framework provides limited support of heterogeneous feature values.
# We cannot have numerical types and string types in one feature vector.
# Let's assume that the first two features are floats and the last feature is integers (encoded a categorical attribute).
X_heter = [[0.5, 1., 300], [-1., -1.5, 400], [0., -2., 100]]

# One popular way to represent categorical is one-hot encoding.
from sklearn.preprocessing import OneHotEncoder
one_hot_encoder = OneHotEncoder(categorical_features=[2])

# Let's initialize a classifier.
# It will be right after the one-hot encoder in our pipeline.
linear_svc = LinearSVC()

# Then, we form a two-stage pipeline.
another_pipeline = make_pipeline(one_hot_encoder, linear_svc)
another_pipeline.fit(X_heter, y)

# Now, we convert, save, and load the converted model.
# For heterogeneous feature vectors, we need to separately specify their types for
# all homogeneous segments.
# The automatic code generator (mlgen) uses the name parameter to generate class names.
another_pipeline_onnx = convert_sklearn(another_pipeline, name='OneHotLinearSVC',
                                        input_features=[('input', FloatTensorType([1, 2])),
                                        ('another_input', Int64TensorType([1, 1]))])
save_model(another_pipeline_onnx, 'another_pipeline.onnx')
from winmltools.utils import load_model
loaded_onnx_model = load_model('another_pipeline.onnx')

# Of course, we can print the ONNX model to see if anything went wrong.
print(another_pipeline_onnx)

转换 TensorFlow 模型Convert TensorFlow models

以下代码是如何将模型从冻结 TensorFlow 模型转换的示例。The following code is an example of how to convert a model from a frozen TensorFlow model. 获取可能的输出的 TensorFlow 模型的名称,则可以使用summarize_graph 工具To get the possible output names of a TensorFlow model, you can use the summarize_graph tool.

import winmltools
import tensorflow

filename = 'frozen-model.pb'
output_names = ['output:0']

graph_def = graph_pb2.GraphDef()
with open(filename, 'rb') as file:
  graph_def.ParseFromString(file.read())
g = tf.import_graph_def(graph_def, name='')

with tf.Session(graph=g) as sess:
  converted_model = winmltools.convert_tensorflow(sess.graph, 7, output_names=['output:0'])
  winmltools.save_model(converted_model)

使用 WinMLTools 转换器tf2onnx.tfonnx.process_tf_graphTF2ONNXThe WinMLTools converter uses tf2onnx.tfonnx.process_tf_graph in TF2ONNX.

将转换为浮点数 16Convert to floating point 16

WinMLTools 支持以浮点数到浮点 32 点 16 表示形式表示的模型的转换 (IEEE 754 一半),从而有效地压缩模型通过减少了一半其大小。WinMLTools supports the conversion of models represented in floating point 32 into a floating point 16 representation (IEEE 754 half), effectively compressing the model by reducing its size in half.

备注

将您的模型转换为浮点数 16 可能导致精度丢失。Converting your model to floating point 16 could result in a loss of accuracy. 请确保部署到你的应用程序之前验证模型的准确性。Make sure you verify the model's accuracy before deploying into your application.

下面是一个完整的示例,如果你想要直接从 ONNX 二进制文件转换。Below is a full example if you want to convert directly from an ONNX binary file.

from winmltools.utils import convert_float_to_float16
from winmltools.utils import load_model, save_model
onnx_model = load_model('model.onnx')
new_onnx_model = convert_float_to_float16(onnx_model)
save_model(new_onnx_model, 'model_fp16.onnx')

使用help(winmltools.utils.convert_float_to_float16),可以找到有关此工具的更多详细信息。With help(winmltools.utils.convert_float_to_float16), you can find more details about this tool. 浮点 16 WinMLTools 中的目前仅符合IEEE 754 浮点位标准 (2008)The floating point 16 in WinMLTools currently only complies with IEEE 754 floating point standard (2008).

后培训权重量化Post-training weight quantization

WinMLTools 还支持模型表示为 8 位整数表示形式的浮动点 32 中的压缩。WinMLTools also supports the compression of models represented in floating point 32 into 8-bit integer representations. 这可能会产生最多 75%,具体取决于模型的磁盘占用空间减少。This could yield a disk footprint reduction of up to 75% depending on the model. 通过一种称为后训练技术来完成这种减少权重的量化,其中对模型进行分析和存储的 tensor 权重从 32 位浮点型数据减少到 8 位数据。This reduction is done via a technique called post-training weight quantization, where the model is analyzed and the stored tensor weights are reduced from 32-bit floating point data into 8-bit data.

备注

定型权重量化后可能会导致丢失在生成的模型的准确性。Post-training weight quantization may result in loss of accuracy in the resulting model. 请确保部署到你的应用程序之前验证模型的准确性。Make sure you verify the model's accuracy before deploying into your application.

下面是演示如何直接从 ONNX 二进制文件将转换的完整示例。The following is a complete example that demonstrates how to convert directly from an ONNX binary file.

import winmltools

model = winmltools.load_model('model.onnx')
packed_model = winmltools.quantize(model, per_channel=True, nbits=8, use_dequantize_linear=True)
winmltools.save_model(packed_model, 'quantized.onnx')

下面是一些有关的输入参数的信息quantize:Here is some information about the input parameters to quantize:

  • per_channel:如果设置为True,这将以线性方式量化每个初始化 tensor [n、 c、 h、 w] 格式的每个通道。per_channel: If set to True, this will linearly quantize each channel for each initialized tensor in [n,c,h,w] format. 默认情况下,此参数设置为TrueBy default, this parameter is set to True.
  • nbits:比特来表示量化的值数。nbits: The number of bits to represent quantized values. 目前支持仅为 8 位。Currently only 8 bits is supported.
  • use_dequantize_linear:如果设置为True,这将以线性方式 dequantize Conv 中的操作员 [n、 c、 h、 w] 格式的初始化 tensors 中的每个通道。use_dequantize_linear: If set to True, this will linearly dequantize each channel in initialized tensors for Conv operators in [n,c,h,w] format. 默认情况下,此设置为TrueBy default, this is set to True.

创建自定义 ONNX 运算符Create custom ONNX operators

当从 Keras 或 Core ML 模型转换,可以编写自定义的运算符函数嵌入自定义运算符到 ONNX 关系图。When converting from a Keras or a Core ML model, you can write a custom operator function to embed custom operators into the ONNX graph. 转换过程中,转换器调用你的函数转换 Keras 层或向 ONNX 操作员,Core ML LayerParameter,然后它将运算符节点连接到整个图。During the conversion, the converter invokes your function to translate the Keras layer or the Core ML LayerParameter to an ONNX operator, and then it connects the operator node into the whole graph.

  1. 创建 ONNX 子图构建的自定义函数。Create the custom function for the ONNX sub-graph building.
  2. 调用winmltools.convert_keraswinmltools.convert_coreml与自定义函数的自定义层名称的映射。Call winmltools.convert_keras or winmltools.convert_coreml with the map of the custom layer name to the custom function.
  3. 如果适用,实现推理运行时的自定义层。If applicable, implement the custom layer for the inference runtime.

下面的示例演示在 Keras 的工作方式。The following example shows how it works in Keras.

# Define the activation layer.
class ParametricSoftplus(keras.layers.Layer):
    def __init__(self, alpha, beta, **kwargs):
    ...
    ...
    ...

# Create the convert function.
def convert_userPSoftplusLayer(scope, operator, container):
      return container.add_node('ParametricSoftplus', operator.input_full_names, operator.output_full_names,
        op_version=1, alpha=operator.original_operator.alpha, beta=operator.original_operator.beta)

winmltools.convert_keras(keras_model, 7,
    custom_conversion_functions={ParametricSoftplus: convert_userPSoftplusLayer })

备注

使用以下资源可获取有关 Windows ML 的帮助:Use the following resources for help with Windows ML:

  • 若要提出或回答有关 Windows ML 的技术问题,请在 Stack Overflow 上使用 windows-machine-learning 标记。To ask or answer technical questions about Windows ML, please use the windows-machine-learning tag on Stack Overflow.
  • 若要报告 bug,请在 GitHub 上提交问题。To report a bug, please file an issue on our GitHub.
  • 若要请求一项功能,请访问 Windows 开发人员反馈To request a feature, please head over to Windows Developer Feedback.