教程:创建 Windows 机器学习 UWP 应用程序 (C#)Tutorial: Create a Windows Machine Learning UWP application (C#)

在本教程中, 我们将构建一个简单的通用 Windows 平台应用程序, 该应用程序使用经过训练的机器学习模型来识别用户绘制的数字。In this tutorial, we'll build a simple Universal Windows Platform application that uses a trained machine learning model to recognize a numeric digit drawn by the user. 本教程主要介绍如何在 UWP 应用程序中加载和使用 Windows ML。This tutorial primarily focuses on how to load and use Windows ML in your UWP application.

以下视频演示了本教程所基于的示例。The following video walks through the sample that this tutorial is based on.


如果只想查看完成的教程的代码, 可以在WinML GitHub 存储库中找到它。If you'd prefer to simply look at the code of the finished tutorial, you can find it on the WinML GitHub repository. 此外, 还可以在 C++/cx中使用。It's also available in C++/CX.

先决条件Prerequisites

1.在 Visual Studio 中打开项目1. Open the project in Visual Studio

后从 GitHub 下载项目后,启动 Visual Studio 并打开 MNIST_Demo.sln 文件 (它应该位于 <存储库路径>\Windows-Machine-Learning\Samples\MNIST\Tutorial\cs)。Once you've downloaded the project from GitHub, launch Visual Studio and open the MNIST_Demo.sln file (it should be located at <Path to repo>\Windows-Machine-Learning\Samples\MNIST\Tutorial\cs). 如果该解决方案显示为 "不可用", 则需要在解决方案资源管理器中右键单击该项目, 然后选择 "重新加载项目"。If the solution is shown as unavailable, you'll need to right-click the project in the Solution Explorer and select Reload Project.

我们提供了已实现 XAML 控件和事件的模板,包括:We've provided a template with implemented XAML controls and events, including:

  • 一个 InkCanvas,用于绘制数字。An InkCanvas to draw the digit.
  • 按钮,用于解释数字并清除画布。Buttons to interpret the digit and clear the canvas.
  • 用于将InkCanvas输出转换为VideoFrame的帮助程序例程。Helper routines to convert the InkCanvas output to a VideoFrame.

解决方案资源管理器中, 该项目有三个主要代码文件:Inside the Solution Explorer, the project has three main code files:

  • MainPage -所有 xaml 代码, 用于创建InkCanvas、按钮和标签的 UI。MainPage.xaml - All of our XAML code to create the UI for the InkCanvas, buttons, and labels.
  • MainPage.xaml.cs -应用程序代码所在的位置。MainPage.xaml.cs - Where our application code lives.
  • Helper.cs -用于裁剪和转换图像格式的帮助程序例程。Helper.cs - Helper routines to crop and convert image formats.

包含项目文件的 Visual Studio 解决方案资源管理器

2.生成并运行项目2. Build and run the project

在 Visual Studio 工具栏中, 将 "解决方案平台" 更改为 " x64 ", 在本地计算机上运行项目 (如果设备为64位) 或x86 (如果它是 32)。In the Visual Studio toolbar, change the Solution Platform to x64 to run the project on your local machine if your device is 64-bit, or x86 if it's 32-bit. (可以签入 Windows 设置应用程序:系统 > > 系统类型 > 设备规格。)(You can check in the Windows Settings app: System > About > Device specifications > System type.)

若要运行项目, 请单击工具栏上的 "启动调试" 按钮, 或按F5To run the project, click the Start Debugging button on the toolbar, or press F5. 应用程序应显示一个InkCanvas , 用户可在其中编写一个数字、一个可识别的按钮来解释该数字、一个空标签字段 (其中解释的数字将显示为文本) 以及一个明文数字按钮以清除InkCanvasThe application should show an InkCanvas where users can write a digit, a Recognize button to interpret the number, an empty label field where the interpreted digit will be displayed as text, and a Clear Digit button to clear the InkCanvas.

应用程序的屏幕截图

备注

如果项目不生成, 可能需要更改项目的部署目标版本。If the project won't build, you might need to change the project's deployment target version. 右键单击 "解决方案资源管理器中的项目, 然后选择"属性"。Right-click the project in the Solution Explorer and select Properties. 在 "应用程序" 选项卡中, 设置目标版本最小版本以与 OS 和 SDK 匹配。In the Application tab, set the Target version and Min version to match your OS and SDK.

备注

如果收到有关应用程序已安装的警告, 只需选择 "是" 以继续进行部署。If you get a warning that the application is already installed, just select Yes to continue with deployment. 如果 Visual Studio 仍然不起作用, 可能需要将其关闭, 然后重新打开。You may need to close Visual Studio and re-open if it still doesn't work.

3.下载模型3. Download a model

接下来, 让我们获取要添加到应用程序中的机器学习模型。Next, let's get a machine learning model to add to our application. 对于本教程, 我们将使用 MNIST (CNTK) Microsoft Cognitive Toolkit训练的预先训练的模型, 并将其导出到 ONNX 格式For this tutorial, we'll use a pre-trained MNIST model that was trained with the Microsoft Cognitive Toolkit (CNTK) and exported to ONNX format.

"资产" 文件夹中已包含 MNIST 模型, 你需要将其作为现有项添加到应用程序。The MNIST model has already been included in your Assets folder, and you will need to add it to your application as an existing item. 你还可以从 GitHub 上的 ONNX Model Zoo 下载经过预先训练的模型。You can also download the pre-trained model from the ONNX Model Zoo on GitHub.

4.添加模型4. Add the model

右键单击 "解决方案资源管理器中的"资产"文件夹, 然后选择"添加 > 现有项"。Right click on the Assets folder in the Solution Explorer, and select Add > Existing Item. 将文件选取器指向 ONNX 模型所在的位置, 然后单击 "添加"。Point the file picker to the location of your ONNX model, and click Add.

该项目现在应该有两个新文件:The project should now have two new files:

  • mnist. onnx -定型模型。mnist.onnx - Your trained model.
  • mnist.cs -Windows ML 生成的代码。mnist.cs - The Windows ML-generated code.

包含新文件的解决方案资源管理器

若要确保在编译应用程序时模型生成, 请右键单击mnist文件, 然后选择 "属性"。To make sure the model builds when we compile our application, right click on the mnist.onnx file, and select Properties. 对于版本操作,选择内容For Build Action, select Content.

现在, 让我们看看mnist.cs文件中新生成的代码。Now, let's take a look at the newly generated code in the mnist.cs file. 我们有三个类:We have three classes:

  • mnistModel创建机器学习模型表示形式, 在系统默认设备上创建一个会话, 将特定的输入和输出绑定到该模型, 并以异步方式对模型进行评估。mnistModel creates the machine learning model representation, creates a session on the system default device, binds the specific inputs and outputs to the model, and evaluates the model asynchronously.
  • mnistInput初始化模型所需的输入类型。mnistInput initializes the input types that the model expects. 在这种情况下, 输入需要ImageFeatureValueIn this case, the input expects an ImageFeatureValue.
  • mnistOutput初始化模型将输出的类型。mnistOutput initializes the types that the model will output. 在这种情况下, 输出将为TensorFloat类型的名为Plus214_Output_0的列表。In this case, the output will be a list called Plus214_Output_0 of type TensorFloat.

现在我们将使用这些类在我们的项目中加载、绑定并评估模型。We'll now use these classes to load, bind, and evaluate the model in our project.

5.加载、绑定和评估模型5. Load, bind, and evaluate the model

对于 Windows ML 应用程序, 我们要遵循的模式如下:计算 > 绑定 > 负载。For Windows ML applications, the pattern we want to follow is: Load > Bind > Evaluate.

  1. 加载机器学习模型。Load the machine learning model.
  2. 将输入和输出绑定到模型。Bind inputs and outputs to the model.
  3. 评估模型并查看结果。Evaluate the model and view results.

我们将使用在mnist.cs中生成的接口代码来加载、绑定和评估应用程序中的模型。We'll use the interface code generated in mnist.cs to load, bind, and evaluate the model in our application.

首先, 在MainPage.xaml.cs中, 让我们实例化模型、输入和输出。First, in MainPage.xaml.cs, let's instantiate the model, inputs, and outputs. 将以下成员变量添加到MainPage类:Add the following member variables to the MainPage class:

private mnistModel ModelGen;
private mnistInput ModelInput = new mnistInput();
private mnistOutput ModelOutput;

然后, 在LoadModelAsync中, 将加载模型。Then, in LoadModelAsync, we'll load the model. 应在使用模型的任何方法之前 (即, 在MainPage加载事件上、在OnNavigatedTo重写中, 或在调用recognizeButton_Click之前的任意位置) 调用此方法。This method should be called before we use any of the model's methods (that is, on MainPage's Loaded event, at an OnNavigatedTo override, or anywhere before recognizeButton_Click is called). MnistModel类表示 MNIST 模型, 并在系统默认设备上创建会话。The mnistModel class represents the MNIST model and creates the session on the system default device. 若要加载该模型, 我们需要调用CreateFromStreamAsync方法, 并将 ONNX 文件作为参数传入。To load the model, we call the CreateFromStreamAsync method, passing in the ONNX file as the parameter.

private async Task LoadModelAsync()
{
    // Load a machine learning model
    StorageFile modelFile = await StorageFile.GetFileFromApplicationUriAsync(new Uri($"ms-appx:///Assets/mnist.onnx"));
    ModelGen = await mnistModel.CreateFromStreamAsync(modelFile as IRandomAccessStreamReference);
}

备注

如果在IRandomAccessStreamReference下出现红色下划线, 则需要包括其命名空间。If you get red underlines under IRandomAccessStreamReference, you need to include its namespace. 将光标置于该光标上方, 按Ctrl +。Put your cursor over it, press Ctrl + . 并在下拉菜单中选择 "使用"。and select using Windows.Storage.Streams from the drop-down menu.

接下来,我们要将输入和输出绑定到模型。Next, we want to bind our inputs and outputs to the model. 生成的代码还包括mnistInputmnistOutput包装器类。The generated code also includes mnistInput and mnistOutput wrapper classes. MnistInput类表示模型的预期输入, mnistOutput类表示模型的预期输出。The mnistInput class represents the model's expected inputs, and the mnistOutput class represents the model's expected outputs.

若要初始化模型的输入对象, 请调用mnistInput类构造函数并传入应用程序数据, 并确保输入数据与模型所需的输入类型匹配。To initialize the model's input object, call the mnistInput class constructor, passing in your application data, and make sure that your input data matches the input type that your model expects. MnistInput类需要ImageFeatureValue, 因此我们使用 helper 方法获取输入的ImageFeatureValueThe mnistInput class expects an ImageFeatureValue, so we use a helper method to get an ImageFeatureValue for the input.

使用我们在helper.cs中包含的帮助器函数, 将复制InkCanvas的内容, 将其转换为类型ImageFeatureValue, 并将其绑定到模型。Using our included helper functions in helper.cs, we will copy the contents of the InkCanvas, convert it to type ImageFeatureValue, and bind it to our model.

private async void recognizeButton_Click(object sender, RoutedEventArgs e)
{
    // Bind model input with contents from InkCanvas
    VideoFrame vf = await helper.GetHandWrittenImage(inkGrid);
    ModelInput.Input3 = ImageFeatureValue.CreateFromVideoFrame(vf);
}

对于输出, 我们只需用指定输入调用EvaluateAsyncFor output, we simply call EvaluateAsync with the specified input. 初始化输入后, 调用模型的EvaluateAsync方法来计算输入数据的模型。Once your inputs are initialized, call the model's EvaluateAsync method to evaluate your model on the input data. EvaluateAsync将输入和输出绑定到模型对象, 并对输入计算模型。EvaluateAsync binds your inputs and outputs to the model object and evaluates the model on the inputs.

由于模型返回一个输出 tensor, 我们将首先要将其转换为友好的数据类型, 然后分析返回的列表, 以确定哪一个数字的概率最高并显示该数字。Since the model returns an output tensor, we'll first want to convert it to a friendly datatype, and then parse the returned list to determine which digit had the highest probability and display that one.

private async void recognizeButton_Click(object sender, RoutedEventArgs e)
{
    // Bind model input with contents from InkCanvas
    VideoFrame vf = await helper.GetHandWrittenImage(inkGrid);
    ModelInput.Input3 = ImageFeatureValue.CreateFromVideoFrame(vf);

    // Evaluate the model
    ModelOutput = await ModelGen.EvaluateAsync(ModelInput);

    // Convert output to datatype
    IReadOnlyList<float> vectorImage = ModelOutput.Plus214_Output_0.GetAsVectorView();
    IList<float> imageList = vectorImage.ToList();

    // Query to check for highest probability digit
    var maxIndex = imageList.IndexOf(imageList.Max());

    // Display the results
    numberLabel.Text = maxIndex.ToString();
}

最后, 我们需要清除InkCanvas以允许用户绘制另一个数字。Finally, we'll want to clear out the InkCanvas to allow users to draw another number.

private void clearButton_Click(object sender, RoutedEventArgs e)
{
    inkCanvas.InkPresenter.StrokeContainer.Clear();
    numberLabel.Text = "";
}

6.启动应用程序6. Launch the application

生成并启动应用程序后 (按F5), 我们将能够识别在InkCanvas上绘制的数字。Once we build and launch the application (press F5), we'll be able to recognize a number drawn on the InkCanvas.

完成应用程序

就是这样, 你已经创建了第一个 Windows ML 应用程序!That's it - you've made your first Windows ML application! 有关演示如何使用 Windows ML 的更多示例, 请查看 GitHub 上的Windows 机器学习存储库。For more samples that demonstrate how to use Windows ML, check out our Windows-Machine-Learning repo on GitHub.

备注

使用以下资源可获取有关 Windows ML 的帮助:Use the following resources for help with Windows ML:

  • 若要提出或回答有关 Windows ML 的技术问题,请在 Stack Overflow 上使用 windows-machine-learning 标记。To ask or answer technical questions about Windows ML, please use the windows-machine-learning tag on Stack Overflow.
  • 若要报告 bug,请在 GitHub 上提交问题。To report a bug, please file an issue on our GitHub.
  • 若要请求一项功能,请访问 Windows 开发人员反馈To request a feature, please head over to Windows Developer Feedback.