Deploy a model to Azure Functions

Learn how to deploy a pre-trained ML.NET machine learning model for predictions over HTTP through an Azure Functions serverless environment.

Note

This sample runs a preview version of the PredictionEnginePool service.

Prerequisites

Azure Functions sample overview

This sample is a C# HTTP Trigger Azure Functions application that uses a pretrained binary classification model to categorize the sentiment of text as positive or negative. Azure Functions provides an easy way to run small pieces of code at scale on a managed serverless environment in the cloud. The code for this sample can be found on the dotnet/machinelearning-samples repository on GitHub.

Create Azure Functions project

  1. Open Visual Studio 2017. Select File > New > Project from the menu bar. In the New Project dialog, select the Visual C# node followed by the Cloud node. Then select the Azure Functions project template. In the Name text box, type "SentimentAnalysisFunctionsApp" and then select the OK button.

  2. In the New Project dialog, open the dropdown above the project options and select Azure Functions v2 (.NET Core). Then, select the Http trigger project and then select the OK button.

  3. Create a directory named MLModels in your project to save your model:

    In Solution Explorer, right-click on your project and select Add > New Folder. Type "MLModels" and hit Enter.

  4. Install the Microsoft.ML NuGet Package version 1.3.1:

    In Solution Explorer, right-click on your project and select Manage NuGet Packages. Choose "nuget.org" as the Package source, select the Browse tab, search for Microsoft.ML, select that package in the list, and select the Install button. Select the OK button on the Preview Changes dialog and then select the I Accept button on the License Acceptance dialog if you agree with the license terms for the packages listed.

  5. Install the Microsoft.Azure.Functions.Extensions NuGet Package:

    In Solution Explorer, right-click on your project and select Manage NuGet Packages. Choose "nuget.org" as the Package source, select the Browse tab, search for Microsoft.Azure.Functions.Extensions, select that package in the list, and select the Install button. Select the OK button on the Preview Changes dialog and then select the I Accept button on the License Acceptance dialog if you agree with the license terms for the packages listed.

  6. Install the Microsoft.Extensions.ML NuGet Package version 0.15.1:

    In Solution Explorer, right-click on your project and select Manage NuGet Packages. Choose "nuget.org" as the Package source, select the Browse tab, search for Microsoft.Extensions.ML, select that package in the list, and select the Install button. Select the OK button on the Preview Changes dialog and then select the I Accept button on the License Acceptance dialog if you agree with the license terms for the packages listed.

  7. Install the Microsoft.NET.Sdk.Functions NuGet Package version 1.0.28+:

    In Solution Explorer, right-click on your project and select Manage NuGet Packages. Choose "nuget.org" as the Package source, select the Installed tab, search for Microsoft.NET.Sdk.Functions, select that package in the list, select 1.0.28 or later from the Version dropdown, and select the Update button. Select the OK button on the Preview Changes dialog and then select the I Accept button on the License Acceptance dialog if you agree with the license terms for the packages listed.

Add pre-trained model to project

  1. Copy your pre-built model to the MLModels folder.
  2. In Solution Explorer, right-click your pre-built model file and select Properties. Under Advanced, change the value of Copy to Output Directory to Copy if newer.

Create Azure Function to analyze sentiment

Create a class to predict sentiment. Add a new class to your project:

  1. In Solution Explorer, right-click the project, and then select Add > New Item.

  2. In the Add New Item dialog box, select Azure Function and change the Name field to AnalyzeSentiment.cs. Then, select the Add button.

  3. In the New Azure Function dialog box, select Http Trigger. Then, select the OK button.

    The AnalyzeSentiment.cs file opens in the code editor. Add the following using statement to the top of AnalyzeSentiment.cs:

    using System;
    using System.IO;
    using System.Threading.Tasks;
    using Microsoft.AspNetCore.Mvc;
    using Microsoft.Azure.WebJobs;
    using Microsoft.Azure.WebJobs.Extensions.Http;
    using Microsoft.AspNetCore.Http;
    using Microsoft.Extensions.Logging;
    using Newtonsoft.Json;
    using Microsoft.Extensions.ML;
    using SentimentAnalysisFunctionsApp.DataModels;
    

    By default, the AnalyzeSentiment class is static. Make sure to remove the static keyword from the class definition.

    public class AnalyzeSentiment
    {
    
    }
    

Create data models

You need to create some classes for your input data and predictions. Add a new class to your project:

  1. Create a directory named DataModels in your project to save your data models: In Solution Explorer, right-click on your project and select Add > New Folder. Type "DataModels" and hit Enter.

  2. In Solution Explorer, right-click the DataModels directory, and then select Add > New Item.

  3. In the Add New Item dialog box, select Class and change the Name field to SentimentData.cs. Then, select the Add button.

    The SentimentData.cs file opens in the code editor. Add the following using statement to the top of SentimentData.cs:

    using Microsoft.ML.Data;
    

    Remove the existing class definition and add the following code to the SentimentData.cs file:

    public class SentimentData
    {
        [LoadColumn(0)]
        public string SentimentText;
    
        [LoadColumn(1)]
        [ColumnName("Label")]
        public bool Sentiment;
    }
    
  4. In Solution Explorer, right-click the DataModels directory, and then select Add > New Item.

  5. In the Add New Item dialog box, select Class and change the Name field to SentimentPrediction.cs. Then, select the Add button. The SentimentPrediction.cs file opens in the code editor. Add the following using statement to the top of SentimentPrediction.cs:

    using Microsoft.ML.Data;
    

    Remove the existing class definition and add the following code to the SentimentPrediction.cs file:

    public class SentimentPrediction : SentimentData
    {
    
        [ColumnName("PredictedLabel")]
        public bool Prediction { get; set; }
    
        public float Probability { get; set; }
    
        public float Score { get; set; }
    }
    

    SentimentPrediction inherits from SentimentData which provides access to the original data in the SentimentText property as well as the output generated by the model.

Register PredictionEnginePool service

To make a single prediction, you have to create a PredictionEngine. PredictionEngine is not thread-safe. Additionally, you have to create an instance of it everywhere it is needed within your application. As your application grows, this process can become unmanageable. For improved performance and thread safety, use a combination of dependency injection and the PredictionEnginePool service, which creates an ObjectPool of PredictionEngine objects for use throughout your application.

The following link provides more information if you want to learn more about dependency injection.

  1. In Solution Explorer, right-click the project, and then select Add > New Item.

  2. In the Add New Item dialog box, select Class and change the Name field to Startup.cs. Then, select the Add button.

  3. Add the following using statements to the top of Startup.cs:

    using System;
    using System.IO;
    using Microsoft.Azure.Functions.Extensions.DependencyInjection;
    using Microsoft.Extensions.ML;
    using SentimentAnalysisFunctionsApp;
    using SentimentAnalysisFunctionsApp.DataModels;
    
  4. Remove the existing code below the using statements and add the following code:

    [assembly: FunctionsStartup(typeof(Startup))]
    namespace SentimentAnalysisFunctionsApp
    {
        public class Startup : FunctionsStartup
        {
    
        }
    }
    
  5. Define variables to store the environment the app is running in and the file path where the model is located inside the Startup class

    private readonly string _environment;
    private readonly string _modelPath;
    
  6. Below that, create a constructor to set the values of the _environment and _modelPath variables. When the application is running locally, the default environment is Development.

    public Startup()
    {
        _environment = Environment.GetEnvironmentVariable("AZURE_FUNCTIONS_ENVIRONMENT");
    
        if (_environment == "Development")
        {
            _modelPath = Path.Combine("MLModels", "sentiment_model.zip");
        }
        else
        {
            string deploymentPath = @"D:\home\site\wwwroot\";
            _modelPath = Path.Combine(deploymentPath, "MLModels", "sentiment_model.zip");
        }
    }
    
  7. Then, add a new method called Configure to register the PredictionEnginePool service below the constructor.

    public override void Configure(IFunctionsHostBuilder builder)
    {
        builder.Services.AddPredictionEnginePool<SentimentData, SentimentPrediction>()
            .FromFile(modelName: "SentimentAnalysisModel", filePath: _modelPath, watchForChanges: true);
    }
    

At a high level, this code initializes the objects and services automatically for later use when requested by the application instead of having to manually do it.

Machine learning models are not static. As new training data becomes available, the model is retrained and redeployed. One way to get the latest version of the model into your application is to redeploy the entire application. However, this introduces application downtime. The PredictionEnginePool service provides a mechanism to reload an updated model without taking your application down.

Set the watchForChanges parameter to true, and the PredictionEnginePool starts a FileSystemWatcher that listens to the file system change notifications and raises events when there is a change to the file. This prompts the PredictionEnginePool to automatically reload the model.

The model is identified by the modelName parameter so that more than one model per application can be reloaded upon change.

Tip

Alternatively, you can use the FromUri method when working with models stored remotely. Rather than watching for file changed events, FromUri polls the remote location for changes. The polling interval defaults to 5 minutes. You can increase or decrease the polling interval based on your application's requirements. In the code sample below, the PredictionEnginePool polls the model stored at the specified URI every minute.

builder.Services.AddPredictionEnginePool<SentimentData, SentimentPrediction>()
  .FromUri(
      modelName: "SentimentAnalysisModel",
      uri:"https://github.com/dotnet/samples/raw/master/machine-learning/models/sentimentanalysis/sentiment_model.zip",
      period: TimeSpan.FromMinutes(1));

Load the model into the function

Insert the following code inside the AnalyzeSentiment class:

private readonly PredictionEnginePool<SentimentData, SentimentPrediction> _predictionEnginePool;

// AnalyzeSentiment class constructor
public AnalyzeSentiment(PredictionEnginePool<SentimentData, SentimentPrediction> predictionEnginePool)
{
    _predictionEnginePool = predictionEnginePool;
}

This code assigns the PredictionEnginePool by passing it to the function's constructor which you get via dependency injection.

Use the model to make predictions

Replace the existing implementation of Run method in AnalyzeSentiment class with the following code:

[FunctionName("AnalyzeSentiment")]
public async Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Function, "post", Route = null)] HttpRequest req,
ILogger log)
{
    log.LogInformation("C# HTTP trigger function processed a request.");

    //Parse HTTP Request Body
    string requestBody = await new StreamReader(req.Body).ReadToEndAsync();
    SentimentData data = JsonConvert.DeserializeObject<SentimentData>(requestBody);

    //Make Prediction
    SentimentPrediction prediction = _predictionEnginePool.Predict(modelName: "SentimentAnalysisModel", example: data);

    //Convert prediction to string
    string sentiment = Convert.ToBoolean(prediction.Prediction) ? "Positive" : "Negative";

    //Return Prediction
    return (ActionResult)new OkObjectResult(sentiment);
}

When the Run method executes, the incoming data from the HTTP request is deserialized and used as input for the PredictionEnginePool. The Predict method is then called to make predictions using the SentimentAnalysisModel registered in the Startup class and returns the results back to the user if successful.

Test locally

Now that everything is set up, it's time to test the application:

  1. Run the application

  2. Open PowerShell and enter the code into the prompt where PORT is the port your application is running on. Typically the port is 7071.

    Invoke-RestMethod "http://localhost:<PORT>/api/AnalyzeSentiment" -Method Post -Body (@{SentimentText="This is a very bad steak"} | ConvertTo-Json) -ContentType "application/json"
    

    If successful, the output should look similar to the text below:

    Negative
    

Congratulations! You have successfully served your model to make predictions over the internet using an Azure Function.

Next Steps