Tutorial: Analyze sentiment of movie reviews using a pre-trained TensorFlow model in ML.NET

This tutorial shows you how to use a pre-trained TensorFlow model to classify sentiment in website comments. The binary sentiment classifier is a C# console application developed using Visual Studio.

The TensorFlow model used in this tutorial was trained using movie reviews from the IMDB database. Once you have finished developing the application, you will be able to supply movie review text and the application will tell you whether the review has positive or negative sentiment.

In this tutorial, you learn how to:

  • Load a pre-trained TensorFlow model
  • Transform website comment text into features suitable for the model
  • Use the model to make a prediction

You can find the source code for this tutorial at the dotnet/samples repository.

Prerequisites

Setup

Create the application

  1. Create a .NET Core Console Application called "TextClassificationTF".

  2. Create a directory named Data in your project to save your data set files.

  3. Install the Microsoft.ML NuGet Package:

    In Solution Explorer, right-click on your project and select Manage NuGet Packages. Choose "nuget.org" as the package source, and then select the Browse tab. Search for Microsoft.ML, select the package you want, and then select the Install button. Proceed with the installation by agreeing to the license terms for the package you choose. Repeat these steps for Microsoft.ML.TensorFlow.

Add the TensorFlow model to the project

Note

The model for this tutorial is from the dotnet/machinelearning-testdata GitHub repo. The model is in TensorFlow SavedModel format.

  1. Download the sentiment_model zip file, and unzip.

    The zip file contains:

    • saved_model.pb: the TensorFlow model itself. The model takes a fixed length (size 600) integer array of features representing the text in an IMDB review string, and outputs two probabilities which sum to 1: the probability that the input review has positive sentiment, and the probability that the input review has negative sentiment.
    • imdb_word_index.csv: a mapping from individual words to an integer value. The mapping is used to generate the input features for the TensorFlow model.
  2. Copy the contents of the innermost sentiment_model directory into your TextClassificationTF project sentiment_model directory. This directory contains the model and additional support files needed for this tutorial, as shown in the following image:

    sentiment_model directory contents

  3. In Solution Explorer, right-click each of the files in the sentiment_model directory and subdirectory and select Properties. Under Advanced, change the value of Copy to Output Directory to Copy if newer.

Add using statements and global variables

  1. Add the following additional using statements to the top of the Program.cs file:

    using System;
    using System.IO;
    using System.Collections.Generic;
    using Microsoft.ML;
    using Microsoft.ML.Data;
    using Microsoft.ML.Transforms;
    
  2. Create two global variables right above the Main method to hold the saved model file path, and the feature vector length.

    public const int FeatureLength = 600;
    static readonly string _modelPath = Path.Combine(Environment.CurrentDirectory, "sentiment_model");
    
    • _modelPath is the file path of the trained model.
    • FeatureLength is the length of the integer feature array that the model is expecting.

Model the data

Movie reviews are free form text. Your application converts the text into the input format expected by the model in a number of discrete stages.

The first is to split the text into separate words and use the provided mapping file to map each word onto an integer encoding. The result of this transformation is a variable length integer array with a length corresponding to the number of words in the sentence.

Property Value Type
ReviewText this film is really good string
VariableLengthFeatures 14,22,9,66,78,... int[]

The variable length feature array is then resized to a fixed length of 600. This is the length that the TensorFlow model expects.

Property Value Type
ReviewText this film is really good string
VariableLengthFeatures 14,22,9,66,78,... int[]
Features 14,22,9,66,78,... int[600]
  1. Create a class for your input data, after the Main method:

    /// <summary>
    /// Class to hold original sentiment data.
    /// </summary>
    public class MovieReview
    {
        public string ReviewText { get; set; }
    }
    

    The input data class, MovieReview, has a string for user comments (ReviewText).

  2. Create a class for the variable length features, after the Main method:

    /// <summary>
    /// Class to hold the variable length feature vector. Used to define the
    /// column names used as input to the custom mapping action.
    /// </summary>
    public class VariableLength
    {
        /// <summary>
        /// This is a variable length vector designated by VectorType attribute.
        /// Variable length vectors are produced by applying operations such as 'TokenizeWords' on strings
        /// resulting in vectors of tokens of variable lengths.
        /// </summary>
        [VectorType]
        public int[] VariableLengthFeatures { get; set; }
    }
    

    The VariableLengthFeatures property has a VectorType attribute to designate it as a vector. All of the vector elements must be the same type. In data sets with a large number of columns, loading multiple columns as a single vector reduces the number of data passes when you apply data transformations.

    This class is used in the ResizeFeatures action. The names of its properties (in this case only one) are used to indicate which columns in the DataView can be used as the input to the custom mapping action.

  3. Create a class for the fixed length features, after the Main method:

    /// <summary>
    /// Class to hold the fixed length feature vector. Used to define the
    /// column names used as output from the custom mapping action,
    /// </summary>
    public class FixedLength
    {
        /// <summary>
        /// This is a fixed length vector designated by VectorType attribute.
        /// </summary>
        [VectorType(FeatureLength)]
        public int[] Features { get; set; }
    }
    

    This class is used in the ResizeFeatures action. The names of its properties (in this case only one) are used to indicate which columns in the DataView can be used as the output of the custom mapping action.

    Note that the name of the property Features is determined by the TensorFlow model. You cannot change this property name.

  4. Create a class for the prediction after the Main method:

    /// <summary>
    /// Class to contain the output values from the transformation.
    /// </summary>
    public class MovieReviewSentimentPrediction
    {
        [VectorType(2)]
        public float[] Prediction { get; set; }
    }
    

    MovieReviewSentimentPrediction is the prediction class used after the model training. MovieReviewSentimentPrediction has a single float array (Prediction) and a VectorType attribute.

Create the MLContext, lookup dictionary, and action to resize features

The MLContext class is a starting point for all ML.NET operations. Initializing mlContext creates a new ML.NET environment that can be shared across the model creation workflow objects. It's similar, conceptually, to DBContext in Entity Framework.

  1. Replace the Console.WriteLine("Hello World!") line in the Main method with the following code to declare and initialize the mlContext variable:

    MLContext mlContext = new MLContext();
    
  2. Create a dictionary to encode words as integers by using the LoadFromTextFile method to load mapping data from a file, as seen in the following table:

    Word Index
    kids 362
    want 181
    wrong 355
    effects 302
    feeling 547

    Add the code below to create the lookup map:

    var lookupMap = mlContext.Data.LoadFromTextFile(Path.Combine(_modelPath, "imdb_word_index.csv"),
        columns: new[]
           {
                new TextLoader.Column("Words", DataKind.String, 0),
                new TextLoader.Column("Ids", DataKind.Int32, 1),
           },
        separatorChar: ','
       );
    
  3. Add an Action to resize the variable length word integer array to an integer array of fixed size, with the next lines of code:

    Action<VariableLength, FixedLength> ResizeFeaturesAction = (s, f) =>
    {
        var features = s.VariableLengthFeatures;
        Array.Resize(ref features, FeatureLength);
        f.Features = features;
    };
    

Load the pre-trained TensorFlow model

  1. Add code to load the TensorFlow model:

    TensorFlowModel tensorFlowModel = mlContext.Model.LoadTensorFlowModel(_modelPath);
    

    Once the model is loaded, you can extract its input and output schema. The schemas are displayed for interest and learning only. You do not need this code for the final application to function:

    DataViewSchema schema = tensorFlowModel.GetModelSchema();
    Console.WriteLine(" =============== TensorFlow Model Schema =============== ");
    var featuresType = (VectorDataViewType)schema["Features"].Type;
    Console.WriteLine($"Name: Features, Type: {featuresType.ItemType.RawType}, Size: ({featuresType.Dimensions[0]})");
    var predictionType = (VectorDataViewType)schema["Prediction/Softmax"].Type;
    Console.WriteLine($"Name: Prediction/Softmax, Type: {predictionType.ItemType.RawType}, Size: ({predictionType.Dimensions[0]})");
    
    

    The input schema is the fixed-length array of integer encoded words. The output schema is a float array of probabilities indicating whether a review's sentiment is negative, or positive . These values sum to 1, as the probability of being positive is the complement of the probability of the sentiment being negative.

Create the ML.NET pipeline

  1. Create the pipeline and split the input text into words using TokenizeIntoWords transform to break the text into words as the next line of code:

    IEstimator<ITransformer> pipeline =
        // Split the text into individual words
        mlContext.Transforms.Text.TokenizeIntoWords("TokenizedWords", "ReviewText")
    

    The TokenizeIntoWords transform uses spaces to parse the text/string into words. It creates a new column and splits each input string to a vector of substrings based on the user-defined separator.

  2. Map the words onto their integer encoding using the lookup table that you declared above:

    // Map each word to an integer value. The array of integer makes up the input features.
    .Append(mlContext.Transforms.Conversion.MapValue("VariableLengthFeatures", lookupMap,
        lookupMap.Schema["Words"], lookupMap.Schema["Ids"], "TokenizedWords"))
    
  3. Resize the variable length integer encodings to the fixed-length one required by the model:

    // Resize variable length vector to fixed length vector.
    .Append(mlContext.Transforms.CustomMapping(ResizeFeaturesAction, "Resize"))
    
  4. Classify the input with the loaded TensorFlow model:

    // Passes the data to TensorFlow for scoring
    .Append(tensorFlowModel.ScoreTensorFlowModel("Prediction/Softmax", "Features"))
    

    The TensorFlow model output is called Prediction/Softmax. Note that the name Prediction/Softmax is determined by the TensorFlow model. You cannot change this name.

  5. Create a new column for the output prediction:

    // Retrieves the 'Prediction' from TensorFlow and and copies to a column
    .Append(mlContext.Transforms.CopyColumns("Prediction", "Prediction/Softmax"));
    

    You need to copy the Prediction/Softmax column into one with a name that can be used as a property in a C# class: Prediction. The / character is not allowed in a C# property name.

Create the ML.NET model from the pipeline

  1. Add the code to create the model from the pipeline:

    // Create an executable model from the estimator pipeline
    IDataView dataView = mlContext.Data.LoadFromEnumerable(new List<MovieReview>());
    ITransformer model = pipeline.Fit(dataView);
    

    An ML.NET model is created from the chain of estimators in the pipeline by calling the Fit method. In this case, we are not fitting any data to create the model, as the TensorFlow model has already been previously trained. We supply an empty data view object to satisfy the requirements of the Fit method.

Use the model to make a prediction

  1. Add the PredictSentiment method below the Main method:

    public static void PredictSentiment(MLContext mlContext, ITransformer model)
    {
    
    }
    
  2. Add the following code to create the PredictionEngine as the first line in the PredictSentiment() method:

    var engine = mlContext.Model.CreatePredictionEngine<MovieReview, MovieReviewSentimentPrediction>(model);
    

    The PredictionEngine is a convenience API, which allows you to perform a prediction on a single instance of data. PredictionEngine is not thread-safe. It's acceptable to use in single-threaded or prototype environments. For improved performance and thread safety in production environments, use the PredictionEnginePool service, which creates an ObjectPool of PredictionEngine objects for use throughout your application. See this guide on how to use PredictionEnginePool in an ASP.NET Core Web API

    Note

    PredictionEnginePool service extension is currently in preview.

  3. Add a comment to test the trained model's prediction in the Predict() method by creating an instance of MovieReview:

    var review = new MovieReview()
    {
        ReviewText = "this film is really good"
    };
    
  4. Pass the test comment data to the Prediction Engine by adding the next lines of code in the PredictSentiment() method:

    var sentimentPrediction = engine.Predict(review);
    
  5. The Predict() function makes a prediction on a single row of data:

    Property Value Type
    Prediction [0.5459937, 0.454006255] float[]
  6. Display sentiment prediction using the following code:

    Console.WriteLine("Number of classes: {0}", sentimentPrediction.Prediction.Length);
    Console.WriteLine("Is sentiment/review positive? {0}", sentimentPrediction.Prediction[1] > 0.5 ? "Yes." : "No.");
    
  7. Add a call to PredictSentiment at the end of the Main method:

    PredictSentiment(mlContext, model);
    

Results

Build and run your application.

Your results should be similar to the following. During processing, messages are displayed. You may see warnings, or processing messages. These messages have been removed from the following results for clarity.

Number of classes: 2
Is sentiment/review positive ? Yes

Congratulations! You've now successfully built a machine learning model for classifying and predicting messages sentiment by reusing a pre-trained TensorFlow model in ML.NET.

You can find the source code for this tutorial at the dotnet/samples repository.

In this tutorial, you learned how to:

  • Load a pre-trained TensorFlow model
  • Transform website comment text into features suitable for the model
  • Use the model to make a prediction