Tutorial: Analyze sentiment of movie reviews using a pre-trained TensorFlow model in ML.NET

Article
11/29/2021

This tutorial shows you how to use a pre-trained TensorFlow model to classify sentiment in website comments. The binary sentiment classifier is a C# console application developed using Visual Studio.

The TensorFlow model used in this tutorial was trained using movie reviews from the IMDB database. Once you have finished developing the application, you will be able to supply movie review text and the application will tell you whether the review has positive or negative sentiment.

In this tutorial, you learn how to:

Load a pre-trained TensorFlow model
Transform website comment text into features suitable for the model
Use the model to make a prediction

You can find the source code for this tutorial at the dotnet/samples repository.

Prerequisites

Visual Studio 2022 with the ".NET Desktop Development" workload installed.

Setup

Create the application

Create a C# Console Application called "TextClassificationTF". Click the Next button.
Choose .NET 6 as the framework to use. Click the Create button.
Create a directory named Data in your project to save your data set files.
Install the Microsoft.ML NuGet Package:

Note

This sample uses the latest stable version of the NuGet packages mentioned unless otherwise stated.

In Solution Explorer, right-click on your project and select Manage NuGet Packages. Choose "nuget.org" as the package source, and then select the Browse tab. Search for Microsoft.ML, select the package you want, and then select the Install button. Proceed with the installation by agreeing to the license terms for the package you choose. Repeat these steps for Microsoft.ML.TensorFlow, Microsoft.ML.SampleUtils and SciSharp.TensorFlow.Redist.

Add the TensorFlow model to the project

Note

The model for this tutorial is from the dotnet/machinelearning-testdata GitHub repo. The model is in TensorFlow SavedModel format.

Download the sentiment_model zip file, and unzip.

The zip file contains:
- saved_model.pb: the TensorFlow model itself. The model takes a fixed length (size 600) integer array of features representing the text in an IMDB review string, and outputs two probabilities which sum to 1: the probability that the input review has positive sentiment, and the probability that the input review has negative sentiment.
- imdb_word_index.csv: a mapping from individual words to an integer value. The mapping is used to generate the input features for the TensorFlow model.
Copy the contents of the innermost sentiment_model directory into your TextClassificationTF project sentiment_model directory. This directory contains the model and additional support files needed for this tutorial, as shown in the following image:
In Solution Explorer, right-click each of the files in the sentiment_model directory and subdirectory and select Properties. Under Advanced, change the value of Copy to Output Directory to Copy if newer.

Add using statements and global variables

Add the following additional using statements to the top of the Program.cs file:

using Microsoft.ML;
using Microsoft.ML.Data;
using Microsoft.ML.Transforms;

Create a global variable right after the using statements to hold the saved model file path.
```
string _modelPath = Path.Combine(Environment.CurrentDirectory, "sentiment_model");
```
- _modelPath is the file path of the trained model.

Model the data

Movie reviews are free form text. Your application converts the text into the input format expected by the model in a number of discrete stages.

The first is to split the text into separate words and use the provided mapping file to map each word onto an integer encoding. The result of this transformation is a variable length integer array with a length corresponding to the number of words in the sentence.

Property	Value	Type
ReviewText	this film is really good	string
VariableLengthFeatures	14,22,9,66,78,...	int[]

The variable length feature array is then resized to a fixed length of 600. This is the length that the TensorFlow model expects.

Property	Value	Type
ReviewText	this film is really good	string
VariableLengthFeatures	14,22,9,66,78,...	int[]
Features	14,22,9,66,78,...	int[600]

Create a class for your input data at the bottom of the Program.cs file:

/// <summary>
/// Class to hold original sentiment data.
/// </summary>
public class MovieReview
{
    public string? ReviewText { get; set; }
}

The input data class, MovieReview, has a string for user comments (ReviewText).

Create a class for the variable length features after the MovieReview class:
```
/// <summary>
/// Class to hold the variable length feature vector. Used to define the
/// column names used as input to the custom mapping action.
/// </summary>
public class VariableLength
{
    /// <summary>
    /// This is a variable length vector designated by VectorType attribute.
    /// Variable length vectors are produced by applying operations such as 'TokenizeWords' on strings
    /// resulting in vectors of tokens of variable lengths.
    /// </summary>
    [VectorType]
    public int[]? VariableLengthFeatures { get; set; }
}
```
The VariableLengthFeatures property has a VectorType attribute to designate it as a vector. All of the vector elements must be the same type. In data sets with a large number of columns, loading multiple columns as a single vector reduces the number of data passes when you apply data transformations.

This class is used in the ResizeFeatures action. The names of its properties (in this case only one) are used to indicate which columns in the DataView can be used as the input to the custom mapping action.
Create a class for the fixed length features, after the VariableLength class:
```
/// <summary>
/// Class to hold the fixed length feature vector. Used to define the
/// column names used as output from the custom mapping action,
/// </summary>
public class FixedLength
{
    /// <summary>
    /// This is a fixed length vector designated by VectorType attribute.
    /// </summary>
    [VectorType(Config.FeatureLength)]
    public int[]? Features { get; set; }
}
```
This class is used in the ResizeFeatures action. The names of its properties (in this case only one) are used to indicate which columns in the DataView can be used as the output of the custom mapping action.

Note that the name of the property Features is determined by the TensorFlow model. You cannot change this property name.
Create a class for the prediction after the FixedLength class:
```
/// <summary>
/// Class to contain the output values from the transformation.
/// </summary>
public class MovieReviewSentimentPrediction
{
    [VectorType(2)]
    public float[]? Prediction { get; set; }
}
```
MovieReviewSentimentPrediction is the prediction class used after the model training. MovieReviewSentimentPrediction has a single float array (Prediction) and a VectorType attribute.
Create another class to hold configuration values, such as the feature vector length:
```
static class Config
{
    public const int FeatureLength = 600;
}
```

Create the MLContext, lookup dictionary, and action to resize features

The MLContext class is a starting point for all ML.NET operations. Initializing mlContext creates a new ML.NET environment that can be shared across the model creation workflow objects. It's similar, conceptually, to DBContext in Entity Framework.

Replace the Console.WriteLine("Hello World!") line with the following code to declare and initialize the mlContext variable:
```
MLContext mlContext = new MLContext();
```

Create a dictionary to encode words as integers by using the LoadFromTextFile method to load mapping data from a file, as seen in the following table:

Word	Index
kids	362
want	181
wrong	355
effects	302
feeling	547

Add the code below to create the lookup map:

var lookupMap = mlContext.Data.LoadFromTextFile(Path.Combine(_modelPath, "imdb_word_index.csv"),
    columns: new[]
        {
            new TextLoader.Column("Words", DataKind.String, 0),
            new TextLoader.Column("Ids", DataKind.Int32, 1),
        },
    separatorChar: ','
    );

Add an Action to resize the variable length word integer array to an integer array of fixed size, with the next lines of code:

Action<VariableLength, FixedLength> ResizeFeaturesAction = (s, f) =>
{
    var features = s.VariableLengthFeatures;
    Array.Resize(ref features, Config.FeatureLength);
    f.Features = features;
};

Load the pre-trained TensorFlow model

Add code to load the TensorFlow model:

TensorFlowModel tensorFlowModel = mlContext.Model.LoadTensorFlowModel(_modelPath);

Once the model is loaded, you can extract its input and output schema. The schemas are displayed for interest and learning only. You do not need this code for the final application to function:

DataViewSchema schema = tensorFlowModel.GetModelSchema();
Console.WriteLine(" =============== TensorFlow Model Schema =============== ");
var featuresType = (VectorDataViewType)schema["Features"].Type;
Console.WriteLine($"Name: Features, Type: {featuresType.ItemType.RawType}, Size: ({featuresType.Dimensions[0]})");
var predictionType = (VectorDataViewType)schema["Prediction/Softmax"].Type;
Console.WriteLine($"Name: Prediction/Softmax, Type: {predictionType.ItemType.RawType}, Size: ({predictionType.Dimensions[0]})");

The input schema is the fixed-length array of integer encoded words. The output schema is a float array of probabilities indicating whether a review's sentiment is negative, or positive . These values sum to 1, as the probability of being positive is the complement of the probability of the sentiment being negative.

Create the ML.NET pipeline

Create the pipeline and split the input text into words using TokenizeIntoWords transform to break the text into words as the next line of code:
```
IEstimator<ITransformer> pipeline =
    // Split the text into individual words
    mlContext.Transforms.Text.TokenizeIntoWords("TokenizedWords", "ReviewText")
```
The TokenizeIntoWords transform uses spaces to parse the text/string into words. It creates a new column and splits each input string to a vector of substrings based on the user-defined separator.

Map the words onto their integer encoding using the lookup table that you declared above:

// Map each word to an integer value. The array of integer makes up the input features.
.Append(mlContext.Transforms.Conversion.MapValue("VariableLengthFeatures", lookupMap,
    lookupMap.Schema["Words"], lookupMap.Schema["Ids"], "TokenizedWords"))

Resize the variable length integer encodings to the fixed-length one required by the model:

// Resize variable length vector to fixed length vector.
.Append(mlContext.Transforms.CustomMapping(ResizeFeaturesAction, "Resize"))

Classify the input with the loaded TensorFlow model:
```
// Passes the data to TensorFlow for scoring
.Append(tensorFlowModel.ScoreTensorFlowModel("Prediction/Softmax", "Features"))
```
The TensorFlow model output is called Prediction/Softmax. Note that the name Prediction/Softmax is determined by the TensorFlow model. You cannot change this name.
Create a new column for the output prediction:
```
// Retrieves the 'Prediction' from TensorFlow and copies to a column
.Append(mlContext.Transforms.CopyColumns("Prediction", "Prediction/Softmax"));
```
You need to copy the Prediction/Softmax column into one with a name that can be used as a property in a C# class: Prediction. The / character is not allowed in a C# property name.

Create the ML.NET model from the pipeline

Add the code to create the model from the pipeline:
```
// Create an executable model from the estimator pipeline
IDataView dataView = mlContext.Data.LoadFromEnumerable(new List<MovieReview>());
ITransformer model = pipeline.Fit(dataView);
```
An ML.NET model is created from the chain of estimators in the pipeline by calling the Fit method. In this case, we are not fitting any data to create the model, as the TensorFlow model has already been previously trained. We supply an empty data view object to satisfy the requirements of the Fit method.

Use the model to make a prediction

Add the PredictSentiment method above the MovieReview class:

void PredictSentiment(MLContext mlContext, ITransformer model)
{

}

Add the following code to create the PredictionEngine as the first line in the PredictSentiment() method:
```
var engine = mlContext.Model.CreatePredictionEngine<MovieReview, MovieReviewSentimentPrediction>(model);
```
The PredictionEngine is a convenience API, which allows you to perform a prediction on a single instance of data. PredictionEngine is not thread-safe. It's acceptable to use in single-threaded or prototype environments. For improved performance and thread safety in production environments, use the PredictionEnginePool service, which creates an ObjectPool of PredictionEngine objects for use throughout your application. See this guide on how to use PredictionEnginePool in an ASP.NET Core Web API.

Note

PredictionEnginePool service extension is currently in preview.
Add a comment to test the trained model's prediction in the Predict() method by creating an instance of MovieReview:
```
var review = new MovieReview()
{
    ReviewText = "this film is really good"
};
```
Pass the test comment data to the Prediction Engine by adding the next lines of code in the PredictSentiment() method:
```
var sentimentPrediction = engine.Predict(review);
```
The Predict() function makes a prediction on a single row of data:

Property Value Type

Prediction [0.5459937, 0.454006255] float[]

Property	Value	Type
Prediction	[0.5459937, 0.454006255]	float[]

Display sentiment prediction using the following code:

Console.WriteLine($"Number of classes: {sentimentPrediction.Prediction?.Length}");
Console.WriteLine($"Is sentiment/review positive? {(sentimentPrediction.Prediction?[1] > 0.5 ? "Yes." : "No.")}");

Add a call to PredictSentiment after calling the Fit() method:
```
PredictSentiment(mlContext, model);
```

Results

Build and run your application.

Your results should be similar to the following. During processing, messages are displayed. You may see warnings, or processing messages. These messages have been removed from the following results for clarity.

Number of classes: 2
Is sentiment/review positive ? Yes

Congratulations! You've now successfully built a machine learning model for classifying and predicting messages sentiment by reusing a pre-trained TensorFlow model in ML.NET.