Tutorial: Analyze sentiment of website comments in a web application using ML.NET Model Builder

11/21/2019
9 min na prečítanie

Learn how to analyze sentiment from comments in real time inside a web application.

This tutorial shows you how to create an ASP.NET Core Razor Pages application that classifies sentiment from website comments in real time.

In this tutorial, you learn how to:

Create an ASP.NET Core Razor Pages application
Prepare and understand the data
Choose a scenario
Load the data
Train the model
Evaluate the model
Use the model for predictions

Note

Model Builder is currently in Preview.

You can find the source code for this tutorial at the dotnet/machinelearning-samples repository.

Pre-requisites

For a list of pre-requisites and installation instructions, visit the Model Builder installation guide.

Create a Razor Pages application

Create a ASP.NET Core Razor Pages Application.
1. Open Visual Studio and select File > New > Project from the menu bar.
2. In the New Project dialog, select the Visual C# node followed by the Web node.
3. Then select the ASP.NET Core Web Application project template.
4. In the Name text box, type "SentimentRazor".
5. Make sure Place solution and project in the same directory is unchecked (VS 2019), or Create directory for solution is checked (VS 2017).
6. Select the OK button.
7. Choose Web Application in the window that displays the different types of ASP.NET Core Projects, and then select the OK button.

Prepare and understand the data

Download Wikipedia detox dataset. When the webpage opens, right-click on the page, select Save As and save the file anywhere on your computer.

Each row in the wikipedia-detox-250-line-data.tsv dataset represents a different review left by a user on Wikipedia. The first column represents the sentiment of the text (0 is non-toxic, 1 is toxic), and the second column represents the comment left by the user. The columns are separated by tabs. The data looks like the following:

Sentiment	SentimentText
1	==RUDE== Dude, you are rude upload that carl picture back, or else.
1	== OK! == IM GOING TO VANDALIZE WILD ONES WIKI THEN!!!
0	I hope this helps.

Choose a scenario

Model Builder wizard in Visual Studio

To train your model, you need to select from the list of available machine learning scenarios provided by Model Builder.

In Solution Explorer, right-click the SentimentRazor project, and select Add > Machine Learning.
For this sample, the scenario is sentiment analysis. In the scenario step of the Model Builder tool, select the Sentiment Analysis scenario.

Load the data

Model Builder accepts data from two sources, a SQL Server database or a local file in csv or tsv format.

In the data step of the Model Builder tool, select File from the data source dropdown.
Select the button next to the Select a file text box and use File Explorer to browse and select the wikipedia-detox-250-line-data.tsv file.
Choose Sentiment in the Column to Predict (Label) dropdown.
Leave the default values for the Input Columns (Features) dropdown.
Select the Train link to move to the next step in the Model Builder tool.

Train the model

The machine learning task used to train the sentiment analysis model in this tutorial is binary classification. During the model training process, Model Builder trains separate models using different binary classification algorithms and settings to find the best performing model for your dataset.

The time required for the model to train is proportionate to the amount of data. Model Builder automatically selects a default value for Time to train (seconds) based on the size of your data source.

Although Model Builder sets the value of Time to train (seconds) to 10 seconds, increase it to 30 seconds. Training for a longer period of time allows Model Builder to explore a larger number of algorithms and combination of parameters in search of the best model.
Select Start Training.

Throughout the training process, progress data is displayed in the Progress section of the train step.
- Status displays the completion status of the training process.
- Best accuracy displays the accuracy of the best performing model found by Model Builder so far. Higher accuracy means the model predicted more correctly on test data.
- Best algorithm displays the name of the best performing algorithm performed found by Model Builder so far.
- Last algorithm displays the name of the algorithm most recently used by Model Builder to train the model.
Once training is complete, select the evaluate link to move to the next step.

Evaluate the model

The result of the training step will be one model that has the best performance. In the evaluate step of the Model Builder tool, the output section will contain the algorithm used by the best-performing model in the Best Model entry along with metrics in Best Model Accuracy. Additionally, a summary table containing the top five models and their metrics is shown.

If you're not satisfied with your accuracy metrics, some easy ways to try to improve model accuracy are to increase the amount of time to train the model or use more data. Otherwise, select the code link to move to the final step in the Model Builder tool.

Add the code to make predictions

Two projects will be created as a result of the training process.

Reference the trained model

In the code step of the Model Builder tool, select Add Projects to add the autogenerated projects to the solution.

The following projects should appear in the Solution Explorer:
- SentimentRazorML.ConsoleApp: A .NET Core Console application that contains the model training and prediction code.
- SentimentRazorML.Model: A .NET Standard class library containing the data models that define the schema of input and output model data as well as the saved version of the best performing model during training.
For this tutorial, only the SentimentRazorML.Model project is used because predictions will be made in the SentimentRazor web application rather than in the console. Although the SentimentRazorML.ConsoleApp won't be used for scoring, it can be used to retrain the model using new data at a later time. Retraining is outside the scope of this tutorial though.

Configure the PredictionEngine pool

To make a single prediction, you have to create a PredictionEngine<TSrc,TDst>. PredictionEngine<TSrc,TDst> is not thread-safe. Additionally, you have to create an instance of it everywhere it's needed within your application. As your application grows, this process can become unmanageable. For improved performance and thread safety, use a combination of dependency injection and the PredictionEnginePool service, which creates an ObjectPool<T> of PredictionEngine<TSrc,TDst> objects for use throughout your application.

Install the Microsoft.Extensions.ML NuGet package:
1. In Solution Explorer, right-click the project and select Manage NuGet Packages.
2. Choose "nuget.org" as the Package source.
3. Select the Browse tab and search for Microsoft.Extensions.ML.
4. Select the package in the list, and select the Install button.
5. Select the OK button on the Preview Changes dialog
6. Select the I Accept button on the License Acceptance dialog if you agree with the license terms for the packages listed.
Open the Startup.cs file in the SentimentRazor project.
Add the following using statements to reference the Microsoft.Extensions.ML NuGet package and SentimentRazorML.Model project:
```
using System.IO;
using Microsoft.Extensions.ML;
using SentimentRazorML.Model;
```
Create a global variable to store the location of the trained model file.
```
private readonly string _modelPath;
```

The model file is stored in the build directory alongside the assembly files of your application. To make it easier to access, create a helper method called GetAbsolutePath after the Configure method

public static string GetAbsolutePath(string relativePath)
{
    FileInfo _dataRoot = new FileInfo(typeof(Program).Assembly.Location);
    string assemblyFolderPath = _dataRoot.Directory.FullName;

    string fullPath = Path.Combine(assemblyFolderPath, relativePath);
    return fullPath;
}

Use the GetAbsolutePath method in the Startup class constructor to set the _modelPath.
```
_modelPath = GetAbsolutePath("MLModel.zip");
```

Configure the PredictionEnginePool for your application in the ConfigureServices method:

services.AddPredictionEnginePool<ModelInput, ModelOutput>()
        .FromFile(_modelPath);

Create sentiment analysis handler

Predictions will be made inside the main page of the application. Therefore, a method that takes the user input and uses the PredictionEnginePool to return a prediction needs to be added.

Open the Index.cshtml.cs file located in the Pages directory and add the following using statements:
```
using Microsoft.Extensions.ML;
using SentimentRazorML.Model;
```
In order to use the PredictionEnginePool configured in the Startup class, you have to inject it into the constructor of the model where you want to use it.

Add a variable to reference the PredictionEnginePool inside the IndexModel class.

private readonly PredictionEnginePool<ModelInput, ModelOutput> _predictionEnginePool;

Create a constructor in the IndexModel class and inject the PredictionEnginePool service into it.

public IndexModel(PredictionEnginePool<ModelInput, ModelOutput> predictionEnginePool)
{
    _predictionEnginePool = predictionEnginePool;
}

Create a method handler that uses the PredictionEnginePool to make predictions from user input received from the web page.
1. Below the OnGet method, create a new method called OnGetAnalyzeSentiment
```
public IActionResult OnGetAnalyzeSentiment([FromQuery] string text)
{

}
```
2. Inside the OnGetAnalyzeSentiment method, return Neutral sentiment if the input from the user is blank or null.
```
if (String.IsNullOrEmpty(text)) return Content("Neutral");
```
3. Given a valid input, create a new instance of ModelInput.
```
var input = new ModelInput { SentimentText = text };
```
4. Use the PredictionEnginePool to predict sentiment.
```
var prediction = _predictionEnginePool.Predict(input);
```
5. Convert the predicted bool value into toxic or not toxic with the following code.
```
var sentiment = Convert.ToBoolean(prediction.Prediction) ? "Toxic" : "Not Toxic";
```
6. Finally, return the sentiment back to the web page.
```
return Content(sentiment);
```

Configure the web page

The results returned by the OnGetAnalyzeSentiment will be dynamically displayed on the Index web page.

Open the Index.cshtml file in the Pages directory and replace its contents with the following code:

@page
@model IndexModel
@{
    ViewData["Title"] = "Home page";
}

<div class="text-center">
    <h2>Live Sentiment</h2>

    <p><textarea id="Message" cols="45" placeholder="Type any text like a short review"></textarea></p>

    <div class="sentiment">
        <h4>Your sentiment is...</h4>
        <p>😡 😐 😍</p>

        <div class="marker">
            <div id="markerPosition" style="left: 45%;">
                <div>▲</div>
                <label id="markerValue">Neutral</label>
            </div>
        </div>
    </div>
</div>

Next, add css styling code to the end of the site.css page in the wwwroot\css directory:

/* Style for sentiment display */

.sentiment {
    background-color: #eee;
    position: relative;
    display: inline-block;
    padding: 1rem;
    padding-bottom: 0;
    border-radius: 1rem;
}

.sentiment h4 {
    font-size: 16px;
    text-align: center;
    margin: 0;
    padding: 0;
}

.sentiment p {
    font-size: 50px;
}

.sentiment .marker {
    position: relative;
    left: 22px;
    width: calc(100% - 68px);
}

.sentiment .marker > div {
    transition: 0.3s ease-in-out;
    position: absolute;
    margin-left: -30px;
    text-align: center;
}

.sentiment .marker > div > div {
    font-size: 50px;
    line-height: 20px;
    color: green;
}

.sentiment .marker > div label {
    font-size: 30px;
    color: gray;
}

After that, add code to send inputs from the web page to the OnGetAnalyzeSentiment handler.

In the site.js file located in the wwwroot\js directory, create a function called getSentiment to make a GET HTTP request with the user input to the OnGetAnalyzeSentiment handler.

function getSentiment(userInput) {
    return fetch(`Index?handler=AnalyzeSentiment&text=${userInput}`)
        .then((response) => {
            return response.text();
        })
}

Below that, add another function called updateMarker to dynamically update the position of the marker on the web page as sentiment is predicted.

function updateMarker(markerPosition, sentiment) {
    $("#markerPosition").attr("style", `left:${markerPosition}%`);
    $("#markerValue").text(sentiment);
}

Create an event handler function called updateSentiment to get the input from the user, send it to the OnGetAnalyzeSentiment function using the getSentiment function and update the marker with the updateMarker function.

function updateSentiment() {

    var userInput = $("#Message").val();

    getSentiment(userInput)
        .then((sentiment) => {
            switch (sentiment) {
                case "Not Toxic":
                    updateMarker(100.0,sentiment);
                    break;
                case "Toxic":
                    updateMarker(0.0,sentiment);
                    break;
                default:
                    updateMarker(45.0, "Neutral");
            }
        });
}

Finally, register the event handler and bind it to the textarea element with the id=Message attribute.
```
$("#Message").on('change input paste', updateSentiment)
```

Run the application

Now that your application is set up, run the application, which should launch in your browser.

When the application launches, enter Model Builder is cool! into the text area. The predicted sentiment displayed should be Not Toxic.

Running window with the predicted sentiment window

If you need to reference the Model Builder generated projects at a later time inside of another solution, you can find them inside the C:\Users\%USERNAME%\AppData\Local\Temp\MLVSTools directory.

Next steps

In this tutorial, you learned how to:

Create an ASP.NET Core Razor Pages application
Prepare and understand the data
Choose a scenario
Load the data
Train the model
Evaluate the model
Use the model for predictions

Additional Resources

To learn more about topics mentioned in this tutorial, visit the following resources: