Quickstart: Use the Content Moderator client library

Get started with the Content Moderator client library for .NET. Follow these steps to install the package and try out the example code for basic tasks. Content Moderator is a cognitive service that checks text, image, and video content for material that is potentially offensive, risky, or otherwise undesirable. When such material is found, the service applies appropriate labels (flags) to the content. Your app can then handle flagged content to comply with regulations or maintain the intended environment for users.

Use the Content Moderator client library for .NET to:

Reference documentation | Library source code | Package (NuGet) | Samples

Prerequisites

Setting up

Create a Content Moderator Azure resource

Azure Cognitive Services are represented by Azure resources that you subscribe to. Create a resource for Content Moderator using the Azure portal or Azure CLI on your local machine. You can also:

After you get a key from your resource, create environment variables for the key and endpoint URL, named CONTENT_MODERATOR_SUBSCRIPTION_KEY and CONTENT_MODERATOR_ENDPOINT, respectively.

Create a new C# application

Create a new .NET Core application in your preferred text editor or IDE.

In a console window (such as cmd, PowerShell, or Bash), use the dotnet new command to create a new console app with the name content-moderator-quickstart. This command creates a simple "Hello World" C# project with a single source file: Program.cs.

dotnet new console -n content-moderator-quickstart

Change your directory to the newly created app folder. You can build the application with:

dotnet build

The build output should contain no warnings or errors.

...
Build succeeded.
 0 Warning(s)
 0 Error(s)
...

From the project directory, open the Program.cs file in your preferred editor or IDE. Add the following using statements:

using Microsoft.Azure.CognitiveServices.ContentModerator;
using Microsoft.Azure.CognitiveServices.ContentModerator.Models;
using Newtonsoft.Json;
using System;
using System.Collections.Generic;
using System.IO;
using System.Text;
using System.Threading;

In the Program class, create variables for your resource's endpoint location and key as environment variables.

// Your Content Moderator subscription key is found in your Azure portal resource on the 'Keys' page. Add to your environment variables.
private static readonly string SubscriptionKey = Environment.GetEnvironmentVariable("CONTENT_MODERATOR_SUBSCRIPTION_KEY");
// Base endpoint URL. Add this to your environment variables. Found on 'Overview' page in Azure resource. For example: https://westus.api.cognitive.microsoft.com
private static readonly string Endpoint = Environment.GetEnvironmentVariable("CONTENT_MODERATOR_ENDPOINT");

Note

If you created the environment variables after you launched the application, you will need to close and reopen the editor, IDE, or shell running it to access the variables.

Install the client library

Within the application directory, install the Content Moderator client library for .NET with the following command:

dotnet add package Microsoft.Azure.CognitiveServices.ContentModerator --version 2.0.0

If you're using the Visual Studio IDE, the client library is available as a downloadable NuGet package.

Object model

The following classes handle some of the major features of the Content Moderator .NET client library.

Name Description
ContentModeratorClient This class is needed for all Content Moderator functionality. You instantiate it with your subscription information, and you use it to produce instances of other classes.
ImageModeration This class provides the functionality for analyzing images for adult content, personal information, or human faces.
TextModeration This class provides the functionality for analyzing text for language, profanity, errors, and personal information.
Reviews This class provides the functionality of the Review APIs, including the methods for creating jobs, custom workflows, and human reviews.

Code examples

These code snippets show you how to do the following tasks with the Content Moderator client library for .NET:

Authenticate the client

In a new method, instantiate client objects with your endpoint and key. You don't need a different client for every scenario, but it can help keep your code organized.

// Create an image review client
ContentModeratorClient clientImage = Authenticate(SubscriptionKey, Endpoint);
// Create a text review client
ContentModeratorClient clientText = Authenticate(SubscriptionKey, Endpoint);
// Create a human reviews client
ContentModeratorClient clientReviews = Authenticate(SubscriptionKey, Endpoint);

Moderate text

The following code uses a Content Moderator client to analyze a body of text and print the results to the console. In the root of your Program class, define input and output files:

// TEXT MODERATION
// Name of the file that contains text
private static readonly string TextFile = "TextFile.txt";
// The name of the file to contain the output from the evaluation.
private static string TextOutputFile = "TextModerationOutput.txt";

Then at the root of your project and add a TextFile.txt file. Add your own text to this file, or use the following sample text:

Is this a grabage email abcdef@abcd.com, phone: 4255550111, IP: 255.255.255.255, 1234 Main Boulevard, Panapolis WA 96555.
Crap is the profanity here. Is this information PII? phone 4255550111

Add the following method call to your Main method:

// Moderate text from text in a file
ModerateText(clientText, TextFile, TextOutputFile);

Then define the text moderation method somewhere in your Program class:

/*
 * TEXT MODERATION
 * This example moderates text from file.
 */
public static void ModerateText(ContentModeratorClient client, string inputFile, string outputFile)
{
    Console.WriteLine("--------------------------------------------------------------");
    Console.WriteLine();
    Console.WriteLine("TEXT MODERATION");
    Console.WriteLine();
    // Load the input text.
    string text = File.ReadAllText(inputFile);

    // Remove carriage returns
    text = text.Replace(Environment.NewLine, " ");
    // Convert string to a byte[], then into a stream (for parameter in ScreenText()).
    byte[] textBytes = Encoding.UTF8.GetBytes(text);
    MemoryStream stream = new MemoryStream(textBytes);

    Console.WriteLine("Screening {0}...", inputFile);
    // Format text

    // Save the moderation results to a file.
    using (StreamWriter outputWriter = new StreamWriter(outputFile, false))
    {
        using (client)
        {
            // Screen the input text: check for profanity, classify the text into three categories,
            // do autocorrect text, and check for personally identifying information (PII)
            outputWriter.WriteLine("Autocorrect typos, check for matching terms, PII, and classify.");

            // Moderate the text
            var screenResult = client.TextModeration.ScreenText("text/plain", stream, "eng", true, true, null, true);
            outputWriter.WriteLine(JsonConvert.SerializeObject(screenResult, Formatting.Indented));
        }

        outputWriter.Flush();
        outputWriter.Close();
    }

    Console.WriteLine("Results written to {0}", outputFile);
    Console.WriteLine();
}

Moderate images

The following code uses a Content Moderator client, along with an ImageModeration object, to analyze remote images for adult and racy content.

Note

You can also analyze the content of a local image. See the reference documentation for methods and operations that work with local images.

Get sample images

Define your input and output files:

// IMAGE MODERATION
//The name of the file that contains the image URLs to evaluate.
private static readonly string ImageUrlFile = "ImageFiles.txt";
// The name of the file to contain the output from the evaluation.
private static string ImageOutputFile = "ImageModerationOutput.json";

Then create the input file, ImageFiles.txt, at the root of your project. In this file, you add the URLs of images to analyze—one URL on each line. You can use the following sample images:

https://moderatorsampleimages.blob.core.windows.net/samples/sample2.jpg
https://moderatorsampleimages.blob.core.windows.net/samples/sample5.png

Pass your input and output files into the following method call in the Main method. You'll define this method at a later step.

// Moderate images from list of image URLs
ModerateImages(clientImage, ImageUrlFile, ImageOutputFile);

Define helper class

Add the following class definition within the Program class. This inner class will handle image moderation results.

// Contains the image moderation results for an image, 
// including text and face detection results.
public class EvaluationData
{
    // The URL of the evaluated image.
    public string ImageUrl;

    // The image moderation results.
    public Evaluate ImageModeration;

    // The text detection results.
    public OCR TextDetection;

    // The face detection results;
    public FoundFaces FaceDetection;
}

Define the image moderation method

The following method iterates through the image URLs in a text file, creates an EvaluationData instance, and analyzes the image for adult/racy content, text, and human faces. Then it adds the final EvaluationData instance to a list and writes the complete list of returned data to the console.

Iterate through image URLs

/*
 * IMAGE MODERATION
 * This example moderates images from URLs.
 */
public static void ModerateImages(ContentModeratorClient client, string urlFile, string outputFile)
{
    Console.WriteLine("--------------------------------------------------------------");
    Console.WriteLine();
    Console.WriteLine("IMAGE MODERATION");
    Console.WriteLine();
    // Create an object to store the image moderation results.
    List<EvaluationData> evaluationData = new List<EvaluationData>();

    using (client)
    {
        // Read image URLs from the input file and evaluate each one.
        using (StreamReader inputReader = new StreamReader(urlFile))
        {
            while (!inputReader.EndOfStream)
            {
                string line = inputReader.ReadLine().Trim();
                if (line != String.Empty)
                {
                    Console.WriteLine("Evaluating {0}...", Path.GetFileName(line));
                    var imageUrl = new BodyModel("URL", line.Trim());

Analyze content

For more information on the image attributes that Content Moderator screens for, see the Image moderation concepts guide.

            var imageData = new EvaluationData
            {
                ImageUrl = imageUrl.Value,

                // Evaluate for adult and racy content.
                ImageModeration =
                client.ImageModeration.EvaluateUrlInput("application/json", imageUrl, true)
            };
            Thread.Sleep(1000);

            // Detect and extract text.
            imageData.TextDetection =
                client.ImageModeration.OCRUrlInput("eng", "application/json", imageUrl, true);
            Thread.Sleep(1000);

            // Detect faces.
            imageData.FaceDetection =
                client.ImageModeration.FindFacesUrlInput("application/json", imageUrl, true);
            Thread.Sleep(1000);

            // Add results to Evaluation object
            evaluationData.Add(imageData);
        }
    }
}

Write moderation results to file

        // Save the moderation results to a file.
        using (StreamWriter outputWriter = new StreamWriter(outputFile, false))
        {
            outputWriter.WriteLine(JsonConvert.SerializeObject(
                evaluationData, Formatting.Indented));

            outputWriter.Flush();
            outputWriter.Close();
        }
        Console.WriteLine();
        Console.WriteLine("Image moderation results written to output file: " + outputFile);
        Console.WriteLine();
    }
}

Create a review

You can use the Content Moderator .NET client library to feed content into the Review tool so that human moderators can review it. To learn more about the Review tool, see the Review tool conceptual guide.

The method in this section uses the Reviews class to create a review, retrieve its ID, and check its details after receiving human input through the Review tool's web portal. It logs all of this information in an output text file. Call the method from your Main method:

// Create image reviews for human reviewers
CreateReviews(clientReviews, IMAGE_URLS_FOR_REVIEW, TEAM_NAME, ReviewsEndpoint);

Get sample images

Declare the following array at the root of your Program class. This variable references a sample image to use to create the review.

// The list of URLs of the images to create review jobs for.
private static readonly string[] IMAGE_URLS_FOR_REVIEW = new string[] { "https://moderatorsampleimages.blob.core.windows.net/samples/sample5.png" };

Get review credentials

Sign in to the Review tool and retrieve your team name. Then assign it to the appropriate variable in the Program class. Optionally, you can set up a callback endpoint to receive updates on the activity of the review.

// The name of the team to assign the review to. Must be the team name used to create your Content Moderator website account. 
// If you do not yet have an account, follow this: https://docs.microsoft.com/en-us/azure/cognitive-services/content-moderator/quick-start
// Select the gear symbol (settings)-->Credentials to retrieve it. Your team name is the Id associated with your subscription.
private static readonly string TEAM_NAME = Environment.GetEnvironmentVariable("CONTENT_MODERATOR_TEAM_NAME");
// The callback endpoint for completed human reviews. Add to your environment variables. 
// For example: https://westus.api.cognitive.microsoft.com/contentmoderator/review/v1.0
// As reviewers complete reviews, results are sent using an HTTP POST request.
private static readonly string ReviewsEndpoint = Environment.GetEnvironmentVariable("CONTENT_MODERATOR_REVIEWS_ENDPOINT");

Define helper class

Add the following class definition within your Program class. This class will be used to represent a single review instance that is submitted to the Review tool.

// Associates the review ID (assigned by the service) to the internal.
public class ReviewItem
{
    // The media type for the item to review. 
    public string Type;
    // The URL of the item to review.
    public string Url;
    // The internal content ID for the item to review.
    public string ContentId;
    // The ID that the service assigned to the review.
    public string ReviewId;
}

Define helper method

Add the following method to the Program class. This method will write the results of review queries to the output text file.

// Helper function that writes a message to the log file, and optionally to the console.
// If echo is set to true, details will be written to the console.
private static void WriteLine(TextWriter writer, string message = null, bool echo = true)
{
    writer.WriteLine(message ?? String.Empty);
    if (echo) { Console.WriteLine(message ?? String.Empty); }
}

Define the review creation method

Now you're ready to define the method that will handle the review creation and querying. Add a new method, CreateReviews, and define the following local variables.

// Create the reviews using the fixed list of images.
private static void CreateReviews(ContentModeratorClient client, string[] ImageUrls, string teamName, string endpoint)
{
    Console.WriteLine("--------------------------------------------------------------");
    Console.WriteLine();
    Console.WriteLine("CREATE HUMAN IMAGE REVIEWS");

    // The minimum amount of time, in milliseconds, to wait between calls to the Image List API.
    const int throttleRate = 2000;
    // The number of seconds to delay after a review has finished before getting the review results from the server.
    const int latencyDelay = 45;

    // The name of the log file to create. Relative paths are relative to the execution directory.
    const string OutputFile = "OutputLog.txt";

    // The optional name of the subteam to assign the review to. Not used for this example.
    const string Subteam = null;

    // The media type for the item to review. Valid values are "image", "text", and "video".
    const string MediaType = "image";

    // The metadata key to initially add to each review item. This is short for 'score'.
    // It will enable the keys to be 'a' (adult) and 'r' (racy) in the response,
    // with a value of true or false if the human reviewer marked them as adult and/or racy.
    const string MetadataKey = "sc";
    // The metadata value to initially add to each review item.
    const string MetadataValue = "true";

    // A static reference to the text writer to use for logging.
    TextWriter writer;

    // The cached review information, associating a local content ID to the created review ID for each item.
    List<ReviewItem> reviewItems = new List<ReviewItem>();

Post reviews to the Review tool

Then, add the following code to iterate through the given sample images, add metadata, and send them to the Review tool in a single batch.

using (TextWriter outputWriter = new StreamWriter(OutputFile, false))
{
    writer = outputWriter;
    WriteLine(writer, null, true);
    WriteLine(writer, "Creating reviews for the following images:", true);

    // Create the structure to hold the request body information.
    List<CreateReviewBodyItem> requestInfo = new List<CreateReviewBodyItem>();

    // Create some standard metadata to add to each item.
    List<CreateReviewBodyItemMetadataItem> metadata =
        new List<CreateReviewBodyItemMetadataItem>(new CreateReviewBodyItemMetadataItem[]
        { new CreateReviewBodyItemMetadataItem(MetadataKey, MetadataValue) });

    // Populate the request body information and the initial cached review information.
    for (int i = 0; i < ImageUrls.Length; i++)
    {
        // Cache the local information with which to create the review.
        var itemInfo = new ReviewItem()
        {
            Type = MediaType,
            ContentId = i.ToString(),
            Url = ImageUrls[i],
            ReviewId = null
        };

        WriteLine(writer, $" {Path.GetFileName(itemInfo.Url)} with id = {itemInfo.ContentId}.", true);

        // Add the item informaton to the request information.
        requestInfo.Add(new CreateReviewBodyItem(itemInfo.Type, itemInfo.Url, itemInfo.ContentId, endpoint, metadata));

        // Cache the review creation information.
        reviewItems.Add(itemInfo);
    }

    var reviewResponse = client.Reviews.CreateReviewsWithHttpMessagesAsync("application/json", teamName, requestInfo);

The object returned from the API call will contain unique ID values for each image uploaded. The following code parses these IDs and then uses them to query Content Moderator for the status of each image in the batch.

// Update the local cache to associate the created review IDs with the associated content.
var reviewIds = reviewResponse.Result.Body;
for (int i = 0; i < reviewIds.Count; i++) { reviewItems[i].ReviewId = reviewIds[i]; }

WriteLine(outputWriter, JsonConvert.SerializeObject(reviewIds, Formatting.Indented));
Thread.Sleep(throttleRate);

// Get details of the reviews created that were sent to the Content Moderator website.
WriteLine(outputWriter, null, true);
WriteLine(outputWriter, "Getting review details:", true);
foreach (var item in reviewItems)
{
    var reviewDetail = client.Reviews.GetReviewWithHttpMessagesAsync(teamName, item.ReviewId);
    WriteLine(outputWriter, $"Review {item.ReviewId} for item ID {item.ContentId} is " +
        $"{reviewDetail.Result.Body.Status}.", true);
    WriteLine(outputWriter, JsonConvert.SerializeObject(reviewDetail.Result.Body, Formatting.Indented));
    Thread.Sleep(throttleRate);
}

Get review details

The following code causes the program to wait for user input. When you come to this step at runtime, you can go to the Review tool yourself, verify that the sample image was uploaded, and interact with it. For information on how to interact with a review, see the Reviews how-to guide. When you're finished, you can press any key to continue the program and retrieve the results of the review process.

        Console.WriteLine();
        Console.WriteLine("Perform manual reviews on the Content Moderator site.");
        Console.WriteLine("Then, press any key to continue.");
        Console.ReadKey();

        // After the human reviews, the results are confirmed.
        Console.WriteLine();
        Console.WriteLine($"Waiting {latencyDelay} seconds for results to propagate.");
        Thread.Sleep(latencyDelay * 1000);

        // Get details from the human review.
        WriteLine(writer, null, true);
        WriteLine(writer, "Getting review details:", true);
        foreach (var item in reviewItems)
        {
            var reviewDetail = client.Reviews.GetReviewWithHttpMessagesAsync(teamName, item.ReviewId);
            WriteLine(writer, $"Review {item.ReviewId} for item ID {item.ContentId} is " + $"{reviewDetail.Result.Body.Status}.", true);
            WriteLine(outputWriter, JsonConvert.SerializeObject(reviewDetail.Result.Body, Formatting.Indented));

            Thread.Sleep(throttleRate);
        }

        Console.WriteLine();
        Console.WriteLine("Check the OutputLog.txt file for results of the review.");

        writer = null;
        outputWriter.Flush();
        outputWriter.Close();
    }
    Console.WriteLine("--------------------------------------------------------------");
}

If you used a callback endpoint in this scenario, it should receive an event in this format:

{'callback_endpoint': 'https://requestb.in/qmsakwqm',
 'content': '',
 'content_id': '3ebe16cb-31ed-4292-8b71-1dfe9b0e821f',
 'created_by': 'cspythonsdk',
 'metadata': [{'key': 'sc', 'value': 'True'}],
 'review_id': '201901i14682e2afe624fee95ebb248643139e7',
 'reviewer_result_tags': [{'key': 'a', 'value': 'True'},
                          {'key': 'r', 'value': 'True'}],
 'status': 'Complete',
 'sub_team': 'public',
 'type': 'Image'}

Run the application

Run the application from your application directory with the dotnet run command.

dotnet run 

Clean up resources

If you want to clean up and remove a Cognitive Services subscription, you can delete the resource or resource group. Deleting the resource group also deletes any other resources associated with it.

Next steps

In this quickstart, you learned how to use the Content Moderator .NET library to do moderation tasks. Next, learn more about the moderation of images or other media by reading a conceptual guide.

Get started with the Content Moderator client library for Java. Follow these steps to install the package and try out the example code for basic tasks. Content Moderator is a cognitive service that checks text, image, and video content for material that is potentially offensive, risky, or otherwise undesirable. When such material is found, the service applies appropriate labels (flags) to the content. Your app can then handle flagged content in order to comply with regulations or maintain the intended environment for users.

Use the Content Moderator client library for Java to:

  • Moderate images for adult or racy content, text, or human faces.

Reference documentation | Artifact (Maven) | Samples

Prerequisites

Setting up

Create a Content Moderator Azure resource

Azure Cognitive Services are represented by Azure resources that you subscribe to. Create a resource for Content Moderator using the Azure portal or Azure CLI on your local machine. You can also:

After getting a key from your resource, create an environment variable for the key, named AZURE_CONTENTMODERATOR_KEY.

Create a new Gradle project

In a console window (such as cmd, PowerShell, or Bash), create a new directory for your app, and navigate to it.

mkdir myapp && cd myapp

Run gradle init. This command will create essential build files for Gradle, including build.gradle.kts, which is used at runtime to create and configure your application. Run this command from your working directory:

gradle init --type basic

When prompted to choose a build script DSL, select Kotlin.

Find build.gradle.kts and open it with your preferred IDE or text editor. Then copy in the following build configuration. This configuration defines the project as a Java application whose entry point is the class ContentModeratorQuickstart. It imports the Content Moderator client library as well as the Gson sdk for JSON serialization.

plugins {
    java
    application
}

application{ 
    mainClassName = "ContentModeratorQuickstart"
}

repositories{
    mavenCentral()
}

dependencies{
    compile(group = "com.microsoft.azure.cognitiveservices", name = "azure-cognitiveservices-contentmoderator", version = "1.0.2-beta")
    compile(group = "com.google.code.gson", name = "gson", version = "2.8.5")
}

From your working directory, run the following command to create a project source folder.

mkdir -p src/main/java

Then create a file named ContentModeratorQuickstart.java in the new folder. Open the file in your preferred editor or IDE and import the following libraries at the top:

import com.google.gson.*;

import com.microsoft.azure.cognitiveservices.vision.contentmoderator.*;
import com.microsoft.azure.cognitiveservices.vision.contentmoderator.models.*;

import java.io.*;
import java.util.*;
import java.util.concurrent.*;

Object model

The following classes handle some of the major features of the Content Moderator Java client library.

Name Description
ContentModeratorClient This class is needed for all Content Moderator functionality. You instantiate it with your subscription information, and you use it to produce instances of other classes.
ImageModeration This class provides the functionality for analyzing images for adult content, personal information, or human faces.
TextModerations This class provides the functionality for analyzing text for language, profanity, errors, and personal information.
Reviews This class provides the functionality of the Review APIs, including the methods for creating jobs, custom workflows, and human reviews.

Code examples

These code snippets show you how to do the following tasks with the Content Moderator client library for Java:

Authenticate the client

Note

This step assumes you've created an environment variable for your Content Moderator key, named AZURE_CONTENTMODERATOR_KEY.

In the application's main method, create a ContentModeratorClient object using your subscription endpoint value and subscription key environment variable.

Note

If you created the environment variable after you launched the application, you will need to close and reopen the editor, IDE, or shell running it to access the variable.

/**
 * Authenticate
 */
// Set CONTENT_MODERATOR_SUBSCRIPTION_KEY in your environment settings, with your key as its value.
// Set COMPUTER_MODERATOR_ENDPOINT in your environment variables with your Azure endpoint.
ContentModeratorClient client = 
    ContentModeratorManager.authenticate(AzureRegionBaseUrl
                           .fromString(System.getenv("CONTENT_MODERATOR_ENDPOINT")), 
                                System.getenv("CONTENT_MODERATOR_SUBSCRIPTION_KEY"));

Moderate images

Get images

In the src/main/ folder of your project, create a resources folder and navigate to it. Then create a new text file, ImageFiles.txt. In this file, you add the URLs of images to analyze—one URL on each line. You can use the following sample images:

https://moderatorsampleimages.blob.core.windows.net/samples/sample2.jpg
https://moderatorsampleimages.blob.core.windows.net/samples/sample5.png

Define helper class

Then, in your ContentModeratorQuickstart.java file, add the following class definition inside the ContentModeratorQuickstart class. This inner class will be used later in the image moderation process.

// Contains the image moderation results for an image, including text and face detection from the image.
public static class EvaluationData {
    // The URL of the evaluated image.
    public String ImageUrl;
    // The image moderation results.
    public Evaluate ImageModeration;
    // The text detection results.
    public OCR TextDetection;
    // The face detection results;
    public FoundFaces FaceDetection;
}

Iterate through images

Next, add the following code to the bottom of the main method. Or, you can add it to a separate method that's called from main. This code steps through each line of the ImageFiles.txt file.

// ImageFiles.txt is the file that contains the image URLs to evaluate.
// Relative paths are relative to the execution directory.
try (BufferedReader inputStream = new BufferedReader(new FileReader(imageListFile) )){
    String line;
    while ((line = inputStream.readLine()) != null) {
        if (line.length() > 0) {
            // Evalutate each line of text
            BodyModelModel url = new BodyModelModel();
            url.withDataRepresentation("URL");
            url.withValue(line);
            // Save to EvaluationData class for later
            EvaluationData imageData = new EvaluationData(); 
            imageData.ImageUrl = url.value();

Check for adult/racy content

This line of code checks the image at the given URL for adult or racy content. See the Image moderation conceptual guide for information on these terms.

// Evaluate for adult and racy content.
imageData.ImageModeration = client.imageModerations().evaluateUrlInput("application/json", url, new EvaluateUrlInputOptionalParameter().withCacheImage(true));
Thread.sleep(1000);

Check for text

This line of code checks the image for visible text.

// Detect and extract text from image.
imageData.TextDetection = client.imageModerations().oCRUrlInput("eng", "application/json", url, new OCRUrlInputOptionalParameter().withCacheImage(true));
Thread.sleep(1000);

Check for faces

This line of code checks the image for human faces.

// Detect faces.
imageData.FaceDetection = client.imageModerations().findFacesUrlInput("application/json", url, new FindFacesUrlInputOptionalParameter().withCacheImage(true));
Thread.sleep(1000);

Finally, store the returned information in the EvaluationData list.

resultsList.add(imageData);

After the while loop, add the following code, which prints the results to the console and to an output file, src/main/resources/ModerationOutput.json.

// Save the moderation results to a file.
// ModerationOutput.json contains the output from the evaluation.
// Relative paths are relative to the execution directory (where pom.xml is located).
BufferedWriter writer = new BufferedWriter(new FileWriter(new File("src\\main\\resources\\ImageModerationOutput.json")));
// For formatting the printed results
Gson gson = new GsonBuilder().setPrettyPrinting().create();  

writer.write(gson.toJson(resultsList).toString());
System.out.println("Check ImageModerationOutput.json to see printed results.");
writer.close();

Close out the try statement and add a catch statement to complete the method.

}   catch (Exception e) {
    System.out.println(e.getMessage());
    e.printStackTrace();
}

Run the application

You can build the app with:

gradle build

Run the application with the gradle run command:

gradle run

Then navigate to the src/main/resources/ModerationOutput.json file and view the results of your content moderation.

Clean up resources

If you want to clean up and remove a Cognitive Services subscription, you can delete the resource or resource group. Deleting the resource group also deletes any other resources associated with it.

Next steps

In this quickstart, you learned how to use the Content Moderator Java library to perform moderation tasks. Next, learn more about the moderation of images or other media by reading a conceptual guide.

Get started with the Content Moderator client library for Python. Follow these steps to install the package and try out the example code for basic tasks. Content Moderator is a cognitive service that checks text, image, and video content for material that is potentially offensive, risky, or otherwise undesirable. When such material is found, the service applies appropriate labels (flags) to the content. Your app can then handle flagged content in order to comply with regulations or maintain the intended environment for users.

Use the Content Moderator client library for Python to:

Reference documentation | Library source code | Package (PiPy) | Samples

Prerequisites

Setting up

Create a Content Moderator Azure resource

Azure Cognitive Services are represented by Azure resources that you subscribe to. Create a resource for Content Moderator using the Azure portal or Azure CLI on your local machine. You can also:

After you get a key from your resource, create environment variables for the key and endpoint URL, named CONTENT_MODERATOR_SUBSCRIPTION_KEY and CONTENT_MODERATOR_ENDPOINT, respectively.

Create a python script

Create a new Python script and open it in your preferred editor or IDE. Then add the following import statements to the top of the file.

import os.path
from pprint import pprint
import time
from io import BytesIO
from random import random
import uuid

from azure.cognitiveservices.vision.contentmoderator import ContentModeratorClient
import azure.cognitiveservices.vision.contentmoderator.models
from msrest.authentication import CognitiveServicesCredentials

Next, create variables for your resource's endpoint location and key as environment variables.

CONTENTMODERATOR_ENDPOINT = os.environ.get("CONTENT_MODERATOR_ENDPOINT")
subscription_key = os.environ.get("CONTENT_MODERATOR_SUBSCRIPTION_KEY")

Note

If you created the environment variables after you launched the application, you will need to close and reopen the editor, IDE, or shell running it to access the variables.

Install the client library

You can install the Content Moderator client library with the following command:

pip install --upgrade azure-cognitiveservices-vision-contentmoderator

Object model

The following classes handle some of the major features of the Content Moderator Python client library.

Name Description
ContentModeratorClient This class is needed for all Content Moderator functionality. You instantiate it with your subscription information, and you use it to produce instances of other classes.
ImageModerationOperations This class provides the functionality for analyzing images for adult content, personal information, or human faces.
TextModerationOperations This class provides the functionality for analyzing text for language, profanity, errors, and personal information.
ReviewsOperations This class provides the functionality of the Review APIs, including the methods for creating jobs, custom workflows, and human reviews.

Code examples

These code snippets show you how to do the following tasks with the Content Moderator client library for Python:

Authenticate the client

Note

This quickstart assumes you've created environment variables for your Content Moderator key and endpoint.

Instantiate a client with your endpoint and key. Create a CognitiveServicesCredentials object with your key, and use it with your endpoint to create an ContentModeratorClient object.

client = ContentModeratorClient(
    endpoint=CONTENT_MODERATOR_ENDPOINT,
    credentials=CognitiveServicesCredentials(subscription_key)
)

Moderate text

The following code uses a Content Moderator client to analyze a body of text and print the results to the console. First, create a text_files/ folder at the root of your project and add a content_moderator_text_moderation.txt file. Add your own text to this file, or use the following sample text:

Is this a grabage email abcdef@abcd.com, phone: 4255550111, IP: 255.255.255.255, 1234 Main Boulevard, Panapolis WA 96555.
Crap is the profanity here. Is this information PII? phone 2065550111

Add a reference to the new folder.

TEXT_FOLDER = os.path.join(os.path.dirname(
    os.path.realpath(__file__)), "text_files")

Then, add the following code to your Python script.

# Screen the input text: check for profanity,
# do autocorrect text, and check for personally identifying
# information (PII)
with open(os.path.join(TEXT_FOLDER, 'content_moderator_text_moderation.txt'), "rb") as text_fd:
    screen = client.text_moderation.screen_text(
        text_content_type="text/plain",
        text_content=text_fd,
        language="eng",
        autocorrect=True,
        pii=True
    )
    assert isinstance(screen, Screen)
    pprint(screen.as_dict())

Use a custom terms list

The following code shows how to manage a list of custom terms for text moderation. You can use the ListManagementTermListsOperations class to create a terms list, manage the individual terms, and screen other bodies of text against it.

Get sample text

To use this sample, you must create a text_files/ folder at the root of your project and add a content_moderator_term_list.txt file. This file should contain organic text that will be checked against the list of terms. You can use the following sample text:

This text contains the terms "term1" and "term2".

Add a reference to the folder if you haven't already defined one.

TEXT_FOLDER = os.path.join(os.path.dirname(
    os.path.realpath(__file__)), "text_files")

Create a list

Add the following code to your Python script to create a custom terms list and save its ID value.

#
# Create list
#
print("\nCreating list")
custom_list = client.list_management_term_lists.create(
    content_type="application/json",
    body={
        "name": "Term list name",
        "description": "Term list description",
    }
)
print("List created:")
assert isinstance(custom_list, TermList)
pprint(custom_list.as_dict())
list_id = custom_list.id

Define list details

You can use a list's ID to edit its name and description.

#
# Update list details
#
print("\nUpdating details for list {}".format(list_id))
updated_list = client.list_management_term_lists.update(
    list_id=list_id,
    content_type="application/json",
    body={
        "name": "New name",
        "description": "New description"
    }
)
assert isinstance(updated_list, TermList)
pprint(updated_list.as_dict())

Add a term to the list

The following code adds the terms "term1" and "term2" to the list.

#
# Add terms
#
print("\nAdding terms to list {}".format(list_id))
client.list_management_term.add_term(
    list_id=list_id,
    term="term1",
    language="eng"
)
client.list_management_term.add_term(
    list_id=list_id,
    term="term2",
    language="eng"
)

Get all terms in the list

You can use the list ID to return all of the terms in the list.

#
# Get all terms ids
#
print("\nGetting all term IDs for list {}".format(list_id))
terms = client.list_management_term.get_all_terms(
    list_id=list_id, language="eng")
assert isinstance(terms, Terms)
terms_data = terms.data
assert isinstance(terms_data, TermsData)
pprint(terms_data.as_dict())

Refresh the list index

Whenever you add or remove terms from the list, you must refresh the index before you can use the updated list.

#
# Refresh the index
#
print("\nRefreshing the search index for list {}".format(list_id))
refresh_index = client.list_management_term_lists.refresh_index_method(
    list_id=list_id, language="eng")
assert isinstance(refresh_index, RefreshIndex)
pprint(refresh_index.as_dict())

print("\nWaiting {} minutes to allow the server time to propagate the index changes.".format(
    LATENCY_DELAY))
time.sleep(LATENCY_DELAY * 60)

Screen text against the list

The main functionality of the custom terms list is to compare a body of text against the list and find whether there are any matching terms.

#
# Screen text
#
with open(os.path.join(TEXT_FOLDER, 'content_moderator_term_list.txt'), "rb") as text_fd:
    screen = client.text_moderation.screen_text(
        text_content_type="text/plain",
        text_content=text_fd,
        language="eng",
        autocorrect=False,
        pii=False,
        list_id=list_id
    )
    assert isinstance(screen, Screen)
    pprint(screen.as_dict())

Remove a term from a list

The following code removes the term "term1" from the list.

#
# Remove terms
#
term_to_remove = "term1"
print("\nRemove term {} from list {}".format(term_to_remove, list_id))
client.list_management_term.delete_term(
    list_id=list_id,
    term=term_to_remove,
    language="eng"
)

Remove all terms from a list

Use the following code to clear a list of all its terms.

#
# Delete all terms
#
print("\nDelete all terms in the image list {}".format(list_id))
client.list_management_term.delete_all_terms(
    list_id=list_id, language="eng")

Delete list

Use the following code to delete a custom terms list.

#
# Delete list
#
print("\nDelete the term list {}".format(list_id))
client.list_management_term_lists.delete(list_id=list_id)

Moderate images

The following code uses a Content Moderator client, along with an ImageModerationOperations object, to analyze images for adult and racy content.

Get images

Define a reference to some images to analyze.

IMAGE_LIST = [
    "https://moderatorsampleimages.blob.core.windows.net/samples/sample2.jpg",
    "https://moderatorsampleimages.blob.core.windows.net/samples/sample5.png"
]

Then add the following code to iterate through your images. The rest of the code in this section will go inside this loop.

for image_url in IMAGE_LIST:
    print("\nEvaluate image {}".format(image_url))

Check for adult/racy content

The following code checks the image at the given URL for adult or racy content and prints results to the console. See the Image moderation concepts guide for information on what these terms mean.

print("\nEvaluate for adult and racy content.")
evaluation = client.image_moderation.evaluate_url_input(
    content_type="application/json",
    cache_image=True,
    data_representation="URL",
    value=image_url
)
assert isinstance(evaluation, Evaluate)
pprint(evaluation.as_dict())

Check for visible text

The following code checks the image for visible text content and prints results to the console.

print("\nDetect and extract text.")
evaluation = client.image_moderation.ocr_url_input(
    language="eng",
    content_type="application/json",
    data_representation="URL",
    value=image_url,
    cache_image=True,
)
assert isinstance(evaluation, OCR)
pprint(evaluation.as_dict())

Check for faces

The following code checks the image for human faces and prints results to the console.

print("\nDetect faces.")
evaluation = client.image_moderation.find_faces_url_input(
    content_type="application/json",
    cache_image=True,
    data_representation="URL",
    value=image_url
)
assert isinstance(evaluation, FoundFaces)
pprint(evaluation.as_dict())

Use a custom image list

The following code shows how to manage a custom list of images for image moderation. This feature is useful if your platform frequently receives instances of the same set of images that you want to screen out. By maintaining a list of these specific images, you can improve performance. The ListManagementImageListsOperations class allows you to create an image list, manage the individual images on the list, and compare other images against it.

Create the following text variables to store the image URLs that you'll use in this scenario.

IMAGE_LIST = {
    "Sports": [
        "https://moderatorsampleimages.blob.core.windows.net/samples/sample4.png",
        "https://moderatorsampleimages.blob.core.windows.net/samples/sample6.png",
        "https://moderatorsampleimages.blob.core.windows.net/samples/sample9.png"
    ],
    "Swimsuit": [
        "https://moderatorsampleimages.blob.core.windows.net/samples/sample1.jpg",
        "https://moderatorsampleimages.blob.core.windows.net/samples/sample3.png",
        "https://moderatorsampleimages.blob.core.windows.net/samples/sample4.png",
        "https://moderatorsampleimages.blob.core.windows.net/samples/sample16.png"
    ]
}

IMAGES_TO_MATCH = [
    "https://moderatorsampleimages.blob.core.windows.net/samples/sample1.jpg",
    "https://moderatorsampleimages.blob.core.windows.net/samples/sample4.png",
    "https://moderatorsampleimages.blob.core.windows.net/samples/sample5.png",
    "https://moderatorsampleimages.blob.core.windows.net/samples/sample16.png"
]

Note

This is not the proper list itself, but an informal list of images that will be added in the add images section of the code.

Create an image list

Add the following code to create an image list and save a reference to its ID.

#
# Create list
#
print("Creating list MyList\n")
custom_list = client.list_management_image_lists.create(
    content_type="application/json",
    body={
        "name": "MyList",
        "description": "A sample list",
        "metadata": {
            "key_one": "Acceptable",
            "key_two": "Potentially racy"
        }
    }
)
print("List created:")
assert isinstance(custom_list, ImageList)
pprint(custom_list.as_dict())
list_id = custom_list.id

Add images to a list

The following code adds all of your images to the list.

print("\nAdding images to list {}".format(list_id))
index = {}  # Keep an index url to id for later removal
for label, urls in IMAGE_LIST.items():
    for url in urls:
        image = add_images(list_id, url, label)
        if image:
            index[url] = image.content_id

Define the add_images helper function elsewhere in your script.

#
# Add images
#
def add_images(list_id, image_url, label):
    """Generic add_images from url and label."""
    print("\nAdding image {} to list {} with label {}.".format(
        image_url, list_id, label))
    try:
        added_image = client.list_management_image.add_image_url_input(
            list_id=list_id,
            content_type="application/json",
            data_representation="URL",
            value=image_url,
            label=label
        )
    except APIErrorException as err:
        # sample4 will fail
        print("Unable to add image to list: {}".format(err))
    else:
        assert isinstance(added_image, Image)
        pprint(added_image.as_dict())
        return added_image

Get images in list

The following code prints the names of all the images in your list.

#
# Get all images ids
#
print("\nGetting all image IDs for list {}".format(list_id))
image_ids = client.list_management_image.get_all_image_ids(list_id=list_id)
assert isinstance(image_ids, ImageIds)
pprint(image_ids.as_dict())

Update list details

You can use the list ID to update the name and description of the list.

#
# Update list details
#
print("\nUpdating details for list {}".format(list_id))
updated_list = client.list_management_image_lists.update(
    list_id=list_id,
    content_type="application/json",
    body={
        "name": "Swimsuits and sports"
    }
)
assert isinstance(updated_list, ImageList)
pprint(updated_list.as_dict())

Get list details

Use the following code to print the current details of your list.

#
# Get list details
#
print("\nGetting details for list {}".format(list_id))
list_details = client.list_management_image_lists.get_details(
    list_id=list_id)
assert isinstance(list_details, ImageList)
pprint(list_details.as_dict())

Refresh the list index

After you add or remove images, you must refresh the list index before you can use it to screen other images.

#
# Refresh the index
#
print("\nRefreshing the search index for list {}".format(list_id))
refresh_index = client.list_management_image_lists.refresh_index_method(
    list_id=list_id)
assert isinstance(refresh_index, RefreshIndex)
pprint(refresh_index.as_dict())

print("\nWaiting {} minutes to allow the server time to propagate the index changes.".format(
    LATENCY_DELAY))
time.sleep(LATENCY_DELAY * 60)

Match images against the list

The main function of image lists is to compare new images and see if there are any matches.

#
# Match images against the image list.
#
for image_url in IMAGES_TO_MATCH:
    print("\nMatching image {} against list {}".format(image_url, list_id))
    match_result = client.image_moderation.match_url_input(
        content_type="application/json",
        list_id=list_id,
        data_representation="URL",
        value=image_url,
    )
    assert isinstance(match_result, MatchResponse)
    print("Is match? {}".format(match_result.is_match))
    print("Complete match details:")
    pprint(match_result.as_dict())

Remove an image from the list

The following code removes an item from the list. In this case, it is an image that does not match the list category.

#
# Remove images
#
correction = "https://moderatorsampleimages.blob.core.windows.net/samples/sample16.png"
print("\nRemove image {} from list {}".format(correction, list_id))
client.list_management_image.delete_image(
    list_id=list_id,
    image_id=index[correction]
)

Remove all images from a list

Use the following code to clear out an image list.

#
# Delete all images
#
print("\nDelete all images in the image list {}".format(list_id))
client.list_management_image.delete_all_images(list_id=list_id)

Delete the image list

Use the following code to delete a given image list.

#
# Delete list
#
print("\nDelete the image list {}".format(list_id))
client.list_management_image_lists.delete(list_id=list_id)

Create a review

You can use the Content Moderator Python client library to feed content into the Review tool so that human moderators can review it. To learn more about the Review tool, see the Review tool conceptual guide.

The following code uses the ReviewsOperations class to create a review, retrieve its ID, and check its details after receiving human input through the Review tool's web portal.

Get Review credentials

First, sign in to the Review tool and retrieve your team name. Then assign it to the appropriate variable in the code. Optionally, you can set up a callback endpoint to receive updates on the activity of the review.

# The name of the team to assign the job to.
# This must be the team name you used to create your Content Moderator account. You can
# retrieve your team name from the Review tool web site. Your team name is the Id
# associated with your subscription.
team_name = "<insert your team name here>"

# An image to review
image_url = "https://moderatorsampleimages.blob.core.windows.net/samples/sample5.png"

# Where you want to receive the approval/refuse event. This is the only way to get this information.
call_back_endpoint = "https://requestb.in/qmsakwqm"

Create an image review

Add the following code to create and post a review for the given image URL. The code saves a reference to the review ID.

# Create review
print("Create review for {}.\n".format(image_url))
review_item = {
    "type": "Image",             # Possible values include: 'Image', 'Text'
    "content": image_url,        # How to download the image
    "content_id": uuid.uuid4(),  # Random id
    "callback_endpoint": call_back_endpoint,
    "metadata": [{
        "key": "sc",
        "value": True  # will be sent to Azure as "str" cast.
    }]
}

reviews = client.reviews.create_reviews(
    url_content_type="application/json",
    team_name=team_name,
    create_review_body=[review_item]  # As many review item as you need
)

# Get review ID
review_id = reviews[0]  # Ordered list of string of review ID

Get review details

Use the following code to check the details of a given review. After you create the review, you can go to the Review tool yourself and interact with the content. For information on how to do this, see the Reviews how-to guide. When you're finished, you can run this code again, and it will retrieve the results of the review process.

print("\nGet review details")
review_details = client.reviews.get_review(
    team_name=team_name, review_id=review_id)
pprint(review_details.as_dict())

If you used a callback endpoint in this scenario, it should receive an event in this format:

{'callback_endpoint': 'https://requestb.in/qmsakwqm',
 'content': '',
 'content_id': '3ebe16cb-31ed-4292-8b71-1dfe9b0e821f',
 'created_by': 'cspythonsdk',
 'metadata': [{'key': 'sc', 'value': 'True'}],
 'review_id': '201901i14682e2afe624fee95ebb248643139e7',
 'reviewer_result_tags': [{'key': 'a', 'value': 'True'},
                          {'key': 'r', 'value': 'True'}],
 'status': 'Complete',
 'sub_team': 'public',
 'type': 'Image'}

Run the application

Run the application with the python command on your quickstart file.

python quickstart-file.py

Clean up resources

If you want to clean up and remove a Cognitive Services subscription, you can delete the resource or resource group. Deleting the resource group also deletes any other resources associated with it.

Next steps

In this quickstart, you learned how to use the Content Moderator Python library to do moderation tasks. Next, learn more about the moderation of images or other media by reading a conceptual guide.