Quickstart: Computer Vision client library for .NET

Get started with the Computer Vision client library for .NET. Follow these steps to install the package and try out the example code for basic tasks. Computer Vision provides you with access to advanced algorithms for processing images and returning information.

Use the Computer Vision client library for .NET to:

  • Analyze an image for tags, text description, faces, adult content, and more.
  • Recognize printed and handwritten text with the Batch Read API.

Reference documentation | Library source code | Package (NuGet) | Samples

Prerequisites

Setting up

Create a Computer Vision Azure resource

Azure Cognitive Services are represented by Azure resources that you subscribe to. Create a resource for Computer Vision using the Azure portal or Azure CLI on your local machine. You can also:

After you get a key from your trial subscription or resource, create environment variables for the key and endpoint URL, named COMPUTER_VISION_SUBSCRIPTION_KEY and COMPUTER_VISION_ENDPOINT, respectively.

Create a new C# application

Create a new .NET Core application in your preferred editor or IDE.

In a console window (such as cmd, PowerShell, or Bash), use the dotnet new command to create a new console app with the name computer-vision-quickstart. This command creates a simple "Hello World" C# project with a single source file: Program.cs.

dotnet new console -n computer-vision-quickstart

Change your directory to the newly created app folder. You can build the application with:

dotnet build

The build output should contain no warnings or errors.

...
Build succeeded.
 0 Warning(s)
 0 Error(s)
...

From the project directory, open the Program.cs file in your preferred editor or IDE. Add the following using directives:

using System;
using System.Collections.Generic;
using Microsoft.Azure.CognitiveServices.Vision.ComputerVision;
using Microsoft.Azure.CognitiveServices.Vision.ComputerVision.Models;
using System.Threading.Tasks;
using System.IO;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;

In the application's Program class, create variables for your resource's Azure endpoint and key.

// Get your COMPUTER_VISION_SUBSCRIPTION_KEY and COMPUTER_VISION_REGION values from your Face resource,
// in the Azure portal, and set your local environment variables accordingly.
// Close/reopen your project for them to take effect.
// Free trial subscription keys are generated in the westcentralus region.
// If you use a free trial subscription key, you shouldn't need to change the region.
static string SubscriptionKey = Environment.GetEnvironmentVariable("COMPUTER_VISION_SUBSCRIPTION_KEY");
static string Endpoint = Environment.GetEnvironmentVariable("COMPUTER_VISION_ENDPOINT");

Install the client library

Within the application directory, install the Computer Vision client library for .NET with the following command:

dotnet add package Microsoft.Azure.CognitiveServices.Vision.ComputerVision --version 5.0.0

If you're using the Visual Studio IDE, the client library is available as a downloadable NuGet package.

Object model

The following classes and interfaces handle some of the major features of the Computer Vision .NET SDK.

Name Description
ComputerVisionClient This class is needed for all Computer Vision functionality. You instantiate it with your subscription information, and you use it to do most image operations.
ComputerVisionClientExtensions This class contains additional methods for the ComputerVisionClient.
VisualFeatureTypes This enum defines the different types of image analysis that can be done in a standard Analyze operation. You specify a set of VisualFeatureTypes values depending on your needs.

Code examples

These code snippets show you how to do the following tasks with the Computer Vision client library for .NET:

Authenticate the client

Note

This quickstart assumes you've created environment variables for your Computer Vision key and endpoint, named COMPUTER_VISION_SUBSCRIPTION_KEY and COMPUTER_VISION_ENDPOINT respectively.

In a new method, instantiate a client with your endpoint and key. Create a CognitiveServicesCredentials object with your key, and use it with your endpoint to create an ComputerVisionClient object.

/*
 * AUTHENTICATE
 * Creates a Computer Vision client used by each example.
 */
public static ComputerVisionClient Authenticate(string endpoint, string key)
{
    ComputerVisionClient client =
        new ComputerVisionClient(new ApiKeyServiceClientCredentials(key))
        { Endpoint = endpoint };
    return client;
}

You'll likely want to call this method in the Main method.

// Create a client
ComputerVisionClient client = Authenticate(Endpoint, SubscriptionKey);

Analyze an image

The following code defines a method, AnalyzeImageUrl, which uses the client object to analyze a remote image and print the results. The method returns a text description, categorization, list of tags, detected faces, adult content flags, main colors, and image type.

Add the method call in your Main method.

// Analyze an image to get features and other properties.
AnalyzeImageUrl(client, ANALYZE_URL_IMAGE).Wait();

Set up test image

In your Program class, save a reference to the URL of the image you want to analyze.

// URL image used for analyzing an image (image of puppy)
private const string ANALYZE_URL_IMAGE = "https://moderatorsampleimages.blob.core.windows.net/samples/sample16.png";

Note

You can also analyze a local image. See the sample code on GitHub for scenarios involving local images.

Specify visual features

Define your new method for image analysis. Add the code below, which specifies visual features you'd like to extract in your analysis. See the VisualFeatureTypes enum for a complete list.

   /*
    * ANALYZE IMAGE - URL IMAGE
* Analyze URL image. Extracts captions, categories, tags, objects, faces, racy/adult content,
* brands, celebrities, landmarks, color scheme, and image types.
    */
   public static async Task AnalyzeImageUrl(ComputerVisionClient client, string imageUrl)
   {
       Console.WriteLine("----------------------------------------------------------");
       Console.WriteLine("ANALYZE IMAGE - URL");
       Console.WriteLine();

       // Creating a list that defines the features to be extracted from the image.
       List<VisualFeatureTypes> features = new List<VisualFeatureTypes>()
           {
               VisualFeatureTypes.Categories, VisualFeatureTypes.Description,
               VisualFeatureTypes.Faces, VisualFeatureTypes.ImageType,
               VisualFeatureTypes.Tags, VisualFeatureTypes.Adult,
               VisualFeatureTypes.Color, VisualFeatureTypes.Brands,
               VisualFeatureTypes.Objects
           };

Analyze

The AnalyzeImageAsync method returns an ImageAnalysis object that contains all of extracted information.

Console.WriteLine($"Analyzing the image {Path.GetFileName(imageUrl)}...");
Console.WriteLine();
// Analyze the URL image
ImageAnalysis results = await client.AnalyzeImageAsync(imageUrl, features);

The following sections show how to parse this information in detail.

Get image description

The following code gets the list of generated captions for the image. See Describe images for more details.

// Summarizes the image content.
Console.WriteLine("Summary:");
foreach (var caption in results.Description.Captions)
{
    Console.WriteLine($"{caption.Text} with confidence {caption.Confidence}");
}
Console.WriteLine();

Get image category

The following code gets the detected category of the image. See Categorize images for more details.

// Display categories the image is divided into.
Console.WriteLine("Categories:");
foreach (var category in results.Categories)
{
    Console.WriteLine($"{category.Name} with confidence {category.Score}");
}
Console.WriteLine();

Get image tags

The following code gets the set of detected tags in the image. See Content tags for more details.

// Image tags and their confidence score
Console.WriteLine("Tags:");
foreach (var tag in results.Tags)
{
    Console.WriteLine($"{tag.Name} {tag.Confidence}");
}
Console.WriteLine();

Detect objects

The following code detects common objects in the image and prints them to the console. See Object detection for more details.

// Objects
Console.WriteLine("Objects:");
foreach (var obj in results.Objects)
{
    Console.WriteLine($"{obj.ObjectProperty} with confidence {obj.Confidence} at location {obj.Rectangle.X}, " +
        $"{obj.Rectangle.X + obj.Rectangle.W}, {obj.Rectangle.Y}, {obj.Rectangle.Y + obj.Rectangle.H}");
}
Console.WriteLine();

Detect brands

The following code detects corporate brands and logos in the image and prints them to the console. See Brand detection for more details.

// Well-known (or custom, if set) brands.
Console.WriteLine("Brands:");
foreach (var brand in results.Brands)
{
    Console.WriteLine($"Logo of {brand.Name} with confidence {brand.Confidence} at location {brand.Rectangle.X}, " +
        $"{brand.Rectangle.X + brand.Rectangle.W}, {brand.Rectangle.Y}, {brand.Rectangle.Y + brand.Rectangle.H}" );
}
Console.WriteLine();

Detect faces

The following code returns the detected faces in the image with their rectangle coordinates and select face attributes. See Face detection for more details.

// Faces
Console.WriteLine("Faces:");
foreach (var face in results.Faces)
{
    Console.WriteLine($"A {face.Gender} of age {face.Age} at location {face.FaceRectangle.Left}, " +
        $"{face.FaceRectangle.Left}, {face.FaceRectangle.Top + face.FaceRectangle.Width}, " +
        $"{face.FaceRectangle.Top + face.FaceRectangle.Height}");
}
Console.WriteLine();

Detect adult, racy, or gory content

The following code prints the detected presence of adult content in the image. See Adult, racy, gory content for more details.

// Adult or racy content, if any.
Console.WriteLine("Adult:");
Console.WriteLine($"Has adult content: {results.Adult.IsAdultContent} with confidence {results.Adult.AdultScore}");
Console.WriteLine($"Has racy content: {results.Adult.IsRacyContent} with confidence {results.Adult.RacyScore}");
Console.WriteLine();

Get image color scheme

The following code prints the detected color attributes in the image, like the dominant colors and accent color. See Color schemes for more details.

// Identifies the color scheme.
Console.WriteLine("Color Scheme:");
Console.WriteLine("Is black and white?: " + results.Color.IsBWImg);
Console.WriteLine("Accent color: " + results.Color.AccentColor);
Console.WriteLine("Dominant background color: " + results.Color.DominantColorBackground);
Console.WriteLine("Dominant foreground color: " + results.Color.DominantColorForeground);
Console.WriteLine("Dominant colors: " + string.Join(",", results.Color.DominantColors));
Console.WriteLine();

Get domain-specific content

Computer Vision can use specialized models to do further analysis on images. See Domain-specific content for more details.

The following code parses data about detected celebrities in the image.

// Celebrities in image, if any.
Console.WriteLine("Celebrities:");
foreach (var category in results.Categories)
{
    if (category.Detail?.Celebrities != null)
    {
        foreach (var celeb in category.Detail.Celebrities)
        {
            Console.WriteLine($"{celeb.Name} with confidence {celeb.Confidence} at location {celeb.FaceRectangle.Left}, " +
                $"{celeb.FaceRectangle.Top}, {celeb.FaceRectangle.Height}, {celeb.FaceRectangle.Width}");
        }
    }
}
Console.WriteLine();

The following code parses data about detected landmarks in the image.

// Popular landmarks in image, if any.
Console.WriteLine("Landmarks:");
foreach (var category in results.Categories)
{
    if (category.Detail?.Landmarks != null)
    {
        foreach (var landmark in category.Detail.Landmarks)
        {
            Console.WriteLine($"{landmark.Name} with confidence {landmark.Confidence}");
        }
    }
}
Console.WriteLine();

Get the image type

The following code prints information about the type of image—whether it is clip art or a line drawing.

// Detects the image types.
Console.WriteLine("Image Type:");
Console.WriteLine("Clip Art Type: " + results.ImageType.ClipArtType);
Console.WriteLine("Line Drawing Type: " + results.ImageType.LineDrawingType);
Console.WriteLine();

Read printed and handwritten text

Computer Vision can read visible text in an image and convert it to a character stream. The code in this section defines a method, ExtractTextUrl, which uses the client object to detect and extract printed or handwritten text in the image.

Add the method call in your Main method.

// Read the batch text from an image (handwriting and/or printed).
BatchReadFileUrl(client, EXTRACT_TEXT_URL_IMAGE).Wait();
BatchReadFileLocal(client, EXTRACT_TEXT_LOCAL_IMAGE).Wait();

Set up test image

In your Program class, save a reference the URL of the image you want to extract text from.

private const string EXTRACT_TEXT_URL_IMAGE = "https://moderatorsampleimages.blob.core.windows.net/samples/sample2.jpg";

Note

You can also extract text from a local image. See the sample code on GitHub for scenarios involving local images.

Call the Read API

Define the new method for reading text. Add the code below, which calls the BatchReadFileAsync method for the given image. This returns an operation ID and starts an asynchronous process to read the content of the image.

   /*
* BATCH READ FILE - URL IMAGE
* Recognizes handwritten text. 
    * This API call offers an improvement of results over the Recognize Text calls.
*/
   public static async Task BatchReadFileUrl(ComputerVisionClient client, string urlImage)
   {
       Console.WriteLine("----------------------------------------------------------");
       Console.WriteLine("BATCH READ FILE - URL IMAGE");
       Console.WriteLine();

       // Read text from URL
       BatchReadFileHeaders textHeaders = await client.BatchReadFileAsync(urlImage);
       // After the request, get the operation location (operation ID)
       string operationLocation = textHeaders.OperationLocation;

Get Read results

Next, get the operation ID returned from the BatchReadFileAsync call, and use it to query the service for operation results. The following code checks the operation at one-second intervals until the results are returned. It then prints the extracted text data to the console.

// Retrieve the URI where the recognized text will be stored from the Operation-Location header.
// We only need the ID and not the full URL
const int numberOfCharsInOperationId = 36;
string operationId = operationLocation.Substring(operationLocation.Length - numberOfCharsInOperationId);

// Extract the text
// Delay is between iterations and tries a maximum of 10 times.
int i = 0;
int maxRetries = 10;
ReadOperationResult results;
Console.WriteLine($"Extracting text from URL image {Path.GetFileName(urlImage)}...");
Console.WriteLine();
do
{
    results = await client.GetReadOperationResultAsync(operationId);
    Console.WriteLine("Server status: {0}, waiting {1} seconds...", results.Status, i);
    await Task.Delay(1000);
}
while ((results.Status == TextOperationStatusCodes.Running ||
        results.Status == TextOperationStatusCodes.NotStarted) && i++ < maxRetries);

Display Read results

Add the following code to parse and display the retrieved text data, and finish the method definition.

    // Display the found text.
    Console.WriteLine();
    var recognitionResults = results.RecognitionResults;
    foreach (TextRecognitionResult result in recognitionResults)
    {
        foreach (Line line in result.Lines)
        {
            Console.WriteLine(line.Text);
        }
    }
    Console.WriteLine();
}

Run the application

Run the application from your application directory with the dotnet run command.

dotnet run

Clean up resources

If you want to clean up and remove a Cognitive Services subscription, you can delete the resource or resource group. Deleting the resource group also deletes any other resources associated with it.

Next steps