Quickstart: Create an image classification project with the Custom Vision .NET SDK

This article provides information and sample code to help you get started using the Custom Vision SDK with C# to build an image classification model. After it's created, you can add tags, upload images, train the project, obtain the project's default prediction endpoint URL, and use the endpoint to programmatically test an image. Use this example as a template for building your own .NET application. If you wish to go through the process of building and using a classification model without code, see the browser-based guidance instead.

Prerequisites

  • Any edition of Visual Studio 2015 or 2017
  • To use the Custom Vision Service you will need to create Custom Vision Training and Prediction resources in Azure. To do so in the Azure portal, fill out the dialog window on the Create Custom Vision page to create both a Training and Prediction resource.

Get the Custom Vision SDK and sample code

To write a .NET app that uses Custom Vision, you'll need the Custom Vision NuGet packages. These packages are included in the sample project you will download, but you can access them individually here.

Clone or download the Cognitive Services .NET Samples project. Navigate to the CustomVision/ImageClassification folder and open ImageClassification.csproj in Visual Studio.

This Visual Studio project creates a new Custom Vision project named My New Project, which can be accessed through the Custom Vision website. It then uploads images to train and test a classifier. In this project, the classifier is intended to determine whether a tree is a Hemlock or a Japanese Cherry.

Get the training and prediction keys

The project needs a valid set of subscription keys to interact with the service. You can find the items at the Custom Vision website. Sign in with the account associated with the Azure account used to create your Custom Vision resources. On the home page (the page with the option to add a new project), select the gear icon in the upper right. Find your training and prediction resources in the list and expand them. Here you can find your training key, prediction key, and prediction resource ID values. Save these values to a temporary location.

Image of the keys UI

Or, you can obtain these keys and ID from the Azure portal by viewing your Custom Vision Training and Prediction resources and navigating to the Keys tab. There you'll find your training key and prediction key. Navigate to the Properties tab of your Prediction resource to get your prediction resource ID.

Understand the code

Open the Program.cs file and inspect the code. Create environment variables for your training and prediction keys named CUSTOM_VISION_TRAINING_KEY and CUSTOM_VISION_PREDICTION_KEY, respectively. The script will look for these.

// Add your training & prediction key from the settings page of the portal
string trainingKey = Environment.GetEnvironmentVariable("CUSTOM_VISION_TRAINING_KEY");
string predictionKey = Environment.GetEnvironmentVariable("CUSTOM_VISION_PREDICTION_KEY");

Also, get your Endpoint URL from the Settings page of the Custom Vision website. Save it to an environment variable called CUSTOM_VISION_ENDPOINT. The script saves a reference to it at the root of your class.

string ENDPOINT = Environment.GetEnvironmentVariable("CUSTOM_VISION_ENDPOINT");

The following lines of code execute the primary functionality of the project.

Create a new Custom Vision service project

The created project will show up on the Custom Vision website that you visited earlier. See the CreateProject method to specify other options when you create your project (explained in the Build a classifier web portal guide).

// Create the Api, passing in the training key
CustomVisionTrainingClient trainingApi = new CustomVisionTrainingClient()
{
    ApiKey = trainingKey,
    Endpoint = ENDPOINT
};

// Create a new project
Console.WriteLine("Creating new project:");
var project = trainingApi.CreateProject("My New Project");

Create tags in the project

// Make two tags in the new project
var hemlockTag = trainingApi.CreateTag(project.Id, "Hemlock");
var japaneseCherryTag = trainingApi.CreateTag(project.Id, "Japanese Cherry");

Upload and tag images

The images for this project are included. They are referenced in the LoadImagesFromDisk method in Program.cs. You can upload up to 64 images in a single batch.

// Add some images to the tags
Console.WriteLine("\tUploading images");
LoadImagesFromDisk();

// Images can be uploaded one at a time
foreach (var image in hemlockImages)
{
    using (var stream = new MemoryStream(File.ReadAllBytes(image)))
    {
        trainingApi.CreateImagesFromData(project.Id, stream, new List<Guid>() { hemlockTag.Id });
    }
}

Train the classifier and publish

This code creates the first iteration in the project and then publishes that iteration to the prediction endpoint. The name given to the published iteration can be used to send prediction requests. An iteration is not available in the prediction endpoint until it is published.

// Now there are images with tags start training the project
Console.WriteLine("\tTraining");
var iteration = trainingApi.TrainProject(project.Id);

// The returned iteration will be in progress, and can be queried periodically to see when it has completed
while (iteration.Status == "Training")
{
    Thread.Sleep(1000);

    // Re-query the iteration to get it's updated status
    iteration = trainingApi.GetIteration(project.Id, iteration.Id);
}

// The iteration is now trained. Publish it to the prediction end point.
var publishedModelName = "treeClassModel";
var predictionResourceId = "<target prediction resource ID>";
trainingApi.PublishIteration(project.Id, iteration.Id, publishedModelName, predictionResourceId);
Console.WriteLine("Done!\n");

Set the prediction endpoint

The prediction endpoint is the reference that you can use to submit an image to the current model and get a classification prediction.

// Create a prediction endpoint, passing in obtained prediction key
CustomVisionPredictionClient endpoint = new CustomVisionPredictionClient()
{
    ApiKey = predictionKey,
    Endpoint = ENDPOINT
};

Submit an image to the default prediction endpoint

In this script, the test image is loaded in the LoadImagesFromDisk method, and the model's prediction output is to be displayed in the console. The value of the publishedModelName variable should correspond to the "Published as" value found on the Custom Vision portal's Performance tab.

// Make a prediction against the new project
Console.WriteLine("Making a prediction:");
var result = endpoint.ClassifyImage(project.Id, publishedModelName, testImage);

// Loop over each prediction and write out the results
foreach (var c in result.Predictions)
{
    Console.WriteLine($"\t{c.TagName}: {c.Probability:P1}");
}
Console.ReadKey();

Run the application

As the application runs, it should open a console window and write the following output:

Creating new project:
        Uploading images
        Training
Done!

Making a prediction:
        Hemlock: 95.0%
        Japanese Cherry: 0.0%

You can then verify that the test image (found in Images/Test/) is tagged appropriately. Press any key to exit the application. You can also go back to the Custom Vision website and see the current state of your newly created project.

Clean up resources

If you wish to implement your own image classification project (or try an object detection project instead), you may want to delete the tree identification project from this example. A free trial allows for two Custom Vision projects.

On the Custom Vision website, navigate to Projects and select the trash can under My New Project.

Screenshot of a panel labeled My New Project with a trash can icon

Next steps

Now you have seen how every step of the image classification process can be done in code. This sample executes a single training iteration, but often you will need to train and test your model multiple times in order to make it more accurate.