Quickstart: Create an image classification project with the Custom Vision client library

Get started with the Custom Vision client library for .NET. Follow these steps to install the package and try out the example code for building an image classification model. You'll create a project, add tags, train the project, and use the project's prediction endpoint URL to programmatically test it. Use this example as a template for building your own image recognition app.

Note

If you want to build and train a classification model without writing code, see the browser-based guidance instead.

Use the Custom Vision client library for .NET to:

  • Create a new Custom Vision project
  • Add tags to the project
  • Upload and tag images
  • Train the project
  • Publish the current iteration
  • Test the prediction endpoint

Reference documentation | Library source code (training) (prediction) | Package (NuGet) (training) (prediction) | Samples

Prerequisites

  • Azure subscription - Create one for free
  • The Visual Studio IDE or current version of .NET Core.
  • Once you have your Azure subscription, create a Custom Vision resource in the Azure portal to create a training and prediction resource and get your keys and endpoint. Wait for it to deploy and click the Go to resource button.
    • You will need the key and endpoint from the resources you create to connect your application to Custom Vision. You'll paste your key and endpoint into the code below later in the quickstart.
    • You can use the free pricing tier (F0) to try the service, and upgrade later to a paid tier for production.

Setting up

Create a new C# application

Using Visual Studio, create a new .NET Core application.

Install the client library

Once you've created a new project, install the client library by right-clicking on the project solution in the Solution Explorer and selecting Manage NuGet Packages. In the package manager that opens select Browse, check Include prerelease, and search for Microsoft.Azure.CognitiveServices.Vision.CustomVision.Training and Microsoft.Azure.CognitiveServices.Vision.CustomVision.Prediction. Select the latest version and then Install.

Tip

Want to view the whole quickstart code file at once? You can find it on GitHub, which contains the code examples in this quickstart.

From the project directory, open the program.cs file and add the following using directives:

using Microsoft.Azure.CognitiveServices.Vision.CustomVision.Prediction;
using Microsoft.Azure.CognitiveServices.Vision.CustomVision.Training;
using Microsoft.Azure.CognitiveServices.Vision.CustomVision.Training.Models;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Threading;

In the application's Main method, create variables for your resource's key and endpoint. You'll also declare some basic objects to be used later.

private const string endpoint = "<your API endpoint here>";
// Add your training & prediction key from the settings page of the portal
private const string trainingKey = "<your training key here>";
private const string predictionKey = "<your prediction key here>";
private static List<string> hemlockImages;
private static List<string> japaneseCherryImages;
private static MemoryStream testImage;

Important

Go to the Azure portal. If the Custom Vision resources you created in the Prerequisites section deployed successfully, click the Go to Resource button under Next Steps. You can find your keys and endpoint in the resources' key and endpoint pages, under resource management. You'll need to get both your training and prediction keys.

Remember to remove the keys from your code when you're done, and never post them publicly. For production, consider using a secure way of storing and accessing your credentials. See the Cognitive Services security article for more information.

In the application's Main method, add calls for the methods used in this quickstart. You will implement these later.

CustomVisionTrainingClient TrainingApi = AuthenticateTraining(endpoint, trainingKey);
CustomVisionPredictionClient predictionApi = AuthenticateTraining(endpoint, predictionKey);

CreateProject(trainingApi);
AddTags(trainingApi);
UploadImages(trainingApi);
TrainProject(trainingApi);
PublishIteration(trainingApi);
TestIteration(predictionApi);

Object model

Name Description
CustomVisionTrainingClient This class handles the creation, training, and publishing of your models.
CustomVisionPredictionClient This class handles the querying of your models for image classification predictions.
PredictionModel This class defines a single prediction on a single image. It includes properties for the object ID and name, and a confidence score.

Code examples

These code snippets show you how to do the following tasks with the Custom Vision client library for .NET:

Authenticate the client

In a new method, instantiate training and prediction clients using your endpoint and keys.

private CustomVisionTrainingClient AuthenticateTraining(string endpoint, string trainingKey, string predictionKey)
{
    // Create the Api, passing in the training key
    CustomVisionTrainingClient trainingApi = new CustomVisionTrainingClient(new Microsoft.Azure.CognitiveServices.Vision.CustomVision.Training.ApiKeyServiceClientCredentials(trainingKey))
    {
        Endpoint = endpoint
    };
    return trainingApi;
}
private CustomVisionPredictionClient AuthenticatePrediction(string endpoint, string predictionKey)
{
    // Create a prediction endpoint, passing in the obtained prediction key
    CustomVisionPredictionClient predictionApi = new CustomVisionPredictionClient(new Microsoft.Azure.CognitiveServices.Vision.CustomVision.Prediction.ApiKeyServiceClientCredentials(predictionKey))
    {
        Endpoint = endpoint
    };
    return predictionApi;
}

Create a new Custom Vision project

This next bit of code creates an image classification project. The created project will show up on the Custom Vision website. See the CreateProject method to specify other options when you create your project (explained in the Build a classifier web portal guide).

private void CreateProject(CustomVisionTrainingClient trainingApi)
{
    // Create a new project
    Console.WriteLine("Creating new project:");
    var project = trainingApi.CreateProject("My New Project");
}

Add tags to the project

This method defines the tags that you will train the model on.

private void AddTags(CustomVisionTrainingClient trainingApi)
{

    // Make two tags in the new project
    var hemlockTag = trainingApi.CreateTag(project.Id, "Hemlock");
    var japaneseCherryTag = trainingApi.CreateTag(project.Id, "Japanese Cherry");
}

Upload and tag images

First, download the sample images for this project. Save the contents of the sample Images folder to your local device.

Then define a helper method to upload the images in this directory. You may need to edit the GetFiles argument to point to the location where your images are saved.

private static void LoadImagesFromDisk()
{
    // this loads the images to be uploaded from disk into memory
    hemlockImages = Directory.GetFiles(Path.Combine("Images", "Hemlock")).ToList();
    japaneseCherryImages = Directory.GetFiles(Path.Combine("Images", "Japanese Cherry")).ToList();
    testImage = new MemoryStream(File.ReadAllBytes(Path.Combine("Images", "Test\\test_image.jpg")));
}

Next, define a method to upload the images, applying tags according to their folder location (the images are already sorted). You can upload and tag images iteratively, or in a batch (up to 64 per batch). This code snippet contains examples of both.

private void UploadImages(CustomVisionTrainingClient trainingApi)
{
    // Add some images to the tags
    Console.WriteLine("\tUploading images");
    LoadImagesFromDisk();

    // Images can be uploaded one at a time
    foreach (var image in hemlockImages)
    {
        using (var stream = new MemoryStream(File.ReadAllBytes(image)))
        {
            trainingApi.CreateImagesFromData(project.Id, stream, new List<Guid>() { hemlockTag.Id });
        }
    }

    // Or uploaded in a single batch 
    var imageFiles = japaneseCherryImages.Select(img => new ImageFileCreateEntry(Path.GetFileName(img), File.ReadAllBytes(img))).ToList();
    trainingApi.CreateImagesFromFiles(project.Id, new ImageFileCreateBatch(imageFiles, new List<Guid>() { japaneseCherryTag.Id }));

}

Train the project

This method creates the first training iteration in the project. It queries the service until training is completed.

private void TrainProject(CustomVisionTrainingClient trainingApi)
{
    // Now there are images with tags start training the project
    Console.WriteLine("\tTraining");
    var iteration = trainingApi.TrainProject(project.Id);

    // The returned iteration will be in progress, and can be queried periodically to see when it has completed
    while (iteration.Status == "Training")
    {
        Thread.Sleep(1000);

        // Re-query the iteration to get it's updated status
        iteration = trainingApi.GetIteration(project.Id, iteration.Id);
    }
}

Tip

Train with selected tags

You can optionally train on only a subset of your applied tags. You may want to do this if you haven't applied enough of certain tags yet, but you do have enough of others. In the TrainProject call, use the trainingParameters parameter. Construct a TrainingParameters and set its SelectedTags property to a list of IDs of the tags you want to use. The model will train to only recognize the tags on that list.

Publish the current iteration

This method makes the current iteration of the model available for querying. You can use the model name as a reference to send prediction requests. You need to enter your own value for predictionResourceId. You can find the prediction resource ID on the resource's Overview tab in the Azure portal, listed as Subscription ID.

private void PublishIteration(CustomVisionTrainingClient trainingApi)
{

    // The iteration is now trained. Publish it to the prediction end point.
    var publishedModelName = "treeClassModel";
    var predictionResourceId = "<target prediction resource ID>";
    trainingApi.PublishIteration(project.Id, iteration.Id, publishedModelName, predictionResourceId);
    Console.WriteLine("Done!\n");

    // Now there is a trained endpoint, it can be used to make a prediction
}

Test the prediction endpoint

This part of the script loads the test image, queries the model endpoint, and outputs prediction data to the console.

private void TestIteration(CustomVisionPredictionClient predictionApi)
{

    // Make a prediction against the new project
    Console.WriteLine("Making a prediction:");
    var result = predictionApi.ClassifyImage(project.Id, publishedModelName, testImage);

    // Loop over each prediction and write out the results
    foreach (var c in result.Predictions)
    {
        Console.WriteLine($"\t{c.TagName}: {c.Probability:P1}");
    }
    Console.ReadKey();
}

Run the application

Run the application by clicking the Debug button at the top of the IDE window.

As the application runs, it should open a console window and write the following output:

Creating new project:
        Uploading images
        Training
Done!

Making a prediction:
        Hemlock: 95.0%
        Japanese Cherry: 0.0%

You can then verify that the test image (found in Images/Test/) is tagged appropriately. Press any key to exit the application. You can also go back to the Custom Vision website and see the current state of your newly created project.

Clean up resources

If you wish to implement your own image classification project (or try an object detection project instead), you may want to delete the tree identification project from this example. A free subscription allows for two Custom Vision projects.

On the Custom Vision website, navigate to Projects and select the trash can under My New Project.

Screenshot of a panel labeled My New Project with a trash can icon

Next steps

Now you've done every step of the image classification process in code. This sample executes a single training iteration, but often you'll need to train and test your model multiple times in order to make it more accurate.

This guide provides instructions and sample code to help you get started using the Custom Vision client library for Go to build an image classification model. You'll create a project, add tags, train the project, and use the project's prediction endpoint URL to programmatically test it. Use this example as a template for building your own image recognition app.

Note

If you want to build and train a classification model without writing code, see the browser-based guidance instead.

Prerequisites

  • Go 1.8+
  • To use the Custom Vision Service you will need to create Custom Vision Training and Prediction resources in Azure. To do so in the Azure portal, fill out the dialog window on the Create Custom Vision page to create both a Training and Prediction resource.

Install the Custom Vision client library

To write an image analysis app with Custom Vision for Go, you'll need the Custom Vision service client library. Run the following command in PowerShell:

go get -u github.com/Azure/azure-sdk-for-go/...

or if you use dep, within your repo run:

dep ensure -add github.com/Azure/azure-sdk-for-go

Get the training and prediction keys

The project needs a valid set of subscription keys to interact with the service. You can find the items at the Custom Vision website. Sign in with the account associated with the Azure account you used to create your Custom Vision resources. On the home page (the page with the option to add a new project), select the gear icon in the upper right. Find your training and prediction resources in the list and expand them. Here you can find your training key, prediction key, and prediction resource ID values. Save these values to a temporary location.

Note

If you're using a Cognitive Services all-in-one key to access Custom Vision, then you'll only see one key on the settings screen. In this case, you'll use the same key for both training and prediction operations.

Image of the keys UI

Or, you can obtain these keys and ID from the Azure portal by viewing your Custom Vision Training and Prediction resources and navigating to the Keys tab. There you'll find your training key and prediction key. Navigate to the Properties tab of your Prediction resource to get your prediction resource ID.

Get the sample images

This example uses the images from the Cognitive Services Python SDK Samples repository on GitHub. Clone or download this repository to your development environment. Remember its folder location for a later step.

Add the code

Create a new file called sample.go in your preferred project directory.

Create the Custom Vision project

Add the following code to your script to create a new Custom Vision service project. Insert your subscription keys in the appropriate definitions. Also, get your Endpoint URL from the Settings page of the Custom Vision website.

See the CreateProject method to specify other options when you create your project (explained in the Build a classifier web portal guide).

import(
    "context"
    "bytes"
    "fmt"
    "io/ioutil"
    "path"
    "log"
    "time"
    "github.com/Azure/azure-sdk-for-go/services/cognitiveservices/v3.0/customvision/training"
    "github.com/Azure/azure-sdk-for-go/services/cognitiveservices/v3.0/customvision/prediction"
)

var (
    training_key string = "<your training key>"
    prediction_key string = "<your prediction key>"
    prediction_resource_id = "<your prediction resource id>"
    endpoint string = "<your endpoint URL>"
    project_name string = "Go Sample Project"
    iteration_publish_name = "classifyModel"
    sampleDataDirectory = "<path to sample images>"
)

func main() {
    fmt.Println("Creating project...")

    ctx = context.Background()

    trainer := training.New(training_key, endpoint)

    project, err := trainer.CreateProject(ctx, project_name, "sample project", nil, string(training.Multilabel))
    if (err != nil) {
        log.Fatal(err)
    }

Create tags in the project

To create classification tags to your project, add the following code to the end of sample.go:

// Make two tags in the new project
hemlockTag, _ := trainer.CreateTag(ctx, *project.ID, "Hemlock", "Hemlock tree tag", string(training.Regular))
cherryTag, _ := trainer.CreateTag(ctx, *project.ID, "Japanese Cherry", "Japanese cherry tree tag", string(training.Regular))

Upload and tag images

To add the sample images to the project, insert the following code after the tag creation. This code uploads each image with its corresponding tag. You can upload up to 64 images in a single batch.

Note

You'll need to change the path to the images based on where you downloaded the Cognitive Services Go SDK Samples project earlier.

fmt.Println("Adding images...")
japaneseCherryImages, err := ioutil.ReadDir(path.Join(sampleDataDirectory, "Japanese Cherry"))
if err != nil {
    fmt.Println("Error finding Sample images")
}

hemLockImages, err := ioutil.ReadDir(path.Join(sampleDataDirectory, "Hemlock"))
if err != nil {
    fmt.Println("Error finding Sample images")
}

for _, file := range hemLockImages {
    imageFile, _ := ioutil.ReadFile(path.Join(sampleDataDirectory, "Hemlock", file.Name()))
    imageData := ioutil.NopCloser(bytes.NewReader(imageFile))

    trainer.CreateImagesFromData(ctx, *project.ID, imageData, []string{ hemlockTag.ID.String() })
}

for _, file := range japaneseCherryImages {
    imageFile, _ := ioutil.ReadFile(path.Join(sampleDataDirectory, "Japanese Cherry", file.Name()))
    imageData := ioutil.NopCloser(bytes.NewReader(imageFile))
    trainer.CreateImagesFromData(ctx, *project.ID, imageData, []string{ cherryTag.ID.String() })
}

Train and publish the project

This code creates the first iteration of the prediction model and then publishes that iteration to the prediction endpoint. The name given to the published iteration can be used to send prediction requests. An iteration is not available in the prediction endpoint until it is published.

fmt.Println("Training...")
iteration, _ := trainer.TrainProject(ctx, *project.ID)
for {
    if *iteration.Status != "Training" {
        break
    }
    fmt.Println("Training status: " + *iteration.Status)
    time.Sleep(1 * time.Second)
    iteration, _ = trainer.GetIteration(ctx, *project.ID, *iteration.ID)
}
fmt.Println("Training status: " + *iteration.Status)

trainer.PublishIteration(ctx, *project.ID, *iteration.ID, iteration_publish_name, prediction_resource_id))

Use the prediction endpoint

To send an image to the prediction endpoint and retrieve the prediction, add the following code to the end of the file:

    fmt.Println("Predicting...")
    predictor := prediction.New(prediction_key, endpoint)

    testImageData, _ := ioutil.ReadFile(path.Join(sampleDataDirectory, "Test", "test_image.jpg"))
    results, _ := predictor.ClassifyImage(ctx, *project.ID, iteration_publish_name, ioutil.NopCloser(bytes.NewReader(testImageData)), "")

    for _, prediction := range *results.Predictions    {
        fmt.Printf("\t%s: %.2f%%", *prediction.TagName, *prediction.Probability * 100)
        fmt.Println("")
    }
}

Run the application

Run sample.go.

go run sample.go

The output of the application should be similar to the following text:

Creating project...
Adding images...
Training...
Training status: Training
Training status: Training
Training status: Training
Training status: Completed
Done!
        Hemlock: 93.53%
        Japanese Cherry: 0.01%

You can then verify that the test image (found in <base_image_url>/Images/Test/) is tagged appropriately. You can also go back to the Custom Vision website and see the current state of your newly created project.

Clean up resources

If you wish to implement your own image classification project (or try an object detection project instead), you may want to delete the tree identification project from this example. A free subscription allows for two Custom Vision projects.

On the Custom Vision website, navigate to Projects and select the trash can under My New Project.

Screenshot of a panel labeled My New Project with a trash can icon

Next steps

Now you've seen how every step of the object detection process can be done in code. This sample executes a single training iteration, but often you'll need to train and test your model multiple times in order to make it more accurate.

Get started using the Custom Vision client library for Java to build an image classification model. Follow these steps to install the package and try out the example code for basic tasks. Use this example as a template for building your own image recognition app.

Note

If you want to build and train a classification model without writing code, see the browser-based guidance instead.

Use the Custom Vision client library for Java to:

  • Create a new Custom Vision project
  • Add tags to the project
  • Upload and tag images
  • Train the project
  • Publish the current iteration
  • Test the prediction endpoint

Reference documentation | Library source code (training) (prediction)| Artifact (Maven) (training) (prediction) | Samples

Prerequisites

  • An Azure subscription - Create one for free
  • The current version of the Java Development Kit(JDK)
  • The Gradle build tool, or another dependency manager.
  • Once you have your Azure subscription, create a Custom Vision resource in the Azure portal to create a training and prediction resource and get your keys and endpoint. Wait for it to deploy and click the Go to resource button.
    • You will need the key and endpoint from the resources you create to connect your application to Custom Vision. You'll paste your key and endpoint into the code below later in the quickstart.
    • You can use the free pricing tier (F0) to try the service, and upgrade later to a paid tier for production.

Setting up

Create a new Gradle project

In a console window (such as cmd, PowerShell, or Bash), create a new directory for your app, and navigate to it.

mkdir myapp && cd myapp

Run the gradle init command from your working directory. This command will create essential build files for Gradle, including build.gradle.kts, which is used at runtime to create and configure your application.

gradle init --type basic

When prompted to choose a DSL, select Kotlin.

Install the client library

Locate build.gradle.kts and open it with your preferred IDE or text editor. Then copy in the following build configuration. This configuration defines the project as a Java application whose entry point is the class CustomVisionQuickstart. It imports the Custom Vision libraries.

plugins {
    java
    application
}
application { 
    mainClassName = "CustomVisionQuickstart"
}
repositories {
    mavenCentral()
}
dependencies {
    compile(group = "com.azure", name = "azure-cognitiveservices-customvision-training", version = "1.1.0-preview.2")
    compile(group = "com.azure", name = "azure-cognitiveservices-customvision-prediction", version = "1.1.0-preview.2")
}

Create a Java file

From your working directory, run the following command to create a project source folder:

mkdir -p src/main/java

Navigate to the new folder and create a file called CustomVisionQuickstart.java. Open it in your preferred editor or IDE and add the following import statements:

import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.UUID;

import com.google.common.io.ByteStreams;

import com.microsoft.azure.cognitiveservices.vision.customvision.training.models.Classifier;
import com.microsoft.azure.cognitiveservices.vision.customvision.training.models.Domain;
import com.microsoft.azure.cognitiveservices.vision.customvision.training.models.DomainType;
import com.microsoft.azure.cognitiveservices.vision.customvision.training.models.ImageFileCreateBatch;
import com.microsoft.azure.cognitiveservices.vision.customvision.training.models.ImageFileCreateEntry;
import com.microsoft.azure.cognitiveservices.vision.customvision.training.models.Iteration;
import com.microsoft.azure.cognitiveservices.vision.customvision.training.models.Project;
import com.microsoft.azure.cognitiveservices.vision.customvision.training.models.Region;
import com.microsoft.azure.cognitiveservices.vision.customvision.training.models.TrainProjectOptionalParameter;
import com.microsoft.azure.cognitiveservices.vision.customvision.training.CustomVisionTrainingClient;
import com.microsoft.azure.cognitiveservices.vision.customvision.training.Trainings;
import com.microsoft.azure.cognitiveservices.vision.customvision.training.CustomVisionTrainingManager;
import com.microsoft.azure.cognitiveservices.vision.customvision.prediction.models.ImagePrediction;
import com.microsoft.azure.cognitiveservices.vision.customvision.prediction.models.Prediction;
import com.microsoft.azure.cognitiveservices.vision.customvision.prediction.CustomVisionPredictionClient;
import com.microsoft.azure.cognitiveservices.vision.customvision.prediction.CustomVisionPredictionManager;
import com.microsoft.azure.cognitiveservices.vision.customvision.training.models.Tag;

Tip

Want to view the whole quickstart code file at once? You can find it on GitHub, which contains the code examples in this quickstart.

In the application's CustomVisionQuickstart class, create variables for your resource's keys and endpoint.

final static String trainingApiKey = "<your-training-subscription-key>";
final static String predictionApiKey = "<your-prediction-subscription-key>";
final static String endpoint = "<your API endpoint>";

Important

Go to the Azure portal. If the [Product name] resource you created in the Prerequisites section deployed successfully, click the Go to Resource button under Next Steps. You can find your key and endpoint in the resource's key and endpoint page, under resource management.

Remember to remove the key from your code when you're done, and never post it publicly. For production, consider using a secure way of storing and accessing your credentials. See the Cognitive Services security article for more information.

In the application's main method, add calls for the methods used in this quickstart. You'll define these later.

Project project = createProject(trainClient);
addTags(trainClient, project);
uploadImages(trainClient, project);
trainProject(trainClient, project);
publishIteration(trainClient, project);
testProject(predictor, project);

Object model

The following classes and interfaces handle some of the major features of the Custom Vision Java client library.

Name Description
CustomVisionTrainingClient This class handles the creation, training, and publishing of your models.
CustomVisionPredictionClient This class handles the querying of your models for image classification predictions.
ImagePrediction This class defines a single prediction on a single image. It includes properties for the object ID and name, and a confidence score.

Code examples

These code snippets show you how to do the following tasks with the Custom Vision client library for Java:

Authenticate the client

In your main method, instantiate training and prediction clients using your endpoint and keys.

// Authenticate
CustomVisionTrainingClient trainClient = CustomVisionTrainingManager
        .authenticate("https://{Endpoint}/customvision/v3.0/training/", CustomVisionTrainingClientKey)
        .withEndpoint(Endpoint);
CustomVisionPredictionClient predictor = CustomVisionPredictionManager
        .authenticate("https://{Endpoint}/customvision/v3.0/prediction/", predictionApiKey)
        .withEndpoint(Endpoint);

Create a Custom Vision project

T## Create a new Custom Vision project

This next method creates an image classification project. The created project will show up on the Custom Vision website that you visited earlier. See the CreateProject method overloads to specify other options when you create your project (explained in the Build a detector web portal guide).

public static Project createProject(CustomVisionTrainingClient trainClient) {
    System.out.println("ImageClassification Sample");
    Trainings trainer = trainClient.trainings();

    System.out.println("Creating project...");
    Project project = trainer.createProject().withName("Sample Java Project").execute();

    return project;
}

Add tags to your project

This method defines the tags that you will train the model on.

public static void addTags(CustomVisionTrainingClient trainClient, Project project) {

    Trainings trainer = trainClient.trainings();

    // create hemlock tag
    Tag hemlockTag = trainer.createTag().withProjectId(project.id()).withName("Hemlock").execute();
    // create cherry tag
    Tag cherryTag = trainer.createTag().withProjectId(project.id()).withName("Japanese Cherry").execute();
}

Upload and tag images

First, download the sample images for this project. Save the contents of the sample Images folder to your local device.

public static void uploadImages(CustomVisionTrainingClient trainClient, Project project) {
    Trainings trainer = trainClient.trainings();
    System.out.println("Adding images...");
    for (int i = 1; i <= 10; i++) {
        String fileName = "hemlock_" + i + ".jpg";
        byte[] contents = GetImage("/Hemlock", fileName);
        AddImageToProject(trainer, project, fileName, contents, hemlockTag.id(), null);
    }

    for (int i = 1; i <= 10; i++) {
        String fileName = "japanese_cherry_" + i + ".jpg";
        byte[] contents = GetImage("/Japanese Cherry", fileName);
        AddImageToProject(trainer, project, fileName, contents, cherryTag.id(), null);
    }
}

The previous code snippet makes use of two helper functions that retrieve the images as resource streams and upload them to the service (you can upload up to 64 images in a single batch).

private static void AddImageToProject(Trainings trainer, Project project, String fileName, byte[] contents,
        UUID tag, double[] regionValues) {
    System.out.println("Adding image: " + fileName);
    ImageFileCreateEntry file = new ImageFileCreateEntry().withName(fileName).withContents(contents);

    ImageFileCreateBatch batch = new ImageFileCreateBatch().withImages(Collections.singletonList(file));

    // If Optional region is specified, tack it on and place the tag there,
    // otherwise
    // add it to the batch.
    if (regionValues != null) {
        Region region = new Region().withTagId(tag).withLeft(regionValues[0]).withTop(regionValues[1])
                .withWidth(regionValues[2]).withHeight(regionValues[3]);
        file = file.withRegions(Collections.singletonList(region));
    } else {
        batch = batch.withTagIds(Collections.singletonList(tag));
    }

    trainer.createImagesFromFiles(project.id(), batch);
}

private static byte[] GetImage(String folder, String fileName) {
    try {
        return ByteStreams.toByteArray(CustomVisionSamples.class.getResourceAsStream(folder + "/" + fileName));
    } catch (Exception e) {
        System.out.println(e.getMessage());
        e.printStackTrace();
    }
    return null;
}

Train the project

This method creates the first training iteration in the project. It queries the service until training is completed.

public static void trainProject(CustomVisionTrainingClient trainClient, Project project) {
    System.out.println("Training...");
    Trainings trainer = trainClient.trainings();

    Iteration iteration = trainer.trainProject(project.id(), new TrainProjectOptionalParameter());

    while (iteration.status().equals("Training")) {
        System.out.println("Training Status: " + iteration.status());
        Thread.sleep(1000);
        iteration = trainer.getIteration(project.id(), iteration.id());
    }
    System.out.println("Training Status: " + iteration.status());
}

Publish the current iteration

This method makes the current iteration of the model available for querying. You can use the model name as a reference to send prediction requests. You need to enter your own value for predictionResourceId. You can find the prediction resource ID on the resource's Overview tab in the Azure portal, listed as Subscription ID.

public static String publishIteration(CustomVisionTrainingClient trainClient, Project project) {
    Trainings trainer = trainClient.trainings();
    // The iteration is now trained. Publish it to the prediction endpoint.
    String publishedModelName = "myModel";
    String predictionResourceId = System.getenv("AZURE_CUSTOMVISION_PREDICTION_ID");
    trainer.publishIteration(project.id(), iteration.id(), publishedModelName, predictionResourceId);
}

Test the prediction endpoint

This method loads the test image, queries the model endpoint, and outputs prediction data to the console.

// load test image
public static void testProject(CustomVisionPredictionClient predictor, Project project) {

    byte[] testImage = GetImage("/Test", "test_image.jpg");

    // predict
    ImagePrediction results = predictor.predictions().classifyImage().withProjectId(project.id())
            .withPublishedName(publishedModelName).withImageData(testImage).execute();

    for (Prediction prediction : results.predictions()) {
        System.out.println(String.format("\t%s: %.2f%%", prediction.tagName(), prediction.probability() * 100.0f));
    }
}

Run the application

Run the application

You can build the app with:

gradle build

Run the application with the gradle run command:

gradle run

Clean up resources

If you want to clean up and remove a Cognitive Services subscription, you can delete the resource or resource group. Deleting the resource group also deletes any other resources associated with it.

Clean up resources

If you wish to implement your own image classification project (or try an object detection project instead), you may want to delete the tree identification project from this example. A free subscription allows for two Custom Vision projects.

On the Custom Vision website, navigate to Projects and select the trash can under My New Project.

Screenshot of a panel labeled My New Project with a trash can icon

Next steps

Now you've seen how every step of the image classification process can be done in code. This sample executes a single training iteration, but often you'll need to train and test your model multiple times in order to make it more accurate.

This guide provides instructions and sample code to help you get started using the Custom Vision client library for Node.js to build an image classification model. You'll create a project, add tags, train the project, and use the project's prediction endpoint URL to programmatically test it. Use this example as a template for building your own image recognition app.

Note

If you want to build and train a classification model without writing code, see the browser-based guidance instead.

Use the Custom Vision client library for .NET to:

  • Create a new Custom Vision project
  • Add tags to the project
  • Upload and tag images
  • Train the project
  • Publish the current iteration
  • Test the prediction endpoint

Reference documentation (training) (prediction) | Library source code (training) (prediction) | Package (npm) (training) (prediction) | Samples

Prerequisites

  • Azure subscription - Create one for free
  • The current version of Node.js
  • Once you have your Azure subscription, create a Custom Vision resource in the Azure portal to create a training and prediction resource and get your keys and endpoint. Wait for it to deploy and click the Go to resource button.
    • You will need the key and endpoint from the resources you create to connect your application to Custom Vision. You'll paste your key and endpoint into the code below later in the quickstart.
    • You can use the free pricing tier (F0) to try the service, and upgrade later to a paid tier for production.

Setting up

Create a new Node.js application

In a console window (such as cmd, PowerShell, or Bash), create a new directory for your app, and navigate to it.

mkdir myapp && cd myapp

Run the npm init command to create a node application with a package.json file.

npm init

Install the client library

To write an image analysis app with Custom Vision for Node.js, you'll need the Custom Vision NPM packages. To install them, run the following command in PowerShell:

npm install @azure/cognitiveservices-customvision-training
npm install @azure/cognitiveservices-customvision-prediction

Your app's package.json file will be updated with the dependencies.

Create a file named index.js and import the following libraries:

const util = require('util');
const fs = require('fs');
const TrainingApi = require("@azure/cognitiveservices-customvision-training");
const PredictionApi = require("@azure/cognitiveservices-customvision-prediction");
const msRest = require("@azure/ms-rest-js");

Tip

Want to view the whole quickstart code file at once? You can find it on GitHub, which contains the code examples in this quickstart.

Create variables for your resource's Azure endpoint and keys.

const trainingKey = "<your training key>";
const predictionKey = "<your prediction key>";
const predictionResourceId = "<your prediction resource id>";
const endPoint = "https://<my-resource-name>.cognitiveservices.azure.com/"

Important

Go to the Azure portal. If the [Product name] resource you created in the Prerequisites section deployed successfully, click the Go to Resource button under Next Steps. You can find your key and endpoint in the resource's key and endpoint page, under resource management.

Remember to remove the key from your code when you're done, and never post it publicly. For production, consider using a secure way of storing and accessing your credentials. See the Cognitive Services security article for more information.

Also add fields for your project name and a timeout parameter for asynchronous calls.

const publishIterationName = "classifyModel";
const setTimeoutPromise = util.promisify(setTimeout);

Object model

Name Description
TrainingAPIClient This class handles the creation, training, and publishing of your models.
PredictionAPIClient This class handles the querying of your models for image classification predictions.
Prediction This interface defines a single prediction on a single image. It includes properties for the object ID and name, and a confidence score.

Code examples

These code snippets show you how to do the following tasks with the Custom Vision client library for JavaScript:

Authenticate the client

Instantiate client objects with your endpoint and key. Create an ApiKeyCredentials object with your key, and use it with your endpoint to create a TrainingAPIClient and PredictionAPIClient object.

const credentials = new msRest.ApiKeyCredentials({ inHeader: { "Training-key": trainingKey } });
const trainer = new TrainingApi.TrainingAPIClient(credentials, endPoint);
const predictor_credentials = new msRest.ApiKeyCredentials({ inHeader: { "Prediction-key": predictionKey } });
const predictor = new PredictionApi.PredictionAPIClient(predictor_credentials, endPoint);

Create a new Custom Vision project

Start a new function to contain all of your Custom Vision function calls. Add the following code to to create a new Custom Vision service project.

(async () => {
    console.log("Creating project...");
    const sampleProject = await trainer.createProject("Sample Project");

Add tags to the project

To create classification tags to your project, add the following code to your function:

const hemlockTag = await trainer.createTag(sampleProject.id, "Hemlock");
const cherryTag = await trainer.createTag(sampleProject.id, "Japanese Cherry");

Upload and tag images

First, download the sample images for this project. Save the contents of the sample Images folder to your local device.

To add the sample images to the project, insert the following code after the tag creation. This code uploads each image with its corresponding tag.

const sampleDataRoot = "<path to image files>";

console.log("Adding images...");
let fileUploadPromises = [];

const hemlockDir = `${sampleDataRoot}/Hemlock`;
const hemlockFiles = fs.readdirSync(hemlockDir);
hemlockFiles.forEach(file => {
    fileUploadPromises.push(trainer.createImagesFromData(sampleProject.id, fs.readFileSync(`${hemlockDir}/${file}`), { tagIds: [hemlockTag.id] }));
});

const cherryDir = `${sampleDataRoot}/Japanese Cherry`;
const japaneseCherryFiles = fs.readdirSync(cherryDir);
japaneseCherryFiles.forEach(file => {
    fileUploadPromises.push(trainer.createImagesFromData(sampleProject.id, fs.readFileSync(`${cherryDir}/${file}`), { tagIds: [cherryTag.id] }));
});

await Promise.all(fileUploadPromises);

Important

You'll need to change the path to the images (sampleDataRoot) based on where you downloaded the Cognitive Services Python SDK Samples repo.

Train the project

This code creates the first iteration of the prediction model.

console.log("Training...");
let trainingIteration = await trainer.trainProject(sampleProject.id);

// Wait for training to complete
console.log("Training started...");
while (trainingIteration.status == "Training") {
    console.log("Training status: " + trainingIteration.status);
    await setTimeoutPromise(1000, null);
    trainingIteration = await trainer.getIteration(sampleProject.id, trainingIteration.id)
}
console.log("Training status: " + trainingIteration.status);

Publish the current iteration

This code publishes the trained iteration to the prediction endpoint. The name given to the published iteration can be used to send prediction requests. An iteration is not available in the prediction endpoint until it is published.

// Publish the iteration to the end point
await trainer.publishIteration(sampleProject.id, trainingIteration.id, publishIterationName, predictionResourceId);

Test the prediction endpoint

To send an image to the prediction endpoint and retrieve the prediction, add the following code to your function.

const testFile = fs.readFileSync(`${sampleDataRoot}/Test/test_image.jpg`);

const results = await predictor.classifyImage(sampleProject.id, publishIterationName, testFile);

// Show results
console.log("Results:");
results.predictions.forEach(predictedResult => {
    console.log(`\t ${predictedResult.tagName}: ${(predictedResult.probability * 100.0).toFixed(2)}%`);
});

Then, close your Custom Vision function and call it.

})()

Run the application

Run the application with the node command on your quickstart file.

node index.js

The output of the application should be similar to the following text:

Creating project...
Adding images...
Training...
Training started...
Training status: Training
Training status: Training
Training status: Training
Training status: Completed
Results:
         Hemlock: 94.97%
         Japanese Cherry: 0.01%

You can then verify that the test image (found in /Test/) is tagged appropriately. You can also go back to the Custom Vision website and see the current state of your newly created project.

Clean up resources

If you wish to implement your own image classification project (or try an object detection project instead), you may want to delete the tree identification project from this example. A free subscription allows for two Custom Vision projects.

On the Custom Vision website, navigate to Projects and select the trash can under My New Project.

Screenshot of a panel labeled My New Project with a trash can icon

Next steps

Now you've seen how every step of the object detection process can be done in code. This sample executes a single training iteration, but often you'll need to train and test your model multiple times in order to make it more accurate.

Get started with the Custom Vision client library for Python. Follow these steps to install the package and try out the example code for building an image classification model. You'll create a project, add tags, train the project, and use the project's prediction endpoint URL to programmatically test it. Use this example as a template for building your own image recognition app.

Note

If you want to build and train a classification model without writing code, see the browser-based guidance instead.

Use the Custom Vision client library for Python to:

  • Create a new Custom Vision project
  • Add tags to the project
  • Upload and tag images
  • Train the project
  • Publish the current iteration
  • Test the prediction endpoint

Reference documentation | Library source code | Package (PyPI) | Samples

Prerequisites

  • Azure subscription - Create one for free
  • Python 3.x
  • Once you have your Azure subscription, create a Custom Vision resource in the Azure portal to create a training and prediction resource and get your keys and endpoint. Wait for it to deploy and click the Go to resource button.
    • You will need the key and endpoint from the resources you create to connect your application to Custom Vision. You'll paste your keys and endpoint into the code below later in the quickstart.
    • You can use the free pricing tier (F0) to try the service, and upgrade later to a paid tier for production.

Setting up

Install the client library

To write an image analysis app with Custom Vision for Python, you'll need the Custom Vision client library. After installing Python, run the following command in PowerShell or a console window:

pip install azure-cognitiveservices-vision-customvision

Create a new python application

Create a new Python file and import the following libraries.

from azure.cognitiveservices.vision.customvision.training import CustomVisionTrainingClient
from azure.cognitiveservices.vision.customvision.prediction import CustomVisionPredictionClient
from azure.cognitiveservices.vision.customvision.training.models import ImageFileCreateBatch, ImageFileCreateEntry, Region
from msrest.authentication import ApiKeyCredentials
import time

Tip

Want to view the whole quickstart code file at once? You can find it on GitHub, which contains the code examples in this quickstart.

Create variables for your resource's Azure endpoint and subscription keys.

# Replace with valid values
ENDPOINT = "<your API endpoint>"
training_key = "<your training key>"
prediction_key = "<your prediction key>"
prediction_resource_id = "<your prediction resource id>"

Important

Go to the Azure portal. If the Custom Vision resources you created in the Prerequisites section deployed successfully, click the Go to Resource button under Next Steps. You can find your keys and endpoint in the resources' key and endpoint pages, under resource management. You'll need to get both your training and prediction keys.

You can find the prediction resource ID value on the resource's Overview tab, listed as Subscription ID.

Remember to remove the keys from your code when you're done, and never post them publicly. For production, consider using a secure way of storing and accessing your credentials. See the Cognitive Services security article for more information.

Object model

Name Description
CustomVisionTrainingClient This class handles the creation, training, and publishing of your models.
CustomVisionPredictionClient This class handles the querying of your models for image classification predictions.
ImagePrediction This class defines a single object prediction on a single image. It includes properties for the object ID and name, the bounding box location of the object, and a confidence score.

Code examples

These code snippets show you how to do the following with the Custom Vision client library for Python:

Authenticate the client

Instantiate a training and prediction client with your endpoint and keys. Create ApiKeyServiceClientCredentials objects with your keys, and use them with your endpoint to create a CustomVisionTrainingClient and CustomVisionPredictionClient object.

credentials = ApiKeyCredentials(in_headers={"Training-key": training_key})
trainer = CustomVisionTrainingClient(ENDPOINT, credentials)
prediction_credentials = ApiKeyCredentials(in_headers={"Prediction-key": prediction_key})
predictor = CustomVisionPredictionClient(ENDPOINT, prediction_credentials)

Create a new Custom Vision project

Add the following code to your script to create a new Custom Vision service project.

See the create_project method to specify other options when you create your project (explained in the Build a classifier web portal guide).

publish_iteration_name = "classifyModel"

credentials = ApiKeyCredentials(in_headers={"Training-key": training_key})
trainer = CustomVisionTrainingClient(ENDPOINT, credentials)

# Create a new project
print ("Creating project...")
project = trainer.create_project("My New Project")

Add tags to the project

To add classification tags to your project, add the following code:

# Make two tags in the new project
hemlock_tag = trainer.create_tag(project.id, "Hemlock")
cherry_tag = trainer.create_tag(project.id, "Japanese Cherry")

Upload and tag images

First, download the sample images for this project. Save the contents of the sample Images folder to your local device.

To add the sample images to the project, insert the following code after the tag creation. This code uploads each image with its corresponding tag. You can upload up to 64 images in a single batch.

# Make two tags in the new project
hemlock_tag = trainer.create_tag(project.id, "Hemlock")
cherry_tag = trainer.create_tag(project.id, "Japanese Cherry")

Note

You'll need to change the path to the images based on where you downloaded the Cognitive Services Python SDK Samples repo.

Train the project

This code creates the first iteration of the prediction model.

print ("Training...")
iteration = trainer.train_project(project.id)
while (iteration.status != "Completed"):
    iteration = trainer.get_iteration(project.id, iteration.id)
    print ("Training status: " + iteration.status)
    time.sleep(1)

# The iteration is now trained. Publish it to the project endpoint
trainer.publish_iteration(project.id, iteration.id, publish_iteration_name, prediction_resource_id)
print ("Done!")

Tip

Train with selected tags

You can optionally train on only a subset of your applied tags. You may want to do this if you haven't applied enough of certain tags yet, but you do have enough of others. In the train_project call, set the optional parameter selected_tags to a list of the ID strings of the tags you want to use. The model will train to only recognize the tags on that list.

Publish the current iteration

An iteration is not available in the prediction endpoint until it is published. The following code makes the current iteration of the model available for querying.

# The iteration is now trained. Publish it to the project endpoint
trainer.publish_iteration(project.id, iteration.id, publish_iteration_name, prediction_resource_id)
print ("Done!")

Test the prediction endpoint

To send an image to the prediction endpoint and retrieve the prediction, add the following code to the end of the file:

# Now there is a trained endpoint that can be used to make a prediction
prediction_credentials = ApiKeyCredentials(in_headers={"Prediction-key": prediction_key})
predictor = CustomVisionPredictionClient(ENDPOINT, prediction_credentials)

with open(base_image_location + "images/Test/test_image.jpg", "rb") as image_contents:
    results = predictor.classify_image(
        project.id, publish_iteration_name, image_contents.read())

    # Display the results.
    for prediction in results.predictions:
        print("\t" + prediction.tag_name +
              ": {0:.2f}%".format(prediction.probability * 100))

Run the application

Run CustomVisionQuickstart.py.

python CustomVisionQuickstart.py

The output of the application should be similar to the following text:

Creating project...
Adding images...
Training...
Training status: Training
Training status: Completed
Done!
        Hemlock: 93.53%
        Japanese Cherry: 0.01%

You can then verify that the test image (found in <base_image_location>/images/Test/) is tagged appropriately. You can also go back to the Custom Vision website and see the current state of your newly created project.

Clean up resources

If you wish to implement your own image classification project (or try an object detection project instead), you may want to delete the tree identification project from this example. A free subscription allows for two Custom Vision projects.

On the Custom Vision website, navigate to Projects and select the trash can under My New Project.

Screenshot of a panel labeled My New Project with a trash can icon

Next steps

Now you've seen how every step of the image classification process can be done in code. This sample executes a single training iteration, but often you'll need to train and test your model multiple times in order to make it more accurate.