Image categorization

In addition to tags and a description, Image Analysis 3.2 can return the taxonomy-based categories detected in an image. Unlike tags, categories are organized in a parent/child hierarchy, and there are fewer of them (86, as opposed to thousands of tags). All category names are in English. Categorization can be done by itself or alongside the newer tags model.

The 86-category taxonomy

Azure AI Vision can categorize an image broadly or specifically, using the list of 86 categories in the following diagram. For the full taxonomy in text format, see Category Taxonomy.

Grouped lists of all the categories in the category taxonomy

Image categorization examples

The following JSON response illustrates what Azure AI Vision returns when categorizing the example image based on its visual features.

A woman on the roof of an apartment building

{
    "categories": [
        {
            "name": "people_",
            "score": 0.81640625
        }
    ],
    "requestId": "bae7f76a-1cc7-4479-8d29-48a694974705",
    "metadata": {
        "height": 200,
        "width": 300,
        "format": "Jpeg"
    }
}

The following table illustrates a typical image set and the category returned by Azure AI Vision for each image.

Image Category
Four people posed together as a family people_group
A puppy sitting in a grassy field animal_dog
A person standing on a mountain rock at sunset outdoor_mountain
A pile of bread roles on a table food_bread

Use the API

The categorization feature is part of the Analyze Image 3.2 API. You can call this API through a native SDK or through REST calls. Include Categories in the visualFeatures query parameter. Then, when you get the full JSON response, simply parse the string for the contents of the "categories" section.

Next steps

Learn the related concepts of tagging images and describing images.