Quickstart: Get image insights using the Bing Visual Search SDK for Python

Use this quickstart to begin getting image insights from the Bing Visual Search service, using the Python SDK. While Bing Visual Search has a REST API compatible with most programming languages, the SDK provides an easy way to integrate the service into your applications. The source code for this sample can be found on GitHub


  • Python 2.x or 3.x
  • It is recommended to use a virtual environment. Install and initialize the virtual environment with the venv module.
  • The Bing Visual Search SDK for Python. You can install it with the following commands:
    1. cd mytestenv
    2. python -m pip install azure-cognitiveservices-search-visualsearch

You must have a Cognitive Services API account with access to the Bing Search APIs. If you don't have an Azure subscription, you can create an account for free. Before continuing, You will need the access key provided after activating your free trial, or a paid subscription key from your Azure dashboard.

Create and initialize the application

  1. Create a new Python file in your favorite IDE or editor, and add the following import statements.

    import http.client, urllib.parse
    import json
    import os.path
    from azure.cognitiveservices.search.visualsearch import VisualSearchClient
    from azure.cognitiveservices.search.visualsearch.models import (
    from msrest.authentication import CognitiveServicesCredentials
  2. Create variables for your subscription key, Custom Configuration ID, and the image you want to upload.

    subscription_key = 'YOUR-VISUAL-SEARCH-ACCESS-KEY'
    PATH = 'C:\\Users\\USER\\azure-cognitive-samples\\mytestenv\\TestImages\\'
    image_path = os.path.join(PATH, "image.jpg")
  3. Instantiate the client

    client = VisualSearchClient(endpoint="https://api.cognitive.microsoft.com", credentials=CognitiveServicesCredentials(subscription_key))

Send the search request

  1. With the image file open, serialize VisualSearchRequest(), and pass it as the knowledge_request parameter for the visual_search().

    with open(image_path, "rb") as image_fd:
        # You need to pass the serialized form of the model
        knowledge_request = json.dumps(VisualSearchRequest().serialize())
        print("\r\nSearch visual search request with binary of dog image")
        result = client.images.visual_search(image=image_fd, knowledge_request=knowledge_request)
  2. If any results were returned, print them, the tags, and the actions in the first tag.

    if not result:
            print("No visual search result data.")
            # Visual Search results
        if result.image.image_insights_token:
            print("Uploaded image insights token: {}".format(result.image.image_insights_token))
            print("Couldn't find image insights token!")
        # List of tags
        if result.tags:
            first_tag = result.tags[0]
            print("Visual search tag count: {}".format(len(result.tags)))
            # List of actions in first tag
            if first_tag.actions:
                first_tag_action = first_tag.actions[0]
                print("First tag action count: {}".format(len(first_tag.actions)))
                print("First tag action type: {}".format(first_tag_action.action_type))
                print("Couldn't find tag actions!")
            print("Couldn't find image tags!")

Next steps