Quickstart: Create a HoloLens app with Azure Object Anchors, in C++/WinRT and DirectX

This quickstart covers how to create a HoloLens app using Azure Object Anchors in C++/WinRT and DirectX. Object Anchors is a managed cloud service that converts 3D assets into AI models that enable object-aware mixed reality experiences for the HoloLens. When you're finished, you'll have a HoloLens app that can detect an object and its pose in a Holographic DirectX 11 (Universal Windows) application.

You'll learn how to:

  • Create and side-load a HoloLens application
  • Detect an object and visualize its model

If you don't have an Azure subscription, create an Azure free account before you begin.

Prerequisites

To complete this quickstart, make sure you have:

  • A physical object in your environment and its 3D model, either CAD or scanned.
  • A Windows computer with the following installed:
  • A HoloLens 2 device that is up to date and has developer mode enabled.
    • To update to the latest release on HoloLens, open the Settings app, go to Update & Security, and then select Check for updates.

Create an Object Anchors account

First, you need to create an account with the Object Anchors service.

  1. Go to the Azure portal and select Create a resource.

    Create a new resource

  2. Search for the Object Anchors resource.

    Search for "Object Anchors".

    Select the Object Anchors Resource

    On the Object Anchors resource in the search results, select Create -> Object Anchors.

    Create an Object Anchors Resource

  3. In the Object Anchors Account dialog box:

    • Enter a unique resource name.
    • Select the subscription you want to attach the resource to.
    • Create or use an existing resource group.
    • Select the region you'd like your resource to exist in.

    Enter Object Anchors resource account details

    Select Create to begin creating the resource.

  4. Once the resource has been created, select Go to resource.

    Go to resource

  5. On the overview page:

    Take note of the Account Domain. You'll need it later.

    Copy the account domain for your Object Anchors resource

    Take note of the Account ID. You'll need it later.

    Copy the account ID for your Object Anchors resource

    Go to the Access Keys page and take note of the Primary key. You'll need it later.

    Copy the account key for your Object Anchors resource

Upload your model

Before you run the app, you'll need to make your models available to the app. If you don't already have an Object Anchors model, follow the instructions in Create a model to create one. Then, return here.

With your HoloLens powered on and connected to the development device (PC), follow these steps to upload a model to the 3D Objects folder on your HoloLens:

  1. Select and copy the models you want to work with by pressing Ctrl key and C together (Ctrl + C).

  2. Press Windows Logo key and E together (Win + E) to launch File Explorer. You should see your HoloLens listed with other drives and folders on the left pane.

    file explorer

  3. Tap on the HoloLens link to show the storage on the HoloLens device on the right pane.

    open HoloLens internal storage

  4. In File Explorer, go to Internal Storage > 3D Objects. Now, you can paste your models in the 3D Objects folder by pressing Ctrl key and V together (Ctrl + V).

    paste models in 3D Objects folder

Open the sample project

Clone the samples repository by running the following commands:

git clone https://github.com/Azure/azure-object-anchors.git

cd ./azure-object-anchors

Open quickstarts/apps/directx/DirectXAoaSampleApp.sln in Visual Studio.

Change the Solution Configuration to Release, change Solution Platform to ARM64, select Device from the deployment target options.

Configure the account information

The next step is to configure the app to use your account information. You took note of the Account Key, Account ID, and Account Domain values, in the "Create an Object Anchors account" section.

Open Assets\ObjectAnchorsConfig.json.

Locate the AccountId field and replace Set me with your Account ID.

Locate the AccountKey field and replace Set me with your Account Key.

Locate the AccountDomain field and replace Set me with your Account Domain.

Now, build the AoaSampleApp project by right-clicking the project and selecting Build.

Screenshot shows configuring Visual Studio project to deploy.

Deploy the app to HoloLens

After compiling the sample project successfully, you can deploy the app to HoloLens.

Ensure the HoloLens device is powered on and connected to the PC through a USB cable. Make sure Device is the chosen deployment target, as above.

Right-click AoaSampleApp project, then select Deploy from the context menu to install the app. If no error shows up in Visual Studio's Output Window, the app will be installed on HoloLens.

Screenshot shows the context menu for the project with Deploy selected.

Before launching the app, you ought to have uploaded an object model, chair.ou for example, to the 3D Objects folder on your HoloLens. If you haven't, follow the instructions in the Upload your model section.

To launch and debug the app, select Debug > Start debugging.

Ingest object model and detect its instance

The AoaSampleApp app is now running on your HoloLens device. Walk close, within 2-meter distance, to the target object (chair) and scan it by looking at it from multiple perspectives. You should see a pink bounding box around the object with some yellow points rendered close to object's surface, which indicates that it was detected. You should also see a yellow box that indicates the search area.

Photograph shows a chair with the bounding box, point cloud, and search area.

You can define a search space for the object in the app by finger clicking in the air with either your right or left hand. The search space will switch among a sphere of 2-meters radius, a 4 m^3 bounding box and a view frustum. For larger objects such as cars, the best choice is usually to use the view frustum selection while standing facing a corner of the object at about a 2-meter distance. Each time the search area changes, the app removes instances currently being tracked. It then tries to find them again in the new search area.

This app can track multiple objects at one time. To do that, upload multiple models to the 3D Objects folder of your device and set a search area that covers all the target objects. It may take longer to detect and track multiple objects.

The app aligns a 3D model to its physical counterpart closely. A user can air tap using their left hand to turn on the high precision tracking mode, which computes a more accurate pose. This feature is still experimental. It consumes more system resources and could result in higher jitter in the estimated pose. Air tap again with the left hand to switch back to the normal tracking mode.

Next steps

FAQ