Spatial anchors

A spatial anchor represents an important point in the world that the system keeps track of over time. Each anchor has a coordinate system that adjusts as needed, relative to other anchors or frames of reference, in order to ensure that anchored holograms stay precisely in place. Rendering a hologram in an anchor's coordinate system gives you the most accurate positioning for that hologram at any given time. This comes at the cost of small adjustments over time to the hologram's position as the system continually moves it back into place relative to the real world.

You can also persist and share spatial anchors across application sessions and across devices:

  • By saving local spatial anchors to disk and loading them back later, your application can calculate the same location in the real world across multiple application sessions on a single HoloLens.
  • By using Azure Spatial Anchors to create a cloud anchor, your application can share a spatial anchor across multiple HoloLens, iOS, and Android devices. By having each device render a hologram using the same spatial anchor, users will see the hologram appear at the same place in the real world. This allows for real-time shared experiences.
  • You can also use Azure Spatial Anchors for asynchronous hologram persistence across HoloLens, iOS and Android devices. By sharing a durable cloud spatial anchor, multiple devices can observe the same persisted hologram over time, even if those devices are not present together at the same time.

For standing-scale or room-scale experiences for tethered desktop headsets that will stay within a 5-meter diameter, you can typically use the stage frame of reference instead of spatial anchors, which provides you a single coordinate system in which to render all content. However, if your application intends to let users wander beyond 5 meters in HoloLens, perhaps operating throughout an entire floor of a building, you'll need spatial anchors to keep content stable.

While spatial anchors are great for holograms that should remain fixed in the world, once an anchor is placed, it can't be moved. There are alternatives to anchors that are more appropriate for dynamic holograms that tag along with the user. It is best to position dynamic holograms using a stationary frame of reference (the foundation for Unity's world coordinates) or an attached frame of reference.

Best practices

These spatial anchor guidelines will help you render stable holograms that accurately track the real world.

Create spatial anchors where users place them

Typically, users are the ones explicitly placing spatial anchors.

For example, on HoloLens, an application can intersect the user's gaze ray with the spatial mapping mesh to let the user decide where to place a hologram. When the user taps to place that hologram, create a spatial anchor at the intersection point, and then place the hologram at the origin of that anchor's coordinate system.

Local spatial anchors are easy and performant to create. The system will consolidate their internal data if multiple anchors can share their underlying sensor data. You should typically create a new local spatial anchor for each hologram that a user explicitly places, except in cases outlined below, such as rigid groups of holograms.

Always render anchored holograms within 3 meters of their anchor

Spatial anchors stabilize their coordinate system near the anchor's origin. If you render holograms more than 3 meters from that origin, those holograms might experience noticeable positional errors in proportion to their distance from that origin due to lever-arm effects. This works if the user stands near the anchor, since the hologram is far away from the user, too. In other words, the angular error of the distant hologram will be small. However, if the user walks up to that distant hologram, it will be large in their view, making the lever-arm effects from the faraway anchor origin quite obvious.

Group holograms that should form a rigid cluster

Multiple holograms can share the same spatial anchor if the application expects those holograms to maintain fixed relationships with one another.

For example, if you are animating a holographic solar system in a room, it's better to tie all of the solar system objects to a single anchor in the center so that they move smoothly relative to each other. In this case, it is the solar system as a whole that is anchored, even though its component parts are moving dynamically around the anchor.

The key caveat to maintain hologram stability is to follow the 3-meter rule above.

Render highly dynamic holograms using the stationary frame of reference instead of a local spatial anchor

If you have a highly dynamic hologram, such as a character walking around a room or a floating UI that follows along the wall near the user, it is best to skip local spatial anchors, and render those holograms directly in the coordinate system provided by the stationary frame of reference. I Unity, you achieve this by placing holograms directly in world coordinates without a WorldAnchor. Holograms in a stationary frame of reference might experience drift when the user is far from the hologram. But this is less likely to be noticeable for dynamic holograms: either the hologram is constantly moving anyway or its motion constantly keeps it close to the user where drift will be minimized.

One interesting case of dynamic holograms is an object that is animating from one anchored coordinate system to another. For example, you might have two castles 10 meters apart, each on their own spatial anchor with one castle firing a cannonball at the other castle. At the moment the cannonball is fired, you can render it at the appropriate location in the stationary frame of reference to coincide with the cannon in the first castle's anchored coordinate system. It can then follow its trajectory in the stationary frame of reference as it flies 10 meters through the air. As the cannonball reaches the other castle, you can choose to move it into the second castle's anchored coordinate system to allow for physics calculations with that castle's rigid bodies.

If you are sharing a highly dynamic hologram across devices, you will need to pick some cloud spatial anchor to act as their parent because stationary frames of reference cannot be shared across devices. However, in this case, you should ensure that either the dynamic hologram or the devices viewing it remain within the anchor's 3 meter radius to ensure the hologram appears stable on all devices.

Avoid creating a grid of spatial anchors

You may be tempted to have your application drop a regular grid of spatial anchors as the user walks around, transitioning dynamic objects from anchor to anchor as they move around. However, this involves a lot of management for your application, without the benefit of the deep sensor data that the system itself maintains internally. For these cases, you will usually achieve better results with less effort by placing your holograms in the stationary frame of reference as described in the section above. When pre-positioning a set of cloud spatial anchors around a static space, consider placing the spatial anchors at the locations of the key holograms the user encounters per the principle above rather than creating an arbitrary grid of anchors. This ensures that you'll get maximum stability for those key holograms.

Release local spatial anchors you no longer need

While a local spatial anchor is active, the system prioritizes retaining the sensor data that is near that anchor. If you are no longer using a spatial anchor, stop accessing its coordinate system. This allows its underlying sensor data to be removed as necessary.

This is especially important for local anchors you have persisted to the spatial anchor store. The sensor data behind these anchors will be kept around permanently to allow your application to find that anchor in future sessions, which reduces the space available to track other anchors. Persist only those local anchors that you need to find again in future sessions, and remove them from the store when they are no longer meaningful to the user.

For cloud spatial anchors, your storage can scale as your scenario requires. You can store as many cloud anchors as needed, releasing them only when you know that your users will not need to locate holograms at that anchor again.

See also