OpenXR

OpenXR logo

OpenXR is an open royalty-free API standard from Khronos, providing engines with native access to a range of devices across the mixed reality spectrum.

You can develop using OpenXR on a HoloLens 2 or Windows Mixed Reality immersive headset on the desktop. If you don't have access to a headset, you can use the HoloLens 2 Emulator or the Windows Mixed Reality Simulator instead.

Why OpenXR?

With OpenXR, you can build engines that target both holographic devices, like HoloLens 2, and immersive devices, like Windows Mixed Reality headsets for desktop PCs. OpenXR lets you write code once that's then portable across a wide range of hardware platforms.

The OpenXR API uses a loader to connect your application directly to your headset's native platform support. End users get maximum performance and minimum latency, whether they're using a Windows Mixed Reality or any other headset.

What is OpenXR?

The OpenXR API provides the core pose prediction, frame timing, and spatial input functionality you'll need to build an engine that can target both holographic and immersive devices.

To learn about the OpenXR API, check out the OpenXR 1.0 specification, API reference, and quick reference guide. For more information, see the Khronos OpenXR page.

To target the full feature set of HoloLens 2, you'll also use cross-vendor and vendor-specific OpenXR extensions that enable additional features beyond the OpenXR 1.0 core, such as articulated hand tracking, eye tracking, spatial mapping, and spatial anchors. For more information, see the Roadmap section below on the extensions coming later this year.

OpenXR isn't itself a mixed reality engine. Instead, OpenXR enables engines like Unity and Unreal to write portable code once that can then access the native platform features of the user's holographic or immersive device, whatever vendor built that platform.

Roadmap

The OpenXR specification defines an extension mechanism that enables runtime implementers to expose additional functionality beyond the core features defined in the base OpenXR 1.0 specification.

There are three kinds of OpenXR extensions:

  • Vendor extensions (for example, MSFT): Enables per-vendor innovation in hardware or software features. Any runtime vendor can introduce and ship a vendor extension at any time.
    • Experimental vendor extensions (for example, MSFT_preview): Experimental vendor extensions being previewed to gather feedback. MSFT_preview extensions are for developer devices only and will be removed when the real extension ships. To experiment with them, you can enable preview extensions on your developer device.
  • Cross-vendor EXT extensions: Cross-vendor extensions that multiple companies define and implement. Groups of interested companies can introduce EXT extensions at any time.
  • Official KHR extensions: Official Khronos extensions ratified as part of a core spec release. KHR extensions are covered by the same license as the core spec itself.

The Windows Mixed Reality OpenXR Runtime supports a set of MSFT and EXT extensions that brings the full set of HoloLens 2 features to OpenXR applications:

Feature area Extension availability
Systems + sessions OpenXR 1.0 core spec:
XrInstance, XrSystemId, XrSession
Reference spaces (view, local, stage) OpenXR 1.0 core spec:
XrSpace
View configurations (mono, stereo) OpenXR 1.0 core spec:
XrView...
Swapchains + frame timing OpenXR 1.0 core spec:
XrSwapchain... + xrWaitFrame
Composition layers
(projection, quad)
OpenXR 1.0 core spec:
XrCompositionLayer... + xrEndFrame
Input and haptics OpenXR 1.0 core spec:
XrAction...
Direct3D 11 integration Official KHR extension released:
XR_KHR_D3D11_enable
Direct3D 12 integration Official KHR extension released:
XR_KHR_D3D12_enable
Unbounded reference space
(world-scale experiences)
MSFT extension released:
XR_MSFT_unbounded_reference_space
Spatial anchors MSFT extension released:
XR_MSFT_spatial_anchor
Hand interaction
(grip/aim pose, air-tap, grasp)

HoloLens 2 only

MSFT extension released:
XR_MSFT_hand_interaction
Hand articulation + hand mesh

HoloLens 2 only

EXT extension released in runtime 102:
XR_EXT_hand_tracking

MSFT extension released in runtime 102:
XR_MSFT_hand_tracking_mesh
Eye gaze

HoloLens 2 only

EXT extension released:
XR_EXT_eye_gaze_interaction
Interop with other HoloLens SDKs
(for example, QR)

HoloLens 2 only

MSFT extension released in runtime 102:
XR_MSFT_spatial_graph_bridge

MSFT extension in runtime 105:
XR_MSFT_perception_anchor_interop

Mixed Reality Capture
(third render from PV camera)

HoloLens 2 only

MSFT extensions released in runtime 102:
XR_MSFT_secondary_view_configuration
XR_MSFT_first_person_observer
Interop with UWP CoreWindow API
(for example, for keyboard/mouse)
MSFT extension released in runtime 103:
XR_MSFT_holographic_window_attachment
Motion controller interaction profiles
(Samsung Odyssey and HP Reverb G2)
MSFT extensions released in runtime 103:
XR_EXT_samsung_odyssey_controller
XR_EXT_hp_mixed_reality_controller
Motion controller render models MSFT extension released in runtime 104:
XR_MSFT_controller_model
Scene understanding (planes, meshes)

HoloLens 2 only

As of runtime 102:
Use XR_MSFT_spatial_graph_bridge with Scene Understanding SDK

MSFT extension in preview runtime 105:
XR_MSFT_scene_understanding_preview2
XR_MSFT_scene_understanding_serialization_preview

Composition layer reprojection modes
(auto-planar or orientation-only reprojection)
MSFT extension in preview runtime 105:
XR_MSFT_composition_layer_reprojection_preview
Other cross-vendor extensions

Official KHR extensions released:

XR_KHR_composition_layer_depth
XR_KHR_visibility_mask
XR_KHR_win32_convert_performance_counter_time
XR_KHR_composition_layer_color_scale_bias

EXT extensions released:

XR_EXT_win32_appcontainer_compatible
XR_EXT_debug_utils

While some of these extensions may start out as vendor-specific MSFT extensions, Microsoft and other OpenXR runtime vendors are working together to design cross-vendor EXT or KHR extensions for many of these feature areas. Cross-vendor extensions will make the code you write for those features portable across runtime vendors, as with the core specification.

Getting started with OpenXR

Screenshot of Minecraft being played by a user wearing a mixed reality headset

Minecraft's new RenderDragon engine has built its desktop VR support using OpenXR!

Microsoft has been working with Unity and Epic Games to ensure the future of mixed reality is open, not just for HoloLens 2, but across the full breadth of PC VR, including HP’s new Reverb G2 headset. OpenXR powers the cross-vendor VR support for major titles shipping today, such as Minecraft and Microsoft Flight Simulator! For more information on developing for HoloLens (1st gen), see the release notes.

To learn how you can get started with OpenXR in Unity, Unreal Engine or your own engine, read on!

OpenXR in Unity

Today, the supported Unity development path for HoloLens 2, HoloLens (1st gen), and Windows Mixed Reality headsets is Unity 2019 LTS with the existing WinRT API backend. You can jump into OpenXR with Unity; If you’re targeting the new HP Reverb G2 controller in a Unity 2019 app, see our HP Reverb G2 input docs.

Starting with Unity 2020 LTS, Unity ships an OpenXR backend that supports HoloLens 2 and Windows Mixed Reality headsets. This includes support for the OpenXR extensions that light up the full capabilities of HoloLens 2 and Windows Mixed Reality headsets, including hand/eye tracking, spatial anchors and HP Reverb G2 controllers. MRTK-Unity supports OpenXR as of MRTK 2.7. For more about the current status of Unity 2020 LTS support for HoloLens 2, see Choosing a Unity version.

Starting in Unity 2021, OpenXR will then graduate to be the only supported Unity backend for targeting HoloLens 2 and Windows Mixed Reality headsets.

OpenXR in Unreal Engine

As of Unreal Engine 4.23, full support for HoloLens 2 and Windows Mixed Reality headsets are available through the Windows Mixed Reality (WinRT) plugin.

Unreal Engine 4.23 was also the first major game engine release to ship preview support for OpenXR 1.0! Now in Unreal Engine 4.26, support for HoloLens 2, Windows Mixed Reality and other desktop VR headsets is available through Unreal Engine's built-in OpenXR support. Unreal Engine 4.26 also supports Microsoft's OpenXR extension plugin, enabling hand interaction and HP Reverb G2 controller support, lighting up the full feature set of HoloLens 2 and Windows Mixed Reality headsets. Unreal Engine 4.26 is available today on the Epic Games Launcher, with MRTK-Unreal support coming later in April.

OpenXR for native development

You can develop using OpenXR on a HoloLens 2 or Windows Mixed Reality immersive headset on the desktop. If you don't have access to a headset, you can use the HoloLens 2 Emulator or the Windows Mixed Reality Simulator instead.

To start developing OpenXR applications for HoloLens 2 or immersive Windows Mixed Reality headsets, see how to get started with OpenXR development.

For a tour through all the major components of the OpenXR API, along with examples of the real-world applications using OpenXR today, check out this 60-minute walkthrough video:

See also