Development overview

Mixed reality apps are built with the Universal Windows Platform. All mixed reality apps are Universal Windows apps, and all Universal Windows apps can be made to run on Windows Mixed Reality devices. With Windows 10 and familiarity with middleware tools like Unity, you can start building mixed reality experiences today.

Basics of mixed reality development

Mixed reality experiences are enabled by new Windows features for environmental understanding. These enable developers to place a hologram in the real world, and allow users to move through digital worlds by literally walking about.

These are the core building blocks for mixed reality development:

Input HoloLens Immersive headsets
Gaze✔️✔️
Gestures✔️
Voice✔️✔️
Gamepad✔️✔️
Motion controllers✔️
Perception and spatial features HoloLens Immersive headsets
World coordinates✔️✔️
Spatial sound✔️✔️
Spatial mapping✔️

The basic interaction model for HoloLens is gaze, gesture, and voice, sometimes referred to as GGV. Windows Mixed Reality immersive headsets also use gaze and voice, but swap motion controllers for gestures.

All mixed reality devices benefit from the input ecosystem available to Windows, including mouse, keyboard, gamepads, and more. With HoloLens, hardware accessories are connected via Bluetooth. With immersive headsets, accessories connect to the host PC via Bluetooth, USB, and other supported protocols.

The environmental understanding features like coordinates, spatial sound, and spatial mapping provide the necessary capabilities for mixing reality. Spatial mapping is unique to HoloLens, and enables holograms to interact with both the user and the physical world around them. Coordinate systems allow the user's movement to affect movement in the digital world.

Holograms are made of light and sound, which rely on rendering. Understanding the experience of placement and persistence, as demonstrated in the Windows Mixed Reality home (sometimes called the "shell") is a great way ground yourself in the user experience.

Tools for developing for mixed reality

The tools you use will depend on the style of app you want to build.

  • Apps with a 2D view leverage tools for building Universal Windows Platform apps suited for environments like Windows Phone, PC, and tablets. These apps are experienced as 2D projections placed in the Windows Mixed Reality home, and can work across multiple device types (including phone and PC).
  • Immersive and holographic apps need tools designed to take advantage of the Windows Mixed Reality APIs. We recommend using Unity to build mixed reality apps. Developers interested in building their own engine can use DirectX and other Windows APIs.

Regardless of the type of app you're building, these tools will facilitate your app development experience:

See also