Development launchpad

Article categories

Overview

Mixed reality applications are built with the Universal Windows Platform (UWP). All mixed reality applications are UWP applications. All UWP applications can be made to run on Windows Mixed Reality devices. With Windows 10 and familiarity with middleware tools, such as Unity, you can build mixed reality experiences today.


Basics of mixed reality development

Mixed reality experiences are enabled by new Windows features for environmental understanding. These features enable developers to place a hologram in the real world, and let users move through digital worlds by literally walking about.

These are the core building blocks for mixed reality development:

InputHoloLens (1st gen)HoloLens 2 Immersive headsets
Head gaze✔️✔️✔️
Eye gaze✔️
Articulated hands✔️
Gestures✔️✔️
Voice✔️✔️✔️
Gamepad✔️✔️✔️
Motion controllers✔️
Perception and spatial featuresHoloLens (1st gen)HoloLens 2 Immersive headsets
World coordinates✔️✔️✔️
Spatial sound✔️✔️✔️
Spatial mapping✔️✔️

The basic interaction model for HoloLens is gaze, gesture, and voice, sometimes referred to as GGV. Windows Mixed Reality immersive headsets also use gaze and voice, but swap motion controllers for gestures.

All mixed reality devices benefit from the input ecosystem available in Windows, including mouse, keyboard, gamepads, and more. With HoloLens, hardware accessories are connected via Bluetooth. With immersive headsets, accessories connect to the host PC via Bluetooth, USB, and other supported protocols.

Environmental understanding features, such as coordinates, spatial sound, and spatial mapping provide the necessary capabilities for mixing reality. Spatial mapping is unique to HoloLens, and enables holograms to interact with both the user and the physical world around them. Coordinate systems allow the user's movement to affect movement in the digital world.

Holograms are made of light and sound that rely on rendering. Understanding the experience of placement and persistence, as demonstrated in the Windows Mixed Reality home, also known as the the "shell", is a great way to ground yourself in the user experience.

Tools for developing mixed reality

The tools you use will depend on the style of app you want to build.

  • Apps with a 2D view leverage tools for building UWP applications suited for environments like Windows Phone, PC, and tablets. These applications are experienced as 2D projections placed in the Windows Mixed Reality home, and work across multiple device types, including phone and PC.
  • Immersive and holographic applications require tools designed to take advantage of Windows Mixed Reality APIs. We recommend using Unity to build your mixed reality applications. Developers interested in building their own engine can use DirectX and other Windows APIs.

Regardless of the type of app you're building, these tools will facilitate your app development experience: