Unreal Development Overview

Getting started with mixed reality applications is a big task. New concepts, platforms, and cutting edge hardware can seem like barriers. However, if you're an Unreal developer you're in luck. Support for Windows Mixed Reality (VR) and HoloLens 2 (AR) is now included in Unreal Engine's newest release. This update includes:

  • Mixed Reality UX Tools plugin support
  • OpenXR support
  • App Remoting from a desktop app
  • Better performance
  • Mixed reality capture
  • Initial support for Azure Spatial Anchors

If you're new to Unreal development don't jump in blind. Explore the Unreal tutorial series to get up to speed and look for assets and support in the Unreal marketplace and mixed reality forums. These resources are your links to the community of builders and problem solvers in todays mixed reality market.

Mixed Reality Toolkit for Unreal

The Mixed Reality Toolkit for Unreal is a set of components designed to speed up your development in Unreal. Each component includes plugins, samples, and documentation for setting up immersive experiences.

UX Tools for Unreal is the first component to be released and is currently only supported on HoloLens 2. The component plugin includes code, blueprints, and example assets of common UX features including:

  • Input simulation
  • Hand interaction actor
  • Press-able button component
  • Manipulator component
  • Follow behavior component

You can dive into the UX Tools for Unreal GitHub repository for feature details and information on setting up your project.

HoloLens 2 platform support

If this is your first time creating or deploying an Unreal app for HoloLens, you'll need to download supporting platform support files from the Epic Launcher.

Tutorial

Building something with your own two hands is the best way to learn a new skill. Learning how to build and deploy a simple chess app for HoloLens 2 with the UX Tools plugin is a great way to start.

The end-to-end tutorial series provides hands-on contact with common interactive UX components and scenarios. You'll work through the project setup, adding interactions to the scene, and deploying to a device or emulator. All you need is Windows 10, an emulator, and Visual Studio 2019.

Debugging

To debug an app running on HoloLens 2 with Visual Studio, follow the instructions here for debugging an installed UWP app on a remote device.

Performance

Developing for mixed reality comes with performance checkpoints that depend on the platform. A HoloLens 2 app must run at 60 frames per second for holograms to appear stable and responsive. Luckily, we have performance recommendations for achieving this in your Unreal applications.

Guides to specific features

There are several key features of mixed reality development that our tutorial series doesn't cover. Check out the following guides for details and practical applications:

Supported Features

HoloLens 2 Feature Earliest Supported Unreal Engine Version
ARM64 support 4.23
Streaming from a PC 4.23
Spatial mapping 4.23
Hand and joint tracking 4.23
Eye tracking 4.23
Voice input 4.23
Spatial anchors 4.23
Camera access 4.23
QR codes 4.23
Spatial audio 4.23
Spectator Screen support for streaming 4.24
Planar LSR over streaming 4.24
Sample apps (HoloLens2Example and Mission AR) 4.24
Mobile multi-View: Performance hits 60 fps 4.25
3rd camera render 4.25
Streaming from a packaged desktop app 4.25.1
Azure Spatial Anchors for HoloLens 2 (beta) 4.25
OpenXR support (beta) 4.25
UX Tools support (0.8) 4.25
Developer docs & tutorials 4.25

See also