Unreal Development Overview
Getting started with mixed reality applications is a big task. New concepts, platforms, and cutting edge hardware can seem like barriers. However, if you're an Unreal developer you're in luck. Support for Windows Mixed Reality (VR) and HoloLens 2 (AR) is now included in Unreal Engine's newest release. This update includes:
- Mixed Reality UX Tools plugin support
- OpenXR support
- App Remoting from a desktop app
- Better performance
- Mixed reality capture
- Initial support for Azure Spatial Anchors
If you're new to Unreal development don't jump in blind. Explore the Unreal tutorial series to get up to speed and look for assets and support in the Unreal marketplace and mixed reality forums. These resources are your links to the community of builders and problem solvers in todays mixed reality market.
Mixed Reality Toolkit for Unreal
The Mixed Reality Toolkit for Unreal is a set of components designed to speed up your development in Unreal. Each component includes plugins, samples, and documentation for setting up immersive experiences.
UX Tools for Unreal is the first component to be released and is currently only supported on HoloLens 2. The component plugin includes code, blueprints, and example assets of common UX features including:
- Input simulation
- Hand interaction actor
- Press-able button component
- Manipulator component
- Follow behavior component
You can dive into the UX Tools for Unreal GitHub repository for feature details and information on setting up your project.
HoloLens 2 platform support
If this is your first time creating or deploying an Unreal app for HoloLens, you'll need to download supporting platform support files from the Epic Launcher.
Building something with your own two hands is the best way to learn a new skill. Learning how to build and deploy a simple chess app for HoloLens 2 with the UX Tools plugin is a great way to start.
The end-to-end tutorial series provides hands-on contact with common interactive UX components and scenarios. You'll work through the project setup, adding interactions to the scene, and deploying to a device or emulator. All you need is Windows 10, an emulator, and Visual Studio 2019.
To debug an app running on HoloLens 2 with Visual Studio, follow the instructions here for debugging an installed UWP app on a remote device.
Developing for mixed reality comes with performance checkpoints that depend on the platform. A HoloLens 2 app must run at 60 frames per second for holograms to appear stable and responsive. Luckily, we have performance recommendations for achieving this in your Unreal applications.
Guides to specific features
There are several key features of mixed reality development that our tutorial series doesn't cover. Check out the following guides for details and practical applications:
- Eye tracking
- Hand tracking
- HoloLens camera
- Spatial anchors
- Spatial mapping
- Spatial audio
- Voice input
- QR codes
- Performance recommendations
|HoloLens 2 Feature||Earliest Supported Unreal Engine Version|
|Streaming from a PC||4.23|
|Hand and joint tracking||4.23|
|Spectator Screen support for streaming||4.24|
|Planar LSR over streaming||4.24|
|Sample apps (HoloLens2Example and Mission AR)||4.24|
|Mobile multi-View: Performance hits 60 fps||4.25|
|3rd camera render||4.25|
|Streaming from a packaged desktop app||4.25.1|
|Azure Spatial Anchors for HoloLens 2 (beta)||4.25|
|OpenXR support (beta)||4.25|
|UX Tools support (0.8)||4.25|
|Developer docs & tutorials||4.25|