Mixed reality resources

  • 10 minutes

Resources are collections of valuable and informative content, articles, and guides that assist you in achieving a specific goal. Now that you understand the fundamental concepts of mixed reality design, it's time to equip yourself with the tools and resources to start your design journey. There are numerous resources available for mixed reality development and design. Here, you'll be learning about a few extensively used resources.

Unity

Unity is the most widely used game engine in the world. It has many features and is adaptable enough to create practically any game you can think of. Unity is popular among both amateurs and AAA studios due to its unparalleled cross-platform functionality. Pokemon Go, Hearthstone, Rimworld, Cuphead, and other games have all used Unity as their development engine.

Programmers love it because of the built-in Visual Studio integration and C# scripting API. For developers looking for an alternative to Visual Studio, Unity also offers JavaScript as a scripting language and MonoDevelop as IDE. On the other hand, artists prefer Unity since it comes with solid animation tools that are easy to use to create 3D sequences and 2D animations from scratch. In Unity, almost anything can be animated.

Additionally, Unity 3D offers a version called "Unity Personal" that creators can use for free as long as any apps they create from it make less than $100,000. Unity offers certain extra features and a flexible license plan under a tiered subscription model for those willing to pay. Users with a premium subscription can access Unity's source code and development support. You can download unity with this link9.

Features of Unity

  • Unity is a game creation environment that combines artist-friendly tools with a component-driven design that makes game development a breeze.
  • Unity takes a component-based, prefab-based approach to game development. Prefabs allow game designers to create items and environments more quickly and efficiently.
  • Unity can produce stunning graphics across the board thanks to its powerful shaders, physics-based materials, post-processing, and high-resolution lighting systems.
  • Today's game developers value cross-platform deployment, and Unity excels in this area. You can play Unity games on almost any platform due to powerful console and operating system compatibility.
  • You can manage mouse, keyboard, and game controller inputs using Unity's editor tools.

Other game engines for mixed reality development

If you've already decided to use Unity, go ahead and jump right into it. However, if you're on the fence or just starting, you can look through other available platform and tool options, understand what they offer, and start creating!

  • Unreal Engine: The Unreal Engine10 is a robust open-source development engine with complete mixed reality support in C++ and Blueprints. HoloLens support is now fully functional and ready for development in Unreal Engine 4.25. Designers may virtually access the full range of concepts and techniques, only available to programmers, due to features like the versatile Blueprints and Visual Scripting framework. Creators in various fields can use the freedom and control to create cutting-edge content, immersive virtual worlds, and interactive experiences.

  • Native OpenXR: OpenXR can be used to create a custom engine by native developers that have experience creating their own 3D renderers. OpenXR is a royalty-free API standard from Khronos that gives engines native access to a wide range of mixed reality devices from various vendors. For example, you can use OpenXR to build on a HoloLens 2 or Windows Mixed Reality immersive headset on a PC. ⁠StereoKit11 is an easy-to-use open source mixed reality library for building HoloLens and VR applications with C# and OpenXR.

  • WebXR: Web developers can use WebXR12 to create compelling cross-browser AR/VR web experiences. Babylon is a JavaScript 3D engine that makes developing 3D content and immersive applications easy.

Mixed Reality Toolkit (MRTK)

MRTK is a Microsoft-led open-source project that provides a set of components and functionalities to speed up your creation of cross-platform MR apps in Unity. You'll be able to find crucial building blocks for spatial interactions and UI for mixed reality that have been designed and polished by Microsoft's mixed reality design and engineering team.

There's no one-size-fits-all audience for MRTK. It has been developed to accommodate various use cases, from first-time hackathons to individuals creating complicated and shared workplace experiences. Some code and APIs may be created to favor one over the other. For example, some elements of the MRTK appear to be more optimized for "one-click configure." It's important to note that some of these are due to historical and resource reasons. As MRTK develops, added features will be scalable to suit various use cases.

MRTK also has to scale seamlessly across VR and AR scenarios. Creating applications that seamlessly fall back in behavior should be straightforward, whether launched on HoloLens 2 or HoloLens 1. Additionally, it should be simple to create OpenVR, Windows Mixed Reality (WMR) applications, and apps on other platforms. While the team may choose to focus a particular iteration on a specific system or platform, the long-term goal is to provide a wide range of support for users creating mixed reality experiences.

MRTK includes various example scenes that demonstrate the functionality and building elements of the toolkit for spatial user experience. Experimenting with and examining example scenes might help you learn and apply features to your projects. Visit Mixed Reality Toolkit Documentation8 to learn more.

MRTK offers the following features:

  • Provides a cross-platform input mechanism and building blocks for spatial interactions and user interface design.
  • Rapid prototyping is possible thanks to an in-editor simulation that allows you to see changes right away.
  • MRTK functions as an extensible framework, allowing developers to swap out core components.
  • Supports various platforms, including: OpenXR (Unity 2020.3.8+), Windows Mixed Reality, Oculus (Unity 2019.3 or newer), OpenVR, Ultraleap Hand Tracking, and Mobile.

MRTK Figma Toolkit

The MRTK Figma Toolkit includes 2D versions of the Microsoft HoloLens 2 style UI controls seen in the MRTK for Unity and MRTK for Unreal toolkits. These can be used to make UI layouts and storyboards. For example, after you create a flat layout with MRTK's components, you can experiment with depth (z-position, rotation, and so on) in Unity or Unreal. Visit the MRTK Figma Toolkit Documentation13 to learn more.

Most of the controls in this Figma file's UI Gallery are a 2D replica of controls available as prefabs in MRTK for Unity. These prefabs are available in the MRTK Toolbox. Since the building components are visually similar, you or your Unity expert can easily recreate any layouts you design here in Unity.

MRTK Figma Bridge for Unity

The MRTK Figma Bridge for Unity allows you to import Figma Toolkit layouts into Unity. The bridge may import a UI layout developed using the MRTK Figma Toolkit and then create the corresponding MRTK prefabs with the correct size and position. Figma Bridge will aid design integration and collaboration between designers and developers.

Learn more about MRTK Figma Bridge for Unity14.

Reference

9-14 See the Additional reading section at the end of this learn module.