Gaze in Unity

Gaze is a primary way for users to target the holograms your app creates in mixed reality.

Implementing Gaze

Conceptually, gaze is implemented by projecting a ray from the user's head where the headset is, in the forward direction they are facing and determining what that ray collides with. In Unity, the user's head position and direction are exposed through the Unity Main Camera, specifically UnityEngine.Camera.main.transform.forward and UnityEngine.Camera.main.transform.position.

Calling Physics.RayCast results in a RaycastHit structure which contains information about the collision including the 3D point where collision occurred and the other GameObject the gaze ray collided with.

Example: Implement Gaze

void Update()
{
       RaycastHit hitInfo;
       if (Physics.Raycast(
               Camera.main.transform.position,
               Camera.main.transform.forward,
               out hitInfo,
               20.0f,
               Physics.DefaultRaycastLayers))
       {
           // If the Raycast has succeeded and hit a hologram
           // hitInfo's point represents the position being gazed at
           // hitInfo's collider GameObject represents the hologram being gazed at
       }
}

Best Practices

While the example above demonstrates how to do a single raycast in an update loop to find the Gaze target, it is recommended to do this in a single object managing gaze instead of doing this in any object that is potentially interested in the object being gazed at. This lets your app save processing by doing just one gaze raycast each frame.

Visualizing Gaze

Just like on the desktop where you use a mouse pointer to target and interact with content, you should implement a cursor that represents the user's gaze. This gives the user confidence in what they're about to interact with.

Gaze in Mixed Reality Toolkit

When you import MRTK release Unity packages or clone the project from the GitHub repository, you are going to find a new menu 'Mixed Reality Toolkit' in Unity. Under 'Configure' menu, you will see the menu 'Apply Mixed Reality Scene Settings'. When you click it, it removes the default camera and adds foundational components - InputManager, MixedRealityCameraParent, and DefaultCursor.

MRTK Menu for scene setup
MRTK Menu for scene setup

Automatic scene setup in MRTK
Automatic scene setup in MRTK

Mixed Reality Toolkit's InputManager prefab includes GazeManager.cs and Gaze Stabilizer. Under SimpleSinglePointerSelector, you can assign your custom Cursor. In default, animated DefaultCursor is assigned.

Cursor.prefab and CursorWithFeedback.prefab shows you how to visualize your Gaze using Cursors.

See also