SpatialGestureRecognizer SpatialGestureRecognizer SpatialGestureRecognizer SpatialGestureRecognizer Class

Definition

Interprets user interactions from hands, motion controllers, and system voice commands to surface spatial gesture events, which users target using their gaze or a motion controller's pointing ray.

public : sealed class SpatialGestureRecognizer : ISpatialGestureRecognizer
public sealed class SpatialGestureRecognizer : ISpatialGestureRecognizer
Public NotInheritable Class SpatialGestureRecognizer Implements ISpatialGestureRecognizer
var spatialGestureRecognizer = new spatialGestureRecognizer(settings);
Attributes
Windows 10 requirements
Device family
Windows 10 (introduced v10.0.10586.0 - for Xbox, see UWP features that aren't yet supported on Xbox)
API contract
Windows.Foundation.UniversalApiContract (introduced v2)

Remarks

Spatial gestures are a key form of input for Mixed Reality headsets such as HoloLens. By routing interactions from the SpatialInteractionManager to a hologram's SpatialGestureRecognizer, apps can detect Tap, Hold, Manipulation, and Navigation events uniformly across hands, voice, and motion controllers.

Note that spatial gestures are not detected for input from gamepads, keyboards or mice.

SpatialGestureRecognizer performs only the minimal disambiguation between the set of gestures that you request. For example, if you request just Tap, the user may hold their finger down as long as they like and a Tap will still occur. If you request both Tap and Hold, after about a second of holding down their finger, the gesture will promote to a Hold and a Tap will no longer occur.

To use SpatialGestureRecognizer, handle the SpatialInteractionManager's InteractionDetected event and grab the SpatialPointerPose exposed there. Use the user's gaze ray from this pose to intersect with the holograms and surface meshes in the user's surroundings, in order to determine what the user is intending to interact with. Then, route the SpatialInteraction in the event arguments to the target hologram's SpatialGestureRecognizer, using its CaptureInteraction method. This starts interpreting that interaction according to the SpatialGestureSettings set on that recognizer at creation time or by TrySetGestureSettings.

When targeting a spatial interaction, such as a hand gesture, motion controller press or voice interaction, apps should choose a pointing ray available from the interaction's SpatialPointerPose, based on the nature of the interaction's SpatialInteractionSource:

  • If the interaction source does not support pointing (IsPointingSupported is false), the app should target based on the user's gaze, available through the Head property.
  • If the interaction source does support pointing (IsPointingSupported is true), the app may instead target based on the source's pointer pose, available through the TryGetInteractionSourcePose method.

The app should then intersect the chosen pointing ray with its own holograms or with the spatial mapping mesh to render cursors and determine what the user is intending to interact with.

Constructors

SpatialGestureRecognizer(SpatialGestureSettings) SpatialGestureRecognizer(SpatialGestureSettings) SpatialGestureRecognizer(SpatialGestureSettings) SpatialGestureRecognizer(SpatialGestureSettings)

Initializes a new SpatialGestureRecognizer with the specified gesture settings.

public : SpatialGestureRecognizer(SpatialGestureSettings settings)
public SpatialGestureRecognizer(SpatialGestureSettings settings)
Public Sub New(settings As SpatialGestureSettings)
var spatialGestureRecognizer = new spatialGestureRecognizer(settings);
Parameters
settings
SpatialGestureSettings SpatialGestureSettings SpatialGestureSettings SpatialGestureSettings

The gesture settings for the new recognizer.

Properties

GestureSettings GestureSettings GestureSettings GestureSettings

Gets the current SpatialGestureSettings for this recognizer.

public : SpatialGestureSettings GestureSettings { get; }
public SpatialGestureSettings GestureSettings { get; }
Public ReadOnly Property GestureSettings As SpatialGestureSettings
var spatialGestureSettings = spatialGestureRecognizer.gestureSettings;

Methods

CancelPendingGestures() CancelPendingGestures() CancelPendingGestures() CancelPendingGestures()

Cancels any in-progress gestures, abandoning any interactions that have been captured.

public : void CancelPendingGestures()
public void CancelPendingGestures()
Public Function CancelPendingGestures() As void
spatialGestureRecognizer.cancelPendingGestures();

CaptureInteraction(SpatialInteraction) CaptureInteraction(SpatialInteraction) CaptureInteraction(SpatialInteraction) CaptureInteraction(SpatialInteraction)

Subscribe this SpatialGestureRecognizer to track all input events that occur as part of this interaction.

public : void CaptureInteraction(SpatialInteraction interaction)
public void CaptureInteraction(SpatialInteraction interaction)
Public Function CaptureInteraction(interaction As SpatialInteraction) As void
spatialGestureRecognizer.captureInteraction(interaction);
Parameters
interaction
SpatialInteraction SpatialInteraction SpatialInteraction SpatialInteraction

The SpatialInteraction to capture from the SpatialInteractionManager's InteractionDetected event.

Remarks

To use SpatialGestureRecognizer, handle the SpatialInteractionManager's InteractionDetected event and get the SpatialPointerPose for that point in time by calling TryGetPointerPose on the event arguments. Grab the user's gaze ray from this SpatialPointerPose and intersect it with the holograms and surfaces in the user's surroundings, in order to determine what the user is intending to interact with. Then, route the SpatialInteraction in the event arguments to the target hologram's SpatialGestureRecognizer, using its CaptureInteraction method. This starts interpreting that interaction according to the SpatialGestureSettings set at creation time on that recognizer.

Note that you may choose to have more than one SpatialGestureRecognizer capture the same interaction. In this case, events will surface from both recognizers according to the SpatialGestureSettings of each. This can be useful if, for example, you have one hologram that supports Tap that is logically contained within a hologram that supports NavigationRailsY for scrolling, and you want to allow either gesture when gazing at the inner hologram.

TrySetGestureSettings(SpatialGestureSettings) TrySetGestureSettings(SpatialGestureSettings) TrySetGestureSettings(SpatialGestureSettings) TrySetGestureSettings(SpatialGestureSettings)

Attempt to change the gesture settings for this recognizer. This will only succeed if there are no gestures in progress at the time.

public : Platform::Boolean TrySetGestureSettings(SpatialGestureSettings settings)
public bool TrySetGestureSettings(SpatialGestureSettings settings)
Public Function TrySetGestureSettings(settings As SpatialGestureSettings) As bool
var bool = spatialGestureRecognizer.trySetGestureSettings(settings);
Parameters
settings
SpatialGestureSettings SpatialGestureSettings SpatialGestureSettings SpatialGestureSettings

The new SpatialGestureSettings to switch to.

Returns
Platform::Boolean bool bool bool

Whether the switch was successful.

Events

HoldCanceled HoldCanceled HoldCanceled HoldCanceled

Occurs when a Hold gesture is canceled.

public : event TypedEventHandler HoldCanceled<SpatialGestureRecognizer, SpatialHoldCanceledEventArgs>
public event TypedEventHandler HoldCanceled<SpatialGestureRecognizer, SpatialHoldCanceledEventArgs>
Public Event TypedEventHandler HoldCanceled( Of ( Of SpatialGestureRecognizer ), ( Of SpatialHoldCanceledEventArgs ))
function onHoldCanceled(eventArgs){/* Your code */}


spatialGestureRecognizer.addEventListener("holdCanceled", onHoldCanceled);
spatialGestureRecognizer.removeEventListener("holdCanceled", onHoldCanceled);

Remarks

For hand interactions, the SpatialHoldCanceledEventArgs event fires when the relevant hand that had a HoldStarted is lost during the gesture.

For voice interactions, this event does not fire.

For motion controllers, this event fires when the relevant controller that had a HoldStarted is lost during the gesture.

HoldCompleted HoldCompleted HoldCompleted HoldCompleted

Occurs when a Hold gesture completes.

public : event TypedEventHandler HoldCompleted<SpatialGestureRecognizer, SpatialHoldCompletedEventArgs>
public event TypedEventHandler HoldCompleted<SpatialGestureRecognizer, SpatialHoldCompletedEventArgs>
Public Event TypedEventHandler HoldCompleted( Of ( Of SpatialGestureRecognizer ), ( Of SpatialHoldCompletedEventArgs ))
function onHoldCompleted(eventArgs){/* Your code */}


spatialGestureRecognizer.addEventListener("holdCompleted", onHoldCompleted);
spatialGestureRecognizer.removeEventListener("holdCompleted", onHoldCompleted);

Remarks

For hand interactions, the SpatialHoldCompletedEventArgs event fires when the finger is released after HoldStarted.

For voice interactions, this event does not fire.

For motion controllers, this event fires when the Select trigger or button is released after HoldStarted.

HoldStarted HoldStarted HoldStarted HoldStarted

Occurs when an interaction becomes a Hold gesture.

public : event TypedEventHandler HoldStarted<SpatialGestureRecognizer, SpatialHoldStartedEventArgs>
public event TypedEventHandler HoldStarted<SpatialGestureRecognizer, SpatialHoldStartedEventArgs>
Public Event TypedEventHandler HoldStarted( Of ( Of SpatialGestureRecognizer ), ( Of SpatialHoldStartedEventArgs ))
function onHoldStarted(eventArgs){/* Your code */}


spatialGestureRecognizer.addEventListener("holdStarted", onHoldStarted);
spatialGestureRecognizer.removeEventListener("holdStarted", onHoldStarted);

Remarks

For hand interactions, the SpatialHoldStartedEventArgs event fires when a finger is held pressed for longer than the Hold time threshold.

For voice interactions, this event does not fire.

For motion controllers, this event fires when the Select trigger or button is held pressed for longer than the Hold time threshold.

ManipulationCanceled ManipulationCanceled ManipulationCanceled ManipulationCanceled

Occurs when a Manipulation gesture is canceled.

public : event TypedEventHandler ManipulationCanceled<SpatialGestureRecognizer, SpatialManipulationCanceledEventArgs>
public event TypedEventHandler ManipulationCanceled<SpatialGestureRecognizer, SpatialManipulationCanceledEventArgs>
Public Event TypedEventHandler ManipulationCanceled( Of ( Of SpatialGestureRecognizer ), ( Of SpatialManipulationCanceledEventArgs ))
function onManipulationCanceled(eventArgs){/* Your code */}


spatialGestureRecognizer.addEventListener("manipulationCanceled", onManipulationCanceled);
spatialGestureRecognizer.removeEventListener("manipulationCanceled", onManipulationCanceled);

Remarks

For hand interactions, the SpatialManipulationCanceledEventArgs event fires when the relevant hand that had a ManipulationStarted is lost during the gesture.

For voice interactions and motion controllers, this event does not fire.

ManipulationCompleted ManipulationCompleted ManipulationCompleted ManipulationCompleted

Occurs when a Manipulation gesture is completed.

public : event TypedEventHandler ManipulationCompleted<SpatialGestureRecognizer, SpatialManipulationCompletedEventArgs>
public event TypedEventHandler ManipulationCompleted<SpatialGestureRecognizer, SpatialManipulationCompletedEventArgs>
Public Event TypedEventHandler ManipulationCompleted( Of ( Of SpatialGestureRecognizer ), ( Of SpatialManipulationCompletedEventArgs ))
function onManipulationCompleted(eventArgs){/* Your code */}


spatialGestureRecognizer.addEventListener("manipulationCompleted", onManipulationCompleted);
spatialGestureRecognizer.removeEventListener("manipulationCompleted", onManipulationCompleted);

Remarks

For hand interactions, the SpatialManipulationCompletedEventArgs event fires when the finger is released after ManipulationStarted.

For voice interactions and motion controllers, this event does not fire.

ManipulationStarted ManipulationStarted ManipulationStarted ManipulationStarted

Occurs when an interaction becomes a Manipulation gesture.

public : event TypedEventHandler ManipulationStarted<SpatialGestureRecognizer, SpatialManipulationStartedEventArgs>
public event TypedEventHandler ManipulationStarted<SpatialGestureRecognizer, SpatialManipulationStartedEventArgs>
Public Event TypedEventHandler ManipulationStarted( Of ( Of SpatialGestureRecognizer ), ( Of SpatialManipulationStartedEventArgs ))
function onManipulationStarted(eventArgs){/* Your code */}


spatialGestureRecognizer.addEventListener("manipulationStarted", onManipulationStarted);
spatialGestureRecognizer.removeEventListener("manipulationStarted", onManipulationStarted);

Remarks

For hand interactions, the SpatialManipulationStartedEventArgs event fires when a finger is pressed and then moves outside of the small Manipulation deadzone.

For voice interactions and motion controllers, this event does not fire.

ManipulationUpdated ManipulationUpdated ManipulationUpdated ManipulationUpdated

Occurs when a Manipulation gesture is updated due to hand movement.

public : event TypedEventHandler ManipulationUpdated<SpatialGestureRecognizer, SpatialManipulationUpdatedEventArgs>
public event TypedEventHandler ManipulationUpdated<SpatialGestureRecognizer, SpatialManipulationUpdatedEventArgs>
Public Event TypedEventHandler ManipulationUpdated( Of ( Of SpatialGestureRecognizer ), ( Of SpatialManipulationUpdatedEventArgs ))
function onManipulationUpdated(eventArgs){/* Your code */}


spatialGestureRecognizer.addEventListener("manipulationUpdated", onManipulationUpdated);
spatialGestureRecognizer.removeEventListener("manipulationUpdated", onManipulationUpdated);

Remarks

For hand interactions, the SpatialManipulationUpdatedEventArgs event fires when the relevant hand that had a ManipulationStarted updates its position.

For voice interactions and motion controllers, this event does not fire.

NavigationCanceled NavigationCanceled NavigationCanceled NavigationCanceled

Occurs when a Navigation gesture is canceled.

public : event TypedEventHandler NavigationCanceled<SpatialGestureRecognizer, SpatialNavigationCanceledEventArgs>
public event TypedEventHandler NavigationCanceled<SpatialGestureRecognizer, SpatialNavigationCanceledEventArgs>
Public Event TypedEventHandler NavigationCanceled( Of ( Of SpatialGestureRecognizer ), ( Of SpatialNavigationCanceledEventArgs ))
function onNavigationCanceled(eventArgs){/* Your code */}


spatialGestureRecognizer.addEventListener("navigationCanceled", onNavigationCanceled);
spatialGestureRecognizer.removeEventListener("navigationCanceled", onNavigationCanceled);

Remarks

For hand interactions, the SpatialNavigationCanceledEventArgs event fires when the relevant hand that had a NavigationStarted is lost during the gesture.

For voice interactions, this event does not fire.

For motion controllers, this event fires when the relevant controller that had a NavigationStarted is lost during the gesture.

NavigationCompleted NavigationCompleted NavigationCompleted NavigationCompleted

Occurs when a Navigation gesture is completed.

public : event TypedEventHandler NavigationCompleted<SpatialGestureRecognizer, SpatialNavigationCompletedEventArgs>
public event TypedEventHandler NavigationCompleted<SpatialGestureRecognizer, SpatialNavigationCompletedEventArgs>
Public Event TypedEventHandler NavigationCompleted( Of ( Of SpatialGestureRecognizer ), ( Of SpatialNavigationCompletedEventArgs ))
function onNavigationCompleted(eventArgs){/* Your code */}


spatialGestureRecognizer.addEventListener("navigationCompleted", onNavigationCompleted);
spatialGestureRecognizer.removeEventListener("navigationCompleted", onNavigationCompleted);

Remarks

For hand interactions, the SpatialNavigationCompletedEventArgs event fires when the finger is released after NavigationStarted.

For voice interactions, this event does not fire.

For motion controllers, this event fires when the Select trigger or button is released after NavigationStarted.

NavigationStarted NavigationStarted NavigationStarted NavigationStarted

Occurs when an interaction becomes a Navigation gesture.

public : event TypedEventHandler NavigationStarted<SpatialGestureRecognizer, SpatialNavigationStartedEventArgs>
public event TypedEventHandler NavigationStarted<SpatialGestureRecognizer, SpatialNavigationStartedEventArgs>
Public Event TypedEventHandler NavigationStarted( Of ( Of SpatialGestureRecognizer ), ( Of SpatialNavigationStartedEventArgs ))
function onNavigationStarted(eventArgs){/* Your code */}


spatialGestureRecognizer.addEventListener("navigationStarted", onNavigationStarted);
spatialGestureRecognizer.removeEventListener("navigationStarted", onNavigationStarted);

Remarks

For hand interactions, the SpatialNavigationStartedEventArgs event fires when a finger is pressed and then moves outside of the small Navigation deadzone.

For voice interactions, this event does not fire.

For motion controllers, this event fires when the Select trigger or button is pressed and then the controller is moved outside of the small Navigation deadzone.

NavigationUpdated NavigationUpdated NavigationUpdated NavigationUpdated

Occurs when a Navigation gesture is updated due to hand or motion controller movement.

public : event TypedEventHandler NavigationUpdated<SpatialGestureRecognizer, SpatialNavigationUpdatedEventArgs>
public event TypedEventHandler NavigationUpdated<SpatialGestureRecognizer, SpatialNavigationUpdatedEventArgs>
Public Event TypedEventHandler NavigationUpdated( Of ( Of SpatialGestureRecognizer ), ( Of SpatialNavigationUpdatedEventArgs ))
function onNavigationUpdated(eventArgs){/* Your code */}


spatialGestureRecognizer.addEventListener("navigationUpdated", onNavigationUpdated);
spatialGestureRecognizer.removeEventListener("navigationUpdated", onNavigationUpdated);

Remarks

For hand interactions, the SpatialNavigationUpdatedEventArgs event fires when the relevant hand that had a NavigationStarted updates its position.

For voice interactions, this event does not fire.

For motion controllers, this event fires when the relevant controller that had a NavigationStarted is moved.

RecognitionEnded RecognitionEnded RecognitionEnded RecognitionEnded

Occurs when recognition of gestures is done, either due to completion of a gesture or cancellation. This is the last event to fire.

public : event TypedEventHandler RecognitionEnded<SpatialGestureRecognizer, SpatialRecognitionEndedEventArgs>
public event TypedEventHandler RecognitionEnded<SpatialGestureRecognizer, SpatialRecognitionEndedEventArgs>
Public Event TypedEventHandler RecognitionEnded( Of ( Of SpatialGestureRecognizer ), ( Of SpatialRecognitionEndedEventArgs ))
function onRecognitionEnded(eventArgs){/* Your code */}


spatialGestureRecognizer.addEventListener("recognitionEnded", onRecognitionEnded);
spatialGestureRecognizer.removeEventListener("recognitionEnded", onRecognitionEnded);

Remarks

For hand interactions, the SpatialRecognitionEndedEventArgs event fires on finger release.

For voice interactions, this event fires after a system voice command like "Select" has been processed.

For motion controllers, this event fires when the Select trigger or button is released.

This also fires if the relevant interaction source is lost during recognition.

RecognitionStarted RecognitionStarted RecognitionStarted RecognitionStarted

Occurs when recognition of gestures begins. This is the first event to fire.

public : event TypedEventHandler RecognitionStarted<SpatialGestureRecognizer, SpatialRecognitionStartedEventArgs>
public event TypedEventHandler RecognitionStarted<SpatialGestureRecognizer, SpatialRecognitionStartedEventArgs>
Public Event TypedEventHandler RecognitionStarted( Of ( Of SpatialGestureRecognizer ), ( Of SpatialRecognitionStartedEventArgs ))
function onRecognitionStarted(eventArgs){/* Your code */}


spatialGestureRecognizer.addEventListener("recognitionStarted", onRecognitionStarted);
spatialGestureRecognizer.removeEventListener("recognitionStarted", onRecognitionStarted);

Remarks

The SpatialRecognitionStartedEventArgs event fires when there is no active gesture and the gesture recognizer is told to capture an interaction. Note that events will only fire if the interaction could possibly trigger at least one of the gestures requested in the SpatialGestureSettings.

For hand interactions, this event fires on finger press.

For voice interactions, this event fires when a system voice command like "Select" is spoken.

For motion controllers, this event fires when the Select trigger or button is pressed.

Tapped Tapped Tapped Tapped

Occurs when a Tap or DoubleTap gesture is recognized.

public : event TypedEventHandler Tapped<SpatialGestureRecognizer, SpatialTappedEventArgs>
public event TypedEventHandler Tapped<SpatialGestureRecognizer, SpatialTappedEventArgs>
Public Event TypedEventHandler Tapped( Of ( Of SpatialGestureRecognizer ), ( Of SpatialTappedEventArgs ))
function onTapped(eventArgs){/* Your code */}


spatialGestureRecognizer.addEventListener("tapped", onTapped);
spatialGestureRecognizer.removeEventListener("tapped", onTapped);

Remarks

For hand interactions, the SpatialTappedEventArgs event fires on finger release after a finger press.

For voice interactions, this event fires after the system voice command "Select" has been processed.

For controllers, this event fires when the Select trigger or button is released after it was pressed.