Voice commands and gestures for Dynamics 365 Remote Assist on HoloLens

Tip

Dynamics 365 Remote Assist customers now have access to Remote Assist in Microsoft Teams mobile. Take advantage of all that Teams has to offer and collaborate with other users! For the best calling experience on HoloLens, use Dynamics 365 Guides.

With Guides, the same remote experts can assist you through a problem while you have the holographic content of a Guide appearing before you.

Learn more and try Guides!

You can navigate Dynamics 365 Remote Assist on HoloLens and HoloLens 2 using voice commands and gaze instead of gestures. Some voice commands are contextual, so they only work in particular areas of the user interface. For example, “Start inking” only works if the Draw tool is already active. You can also gaze at any button and say “Select.”

At this time, voice commands and speech recognition are available in specific languages for Hololens.

When you’re in a call, a good way to learn the features of Dynamics 365 Remote Assist is to say, “Remote Assist” without saying a follow-up command. The app displays what it believes you said by showing a check mark and “Remote assist.” Then the app displays the name of each button. After a few seconds, the labels automatically disappear.

When you’re outside a call (for example, before you’ve picked up, or when you’re looking at a list of contacts but have not called anyone), you can also say “Remote Assist” to remind yourself of the voice command for each button you see. The labels will pop up for a few seconds, and then disappear. However, you won't receive confirmation of your voice command.

Voice commands

The table below provides the full list of voice commands that you can use with Dynamics 365 Remote Assist on HoloLens.

To use a voice command, say “Remote Assist”, and then say one of the voice commands. For example, say "Remote Assist, Move" to enter Move mode for a window. You can also gaze at any button and say “Remote Assist, Select” to select that button.

Voice command	Description
General
Remote Assist	If you say “Remote Assist” without following it with a command, displays labels with the name of each button. After a few seconds, the labels automatically disappear.
Start listening	Put Dynamics 365 Remote Assist into state expecting a list of commands to follow.
Stop listening	Remove Dynamics 365 Remote Assist from state expecting a list of commands to follow.
Next	Go to next page.
Cancel	Close a dialog box without taking any action.
Undo	Undo the previous action.
Call management
Call	Call a contact.
Launch call	Launch a call from the calling confirmation dialog box.
Search	Display the Search box to search for a contact.
Retry	Retry searching the Contacts list.
Video	Answer an incoming call with video.
Audio	Answer an incoming call with audio only.
Hang up	End a call (including ending a ringing outgoing call).
Mute	Mute an audio call.
Unmute	Unmute an audio call.
Record call	Start video recording.
Stop recording	Stop video recording.
Camera	Select the Camera/Capture tool.
Start video	Turn on the video for a call.
Stop video	Stop the video for a call.
Turn on snapshot	Turn on Snapshot mode for a call in low-bandwidth mode.
Turn off snapshot	Turn off Snapshot mode for a call in low-bandwidth mode.
Take a snapshot	Take a snapshot when the Snapshot dialog box appears for low-bandwidth mode.
Join meeting	Join a Microsoft Teams meeting.
Open participants	Open the participants list.
Invite	Confirm inviting a participant to join a call.
Close participants	Close the participants list.
Next meeting	Show the next meeting notification.
Previous meeting	Show the previous meeting notification.
Narrator on	Turn the narrator on to have HoloLens announce the name of an incoming caller.
Narrator off	Turn off the narrator.
Dictation	Turn on dictation for text input in a chat.
Window management
Select slate	Select the window you're currently gazing at so you can move or resize it.
Move	Select the Move tool to move a window.
Accept	Accept the placement of a moved window.
There	Accept movement of the slate, and lock it to specific location.
Place here	Place the slate at specific location.
Cancel	Cancel placement of a moved window.
Reset	Reset the window to the previous placement.
Scale	Select the Scale tool to resize a window (PDF document, for example). It does not resize an annotation or ink.
Scale off	Turn off the Scale tool.
Bigger	When the Scale tool is active, make the current window bigger.
Smaller	When the Scale tool is active, make the current window smaller.
Close	Close the open window.
Pin	Pin a window in your space so it doesn’t follow you as you move around in your environment.
Unpin	Unpin a window so it follows you as you move around in your environment.
Follow me	Set the panels to follow the user.
Open chat	Open the Text Chat window (attached to the video feed).
Close chat	Close the Text Chat window.
Dock	Dock the text chat during a call.
Undock	Undock the text chat during a call.
Annotations
Arrow	Select the Arrow tool to create a holographic arrow. Use arrows to point to parts of your environment.
Arrow off	Turn off the Arrow tool.
Arrow	Place an arrow at the current gaze cursor location.
Ink	Select the Draw tool to draw on the walls and surfaces of your environment.
Ink off	Turn off the Draw tool.
Start inking	Start adding ink.
Stop inking	Stop adding ink.
Colors	Open color picker to determine color of the arrow or ink, and then say "blue", "gray", "green", "red", or "yellow".
Undo	Undo the previous action.
Erase all	Remove all annotations that you placed in the environment.
Toggle hand rays	Turn hand rays on or off. Note that turning hand rays off disables far-field interactions. Turning hand rays back on re-enables far-field interactions. Activating any tools that require far-field interactions (inking and arrows, for example) also automatically turns hand rays back on.
Hand rays on	Show hand rays.
Hand rays off	Hide hand rays. Note that turning hand rays off disables far-field interactions. Turning hand rays back on re-enables far-field interactions. Activating any tools that require far-field interactions (inking and arrows, for example) also automatically turns hand rays back on.
Capture photos or videos
Snap, Take picture, Take photo, Capture	Take a photo of the environment.
Take photo	Select the Take Photo button.
Cancel	Cancel the photo or video capture.
Photo	Enable the image capture tool.
Photo off	Disable the image capture tool.
Retake snapshot	Retake the photo during Snapshot mode.
Save to chat	During a call when the Snapshot box is active, save the snapshot to the chat.
Record video	Record a video capture of the environment.
End recording	Stop the video capture.
Video off	Disable the Video capture tool.
Save to booking	Save the captured photo to a Dynamics 365 Field Service booking.
Save to OneDrive	Save the captured photo to OneDrive.
Dismiss	Select the Dismiss button in the poor connection dialog box.
OneDrive file management
Files	Switch to the Files tab to add a file from OneDrive to your environment.
Home	When the Files tab is open, go to the root OneDrive folder.
Up	When the Files tab is open, scroll up in the file directory.
Back	When the Files tab is open, go back.
Previous page	Go to the previous page in OneDrive.
Next page	Go to the next page in OneDrive.
Previous	When viewing a PDF file, go back to the previous page.
Next	When viewing a PDF file, go to the next page.
Set page	Set the page of the open PDF file.
Close	Close OneDrive.
Asset management
New asset	Add a new asset.
Rename	Rename an asset.
Submit	Submit a new name when renaming an asset.
Post, Post all	Post the Dynamics 365 Field Service items to the work order.
Discard, Discard all	Discard the Dynamics 365 Field Service items at the end of a call.

Use the Narrator to announce incoming calls

You can turn on the Narrator if you want to take advantage of voice narration for incoming calls. When you turn on incoming call narration, in addition to displaying the incoming caller's name, the caller's name is announced. You can then say "Remote Assist, Video," "Remote Assist, Audio," or "Remote Assist, Ignore," to respond to the incoming call.

To turn on the Narrator for incoming calls:

• Go to Settings, and then select the On button for the Narrator.

Gestures

The April 2021 release of Dynamics 365 Remote Assist improved gestures on HoloLens 2 to make them more instinctual:

• Improved audio and visual cues make it easier to grab, move, and resize windows, whether you’re directly manipulating windows up close or manipulating them from a distance with hand rays.

• Improvements to the near finger cursor make it easier to grab windows and select buttons.