MRTK 101:如何使用混合现实工具包 Unity 进行常见的空间交互MRTK 101: How to use Mixed Reality Toolkit Unity for common spatial interactions

MRTK

了解如何使用 MRTK 在混合现实中实现最常用的交互模式。Learn about how to use MRTK to achieve some of the most widely used common interaction patterns in mixed reality.

  • 如何在 Unity 编辑器中模拟输入交互?How to simulate input interactions in Unity editor?
  • 如何抓取和移动对象?How to grab and move an object?
  • 如何调整对象的大小?How to resize an object?
  • 如何精确移动或旋转对象?How to move or rotate an object with precision?
  • 如何使对象响应输入事件?How to make an object respond to input events?
  • 如何添加视觉反馈?How to add visual feedback?
  • 如何添加音频反馈?How to add audio feedback?
  • 如何使用 HoloLens 2 样式按钮预制件?How to use HoloLens 2 style button prefabs?
  • 如何使对象跟随你?How to make an object follow you?
  • 如何使对象朝向你?How to make an object face you?

备注

本文已经过更新,反映了 MRTK v2.5.1 版本中的更改This article has been updated to reflect the changes in MRTK v2.5.1 release

本页中的所有内容都可在 Unity 中使用 MRTK 的输入模拟进行测试。All contents in this page can be tested in Unity editor with MRTK's Input Simulation. 如果你还没有,可按照 MRTK 安装指南 (GitHub) 安装最新版本的 MRTK。If you haven't, follow MRTK Installation Guide (GitHub) to install the latest version of MRTK.

如何在 Unity 编辑器中模拟输入交互?How to simulate input interactions in Unity editor?

MRTK 支持编辑器中的输入模拟。MRTK supports in-editor input simulation. 单击 Unity 的播放按钮以运行场景,然后使用以下按键来模拟输入:Run your scene by clicking Unity's play button, then use the following keys to simulate input:

  • 按 W、A、S、D 键可移动相机。Press W, A, S, D keys to move the camera.
  • 在按住鼠标右键的同时移动鼠标可以四处浏览。Hold the right mouse button and move the mouse to look around.
  • 按空格键(右手)或左 Shift 键(左手)以显示模拟双手Press Space bar(Right hand) or left Shift key(Left hand) to bring up the simulated hands
  • 按 T 或 Y 键以将模拟双手保持在视野中Press T or Y keys to keep simulated hands in view
  • 按 Q 或 E(水平)/R 或 F(垂直)来旋转模拟双手Press Q or E(horizontal) / R or F(vertical) to rotate simulated hands

可以在 MRTK 文档中详细了解输入模拟。You can learn more about Input Simulation in the MRTK documentation.

如何抓取和移动对象?How to grab and move an object?

附加 ObjectManipulator.cs 和 NearInteractionGrabbable.cs 脚本,使对象可抓取 。Attach the ObjectManipulator.cs and NearInteractionGrabbable.cs scripts to make an object grabbable. ObjectManipulator 支持近距离和远距离交互。ObjectManipulator supports both near and far interactions. 可以使用 HoloLens 2 的精确手动跟踪输入(近距)、手部射线(远距)、运动控制器光束(远距)、HoloLens 凝视光标和隔空敲击(远距)来抓取和移动对象。You can grab and move an object with HoloLens 2's articulated hand tracking input(near), hand ray(far), motion controller's beam(far), and HoloLens gaze cursor and air-tap(far).


NearInteractionGrabbable and ObjectManipulator.cs assigned to an object


NearInteractionGrabbable and ObjectManipulator.cs assigned to an object for grab and move

如何调整对象的大小?How to resize an object?

ObjectManipulator.cs 支持双手缩放/旋转。ObjectManipulator.cs supports two-handed scale/rotation. 此脚本适用于各种输入类型,例如 HoloLens 2 的精确手动输入、HoloLens 1 的凝视 + 手势输入,以及 Windows Mixed Reality 沉浸式头戴显示设备的运动控制器输入。The script works with various input types, such as HoloLens 2's articulated hand input, HoloLens 1's gaze + gesture input, and Windows Mixed Reality immersive headset's motion controller input.


NearInteractionGrabbable and ObjectManipulator.cs assigned to an object for manipulation

如何精确移动或旋转对象?How to move or rotate an object with precision?

将 BoundsControl.cs 分配到某个对象以使用边界框(用于缩放和旋转对象的界面)。Assign BoundsControl.cs to an object to use Bounding Box, which is the interface for scaling and rotating an object. 默认情况下,该脚本会显示 HoloLens 1 样式的蓝色控点和线条。By default, it shows HoloLens 1 style blue handles and wires. 若要使用基于 HoloLens 2 样式邻近性的动画控点,需要分配预制件和材料。To use HoloLens 2 style proximity-based animated handles, you need to assign prefabs and materials.


BoundsControl.cs assigned to an object image


BoundsControl.cs assigned to an object gif

如何使对象响应输入事件?How to make an object respond to input events?

将 PointerHandler.cs 分配到某个对象。Assign PointerHandler.cs to an object. 在检查器中,可使用事件 OnPointerDown()、OnPointerUp()、OnPointerClicked() 和 OnPointerDragged()。若要在脚本中使用这些事件,请实现 IMixedRealityPointerHandler。In the inspector, you can use events OnPointerDown(), OnPointerUp(), OnPointerClicked(), OnPointerDragged() To use these events in a script, implement IMixedRealityPointerHandler.


PointerHandler.cs assigned to an object image

如何添加视觉反馈?How to add visual feedback?

将 Interactable.cs 分配到某个对象。Assign Interactable.cs to an object. 在检查器中,添加目标对象并创建新主题。In the inspector, add target object and create a new theme. 使用 Interactable 的主题配置文件可以轻松将视觉反馈添加到所有可用的输入交互状态。Using Interactable's theme profiles, you can easily add visual feedback to all available input interaction states.


Image of PointerHandler.cs assigned to an object


Interactable gif

Interactable 提供各种类型的主题,包括着色器主题,该主题可用于控制每种交互状态的着色器的属性。Interactable provides various types of themes including the shader theme, which allows you to control properties of the shader per interaction state.

视觉反馈的另一个重要构建基块是 MRTK 标准着色器。Another important building block for visual feedback is the MRTK Standard Shader. 使用 MRTK 标准着色器可以轻松添加视觉反馈效果,例如悬停光线和邻近光线。With MRTK Standard Shader, you can easily add visual feedback effects such as hover light and proximity light. 由于 MRTK 标准着色器占用的计算资源要比 Unity 标准着色器少,因此你可以创建高性能的体验。Since MRTK Standard shader performs less computation than the Unity Standard shader, you can create a performant experience.

创建新材料,然后选择着色器的“混合现实工具包”>“标准”。Create a new material and select the Shader 'Mixed Reality Toolkit > Standard'. 或者,可以选择某个使用 MRTK 标准着色器的现有材料。Or you can pick one of the existing materials that use MRTK Standard Shader.


MRTK Standard Shader image 1

MRTK Standard Shader image 2

MRTK Standard Shader image 3

如何添加音频反馈?How to add audio feedback?

将 AudioSource 添加到某个对象。Add AudioSource to an object. 然后,在公开输入事件的脚本(例如 Interactable.cs 或 PointerHandler.cs)中,将带有 AudioSource 的对象分配到该事件,并选择 AudioSource.PlayOneShot()。Then, in the scripts that expose input events(e.g. Interactable.cs or PointerHandler.cs), assign the object with AudioSource to the event and select AudioSource.PlayOneShot(). 可以使用自己的音频剪辑,或从 MRTK 的音频资产中进行选择。You can use your audio clips or choose one from MRTK's audio assets.


Audio Source assigned to an object. AudioSource.PlayOneShot configured in the Interactable's OnPress() and OnRelease() events.

如何使用 HoloLens 2 样式按钮预制件?How to use HoloLens 2 style button prefabs?

MRTK 提供了各种类型的 HoloLens 2 shell (OS) 样式按钮,包括邻近感应光、压缩框和按钮表面的波纹效果等视觉反馈,可增强用户的信心。MRTK provides various types of HoloLens 2's shell (OS) style buttons, including visual feedback like proximity light, compressing box, and a ripple effect on the button surface that improve the user's confidence.


Interactable button

将某个 HoloLens 2 样式的可按按钮预制件拖放到场景中即可。Drag and drop one of the HoloLens 2 style pressable button prefab into your scene. 该预制件将使用前面介绍的 Interactable.cs。The prefab uses Interactable.cs introduced above. 可以使用 Interactable 中公开的事件(例如 OnClick())来触发操作。You can use exposed events such as OnClick() in the Interactable to trigger actions.


HoloLens 2 Button Prefab

如何使对象跟随你?How to make an object follow you?

将 RadialView.cs 或 Follow.cs 脚本分配到某个对象 。Assign RadialView.cs or Follow.cs script to an object. 此脚本是解算器脚本系列的一部分,可用于在 3D 空间中实现各种类型的对象定位。It's part of the Solver script series that allows you to achieve various types of object positioning in 3D space. 将自动添加 SolverHandler.cs。SolverHandler.cs will be automatically added. 下面是 RadialView 配置的一个示例,它可以实现“惰性跟踪”追随行为,就如同使用 HoloLens shell 中的“开始”菜单一样。Below is an example of RadialView configuration to achieve 'lazy follow' tag-along behavior just like the Start menu in the HoloLens shell. 可以指定最小/最大距离和最小/最大视图角度。You can specify the minimum/maximum distance and minimum/maximum view degrees. 以下示例演示如何在 0.4 到 0.8 米范围内以 15° 的视图角度定位对象。The example below shows positioning the object between 0.4 m and 0.8-m range within 15°. 调整“线性插值时间”值,以加快或减慢定位更新的速度。Adjust Lerp Time values to make the positional update faster or slower.


MRTK Standard Shader for solver


Interactable radial solver

如何使对象朝向你?How to make an object face you?

将 Billboard.cs 脚本分配到某个对象。Assign Billboard.cs script to an object. 不管你处于哪个位置,该对象都会朝向你。It will always face you, whatever your position. 可以指定枢轴选项。You can specify the pivot axis option.


Image of Billboard.cs script assigned to an object with Pivot Axis option Y


Billboard.cs script assigned to an object with Pivot Axis option Y

准备好创建令人惊叹的混合现实体验了吗?Ready to create amazing experiences for mixed reality? 请访问以下页面来详细了解 MRTK 和混合现实。Visit the pages below and learn more about MRTK and mixed reality.

关于作者About the author

Picture of Dong Yoon Park Dong Yoon ParkDong Yoon Park
用户体验设计师 @MicrosoftUX Designer @Microsoft

另请参阅See also