MR 输入 213:运动控制器MR Input 213: Motion controllers


混合现实学院教程在制作时考虑到了 HoloLens(第一代)和混合现实沉浸式头戴显示设备。The Mixed Reality Academy tutorials were designed with HoloLens (1st gen) and Mixed Reality Immersive Headsets in mind. 因此,对于仍在寻求这些设备的开发指导的开发人员而言,我们觉得很有必要保留这些教程。As such, we feel it is important to leave these tutorials in place for developers who are still looking for guidance in developing for those devices. 我们 不会 在这些教程中更新 HoloLens 2 所用的最新工具集或集成相关的内容。These tutorials will not be updated with the latest toolsets or interactions being used for HoloLens 2. 我们将维护这些教程,使之持续适用于支持的设备。They will be maintained to continue working on the supported devices. 已经为 HoloLens 2 发布了一系列新教程A new series of tutorials has been posted for HoloLens 2.

混合现实世界中的运动控制器增加了另一层的交互性。Motion controllers in the mixed reality world add another level of interactivity. 借助 运动控制器,我们可以以更自然的方式与对象直接交互,类似于真实生活中的物理交互,增加浸入式和感到满意的应用体验。With motion controllers, we can directly interact with objects in a more natural way, similar to our physical interactions in real life, increasing immersion and delight in your app experience.

在 MR 输入213中,我们将通过创建一个简单的空间绘制体验来浏览运动控制器的输入事件。In MR Input 213, we will explore the motion controller's input events by creating a simple spatial painting experience. 对于此应用程序,用户可以在具有各种类型的画笔和颜色的三维空间中进行绘制。With this app, users can paint in three-dimensional space with various types of brushes and colors.

本教程中介绍的主题Topics covered in this tutorial

MixedReality213 Topic1 MixedReality213 Topic2 MixedReality213 Topic3
控制器可视化Controller visualization 控制器输入事件Controller input events 自定义控制器和 UICustom controller and UI
了解如何在 Unity 的游戏模式和运行时中呈现运动控制器模型。Learn how to render motion controller models in Unity's game mode and runtime. 了解不同类型的按钮事件及其应用程序。Understand different types of button events and their applications. 了解如何在控制器顶部覆盖 UI 元素或对其进行完全自定义。Learn how to overlay UI elements on top of the controller or fully customize it.

设备支持Device support

课程Course HoloLensHoloLens 沉浸式头戴显示设备Immersive headsets
MR 输入 213:运动控制器MR Input 213: Motion controllers ✔️✔️

准备工作Before you start


请参阅 本页上的沉浸式耳机安装清单。See the installation checklist for immersive headsets on this page.

项目文件Project files

  • 下载项目所需的文件,并将文件解压缩到桌面。Download the files required by the project and extract the files to the Desktop.


如果要在下载之前查看源代码, 可在 GitHub 上找到。If you want to look through the source code before downloading, it's available on GitHub.

Unity 设置Unity setup


  • 优化 Unity 以实现 Windows Mixed Reality 开发Optimize Unity for Windows Mixed Reality development
  • 设置混合现实照相机Setup Mixed Reality Camera
  • 设置环境Setup environment


  • 启动 Unity。Start Unity.

  • 选择“打开” 。Select Open.

  • 导航到桌面并找到之前 unarchived 的 MixedReality213 文件夹。Navigate to your Desktop and find the MixedReality213-master folder you previously unarchived.

  • 单击“选择文件夹”。Click Select Folder.

  • Unity 完成加载项目文件后,你将能够看到 Unity 编辑器。Once Unity finishes loading project files, you will be able to see Unity editor.

  • 在 Unity 中,选择 " 文件 > 生成设置"。In Unity, select File > Build Settings.


  • 选择 "平台" 列表中的 "通用 Windows 平台",然后单击 "切换平台" 按钮。Select Universal Windows Platform in the Platform list and click the Switch Platform button.

  • 将目标设备设置为 任何设备Set Target Device to Any device

  • 将生成类型设置为 D3DSet Build Type to D3D

  • 将 SDK 设置为 安装的最新版本Set SDK to Latest Installed

  • 检查 Unity c # 项目Check Unity C# Projects

    • 这样,便可以修改 Visual Studio 项目中的脚本文件,而无需重新生成 Unity 项目。This allows you modify script files in the Visual Studio project without rebuilding Unity project.
  • 单击 " 播放机设置"。Click Player Settings.

  • 在 " 检查器 " 面板中,向下滚动到底部In the Inspector panel, scroll down to the bottom

  • 在 XR 设置中,检查 支持的虚拟现实In XR Settings, check Virtual Reality Supported

  • 在虚拟现实 Sdk 下,选择 " Windows Mixed Reality "Under Virtual Reality SDKs, select Windows Mixed Reality


  • 关闭 " 生成设置 " 窗口。Close Build Settings window.

项目结构Project structure

本教程使用 混合现实工具包-UnityThis tutorial uses Mixed Reality Toolkit - Unity. 可以在 此页上找到发布。You can find the releases on this page.


已完成用于引用的场景Completed scenes for your reference

  • 你会在 场景 文件夹中找到两个已完成的 Unity 场景。You will find two completed Unity scenes under Scenes folder.
    • MixedReality213:通过单画笔完成场景MixedReality213: Completed scene with single brush
    • MixedReality213Advanced:已完成用于具有多个画笔的高级设计的场景MixedReality213Advanced: Completed scene for advanced design with multiple brushes

教程的新场景设置New Scene setup for the tutorial

  • 在 Unity 中,单击 " 文件" > 新建场景In Unity, click File > New Scene

  • 删除 摄像机方向灯Delete Main Camera and Directional Light

  • 在 " 项目" 面板 中,搜索并将以下 prototyping 拖到 " 层次结构 " 面板中:From the Project panel, search and drag the following prefabs into the Hierarchy panel:

    • 资产/HoloToolkit/Input/Prototyping/MixedRealityCameraAssets/HoloToolkit/Input/Prefabs/MixedRealityCamera
    • 资产/AppPrefabs/环境Assets/AppPrefabs/Environment


  • 混合现实工具包中有两个照相机 prototyping:There are two camera prefabs in Mixed Reality Toolkit:

    • MixedRealityCamera. prefab:仅照相机MixedRealityCamera.prefab: Camera only
    • MixedRealityCameraParent. prefab:摄像 + Teleportation + 边界MixedRealityCameraParent.prefab: Camera + Teleportation + Boundary
    • 在本教程中,我们将使用不带 teleportation 功能的 MixedRealityCameraIn this tutorial, we will use MixedRealityCamera without teleportation feature. 为此,我们添加了简单的 环境 prefab,其中包含一个基本楼层,使用户感觉不到接地。Because of this, we added simple Environment prefab which contains a basic floor to make the user feel grounded.
    • 若要了解有关 teleportation 与 MixedRealityCameraParent 的详细信息,请参阅 高级设计-teleportation 和 locomotionTo learn more about the teleportation with MixedRealityCameraParent, see Advanced design - Teleportation and locomotion

Skybox 安装程序Skybox setup

  • 单击 " 窗口" > 照明 > 设置Click Window > Lighting > Settings

  • 单击 Skybox 材料字段 右侧的圆圈Click the circle on the right side of the Skybox Material field

  • 键入 "灰色" 并选择 SkyboxGray (资产/AppPrefabs/支持/材料/SkyboxGray) Type in ‘gray’ and select SkyboxGray (Assets/AppPrefabs/Support/Materials/SkyboxGray.mat)

    设置 skybox

  • 检查 Skybox 选项以查看已分配的灰色渐变 SkyboxCheck Skybox option to be able to see assigned gray gradient skybox

    切换 skybox 选项

  • 带有 MixedRealityCamera、环境和灰色 skybox 的场景将如下所示。The scene with MixedRealityCamera, Environment and gray skybox will look like this.

    MixedReality213 环境

  • 单击 " 文件" > 将场景另存为Click File > Save Scene as

  • 用任意名称 将场景保存 在场景文件夹下Save your scene under Scenes folder with any name

第1章-控制器可视化Chapter 1 - Controller visualization


  • 了解如何在 Unity 的游戏模式下和在运行时呈现运动控制器模型。Learn how to render motion controller models in Unity's game mode and at runtime.

Windows Mixed Reality 提供适用于控制器可视化的动画控制器模型。Windows Mixed Reality provides an animated controller model for controller visualization. 可在应用中使用以下几种方法来实现控制器可视化:There are several approaches you can take for controller visualization in your app:

  • 默认-使用默认控制器但不修改Default - Using default controller without modification
  • 混合-使用默认控制器,但自定义其部分元素或覆盖 UI 组件Hybrid - Using default controller, but customizing some of its elements or overlaying UI components
  • 替换-将你自己的自定义3D 模型用于控制器Replacement - Using your own customized 3D model for the controller

在本章中,我们将了解这些控制器自定义的示例。In this chapter, we will learn about the examples of these controller customizations.


  • 在 " 项目 " 面板的 "搜索" 框中,键入 MotionControllersIn the Project panel, type MotionControllers in the search box . 还可以在 "资产/HoloToolkit/输入/Prototyping/" 下找到它。You can also find it under Assets/HoloToolkit/Input/Prefabs/.
  • MotionControllers prefab 拖到 " 层次结构 " 面板中。Drag the MotionControllers prefab into the Hierarchy panel.
  • 单击 "层次结构" 面板中的 MotionControllers prefab。Click on the MotionControllers prefab in the Hierarchy panel.

MotionControllers prefabMotionControllers prefab

MotionControllers prefab 有一个 MotionControllerVisualizer 脚本,该脚本提供备用控制器型号的槽。MotionControllers prefab has a MotionControllerVisualizer script which provides the slots for alternate controller models. 如果您为自己的自定义3D 模型(如手或剑)指定了一个,并选中 "始终使用备用的左/右模型",则会看到它们,而不是默认模型。If you assign your own custom 3D models such as a hand or a sword and check 'Always Use Alternate Left/Right Model', you will see them instead of the default model. 我们将在第4章中使用此槽,用画笔替换控制器模型。We will use this slot in Chapter 4 to replace the controller model with a brush.



  • 在 " 检查器 " 面板中,双击 " MotionControllerVisualizer Script" 以查看 Visual Studio 中的代码In the Inspector panel, double click MotionControllerVisualizer script to see the code in the Visual Studio

MotionControllerVisualizer 脚本MotionControllerVisualizer script

MotionControllerVisualizerMotionControllerInfo 类提供访问 & 修改默认控制器模型的方法。The MotionControllerVisualizer and MotionControllerInfo classes provide the means to access & modify the default controller models. MotionControllerVisualizer 订阅 Unity 的 InteractionSourceDetected 事件,并在发现控制器模型时自动将其实例化。MotionControllerVisualizer subscribes to Unity's InteractionSourceDetected event and automatically instantiates controller models when they are found.

protected override void Awake()
    InteractionManager.InteractionSourceDetected += InteractionManager_InteractionSourceDetected;
    InteractionManager.InteractionSourceLost += InteractionManager_InteractionSourceLost;

控制器模型是根据 glTF 规范传递的。The controller models are delivered according to the glTF specification. 创建此格式是为了提供通用格式,同时改进了在传输和解包3D 资产后的过程。This format has been created to provide a common format, while improving the process behind transmitting and unpacking 3D assets. 在这种情况下,我们需要在运行时检索并加载控制器模型,因为我们希望尽可能使用户体验尽可能顺畅,并不保证用户可能正在使用的运动控制器版本。In this case, we need to retrieve and load the controller models at runtime, as we want to make the user's experience as seamless as possible, and it's not guaranteed which version of the motion controllers the user might be using. 本课程通过混合现实工具包使用 Khronos 组的 UnityGLTF 项目版本。This course, via the Mixed Reality Toolkit, uses a version of the Khronos Group's UnityGLTF project.

提供控制器后,脚本可以使用 MotionControllerInfo 查找特定控制器元素的转换,以便它们可以正确定位。Once the controller has been delivered, scripts can use MotionControllerInfo to find the transforms for specific controller elements so they can correctly position themselves.

在后面的章节中,我们将了解如何使用这些脚本将 UI 元素附加到控制器。In a later chapter, we will learn how to use these scripts to attach UI elements to the controllers.

在某些脚本中,你将找到带有 #if 的代码块 !UNITY_EDITORUNITY_WSA。这些代码块仅在部署到 Windows 时在 UWP 运行时上运行。这是因为 Unity 编辑器和 UWP 应用运行时所使用的 Api 集是不同的。In some scripts, you will find code blocks with #if !UNITY_EDITOR or UNITY_WSA. These code blocks run only on the UWP runtime when you deploy to Windows. This is because the set of APIs used by the Unity editor and the UWP app runtime are different.

  • 保存 场景并单击 " 播放 " 按钮。Save the scene and click the play button.

你将能够在耳机中看到带有运动控制器的场景。You will be able to see the scene with motion controllers in your headset. 可以查看按钮单击、操纵杆运动和触摸板触摸突出显示的详细动画。You can see detailed animations for button clicks, thumbstick movement, and touchpad touch highlighting.

MR213_Controller 可视化默认值

第2章-将 UI 元素附加到控制器Chapter 2 - Attaching UI elements to the controller


  • 了解运动控制器的元素Learn about the elements of the motion controllers
  • 了解如何将对象附加到控制器的特定部分Learn how to attach objects to specific parts of the controllers

在本章中,你将学习如何将用户界面元素添加到用户可随时轻松访问和操作的控制器。In this chapter, you will learn how to add user interface elements to the controller which the user can easily access and manipulate at anytime. 你还将了解如何使用触摸板输入添加简单的颜色选取器 UI。You will also learn how to add a simple color picker UI using the touchpad input.


  • 在 " 项目 " 面板中,搜索 " MotionControllerInfo script"。In the Project panel, search MotionControllerInfo script.
  • 在搜索结果中,双击 " MotionControllerInfo script" 以查看 Visual Studio 中的代码。From the search result, double click MotionControllerInfo script to see the code in Visual Studio.

MotionControllerInfo 脚本MotionControllerInfo script

第一步是选择要将 UI 附加到的控制器的元素。The first step is to choose which element of the controller you want the UI to attach to. 这些元素在 MotionControllerInfo.csControllerElementEnum 中定义。These elements are defined in ControllerElementEnum in MotionControllerInfo.cs.

MR213 MotionControllerElements

  • 主页Home
  • 菜单Menu
  • 掌握Grasp
  • 控制Thumbstick
  • SelectSelect
  • 触摸板Touchpad
  • 指针姿势 –此元素表示控制器向后方向的笔尖。Pointing pose – this element represents the tip of the controller pointing forward direction.


  • 在 " 项目 " 面板中,搜索 " AttachToController script"。In the Project panel, search AttachToController script.
  • 在搜索结果中,双击 " AttachToController script" 以查看 Visual Studio 中的代码。From the search result, double click AttachToController script to see the code in Visual Studio.

AttachToController 脚本AttachToController script

AttachToController 脚本提供了一种简单的方法,可将任何对象附加到指定的控制器左右手使用习惯和元素。The AttachToController script provides a simple way to attach any objects to a specified controller handedness and element.

AttachElementToController ( # B1 中,In AttachElementToController(),

  • 使用 MotionControllerInfo 检查左右手使用习惯 。左右手使用习惯Check handedness using MotionControllerInfo.Handedness
  • 使用 MotionControllerInfo. TryGetElement ( 获取控制器的特定元素Get specific element of the controller using MotionControllerInfo.TryGetElement()
  • 从控制器模型检索该元素的转换后,将其父对象置于其下,并将对象的本地位置 & 旋转到零。After retrieving the element's transform from the controller model, parent the object under it and set object's local position & rotation to zero.
public MotionControllerInfo.ControllerElementEnum Element { get { return element; } }

private void AttachElementToController(MotionControllerInfo newController)
     if (!IsAttached && newController.Handedness == handedness)
          if (!newController.TryGetElement(element, out elementTransform))
               Debug.LogError("Unable to find element of type " + element + " under controller " + + "; not attaching.");

          controller = newController;


          // Parent ourselves under the element and set our offsets
          transform.parent = elementTransform;
          transform.localPosition = positionOffset;
          transform.localEulerAngles = rotationOffset;
          if (setScaleOnAttach)
               transform.localScale = scale;

          // Announce that we're attached
          IsAttached = true;

使用 AttachToController 脚本的最简单方法是从其继承,就像我们在 ColorPickerWheel 的情况下所做的那样 The simplest way to use AttachToController script is to inherit from it, as we've done in the case of ColorPickerWheel. 只需重写 OnAttachToControllerOnDetachFromController 函数,以便在检测到控制器/断开连接时执行设置/细分。Simply override the OnAttachToController and OnDetachFromController functions to perform your setup / breakdown when the controller is detected / disconnected.


  • 在 " 项目 " 面板的 "搜索" 框中键入 ColorPickerWheelIn the Project panel, type in the search box ColorPickerWheel. 还可以在 "资产/AppPrefabs/" 下找到它。You can also find it under Assets/AppPrefabs/.
  • ColorPickerWheel prefab 拖到 " 层次结构 " 面板中。Drag ColorPickerWheel prefab into the Hierarchy panel.
  • 单击 "层次结构" 面板中的 ColorPickerWheel prefab。Click the ColorPickerWheel prefab in the Hierarchy panel.
  • 在 " 检查器 " 面板中,双击 " ColorPickerWheel Script" 以查看 Visual Studio 中的代码。In the Inspector panel, double click ColorPickerWheel Script to see the code in Visual Studio.

ColorPickerWheel prefab

ColorPickerWheel 脚本ColorPickerWheel script

由于 ColorPickerWheel 继承 了 AttachToController,它会在 "检查器" 面板中显示 左右手使用习惯元素Since ColorPickerWheel inherits AttachToController, it shows Handedness and Element in the Inspector panel. 我们会将 UI 附加到左侧控制器上的触摸板元素。We'll be attaching the UI to the Touchpad element on the left controller.

ColorPickerWheel 脚本

ColorPickerWheel 重写 OnAttachToControllerOnDetachFromController 以订阅输入事件,此事件将在下一章中用于使用触摸板输入的颜色选择。ColorPickerWheel overrides the OnAttachToController and OnDetachFromController to subscribe to the input event which will be used in next chapter for color selection with touchpad input.

public class ColorPickerWheel : AttachToController, IPointerTarget
    protected override void OnAttachToController()
        // Subscribe to input now that we're parented under the controller
        InteractionManager.InteractionSourceUpdated += InteractionSourceUpdated;

    protected override void OnDetachFromController()
        Visible = false;

        // Unsubscribe from input now that we've detached from the controller
        InteractionManager.InteractionSourceUpdated -= InteractionSourceUpdated;
  • 保存 场景并单击 " 播放 " 按钮。Save the scene and click the play button.

将对象附加到控制器的替代方法Alternative method for attaching objects to the controllers

建议你的脚本继承自 AttachToController ,并重写 OnAttachToControllerWe recommend that your scripts inherit from AttachToController and override OnAttachToController. 但这并不总是可行。However, this may not always be possible. 替代方法是将其用作独立组件。An alternative is using it as a standalone component. 如果要将现有的 prefab 附加到控制器而不重构脚本,这会很有用。This can be useful when you want to attach an existing prefab to a controller without refactoring your scripts. 只需将 IsAttached 设置为 true,然后再执行任何设置。Simply have your class wait for IsAttached to be set to true before performing any setup. 执行此操作的最简单方法是使用协同程序作为 "Start"。The simplest way to do this is by using a coroutine for 'Start.'

private IEnumerator Start() {
    AttachToController attach = gameObject.GetComponent<AttachToController>();

    while (!attach.IsAttached) {
        yield return null;

    // Perform setup here

第3章-使用触摸板输入Chapter 3 - Working with touchpad input


  • 了解如何获取触摸板输入数据事件Learn how to get touchpad input data events
  • 了解如何将触摸板轴位置信息用于应用体验Learn how to use touchpad axis position information for your app experience


  • 在 "层次结构" 面板中,单击 " ColorPickerWheel "In the Hierarchy panel, click ColorPickerWheel
  • 在 "检查器" 面板中的 " Animator" 下,双击 " ColorPickerWheelController "In the Inspector panel, under Animator, double click ColorPickerWheelController
  • 你将能够看到 " Animator " 选项卡已打开You will be able to see Animator tab opened

显示/隐藏具有 Unity 动画控制器的 UIShowing/hiding UI with Unity's Animation controller

若要显示和隐藏带有动画的 ColorPickerWheel UI,我们使用 的是 Unity 的动画系统To show and hide the ColorPickerWheel UI with animation, we are using Unity's animation system. 如果将 ColorPickerWheelVisible 属性设置为 true 或 False,则将 显示隐藏 动画触发器。Setting the ColorPickerWheel's Visible property to true or false triggers Show and Hide animation triggers. ColorPickerWheelController 动画控制器中定义了 显示隐藏 参数。Show and Hide parameters are defined in the ColorPickerWheelController animation controller.

Unity 动画控制器


  • 在 " 层次结构 " 面板中,选择 ColorPickerWheel prefabIn the Hierarchy panel, select ColorPickerWheel prefab
  • 在 " 检查器 " 面板中,双击 " ColorPickerWheel Script" 以查看 Visual Studio 中的代码In the Inspector panel, double click ColorPickerWheel script to see the code in the Visual Studio

ColorPickerWheel 脚本ColorPickerWheel script

ColorPickerWheel 订阅 Unity 的 InteractionSourceUpdated 事件来侦听触摸板事件。ColorPickerWheel subscribes to Unity's InteractionSourceUpdated event to listen for touchpad events.

InteractionSourceUpdated ( # B1 中,脚本首先检查以确保:In InteractionSourceUpdated(), the script first checks to ensure that it:

  • 实际上为触摸板事件 (的 obj。touchpadTouched) is actually a touchpad event (obj.state.touchpadTouched)
  • 源自左侧控制器 (obj。左右手使用习惯) originates from the left controller (obj.state.source.handedness)

如果两者都为 true,则触摸板位置 (为 "状态"。touchpadPosition) 分配给 selectorPositionIf both are true, the touchpad position (obj.state.touchpadPosition) is assigned to selectorPosition.

private void InteractionSourceUpdated(InteractionSourceUpdatedEventArgs obj)
    if (obj.state.source.handedness == handedness && obj.state.touchpadTouched)
        Visible = true;
        selectorPosition = obj.state.touchpadPosition;

更新 ( # B1 基于 visible 属性时,它会触发在颜色选取器的 animator 组件中显示和隐藏动画触发器In Update(), based on visible property, it triggers Show and Hide animation triggers in the color picker's animator component

if (visible != visibleLastFrame)
    if (visible)

Update ( # B1 中, selectorPosition 用于在色轮的网格碰撞器上强制转换射线,这会返回一个 UV 位置。In Update(), selectorPosition is used to cast a ray at the color wheel's mesh collider, which returns a UV position. 然后,可以使用此位置查找色轮纹理的像素坐标和颜色值。This position can then be used to find the pixel coordinate and color value of the color wheel's texture. 其他脚本可以通过 SelectedColor 属性访问此值。This value is accessible to other scripts via the SelectedColor property.

颜色选取器轮 Raycasting

    // Clamp selector position to a radius of 1
    Vector3 localPosition = new Vector3(selectorPosition.x * inputScale, 0.15f, selectorPosition.y * inputScale);
    if (localPosition.magnitude > 1)
        localPosition = localPosition.normalized;
    selectorTransform.localPosition = localPosition;

    // Raycast the wheel mesh and get its UV coordinates
    Vector3 raycastStart = selectorTransform.position + selectorTransform.up * 0.15f;
    RaycastHit hit;
    Debug.DrawLine(raycastStart, raycastStart - (selectorTransform.up * 0.25f));

    if (Physics.Raycast(raycastStart, -selectorTransform.up, out hit, 0.25f, 1 << colorWheelObject.layer, QueryTriggerInteraction.Ignore))
        // Get pixel from the color wheel texture using UV coordinates
        Vector2 uv = hit.textureCoord;
        int pixelX = Mathf.FloorToInt(colorWheelTexture.width * uv.x);
        int pixelY = Mathf.FloorToInt(colorWheelTexture.height * uv.y);
        selectedColor = colorWheelTexture.GetPixel(pixelX, pixelY);
        selectedColor.a = 1f;
    // Set the selector's color and blend it with white to make it visible on top of the wheel
    selectorRenderer.material.color = Color.Lerp (selectedColor, Color.white, 0.5f);

第4章-覆盖控制器模型Chapter 4 - Overriding controller model


  • 了解如何使用自定义3D 模型替代控制器模型。Learn how to override the controller model with a custom 3D model.



  • 在 "层次结构" 面板中单击 " MotionControllers "。Click MotionControllers in the Hierarchy panel.
  • 单击 替换右侧控制器 字段右侧的圆圈。Click the circle on the right side of the Alternate Right Controller field.
  • 键入 "BrushController",并从结果中选择 prefab。Type in 'BrushController' and select the prefab from the result. 可以在 "资产/AppPrefabs/BrushController" 下找到它。You can find it under Assets/AppPrefabs/BrushController.
  • 选中 "始终使用备用模式"Check Always Use Alternate Right Model


不需要在 层次结构 面板中包含 BrushController prefab。The BrushController prefab does not have to be included in the Hierarchy panel. 但是,若要查看子组件,请执行以下操作:However, to check out its child components:

  • 在 " 项目 " 面板中,键入 BrushController 并将 BrushController prefab 拖到 " 层次结构 " 面板中。In the Project panel, type in BrushController and drag BrushController prefab into the Hierarchy panel.


你会在 BrushController 中找到 Tip 组件。You will find the Tip component in BrushController. 我们将使用其转换来启动/停止绘制线条。We will use its transform to start/stop drawing lines.

  • 层次结构 面板中删除 BrushControllerDelete the BrushController from the Hierarchy panel.
  • 保存 场景并单击 " 播放 " 按钮。Save the scene and click the play button. 你将能够看到画笔模型已替换右手运动控制器。You will be able to see the brush model replaced the right-hand motion controller.

第5章-用选择输入进行绘制Chapter 5 - Painting with Select input


  • 了解如何使用 "选择按钮" 事件来启动和停止线条绘制Learn how to use the Select button event to start and stop a line drawing


  • 在 "项目" 面板中搜索 BrushController prefab。Search BrushController prefab in the Project panel.
  • 在 " 检查器 " 面板中,双击 " BrushController Script" 以查看 Visual Studio 中的代码In the Inspector panel, double click BrushController Script to see the code in Visual Studio

BrushController 脚本BrushController script

BrushController 订阅 InteractionManager 的 InteractionSourcePressedInteractionSourceReleased 事件。BrushController subscribes to the InteractionManager's InteractionSourcePressed and InteractionSourceReleased events. 触发 InteractionSourcePressed 事件后,画笔的 Draw 属性将设置为 true;触发 InteractionSourceReleased 事件后,画笔的 Draw 属性将设置为 false。When InteractionSourcePressed event is triggered, the brush's Draw property is set to true; when InteractionSourceReleased event is triggered, the brush's Draw property is set to false.

private void InteractionSourcePressed(InteractionSourcePressedEventArgs obj)
    if (obj.state.source.handedness == InteractionSourceHandedness.Right && obj.pressType == InteractionSourcePressType.Select)
        Draw = true;

private void InteractionSourceReleased(InteractionSourceReleasedEventArgs obj)
    if (obj.state.source.handedness == InteractionSourceHandedness.Right && obj.pressType == InteractionSourcePressType.Select)
        Draw = false;

绘图 设置为 true 时,画笔将在实例化的 Unity LineRenderer 中生成点。While Draw is set to true, the brush will generate points in an instantiated Unity LineRenderer. 对此 prefab 的引用将保存在画笔的 Stroke prefab 字段中。A reference to this prefab is kept in the brush's Stroke Prefab field.

private IEnumerator DrawOverTime()
    // Get the position of the tip
    Vector3 lastPointPosition = tip.position;


    // Create a new brush stroke
    GameObject newStroke = Instantiate(strokePrefab);
    LineRenderer line = newStroke.GetComponent<LineRenderer>();
    newStroke.transform.position = startPosition;
    line.SetPosition(0, tip.position);
    float initialWidth = line.widthMultiplier;

    // Generate points in an instantiated Unity LineRenderer
    while (draw)
        // Move the last point to the draw point position
        line.SetPosition(line.positionCount - 1, tip.position);
        line.material.color = colorPicker.SelectedColor;
        brushRenderer.material.color = colorPicker.SelectedColor;
        lastPointAddedTime = Time.unscaledTime;
        // Adjust the width between 1x and 2x width based on strength of trigger pull
        line.widthMultiplier = Mathf.Lerp(initialWidth, initialWidth * 2, width);

        if (Vector3.Distance(lastPointPosition, tip.position) > minPositionDelta || Time.unscaledTime > lastPointAddedTime + maxTimeDelta)
            // Spawn a new point
            lastPointAddedTime = Time.unscaledTime;
            lastPointPosition = tip.position;
            line.positionCount += 1;
            line.SetPosition(line.positionCount - 1, lastPointPosition);
        yield return null;

若要从颜色选取器滚轮 UI 使用当前选定的颜色, BrushController 需要具有对 ColorPickerWheel 对象的引用。To use the currently selected color from the color picker wheel UI, BrushController needs to have a reference to the ColorPickerWheel object. 因为在运行时将 BrushController prefab 实例化为替换控制器,所以必须在运行时设置对场景中对象的任何引用。Because the BrushController prefab is instantiated at runtime as a replacement controller, any references to objects in the scene will have to be set at runtime. 在此示例中,我们使用 GameObject 来查找 ColorPickerWheelIn this case we use GameObject.FindObjectOfType to locate the ColorPickerWheel:

private void OnEnable()
    // Locate the ColorPickerWheel
    colorPicker = FindObjectOfType<ColorPickerWheel>();

    // Assign currently selected color to the brush’s material color
    brushRenderer.material.color = colorPicker.SelectedColor;
  • 保存 场景并单击 " 播放 " 按钮。Save the scene and click the play button. 你将能够使用右侧控制器上的 "选择" 按钮绘制线条并进行绘制。You will be able to draw the lines and paint using the select button on the right-hand controller.

第6章-带有选择输入的对象生成Chapter 6 - Object spawning with Select input


  • 了解如何使用 "选择并抓住" 按钮输入事件Learn how to use Select and Grasp button input events
  • 了解如何实例化对象Learn how to instantiate objects


  • 在 " 项目 " 面板的 "搜索" 框中,键入 ObjectSpawnerIn the Project panel, type ObjectSpawner in the search box. 还可以在 "资产/AppPrefabs/You can also find it under Assets/AppPrefabs/

  • ObjectSpawner prefab 拖到 " 层次结构 " 面板中。Drag the ObjectSpawner prefab into the Hierarchy panel.

  • 在 "层次结构" 面板中单击 " ObjectSpawner "。Click ObjectSpawner in the Hierarchy panel.

  • ObjectSpawner 有一个名为 " 颜色源" 的字段。ObjectSpawner has a field named Color Source.

  • 从 " 层次结构 " 面板中,将 " ColorPickerWheel " 引用拖到此字段中。From the Hierarchy panel, drag the ColorPickerWheel reference into this field.

    对象 Spawner 检查器

  • 单击 "层次结构" 面板中的 ObjectSpawner prefab。Click the ObjectSpawner prefab in the Hierarchy panel.

  • 在 " 检查器 " 面板中,双击 " ObjectSpawner Script" 以查看 Visual Studio 中的代码。In the Inspector panel, double click ObjectSpawner Script to see the code in Visual Studio.

ObjectSpawner 脚本ObjectSpawner script

ObjectSpawner 将基元网格的副本( (cube、球面、圆柱) 实例化到空间中。The ObjectSpawner instantiates copies of a primitive mesh (cube, sphere, cylinder) into the space. 检测到 InteractionSourcePressed 时,它会检查左右手使用习惯,并检查其是否为 InteractionSourcePressTypeInteractionSourcePressType。When a InteractionSourcePressed is detected it checks the handedness and if it's an InteractionSourcePressType.Grasp or InteractionSourcePressType.Select event.

对于 抓住 事件,它会递增当前网格类型 (球、cube、圆柱) 的索引For a Grasp event, it increments the index of current mesh type (sphere, cube, cylinder)

private void InteractionSourcePressed(InteractionSourcePressedEventArgs obj)
    // Check handedness, see if it is left controller
    if (obj.state.source.handedness == handedness)
        switch (obj.pressType)
            // If it is Select button event, spawn object
            case InteractionSourcePressType.Select:
                if (state == StateEnum.Idle)
                    // We've pressed the grasp - enter spawning state
                    state = StateEnum.Spawning;

            // If it is Grasp button event
            case InteractionSourcePressType.Grasp:

                // Increment the index of current mesh type (sphere, cube, cylinder)
                if (meshIndex >= NumAvailableMeshes)
                    meshIndex = 0;


对于 Select 事件,在 SpawnObject ( # B1 中,会实例化一个新的对象,并将其取消父级并发布到世界中。For a Select event, in SpawnObject(), a new object is instantiated, un-parented and released into the world.

private void SpawnObject()
    // Instantiate the spawned object
    GameObject newObject = Instantiate(displayObject.gameObject, spawnParent);
    // Detach the newly spawned object
    newObject.transform.parent = null;
    // Reset the scale transform to 1
    scaleParent.localScale =;
    // Set its material color so its material gets instantiated
    newObject.GetComponent<Renderer>().material.color = colorSource.SelectedColor;

ObjectSpawner 使用 ColorPickerWheel 设置显示对象材料的颜色。The ObjectSpawner uses the ColorPickerWheel to set the color of the display object's material. 将为生成的对象提供此材料的实例,以使它们保留其颜色。Spawned objects are given an instance of this material so they will retain their color.

  • 保存 场景并单击 " 播放 " 按钮。Save the scene and click the play button.

你将能够通过 "抓住" 按钮更改对象,并通过 "选择" 按钮生成对象。You will be able to change the objects with the Grasp button and spawn objects with the Select button.

构建应用并将其部署到混合现实门户Build and deploy app to Mixed Reality Portal

  • 在 Unity 中,选择 " 文件 > 生成设置"。In Unity, select File > Build Settings.
  • 单击 " 添加打开的场景 ",将当前场景添加到 生成的场景中Click Add Open Scenes to add current scene to the Scenes In Build.
  • 单击“生成”。Click Build.
  • 创建名为 "App" 的 新文件夹Create a New Folder named "App".
  • 单击 应用 文件夹。Single click the App folder.
  • 单击“选择文件夹”。Click Select Folder.
  • 当 Unity 完成后,将显示文件资源管理器窗口。When Unity is done, a File Explorer window will appear.
  • 打开 应用程序 文件夹。Open the App folder.
  • 双击 " YourSceneName " Visual Studio 解决方案文件。Double click YourSceneName.sln Visual Studio Solution file.
  • 使用 Visual Studio 中的顶部工具栏,将目标从 "调试" 更改为 " 发布 ",将 "从 ARM" 更改为 " X64"。Using the top toolbar in Visual Studio, change the target from Debug to Release and from ARM to X64.
  • 单击 "设备" 按钮旁边的下拉箭头,然后选择 " 本地计算机"。Click on the drop-down arrow next to the Device button, and select Local Machine.
  • 单击菜单中的 " 调试-> 启动但不调试 ",或按 Ctrl + F5Click Debug -> Start Without debugging in the menu or press Ctrl + F5.

现在,在混合现实门户中生成并安装应用。Now the app is built and installed in Mixed Reality Portal. 可以通过混合现实门户中的 "开始" 菜单再次启动它。You can launch it again through Start menu in Mixed Reality Portal.

高级设计-具有径向布局的画笔工具Advanced design - Brush tools with radial layout

MixedReality213 Main

在本章中,你将学习如何将默认运动控制器模型替换为自定义画笔工具集合。In this chapter, you will learn how to replace the default motion controller model with a custom brush tool collection. 对于引用,可以在 "场景" 文件夹下找到已完成的场景 MixedReality213AdvancedFor your reference, you can find the completed scene MixedReality213Advanced under Scenes folder.


  • 在 " 项目 " 面板的 "搜索" 框中,键入 BrushSelectorIn the Project panel, type BrushSelector in the search box . 还可以在 "资产/AppPrefabs/You can also find it under Assets/AppPrefabs/

  • BrushSelector prefab 拖到 " 层次结构 " 面板中。Drag the BrushSelector prefab into the Hierarchy panel.

  • 对于组织,创建名为 "画笔" 的空 GameObjectFor organization, create an empty GameObject called Brushes

  • 将以下 prototyping 从 "项目" 面板拖到 "画笔"Drag following prefabs from the Project panel into Brushes

    • 资产/AppPrefabs/BrushFatAssets/AppPrefabs/BrushFat
    • 资产/AppPrefabs/BrushThinAssets/AppPrefabs/BrushThin
    • 资产/AppPrefabs/橡皮擦Assets/AppPrefabs/Eraser
    • 资产/AppPrefabs/MarkerFatAssets/AppPrefabs/MarkerFat
    • 资产/AppPrefabs/MarkerThinAssets/AppPrefabs/MarkerThin
    • 资产/AppPrefabs/铅笔Assets/AppPrefabs/Pencil


  • 在 "层次结构" 面板中单击 " MotionControllers prefab"。Click MotionControllers prefab in the Hierarchy panel.

  • 在 "检查器" 面板中,取消选中 "在 运动控制器可视化工具始终使用替代右模型"In the Inspector panel, uncheck Always Use Alternate Right Model on the Motion Controller Visualizer

  • 在 "层次结构" 面板中,单击 " BrushSelector "In the Hierarchy panel, click BrushSelector

  • BrushSelector 有一个名为 ColorPicker 的字段BrushSelector has a field named ColorPicker

  • 从 "层次结构" 面板中,将 " ColorPickerWheel " 拖动到 检查器 面板中的 " ColorPicker " 字段。From the Hierarchy panel, drag the ColorPickerWheel into ColorPicker field in the Inspector panel.

    将 ColorPickerWheel 分配给画笔选择器

  • 在 " 层次结构 " 面板中的 " BrushSelector prefab" 下,选择 " Menu " 对象。In the Hierarchy panel, under BrushSelector prefab, select the Menu object.

  • 在 " 检查器 " 面板中的 LineObjectCollection 组件下,打开 " 对象 数组" 下拉列表。In the Inspector panel, under the LineObjectCollection component, open the Objects array dropdown. 你将看到6个空槽。You will see 6 empty slots.

  • 从 " 层次结构 " 面板中,将 " 画笔 " 下的每个 prototyping 父级以任意顺序拖动到这些槽。From the Hierarchy panel, drag each of the prefabs parented under the Brushes GameObject into these slots in any order. (确保从场景中拖动 prototyping,而不是从项目文件夹中的 prototyping 中拖动。 ) (Make sure you're dragging the prefabs from the scene, not the prefabs in the project folder.)


BrushSelector prefabBrushSelector prefab

由于 BrushSelector 继承了 AttachToController,它将在 "检查器" 面板中显示 左右手使用习惯元素 选项。Since the BrushSelector inherits AttachToController, it shows Handedness and Element options in the Inspector panel. 我们选择了 " 向右 " 并将画笔工具 连接到右侧 的右控制器。We selected Right and Pointing Pose to attach brush tools to the right hand controller with forward direction.

BrushSelector 利用了两个实用工具:The BrushSelector makes use of two utilities:

  • 椭圆形:用于沿椭圆形状生成空间中的点。Ellipse: used to generate points in space along an ellipse shape.
  • LineObjectCollection:使用任意 Line 类生成的点( (例如) 的椭圆形)来分发对象。LineObjectCollection: distributes objects using the points generated by any Line class (eg, Ellipse). 这就是我们用来沿椭圆形状放置画笔的内容。This is what we'll be using to place our brushes along the Ellipse shape.

结合使用时,可以使用这些实用程序创建放射状菜单。When combined, these utilities can be used to create a radial menu.

LineObjectCollection 脚本LineObjectCollection script

LineObjectCollection 具有控件沿行分布的对象的大小、位置和旋转。LineObjectCollection has controls for the size, position and rotation of objects distributed along its line. 这对于创建径向菜单(如画笔选择器)很有用。This is useful for creating radial menus like the brush selector. 若要创建画笔的外观,使其在接近中心选定位置的情况下进行缩放, ObjectScale 曲线会在中心处峰值,并在边缘处关闭 tapers。To create the appearance of brushes that scale up from nothing as they approach the center selected position, the ObjectScale curve peaks in the center and tapers off at the edges.

BrushSelector 脚本BrushSelector script

对于 BrushSelector,我们选择了使用过程动画。In the case of the BrushSelector, we've chosen to use procedural animation. 首先,画笔模型由 LineObjectCollection 脚本分布在一个椭圆中。First, brush models are distributed in an ellipse by the LineObjectCollection script. 然后,每个画笔负责根据其 DisplayMode 值来维护其在用户的手边,这会根据所选内容进行更改。Then, each brush is responsible for maintaining its position in the user's hand based on its DisplayMode value, which changes based on the selection. 我们选择了过程方法,因为当用户选择画笔时,画笔位置转换的概率很高。We chose a procedural approach because of the high probability of brush position transitions being interrupted as the user selects brushes. Mecanim 动画可以合理地处理中断,但比简单的 Lerp 操作要复杂得多。Mecanim animations can handle interruptions gracefully, but it tends to be more complicated than a simple Lerp operation.

BrushSelector 结合使用这两种方法。BrushSelector uses a combination of both. 检测到触摸板输入后,画笔选项将变为可见,并沿径向菜单向上缩放。When touchpad input is detected, brush options become visible and scale up along the radial menu. 超时时间段之后 (表示用户选择了) 画笔选项再次缩小,只留下所选的画笔。After a timeout period (which indicates that the user has made a selection) the brush options scale down again, leaving only the selected brush.

可视化触摸板输入Visualizing touchpad input

即使在完全替换控制器模型的情况下,显示原始模型输入的输入也可能会很有帮助。Even in cases where the controller model has been completely replaced, it can be helpful to show input on the original model inputs. 这有助于使用户的操作成为现实。This helps to ground the user's actions in reality. 对于 BrushSelector ,我们选择了在收到输入时使触摸板短暂可见。For the BrushSelector we've chosen to make the touchpad briefly visible when the input is received. 这是通过从控制器检索触摸板元素,将其材料替换为自定义材料,然后根据收到的触摸板输入的最后时间向该材料的颜色应用渐变来完成的。This was done by retrieving the Touchpad element from the controller, replacing its material with a custom material, then applying a gradient to that material's color based on the last time touchpad input was received.

protected override void OnAttachToController()
    // Turn off the default controller's renderers

    // Get the touchpad and assign our custom material to it
    Transform touchpad;
    if (controller.TryGetElement(MotionControllerInfo.ControllerElementEnum.Touchpad, out touchpad))
        touchpadRenderer = touchpad.GetComponentInChildren<MeshRenderer>();
        originalTouchpadMaterial = touchpadRenderer.material;
        touchpadRenderer.material = touchpadMaterial;
        touchpadRenderer.enabled = true;

    // Subscribe to input now that we're parented under the controller
    InteractionManager.InteractionSourceUpdated += InteractionSourceUpdated;

private void Update()
    // Update our touchpad material
    Color glowColor = touchpadColor.Evaluate((Time.unscaledTime - touchpadTouchTime) / touchpadGlowLossTime);
    touchpadMaterial.SetColor("_EmissionColor", glowColor);
    touchpadMaterial.SetColor("_Color", glowColor);

带有触摸板输入的画笔工具选择Brush tool selection with touchpad input

当画笔选择器检测到触摸板的按下输入时,它将检查输入的位置以确定其是否在左侧或右侧。When the brush selector detects touchpad's pressed input, it checks the position of the input to determine if it was to the left or right.

笔划粗细与 selectPressedAmountStroke thickness with selectPressedAmount

可以通过 selectPressedAmount 获取按下的量的模拟值,而不是 InteractionSourcePressed ( # B1 中的 InteractionSourcePressType 事件。Instead of the InteractionSourcePressType.Select event in the InteractionSourcePressed(), you can get the analog value of the pressed amount through selectPressedAmount. 可以在 InteractionSourceUpdated ( # B1 中检索此值。This value can be retrieved in InteractionSourceUpdated().

private void InteractionSourceUpdated(InteractionSourceUpdatedEventArgs obj)
    if (obj.state.source.handedness == handedness)
        if (obj.state.touchpadPressed)
            // Check which side we clicked
            if (obj.state.touchpadPosition.x < 0)
                currentAction = SwipeEnum.Left;
                currentAction = SwipeEnum.Right;

            // Ping the touchpad material so it gets bright
            touchpadTouchTime = Time.unscaledTime;

        if (activeBrush != null)
            // If the pressed amount is greater than our threshold, draw
            if (obj.state.selectPressedAmount >= selectPressedDrawThreshold)
                activeBrush.Draw = true;
                activeBrush.Width = ProcessSelectPressedAmount(obj.state.selectPressedAmount);
                // Otherwise, stop drawing
                activeBrush.Draw = false;
                selectPressedSmooth = 0f;

橡皮擦脚本Eraser script

橡皮擦 是一种特殊类型的画笔,用于重写基 画笔DrawOverTime ( # B1 函数。Eraser is a special type of brush that overrides the base Brush's DrawOverTime() function. 绘图为 true 时,橡皮擦检查其刀尖是否与任何现有画笔笔划相交。While Draw is true, the eraser checks to see if its tip intersects with any existing brush strokes. 如果已添加到队列中,则会将它们添加到要缩减和删除的队列中。If it does, they are added to a queue to be shrunk down and deleted.

高级设计-Teleportation 和 locomotionAdvanced design - Teleportation and locomotion

如果要允许用户使用 teleportation 在场景周围移动,请使用 MixedRealityCameraParent 而不是 MixedRealityCameraIf you want to allow the user to move around the scene with teleportation using thumbstick, use MixedRealityCameraParent instead of MixedRealityCamera. 还需要添加 InputManagerDefaultCursorYou also need to add InputManager and DefaultCursor. 由于 MixedRealityCameraParent 已将 MotionControllers边界 包含为子组件,因此应该删除现有的 MotionControllers环境 prefab。Since MixedRealityCameraParent already includes MotionControllers and Boundary as child components, you should remove existing MotionControllers and Environment prefab.


  • 层次结构 面板中删除 "MixedRealityCamera"、"环境" 和 " MotionControllers "In the Hierarchy panel, delete MixedRealityCamera, Environment and MotionControllers

  • 在 " 项目" 面板 中,搜索并将以下 prototyping 拖到 " 层次结构 " 面板中:From the Project panel, search and drag the following prefabs into the Hierarchy panel:

    • 资产/AppPrefabs/Input/Prototyping/MixedRealityCameraParentAssets/AppPrefabs/Input/Prefabs/MixedRealityCameraParent
    • 资产/AppPrefabs/Input/Prototyping/InputManagerAssets/AppPrefabs/Input/Prefabs/InputManager
    • 资产/AppPrefabs/Input/Prototyping/Cursor/DefaultCursorAssets/AppPrefabs/Input/Prefabs/Cursor/DefaultCursor


  • 层次结构 面板中,单击 "输入管理器"In the Hierarchy panel, click Input Manager

  • 在 " 检查器 " 面板中,向下滚动到 简单的单指针选择器 部分In the Inspector panel, scroll down to the Simple Single Pointer Selector section

  • 从 " 层次结构 " 面板中,将 DefaultCursor 拖到 Cursor 字段From the Hierarchy panel, drag DefaultCursor into Cursor field

    分配 DefaultCursor

  • 保存 场景并单击 " 播放 " 按钮。Save the scene and click the play button. 你将能够使用操纵杆向左或向右旋转。You will be able to use the thumbstick to rotate left/right or teleport.

结束The end

这就是本教程的结尾!And that's the end of this tutorial! 你已了解:You learned:

  • 如何使用 Unity 的游戏模式和运行时中的运动控制器模型。How to work with motion controller models in Unity's game mode and runtime.
  • 如何使用不同类型的按钮事件及其应用程序。How to use different types of button events and their applications.
  • 如何在控制器顶部覆盖 UI 元素或对其进行完全自定义。How to overlay UI elements on top of the controller or fully customize it.

现在,你可以开始创建自己的具有运动控制器的沉浸式体验!You are now ready to start creating your own immersive experience with motion controllers!

已完成场景Completed scenes

  • 在 Unity 的 " 项目 " 面板中,单击 " 场景 " 文件夹。In Unity's Project panel click on the Scenes folder.
  • 你将发现两个 Unity 场景 MixedReality213MixedReality213AdvancedYou will find two Unity scenes MixedReality213 and MixedReality213Advanced.
    • MixedReality213:通过单画笔完成场景MixedReality213: Completed scene with single brush
    • MixedReality213Advanced:具有多个画笔并带有 "选择按钮的按量" 示例的已完成场景MixedReality213Advanced: Completed scene with multiple brush with select button's press amount example

另请参阅See also