1.电脑全息远程处理入门1. Getting started with PC Holographic Remoting

欢迎学习 HoloLens 2 教程。Welcome to the HoloLens 2 tutorials. 该系列由两部分组成,其中你将学习如何创建混合现实体验演示,并了解如何创建用于全息远程处理的电脑应用。In this two-part tutorial series, you will learn how to create a mixed reality experience demonstration and how to create a PC app for Holographic Remoting.

本教程介绍如何创建混合现实体验。In this tutorial, you'll learn how to create a mixed reality experience. 它将演示 UI 元素、3D 模型操作、模型剪辑和眼动跟踪功能。It will demonstrate UI elements, 3D model manipulation, model clipping, and eye-tracking features.

创建全息远程处理应用程序(第二篇教程)中,你将学习如何创建用于全息远程处理的电脑应用。In the second tutorial, Create a Holographic Remoting application, you will learn how to create a PC app for Holographic Remoting. 你还可随时连接到 HoloLens 2,从而在混合现实中直观呈现 3D 内容。And connect to HoloLens 2 at any point, providing a way to visualize 3D content in mixed reality.

目标Objectives

  • 导入资产并设置场景Import assets and set up the scene
  • 使用 UI 元素和按钮来与全息影像交互Interact with holograms using UI elements and buttons
  • 为剪辑功能配置3D 对象Configure 3D objects for the clipping feature
  • 了解如何通过眼动跟踪激活工具提示Learn about activating tooltips with eye-tracking

必备条件Prerequisites

  • 一台 Windows 10 电脑,其中已安装并配置正确的工具A Windows 10 PC configured with the correct tools installed
  • 基本 C# 编程知识Basic c# programming knowledge
  • 一个针对开发配置的 HoloLens 2 设备A HoloLens 2 device configured for development
  • Unity Hub,其中已装载 Unity 2019 LTS 且已添加通用 Windows 平台生成支持模块Unity Hub with Unity 2019 LTS mounted, and the Universal Windows Platform Build Support module added

继续学习之前,强烈建议完成入门教程系列,或者具备一些使用 Unity 和 MRTK 的基本经验。We strongly recommend completing the Getting started tutorials series or some basic prior experience with Unity and MRTK before continuing.

重要

  • 建议对本系列教程使用 Unity 2019 LTS。The recommended Unity version for this tutorial series is Unity 2019 LTS. 这将取代上述链接的先决条件中所述的所有 Unity 版本要求或建议。It supersedes any Unity version requirements or recommendations stated in the prerequisites linked above.
  • 只能在旧版 XR 中将全息远程用于 MRTK 项目。Holographic Remoting with MRTK projects will only work with legacy XR. 目前不支持 XR SDK。XR SDK is not supported at this time.

创建和准备 Unity 项目Creating and preparing the Unity project

在本部分,你将创建一个新的 Unity 项目,并使其准备好用于 MRTK 开发。In this section, you will create a new Unity project and get it ready for MRTK development.

为此,请先执行初始化项目和第一个应用程序中的以下步骤,但请忽略有关在设备上生成应用程序的说明:For this, first follow the Initializing your project and first application, excluding the Build your application to your device instructions, which includes the following steps:

  1. 创建 Unity 项目并为其指定适当的名称,例如“MRTK 教程”Creating the Unity project and give it a suitable name, for example, MRTK Tutorials

  2. 切换生成平台Switching the build platform

  3. 导入 TextMeshPro 基本资源Importing the TextMeshPro Essential Resources

  4. 导入混合现实工具包Importing the Mixed Reality Toolkit

  5. 配置 Unity 项目Configuring the Unity project

  6. 创建和设置场景,并为场景提供一个合适的名称,例如“电脑全息远程处理”Creating and setting the scene and give the scene a suitable name, for example, PC Holographic Remoting

然后,按照更改空间感知显示选项的说明,将场景的 MRTK 配置配置文件更改为“DefaultHoloLens2ConfigurationProfile”。Then follow the Changing the Spatial Awareness Display Option instructions to change the MRTK configuration profile for your scene to the DefaultHoloLens2ConfigurationProfile. 将空间感知网格的显示选项更改为“遮挡”。Change the display options for the spatial awareness mesh to Occlusion.

导入教程资产Importing the tutorial assets

下载并导入 MRTK.Tutorials.PCHolographicRemoting.unitypackageDownload and import the MRTK.Tutorials.PCHolographicRemoting.unitypackage.

提示

有关如何导入 Unity 自定义包的提示,可参阅导入混合现实工具包中的说明。For a reminder on how to import a Unity custom package, you can refer to the Import the Mixed Reality Toolkit instructions.

导入教程资产后,项目窗口应如下所示:After importing the tutorial assets, your Project window should look similar to this:

导入教程资产后的 Unity“层次结构”、“场景”和“项目”窗口

配置和准备场景Configuring and preparing the scene

在本部分,你将通过添加一些教程预制件来准备场景。In this section, you will prepare the scene by adding some of the tutorial prefabs.

在“项目”窗口中,导航到“资产” > “MRTK.Tutorials.PCHolograhicRemoting” > “预制件”文件夹 。In the Project window, navigate to Assets > MRTK.Tutorials.PCHolograhicRemoting > Prefabs folder. 在按住 Ctrl 按钮时单击下面 6 个预制件。While holding down the CTRL button, click on the below six prefabs.

  • ButtonParentButtonParent
  • ClippingObjectsClippingObjects
  • HandSpatialMapButtonHandSpatialMapButton
  • 说明Instructions
  • ModelParentModelParent
  • 平台Platform

显示了要添加到所选场景的预制件的 Unity

将这些模型从预制件文件夹拖放到“层次结构”窗口。Drag-and-drop these models from the prefabs folder into the Hierarchy window.

新增的预制件仍处于选中状态的 Unity

若要将焦点置于场景中的对象上,可双击 ModelParent 对象,然后再次放大一些:To focus in on the objects in the scene, you can double-click on the ModelParent object, and then zoom slightly in again:

ModelParent 对象处于对焦状态的 Unity

提示

如果在场景中看到大图标(例如带边框的“T”图标会分散注意力),可将调节器 (Gizmos) 切换到关闭位置来隐藏这些图标。If you find the large icons in your scene, such as, the large framed 'T' icons distracting, you can hide these by toggling the Gizmos to the off position.

配置按钮以操作场景Configuring the buttons to operate the scene

在本部分,你将向场景中添加脚本来创建按钮事件,从而演示模型切换和剪辑功能的基本原理。In this section, you will add scripts into the scene to create button events demonstrating the fundamentals of model switching and clipping functionality.

1.配置“可交互(脚本)”组件1. Configuring the Interactable (Script) component

在“层次结构”窗口中,展开 ButtonParent 对象并选择“NextButton” 。In the Hierarchy window, expand the ButtonParent object and select the NextButton. 在检查器窗口中,找到“可交互(脚本)”组件,然后单击 OnClick () 事件下的 + 图标 。In the Inspector window, locate the Interactable (Script) component and click on + icon under OnClick () event.

添加了 Nextbutton OnClick 事件的 Unity

在“层次结构”窗口中仍选中 NextButton 对象的情况下,单击“层次结构”窗口中的 ButtonParent 对象并将其拖放到刚刚添加的事件的空“无(对象)”字段中,使 ButtonParent 对象侦听此按钮的按钮单击事件 :With the NextButton object still selected in the Hierarchy window, click-and-drag the ButtonParent object from the Hierarchy window into the empty None (Object) field of the event you just added to make the ButtonParent object listen for button click event from this button:

配置了 NextButton OnClick 事件侦听器的 Unity

单击同一事件的“无函数”下拉列表。Click the No Function dropdown of the same event. 然后选择“ViewButtonControl” > NextModel (),将 NextModel () 函数设置为从该按钮触发按钮按下事件时触发的操作 :Then select ViewButtonControl > NextModel () to set the NextModel () function as the action that is triggered when the button pressed events is fired from this button:

具有 NextButton OnClick 事件操作选择路径的 Unity

2.配置其余的按钮2. Configuring the remaining buttons

对于剩余的每个按钮,请完成上述的过程,将函数分配给“OnClick ()”事件:For each of the remaining buttons, complete the process outlined above to assign functions to the OnClick () events:

  • 对于 PreviousButton 对象,请分配“ViewButtonControl”>“PreviousModel ()”函数 。For PreviousButton object, assign the ViewButtonContro l > PreviousModel () function.

  • 对于 ClippingButton,请选择“ToggleButton” > “ToggleClipping ()”函数 。For ClippingButton select ToggleButton > ToggleClipping () function.

3.配置“视图按钮控件(脚本)”和“切换按钮(脚本)”组件3. Configuring the View Button Control (Script) and Toggle Button (Script) components

现在根据配置,你的按钮会演示模型切换和剪辑功能。Now your buttons are configured to demonstrate the model switching and clipping functionality. 现可向脚本中添加 3D 模型和剪辑对象。It is time to add 3D models and the clipping objects to the script.

我们提供了 6 种不同的 3D 模型用于演示,展开 ModelParentobject 可公开这些 3D 模型。We have provided six different 3D models for demonstration, expand the *ModelParentobject _ to expose these 3D models.

在“层次结构”窗口中仍选中 ButtonParent 对象的情况下,在检查器窗口中找到“视图按钮控件(脚本)”组件并展开“模型”变量。With the ButtonParent object still selected in the Hierarchy window, in the Inspector window, locate the _ View Button Control (Script)* component and expand the Models variable.

在“大小”字段中,输入要在场景中拥有的 3D 模型的数量。In the Size field, enter the number of 3D models you would like to have in your scene. 在本例中,为 6 个模型。In this case, it would be six. 它将创建用于添加新 3D 模型的字段。It will create fields for adding new 3D models.

具有 ViewButtonControl 脚本组件字段的 Unity

将 ModelParent 对象的每个子对象拖放到这些字段中。Drag and drop each child object of ModelParent Object into these fields.

配置了 ViewButtonControl 脚本组件字段的 Unity

将 ClippingObjects 对象从“层次结构”窗口拖放到“切换按钮(脚本)”组件的“剪辑对象”字段 。Drag and drop the ClippingObjects object from the Hierarchy window to the Toggle Button (Script) component Clipping Object field.

备注

仅保留按钮父对象。Stay in button parent object only.

配置了 ToggleButton 脚本组件字段的 Unity

在“层次结构”窗口中,选择 ClippingObjects 预制件,并在检查器窗口中启用它来打开剪辑对象。In the Hierarchy window, select the ClippingObjects prefab and enable it in the Inspector window to turn on the Clipping objects.

配置剪辑对象以启用剪辑功能Configuring the clipping objects to enable clipping feature

在本部分中,你将向单个剪辑对象添加 MarsCuriosityRover 对象的子对象渲染器,演示对 MarsCuriosityRover 模型的剪辑。In this section, you will add MarsCuriosityRover object's child objects renderer into an individual clipping object to demonstrate the clipping of the MarsCuriosityRover model.

在“层次结构”窗口中,展开 ClippingObjects 对象,显示你将在此项目中使用的 3 个不同的剪辑对象。In the Hierarchy window, expand the ClippingObjects object to expose the three different clipping objects that you will be using in this project.

若要配置 ClippingSphere 对象,请单击它,然后在检查器窗口中找到“剪辑球体(脚本)”组件 。To configure the ClippingSphere object, click on it, and in the Inspector window, locate the Clipping Sphere (Script) component. 在“大小”字段中输入需要为 3D 模型添加的渲染器数量。Enter the number of renderers in the size field that you need to add for your 3D model. 在本例中,为 MarsCuriosityRover 子对象添加 10 个渲染器。In this case, add 10 for MarsCuriosityRover child objects. 它将创建用于添加渲染器的字段,并将 MarsCuriosityRover 对象的子模型对象拖放到这些字段中。It will create fields for adding renderers, drag and drop MarsCuriosityRover Object's child model objects into these fields.

配置了 ClippingSphere 脚本组件字段的 Unity

按照相同的过程,将 MarsCuriosityRover 的子对象渲染器添加到 ClippingBox 和 ClippingPlane 对象 。Follow the same process and add MarsCuriosityRover's child objects renderers to the ClippingBox and ClippingPlane objects.

在本教程中,只有 MarsCuriosityRover 模型将用于演示剪辑功能。In this tutorial, only the MarsCuriosityRover model will be used for demonstrating the clipping feature. 它们向更多的模型添加剪辑功能,从而增加了渲染器的大小,并添加了其各自的网格渲染器。They were adding clipping features to more models, increasing the size of the renderer, and adding their individual mesh renderers.

配置眼动跟踪以突出显示工具提示Configuring eye-tracking to highlight tooltips

本部分探讨如何在项目中启用眼动跟踪。In this section, you will explore how to enable eye tracking in your project. 例如,你将实现这样的功能,即在查看 MarsCuriosityRover 各部分时突出显示附加到它们的工具提示,在不看这些部分时隐藏这些提示。For example, you will implement the functionality to highlight tooltips attached to MarsCuriosityRover's parts while looking at them and hiding them, while you are looking away from them.

1.确定目标对象及关联的工具提示1. Identify target objects and associated tooltips

在“层次结构”窗口中,选择 ModelParent 对象。In the Hierarchy window, select the ModelParent object. 展开“MarsCuriosity”->“探测器”,找到 MarsCuriosityRover 的五个主要部分:POI-Camera、POI-Wheels、POI-Antena、POI-Spectrometer、POI-RUHF Antenna 。Expand the *MarsCuriosity -> Rover_ to find five main parts of the MarsCuriosityRover: _* POI-Camera**, POI-Wheels, POI-Antena, POI-Spectrometer, POI-RUHF Antenna.

  • 在“层次结构”窗口中观察与 MarsCuriosityRover 各部分关联的 5 个对应的工具提示对象。Observe five corresponding tooltip objects associated with MarsCuriosityRover parts in the Hierarchy window.
  • 你将配置这些对象,以便在查看 MarsCuriosityRover 各部分时突出显示体验。You will be configuring these objects to highlight the experience when you look at the MarsCuriosityRover parts.

选中并展开了“探测器”对象的 Unity

2.实现 While Looking At Target () 和 On Look Away () 事件2. Implement While Looking At Target () & On Look Away () events

在“层次结构”窗口中,选择 POI-Camera 对象。In the Hierarchy window, select the *POI-Camera _ object. 在检查器窗口中,找到“眼动跟踪目标(脚本)”组件,并配置“While Looking At Target ()” & “On Look Away ()”事件,如下所示 :In the Inspector window, locate the _ Eye Tracking Target (Script)* component and configure the While Looking At Target () & On Look Away () events as follows:

  • 对于“无(对象)”字段,指定“POI Camera 工具提示”对象 To None (Object) field, assign the POI-Camera ToolTip object
  • 从“While Looking At Target ()”事件的“无函数”下拉列表中,选择“GameObject” > “SetActive (bool)” 。From No Function dropdown of While Looking At Target () event, select GameObject > SetActive (bool). 选中它下面的“复选框”,将突出显示工具提示作为在查看目标对象时触发的操作。Select the Checkbox under it to highlight the tooltip as the action that is triggered when you look at the target object.

Unity 正在配置 EyeTrackingTarget WhileLookingAtTarget 事件

  • 按照相同的过程,单击“On Look Away ()”事件侦听器上的“无函数”下拉列表 。Follow the same process and click on the No Function dropdown of the On Look Away () event listener. 然后选择“GameObject” > “SetActive (bool)”,并将复选框留空,将隐藏工具提示作为在不看目标对象时触发的操作 。Then select GameObject > SetActive (bool) and leave the Checkbox empty to hide the tooltip as the action that is triggered when you look away from the target object.

配置了 EyeTrackingTarget OnLookAway 事件的 Unity

按照相同的过程,将各自的工具提示对象分配给对应的 MarsCuriosityRover 部分的“While Looking At Target ()”和“On Look Away ()”事件 。Follow the same process and assign respective tooltip objects to their same MarsCuriosityRover parts While Looking At Target () & On Look Away () events.

若要启用眼动跟踪,请遵循这些准则To enable eye tracking, please follow these guidelines.

祝贺Congratulations

在本教程中,你学习了如何构建混合现实体验来演示 UI 元素、3D 模型操作、模型剪辑和眼动跟踪功能。In this tutorial, you learned to build a mixed reality experience demonstrating UI elements, 3D model manipulation, model clipping, and eye-tracking features. 本教程提供了 NextButton 和 PreviousButton,便于你探索 3D 模型查看器的体验。The tutorial provided you with NextButton and PreviousButton that let you explore the 3D model viewer experience. 通过 ClippingObjectButton,你打开了剪辑对象,还体验了剪辑功能。The ClippingObjectButton made you turn on clipping objects and experience clipping feature. 本教程还为你提供了眼动跟踪元素,以便在体验中突出显示工具提示。The tutorial also provided you with an eye-tracking element to enable highlighting the tooltips in the experience.

在下堂课中,你将学习如何为电脑创建全息远程处理应用程序来随时连接 HoloLens 2,从而可在混合现实中直观呈现 3D 内容。In the next lesson, you will learn how to create a Holographic Remoting application for PC to connect HoloLens 2 at any point, providing a way to Visualize 3D content in mixed reality.