MR 共享 250:HoloLens 和沉浸式头戴显示设备MR Sharing 250: HoloLens and immersive headsets

备注

混合现实学院教程在制作时考虑到了 HoloLens(第一代)和混合现实沉浸式头戴显示设备。The Mixed Reality Academy tutorials were designed with HoloLens (1st gen) and Mixed Reality Immersive Headsets in mind. 因此,对于仍在寻求这些设备的开发指导的开发人员而言,我们觉得很有必要保留这些教程。As such, we feel it is important to leave these tutorials in place for developers who are still looking for guidance in developing for those devices. 我们 不会 在这些教程中更新 HoloLens 2 所用的最新工具集或集成相关的内容。These tutorials will not be updated with the latest toolsets or interactions being used for HoloLens 2. 我们将维护这些教程,使之持续适用于支持的设备。They will be maintained to continue working on the supported devices. 已经为 HoloLens 2 发布了一系列新教程A new series of tutorials has been posted for HoloLens 2.

利用通用 Windows 平台 (UWP) 的灵活性,可以轻松创建跨多个设备的应用程序。With the flexibility of Universal Windows Platform (UWP), it is easy to create an application that spans multiple devices. 通过这种灵活性,我们可以创建利用每个设备的优势的体验。With this flexibility, we can create experiences that leverage the strengths of each device. 本教程将介绍在 HoloLens 和 Windows Mixed Reality 沉浸式耳机上运行的基本共享体验。This tutorial will cover a basic shared experience that runs on both HoloLens and Windows Mixed Reality immersive headsets. 此内容最初在华盛顿州西雅图的 Microsoft Build 2017 大会上交付。This content was originally delivered at the Microsoft Build 2017 conference in Seattle, WA.

在本教程中,我们将:In this tutorial, we will:

  • 使用 UNET 设置网络。Setup a network using UNET.
  • 跨混合现实设备共享全息影像。Share holograms across mixed reality devices.
  • 根据正在使用的混合现实设备,建立不同的应用程序视图。Establish a different view of the application depending on which mixed reality device is being used.
  • 通过一些简单的测验题,为 HoloLens 用户指导沉浸式耳机用户。Create a shared experience where HoloLens users guide immersive headsets users through some simple puzzles.

设备支持Device support

课程Course HoloLensHoloLens 沉浸式头戴显示设备Immersive headsets
MR 共享 250:HoloLens 和沉浸式头戴显示设备MR Sharing 250: HoloLens and immersive headsets ✔️✔️ ✔️✔️

准备工作Before you start

必备条件Prerequisites

项目文件Project files

备注

如果要在下载之前查看源代码, 可在 GitHub 上找到。If you want to look through the source code before downloading, it's available on GitHub.

第1章-Holo WorldChapter 1 - Holo World

目标Objectives

请确保开发环境已准备好使用简单项目。Make sure the development environment is ready to go with a simple project.

要生成的内容What we will build

在 HoloLens 或 Windows Mixed Reality 沉浸式耳机上显示全息图的应用程序。An application that shows a hologram on either HoloLens or a Windows Mixed Reality immersive headset.

步骤Steps

  • 打开 Unity。Open Unity.
    • 选择“打开” 。Select Open.
    • 导航到在其中提取项目文件的位置。Navigate to where you extracted the project files.
    • 单击“选择文件夹”。Click Select Folder.
    • Unity 第一次处理项目需要一些时间。It will take a little while for Unity to process the project the first time.
  • 检查 Unity 中是否已启用混合现实。Check that Mixed Reality is enabled in Unity.
    • 打开 "生成设置" 对话框 (Control + Shift + B> 生成设置 ...) "。Open the build settings dialog (Control+Shift+B or File > Build Settings...).
    • 选择 通用 Windows 平台 然后单击 " 切换平台"。Select Universal Windows Platform then click Switch Platform.
    • 选择 " 编辑>播放机设置"。Select Edit>Player Settings.
    • 在右侧的 检查器 面板中,展开 " XR 设置"。In the Inspector panel on the right hand side, expand XR Settings.
    • 选中 " 支持虚拟现实 " 框。Check the Virtual Reality Supported box.
    • Windows Mixed Reality 应为虚拟现实 SDK。Windows Mixed Reality should be the Virtual Reality SDK.
  • 创建场景。Create a scene.
    • 层次结构 中右键单击 " 主相机 ",然后选择 " 删除"。In the Hierarchy right click Main Camera select Delete.
    • HoloToolkit > 输入 > prototypingMixedRealityCameraParent 拖到 层次结构 中。From HoloToolkit > Input > Prefabs drag MixedRealityCameraParent to the Hierarchy.
  • 向场景中添加全息影像Add Holograms to the scene
    • From AppPrefabsSkybox 拖到 场景视图From AppPrefabs drag Skybox to the Scene View.
    • From AppPrefabs经理 拖到 层次结构 中。From AppPrefabs drag Managers to the Hierarchy.
    • From AppPrefabs 拖到 层次结构 中。From AppPrefabs drag Island to the Hierarchy.
  • 保存并生成Save And build
    • 保存 (Control + SFile > save 场景) Save (either Control+S or File > Save Scene)
    • 由于这是新场景,因此需要将其命名为。Since this is a new scene, you'll need to name it. 名称并不重要,但我们使用的是 SharedMixedReality。Name doesn't matter, but we use SharedMixedReality.
  • 导出到 Visual StudioExport To Visual Studio
    • 打开 "生成" 菜单 (Control + Shift + BFile > 生成设置 ") Open the build menu (Control+Shift+B or File > Build Settings)
    • 单击 " 添加打开的场景"。Click Add Open Scenes.
    • 检查 Unity c # 项目Check Unity C# Projects
    • 单击“生成”。Click Build.
    • 在出现的 "文件资源管理器" 窗口中,创建一个名为 " 应用" 的新文件夹。In the file explorer window that appears, create a New Folder named App.
    • 单击 应用 文件夹。Single click the App folder.
    • 按 " 选择文件夹"。Press Select Folder.
    • 等待生成完成Wait for the build to complete
    • 在出现的 "文件资源管理器" 窗口中,导航到 " 应用 " 文件夹。In the file explorer window that appears, navigate into the App folder.
    • 双击 " SharedMixedReality " 以启动 Visual StudioDouble-click SharedMixedReality.sln to launch Visual Studio
  • 从 Visual Studio 生成Build From Visual Studio
    • 使用顶部工具栏将目标更改为 " 发布 " 和 " x86"。Using the top toolbar change target to Release and x86.
    • 单击 " 本地计算机 " 旁边的箭头,并选择 "要部署到 HoloLens 的 设备 "Click the arrow next to Local Machine and select Device to deploy to HoloLens
    • 单击 " 设备 " 旁边的箭头,并选择 "要为混合现实耳机部署的 本地计算机 "。Click the arrow next to Device and select Local Machine to deploy for the mixed reality headset.
    • 单击 " 调试"->"无调试开始 " 或按 F5 启动应用程序。Click Debug->Start Without Debugging or Control+F5 to start the application.

深入到代码中Digging into the code

在 "项目" 面板中,导航到 " Assets\HoloToolkit\Input\Scripts\Utilities ",然后双击 " MixedRealityCameraManager.cs " 将其打开。In the project panel, navigate to Assets\HoloToolkit\Input\Scripts\Utilities and double click MixedRealityCameraManager.cs to open it.

概述: MixedRealityCameraManager.cs 是一个简单的脚本,可根据设备调整质量级别和背景设置。Overview: MixedRealityCameraManager.cs is a simple script that adjusts quality level and background settings based on the device. 此处的密钥为 HolographicSettings。 IsDisplayOpaque,它允许脚本检测设备是否为 HoloLens (IsDisplayOpaque 返回 false) 或沉浸式耳机 (IsDisplayOpaque 返回 true) 。Key here is HolographicSettings.IsDisplayOpaque, which allows a script to detect if the device is a HoloLens (IsDisplayOpaque returns false) or an immersive headset (IsDisplayOpaque returns true).

欣赏你的进度Enjoy your progress

此时,应用程序将只呈现一个全息图。At this point the application will just render a hologram. 稍后我们将向全息图添加交互。We will add interaction to the hologram later. 这两个设备会将两个全息图呈现相同的。Both devices will render the hologram the same. 沉浸式耳机还会呈现蓝天和云背景。The immersive headset will also render a blue sky and clouds background.

第2章-交互Chapter 2 - Interaction

目标Objectives

演示如何处理 Windows Mixed Reality 应用程序的输入。Show how to handle input for a Windows Mixed Reality application.

要生成的内容What we will build

基于第1章中的应用程序,我们将添加功能,使用户能够选择全息影像,并将其放在 HoloLens 的真实表面上,或将其放置在沉浸式耳机中的虚拟表上。Building on the application from chapter 1, we will add functionality to allow the user to pick up the hologram and place it on a real world surface in HoloLens or on a virtual table in an immersive headset.

输入刷新器: 在 HoloLens 上,选择手势是 点击Input Refresher: On HoloLens the select gesture is the air tap. 在沉浸式耳机上,我们将使用 Xbox 控制器上 的按钮。On immersive headsets, we will use the A button on the Xbox controller. 有关详细信息,请查看 交互模型概述For more information check out the interaction model overview.

步骤Steps

  • 添加输入管理器Add Input manager
    • HoloToolkit > 输入 > prototypingInputManager 作为 经理 的子项拖到 层次结构 中。From HoloToolkit > Input > Prefabs drag InputManager to Hierarchy as a child of Managers.
    • HoloToolkit > 输入 > prototyping > 光标光标 拖到 层次结构 中。From HoloToolkit > Input > Prefabs > Cursor drag Cursor to Hierarchy.
  • 添加空间映射Add Spatial Mapping
    • From HoloToolkit > SpatialMapping > prototypingSpatialMapping 拖到 层次结构 中。From HoloToolkit > SpatialMapping > Prefabs drag SpatialMapping to Hierarchy.
  • 添加虚拟 PlayspaceAdd Virtual Playspace
    • 层次结构 中展开 MixedRealityCameraParent 选择 边界In Hierarchy expand MixedRealityCameraParent select Boundary
    • 检查 面板中的复选框以启用 边界In Inspector panel check the box to enable Boundary
    • From AppPrefabsVRRoom 拖到 层次结构 中。From AppPrefabs drag VRRoom to Hierarchy.
  • 添加 WorldAnchorManagerAdd WorldAnchorManager
    • 层次结构 中,选择 " 管理器"。In Hierarchy, Select Managers.
    • 检查器 中,单击 " 添加组件"。In Inspector, click Add Component.
    • 键入 " 世界定位管理器"。Type World Anchor Manager.
    • 选择 " 世界锚管理器 " 以添加它。Select World Anchor Manager to add it.
  • 将 TapToPlace 添加到岛Add TapToPlace to the Island
    • 层次结构 中,展开 " "。In Hierarchy, expand Island.
    • 选择 MixedRealityLandSelect MixedRealityLand.
    • 检查器 中,单击 " 添加组件"。In Inspector, click Add Component.
    • 键入 " 点击" 以将 其选中。Type Tap To Place and select it.
    • 在点击时选中 "父项"Check Place Parent On Tap.
    • 放置偏移 设置为 (0,0.1,0)Set Placement Offset to (0, 0.1, 0).
  • 像以前一样保存并生成Save and Build as before

深入到代码中Digging into the code

脚本 1-GamepadInput.csScript 1 - GamepadInput.cs

在 "项目" 面板中,导航到 " Assets\HoloToolkit\Input\Scripts\InputSources ",然后双击 " GamepadInput.cs " 将其打开。In the project panel navigate to Assets\HoloToolkit\Input\Scripts\InputSources and double click GamepadInput.cs to open it. 在 "项目" 面板的同一路径中,双击 " InteractionSourceInputSource.cs"。From the same path in the project panel, also double click InteractionSourceInputSource.cs.

请注意,这两个脚本都有一个公共的基类 BaseInputSource。Note that both scripts have a common base class, BaseInputSource.

BaseInputSource 保留对 InputManager 的引用,这允许脚本触发事件。BaseInputSource keeps a reference to an InputManager, which allows a script to trigger events. 在这种情况下,InputClicked 事件是相关的。In this case, the InputClicked event is relevant. 当我们进入脚本 2 TapToPlace 时,请务必记住这一点。This will be important to remember when we get to script 2, TapToPlace. 对于 GamePadInput,我们轮询控制器上要按下的按钮,然后引发 InputClicked 事件。In the case of GamePadInput, we poll for the A button on the controller to be pressed, then we raise the InputClicked event. 对于 InteractionSourceInputSource,我们将引发 InputClicked 事件以响应 TappedEvent。In the case of InteractionSourceInputSource, we raise the InputClicked event in response to the TappedEvent.

脚本 2-TapToPlace.csScript 2 - TapToPlace.cs

在 "项目" 面板中,导航到 " Assets\HoloToolkit\SpatialMapping\Scripts ",然后双击 " TapToPlace.cs " 将其打开。In the project panel navigate to Assets\HoloToolkit\SpatialMapping\Scripts and double click TapToPlace.cs to open it.

许多开发人员在创建全息应用程序时要实现的第一件事是使用手势输入来移动全息影像。The first thing many developers want to implement when creating a Holographic application is moving Holograms with gesture input. 同样,我们已 endeavored 对此脚本进行全面注释。As such, we've endeavored to thoroughly comment this script. 对于本教程,有几个值得注意的事项。A few things are worth highlighting for this tutorial.

首先,请注意,TapToPlace 实现了 IInputClickHandler。First, note that TapToPlace implements IInputClickHandler. IInputClickHandler 公开处理 GamePadInput.cs 或 InteractionSourceInputSource.cs 引发的 InputClicked 事件的函数。IInputClickHandler exposes the functions that handle the InputClicked event raised by GamePadInput.cs or InteractionSourceInputSource.cs. 当 BaseInputSource 检测到具有 TapToPlace 的对象时,将调用 OnInputClicked。OnInputClicked is called when a BaseInputSource detects a click while the object with TapToPlace is in focus. 在 HoloLens 上 airtapping 或按下 Xbox 控制器上的一个按钮将触发该事件。Either airtapping on HoloLens or pressing the A button on the Xbox controller will trigger the event.

第二是在更新中执行代码以查看是否正在查看某个面,以便我们可以将游戏对象放置在图面上,如表。Second is the code be executed in update to see if a surface is being looked at so we can place the game object on a surface, like a table. 沉浸式头戴式耳机没有实表面的概念,因此表示表顶部 (Vroom > TableThingy > 多维数据集) 的对象已使用 SpatialMapping 物理学层进行标记,因此更新中的 ray 转换将与虚拟表的顶部发生冲突。The immersive headset doesn't have a concept of real surfaces, so the object that represents the table top (Vroom > TableThingy > Cube) has been marked with the SpatialMapping physics layer, so the ray cast in Update will collide with the virtual table top.

欣赏你的进度Enjoy your progress

这一次,您可以选择要移动的岛。This time you can select the island to move it. 在 HoloLens 上,你可以将岛移动到真实表面。On HoloLens you can move the island to a real surface. 在沉浸式耳机中,你可以将岛移动到我们添加的虚拟表。In the immersive headset you can move the island to the virtual table we added.

第3章-共享Chapter 3 - Sharing

目标Objectives

确保已正确配置网络,并详细说明如何在设备之间共享空间锚。Ensure that the network is correctly configured and detail how spatial anchors are shared between devices.

要生成的内容What we will build

我们会将项目转换为多玩家项目。We will convert our project to a multiplayer project. 我们会将 UI 和逻辑添加到主机或加入会话。We will add UI and logic to host or join sessions. HoloLens 用户会在会话中使用其打印头上的云查看彼此,并且沉浸式头戴式耳机用户在锚定到的位置附近有云。HoloLens users will see each other in the session with clouds over their heads, and immersive headset users have clouds near to where the anchor is. 沉浸式耳机中的用户将看到 HoloLens 用户相对场景的原点。Users in the immersive headsets will see the HoloLens users relative to the origin of the scene. HoloLens 用户将在同一位置看到岛的全息影像。HoloLens users will all see the hologram of the island in the same place. 需要注意的是,沉浸式耳机中的用户将不会在本章节中的岛上出现,但会表现得非常类似于 HoloLens,并具有岛的鸟瞰视图。It is key to note that the users in the immersive headsets will not be on the island during this chapter, but will behave very similarly to HoloLens, with a birds eye view of the island.

步骤Steps

  • 删除岛和 VRRoomRemove Island and VRRoom
    • 层次结构 中右键单击 选择 删除In Hierarchy right-click Island select Delete
    • 层次结构 中右键单击 VRRoom 选择 删除In Hierarchy right-click VRRoom select Delete
  • 添加 UslandAdd Usland
    • From AppPrefabsUsland 拖到 层次结构 中。From AppPrefabs drag Usland to Hierarchy.
  • AppPrefabs 将以下各项拖动到 层次结构From AppPrefabs drag each of the following to Hierarchy:
    • UNETSharingStageUNETSharingStage
    • UNetAnchorRootUNetAnchorRoot
    • UIContainerUIContainer
    • DebugPanelButtonDebugPanelButton
  • 像以前一样保存并生成Save and Build as before

深入到代码中Digging into the code

在 "项目" 面板中,导航到 " Assets\AppPrefabs\Support\SharingWithUnet\Scripts ",然后双击 " UnetAnchorManager.cs"。In the project panel, navigate to Assets\AppPrefabs\Support\SharingWithUnet\Scripts and double-click on UnetAnchorManager.cs. 一个 HoloLens 与另一个 HoloLens 共享跟踪信息,使这两个设备可以共享相同空间的功能在神奇附近。The ability for one HoloLens to share tracking information with another HoloLens such that both devices can share the same space is near magical. 当两个或多个用户可以使用相同的数字数据进行协作时,混合现实的强大功能将处于活动状态。The power of mixed reality comes alive when two or more people can collaborate using the same digital data.

在此脚本中需要指出的几点:A few things to point out in this script:

在 start 函数中,请注意对 IsDisplayOpaque 的检查。In the start function, notice the check for IsDisplayOpaque. 在这种情况下,我们假设已建立定位点。In this case, we pretend that the Anchor is established. 这是因为沉浸式耳机不公开导入或导出锚的方式。This is because the immersive headsets do not expose a way to import or export anchors. 但是,如果我们在 HoloLens 上运行,则此脚本在设备之间实现共享锚。If we are running on a HoloLens, however, this script implements sharing anchors between the devices. 启动会话的设备将创建用于导出的定位点。The device that starts the session will create an anchor for exporting. 加入会话的设备将从启动会话的设备请求定位点。The device that joins a session will request the anchor from the device that started the session.

输出Exporting:

当用户创建会话时,NetworkDiscoveryWithAnchors 将调用 UNETAnchorManagers CreateAnchor 函数。When a user creates a session, NetworkDiscoveryWithAnchors will call UNETAnchorManagers CreateAnchor function. 我们来看看 CreateAnchor flow。Let's follow CreateAnchor flow.

首先,我们会进行一些内务处理,清除我们为先前定位点收集的任何数据。We start by doing some housekeeping, clearing out any data we may have collected for previous anchors. 然后检查是否有要加载的缓存锚。Then we check if there is a cached anchor to load. 定位点数据的大小通常介于5到 20 MB 之间,因此重复使用缓存的定位点可以节省需要通过网络传输的数据量。The anchor data tends to be between 5 and 20 MB, so reusing cached anchors can save on the amount of data we need to transfer over the network. 稍后我们将介绍此功能的工作原理。We'll see how this works a bit later. 即使我们正在重用定位点,我们也需要在没有定位点的新客户端联接的情况下获取定位点数据。Even if we are reusing the anchor, we need to get the anchor data ready in case a new client joins that doesn't have the anchor.

说到定位点数据已准备就绪,WorldAnchorTransferBatch 类公开了准备用于发送到其他设备或应用程序的定位点数据的功能,并提供了导入定位数据的功能。Speaking of getting the anchor data ready, the WorldAnchorTransferBatch class exposes the functionality to prepare anchor data for sending to another device or application and the functionality to import the anchor data. 由于我们的是导出路径,因此,我们会将定位点添加到 WorldAnchorTransferBatch 并调用 ExportAsync 函数。Since we're on the export path, we will add our anchor to the WorldAnchorTransferBatch and call the ExportAsync function. 然后,ExportAsync 将调用 WriteBuffer 回调,因为它会生成导出的数据。ExportAsync will then call the WriteBuffer callback as it generates data for export. 导出所有数据后,将调用 ExportComplete。When all of the data has been exported ExportComplete will be called. 在 WriteBuffer 中,我们将数据块添加到要导出的列表。In WriteBuffer we add the chunk of data to a list we keep for exporting. 在 ExportComplete 中,我们将列表转换为数组。In ExportComplete we convert the list to an array. 还设置了 AnchorName 变量,这会触发其他设备请求定位点(如果没有)。The AnchorName variable is also set, which will trigger other devices to request the anchor if they don't have it.

在某些情况下,定位点将不会导出或创建很少的数据,我们将重试。In some cases the anchor won't export or will create so little data that we will try again. 这里我们再次调用 CreateAnchor。Here we just call CreateAnchor again.

导出路径中的 final 函数为 AnchorFoundRemotely。A final function in the export path is AnchorFoundRemotely. 当另一个设备找到定位点时,该设备将通知主机,主机会将其用作信号,指示锚点为 "良好锚定" 并可缓存。When another device finds the anchor, that device will tell the host, and the host will use that as a signal that the anchor is a "good anchor" and can be cached.

导入Importing:

当 HoloLens 加入会话时,它需要导入一个定位点。When a HoloLens joins a session, it needs to import an anchor. 在 UNETAnchorManager 的 Update 函数中,将对 AnchorName 进行轮询。In UNETAnchorManager's Update function, the AnchorName is polled. 定位点名称更改时,导入过程开始。When the anchor name changes, the import process begins. 首先,我们尝试从本地定位点存储中加载具有指定名称的定位点。First, we try to load the anchor with the specified name from the local anchor store. 如果已有该数据,则可以使用它,而无需再次下载数据。If we already have it, we can use it without downloading the data again. 如果没有,请调用 WaitForAnchor,这将启动下载。If we don't have it, then we call WaitForAnchor which will initiate the download.

下载完成后,将调用 NetworkTransmitter_dataReadyEvent。When the download is completed, NetworkTransmitter_dataReadyEvent is called. 这会向更新循环发出信号,以便对下载的数据调用 ImportAsync。This will signal the Update loop to call ImportAsync with the downloaded data. 导入过程完成后,ImportAsync 将调用 ImportComplete。ImportAsync will call ImportComplete when the import process is complete. 如果导入成功,定位点将保存在本地播放机存储区中。If the import is successful, the anchor will be saved in the local player store. PlayerController.cs 实际上会调用 AnchorFoundRemotely,让主机知道已经建立好锚。PlayerController.cs actually makes the call to AnchorFoundRemotely to let the host know that a good anchor has been established.

欣赏你的进度Enjoy your progress

这次使用 HoloLens 的用户将使用用户界面中的 " 启动会话 " 按钮来托管会话。This time a user with a HoloLens will host a session using the start session button in the UI. 在 HoloLens 或沉浸式耳机上,其他用户将选择该会话,然后在 UI 中选择 " 加入会话 " 按钮。Other users, both on HoloLens or an immersive headset, will select the session and then select the join session button in the UI. 如果有多个用户使用 HoloLens 设备,它们会在其打印头上包含红色的云。If you have multiple people with HoloLens devices, they will have red clouds over their heads. 对于每个沉浸式耳机,还会有一个蓝色的云,但蓝色云不会位于耳机上方,因为耳机不会尝试查找与 HoloLens 设备相同的世界坐标空间。There will also be a blue cloud for each immersive headset, but the blue clouds will not be above the headsets, as the headsets are not trying to find the same world coordinate space as the HoloLens devices.

项目中的这一点是包含共享应用程序;这并不是很多,而且可以充当基线。This point in the project is a contained sharing application; it doesn't do very much, and could act as a baseline. 在接下来的章节中,我们将开始为用户带来体验。In the next chapters, we will start building an experience for people to enjoy. 若要获取有关共享体验设计的更多指导,请转到此处。To get further guidance on shared experience design, go here.

第4章-浸入式和 teleportingChapter 4 - Immersion and teleporting

目标Objectives

为每种类型的混合现实设备满足经验。Cater the experience to each type of mixed reality device.

要生成的内容What we will build

我们将更新应用程序,以通过沉浸式视图将沉浸式头戴式耳机用户置于岛中。We will update the application to put immersive headset users on the island with an immersive view. HoloLens 用户仍将获得岛的鸟瞰视图。HoloLens users will still have the bird's eye view of the island. 每种设备类型的用户可以看到其他用户出现在世界中。Users of each device type can see other users as they appear in the world. 例如,沉浸式耳机用户可以查看岛上其他路径上的其他头像,并在岛上看到 HoloLens 用户为巨大的云。For instance, immersive headset users can see the other avatars on other paths on the island, and they see the HoloLens users as giant clouds above the island. 如果 HoloLens 用户正在查看岛,则沉浸式耳机用户也会看到 HoloLens 用户的 "注视" 的光标。Immersive headset users will also see the cursor of the HoloLens user's gaze ray if the HoloLens user is looking at the island. HoloLens 用户将看到岛上的头像来表示每个沉浸式耳机用户。HoloLens users will see an avatar on the island to represent each immersive headset user.

已更新沉浸式设备的输入:Updated Input for the Immersive device:

  • Xbox 控制器上的左缓冲器和右缓冲器按钮将旋转播放机The left bumper and right bumper buttons on the Xbox controller rotate the player
  • 按住 Xbox 控制器上的 Y 按钮将启用 传送 光标。Holding the Y button on the Xbox controller will enable a teleport cursor. 如果在释放 Y 按钮时光标有旋转箭头指示器,则会 teleported 光标所在的位置。If the cursor has a spinning arrow indicator when you release the Y button, you will be teleported to the cursor's location.

步骤Steps

  • 将 MixedRealityTeleport 添加到 MixedRealityCameraParentAdd MixedRealityTeleport to MixedRealityCameraParent
    • 层次结构 中,选择 UslandIn Hierarchy, select Usland.
    • 检查器 中,启用 级别控制In Inspector, enable Level Control.
    • 层次结构 中,选择 MixedRealityCameraParentIn Hierarchy, select MixedRealityCameraParent.
    • 检查器 中,单击 " 添加组件"。In Inspector, click Add Component.
    • 键入 混合现实传送 并将其选中。Type Mixed Reality Teleport and select it.

深入到代码中Digging into the code

沉浸式耳机用户将通过电缆受限到其电脑上,但岛比电缆长。Immersive headset users will be tethered to their PCs with a cable, but our island is larger than the cable is long. 若要进行补偿,我们需要能够独立于用户的动作移动相机。To compensate, we need the ability to move the camera independently of the user's motion. 有关设计混合现实应用 (程序的详细 信息,请参阅 locomotion) 。Please see the comfort page for more information about designing your mixed reality application (in particular self motion and locomotion).

若要描述此过程,请定义两个术语。In order to describe this process it will be useful to define two terms. 首先, dolly 将是独立于用户移动相机的对象。First, dolly will be the object that moves the camera independently from the user. Dolly 的子游戏对象将是 主摄像机A child game object of the dolly will be the main camera. 将相机连接到用户的头。The main camera is attached to the user's head.

在 "项目" 面板中,导航到 " Assets\AppPrefabs\Support\Scripts\GameLogic ",然后双击 " MixedRealityTeleport.cs"。In the project panel, navigate to Assets\AppPrefabs\Support\Scripts\GameLogic and double-click on MixedRealityTeleport.cs.

MixedRealityTeleport 有两个作业。MixedRealityTeleport has two jobs. 首先,它使用缓冲器处理旋转。First, it handles rotation using the bumpers. 在更新函数中,我们轮询 LeftBumper 和 RightBumper 上的 "ButtonUp"。In the update function we poll for 'ButtonUp' on LeftBumper and RightBumper. GetButtonUp 仅在第一帧上返回 true,按钮在关闭后就会正常运行。GetButtonUp only returns true on the first frame a button is up after having been down. 如果引发了任一按钮,则我们知道用户需要旋转。If either button had been raised, then we know the user needs to rotate.

旋转时,我们使用名为 "淡化 control" 的简单脚本,进行淡出并淡出。When we rotate we do a fade out and fade in using a simple script called 'fade control'. 这样做是为了防止用户看到可能导致 discomfort 的非自然移动。We do this to prevent the user from seeing an unnatural movement which could lead to discomfort. 淡入和淡出效果非常简单。The fade in and out effect is fairly simple. 摄像机 前面有一个黑色的故障诊断。We have a black quad hanging in front of the main camera. 淡出时,将从0到1转换 alpha 值。When fading out we transition the alpha value from 0 to 1. 这会逐渐导致四个黑色像素渲染和遮蔽。This gradually causes the black pixels of the quad to render and obscure anything behind them. 当淡化后,转换后的 alpha 值将返回零。When fading back in we transition the alpha value back to zero.

计算旋转时,请注意,我们要旋转 dolly ,但计算出 摄像机 周围的旋转。When we calculate the rotation, note that we are rotating our dolly but calculating the rotation around the main camera. 这一点很重要,因为在 摄像机 远离0,0,0时,dolly 的旋转就会从用户的角度来看。This is important as the farther the main camera is away from 0,0,0, the less accurate a rotation around the dolly would become from the point of view of the user. 事实上,如果不围绕相机位置旋转,用户将在 dolly 上移动,而不是旋转。In fact, if you do not rotate around the camera position, the user will move on an arc around the dolly rather than rotating.

MixedRealityTeleport 的第二个作业是处理移动 dollyThe second job for MixedRealityTeleport is to handle moving the dolly. 这是在 SetWorldPosition 中完成的。This is done in SetWorldPosition. SetWorldPosition 采用所需的世界位置,该位置是用户想要 percieve 具有的位置。SetWorldPosition takes the desired world position, the position where the user wants to percieve that they inhabit. 我们需要将 dolly 置于该位置,而不是 主摄像机 的本地位置,因为每帧都会添加该偏移量。We need to put our dolly at that position minus the local position of the main camera, as that offset will be added each frame.

第二个脚本调用 SetWorldPosition。A second script calls SetWorldPosition. 让我们看看该脚本。Let's look at that script. 在 "项目" 面板中,导航到 " Assets\AppPrefabs\Support\Scripts\GameLogic ",然后双击 " TeleportScript.cs"。In the project panel, navigate to Assets\AppPrefabs\Support\Scripts\GameLogic and double-click on TeleportScript.cs.

此脚本比 MixedRealityTeleport 更复杂一些。This script is a little more involved than MixedRealityTeleport. 脚本正在检查 Xbox 控制器上的 Y 按钮是否已关闭。The script is checking for the Y Button on the Xbox controller to be held down. 当按钮被按下时,将呈现一个传送光标,并且该脚本将从用户的注视位置转换一条射线。While the button is held down a teleport cursor is rendered and the script casts a ray from the user's gaze position. 如果该射线与更多或更少地指向的图面冲突,则会将该图面视为传送到的一个好图面,并将启用传送光标上的动画。If that ray collides with a surface that is more or less pointing up, the surface will be considered a good surface to teleport to, and the animation on the teleport cursor will be enabled. 如果该射线不会与某个面上更多或更少的向上箭头发生冲突,则将禁用游标上的动画。If the ray does not collide with a surface more or less pointing up, then the animation on the cursor will be disabled. 当 "Y" 按钮被释放并且射线的计算点为有效位置时,该脚本将使用与该射线相交的位置来调用 SetWorldPosition。When the Y button is released and the calculated point of the ray is a valid position, the script calls SetWorldPosition with the position the ray intersected.

欣赏你的进度Enjoy your progress

这次您需要查找朋友。This time you'll need to find a friend.

同样,具有 HoloLens 的用户将托管一个会话。Once again, a user with the HoloLens will host a session. 其他用户将加入会话。Other users will join the session. 应用程序将把前三名用户加入到岛上三个路径之一上的沉浸式耳机上。The application will place the first three users to join from an immersive headset on one of the three paths on the island. 请随时浏览本部分中的岛。Feel free to explore the island in this section.

要注意的详细信息:Details to notice:

  1. 你可以在云中看到人脸,这有助于沉浸用户查看 HoloLens 用户正在寻找的方向。You can see faces in the clouds, which helps an immersed user see which direction a HoloLens user is looking.
  2. 岛上的头像具有旋转 necks。The avatars on the island have necks that rotate. 它们不会遵循用户正在执行的操作, (我们没有) 的信息,而是为了获得良好的体验。They won't follow what the user is doing is real reality (we don't have that information) but it makes for a nice experience.
  3. 如果 HoloLens 用户正在查看岛,沉浸用户会看到其光标。If the HoloLens user is looking at the Island, the immersed users can see their cursor.
  4. 表示 HoloLens 用户投影的云。The clouds that represent the HoloLens users cast shadows.

第5章-FinaleChapter 5 - Finale

目标Objectives

在这两种设备类型之间创建协作式交互式体验。Create a collaborative interactive experience between the two device types.

要生成的内容What we will build

在第4章,当具有沉浸式耳机的用户在岛上出现谜题附近时,HoloLens 用户将收到一条工具提示,其中包含对测验题的线索。Building on chapter 4, when a user with an immersive headset gets near a puzzle on the island, the HoloLens users will get a tool tip with a clue to the puzzle. 所有沉浸式耳机用户都在火箭房间内进入了测验题后,就会启动火箭。Once all of the immersive headset users get past their puzzles and onto the "ready pad" in the rocket room, the rocket will launch.

步骤Steps

  • 层次结构 中,选择 UslandIn Hierarchy, select Usland.
  • 检查器 的 " 级别控制" 中,选中 " 启用协作"。In Inspector, in Level Control, check Enable Collaboration.

深入到代码中Digging into the code

现在让我们看看 LevelControl.cs。Now let us look at LevelControl.cs. 此脚本是游戏逻辑的核心,并且保持游戏状态。This script is the core of the game logic and maintains the game state. 由于这是一个使用 UNET 的多玩家游戏,我们需要了解数据的流动方式,至少足以修改本教程。Since this is a multiplayer game using UNET we need to understand how data flows, at least well enough to modify this tutorial. 有关 UNET 的更完整概述,请参阅 Unity 的文档。For a more complete overview of UNET, please refer to Unity's documentation.

在 "项目" 面板中,导航到 " Assets\AppPrefabs\Support\Scripts\GameLogic ",然后双击 " LevelControl.cs"。In the project panel, navigate to Assets\AppPrefabs\Support\Scripts\GameLogic and double-click on LevelControl.cs.

让我们了解沉浸式耳机如何指示它们已准备好进行火箭启动。Let us understand how an immersive headset indicates that they are ready for the rocket launch. 通过在与岛上的三个路径相对应的布尔列表中设置三个布尔之一,可以传达火箭的启动准备情况。Rocket Launch readiness is communicated by setting one of three bools in a list of bools that correspond to the three paths on the island. 当分配到路径的用户在火箭房间内的棕色垫的顶部时,将设置路径的 bool。A path's bool will be set when the user assigned to the path is on top of the brown pad inside the rocket room. 好了,现在来了解详细信息。Okay, now to the details.

我们将从更新 ( # A1 函数开始。We will start in the Update() function. 你会注意到有一个 "使用" 函数。You will note that there is a 'cheat' function. 我们在开发过程中使用这种测试火箭启动和重置顺序。We used this in development to test the rocket launch and reset sequence. 它在多用户体验中不起作用。It won't work in the multi user experience. 但愿您内在化以下信息可以使其正常工作。Hopefully by the time you internalize the following infromation you can make it work. 检查是否应该是,我们会检查是否已沉浸本地播放器。After we check to see if we should cheat, we check to see if the local player is immersed. 我们想要重点介绍我们的目标。We want to focus on how we find that we're at the goal. 在 if (沉浸) 检查的情况下,调用 CheckGoal 隐藏在 EnableCollaboration bool 后面。Inside of the if (Immersed) check, there is a call to CheckGoal hiding behind the EnableCollaboration bool. 这对应于完成本章中的步骤时选中的复选框。This corresponds to the checkbox you checked while completing the steps for this chapter. 在 EnableCollaboration 内部,我们看到对 CheckGoal ( # A1 的调用。Inside of EnableCollaboration we see a call to CheckGoal().

CheckGoal 进行一些数学计算,看看是否有更多或更少的时间。CheckGoal does some math to see if we are more or less standing on the pad. 当我们进行调试时,我们将进行调试。记录 "到达目标",然后调用 "SendAtGoalMessage ( # A1"。When we are, we Debug.Log "Arrived at goal" and then we call 'SendAtGoalMessage()'. 在 SendAtGoalMessage 中,我们调用 playerController. SendAtGoal。In SendAtGoalMessage we call playerController.SendAtGoal. 为节省时间,请参阅以下代码:To save you some time, here's the code:

private void CmdSendAtGoal(int GoalIndex)
{
    levelState.SetGoalIndex(GoalIndex);
}
public void SendAtGoal(int GoalIndex)
{
    if (isLocalPlayer)
    {
        Debug.Log("sending at goal " + GoalIndex);
        CmdSendAtGoal(GoalIndex);
    }
}

请注意,SendAtGoalMessage 调用 CmdSendAtGoal,后者会调用中的 levelState。Note that SendAtGoalMessage calls CmdSendAtGoal, which calls levelState.SetGoalIndex, which is back in LevelControl.cs. 乍一看,这似乎很奇怪。At first glance this seems strange. 为什么不只调用 SetGoalIndex 而不是通过播放机控制器进行这种奇怪的路由?Why not just call SetGoalIndex rather than this strange routing through the player controller? 原因是我们符合数据模型,UNET 使用该数据来使数据保持同步。若要防止作弊和抖动,UNET 要求每个对象都有一个有权更改同步变量的用户。The reason is that we are conforming to the data model UNET uses to keep data in sync. To prevent cheating and thrashing, UNET requires that each object has a user who has authority to change the synchronized variables. 此外,只有启动会话) 用户 (的主机可以直接更改数据。Further, only the host (the user that started the session) can change data directly. 如果用户不是宿主但具有权限,则需要将 "command" 发送到将更改变量的主机。Users who are not the host, but have authority, need to send a 'command' to the host which will change the variable. 默认情况下,主机拥有所有对象的权限,但生成的对象除外表示用户。By default the host has authority over all objects, except for the object spawned to represent the user. 在本例中,此对象具有 playercontroller 脚本。In our case this object has the playercontroller script. 有一种方法可以为对象请求授权,然后进行更改,但我们选择利用这一事实:播放机控制器具有己方授权,并通过播放机控制器路由命令。There is a way to request authority for an object and then make changes, but we choose to leverage the fact that the player controller has self authority and route commands through the player controller.

另一种方法是,当我们以我们的目标找到自己时,玩家需要告诉主机,主机会告诉所有其他人。Said another way, when we've found ourselves at our goal, the player needs to tell the host, and the host will tell everyone else.

返回 LevelControl.cs 查看 SetGoalIndex。Back in LevelControl.cs look at SetGoalIndex. 这里,我们要在 synclist (AtGoal) 中设置值的值。Here we are setting the value of a value in a synclist (AtGoal). 请记住,在执行此操作时,我们处于主机的上下文中。Remember that we are in the context of the host while we do this. 与命令相似,RPC 是指主机可以发出的,这会导致所有客户端运行一些代码。Similar to a command, an RPC is something the host can issue that will cause all clients to run some code. 这里我们称之为 "RpcCheckAllGoals"。Here we call 'RpcCheckAllGoals'. 每个客户端将分别检查是否设置了所有三个 AtGoals,如果已设置,则启动火箭。Each client will individually check to see if all three AtGoals are set, and if so, launch the rocket.

欣赏你的进度Enjoy your progress

在上一章中生成,我们将像以前一样启动会话。Building on the previous chapter, we will start the session as before. 这一次,沉浸式头戴显示设备上的用户到达其路径上的 "门" 时,会出现一个工具提示,只有 HoloLens 用户可以看到。This time as the users in the immersive headset get to the "door" on their path, a tooltip will appear that only the HoloLens users can see. HoloLens 用户负责将此线索传达给沉浸式耳机中的用户。The HoloLens users are responsible for communicating this clue to the users in the immersive headset. 一旦每个虚拟形象在火山内的相应的棕色垫上火箭,就会启动空间。The rocket will launch to space once each avatar has stepped on its corresponding brown pad inside the volcano. 场景将在60秒后重置,因此你可以重新执行此操作。The scene will reset after 60 seconds so you can do it again.

另请参阅See also