MR Sharing 250:HoloLens 和沉浸式頭戴裝置MR Sharing 250: HoloLens and immersive headsets

注意

混合實境學院教學課程的設計是以 HoloLens (第 1 代) 和混合實境沉浸式頭戴裝置為準。The Mixed Reality Academy tutorials were designed with HoloLens (1st gen) and Mixed Reality Immersive Headsets in mind. 因此,對於仍在尋找這些裝置開發指引的開發人員而言,我們覺得這些教學課程很重要。As such, we feel it is important to leave these tutorials in place for developers who are still looking for guidance in developing for those devices. 這些教學課程 不會 使用用於 HoloLens 2 的最新工具組或互動進行更新。These tutorials will not be updated with the latest toolsets or interactions being used for HoloLens 2. 系統會保留這些資訊,以繼續在支援的裝置上運作。They will be maintained to continue working on the supported devices. 已針對 HoloLens 2 公佈一系列新的教學課程A new series of tutorials has been posted for HoloLens 2.

利用通用 Windows 平臺 (UWP) 的彈性,您可以輕鬆地建立跨多個裝置的應用程式。With the flexibility of Universal Windows Platform (UWP), it is easy to create an application that spans multiple devices. 有了這種彈性,我們可以建立利用每部裝置優勢的體驗。With this flexibility, we can create experiences that leverage the strengths of each device. 本教學課程將涵蓋在 HoloLens 和 Windows Mixed Reality 沉浸式耳機上執行的基本共用體驗。This tutorial will cover a basic shared experience that runs on both HoloLens and Windows Mixed Reality immersive headsets. 這項內容最初是在華盛頓州西雅圖的 Microsoft Build 2017 會議中提供。This content was originally delivered at the Microsoft Build 2017 conference in Seattle, WA.

在此教學課程中,我們將:In this tutorial, we will:

  • 使用 UNET 設定網路。Setup a network using UNET.
  • 跨混合現實裝置共用全像影像。Share holograms across mixed reality devices.
  • 根據所使用的混合現實裝置,建立不同的應用程式視圖。Establish a different view of the application depending on which mixed reality device is being used.
  • 建立共用體驗,讓 HoloLens 使用者透過一些簡單的謎題引導沉浸式耳機使用者。Create a shared experience where HoloLens users guide immersive headsets users through some simple puzzles.

裝置支援Device support

課程Course HoloLensHoloLens 沉浸式頭戴裝置Immersive headsets
MR Sharing 250:HoloLens 和沉浸式頭戴裝置MR Sharing 250: HoloLens and immersive headsets ✔️✔️ ✔️✔️

在您開始使用 Intune 之前Before you start

必要條件Prerequisites

專案檔Project files

注意

如果您想要在下載之前查看原始程式碼, 可在 GitHub 上取得。If you want to look through the source code before downloading, it's available on GitHub.

第1章-Hololens WorldChapter 1 - Holo World

目標Objectives

請確定開發環境已準備好使用簡單的專案。Make sure the development environment is ready to go with a simple project.

我們將建立的內容What we will build

一種應用程式,會在 HoloLens 或 Windows Mixed Reality 沉浸式耳機上顯示全像投影。An application that shows a hologram on either HoloLens or a Windows Mixed Reality immersive headset.

步驟Steps

  • 開啟 Unity。Open Unity.
    • 選取 [開啟] 。Select Open.
    • 流覽至您解壓縮專案檔的位置。Navigate to where you extracted the project files.
    • 按一下 [選擇資料夾]。Click Select Folder.
    • Unity 需要一些時間才會第一次處理專案。It will take a little while for Unity to process the project the first time.
  • 檢查 Unity 中是否已啟用 Mixed Reality。Check that Mixed Reality is enabled in Unity.
    • 開啟 [組建設定] 對話方塊, (Control + Shift + BFile > 組建設定 ... ]) 。Open the build settings dialog (Control+Shift+B or File > Build Settings...).
    • 選取 通用 Windows 平臺 然後按一下 [ 切換平臺]。Select Universal Windows Platform then click Switch Platform.
    • 選取 [ 編輯>播放機設定]。Select Edit>Player Settings.
    • 在右側的 [偵測 ] 面板中,展開 [ XR 設定]。In the Inspector panel on the right hand side, expand XR Settings.
    • 勾選 [ 支援虛擬實境 ] 方塊。Check the Virtual Reality Supported box.
    • Windows Mixed Reality 應為虛擬實境 SDK。Windows Mixed Reality should be the Virtual Reality SDK.
  • 建立場景。Create a scene.
    • 在階層中, 以滑鼠右鍵 按一下 [ 主要攝影機 ] 選取 [ 刪除]。In the Hierarchy right click Main Camera select Delete.
    • HoloToolkit > 輸入 > PrefabsMixedRealityCameraParent 拖曳至 階層。From HoloToolkit > Input > Prefabs drag MixedRealityCameraParent to the Hierarchy.
  • 將全像影像新增至場景Add Holograms to the scene
    • AppPrefabsSkybox 拖曳至 場景視圖From AppPrefabs drag Skybox to the Scene View.
    • AppPrefabs管理員 拖曳至 階層。From AppPrefabs drag Managers to the Hierarchy.
    • AppPrefabs 拖曳至 階層。From AppPrefabs drag Island to the Hierarchy.
  • 儲存並建立Save And build
    • 儲存 (Control + SFile > 儲存場景) Save (either Control+S or File > Save Scene)
    • 因為這是新的場景,所以您需要為它命名。Since this is a new scene, you'll need to name it. 名稱並不重要,但我們使用 SharedMixedReality。Name doesn't matter, but we use SharedMixedReality.
  • 匯出至 Visual StudioExport To Visual Studio
    • 開啟 [組建] 功能表 (Control + Shift + BFile > 組建設定) Open the build menu (Control+Shift+B or File > Build Settings)
    • 按一下 [ 新增開啟的場景]。Click Add Open Scenes.
    • 檢查 Unity c # 專案Check Unity C# Projects
    • 按一下 [建置]。Click Build.
    • 在出現的 [檔案瀏覽器] 視窗中,建立名為 App 的新資料夾。In the file explorer window that appears, create a New Folder named App.
    • 按一下 應用程式 資料夾。Single click the App folder.
    • 按下 [ 選取資料夾]。Press Select Folder.
    • 等候組建完成Wait for the build to complete
    • 在出現的 [檔案瀏覽器] 視窗中,流覽至 應用程式 資料夾。In the file explorer window that appears, navigate into the App folder.
    • 按兩下 SharedMixedReality .sln 以啟動 Visual StudioDouble-click SharedMixedReality.sln to launch Visual Studio
  • 從 Visual Studio 建立Build From Visual Studio
    • 使用頂端工具列變更目標為 Releasex86Using the top toolbar change target to Release and x86.
    • 按一下 [本機電腦] 旁邊的箭號,然後選取要部署到 HoloLens 的 裝置Click the arrow next to Local Machine and select Device to deploy to HoloLens
    • 按一下 [ 裝置 ] 旁的箭號,然後選取 [ 本機電腦 ] 以部署混合現實耳機。Click the arrow next to Device and select Local Machine to deploy for the mixed reality headset.
    • 按一下 [ Debug->啟動但不調試 ] 或 [ Control + F5 ] 以啟動應用程式。Click Debug->Start Without Debugging or Control+F5 to start the application.

深入探討至程式碼Digging into the code

在 [專案] 面板中,流覽至 [ Assets\HoloToolkit\Input\Scripts\Utilities ],然後按兩下 [ MixedRealityCameraManager.cs ] 將它開啟。In the project panel, navigate to Assets\HoloToolkit\Input\Scripts\Utilities and double click MixedRealityCameraManager.cs to open it.

總覽: MixedRealityCameraManager.cs 是一個簡單的腳本,可根據裝置來調整品質層級和背景設定。Overview: MixedRealityCameraManager.cs is a simple script that adjusts quality level and background settings based on the device. 這裡的金鑰是 HolographicSettings IsDisplayOpaque,可讓腳本偵測裝置是否為 HoloLens (IsDisplayOpaque 會傳回 false) 或沉浸式耳機 (IsDisplayOpaque 會傳回 true) 。Key here is HolographicSettings.IsDisplayOpaque, which allows a script to detect if the device is a HoloLens (IsDisplayOpaque returns false) or an immersive headset (IsDisplayOpaque returns true).

享受您的進度Enjoy your progress

此時,應用程式只會轉譯一個全像影像。At this point the application will just render a hologram. 我們稍後會將互動新增至影像。We will add interaction to the hologram later. 這兩個裝置都會轉譯相同的全像投影。Both devices will render the hologram the same. 沉浸式耳機也會呈現藍色天空和雲端背景。The immersive headset will also render a blue sky and clouds background.

第2章-互動Chapter 2 - Interaction

目標Objectives

示範如何處理 Windows Mixed Reality 應用程式的輸入。Show how to handle input for a Windows Mixed Reality application.

我們將建立的內容What we will build

以第1章的應用程式為基礎,我們將新增功能,讓使用者能夠挑選全像,然後將它放在 HoloLens 的真實世界表面,或放在沉浸式耳機的虛擬資料表中。Building on the application from chapter 1, we will add functionality to allow the user to pick up the hologram and place it on a real world surface in HoloLens or on a virtual table in an immersive headset.

輸入複習: 在 HoloLens 上,選取手勢是 點擊Input Refresher: On HoloLens the select gesture is the air tap. 在沉浸式耳機上,我們將使用 Xbox 控制器上 的按鈕。On immersive headsets, we will use the A button on the Xbox controller. 如需詳細資訊,請參閱 互動模型的總覽For more information check out the interaction model overview.

步驟Steps

  • 新增輸入管理員Add Input manager
    • HoloToolkit > 輸入 > PrefabsInputManager 拖曳 到階層 作為 經理 的子系。From HoloToolkit > Input > Prefabs drag InputManager to Hierarchy as a child of Managers.
    • HoloToolkit > 輸入 > Prefabs > 資料指標游標 拖曳至 階層。From HoloToolkit > Input > Prefabs > Cursor drag Cursor to Hierarchy.
  • 新增空間對應Add Spatial Mapping
    • HoloToolkit > SpatialMapping > PrefabsSpatialMapping 拖曳至 階層。From HoloToolkit > SpatialMapping > Prefabs drag SpatialMapping to Hierarchy.
  • 新增虛擬 PlayspaceAdd Virtual Playspace
    • 階層 中展開 MixedRealityCameraParent 選取 界限In Hierarchy expand MixedRealityCameraParent select Boundary
    • 在 [偵測 ] 面板中,核取 [啟用 界限] 核取方塊In Inspector panel check the box to enable Boundary
    • AppPrefabsVRRoom 拖曳至 階層。From AppPrefabs drag VRRoom to Hierarchy.
  • 新增 WorldAnchorManagerAdd WorldAnchorManager
    • [階層] 中選取[ 管理員]。In Hierarchy, Select Managers.
    • 在 [偵測 ] 中,按一下 [ 新增元件]。In Inspector, click Add Component.
    • 輸入 World 錨點管理員Type World Anchor Manager.
    • 選取 [ 世界錨點管理員 ] 將它加入。Select World Anchor Manager to add it.
  • 將 TapToPlace 新增至島Add TapToPlace to the Island
    • [階層] 中, 展開 [ ]。In Hierarchy, expand Island.
    • 選取 [ MixedRealityLand]。Select MixedRealityLand.
    • 在 [偵測 ] 中,按一下 [ 新增元件]。In Inspector, click Add Component.
    • 輸入點一下 以放置 並選取它。Type Tap To Place and select it.
    • 選取 [在點處放置父系]。Check Place Parent On Tap.
    • 放置位移 設定為 (0、0.1、0)Set Placement Offset to (0, 0.1, 0).
  • 儲存並建立成先前的Save and Build as before

深入探討至程式碼Digging into the code

腳本 1-GamepadInput.csScript 1 - GamepadInput.cs

在 [專案] 面板中,流覽至 [ Assets\HoloToolkit\Input\Scripts\InputSources ],然後按兩下 [ GamepadInput.cs ] 將它開啟。In the project panel navigate to Assets\HoloToolkit\Input\Scripts\InputSources and double click GamepadInput.cs to open it. 在 [專案] 面板中的相同路徑中,也按兩下 [ InteractionSourceInputSource.cs]。From the same path in the project panel, also double click InteractionSourceInputSource.cs.

請注意,這兩個腳本都有一個通用的基類 BaseInputSource。Note that both scripts have a common base class, BaseInputSource.

BaseInputSource 會保留對 InputManager 的參考,讓腳本能夠觸發事件。BaseInputSource keeps a reference to an InputManager, which allows a script to trigger events. 在此情況下,InputClicked 事件是相關的。In this case, the InputClicked event is relevant. 當我們進入腳本2時,請務必記住這一點 TapToPlace。This will be important to remember when we get to script 2, TapToPlace. 在 GamePadInput 的案例中,我們會輪詢控制器上要按下的按鈕,然後引發 InputClicked 事件。In the case of GamePadInput, we poll for the A button on the controller to be pressed, then we raise the InputClicked event. 在 InteractionSourceInputSource 的案例中,我們會引發 InputClicked 事件以回應 TappedEvent。In the case of InteractionSourceInputSource, we raise the InputClicked event in response to the TappedEvent.

腳本 2-TapToPlace.csScript 2 - TapToPlace.cs

在 [專案] 面板中,流覽至 [ Assets\HoloToolkit\SpatialMapping\Scripts ],然後按兩下 [ TapToPlace.cs ] 將它開啟。In the project panel navigate to Assets\HoloToolkit\SpatialMapping\Scripts and double click TapToPlace.cs to open it.

許多開發人員想要在建立全像的應用程式時執行的第一件事,就是移動帶有手勢輸入的全息影像The first thing many developers want to implement when creating a Holographic application is moving Holograms with gesture input. 因此,我們已 endeavored 徹底批註此腳本。As such, we've endeavored to thoroughly comment this script. 在本教學課程中,有一些值得注意的事項。A few things are worth highlighting for this tutorial.

首先,請注意,TapToPlace 會執行 IInputClickHandler。First, note that TapToPlace implements IInputClickHandler. IInputClickHandler 會公開函式,這些函式會處理 GamePadInput.cs 或 InteractionSourceInputSource.cs 所引發的 InputClicked 事件。IInputClickHandler exposes the functions that handle the InputClicked event raised by GamePadInput.cs or InteractionSourceInputSource.cs. 當 BaseInputSource 偵測到 click,而具有 TapToPlace 的物件處於焦點時,就會呼叫 OnInputClicked。OnInputClicked is called when a BaseInputSource detects a click while the object with TapToPlace is in focus. Airtapping 在 HoloLens 上或按下 Xbox 控制器上的按鈕將會觸發事件。Either airtapping on HoloLens or pressing the A button on the Xbox controller will trigger the event.

其次是在 update 中執行程式碼,以查看是否正在查看介面,因此我們可以將遊戲物件放在介面上,例如資料表。Second is the code be executed in update to see if a surface is being looked at so we can place the game object on a surface, like a table. 沉浸式耳機沒有實際表面的概念,因此代表資料表 top (隆隆 > TableThingy > Cube) 的物件已標記 SpatialMapping 實體層,因此更新中的光線轉換將會與虛擬資料表上的資料衝突。The immersive headset doesn't have a concept of real surfaces, so the object that represents the table top (Vroom > TableThingy > Cube) has been marked with the SpatialMapping physics layer, so the ray cast in Update will collide with the virtual table top.

享受您的進度Enjoy your progress

這次您可以選取島來移動它。This time you can select the island to move it. 在 HoloLens 上,您可以將島移至真實表面。On HoloLens you can move the island to a real surface. 在沉浸式耳機中,您可以將島移至我們所加入的虛擬資料表。In the immersive headset you can move the island to the virtual table we added.

第3章-共用Chapter 3 - Sharing

目標Objectives

確定已正確設定網路,並詳細說明如何在裝置之間共用空間錨點。Ensure that the network is correctly configured and detail how spatial anchors are shared between devices.

我們將建立的內容What we will build

我們會將專案轉換成多人的專案。We will convert our project to a multiplayer project. 我們會將 UI 和邏輯新增至主機或加入會話。We will add UI and logic to host or join sessions. HoloLens 使用者會在與雲端一起的會話中看到彼此,而且沉浸式耳機使用者在錨點附近有雲端。HoloLens users will see each other in the session with clouds over their heads, and immersive headset users have clouds near to where the anchor is. 沉浸式耳機中的使用者會看到 HoloLens 使用者相對於場景的原點。Users in the immersive headsets will see the HoloLens users relative to the origin of the scene. HoloLens 使用者將會在同一處看到島的全息圖。HoloLens users will all see the hologram of the island in the same place. 要注意的是,沉浸式耳機中的使用者不會在這一章的島上,而是與 HoloLens 一樣的行為,而且會有鳥的鳥觀點。It is key to note that the users in the immersive headsets will not be on the island during this chapter, but will behave very similarly to HoloLens, with a birds eye view of the island.

步驟Steps

  • 移除島和 VRRoomRemove Island and VRRoom
    • 階層中,以 滑鼠右鍵按一下 [] 選取 [刪除]In Hierarchy right-click Island select Delete
    • 階層中,以 滑鼠右鍵按一下 VRRoom 選取 [刪除]In Hierarchy right-click VRRoom select Delete
  • 新增 UslandAdd Usland
    • AppPrefabsUsland 拖曳至 階層。From AppPrefabs drag Usland to Hierarchy.
  • AppPrefabs 將下列各項拖曳至 階層:From AppPrefabs drag each of the following to Hierarchy:
    • UNETSharingStageUNETSharingStage
    • UNetAnchorRootUNetAnchorRoot
    • UIContainerUIContainer
    • DebugPanelButtonDebugPanelButton
  • 儲存並建立成先前的Save and Build as before

深入探討至程式碼Digging into the code

在 [專案] 面板中,流覽至 [ Assets\AppPrefabs\Support\SharingWithUnet\Scripts ],然後按兩下 [ UnetAnchorManager.cs]。In the project panel, navigate to Assets\AppPrefabs\Support\SharingWithUnet\Scripts and double-click on UnetAnchorManager.cs. 一個 HoloLens 與另一個 HoloLens 共用追蹤資訊的能力,使得這兩個裝置都可以共用相同的空間,幾乎神奇。The ability for one HoloLens to share tracking information with another HoloLens such that both devices can share the same space is near magical. 當有兩個以上的人員可以使用相同的數位資料共同作業時,混合現實的威力就會保持運作。The power of mixed reality comes alive when two or more people can collaborate using the same digital data.

這段腳本中有幾點要指出:A few things to point out in this script:

在 start 函數中,請注意 IsDisplayOpaque 的檢查。In the start function, notice the check for IsDisplayOpaque. 在此情況下,我們會假設已建立錨點。In this case, we pretend that the Anchor is established. 這是因為沉浸式耳機未公開匯入或匯出錨點的方式。This is because the immersive headsets do not expose a way to import or export anchors. 但是,如果我們是在 HoloLens 上執行,此腳本會在裝置之間執行共用錨點。If we are running on a HoloLens, however, this script implements sharing anchors between the devices. 啟動會話的裝置將會建立用於匯出的錨點。The device that starts the session will create an anchor for exporting. 加入會話的裝置將會從啟動會話的裝置要求錨點。The device that joins a session will request the anchor from the device that started the session.

出口:Exporting:

當使用者建立會話時,NetworkDiscoveryWithAnchors 會呼叫 UNETAnchorManagers CreateAnchor 函數。When a user creates a session, NetworkDiscoveryWithAnchors will call UNETAnchorManagers CreateAnchor function. 讓我們遵循 CreateAnchor flow。Let's follow CreateAnchor flow.

我們一開始會先進行一些維護,並清除我們針對先前錨點收集到的任何資料。We start by doing some housekeeping, clearing out any data we may have collected for previous anchors. 然後檢查是否有要載入的快取錨點。Then we check if there is a cached anchor to load. 錨定資料的長度通常是5到 20 MB,因此重複使用快取的錨點可以節省透過網路傳輸所需的資料量。The anchor data tends to be between 5 and 20 MB, so reusing cached anchors can save on the amount of data we need to transfer over the network. 稍後我們會看到其運作方式。We'll see how this works a bit later. 即使我們要重複使用錨點,但如果新的用戶端聯結沒有錨點,就需要讓錨點資料備妥。Even if we are reusing the anchor, we need to get the anchor data ready in case a new client joins that doesn't have the anchor.

說到準備錨點資料,WorldAnchorTransferBatch 類別會公開準備錨點資料以傳送至另一個裝置或應用程式的功能,以及匯入錨點資料的功能。Speaking of getting the anchor data ready, the WorldAnchorTransferBatch class exposes the functionality to prepare anchor data for sending to another device or application and the functionality to import the anchor data. 由於我們是在匯出路徑上,因此我們會將錨點新增至 WorldAnchorTransferBatch,並呼叫 ExportAsync 函式。Since we're on the export path, we will add our anchor to the WorldAnchorTransferBatch and call the ExportAsync function. 然後,ExportAsync 會在產生要匯出的資料時呼叫 WriteBuffer 回呼。ExportAsync will then call the WriteBuffer callback as it generates data for export. 當匯出所有資料時,會呼叫 ExportComplete。When all of the data has been exported ExportComplete will be called. 在 WriteBuffer 中,我們會將資料區塊新增至我們保留的匯出清單。In WriteBuffer we add the chunk of data to a list we keep for exporting. 在 ExportComplete 中,我們會將清單轉換成陣列。In ExportComplete we convert the list to an array. 此外也會設定 AnchorName 變數,以觸發其他裝置要求錨點(如果沒有的話)。The AnchorName variable is also set, which will trigger other devices to request the anchor if they don't have it.

在某些情況下,錨點將不會匯出或建立少量的資料,我們會再試一次。In some cases the anchor won't export or will create so little data that we will try again. 我們在這裡直接呼叫 CreateAnchor。Here we just call CreateAnchor again.

匯出路徑中的最後一個函數是 AnchorFoundRemotely。A final function in the export path is AnchorFoundRemotely. 當另一部裝置找到錨點時,該裝置將會告訴主機,而主機會使用它作為信號,讓錨點成為「良好錨點」並可進行快取。When another device finds the anchor, that device will tell the host, and the host will use that as a signal that the anchor is a "good anchor" and can be cached.

進口:Importing:

當 HoloLens 加入會話時,它需要匯入錨點。When a HoloLens joins a session, it needs to import an anchor. 在 UNETAnchorManager 的 Update 函數中,會輪詢 AnchorName。In UNETAnchorManager's Update function, the AnchorName is polled. 當錨點名稱變更時,匯入程式就會開始。When the anchor name changes, the import process begins. 首先,我們會嘗試從本機錨點存放區載入具有指定名稱的錨點。First, we try to load the anchor with the specified name from the local anchor store. 如果已經有,我們可以使用它,而不需要再次下載資料。If we already have it, we can use it without downloading the data again. 如果沒有,則會呼叫 WaitForAnchor,以起始下載。If we don't have it, then we call WaitForAnchor which will initiate the download.

當下載完成時,會呼叫 NetworkTransmitter_dataReadyEvent。When the download is completed, NetworkTransmitter_dataReadyEvent is called. 這會通知更新迴圈使用下載的資料來呼叫 ImportAsync。This will signal the Update loop to call ImportAsync with the downloaded data. 當匯入程式完成時,ImportAsync 會呼叫 ImportComplete。ImportAsync will call ImportComplete when the import process is complete. 如果匯入成功,錨點將會儲存在本機播放程式存放區中。If the import is successful, the anchor will be saved in the local player store. PlayerController.cs 實際上會呼叫 AnchorFoundRemotely,讓主機知道已建立良好的錨點。PlayerController.cs actually makes the call to AnchorFoundRemotely to let the host know that a good anchor has been established.

享受您的進度Enjoy your progress

這次具有 HoloLens 的使用者將使用 UI 中的 [ 啟動會話 ] 按鈕來裝載會話。This time a user with a HoloLens will host a session using the start session button in the UI. 其他使用者(在 HoloLens 或沉浸式耳機上)會選取會話,然後在 UI 中選取 [ 加入會話 ] 按鈕。Other users, both on HoloLens or an immersive headset, will select the session and then select the join session button in the UI. 如果您有多人使用 HoloLens 裝置,他們的標頭將會有 red cloud。If you have multiple people with HoloLens devices, they will have red clouds over their heads. 每個沉浸式耳機也會有一個藍色的雲端,但藍色的雲端不會在耳機上方,因為耳機不會嘗試尋找與 HoloLens 裝置相同的全局座標空間。There will also be a blue cloud for each immersive headset, but the blue clouds will not be above the headsets, as the headsets are not trying to find the same world coordinate space as the HoloLens devices.

專案中的這個點是包含的共用應用程式;它並不會有太大的作用,而且可以作為基準。This point in the project is a contained sharing application; it doesn't do very much, and could act as a baseline. 在下一章中,我們將開始打造體驗,讓人們享受體驗。In the next chapters, we will start building an experience for people to enjoy. 若要取得有關共用體驗設計的進一步指引,請移至這裡。To get further guidance on shared experience design, go here.

第4章-深度和 teleportingChapter 4 - Immersion and teleporting

目標Objectives

滿足每種混合現實裝置的體驗。Cater the experience to each type of mixed reality device.

我們將建立的內容What we will build

我們將會更新應用程式,將沉浸式耳機使用者放在島上(具有沉浸式視圖)。We will update the application to put immersive headset users on the island with an immersive view. HoloLens 使用者仍會有此島的鳥眼睛。HoloLens users will still have the bird's eye view of the island. 每種裝置類型的使用者都可以看到出現在世界中的其他使用者。Users of each device type can see other users as they appear in the world. 比方說,沉浸式耳機使用者可以看到其他虛擬人偶在島上的其他路徑,並將 HoloLens 使用者視為大型雲端。For instance, immersive headset users can see the other avatars on other paths on the island, and they see the HoloLens users as giant clouds above the island. 如果 HoloLens 使用者正在查看島,則沉浸式耳機使用者也會看到 HoloLens 使用者的注視光線游標。Immersive headset users will also see the cursor of the HoloLens user's gaze ray if the HoloLens user is looking at the island. HoloLens 使用者會看到島上的圖片,代表每個沉浸式耳機使用者。HoloLens users will see an avatar on the island to represent each immersive headset user.

已更新沉浸式裝置的輸入:Updated Input for the Immersive device:

  • Xbox 控制器上的左緩衝和右緩衝按鈕會旋轉玩家The left bumper and right bumper buttons on the Xbox controller rotate the player
  • 按住 Xbox 控制器上的 Y 按鈕將會啟用 傳送 游標。Holding the Y button on the Xbox controller will enable a teleport cursor. 當您放開 Y 按鈕時,如果游標具有旋轉箭號指標,則會傳送至游標的位置。If the cursor has a spinning arrow indicator when you release the Y button, you will be teleported to the cursor's location.

步驟Steps

  • 將 MixedRealityTeleport 新增至 MixedRealityCameraParentAdd MixedRealityTeleport to MixedRealityCameraParent
    • [階層] 中選取[ Usland]。In Hierarchy, select Usland.
    • 在 [偵測 ] 中,啟用 層級控制In Inspector, enable Level Control.
    • [階層] 中選取[ MixedRealityCameraParent]。In Hierarchy, select MixedRealityCameraParent.
    • 在 [偵測 ] 中,按一下 [ 新增元件]。In Inspector, click Add Component.
    • 輸入 Mixed Reality 傳送 ,然後選取它。Type Mixed Reality Teleport and select it.

深入探討至程式碼Digging into the code

沉浸式耳機使用者將使用纜線行動網卡到他們的電腦,但我們的島大於纜線的長度。Immersive headset users will be tethered to their PCs with a cable, but our island is larger than the cable is long. 為了彌補這一點,我們需要能夠在使用者的動作之外獨立移動相機。To compensate, we need the ability to move the camera independently of the user's motion. 請參閱 [緩和] 頁面 ,以取得設計混合現實應用程式 (特定的自我運動和 locomotion) 的詳細資訊。Please see the comfort page for more information about designing your mixed reality application (in particular self motion and locomotion).

為了說明此程式,定義兩個詞彙將會很有説明。In order to describe this process it will be useful to define two terms. 首先, dolly 會是與使用者分開移動相機的物件。First, dolly will be the object that moves the camera independently from the user. Dolly 的子遊戲物件將會是 主相機A child game object of the dolly will be the main camera. 主要相機會附加至使用者的標頭。The main camera is attached to the user's head.

在 [專案] 面板中,流覽至 [ Assets\AppPrefabs\Support\Scripts\GameLogic ],然後按兩下 [ MixedRealityTeleport.cs]。In the project panel, navigate to Assets\AppPrefabs\Support\Scripts\GameLogic and double-click on MixedRealityTeleport.cs.

MixedRealityTeleport 有兩個作業。MixedRealityTeleport has two jobs. 首先,它會使用緩衝器來處理旋轉。First, it handles rotation using the bumpers. 在 update 函數中,我們會在 LeftBumper 和 RightBumper 上輪詢 ' ButtonUp '。In the update function we poll for 'ButtonUp' on LeftBumper and RightBumper. GetButtonUp 只會在第一個畫面格上傳回 true。GetButtonUp only returns true on the first frame a button is up after having been down. 如果已引發任一個按鈕,則我們知道使用者需要輪替。If either button had been raised, then we know the user needs to rotate.

當我們輪替時,請使用稱為「淡化控制項」的簡單腳本,淡出並淡出。When we rotate we do a fade out and fade in using a simple script called 'fade control'. 這麼做是為了防止使用者看見可能導致不適感的非自然移動。We do this to prevent the user from seeing an unnatural movement which could lead to discomfort. 淡入和淡出效果相當簡單。The fade in and out effect is fairly simple. 主要攝影機 前面有黑色的間隙。We have a black quad hanging in front of the main camera. 當您淡出時,會將 Alpha 值從0轉換成1。When fading out we transition the alpha value from 0 to 1. 這會逐漸產生四的黑色圖元來呈現和遮蔽背後的任何專案。This gradually causes the black pixels of the quad to render and obscure anything behind them. 當您淡出時,我們會將 Alpha 值轉換回零。When fading back in we transition the alpha value back to zero.

當我們計算旋轉時,請注意我們會旋轉 dolly ,但會計算 主要攝影機 的旋轉。When we calculate the rotation, note that we are rotating our dolly but calculating the rotation around the main camera. 這點很重要,因為 主要相機 離0、0、0愈遠,dolly 的旋轉就會從使用者的觀點來看,不精確。This is important as the farther the main camera is away from 0,0,0, the less accurate a rotation around the dolly would become from the point of view of the user. 事實上,如果您沒有繞著相機位置旋轉,則使用者會在 dolly 前後移動弧形,而不是旋轉。In fact, if you do not rotate around the camera position, the user will move on an arc around the dolly rather than rotating.

MixedRealityTeleport 的第二個工作是處理 dolly 的移動。The second job for MixedRealityTeleport is to handle moving the dolly. 這是在 SetWorldPosition 中完成。This is done in SetWorldPosition. SetWorldPosition 會採用所需的世界位置,也就是使用者想要 percieve 占的位置。SetWorldPosition takes the desired world position, the position where the user wants to percieve that they inhabit. 我們需要將 dolly 放在該位置減去 主要攝影機 的本機位置,因為該位移將會新增至每個畫面格。We need to put our dolly at that position minus the local position of the main camera, as that offset will be added each frame.

第二個腳本會呼叫 SetWorldPosition。A second script calls SetWorldPosition. 讓我們來看看那個腳本。Let's look at that script. 在 [專案] 面板中,流覽至 [ Assets\AppPrefabs\Support\Scripts\GameLogic ],然後按兩下 [ TeleportScript.cs]。In the project panel, navigate to Assets\AppPrefabs\Support\Scripts\GameLogic and double-click on TeleportScript.cs.

此腳本比 MixedRealityTeleport 更簡單。This script is a little more involved than MixedRealityTeleport. 腳本正在檢查 Xbox 控制器上的 Y 按鈕是否已關閉。The script is checking for the Y Button on the Xbox controller to be held down. 當按鈕關閉時,會轉譯傳送游標,而且腳本會將光線從使用者的注視位置轉換出來。While the button is held down a teleport cursor is rendered and the script casts a ray from the user's gaze position. 如果該光線與較多或較少指標的表面衝突,則會將介面視為適合用來傳送的介面,並且會啟用傳送游標上的動畫。If that ray collides with a surface that is more or less pointing up, the surface will be considered a good surface to teleport to, and the animation on the teleport cursor will be enabled. 如果光線未與某個介面相較或更低,則會停用游標上的動畫。If the ray does not collide with a surface more or less pointing up, then the animation on the cursor will be disabled. 當您放開 Y 按鈕,且光線的計算點是有效的位置時,腳本會使用與光線交集的位置來呼叫 SetWorldPosition。When the Y button is released and the calculated point of the ray is a valid position, the script calls SetWorldPosition with the position the ray intersected.

享受您的進度Enjoy your progress

這次您將需要尋找 friend。This time you'll need to find a friend.

同樣地,具有 HoloLens 的使用者將會裝載一個會話。Once again, a user with the HoloLens will host a session. 其他使用者將加入會話。Other users will join the session. 應用程式會將前三位使用者從島的三個路徑中的其中一個上的沉浸式耳機加入。The application will place the first three users to join from an immersive headset on one of the three paths on the island. 您可以隨時流覽這一節中的孤島。Feel free to explore the island in this section.

要注意的詳細資料:Details to notice:

  1. 您可以看到雲端中的臉部,以協助可以沉浸使用者查看 HoloLens 使用者的期望方向。You can see faces in the clouds, which helps an immersed user see which direction a HoloLens user is looking.
  2. 島上的虛擬人偶具有旋轉的 necks。The avatars on the island have necks that rotate. 他們不會遵循使用者正在做的事, (我們沒有這項資訊) 但可提供絕佳的體驗。They won't follow what the user is doing is real reality (we don't have that information) but it makes for a nice experience.
  3. 如果 HoloLens 使用者正在查看島,可以沉浸使用者可以看到他們的游標。If the HoloLens user is looking at the Island, the immersed users can see their cursor.
  4. 代表 HoloLens 使用者轉換陰影的雲端。The clouds that represent the HoloLens users cast shadows.

第5章-FinaleChapter 5 - Finale

目標Objectives

建立兩個裝置類型之間的共同作業互動體驗。Create a collaborative interactive experience between the two device types.

我們將建立的內容What we will build

以第4章為基礎,當具有沉浸式耳機的使用者接近島上的拼圖時,HoloLens 使用者將會取得工具提示,其中包含謎題的線索。Building on chapter 4, when a user with an immersive headset gets near a puzzle on the island, the HoloLens users will get a tool tip with a clue to the puzzle. 一旦所有的沉浸式耳機使用者都超過其謎題,並放到 rocket 室中的「就緒板」之後,rocket 就會啟動。Once all of the immersive headset users get past their puzzles and onto the "ready pad" in the rocket room, the rocket will launch.

步驟Steps

  • [階層] 中選取[ Usland]。In Hierarchy, select Usland.
  • 在 [ 檢查] 的 [ 層級控制] 中,勾選 [ 啟用 共同作業]。In Inspector, in Level Control, check Enable Collaboration.

深入探討至程式碼Digging into the code

現在讓我們看看 LevelControl.cs。Now let us look at LevelControl.cs. 此腳本是遊戲邏輯的核心,可維持遊戲狀態。This script is the core of the game logic and maintains the game state. 由於這是使用 UNET 的多人遊戲,我們需要瞭解資料流程的運作方式,而且至少要有足夠的時間來修改本教學課程。Since this is a multiplayer game using UNET we need to understand how data flows, at least well enough to modify this tutorial. 如需更完整的 UNET 總覽,請參閱 Unity 的檔。For a more complete overview of UNET, please refer to Unity's documentation.

在 [專案] 面板中,流覽至 [ Assets\AppPrefabs\Support\Scripts\GameLogic ],然後按兩下 [ LevelControl.cs]。In the project panel, navigate to Assets\AppPrefabs\Support\Scripts\GameLogic and double-click on LevelControl.cs.

請讓我們瞭解沉浸式耳機如何表示已準備好開始 rocket。Let us understand how an immersive headset indicates that they are ready for the rocket launch. Rocket 啟動準備工作的方式,是在對應到島上三個路徑的 bool 清單中設定三個 bool 的其中一個。Rocket Launch readiness is communicated by setting one of three bools in a list of bools that correspond to the three paths on the island. 當指派給路徑的使用者位於 rocket 房間內棕色板的上方時,就會設定路徑的 bool。A path's bool will be set when the user assigned to the path is on top of the brown pad inside the rocket room. 好的,現在的詳細資料。Okay, now to the details.

我們將從 Update ( # A1 函數開始。We will start in the Update() function. 您將會注意到有「功能提要」功能。You will note that there is a 'cheat' function. 我們在開發過程中使用此功能來測試 rocket 的啟動和重設順序。We used this in development to test the rocket launch and reset sequence. 它無法在多使用者體驗中使用。It won't work in the multi user experience. 希望您旨在下列資訊的時間,您可以讓它正常運作。Hopefully by the time you internalize the following infromation you can make it work. 在我們檢查看看是否有任何功能時,我們會檢查本機播放機是否可以沉浸。After we check to see if we should cheat, we check to see if the local player is immersed. 我們希望專注于我們的目標。We want to focus on how we find that we're at the goal. 在中,如果 (可以沉浸) 檢查,則會呼叫 CheckGoal 在 EnableCollaboration bool 後方隱藏。Inside of the if (Immersed) check, there is a call to CheckGoal hiding behind the EnableCollaboration bool. 這會對應到您在完成本章的步驟時所檢查的核取方塊。This corresponds to the checkbox you checked while completing the steps for this chapter. 在 EnableCollaboration 中,我們會看到 CheckGoal ( # A1 的呼叫。Inside of EnableCollaboration we see a call to CheckGoal().

CheckGoal 會進行一些數學運算,以查看是否有更多或更少的站在板上。CheckGoal does some math to see if we are more or less standing on the pad. 當我們在進行時,我們會將「抵達目標」記錄檔,然後呼叫「SendAtGoalMessage ( # A1」。When we are, we Debug.Log "Arrived at goal" and then we call 'SendAtGoalMessage()'. 在 SendAtGoalMessage 中,我們稱之為 >playercontroller. SendAtGoal。In SendAtGoalMessage we call playerController.SendAtGoal. 為了節省時間,以下是程式碼:To save you some time, here's the code:

private void CmdSendAtGoal(int GoalIndex)
{
    levelState.SetGoalIndex(GoalIndex);
}
public void SendAtGoal(int GoalIndex)
{
    if (isLocalPlayer)
    {
        Debug.Log("sending at goal " + GoalIndex);
        CmdSendAtGoal(GoalIndex);
    }
}

請注意,SendAtGoalMessage 會呼叫 CmdSendAtGoal,其會呼叫 levelState SetGoalIndex,這會傳回 LevelControl.cs。Note that SendAtGoalMessage calls CmdSendAtGoal, which calls levelState.SetGoalIndex, which is back in LevelControl.cs. 乍看之下,這似乎很奇怪。At first glance this seems strange. 為什麼不直接呼叫 SetGoalIndex,而不是透過 player 控制器來進行這種奇怪的路由?Why not just call SetGoalIndex rather than this strange routing through the player controller? 原因是我們符合資料模型 UNET 用來保持資料同步。為了避免作弊並使其失效,UNET 要求每個物件都有一個具有變更已同步變數之許可權的使用者。The reason is that we are conforming to the data model UNET uses to keep data in sync. To prevent cheating and thrashing, UNET requires that each object has a user who has authority to change the synchronized variables. 此外,只有主控制項 (啟動會話) 的使用者可以直接變更資料。Further, only the host (the user that started the session) can change data directly. 如果使用者不是主機,但具有授權單位,則必須將「命令」傳送至將變更變數的主控制項。Users who are not the host, but have authority, need to send a 'command' to the host which will change the variable. 依預設,主機具有所有物件的許可權,但衍生來代表使用者的物件除外。By default the host has authority over all objects, except for the object spawned to represent the user. 在我們的案例中,此物件具有 >playercontroller 腳本。In our case this object has the playercontroller script. 有一種方法可以要求物件的授權單位,然後進行變更,但我們選擇利用播放程式控制器具有自我授權,並透過 player 控制器路由傳送命令的事實。There is a way to request authority for an object and then make changes, but we choose to leverage the fact that the player controller has self authority and route commands through the player controller.

另一種方式是,當我們找到自己的目標時,玩家需要告訴主機,而主機會告訴大家其他人。Said another way, when we've found ourselves at our goal, the player needs to tell the host, and the host will tell everyone else.

回到 LevelControl.cs 查看 SetGoalIndex。Back in LevelControl.cs look at SetGoalIndex. 在這裡,我們要設定 synclist (AtGoal) 中值的值。Here we are setting the value of a value in a synclist (AtGoal). 請記住,我們會在主機的內容中進行這項作業。Remember that we are in the context of the host while we do this. 類似于命令,RPC 是主機可發出的內容,會導致所有用戶端執行一些程式碼。Similar to a command, an RPC is something the host can issue that will cause all clients to run some code. 在這裡,我們稱之為「RpcCheckAllGoals」。Here we call 'RpcCheckAllGoals'. 每個用戶端都會個別檢查是否已設定全部三個 AtGoals,如果是,則啟動 rocket。Each client will individually check to see if all three AtGoals are set, and if so, launch the rocket.

享受您的進度Enjoy your progress

在上一章中,我們將依之前的方式啟動會話。Building on the previous chapter, we will start the session as before. 這次,當沉浸式耳機中的使用者到達其路徑上的「門」時,就會出現只有 HoloLens 使用者可以看到的工具提示。This time as the users in the immersive headset get to the "door" on their path, a tooltip will appear that only the HoloLens users can see. HoloLens 使用者負責將這個線索傳達給沉浸式耳機中的使用者。The HoloLens users are responsible for communicating this clue to the users in the immersive headset. 一旦每個頭像在火山內的對應棕色板上進行了 rocket,就會啟動空間。The rocket will launch to space once each avatar has stepped on its corresponding brown pad inside the volcano. 場景會在60秒後重設,以便您可以再次進行。The scene will reset after 60 seconds so you can do it again.

另請參閱See also