MR Input 211:手勢MR Input 211: Gesture

注意

混合實境學院教學課程的設計是以 HoloLens (第 1 代) 和混合實境沉浸式頭戴裝置為準。The Mixed Reality Academy tutorials were designed with HoloLens (1st gen) and Mixed Reality Immersive Headsets in mind. 因此,對於仍在尋找這些裝置開發指引的開發人員而言,我們覺得這些教學課程很重要。As such, we feel it is important to leave these tutorials in place for developers who are still looking for guidance in developing for those devices. 這些教學課程 不會 使用用於 HoloLens 2 的最新工具組或互動進行更新。These tutorials will not be updated with the latest toolsets or interactions being used for HoloLens 2. 系統會保留這些資訊,以繼續在支援的裝置上運作。They will be maintained to continue working on the supported devices. 已針對 HoloLens 2 公佈一系列新的教學課程A new series of tutorials has been posted for HoloLens 2.

手勢 將使用者意圖變成行動。Gestures turn user intention into action. 透過手勢,使用者可以與全像投影互動。With gestures, users can interact with holograms. 在此課程中,我們將瞭解如何追蹤使用者的手、回應使用者輸入,並根據手邊的州和地點將意見反應提供給使用者。In this course, we'll learn how to track the user's hands, respond to user input, and give feedback to the user based on hand state and location.

MR 基本的 101中,我們使用了簡單的點一下手勢來與我們的全像投影互動。In MR Basics 101, we used a simple air-tap gesture to interact with our holograms. 現在,我們將移至 [輕量] 手勢之外,並探索新的概念以:Now, we'll move beyond the air-tap gesture and explore new concepts to:

  • 偵測使用者的手,並提供意見反應給使用者。Detect when the user's hand is being tracked and provide feedback to the user.
  • 使用導覽手勢來旋轉我們的全像投影。Use a navigation gesture to rotate our holograms.
  • 當使用者的手即將消失時,請提供意見反應。Provide feedback when the user's hand is about to go out of view.
  • 使用操作事件,讓使用者能夠以手移入全像移動影像。Use manipulation events to allow users to move holograms with their hands.

在此課程中,我們將重新流覽 Unity 專案 模型瀏覽器,這是我們在 MR 輸入 210中建立的。In this course, we'll revisit the Unity project Model Explorer, which we built in MR Input 210. 我們太空人的朋友會回頭協助我們探索這些新的手勢概念。Our astronaut friend is back to assist us in our exploration of these new gesture concepts.

重要

下列各章節中內嵌的影片是使用較舊版本的 Unity 和混合現實工具組所記錄。The videos embedded in each of the chapters below were recorded using an older version of Unity and the Mixed Reality Toolkit. 雖然逐步指示是正確且最新的,但您可能會在相對應的影片中看到已過期的腳本和視覺效果。While the step-by-step instructions are accurate and current, you may see scripts and visuals in the corresponding videos that are out-of-date. 影片仍隨附于 posterity,因為所涵蓋的概念仍然適用。The videos remain included for posterity and because the concepts covered still apply.

裝置支援Device support

課程Course HoloLensHoloLens 沉浸式頭戴裝置Immersive headsets
MR Input 211:手勢MR Input 211: Gesture ✔️✔️ ✔️✔️

開始之前Before you start

先決條件Prerequisites

專案檔Project files

  • 下載專案 所需的 檔案。Download the files required by the project. 需要 Unity 2017.2 或更新版本。Requires Unity 2017.2 or later.
  • 取消將檔案封存到您的桌面或其他易於觸及的位置。Un-archive the files to your desktop or other easy to reach location.

注意

如果您想要在下載之前查看原始程式碼, 可在 GitHub 上取得。If you want to look through the source code before downloading, it's available on GitHub.

勘誤表和記事Errata and Notes

  • 您必須在 [工具]-[>選項] 下的 Visual Studio 中停用 [啟用 Just My Code] ( 核取) ,才能在程式碼中叫用中斷點。"Enable Just My Code" needs to be disabled (unchecked) in Visual Studio under Tools->Options->Debugging in order to hit breakpoints in your code.

第0章-Unity 設定Chapter 0 - Unity Setup

指示Instructions

  1. 啟動 Unity。Start Unity.
  2. 選取 [開啟] 。Select Open.
  3. 流覽至您先前取消封存的 手勢 資料夾。Navigate to the Gesture folder you previously un-archived.
  4. 尋找並選取 [啟動 / 模型瀏覽器] 資料夾。Find and select the Starting/Model Explorer folder.
  5. 按一下 [ 選取資料夾 ] 按鈕。Click the Select Folder button.
  6. 在 [ 專案 ] 面板中,展開 [ 場景 ] 資料夾。In the Project panel, expand the Scenes folder.
  7. 按兩下 [ ModelExplorer 場景],以在 Unity 中載入。Double-click ModelExplorer scene to load it in Unity.

建置Building

  1. 在 Unity 中,選取 [ File > Build Settings]。In Unity, select File > Build Settings.
  2. 如果 場景/ModelExplorer 未列在 組建的場景 中,請按一下 [ 新增開啟場景 ] 以加入場景。If Scenes/ModelExplorer is not listed in Scenes In Build, click Add Open Scenes to add the scene.
  3. 如果您是特別針對 HoloLens 進行開發,請將 目標裝置 設定為 hololensIf you're specifically developing for HoloLens, set Target device to HoloLens. 否則,請將它保留在 任何裝置 上。Otherwise, leave it on Any device.
  4. 請確定 組建類型 設定為 [ D3D ],並將 [ Sdk ] 設定為 [ 最新安裝 的 (,其應為 SDK 16299 或更新版本的) 。Ensure Build Type is set to D3D and SDK is set to Latest installed (which should be SDK 16299 or newer).
  5. 按一下 [建置]。Click Build.
  6. 建立名為 "App" 的 新資料夾Create a New Folder named "App".
  7. 按一下 應用程式 資料夾。Single click the App folder.
  8. 按下 [ 選取資料夾 ],Unity 就會開始建立 Visual Studio 的專案。Press Select Folder and Unity will start building the project for Visual Studio.

當 Unity 完成時,將會出現檔案總管視窗。When Unity is done, a File Explorer window will appear.

  1. 開啟 應用程式 資料夾。Open the App folder.
  2. 開啟 ModelExplorer Visual Studio 方案Open the ModelExplorer Visual Studio Solution.

如果要部署到 HoloLens:If deploying to HoloLens:

  1. 使用 Visual Studio 中的頂端工具列,將目標從 Debug 變更為 Release ,以及從 ARM 變更為 x86Using the top toolbar in Visual Studio, change the target from Debug to Release and from ARM to x86.
  2. 按一下 [本機電腦] 按鈕旁的下拉箭號,然後選取 [ 遠端電腦]。Click on the drop down arrow next to the Local Machine button, and select Remote Machine.
  3. 輸入 您的 HoloLens 裝置 IP 位址 ,並將驗證模式設定為 通用 (未加密的通訊協定)Enter your HoloLens device IP address and set Authentication Mode to Universal (Unencrypted Protocol). 按一下 [選取]。Click Select. 如果您不知道您的裝置 IP 位址,請查看 [ 設定] > Network & Internet > Advanced 選項If you do not know your device IP address, look in Settings > Network & Internet > Advanced Options.
  4. 在頂端功能表列中,按一下 [ Debug-> 啟動但不進行調試 ],或按 Ctrl + F5In the top menu bar, click Debug -> Start Without debugging or press Ctrl + F5. 如果這是您第一次部署至您的裝置,您必須將 它與 Visual Studio 配對If this is the first time deploying to your device, you will need to pair it with Visual Studio.
  5. 部署應用程式之後,請使用 選取手勢 來關閉 FitboxWhen the app has deployed, dismiss the Fitbox with a select gesture.

如果部署到沉浸式耳機:If deploying to an immersive headset:

  1. 使用 Visual Studio 中的頂端工具列,將目標從 Debug 變更為 Release ,以及從 ARM 變更為 x64Using the top toolbar in Visual Studio, change the target from Debug to Release and from ARM to x64.
  2. 請確定部署目標設定為 [ 本機電腦]。Make sure the deployment target is set to Local Machine.
  3. 在頂端功能表列中,按一下 [ Debug-> 啟動但不進行調試 ],或按 Ctrl + F5In the top menu bar, click Debug -> Start Without debugging or press Ctrl + F5.
  4. 部署應用程式之後,請在動作控制器上提取觸發程式來關閉 FitboxWhen the app has deployed, dismiss the Fitbox by pulling the trigger on a motion controller.

注意

您可能會注意到 Visual Studio 錯誤面板中有一些紅色錯誤。You might notice some red errors in the Visual Studio Errors panel. 您可以放心地忽略它們。It is safe to ignore them. 切換至 [輸出] 面板以查看實際的組建進度。Switch to the Output panel to view actual build progress. 輸出面板中的錯誤會要求您進行修正 (最常見的原因是腳本) 中發生錯誤。Errors in the Output panel will require you to make a fix (most often they are caused by a mistake in a script).

第1章-手動偵測到的意見反應Chapter 1 - Hand detected feedback

目標Objectives

  • 訂閱手邊的追蹤事件。Subscribe to hand tracking events.
  • 使用資料指標意見反應,在追蹤手時顯示使用者。Use cursor feedback to show users when a hand is being tracked.

注意

在 HoloLens 2 上,只要手指出現時,就會引發 (,而不只是手指指向) 。On HoloLens 2 , hands detected fires whenever hands are visible (not just when a finger is pointing up).

指示Instructions

  • 在 [階層] 面板中,展開 [ InputManager ] 物件。In the Hierarchy panel, expand the InputManager object.
  • 尋找並選取 [ GesturesInput ] 物件。Look for and select the GesturesInput object.

InteractionInputSource.cs 腳本會執行這些步驟:The InteractionInputSource.cs script performs these steps:

  1. 訂閱 InteractionSourceDetected 和 InteractionSourceLost 事件。Subscribes to the InteractionSourceDetected and InteractionSourceLost events.
  2. 設定 HandDetected 狀態。Sets the HandDetected state.
  3. 從 InteractionSourceDetected 和 InteractionSourceLost 事件取消訂閱。Unsubscribes from the InteractionSourceDetected and InteractionSourceLost events.

接下來,我們會根據使用者的動作,將我們的資料指標從 MR 輸入 210 升級為一個顯示意見反應的資料指標。Next, we'll upgrade our cursor from MR Input 210 into one that shows feedback depending on the user's actions.

  1. 在 [階層] 面板中,選取資料 指標 物件,並將其刪除。In the Hierarchy panel, select the Cursor object and delete it.
  2. 在 [ 專案 ] 面板中,搜尋 CursorWithFeedback ,然後將它拖曳 至 [階層 ] 面板中。In the Project panel, search for CursorWithFeedback and drag it into the Hierarchy panel.
  3. 按一下 [階層 ] 面板中的 [ InputManager ],然後將 [ CursorWithFeedback ] 物件從階層 拖曳至偵測****器 底部的 InputManager SimpleSinglePointerSelector 資料 指標 欄位中。Click on InputManager in the Hierarchy panel, then drag the CursorWithFeedback object from the Hierarchy into the InputManager's SimpleSinglePointerSelector's Cursor field, at the bottom of the Inspector.
  4. 按一下階層 中的 CursorWithFeedbackClick on the CursorWithFeedback in the Hierarchy.
  5. 在 [偵測 ] 面板中,展開物件資料 指標 腳本上的 資料指標狀態資料In the Inspector panel, expand Cursor State Data on the Object Cursor script.

資料指標狀態資料 的運作方式如下:The Cursor State Data works like this:

  • 任何 觀察 狀態都表示未偵測到任何東西,而使用者只需要四處尋找。Any Observe state means that no hand is detected and the user is simply looking around.
  • 任何 互動 狀態表示偵測到手或控制器。Any Interact state means that a hand or controller is detected.
  • 任何 止狀態都表示使用者正在查看全像影像。Any Hover state means the user is looking at a hologram.

建置和部署Build and Deploy

  • 在 Unity 中,使用檔案 > 組建設定 來重建應用程式。In Unity, use File > Build Settings to rebuild the application.
  • 開啟 應用程式 資料夾。Open the App folder.
  • 如果尚未開啟,請開啟 ModelExplorer Visual Studio 方案If it's not already open, open the ModelExplorer Visual Studio Solution.
    • (如果您已在安裝期間于 Visual Studio 中建立或部署此專案,則您可以開啟 VS 的實例,然後在出現) 提示時按一下 [全部重載]。(If you already built/deployed this project in Visual Studio during set-up, then you can open that instance of VS and click 'Reload All' when prompted).
  • 在 Visual Studio 中,按一下 [ Debug-> 啟動但不進行調試 ],或按 Ctrl + F5In Visual Studio, click Debug -> Start Without debugging or press Ctrl + F5.
  • 將應用程式部署到 HoloLens 之後,請使用 [點一下手勢] 來關閉 fitbox。After the application deploys to the HoloLens, dismiss the fitbox using the air-tap gesture.
  • 將您的手移至視野,並將您的索引手指指向天空以開始進行追蹤。Move your hand into view and point your index finger to the sky to start hand tracking.
  • 將您的手靠左、靠右、向上和向下移動。Move your hand left, right, up and down.
  • 觀賞偵測到您手上的游標變更,然後從 view 遺失。Watch how the cursor changes when your hand is detected and then lost from view.
  • 如果您是在沉浸式耳機上,則必須連接並中斷控制器的連線。If you're on an immersive headset, you'll have to connect and disconnect your controller. 這種意見反應在沉浸式裝置上變得較不有趣,因為連接的控制器一律會「可用」。This feedback becomes less interesting on an immersive device, as a connected controller will always be "available".

第2章-導覽Chapter 2 - Navigation

目標Objectives

  • 使用導覽手勢事件來旋轉太空人。Use Navigation gesture events to rotate the astronaut.

指示Instructions

若要在我們的應用程式中使用導覽手勢,我們將會在導覽手勢發生時,編輯 GestureAction.cs 來旋轉物件。To use Navigation gestures in our app, we are going to edit GestureAction.cs to rotate objects when the Navigation gesture occurs. 此外,我們會將意見反應新增至游標,以在流覽可供使用時顯示。Additionally, we'll add feedback to the cursor to display when Navigation is available.

  1. 在 [階層] 面板中,展開 [ CursorWithFeedback]。In the Hierarchy panel, expand CursorWithFeedback.
  2. 在 [全像 ] 資料夾中 ,尋找 ScrollFeedback 資產。In the Holograms folder, find the ScrollFeedback asset.
  3. ScrollFeedback 預製專案拖放到階層中的 CursorWithFeedback GameObject。Drag and drop the ScrollFeedback prefab onto the CursorWithFeedback GameObject in the Hierarchy.
  4. 按一下 [ CursorWithFeedback]。Click on CursorWithFeedback.
  5. 在 [偵測 ] 面板中,按一下 [ 新增元件 ] 按鈕。In the Inspector panel, click the Add Component button.
  6. 在功能表的 [搜尋] 方塊中,輸入 CursorFeedbackIn the menu, type in the search box CursorFeedback. 選取搜尋結果。Select the search result.
  7. 從階層中,將 ScrollFeedback 物件拖放至偵測****器游標回饋 元件的 [ Scroll 偵測到的遊戲物件] 屬性。Drag and drop the ScrollFeedback object from the Hierarchy onto the Scroll Detected Game Object property in the Cursor Feedback component in the Inspector.
  8. 在 [階層] 面板中,選取 [ AstroMan ] 物件。In the Hierarchy panel, select the AstroMan object.
  9. 在 [偵測 ] 面板中,按一下 [ 新增元件 ] 按鈕。In the Inspector panel, click the Add Component button.
  10. 在功能表中,輸入搜尋方塊的 手勢動作In the menu, type in the search box Gesture Action. 選取搜尋結果。Select the search result.

接下來,在 Visual Studio 中開啟 GestureAction.csNext, open GestureAction.cs in Visual Studio. 在編碼練習 2. c 中,編輯腳本以執行下列動作:In coding exercise 2.c, edit the script to do the following:

  1. 每當執行導覽手勢時 ,旋轉 AstroMan 物件。Rotate the AstroMan object whenever a Navigation gesture is performed.
  2. 計算 rotationFactor 以控制套用至物件的旋轉量。Calculate the rotationFactor to control the amount of rotation applied to the object.
  3. 當使用者向左或向右移動時,旋轉 y 軸周圍的 物件Rotate the object around the y-axis when the user moves their hand left or right.

在腳本中完成編碼練習 2. c,或以下列完成的解決方案取代程式碼:Complete coding exercises 2.c in the script, or replace the code with the completed solution below:

using HoloToolkit.Unity.InputModule;
using UnityEngine;

/// <summary>
/// GestureAction performs custom actions based on
/// which gesture is being performed.
/// </summary>
public class GestureAction : MonoBehaviour, INavigationHandler, IManipulationHandler, ISpeechHandler
{
    [Tooltip("Rotation max speed controls amount of rotation.")]
    [SerializeField]
    private float RotationSensitivity = 10.0f;

    private bool isNavigationEnabled = true;
    public bool IsNavigationEnabled
    {
        get { return isNavigationEnabled; }
        set { isNavigationEnabled = value; }
    }

    private Vector3 manipulationOriginalPosition = Vector3.zero;

    void INavigationHandler.OnNavigationStarted(NavigationEventData eventData)
    {
        InputManager.Instance.PushModalInputHandler(gameObject);
    }

    void INavigationHandler.OnNavigationUpdated(NavigationEventData eventData)
    {
        if (isNavigationEnabled)
        {
            /* TODO: DEVELOPER CODING EXERCISE 2.c */

            // 2.c: Calculate a float rotationFactor based on eventData's NormalizedOffset.x multiplied by RotationSensitivity.
            // This will help control the amount of rotation.
            float rotationFactor = eventData.NormalizedOffset.x * RotationSensitivity;

            // 2.c: transform.Rotate around the Y axis using rotationFactor.
            transform.Rotate(new Vector3(0, -1 * rotationFactor, 0));
        }
    }

    void INavigationHandler.OnNavigationCompleted(NavigationEventData eventData)
    {
        InputManager.Instance.PopModalInputHandler();
    }

    void INavigationHandler.OnNavigationCanceled(NavigationEventData eventData)
    {
        InputManager.Instance.PopModalInputHandler();
    }

    void IManipulationHandler.OnManipulationStarted(ManipulationEventData eventData)
    {
        if (!isNavigationEnabled)
        {
            InputManager.Instance.PushModalInputHandler(gameObject);

            manipulationOriginalPosition = transform.position;
        }
    }

    void IManipulationHandler.OnManipulationUpdated(ManipulationEventData eventData)
    {
        if (!isNavigationEnabled)
        {
            /* TODO: DEVELOPER CODING EXERCISE 4.a */

            // 4.a: Make this transform's position be the manipulationOriginalPosition + eventData.CumulativeDelta
        }
    }

    void IManipulationHandler.OnManipulationCompleted(ManipulationEventData eventData)
    {
        InputManager.Instance.PopModalInputHandler();
    }

    void IManipulationHandler.OnManipulationCanceled(ManipulationEventData eventData)
    {
        InputManager.Instance.PopModalInputHandler();
    }

    void ISpeechHandler.OnSpeechKeywordRecognized(SpeechEventData eventData)
    {
        if (eventData.RecognizedText.Equals("Move Astronaut"))
        {
            isNavigationEnabled = false;
        }
        else if (eventData.RecognizedText.Equals("Rotate Astronaut"))
        {
            isNavigationEnabled = true;
        }
        else
        {
            return;
        }

        eventData.Use();
    }
}

您將會注意到其他導覽事件已填入部分資訊。You'll notice that the other navigation events are already filled in with some info. 我們會將 GameObject 推送至工具組的 InputSystem's 強制回應堆疊,如此一來,使用者就不需要在旋轉開始之後,將焦點放在太空人上。We push the GameObject onto the Toolkit's InputSystem's modal stack, so the user doesn't have to maintain focus on the Astronaut once rotation has begun. 同樣地,當手勢完成時,我們會將 GameObject 從堆疊中取出。Correspondingly, we pop the GameObject off the stack once the gesture is completed.

建置和部署Build and Deploy

  1. 在 Unity 中重建應用程式,然後從 Visual Studio 建立和部署應用程式,以在 HoloLens 中執行該應用程式。Rebuild the application in Unity and then build and deploy from Visual Studio to run it in the HoloLens.
  2. 在太空人中,兩個箭號應該會出現在游標的任一邊。Gaze at the astronaut, two arrows should appear on either side of the cursor. 這個新的視覺效果表示可以旋轉太空人。This new visual indicates that the astronaut can be rotated.
  3. 將您的手移至天空 (的索引手指) ,讓 HoloLens 開始追蹤您的手。Place your hand in the ready position (index finger pointed towards the sky) so the HoloLens will start tracking your hand.
  4. 若要旋轉太空人,請將您的索引指向較低的位置,然後向左或向右移動以觸發 NavigationX 手勢。To rotate the astronaut, lower your index finger to a pinch position, and then move your hand left or right to trigger the NavigationX gesture.

第3章-手指引Chapter 3 - Hand Guidance

目標Objectives

  • 使用 手動指引分數 來協助預測何時會遺失手動追蹤。Use hand guidance score to help predict when hand tracking will be lost.
  • 提供資料 指標的意見 反應,以在使用者的手中接近相機的視圖邊緣時顯示。Provide feedback on the cursor to show when the user's hand nears the camera's edge of view.

指示Instructions

  1. 在 [階層] 面板中,選取 [ CursorWithFeedback ] 物件。In the Hierarchy panel, select the CursorWithFeedback object.
  2. 在 [偵測 ] 面板中,按一下 [ 新增元件 ] 按鈕。In the Inspector panel, click the Add Component button.
  3. 在功能表中,輸入搜尋方塊 手動指引In the menu, type in the search box Hand Guidance. 選取搜尋結果。Select the search result.
  4. 在 [ 專案 ] 面板的 [全像] 資料夾中,尋找 HandGuidanceFeedback 資產。In the Project panel Holograms folder, find the HandGuidanceFeedback asset.
  5. HandGuidanceFeedback 資產拖放到 [偵測 ] 面板中的 [手形指導指標] 屬性。Drag and drop the HandGuidanceFeedback asset onto the Hand Guidance Indicator property in the Inspector panel.

建置和部署Build and Deploy

  • 在 Unity 中重建應用程式,然後從 Visual Studio 建立並部署,以在 HoloLens 上體驗應用程式。Rebuild the application in Unity and then build and deploy from Visual Studio to experience the app on HoloLens.
  • 將您的手移至視野,並提升您的食指以進行追蹤。Bring your hand into view and raise your index finger to get tracked.
  • 使用導覽手勢開始旋轉太空人 (將您的索引手指和) 一起縮小。Start rotating the astronaut with the Navigation gesture (pinch your index finger and thumb together).
  • 將您的手上移、靠右、向上和向下移動。Move your hand far left, right, up, and down.
  • 當您手接近手勢框架的邊緣時,游標旁邊應該會出現箭號,以警告您將會遺失手動追蹤。As your hand nears the edge of the gesture frame, an arrow should appear next to the cursor to warn you that hand tracking will be lost. 箭號會指出移動手的方向,以防止追蹤遺失。The arrow indicates which direction to move your hand in order to prevent tracking from being lost.

第4章-操作Chapter 4 - Manipulation

目標Objectives

  • 使用操作事件以您的手移太空人。Use Manipulation events to move the astronaut with your hands.
  • 提供資料指標的意見反應,讓使用者知道可以使用操作的時機。Provide feedback on the cursor to let the user know when Manipulation can be used.

指示Instructions

GestureManager.cs 和 AstronautManager.cs 可讓我們執行下列作業:GestureManager.cs and AstronautManager.cs will allow us to do the following:

  1. 使用語音關鍵字 "Move 太空人" 來啟用 操作 手勢和「旋轉太空人」以停用它們。Use the speech keyword "Move Astronaut" to enable Manipulation gestures and "Rotate Astronaut" to disable them.
  2. 切換以回應 操作手勢辨識器Switch to responding to the Manipulation Gesture Recognizer.

現在就開始吧。Let's get started.

  1. 在 [階層] 面板中,建立新的空白 GameObject。In the Hierarchy panel, create a new empty GameObject. 將它命名為 "AstronautManager"。Name it "AstronautManager".
  2. 在 [偵測 ] 面板中,按一下 [ 新增元件 ] 按鈕。In the Inspector panel, click the Add Component button.
  3. 在功能表中,于 [搜尋 box 太空人管理員] 中輸入。In the menu, type in the search box Astronaut Manager. 選取搜尋結果。Select the search result.
  4. 在 [偵測 ] 面板中,按一下 [ 新增元件 ] 按鈕。In the Inspector panel, click the Add Component button.
  5. 在功能表中,輸入搜尋方塊 語音輸入來源In the menu, type in the search box Speech Input Source. 選取搜尋結果。Select the search result.

我們現在會新增控制太空人互動狀態所需的語音命令。We'll now add the speech commands required to control the interaction state of the astronaut.

  1. 展開 [檢查] 中的 [關鍵字] 區段。Expand the Keywords section in the Inspector.
  2. 按一下右側的, + 以加入新的關鍵字。Click the + on the right hand side to add a new keyword.
  3. 輸入關鍵字做為 移動太空人Type the Keyword as Move Astronaut. 如有需要,請隨意新增按鍵快捷方式。Feel free to add a Key Shortcut if desired.
  4. 按一下右側的, + 以加入新的關鍵字。Click the + on the right hand side to add a new keyword.
  5. 將關鍵字輸入為 旋轉太空人Type the Keyword as Rotate Astronaut. 如有需要,請隨意新增按鍵快捷方式。Feel free to add a Key Shortcut if desired.
  6. 對應的處理常式程式碼可以在 GestureAction.csISpeechHandler. OnSpeechKeywordRecognized 處理常式中找到。The corresponding handler code can be found in GestureAction.cs, in the ISpeechHandler.OnSpeechKeywordRecognized handler.

如何設定第4章的語音輸入來源

接下來,我們將設定資料指標的操作意見反應。Next, we'll setup the manipulation feedback on the cursor.

  1. 在 [ 專案 ] 面板的 [全像] 資料夾中,尋找 PathingFeedback 資產。In the Project panel Holograms folder, find the PathingFeedback asset.
  2. PathingFeedback 預製專案拖放到階層中的 CursorWithFeedback 物件。Drag and drop the PathingFeedback prefab onto the CursorWithFeedback object in the Hierarchy.
  3. 在 [階層] 面板中,按一下 [ CursorWithFeedback]。In the Hierarchy panel, click on CursorWithFeedback.
  4. 從階層中,將 PathingFeedback 物件拖放至偵測****器 的 [資料 指標意見] 元件中的 [偵測 到的遊戲物件] 屬性。Drag and drop the PathingFeedback object from the Hierarchy onto the Pathing Detected Game Object property in the Cursor Feedback component in the Inspector.

現在,我們需要將程式碼新增至 GestureAction.cs ,以啟用下列各項:Now we need to add code to GestureAction.cs to enable the following:

  1. 將程式碼加入至 IManipulationHandler. OnManipulationUpdated 函式,這會在偵測到 操作 手勢時移動太空人。Add code to the IManipulationHandler.OnManipulationUpdated function, which will move the astronaut when a Manipulation gesture is detected.
  2. 計算 移動向量 ,以根據手上的位置決定應將太空人移至何處。Calculate the movement vector to determine where the astronaut should be moved to based on hand position.
  3. 將太空人移 至新的位置。Move the astronaut to the new position.

完成編碼練習 4. GestureAction.cs,或使用我們完成的解決方案:Complete coding exercise 4.a in GestureAction.cs, or use our completed solution below:

using HoloToolkit.Unity.InputModule;
using UnityEngine;

/// <summary>
/// GestureAction performs custom actions based on
/// which gesture is being performed.
/// </summary>
public class GestureAction : MonoBehaviour, INavigationHandler, IManipulationHandler, ISpeechHandler
{
    [Tooltip("Rotation max speed controls amount of rotation.")]
    [SerializeField]
    private float RotationSensitivity = 10.0f;

    private bool isNavigationEnabled = true;
    public bool IsNavigationEnabled
    {
        get { return isNavigationEnabled; }
        set { isNavigationEnabled = value; }
    }

    private Vector3 manipulationOriginalPosition = Vector3.zero;

    void INavigationHandler.OnNavigationStarted(NavigationEventData eventData)
    {
        InputManager.Instance.PushModalInputHandler(gameObject);
    }

    void INavigationHandler.OnNavigationUpdated(NavigationEventData eventData)
    {
        if (isNavigationEnabled)
        {
            /* TODO: DEVELOPER CODING EXERCISE 2.c */

            // 2.c: Calculate a float rotationFactor based on eventData's NormalizedOffset.x multiplied by RotationSensitivity.
            // This will help control the amount of rotation.
            float rotationFactor = eventData.NormalizedOffset.x * RotationSensitivity;

            // 2.c: transform.Rotate around the Y axis using rotationFactor.
            transform.Rotate(new Vector3(0, -1 * rotationFactor, 0));
        }
    }

    void INavigationHandler.OnNavigationCompleted(NavigationEventData eventData)
    {
        InputManager.Instance.PopModalInputHandler();
    }

    void INavigationHandler.OnNavigationCanceled(NavigationEventData eventData)
    {
        InputManager.Instance.PopModalInputHandler();
    }

    void IManipulationHandler.OnManipulationStarted(ManipulationEventData eventData)
    {
        if (!isNavigationEnabled)
        {
            InputManager.Instance.PushModalInputHandler(gameObject);

            manipulationOriginalPosition = transform.position;
        }
    }

    void IManipulationHandler.OnManipulationUpdated(ManipulationEventData eventData)
    {
        if (!isNavigationEnabled)
        {
            /* TODO: DEVELOPER CODING EXERCISE 4.a */

            // 4.a: Make this transform's position be the manipulationOriginalPosition + eventData.CumulativeDelta
            transform.position = manipulationOriginalPosition + eventData.CumulativeDelta;
        }
    }

    void IManipulationHandler.OnManipulationCompleted(ManipulationEventData eventData)
    {
        InputManager.Instance.PopModalInputHandler();
    }

    void IManipulationHandler.OnManipulationCanceled(ManipulationEventData eventData)
    {
        InputManager.Instance.PopModalInputHandler();
    }

    void ISpeechHandler.OnSpeechKeywordRecognized(SpeechEventData eventData)
    {
        if (eventData.RecognizedText.Equals("Move Astronaut"))
        {
            isNavigationEnabled = false;
        }
        else if (eventData.RecognizedText.Equals("Rotate Astronaut"))
        {
            isNavigationEnabled = true;
        }
        else
        {
            return;
        }

        eventData.Use();
    }
}

建置和部署Build and Deploy

  • 在 Unity 中重建,然後從 Visual Studio 建立並部署,以在 HoloLens 中執行應用程式。Rebuild in Unity and then build and deploy from Visual Studio to run the app in HoloLens.
  • 將您手上的手移至 HoloLens 之前,並提起您的食指以進行追蹤。Move your hand in front of the HoloLens and raise your index finger so that it can be tracked.
  • 將游標放在太空人上。Focus the cursor over the astronaut.
  • 說 ' Move 太空人 ' 可將太空人與操作手勢移動。Say 'Move Astronaut' to move the astronaut with a Manipulation gesture.
  • 游標周圍應該會出現四個箭號,以指出程式現在會回應操作事件。Four arrows should appear around the cursor to indicate that the program will now respond to Manipulation events.
  • 將您的索引向下移至您的 thumb,並讓它們 pinched 在一起。Lower your index finger down to your thumb, and keep them pinched together.
  • 當您手上四處移動時,太空人也會移動 (這是) 的操作。As you move your hand around, the astronaut will move too (this is Manipulation).
  • 提高您的索引手指,以停止操作太空人。Raise your index finger to stop manipulating the astronaut.
  • 注意:如果您在移動手之前沒有說「移動太空人」,則會改用導覽手勢。Note: If you do not say 'Move Astronaut' before moving your hand, then the Navigation gesture will be used instead.
  • 說「旋轉太空人」以返回 rotatable 狀態。Say 'Rotate Astronaut' to return to the rotatable state.

第5章-模型擴充Chapter 5 - Model expansion

目標Objectives

  • 將太空人模型擴充為多個較小的片段,讓使用者可以與之互動。Expand the Astronaut model into multiple, smaller pieces that the user can interact with.
  • 使用導覽和操作手勢來個別移動每個片段。Move each piece individually using Navigation and Manipulation gestures.

指示Instructions

在本節中,我們將完成下列工作:In this section, we will accomplish the following tasks:

  1. 加入新的關鍵字「展開模型」以展開太空人模型。Add a new keyword "Expand Model" to expand the astronaut model.
  2. 加入新的關鍵字 [重設模型],將模型傳回至其原始表單。Add a new Keyword "Reset Model" to return the model to its original form.

我們會藉由將兩個以上的關鍵字新增至上一章中的語音輸入來源來完成這項作業。We'll do this by adding two more keywords to the Speech Input Source from the previous chapter. 我們也將示範另一個處理辨識事件的方式。We'll also demonstrate another way to handle recognition events.

  1. 在 [偵測 ] 中按一下 [上一步],然後在 [檢查 ] 中展開[關鍵字] 區段。Click back on AstronautManager in the Inspector and expand the Keywords section in the Inspector.
  2. 按一下右側的, + 以加入新的關鍵字。Click the + on the right hand side to add a new keyword.
  3. 輸入關鍵字做為 展開模型Type the Keyword as Expand Model. 如有需要,請隨意新增按鍵快捷方式。Feel free to add a Key Shortcut if desired.
  4. 按一下右側的, + 以加入新的關鍵字。Click the + on the right hand side to add a new keyword.
  5. 輸入關鍵字做為 重設模型Type the Keyword as Reset Model. 如有需要,請隨意新增按鍵快捷方式。Feel free to add a Key Shortcut if desired.
  6. 在 [偵測 ] 面板中,按一下 [ 新增元件 ] 按鈕。In the Inspector panel, click the Add Component button.
  7. 在功能表中,輸入搜尋方塊 語音輸入處理常式In the menu, type in the search box Speech Input Handler. 選取搜尋結果。Select the search result.
  8. 檢查 是否為全域接聽程式,因為我們想要讓這些命令運作,而不論我們所關注的 GameObject 為何。Check Is Global Listener, since we want these commands to work regardless of the GameObject we're focusing.
  9. 按一下該 + 按鈕,然後從 [關鍵字] 下拉式清單中選取 [ 展開模型 ]。Click the + button and select Expand Model from the Keyword dropdown.
  10. 按一下 [ + 回應] 下的 [AstronautManager],然後 Hierarchy 將階層中的 [ AstronautManager ] 拖曳到 [無 (物件) ] 欄位中。Click the + under Response, and drag the AstronautManager from the Hierarchy into the None (Object) field.
  11. 現在,按一下 [ 沒有 函式] 下拉式清單,選取 [ AstronautManager],然後按一下 [ ExpandModelCommand]。Now, click the No Function dropdown, select AstronautManager, then ExpandModelCommand.
  12. 按一下 [語音輸入處理常式 + ] 按鈕,然後從 [關鍵字] 下拉式清單中選取 [ 重設模型 ]。Click the Speech Input Handler's + button and select Reset Model from the Keyword dropdown.
  13. 按一下 [ + 回應] 下的 [AstronautManager],然後 Hierarchy 將階層中的 [ AstronautManager ] 拖曳到 [無 (物件) ] 欄位中。Click the + under Response, and drag the AstronautManager from the Hierarchy into the None (Object) field.
  14. 現在,按一下 [ 沒有 函式] 下拉式清單,選取 [ AstronautManager],然後按一下 [ ResetModelCommand]。Now, click the No Function dropdown, select AstronautManager, then ResetModelCommand.

如何設定第5章的語音輸入來源和處理常式

建置和部署Build and Deploy

  • 試試看!Try it! 建立應用程式,並將其部署到 HoloLens。Build and deploy the app to the HoloLens.
  • 展開 [展開模型] 以查看展開的太空人模型。Say Expand Model to see the expanded astronaut model.
  • 使用 導覽 來旋轉太空人花色的個別部分。Use Navigation to rotate individual pieces of the astronaut suit.
  • 假設 Move 太空人 ,然後使用 操作 來移動太空人花色的個別部分。Say Move Astronaut and then use Manipulation to move individual pieces of the astronaut suit.
  • 例如, 旋轉太空人 可再次旋轉零件。Say Rotate Astronaut to rotate the pieces again.
  • 假設 重設模型 將太空人傳回其原始表單。Say Reset Model to return the astronaut to its original form.

結束The End

恭喜!Congratulations! 您現在已完成 MR 輸入211:手勢You have now completed MR Input 211: Gesture.

  • 您知道如何偵測和回應手動追蹤、流覽和操作事件。You know how to detect and respond to hand tracking, navigation and manipulation events.
  • 您瞭解導覽和操作手勢之間的差異。You understand the difference between Navigation and Manipulation gestures.
  • 您知道如何變更資料指標,以提供有關何時偵測到手的視覺回應、當手即將遺失,以及物件支援不同互動 (導覽與操作) 之間的視覺化意見反應。You know how to change the cursor to provide visual feedback for when a hand is detected, when a hand is about to be lost, and for when an object supports different interactions (Navigation vs Manipulation).