手勢指導Hand coach

範例:手形指導

當系統未偵測到使用者的手時,手上的教練會觸發3D 模型化的手。Hand coach triggers 3D modeled hands when the system doesn't detect the user’s hands. 這項功能是一個「教學」元件,可在未教授手勢時協助引導使用者。This feature is a “teaching” component that helps guide the user when the gesture hasn't been taught. 如果使用者尚未針對某個句號完成指定的手勢,則手會有延遲的迴圈。If users haven't done the specified gesture for a period, the hands will loop with a delay. 手形指導可用來代表按下按鈕或挑選全像影像。The Hand coach could be used to represent pressing a button or picking up a hologram.

提供的手勢Hand coach provided

目前的互動模型代表各式各樣的手勢控制項,例如滾動、遠選和近端點擊。The current interaction model represents a wide variety of gesture controls such as scrolling, far select, and near tap. 以下是 MRTK中所提供之現有手勢的完整清單:Below is a full list of existing hand gestures provided in MRTK:

接近 Select 的範例Example of Near Select
接近選擇使用的範例顯示如何選取按鈕或關閉互動物件Example of Near Select - Used show how to select buttons or close interactable objects

點擊點範例Example of Air Tap
用來顯示如何選取遠距離的物件的點擊範例Example of Air Tap - Used to show how to select objects that are far away

移動的範例Example of Move
移動空間中物件的範例,用來示範如何在空間中移動全息圖Example of Moving an object in space-Used to show how to move a hologram in space

旋轉範例Example of Rotate
示範如何旋轉全像全像是影像或物件的 Rotate-Used 範例Example of Rotate-Used to show how to rotate holograms or objects

調整規模範例Example of Scale
相應縮小範例,用來示範如何操作大容量Example of Scale- Used to show how to manipulate holograms to be bigger or smaller

手掌的範例Example of Palm Up
用來顯示快顯功能表的上方範例-建議使用Example of Palm up – Suggested use, to bring up hand menus

HandFlip 範例Example of HandFlip
手勢範例-另一種顯示功能表的方法Exmaple of Hand Flip – Another way to bring up Hand Menus

滾動的範例Example of Scroll
滾動範例-用於滾動清單或長檔Example of Scroll – Used for scrolling a list or a long document

設計概念Design concepts

針對 Hololens2,我們根據 instinctual 和自然手勢來設計出互動。For Hololens2, we designed out hand interactions based on instinctual and natural hand gestures. 我們相信大部分的使用者都可以直覺化,因此我們不會建立專用的手勢學習時間。We believe these to be intuitive to most users, so we didn't create dedicated gesture learning moments. 相反地,我們建立了一項指導,以協助使用者瞭解這些手勢是否停滯,或不熟悉全像全息圖互動。Instead, we created the hand coach to help users learn about these gestures if they get stuck or are unfamiliar with hologram interactions. 在沒有學習的時候,我們認為顯示使用者如何藉由示範來執行動作,這是最佳選項。Without a learning moment, we felt that showing users how to perform an action by demonstrating it would be the best option. 我們發現使用者能夠找出手勢,但需要一些指引。We found that users were able to figure out the gesture but needed a little guidance. 如果我們偵測到使用者未與某個物件進行互動,則會觸發適當的手勢和手指放置。If we detect a user doesn't interact with an object for a period, a Hand coach would be triggered demonstrating the correct hand and finger placement.

直觀Intuitive

製作手動畫時,應該很明顯,不會造成任何混淆。When animating hands, it should be obvious and shouldn't cause any confusion. 手動畫是您嘗試提示使用者瞭解的手勢標記法。The hand animation is a representation of the gesture you're trying to prompt the user to understand.

例如,如果您希望使用者按下按鈕,就會觸發按下按鈕的動作。For example, if you wish a user to press a button, a hand pressing a button would be triggered.

範例:近乎點擊的手勢Example: Hand coach Near Tap
示範近乎點擊 Gem 的手勢Hand Coach demonstrating Near Tapping a Gem

手擴展Hand scale

我們使用 UI 功能表來測試各種手上的大小,並覺得如果手上的大小為真,則會提供阻感受。We tested various hand sizes with the UI menus and felt that if the hands were true to size, it gave a menacing feeling. 如果太小,就很難查看並瞭解手勢。If they were too small, it was hard to see and understand the gesture.

語氣和手Voice over and hands

不要預期使用者可透過語音聆聽一組指示,並透過手動指導來觀看不同的指示。Don’t expect users can listen to one set of instructions via voice over and watch different instructions via Hand coach. 排序您的指示,以協助使用者專注于競爭,以減少感應式多載。Sequence your instructions to help users focus versus compete for their attention to reduce sensory overload.

我可以建立自己的嗎?Can I create my own?

可以!Yes! 建議您為您的遊戲建立自己的獨特手勢,並貢獻回該社區!We encourage you to create your own unique gesture for your game and contribute back to the community! 我們提供 Rigged 手的 Maya 檔,可用於您的應用程式,您可以在這裡下載: 下載 HandCoach_MRTK.zip We've provided a Maya file of a Rigged hand that can be used for your app, which can be downloaded here: Download HandCoach_MRTK.zip

Maya 中的動畫指標範例Example of Animated Hands in Maya
在 Maya 中刺探方塊的動畫手勢範例Example of animated Hand Poking a box in Maya

建議的 authoring toolRecommended authoring tool

在3D 演出者之間,許多選擇使用 Autodesk 的 Maya,可以使用 HoloLens 來轉換資產的建立方式。Among 3D artists, many choose to use Autodesk’s Maya, which can use HoloLens to transform the way assets are created. 提供的手入檔案是 Maya 二進位檔案,因此建議使用 Maya 來建立動畫和匯出手。The hands file provided is a Maya Binary File, so it's recommended to use Maya to animate and export the hands. 如果您想要使用其他3D 程式,以下是 。FBX 下載 HandCoachMRTK_FBX.zip 以建立您自己的控制器設定。If you prefer to use another 3D program, here's a .FBX: Download HandCoachMRTK_FBX.zip to create your own controller setup.

如果使用提供的可下載 maya 檔,建議您將 unity 中的手縮小為0.6。If using the downloadable maya Hand File provided, it's suggested to scale down the hands in unity to 0.6.

範例: Maya 中的手教練裝備Example: Hand coach rig in Maya
Rigged 手Rigged Hands

技術規格Technical Specs

  • Maya Ascii 格式有兩個右手檔案可用Two handed File is available in Maya Ascii format
  • 右邊和左方都提供 Maya 二進位格式Right and Left Hand is available in Maya Binary format
  • 將 Maya 檔案設定為 24 FPSSet your Maya file to 24 FPS
  • 在檔案中,有左方和右手邊,可用於兩個右手或單次手勢。Within the file, there's a left and right hand, which can be used for two handed or single-handed gestures. 預設只會顯示右手邊。The right hand will only be visible by default.
  • 建議將大約10個框架的緩衝區留在開頭和結尾,以淡化It's suggested to leave a buffer of about 10 frames at the beginning and end for fades
  • 如果使用指定的目標建立物件的動畫,它的最佳作法是建立預設方塊或 Null 的動畫。If animating an object with a specified target, its best practice to animate to a Default box or Null.
  • 如果手上有一個實體物件(例如方塊)的動畫,它的最佳作法是不要在 Maya 中製作轉譯的動畫,但等候在 Unity 或程式碼中建立動畫。If the hand is animating a physical object such as a box, its best practice to not animate the translation in Maya but wait to animate it in Unity or in Code.
  • 可見的動畫應為1.5 秒,才能傳達有意義的資訊Visible Animation should be 1.5 secs for any meaningful information to be conveyed
  • 當您感到滿意動畫時:When you feel satisfied with your animation:

從 Maya 匯出Exporting from Maya

當您滿意動畫之後After you're satisfied with your animation

  • 選取所有接點:選取 > 階層Select all joints: Select > Hierarchy

    範例:功能表中的階層

  • 製作您的動畫:切換至動畫 > 按鍵 > 製作動畫Bake out your animation: Switch to Animation > Key > Bake Animation

    範例:製作動畫功能表位置

  • 刪除控制器 Rig: Outliner > MainR_Grp 或 MainL_GrpDelete the Controller Rig: Outliner > MainR_Grp or MainL_Grp

    範例:控制器 Rig 功能表位置

  • 匯出為 FBX:選取 .JNT + 網格:檔案 > 匯出選取範圍 (選項方塊) > 匯出選取專案Export as FBX: Select JNT + Mesh: File > Export Selection (option box) > Export Selection

    範例:匯出選取專案功能表位置

    範例:功能表位置

    範例:匯出選項功能表位置

當匯出為 FBX 並帶入 Unity 時,請將手向下調整為0.6。When exporting as an FBX and brought into Unity, scale the hands down to 0.6. 我們發現這是實際顯示的完美平衡。We found that this was perfect balance for displaying the hands.

範例: Unity 設定Example: Unity Settings
在 MRTK 中找到 HandCoach_R 預製專案的 Unity 設定Unity Settings for HandCoach_R prefab found in MRTK

實際執行您的 Unity 專案Implementing Hands into your Unity project

最佳作法Best practices

  • 建議您將 unity 中的手縮小至0。6It's suggested to scale down the hands in unity to 0.6

  • 雙手應該播放兩次,如果未完成,則會持續迴圈直到手勢完成為止。Hands should be played twice and if not completed then continuously looped until gesture is completed. 雙手應迴圈兩次,以確保使用者有時間註冊並查看手勢。The hands should be looped twice to ensure the user had time to register and see the gesture. 手應該在迴圈之間淡入和縮小。The hands should fade in and out between loops.

  • 如果 HL2 攝影機看到使用者的手,但使用者未進行需要的互動,則手會在10秒後出現。If user’s hands are visible by HL2 cameras but users aren't doing the interaction needed of them the hands will appear after 10 seconds.

  • 如果 HL2 攝影機看不到使用者的手,就會在5秒後出現手。If user’s hands are NOT visible by HL2 cameras, the hands will appear after 5 seconds.

  • 如果在動畫中間的 HL2 攝影機會以明顯的方式追蹤使用者的手,動畫將會完成並淡出。If user’s hands are visibly tracked by HL2 cameras in the middle of the animation, the animation will complete and fade out.

  • 如果您要包含語音,建議您將其對應至手的手勢。If you're including voice over, we suggest that it corresponds to the gesture of the hand.

  • 如果您至少教授過一次,只要偵測到使用者停滯,請只重複手勢。If you've taught the hands at least once, only repeat the gesture if it's detected that the user is stuck.

  • 如果特定的手指/手位置很重要,請確保使用者可以清楚地在動畫中看到這些細微差異。If specific finger/hand positions are critical, ensure users can clearly see these nuances in the animation. 試著 angling 手,讓最重要的部分清楚可見。Try angling the hands so the most important parts are clearly visible.

  • 如果您注意到手上的失真,您需要移至 Unity 的品質設定來提高骨骼數目。If you notice distortion on the hands, you need to go to Unity's Quality settings increase the number of bones. 移至 Unity 的 [編輯] > 專案設定 > 品質 > 其他 > Blend 加權。Go to Unity's Edit > Project Settings > Quality > Other > Blend Weights. 確定已選取「4個骨骼」以查看平滑接點。Make sure "4 bones" are selected to see Smooth Joints.

    範例:專案設定視窗

避免事項What to avoid

  • 調整手太大Scaling the Hands too large
  • 把手放在使用者附近placing the Hands too close to the user
  • 手中應只教授一次。Hands should only be taught once. 透過教學可能會造成混淆和 messinessOver teaching can cause confusion and messiness
  • 將它帶入 Unity,請在這裡下載最新的 MRTK: https://github.com/microsoft/MixedRealityToolkit-UnityBringing it into Unity, download the latest MRTK here: https://github.com/microsoft/MixedRealityToolkit-Unity
    • 材質: Teaching_Hand2Material: Teaching_Hand2
    • 腳本:請參閱 MRTK 指導方針以取得MRTK的指導方針Scripts: Refer to MRTK guidelines for MRTK hand coach
    • 每個專案的設定Per- project setting
      • 設定為 UWP 的場景:您可以在設定 Unity 專案 中找到 Windows Mixed Reality 的指示。Scene set to UWP: Instruction can be found on the Configure Unity Project for Windows Mixed Reality

另請參閱See also