本能互動簡介Introducing instinctual interactions

遠距手部操作

簡單、直覺式互動的原理貫穿整個混合實境 (MR) 平台。The philosophy of simple, instinctual interactions is interwoven throughout the mixed reality (MR) platform. 我們採取了三個步驟,以確保應用程式設計人員和開發人員可為其客戶提供簡單且直覺式的互動。We've taken three steps to ensure that application designers and developers can provide their customers with easy and intuitive interactions.

首先,我們已確保將感應器和輸入技術結合成多樣式互動模型。First, we've made sure our sensors and input technologies combine into multimodal interaction models. 這些互動模型包括手部和眼睛追蹤以及自然語言輸入。These interaction models include hand and eye tracking along with natural language input. 根據我們的研究,多樣式架構內的設計和開發 (而不是根據個別輸入) 是創造直覺式體驗的關鍵。Based on our research, designing and developing within a multimodal framework (and not based on individual inputs) is the key to creating instinctual experiences.

其次,我們了解許多開發人員都以多個 HoloLens 裝置為目標,例如 HoloLens 2 和 HoloLens (第 1 代) 或 HoloLens 和 VR。Second, we recognize that many developers target multiple HoloLens devices, such as HoloLens 2 and HoloLens (1st gen) or HoloLens and VR. 因此,我們將互動模型設計為可跨裝置運作,即使每個裝置上的輸入技術有所不同。So we've designed our interaction models to work across devices, even if the input technology varies on each device. 例如,在採用 6DOF 控制器以及 HoloLens 2 的 Windows 沉浸式頭戴裝置上的遠距互動上均使用相同的能供性和模式。For example, far interaction on a Windows Immersive headset with a 6DoF controller and HoloLens 2 both use identical affordances and patterns. 這可讓您輕鬆地進行跨裝置應用程式開發,並為使用者互動提供自然的感覺。This makes it easy for cross-device application development and provides a natural feel to user interactions.

我們承認 MR 中有數千種有效、吸引人和神奇的可能互動,但我們也發現在應用程式中刻意運用單一互動模型,是確保使用者順利擁有絕佳體驗的最佳方法。While we recognize that there are thousands of effective, engaging, and magical interactions possible in MR, we've found that intentionally employing a single interaction model in an application is the best way to ensure users are successful and have a great experience. 為此,我們在此互動指引中納入了三件事:To that end, we've included three things in this interaction guidance:

  • 特定指引,以三個主要互動模型以及每個模型所需的元件和模式為主。Specific guidance around the three primary interaction models and the components and patterns required for each.
  • 補充指引,說明我們的平台具備的其他優勢。Supplemental guidance about other benefits that our platform provides.
  • 一般指引,可針對您的開發案例協助您選取適當的互動模型。General guidance to help select the appropriate interaction model for your development scenario.

多樣式互動模型Multimodal interaction models

根據我們的研究以及客戶的意見反應,我們發現有三個主要互動模型適合大部分的混合實境體驗。Based on our research and feedback from customers, we've discovered that three primary interaction models suit most mixed reality experiences. 在許多方面,互動模型是使用者如何完成流程的心智模型。In many ways, the interaction model is the user's mental model for how to complete a workflow. 上述每種互動模型都已針對一套客戶需求進行最佳化,正確使用時每種都很方便、強大、實用。Each of these interaction models is optimized for a set of customer needs and is convenient, powerful, and usable when used correctly.

下圖是簡化的概觀。The chart below is a simplified overview. 在底下的頁面中,使用每個互動模型的詳細資訊都有連結,其中包含影像和程式碼範例。Detailed information for using each interaction model is linked in the pages below with images and code samples.


型號Model 範例案例Example scenarios 適合Fit 硬體Hardware
手部和運動控制器Hands and motion controllers 3D 空間體驗,例如空間版面配置與設計、內容操作或模擬。3D spatial experiences, such as spatial layout and design, content manipulation, or simulation. 很適用於新使用者,可搭配語音、眼球追蹤或頭部注視使用。Great for new users coupled with voice, eye tracking or head gaze. 學習曲線很低。Low learning curve. 手部追蹤和 6DoF 控制器之間的一致 UX。Consistent UX across hand tracking and 6DoF controllers. HoloLens 2HoloLens 2
沉浸式頭戴裝置Immersive headsets
免持式Hands-free 使用者雙手沒空 (例如在職培訓和維護) 時的情境體驗。Contextual experiences where a user's hands are occupied, such as on-the-job learning and maintenance. 需要學習一下。Some learning required. 如果無法使用手部,裝置可搭配語音和自然語言使用。If hands are unavailable, the device pairs well with voice and natural language. HoloLens 2HoloLens 2
HoloLens (第 1 代)HoloLens (1st gen)
沉浸式頭戴裝置Immersive headsets
目光和行動Gaze and commit 點擊體驗,例如 3D 簡報、示範。Click-through experiences, for example, 3D presentations, demos. 需要 HMD 訓練,但不是在行動裝置上。Requires training on HMDs but not on mobile. 最適合用於可存取的控制器。Best for accessible controllers. 最適合用於 HoloLens (第 1 代)。Best for HoloLens (1st gen). HoloLens 2HoloLens 2
HoloLens (第 1 代)HoloLens (1st gen)
沉浸式頭戴裝置Immersive headsets
行動 ARMobile AR

為了避免使用者互動體驗中發生落差,從頭到尾遵循單一模型的指引將是最佳方式。To avoid gaps in the user interaction experience, it's best to follow the guidance for a single model from beginning to end.

下列各節會逐步說明選取及實作其中一個互動模型的步驟。The sections below walk through the steps for selecting and implementing one of these interaction models.

在此頁面結束前,您會了解我們對於下列各項的指導:By the end of this page, you'll understand our guidance on:

  • 為您的客戶選擇互動模型Choosing an interaction model for your customer
  • 實作互動模型Implementing the interaction model
  • 互動模型之間的轉換Transitioning between interaction models
  • 設計後續步驟Design next steps

為您的客戶選擇互動模型Choose an interaction model for your customer

開發人員和創作者通常都已考量其客戶可使用的互動類型。Typically, developers and creators have thought through the types of interactions that their customers can have. 為了鼓勵以客戶為主的設計方式,我們建議您遵循下列指引來選取最適合客戶的互動模型。To encourage a customer-focused approach to design, we recommend the following guidance for selecting the interaction model that's optimized for your customer.

為什麼要遵循本指引?Why follow this guidance?

  • 我們均針對客觀和主觀準則對互動模型進行測試,例如實際和認知投入、直覺性和學習力。We test our interaction models for objective and subjective criteria, including physical and cognitive effort, intuitiveness, and learnability.
  • 由於互動不同,互動模型之間的視訊/音訊能供性及物件行為也可能不同。Because interactions differ, visual/audio affordances and object behavior might differ between interaction models.
  • 結合多個互動模型的組件,會引起競用能供性 (例如同時手部光線和頭部注視游標) 的風險。Combining parts of multiple interaction models creates the risk of competing affordances, such as simultaneous hand rays and a head-gaze cursor. 這可能會造成使用者的負荷和困惑。This can overwhelm and confuse users.

以下是如何針對每個互動模型將能供性和行為最佳化的一些範例。Here are some examples of how affordances and behaviors are optimized for each interaction model. 我們常會看到新使用者有類似的問題,例如「如何得知系統是否能運作」 、「如何得知我可以做什麼」 、「如何得知它是否了解我剛做了什麼?」 We often see new users have similar questions, such as "how do I know the system is working", "how do I know what I can do", and "how do I know if it understood what I just did?"


型號Model 如何得知它是否能運作?How do I know it's working? 如何得知我可以做什麼?How do I know what I can do? 如何得知我剛做了什麼?How do I know what I just did?
手部和運動控制器Hands and motion controllers 我看到手狀網格、指尖能供性或手部/控制器光線。I see a hand mesh, a fingertip affordance, or hand/controller rays. 我看到當我的手部靠近物件時會出現可握住的把手或週框方塊。I see grabbable handles, or a bounding box appears when my hand is near an object. 我會在握住和放開時聽到聽得見的音調並看到動畫。I hear audible tones and see animations on grab and release.
頭部目光和行動Head-gaze and commit 我在視野中央看到一個游標。I see a cursor in the center of my field of view. 游標會在其掃過某些物件時變更狀態。The cursor changes state when it's over certain objects. 當我採取動作時,我會看到/聽到看得見和聽得見的確認。I see/hear visual and audible confirmations when I take action.
免持式 (頭部注視並佇留)Hands-free (Head-gaze and dwell) 我在視野中央看到一個游標。I see a cursor in the center of my field of view. 當我佇留在可互動的物件上時,我會看到進度列指示器。I see a progress indicator when I dwell on an interactable object. 當我採取動作時,我會看到/聽到看得見和聽得見的確認。I see/hear visual and audible confirmations when I take action.
免持式 (語音命令)Hands-free (Voice commanding) 我看到接聽指示器以及可顯示系統所聽內容的字幕。I see a listening indicator and captions that show what the system heard. 我取得語音提示。I get voice prompts and hints. 當我說:「我可以說什麼?」When I say: "What can I say?" 我看到回饋。I see feedback. 當我發出命令時,我會看到/聽到看得見和聽得見的確認,或在必要時取得去除混淆 UX。I see/hear visual and audible confirmations when I give a command, or get disambiguation UX when needed.

以下是我們發現有助於小組選取互動模型的問題:Below are questions that we've found help teams select an interaction model:

  1. Q:我的使用者想要觸控全像投影和執行精準全像攝影操作嗎?Q: Do my users want to touch holograms and perform precision holographic manipulations?

    答:若是如此,請查看「雙手和運動控制器」互動模型,以進行精準定向和操作。A: If so, check out the Hands and motion controllers interaction model for precision targeting and manipulation.

  2. Q:我的使用者需要為了實際操作而騰出雙手嗎?Q: Do my users need to keep their hands free for real-world tasks?

    答:若是如此,請查看免持式互動模型,此模型可透過注視和語音型互動提供絕佳的免持式體驗。A: If so, take a look at the Hands-free interaction model, which provides a great hands-free experience through gaze and voice-based interactions.

  3. Q:我的使用者是否有時間了解我的 MR 應用程式互動,或者他們需要最低學習曲線的互動?Q: Do my users have time to learn interactions for my MR application or do they need the interactions with the lowest learning curve possible?

    答:針對最低學習曲線和最直覺的互動,我們建議使用「雙手和運動控制器」模型,只要使用者能夠使用其雙手進行互動即可。A: For the lowest learning curve and most intuitive interactions, we recommend the Hands and motion controllers model, as long as users can use their hands for interaction.

  4. Q:我的使用者是否要使用運動控制器來指向及操作?Q: Do my users use motion controllers for pointing and manipulation?

    答:「雙手和運動控制器」模型包含優質運動控制器體驗的所有指引。A: The Hands and motion controllers model includes all guidance for a great experience with motion controllers.

  5. Q:我的使用者是否使用協助工具控制器或通用藍牙控制器,例如 Clicker?Q: Do my users use an accessibility controller or a common Bluetooth controller, such as a clicker?

    答:我們建議使用所有非追蹤式控制器都適用的「頭部注視並認可」模型。A: We recommend the Head-gaze and commit model for all non-tracked controllers. 其設計訴求是可讓使用者使用簡單的「定向和行動」經歷整個體驗。It's designed to allow a user to traverse an entire experience with a simple "target and commit" mechanism.

  6. Q:相對於瀏覽 UI 控制項的密集版面配置,我的使用者只是經由「點選」來進行體驗 (例如在類似 3D 投影片的環境中) 嗎?Q: Do my users only progress through an experience by "clicking through" (for example in a 3D slideshow-like environment), as opposed to navigating dense layouts of UI controls?

    答:如果使用者不需要控制大量 UI,則「頭部注視並認可」會提供學得會的選項,使用者就不必擔心如何定向。A: If users don't need to control a lot of UI, Head-gaze and commit offers a learnable option where users don't have to worry about targeting.

  7. Q:我的使用者是否同時使用 HoloLens (第 1 代) 和 HoloLens 2/Windows Mixed Reality 沉浸式頭戴裝置 (VR)?Q: Do my users use both HoloLens (1st gen) and HoloLens 2/Windows Mixed Reality immersive headsets (VR)?

    答:由於「頭部注視並認可」是 HoloLens (第 1 代) 適用的互動模型,我們建議支援 HoloLens (第 1 代) 的創作者將「頭部注視並認可」用於使用者在 HoloLens (第 1 代) 頭戴式裝置上所將體驗的任何功能或模式。A: Since Head-gaze and commit are the interaction model for HoloLens (1st gen), we recommend that creators who support HoloLens (1st gen) use Head-gaze and commit for any features or modes that users will experience on a HoloLens (1st gen) headset. 如需了解如何為多代 HoloLens 創造絕佳體驗的詳細資訊,請參閱「轉換互動模型」一節。See the next section on Transitioning interaction models for details on making a great experience for multiple HoloLens generations.

  8. Q:對於移動 (涵蓋大型空間或在空間之間移動) 的使用者,與傾向在單一空間工作的使用者而言,有何差別?Q: What about users who are mobile, covering a large space or moving between spaces, versus users who tend to work in a single space?

    答:任何互動模型都適用於這些使用者。A: Any of the interaction models will work for these users.

注意

即將推出應用程式設計專屬的詳細指引。More guidance specific to app design coming soon.

轉換互動模型Transitioning interaction models

在某些使用案例中,也可能需要使用多個互動模型。There are also use cases that might require using more than one interaction model. 例如,您應用程式的「建立流程」使用「雙手和運動控制器」互動模型,但您想要讓現場技術人員使用語音輸入模式。For example, your application's creation flow uses the "hands and motion controllers" interaction model, but you want to employ a hands-free mode for field technicians. 如果您的經驗的確需要多種互動模型,使用者可能會在轉換模型時遇到困難 (尤其是不熟悉混合實境的使用者)。If your experience does require multiple interaction models, users might have difficulty transitioning from one model to another, especially when they're new to mixed reality.

注意

我們會繼續撰寫更多適用於開發人員和設計人員的指引,讓他們了解使用多個 MR 互動模型的方式、時機和原因。We're constantly working on more guidance that will be available to developers and designers, informing them about the how, when, and why for using multiple MR interaction models.

另請參閱See also