注視並認可Gaze and commit

注視」和「認可」是一種基本的輸入模型,與使用滑鼠:點 & 按一下 來與電腦互動的方式密切相關。Gaze and commit is a fundamental input model that is closely related to the way we're interacting with our computers using the mouse: Point & click. 在這個頁面中,我們會介紹兩種類型的注視輸入 (前端和眼睛) 以及不同類型的認可動作。On this page, we introduce two types of gaze input (head- and eye-gaze) and different types of commit actions. 您可以使用間接操作將 注視和 commit 視為最遠的輸入模型。Gaze and commit is considered a far input model with indirect manipulation. 它最適合用來與不觸及的全像內容互動。It's best used for interacting with holographic content that is out of reach.

混合現實耳機可以使用使用者的標頭位置和方向來決定其前端方向向量。Mixed reality headsets can use the position and orientation of the user's head to determine their head direction vector. 把眼睛想像成直接從使用者的眼睛之間直接指向的鐳射。Think of gaze as a laser pointing straight ahead from directly between the user's eyes. 這是相當粗略的使用者觀看位置概算。This is a fairly coarse approximation of where the user is looking. 您的應用程式可以將此光線與虛擬或真實世界物件相交,並在該位置繪製游標,讓使用者知道其目標。Your application can intersect this ray with virtual or real-world objects, and draw a cursor at that location to let the user know what they're targeting.

除了頭部眼,某些混合的現實耳機(例如 HoloLens 2)包含產生眼睛向量的眼睛追蹤系統。In addition to head gaze, some mixed reality headsets, such as HoloLens 2, include eye tracking systems that produce an eye-gaze vector. 這會對使用者觀看的位置提供精細的測量。This provides a fine-grained measurement of where the user is looking. 在這兩種情況下,注視都代表使用者意圖的重要信號。In both cases, the gaze represents an important signal for the user's intent. 系統可以解讀和預測使用者的預期動作,更好的使用者滿意度和效能也會改善。The better the system can interpret and predict the user's intended actions, the more user satisfaction and performance improves.

以下幾個範例可讓您以混合現實開發人員的身分來受益于 head 或眼睛:Below are a few examples for how you as a mixed reality developer can benefit from head- or eye-gaze:

  • 您的應用程式可以與場景中的全像眼睛相同,以判斷使用者的注意力 (更精確的眼睛) 。Your app can intersect gaze with the holograms in your scene to determine where the user's attention is (more precise with eye-gaze).
  • 您的應用程式可以根據使用者的注視來進行頻道手勢和控制器按,讓使用者可以順暢地選取、啟用、抓取、滾動或以其他方式與其全息的互動。Your app can channel gestures and controller presses based on the user's gaze, which lets the user seamlessly select, activate, grab, scroll, or otherwise interact with their holograms.
  • 您的應用程式可以讓使用者將表面光線與空間對應網格相交,以將影像放在真實世界的表面上。Your app can let the user place holograms on real-world surfaces by intersecting their gaze ray with the spatial mapping mesh.
  • 您的應用程式可以知道使用者何時不會查看重要物件的方向,這可能會導致您的應用程式提供視覺效果和音訊提示,以轉向該物件。Your app can know when the user isn't looking in the direction of an important object, which can lead your app to give visual and audio cues to turn towards that object.

裝置支援Device support

輸入模型Input model HoloLens (第 1 代)HoloLens (1st gen) HoloLens 2HoloLens 2 沉浸式頭戴裝置Immersive headsets
頭部注視並認可Head-gaze and commit ✔️ 建議使用✔️ Recommended ✔️ 建議使用 (第三個選擇 - 查看其他選項)✔️ Recommended (third choice - See the other options) ➕ 替代選項➕ Alternate option
眼部注視並認可Eye-gaze and commit ❌ 無法使用❌ Not available ✔️ 建議使用 (第三個選擇 - 查看其他選項)✔️ Recommended (third choice - See the other options) ❌ 無法使用❌ Not available

注視Gaze

眼睛或頭部?Eye- or head-gaze?

無論您是否應該使用「眼睛和認可」或「前端和認可」輸入模型,都有幾個考慮。There are several considerations when faced with the question whether you should use the "eye-gaze and commit" or "head-gaze and commit" input model. 如果您正在針對沉浸式耳機或 HoloLens (第一代) 進行開發,則選擇很簡單:標上和認可。If you're developing for an immersive headset or for HoloLens (1st gen), then the choice is simple: Head-gaze and commit. 如果您是針對 HoloLens 2 進行開發,則選擇會變得更困難。If you're developing for HoloLens 2, the choice becomes a little harder. 請務必瞭解每個都有的優點和挑戰。It's important to understand the advantages and challenges that come with each of them. 在下表中,我們編譯了一些很廣泛的專業人員和 con,以對比眼睛的目標。We compiled some broad pro's and con's in the table below to contrast head- vs. eye-gaze targeting. 這並不是完整的,我們建議您在此處深入瞭解混合現實中的眼睛目標:This is far from complete and we suggest learning more about eye-gaze targeting in mixed reality here:

眼睛的目標Eye-gaze targeting 頭部注視定向Head-gaze targeting
快速!Fast! Slower
低工作量 (幾乎不需要任何主體移動) Low effort (barely any body movements necessary) 可以是 fatiguing 可能的不適感 (例如,頸部壓力) Can be fatiguing - Possible discomfort (for example, neck strain)
不需要資料指標,但建議使用微妙的意見反應Doesn't require a cursor, but subtle feedback is recommended 需要顯示資料指標Requires to show a cursor
無平滑的移動–例如,不適合繪圖No smooth eye movements – for example, not good for drawing 更受控制且明確More controlled and explicit
很難進行小型目標 (例如,小按鈕或網頁連結) Difficult for small targets (for example, tiny buttons or weblinks) 可靠!Reliable! 絕佳的回復!Great fallback!
...... ......

無論您是針對您的注視和認可輸入模型使用頭部眼或眼睛,都有不同的設計條件約束集合。Whether you use head-gaze or eye-gaze for your gaze-and-commit input model, each comes with different sets of design constraints. 這些都是在眼睛和 認可認可文章中 分開討論。These are covered separately in the eye-gaze and commit and head-gaze and commit articles.



資料指標Cursor

針對 head,大部分的應用程式應該使用資料 指標 或其他聽覺/視覺指示,讓使用者在他們即將與其互動的方面具有信賴度。For head gaze, most apps should use a cursor or other auditory/visual indication to give the user confidence in what they're about to interact with. 您通常會將這個資料指標放在世界中,其中的前端光線最先與物件(可能是全息圖或真實世界表面)相交。You typically position this cursor in the world where their head gaze ray first intersects an object, which may be a hologram or a real-world surface.

針對眼睛,我們通常不建議您 不要 顯示資料指標,因為這可能會對使用者造成干擾和討厭。For eye gaze, we generally recommend not to show a cursor, as this can quickly become distracting and annoying for the user. 相反地,請特別強調視覺效果目標,或使用不清楚的眼睛游標,以提供使用者即將互動的相關資訊。Instead subtly highlight visual targets or use a faint eye cursor to provide confidence about what the user is about to interact with. 如需詳細資訊,請參閱 HoloLens 2 上 的眼睛輸入的設計指引For more information, please check out our design guidance for eye-based input on HoloLens 2.

顯示注視的視覺效果游標範例An example visual cursor to show gaze
影像:顯示注視的視覺效果游標範例Image: An example visual cursor to show gaze



CommitCommit

談到不同的 方式來看 看某個目標之後,讓我們再談談一下看看 和認可認可 部分。After talking about different ways to gaze at a target, let's talk a bit more about the commit part in gaze and commit. 以物件或 UI 元素為目標之後,使用者可以使用次要輸入進行互動或按一下。After targeting an object or UI element, the user can interact or click on it using a secondary input. 這就是所謂的輸入模型認可步驟。This is known as the commit step of the input model.

支援的認可方法如下:The following commit methods are supported:

  • 點擊手勢 (也就是提高您手上的手,並將您的食指和 thumb 合在一起) Air tap hand gesture (that is, raise your hand in front of you and bring together your index finger and thumb)
  • 說「 select 」或其中一個目標語音命令Say "select" or one of the targeted voice commands
  • HoloLens Clicker上按單一按鈕Press a single button on a HoloLens Clicker
  • 按下 Xbox 遊戲台上的 [A] 按鈕Press the 'A' button on an Xbox gamepad
  • 按下 Xbox 適應性控制器上的 [A] 按鈕Press the 'A' button on an Xbox adaptive controller

注視和點擊手勢Gaze and air tap gesture

空中點選是手部直立的點選手勢。Air tap is a tapping gesture with the hand held upright. 若要使用 [點一下],請將您的索引指向 [就緒] 位置,然後使用您的 thumb 來縮小,然後將您的索引手指重設為 [發行]。To use an air tap, raise your index finger to the ready position, then pinch with your thumb, and raise your index finger back up to release. 在 HoloLens (第1代) 上,按一下是最常見的次要輸入。On HoloLens (1st gen), air tap is the most common secondary input.

觸達就緒位置Finger in the ready position
觸達就緒位置Finger in the ready position

按下手指以點擊或按一下Press finger down to tap or click
按下手指以點擊或按一下Press finger down to tap or click

HoloLens 2 也可以使用 [點一下]。Air tap is also available on HoloLens 2. 它已從原始版本中放寬。It has been relaxed from the original version. 幾乎所有類型的 pinches 現在都可支援,只要手邊的手是直立的,且仍維持。Nearly all types of pinches are now supported as long as the hand is upright and holding still. 這讓使用者更容易學習和使用手勢。This makes it much easier for users to learn and use the gesture. 這個新的空中會透過相同的 API 來取代舊的,讓現有的應用程式在重新編譯以進行 HoloLens 2 之後,會自動有新的行為。This new air tap replaces the old one through the same API, so existing applications will have the new behavior automatically after recompiling for HoloLens 2.



注視和「選取」聲音命令Gaze and "Select" voice command

語音命令是混合現實中的其中一個主要互動方法。Voice commanding is one of the primary interaction methods in mixed reality. 它提供強大的無人參與機制來控制系統。It provides a powerful hands-free mechanism to control the system. 語音互動模型有不同的類型:There are different types of voice interaction models:

  • 使用 click actuation 或 commit 作為次要輸入的泛型 "Select" 命令。The generic "Select" command that uses a click actuation or commit as a secondary input.
  • 物件命令 (例如,「關閉」或「變得更大」 ) 執行並認可至動作做為次要輸入。Object commands (for example, "Close" or "Make it bigger") perform and commit to an action as a secondary input.
  • 全域命令 (例如「移至開始」 ) 不需要目標。Global commands (for example, "Go to start") don't require a target.
  • 對話使用者介面或 Cortana 之類的實體都具有 AI 自然語言功能。Conversation user interfaces or entities like Cortana have an AI natural language capability.
  • 自訂語音命令Custom voice commands

若要深入瞭解詳細資料和可用語音命令的完整清單,以及如何使用這些命令,請參閱我們的 語音命令 指引。To find out more about details and a comprehensive list of available voice commands and how to use them, check out our voice commanding guidance.



注視和 HoloLens ClickerGaze and HoloLens Clicker

HoloLens Clicker 是特別為 HoloLens 建立的第一個週邊設備。The HoloLens Clicker is the first peripheral device built specifically for HoloLens. 它隨附于 HoloLens (第一代) 開發版。It's included with HoloLens (1st gen) Development Edition. HoloLens Clicker 可讓使用者按下最少量的動作,並認可為次要輸入。The HoloLens Clicker lets a user click with minimal hand motion, and commit as a secondary input. HoloLens Clicker 會連接到 HoloLens (第1代) 或使用藍牙低能源 (BTLE) HoloLens 2。The HoloLens Clicker connects to HoloLens (1st gen) or HoloLens 2 using Bluetooth Low Energy (BTLE).

有關將裝置配對的詳細資訊和指示More information and instructions to pair the device

映射: HoloLens ClickerImage: HoloLens Clicker

HoloLens Clicker



注視和 Xbox 無線控制器Gaze and Xbox Wireless Controller

Xbox 無線控制器會使用 [A] 按鈕,以次要輸入的形式執行 click actuation。The Xbox Wireless Controller performs a click actuation as a secondary input by using the 'A' button. 裝置會對應至一組預設的動作,以協助流覽和控制系統。The device is mapped to a default set of actions that help navigate and control the system. 如果您想要自訂控制器,請使用 Xbox 配件應用程式來設定 Xbox 無線控制器。If you want to customize the controller, use the Xbox Accessories application to configure your Xbox Wireless Controller.

如何將 Xbox 控制器與您的電腦配對How to pair an Xbox controller with your PC

影像: Xbox 無線控制器Image: Xbox Wireless Controller

Xbox 無線控制器



注視和 Xbox 調適型控制器Gaze and Xbox Adaptive Controller

Xbox 調適型控制器是專為滿足行動裝置的需求而設計,可協助讓混合現實更容易存取的裝置具有統一的中樞。Designed primarily to meet the needs of gamers with limited mobility, the Xbox Adaptive Controller is a unified hub for devices that helps make mixed reality more accessible.

Xbox 適應性控制器會使用 [A] 按鈕,以次要輸入的形式執行 click actuation。The Xbox Adaptive Controller performs a click actuation as a secondary input by using the 'A' button. 裝置會對應至一組預設的動作,以協助流覽和控制系統。The device is mapped to a default set of actions that help navigate and control the system. 如果您想要自訂控制器,請使用 Xbox 配件應用程式來設定 Xbox 調適型控制器。If you want to customize the controller, use the Xbox Accessories application to configure your Xbox Adaptive Controller.

Xbox Adaptive ControllerXbox Adaptive Controller
Xbox Adaptive ControllerXbox Adaptive Controller

連接外部裝置(例如交換器、按鈕、掛接和操作杆)來建立獨特的自訂控制器體驗。Connect external devices such as switches, buttons, mounts, and joysticks to create a custom controller experience that is uniquely yours. 按鈕、操縱杆和觸發程式輸入都是由透過 3.5-mm 插座和 USB 埠連接的輔助裝置所控制。Button, thumbstick, and trigger inputs are controlled with assistive devices connected through 3.5-mm jacks and USB ports.

Xbox Adaptive Controller 連接埠Xbox Adaptive Controller ports
Xbox Adaptive Controller 連接埠Xbox Adaptive Controller ports

裝置配對指示Instructions to pair the device

在 Xbox 網站上可取得更多資訊More info available on the Xbox site



複合手勢Composite gestures

空中點選Air tap

點擊手勢 (,下的其他手勢) 只會回應特定的點。The air tap gesture (and the other gestures below) reacts only to a specific tap. 若要偵測其他點擊(例如功能表或理解),您的應用程式必須直接使用上述兩個主要元件手勢區段中所述的較低層級互動。To detect other taps, such as Menu or Grasp, your application must directly use the lower-level interactions described in the two key component gestures section above.

點選並按住Tap and hold

按住只是維持空中點選的手指朝下位置。Hold is simply maintaining the downward finger position of the air tap. 結合了「點擊和拖曳」互動與 arm 移動(例如挑選物件),而不是啟用或 mousedown 次要互動(例如顯示操作功能表)。The combination of air tap and hold allows for various more complex "click and drag" interactions when combined with arm movement such as picking up an object instead of activating it or mousedown secondary interactions such as showing a context menu. 不過,在設計此筆勢時應謹慎使用,因為使用者可能很容易在任何延長的手勢中放寬其手做法。Caution should be used when designing for this gesture however, as users can be prone to relaxing their hand postures during any extended gesture.

操作Manipulation

當您想1:1 要讓全像全像移動、調整大小或旋轉全像移動、調整大小或旋轉影像時,可以使用操作手勢來移動、調整大小或旋轉。Manipulation gestures can be used to move, resize, or rotate a hologram when you want the hologram to react 1:1 to the user's hand movements. 這類 1:1 移動的用途之一是要讓使用者可以實際繪圖。One use for such 1:1 movements is to let the user draw or paint in the world. 操作手勢的初始定向應經由注視或指向來完成。The initial targeting for a manipulation gesture should be done by gaze or pointing. 當點一下和按住開始之後,任何物件操作都會由手動移動處理,如此可讓使用者在操作時進行查詢。Once the tap and hold starts, any object manipulation is handled by hand movements, which frees the user to look around while they manipulate.

瀏覽手勢的運作方式如同虛擬搖桿,可用來瀏覽 UI 小工具,例如放射狀功能表。Navigation gestures operate like a virtual joystick, and can be used to navigate UI widgets, such as radial menus. 您可點選並按住來啟動手勢,然後在標準化 3D Cube (在初次按壓時置中) 中移動您的手。You tap and hold to start the gesture and then move your hand within a normalized 3D cube, centered around the initial press. 您可以將來自 X、Y 或 Z 軸的手移至從-1 到1的值,並將0視為起始點。You can move your hand along the X, Y, or Z axis from a value of -1 to 1, with 0 being the starting point. 瀏覽可用來建置以速度為基礎的連續捲動或縮放手勢,類似於藉由按一下滑鼠中間按鈕,然後上下移動滑鼠來捲動 2D UI。Navigation can be used to build velocity-based continuous scrolling or zooming gestures, similar to scrolling a 2D UI by clicking the middle mouse button and then moving the mouse up and down.

使用 rails 的導覽指的是在特定軸中辨識移動的能力,直到到達該軸上的特定臨界值為止。Navigation with rails refers to the ability of recognizing movements in certain axis until a certain threshold is reached on that axis. 當開發人員在應用程式中啟用多個軸的移動時(例如,如果應用程式設定為辨識跨 X、Y 軸的導覽手勢,但同時指定 X 軸與 rails),這項功能就很有用。This is only useful when movement in more than one axis is enabled in an application by the developer, such as if an application is configured to recognize navigation gestures across X, Y axis but also specified X axis with rails. 在這種情況下,系統會在 X 軸上辨識手間的移動,只要它們留在 X 軸的虛數滑軌 (指南) ,如果在 Y 軸上也會發生手移動。In this case, the system will recognize hand movements across X axis as long as they remain within an imaginary rails (guide) on the X axis, if hand movement also occurs on the Y axis.

在 2D 應用程式內,使用者可以使用垂直瀏覽手勢在應用程式內捲動、縮放或拖曳。Within 2D apps, users can use vertical navigation gestures to scroll, zoom, or drag inside the app. 這會在應用程式中插入虛擬手指觸控,以模擬同類型的觸控手勢。This injects virtual finger touches to the app to simulate touch gestures of the same type. 使用者可以在應用程式上方的工具列上切換,藉由選取按鈕或說「<滾動/拖曳/縮放> 工具」,來選取要進行的動作。Users can select which of these actions take place by toggling between the tools on the bar above the application, either by selecting the button or saying '<Scroll/Drag/Zoom> Tool'.

複合手勢的詳細資訊More info on composite gestures

手勢辨識器Gesture recognizers

使用手勢辨識的其中一個優點是,您只能針對目前目標的全像全像影像所能接受的手勢設定手勢辨識器。One benefit of using gesture recognition is that you can configure a gesture recognizer only for the gestures the currently targeted hologram can accept. 平臺只會視需要去除混淆,以區別這些特定支援的手勢。The platform only does disambiguation as necessary to distinguish those particular supported gestures. 如此一來,只支援「敲擊」的全像是在按下和放開之間有任何時間長度,而同時支援點一下和按住的全像投影可以在按住時間閾值之後升階。In this way, a hologram that just supports air tap can accept any length of time between press and release, while a hologram that supports both tap and hold can promote the tap to a hold after the hold time threshold.

手部辨識Hand recognition

HoloLens 可藉由追蹤裝置可見的任一手或雙手位置來辨識手勢。HoloLens recognizes hand gestures by tracking the position of either or both hands that are visible to the device. 當手部處於就緒狀態 (手背朝向您且食指朝上) 或按下狀態 (手背朝向您且食指朝下) 時,HoloLens 就會看見手部。HoloLens sees hands when they are in either the ready state (back of the hand facing you with index finger up) or the pressed state (back of the hand facing you with the index finger down). 當手中有其他人時,HoloLens 會忽略它們。When hands are in other poses, HoloLens ignores them. 針對 HoloLens 偵測到的每個手,您可以存取其位置,而不需要方向和按下的狀態。For each hand that HoloLens detects, you can access its position without orientation and its pressed state. 當手部接近手勢框架邊緣時,您也會取得方向向量,而您可對使用者顯示該向量,讓他們知道如何移動其手部以回到 HoloLens 可看見手部的位置。As the hand nears the edge of the gesture frame, you're also provided with a direction vector, which you can show to the user so they know how to move their hand to get it back where HoloLens can see it.

手勢框架Gesture frame

針對 HoloLens 上的手勢,手必須位於手勢框架內、手勢檢測攝影機可適當地查看的範圍、從鼻子到 waist,以及肩膀之間。For gestures on HoloLens, the hand must be within a gesture frame, in a range that the gesture-sensing cameras can see appropriately, from nose to waist and between the shoulders. 使用者必須在此辨識區域上進行訓練,以獲得動作的成功,並讓他們自己的舒適。Users need to be trained on this area of recognition both for success of action and for their own comfort. 許多使用者一開始會假設手勢框架必須在其觀看的情況下,透過 HoloLens,並讓其 arm uncomfortably 互動。Many users will initially assume that the gesture frame must be within their view through HoloLens, and hold up their arms uncomfortably to interact. 使用 HoloLens Clicker 時,不一定要將手放在手勢框架內。When using the HoloLens Clicker, it's not necessary for hands to be within the gesture frame.

尤其是針對連續手勢,當移動全像投影片的物件時,使用者可能會有一些風險將手移至手勢框架之外,而不會發生預期的結果。For continuous gestures in particular, there's some risk of users moving their hands outside of the gesture frame while in mid-gesture when moving a holographic object, for example, and losing their intended outcome.

您應該考量下列三個事項:There are three things that you should consider:

  • 手勢框架存在和大約界限的使用者教育。User education on the gesture frame's existence and approximate boundaries. 這是在 HoloLens 安裝期間教授。This is taught during HoloLens setup.

  • 當使用者的手勢接近或中斷應用程式內的手勢框架界限時,通知使用者,表示遺失的手勢會導致不想要的結果。Notifying users when their gestures are nearing or breaking the gesture frame boundaries within an application to the degree that a lost gesture leads to undesired outcomes. 研究已展示這類通知系統的主要品質。Research has shown the key qualities of such a notification system. HoloLens shell 會在中央游標上提供此類型通知(視覺效果)的良好範例,指出界限跨越的方向。The HoloLens shell provides a good example of this type of notification--visual, on the central cursor, indicating the direction in which boundary crossing is taking place.

  • 您應該將突破手勢框架界限的後果最小化。Consequences of breaking the gesture frame boundaries should be minimized. 一般來說,這表示手勢的結果應該在界限停止,而不是反轉。In general, this means that the outcome of a gesture should be stopped at the boundary, and not reversed. 比方說,如果使用者要在房間內移動一些全像的物件,則當軌跡框架被入侵時,移動應該會停止,而且不會傳回至起始點。For example, if a user is moving some holographic object across a room, the movement should stop when the gesture frame is breached, and not returned to the starting point. 使用者可能會遇到一些挫折,但可能更快速地瞭解界限,而不必每次重新開機其完整的預期動作。The user might experience some frustration, but might more quickly understand the boundaries, and not have to restart their full intended actions each time.

另請參閱See also