什麼是混合實境?What is Mixed Reality?

在 HoloLens 2 上手部指向和行動

混合實境是實體和數位世界的融合,消除了人類、電腦和環境彼此互動的藩籬。Mixed Reality is a blend of physical and digital worlds, unlocking the links between human, computer, and environment interaction. 這種新型的實境功能奠基於電腦視覺、圖形處理能力、顯示器技術和輸入系統的強化。This new reality is based on advancements in computer vision, graphical processing power, display technology, and input systems. 不過,「混合實境」一詞最初是由 Paul Milgram 和 Fumio Kishino 在 1994 年的論文《混合實境視覺顯示器分類法 (A Taxonomy of Mixed Reality Visual Displays)》中提出。However, the term Mixed Reality was introduced in a 1994 paper by Paul Milgram and Fumio Kishino, "A Taxonomy of Mixed Reality Visual Displays." 他們的論文探討了 Virtuality Continuum 的概念,以及如何分類套用至顯示器的分類法。Their paper explored the concept of the virtuality continuum and the categorization of taxonomy applied to displays. 從那時起,混合實境的應用就不再侷限於顯示器,而也包括:Since then, the application of Mixed Reality has gone beyond displays to include:

  • 環境輸入Environmental input
  • 空間音效Spatial sound
  • 實際和虛擬空間中的位置和定位Locations and positioning in both real and virtual spaces

混合實境頻譜影像The Mixed Reality spectrum image
影像:混合實境是實體環境與數位世界混合的結果。Image: Mixed Reality is the result of blending the physical world with the digital world.



環境輸入和認知Environmental input and perception

過去數十年來,眾人持續探索人類與電腦輸入之間的關聯性,因而造就了所謂的 人機互動 (或簡稱 HCI) 的專業領域。Over the past several decades, exploration into the relationship between human and computer input has continued, leading to the discipline known as human computer interaction or HCI. 人類輸入會透過各種不同的方式進行,包括鍵盤、滑鼠、觸控、筆跡、語音,甚至是 Kinect 的框架追蹤。Human input happens through different means, including keyboards, mice, touch, ink, voice, and even Kinect skeletal tracking.

在感應器和處理能力方面的提升,逐漸使環境中的電腦輸入形成新的區域。Advancements in sensors and processing are creating new areas of computer input from environments. 電腦與環境之間的互動是環境理解或 感知,而正因如此,Windows 中顯示環境資訊的 API 名稱即稱為 認知 APIThe interaction between computers and environments is environmental understanding or perception, which is why the API names in Windows that reveal environmental information are called the perception APIs. 環境輸入會擷取諸多事物,像是使用者所處的位置 (例如頭部追蹤)、表面和界限 (例如空間對應場景理解)、環境光源、環境音效、物件辨識和位置。Environmental input captures things like a person's position in the world (head tracking), surfaces, and boundaries (spatial mapping and scene understanding), ambient lighting, environmental sound, object recognition, and location.


顯示電腦、人類與環境之間互動的卞氏圖表Venn diagram showing interactions between computers, humans and environments

影像:電腦、人類與環境之間的互動。

Image: The interactions between computers, humans, and environments.


電腦處理、人類輸入和環境輸入 這三者的組合,造就了建立真正混合實境體驗的舞台。The combination of all three - computer processing, human input, and environmental input - sets the stage for creating true Mixed Reality experiences. 實體世界中的移動會轉譯成數位世界中的移動。Movement through the physical world translates to movement in the digital world. 實體世界的界限會影響數位世界中的應用程式體驗,例如遊戲進行。Boundaries in the physical world influence application experiences, such as game play, in the digital world. 如果沒有環境輸入,體驗就無法在混合實體與數位實境間融合。Without environmental input, experiences can't blend between physical and digital realities.



混合實境頻譜The Mixed Reality spectrum

因為混合實境融合了實際和數位世界,這兩個實境就定義了頻譜的兩極,稱為 Virtuality Continuum。Since Mixed Reality blends both physical and digital worlds, these two realities define the polar ends of a spectrum known as the virtuality continuum. 我們將實境的陣列稱為 混合實境頻譜We refer to the array of realities as the Mixed Reality spectrum. 左側是我們人類所在之處的實體實境。On the left-hand side, we have the physical reality that we as humans exist in. 右側則是對應的數位實境。On the right-hand side, we have the corresponding digital reality.



擴增與虛擬實境Augmented vs. virtual reality

現今市場上大部分的行動電話幾乎都沒有環境理解功能。Most mobile phones on the market today have little to no environmental understanding capabilities. 他們所提供的體驗無法混合實體和數位實境。The experiences they offer can't mix physical and digital realities. 在實體世界的影片串流上重疊圖形的體驗即為「擴增實境」 。The experiences that overlay graphics on video streams of the physical world are augmented reality. 遮蔽您的檢視來呈現數位體驗的體驗即為「虛擬實境」 。The experiences that occlude your view to present a digital experience are virtual reality. 擴增實境與虛擬實境之間的體驗,形成了 混合實境The experiences enabled between augmented and virtual reality form Mixed Reality:

  • 從真實世界開始,放置數位物件 (例如全像投影),如同真實存在一般。Starting with the physical world, placing a digital object, such as a hologram, as if it was there.
  • 從實體世界開始,另一個人的數位表示法 -- [頭像] -- 顯示他們在留言時所處的位置。Starting with the physical world, a digital representation of another person--an avatar--shows the location where they were standing when leaving notes. 換句話說,即為不同時間點表示非同步共同作業的體驗。In other words, experiences that represent asynchronous collaboration at different points in time.
  • 從數位世界開始,實體世界 (例如牆和傢俱) 的實體界限會以數位方式出現在體驗中,以協助使用者避開實體物件。Starting with a digital world, physical boundaries from the physical world like walls and furniture appear digitally within the experience to help users avoid physical objects.

混合實境頻譜The Mixed Reality spectrum
影像:混合實境頻譜Image: The Mixed Reality spectrum


現今提供的多數擴增實境和虛擬實境產品,都代表此頻譜的一小部分,且被視為較大混合實境頻譜的子集。Most augmented reality and virtual reality offerings available today represent a small part of this spectrum and are considered subsets of the larger Mixed Reality spectrum. Windows 10 以整個頻譜為基礎,並可將數位方式呈現的人員、環境和事物與真實世界混合。Windows 10 is built with the entire spectrum in mind, and allows blending digital representations of people, places, and things with the real world.

裝置和體驗Devices and experiences

有兩種主要的裝置類型可提供 Windows Mixed Reality 體驗:There are two main types of devices that deliver Windows Mixed Reality experiences:

  1. 裝置能夠將數位內容放在真實世界中,如同真實存在一般,是 Holographic 裝置 的特性。Holographic devices are characterized by the device's ability to place digital content in the real world as if it were there.
  2. 裝置能夠建立「存在感」,隱藏真實世界,而以數位體驗取而代之,是 沉浸式裝置 的特性。Immersive devices are characterized by the device's ability to create a sense of "presence"--hiding the physical world, and replacing it with a digital experience.
特性Characteristic 全像攝影裝置Holographic devices 沉浸式裝置Immersive devices
範例裝置Example device Microsoft HoloLensMicrosoft HoloLens

Microsoft HoloLens 2 image
Samsung HMD Odyssey+Samsung HMD Odyssey+

Samsung HMD Odyssey+ image
顯示器Display 透明顯示器。See-through display. 讓使用者在穿戴頭戴式裝置時可看到實體環境。Allows user to see the physical environment while wearing the headset. 不透明顯示器。Opaque display. 在穿戴頭戴式裝置時封鎖實體環境。Blocks out the physical environment while wearing the headset.
移動Movement 完整的六度自由移動,包括旋轉和轉譯。Full six-degrees-of-freedom movement, both rotation and translation. 完整的六度自由移動,包括旋轉和轉譯。Full six-degrees-of-freedom movement, both rotation and translation.

注意

無論裝置是連線到其他電腦或以行動網卡連線到其他電腦 (透過 USB 纜線或 Wi-Fi) 或是獨立的 (未連線),都不會反映裝置是否為全像攝影還是沉浸式。Whether a device is connected to or tethered to a separate PC (via USB cable or Wi-Fi) or self-contained (untethered) doesn't reflect whether a device is holographic or immersive. 增加行動性的功能會帶來更好的體驗,且全像攝影和沉浸式裝置都可能是以行動網卡連線或是未連線的。Features that improve mobility lead to better experiences, and both holographic and immersive devices could be tethered or untethered.

技術提升可實現混合實境的體驗,但目前沒有可在整個頻譜中執行體驗的任何裝置。Technological advancement enables Mixed Reality experiences, but there aren't any devices today that can run experiences across the entire spectrum. Windows 10 同時為裝置製造商和開發人員提供通用的混合實境平台。Windows 10 provides a common Mixed Reality platform for both device manufacturers and developers. 現今的裝置可以支援混合實境頻譜內的特定範圍,而新型裝置則可擴大該範圍。Devices today can support a specific range within the Mixed Reality spectrum, with new devices expanding that range. 在未來,全像攝影裝置會變成沉浸式,而沉浸式裝置會變得更全像攝影。In the future, holographic devices will be immersive, and immersive devices will be more holographic.


混合實境頻譜中的裝置類型Device types in the Mixed Reality spectrum
影像:裝置在混合實境頻譜上的位置Image: Where devices exist on the Mixed Reality spectrum

最好能考量應用程式或遊戲開發人員想要建立的體驗類型。It's best to think what type of experience an application or game developer wants to create. 這些體驗通常會以頻譜上的特定點或部分作為目標。The experiences will typically target a specific point or part on the spectrum. 開發人員應考慮他們想要作為目標的裝置功能。Developers should consider the capabilities of devices they want to target. 仰賴實體世界的體驗在 HoloLens 上的效果最佳。Experiences that rely on the physical world will run best on HoloLens.

  • 向左 (接近實體實境)。Towards the left (near physical reality). 使用者仍然存在於其實體環境中,且永遠不會相信他們已離開該環境。Users remain present in their physical environment and are never made to believe they have left that environment.
  • 中間 (完全混合實境)。In the middle (fully Mixed Reality). 這些體驗混合了真實世界與數位世界。These experiences blend the real world and the digital world. 看過野蠻遊戲這部電影的觀看者,可以調整故事中所出現房屋的實體結構與叢林環境的混合方式。Viewers who have seen the movie Jumanji can reconcile how the physical structure of the house where the story took place was blended with a jungle environment.
  • 向右 (接近數位實境)。Towards the right (near digital reality). 使用者會體驗到數位環境,且不會察覺周遭實體環境發生的情況。Users experience a digital environment, and are unaware of what occurs in the physical environment around them.

下一個探索檢查點Next Discovery Checkpoint

如果您有遵循我們安排的探索旅程,則您正處於探索混合實境基本概念的途中。If you're following the discovery journey we've laid out, you're in the midst of exploring the basics of Mixed Reality. 您可以從這裡繼續進行下一個基本主題:From here, you can continue to the next foundational topic: