個案研究-追求更多個人運算的能力Case study - The pursuit of more personal computing

現在就建立產品以開發未來的商機。Tomorrow’s opportunities are uncovered by building products today. 這些產品提供的解決方案可讓您清楚掌握未來的必要條件。The solutions these products provide reveal what’s necessary to advance the future. 有了混合的現實,更是如此:有意義的見解來自實際工作的實際操作—真實的裝置、真實客戶、真正的問題。With mixed reality this is especially true: Meaningful insight comes from getting hands-on with real work — real devices, real customers, real problems.

在 Microsoft,我是設計團隊的一部分,可協助企業合作夥伴使用 Windows Mixed Reality 為其企業打造體驗。At Microsoft, I’m part of the design team helping enterprise partners build experiences for their business using Windows Mixed Reality. 過去一年來,我們的團隊專注于 HoloLens,並瞭解 Microsoft 的旗艦全像裝置可以如何為現今的客戶提供價值。Over the past year, our team has focused on HoloLens and understanding how Microsoft’s flagship holographic device can deliver value to customers today. 與這些公司的設計人員和開發人員密切合作,我們的小組專注于找出技術上不可行、不切合實際或不具 HoloLens 的解決方案。Working closely with designers and developers from these companies, our team focuses on uncovering solutions that would be technically unfeasible, financially impractical, or otherwise impossible without HoloLens.

案例西部保留大學Case Western Reserve University
HoloAnatomy from Case 西部預約大學HoloAnatomy from Case Western Reserve University

建立這些解決方案可協助 Microsoft 的內部團隊做好新一代運算的準備。Building these solutions helps Microsoft’s internal teams prepare for the next generation of computing. 瞭解個人和企業與混合現實、語音和 AI 等核心技術的互動方式,協助 Microsoft 為開發人員打造更好的裝置、平臺和工具。Learning how individuals and businesses interact with core technologies like mixed reality, voice, and AI, helps Microsoft build better devices, platforms, and tools for developers. 如果您是設計人員或探索此領域的開發人員,請瞭解我們的小組目前從合作夥伴學習的概念,對於準備明天的混合現實商機而言很重要。If you are a designer or a developer exploring this space, understanding what our teams are learning from partners today is critical to preparing for tomorrow’s mixed reality opportunities.

Microsoft 的混合現實野心Microsoft's ambition for mixed reality

我們生活在兩個領域:實體和數位。We live our lives among two worlds: the physical and the digital. 兩者都有基本的優勢,可讓我們用來加強和擴充我們的能力。Both have fundamental strengths that we leverage to augment and extend our abilities. 我們可以與朋友討論、使用我們的實體解讀來瞭解內文語言之類的專案,或者我們可以藉由影片與來自英里離開的朋友聊天,來增加我們的能力。We can talk in-person with a friend, using our physical senses to understand things like body language, or we can augment our ability to talk by video-chatting with a friend from miles away. 到目前為止,這兩個世界及其各自的強項都已以根本上區隔開。Until now these two worlds, and their respective strengths, have been fundamentally separated.

實體世界是其中一個原子和物理。The physical world is one of atoms and physics. 我們使用我們的方式來進行決策,利用多年來學習的行為,在我們的環境中與物件和人員互動。We use our senses to make decisions, leveraging years of learned behavior to interact with objects and people in our environments. 雖然這些互動很簡單,但我們受限於我們的實體能力和本質法則。Despite the ease of these interactions, we are limited by our physical abilities and the laws of nature.

數位世界是其中一種位和邏輯。The digital world is one of bits and logic. 計算是立即進行的,而資訊可以輕鬆散發。Computations are made instantly while information can be distributed effortlessly. 儘管有資訊的速度和流程,但互動通常僅限於小型螢幕、摘要輸入和雜訊摘要。Despite the speed and flow of information, interactions are too often limited to small screens, abstract inputs, and noisy feeds.


如果我們可以合併實體和數位世界的優點,該怎麼辦?What if we could combine the advantages of the physical and digital worlds? 這是橫跨混合現實範圍之體驗的基石:實體和數位共同存在並無縫互動的媒體。This is the cornerstone of experiences across the spectrum of mixed reality: a medium where the physical and digital co-exist and seamlessly interact. 結合這些領域可建立新的基礎,以更自然地與技術互動,也就是個人運算的演進。Combining these worlds builds a new foundation for interacting more naturally with technology — an evolution in personal computing.

由於開發人員已經開始探索深度和展示的機會,因此混合現實的範圍已分裂。The spectrum of mixed reality has exploded as developers have begun exploring the opportunities of immersion and presence. 使用沉浸式 (虛擬實境) 體驗,將使用者帶入數位世界,並以全像 (的真實世界) 體驗在實體世界中建立數位物件。Bringing users into the digital world with immersive (virtual reality) experiences and creating digital objects in the physical world with holographic (augmented reality) experiences. 但是將實體世界對應到數位世界有何好處?But what are the benefits of mapping the physical world to the digital world? 當我們讓電腦看得到這種情況時,會發生什麼事?What happens when we give computers eyes to see?

全像投影的基本攝影機視覺技術,就像是一對電腦,看看您的環境:世界各地的物件、您的人,以及發生的變更。The fundamental camera vision technology behind holograms acts like a pair of eyes for the computer to see the environment around you: Objects in the world, people around you, changes as they happen. 對實體世界中的內容進行數位瞭解。A digital understanding of your context in the physical world. 這會導致大量的資訊,也就是我們只開始瞭解的含意。This leads to an enormous amount of information, the implications of which we are only beginning to understand.

累積核心技術Culminating core technologies

運算通常太重要。Computing is too often a tangible thing. 抓取我們的裝置,告訴他們我們和我們想要的內容。Grabbing our devices to tell them who we are and what we want. Contorting 我們的想法,並調整我們所說的內容,以符合我們認為電腦需要聽到的內容。Contorting our thinking and aligning what we say to match what we believe the computer needs to hear.

混合現實的承諾(尤其是在具有全像全球體驗的真實世界中),是減少與技術互動的負擔。The promise of mixed reality, especially in the real world with holographic experiences, is to lessen the burden of interacting with technology. 減少認知負載,因為使用者可以流覽目前的運算所固有的抽象層。Reducing cognitive load as users navigate the layers of abstraction inherent to computing today. 我們要如何設計體驗,不僅能利用內容相關的理解,還可以更輕鬆地繪製見解並採取行動嗎?How can we design experiences that not only take advantage of contextual understanding, but make it easier to draw insight and take action? 有兩個參與的技術也正在嘗試解決此問題:Two contributing technologies are also attempting to solve this problem:

  • 語音和對話方面的 語音 ,可讓使用者透過更自然的方式與電腦通訊,也就是透過文字來回應 bot 或發出命令給對話式硬體。Voice , in terms of speech and conversation, is enabling users to communicate with computers through more natural means — answering bots through text or issuing commands to conversational hardware.
  • AI 的電源體驗可從逐漸複雜的資料集中提煉出見解。AI is powering experiences that distill insights from increasingly complex datasets. 雖然 AI 是很大的主題,但最近的進度已針對依賴電腦視覺的裝置、更自然的數位助理,以及向使用者建議動作提供基礎。While AI is an enormous topic, recent progress has provided the groundwork for devices that rely on computer vision, more natural digital assistants, and recommending actions to users.

Mixed reality 提供將這些技術結合成單一使用者體驗的媒體。Mixed reality provides a medium to combine these technologies into a single user experience. 聲音變成一種強大的自然方法,可在佩戴全像攝影耳機時輸入。Voice becomes a powerful, natural method for input when wearing a holographic headset. AI 是一種重要的密碼,可情境化連接實體和數位世界的大量資訊。AI acts as a critical cipher to contextualize the enormous amounts of information connecting the physical and digital worlds. 這就是為什麼 Sataya Nadella 指的是「終極電腦」,這是適用于三核心技術的累積裝置。This is why Sataya Nadella refers to HoloLens as ‘the ultimate computer’, it’s a culminating device for three core technologies. 一種平臺,可讓人們更輕鬆地與不斷成長的裝置和服務複雜性互動。A platform to empower humans to more easily interact with the growing complexity of devices and services.

臉部中較少的介面Less interface in your face

將實體世界連接到數位世界的累積裝置,可讓我們設計更自然的體驗,而不需要繁瑣的抽象概念。A culminating device that connects the physical world to the digital world allows us to design experiences that fit more naturally, without cumbersome abstractions. 請考慮您所建立的體驗:當您移除抽象概念時,您的介面有多少保留?Consider the experiences you’ve created: When the barriers of abstraction are removed, how much of your interface is left? 當您知道使用者及其內容時,應用程式流程的哪些部分會變更?Which parts of your app flow change when you know the user and their context? 有多少個功能表和按鈕?How many menus and buttons remain?

例如,請考慮混合現實中的共用體驗,例如針對科學家打造的 OnSight tool NASA Jet 推進實驗。For example, think about shared experiences in mixed reality like the OnSight tool NASA’s Jet Propulsion Laboratory built for scientists. 與其建立系統來查看 Martian 資料 (將資料抽象化至螢幕或顯示) ,他們將資料帶入資料內,並將它們有效地放在 Mars 的表面上。Instead of building a system to look at Martian data (abstracting the data onto screens or displays), they brought scientists inside the data, effectively putting them on the surface of Mars as they collaborated.

NASA OnSightNASA OnSight

科學家可以直接指向 Martian 的地質學,而不是尋找將注意力吸引到某些的按鈕。Instead of finding the button to draw attention to some Martian geology, scientists can point to it directly. 沒有功能表,沒有畫筆,沒有有效使用工具的學習曲線。No menus, no pens, no learning curve to using the tool effectively. 藉由充分利用實體世界的已知能力,混合現實的自然互動也可以規避深度的技術讀寫能力,甚至是最先進的業界工具。By leveraging our known abilities from the physical world, more natural interactions in mixed reality can circumvent deep technical literacy in even the most advanced industry tools.

同樣地,語音和 AI 也可以像這樣在體驗中擴充自然互動。Likewise, voice and AI can extend natural interaction in experience like this. 當數位助理可以「見」到全球各地的對話時,交談就比較不麻煩。When digital assistants can ‘see’ into the world around us, conversations will feel less cumbersome. NASA 工具中的 bot 可能會從科學家授予透過 Martian 地質學來提取內容,作為就緒的 (,並在科學家手勢並詢問「這個」或「該」時,使用交談式) 資訊來源。A bot in NASA’s tool might extract context from the scientists conferring over Martian geology, acting as a ready (and conversational) source of information when scientists gesture and ask about ‘this’ or ‘that’. 知道您的內容的電腦已準備好透過最適合您的內容的互動方法,使用適合您的資訊來跳入。A computer that knows your context is ready to jump in with information appropriate to you, through a method of interaction most appropriate to your context.

打造基礎Building on a foundation

在這一世界的內容體驗中,使用者的行動性可做為關鍵。In this world of contextual experiences, catering to the mobility of the user is key. 透過高度個人化的內容,在裝置與服務之間移動流暢地,使用者將可順暢地移動實體世界,進而帶來 大規模 的平臺挑戰。Moving fluidly between devices and services, with highly personalized contexts, users will travel about the physical world seamlessly — creating a massive platform challenge. 當實體世界變成數位端點時,互通性成為頂尖優勢。When the physical world becomes a digital end-point, interoperability reigns supreme.

「將數位世界與實體世界結合在一起,就能發揮人們的潛能 .。。讓地球上的每個人和組織都能達到更多目標。」“Bringing together the digital universe and the physical world will unlock human potential… enabling every person and organization on the planet to achieve more.”
— Satya Nadella— Satya Nadella

Windows Mixed Reality 是為了建立裝置生態系統的平臺,讓開發人員能夠為最大可能的物件建立沉浸式、經濟實惠且相容的體驗。Windows Mixed Reality is an effort to create a platform for an ecosystem of devices, enabling developers to create immersive, affordable, and compatible experiences for the largest possible audience. 未來將不會限制為單一的製造商,只會讓單一裝置。The future will not be limited to a single manufacturer, let alone a single device. 耳機、行動裝置、電腦、配件 .。。Headsets, mobile devices, PCs, accessories… 所有這些實體東西都必須交互操作 (以及像是圖形和服務等數位東西) 透過基礎平臺,才能成功地實現混合現實的承諾。all these physical things must interoperate (as well as digital things like graphs and services) through an underlying platform to successfully deliver on the promise of mixed reality.

立即為明天的體驗設計Designing for tomorrow's experiences today

立即為明天的體驗設計 Designing for tomorrow's experiences today
這一種全新經驗背後的核心技術,讓設計人員和開發人員能夠立即建立引人注目且成功的體驗。Each one of the core technologies behind this new class of experiences are enabling designers and developers to create compelling and successful experiences today. 藉由減少抽象化,我們可以直接與數位世界互動,讓我們以增強和增強人類能力的方式進行設計。By reducing abstraction, we can interact more directly with the digital world, allowing us to design in ways that augment and amplify human abilities. 語音技術 ((例如 Cortana) )可讓使用者執行日益複雜的交談和案例,而 AI 技術 (透過 Microsoft 認知) 服務等工具來,則會導致公司重新思考使用者將如何與社交媒體中的所有專案互動以提供連鎖管理。Voice technology (through bots and digital assistants like Cortana) is allowing users to carry out increasingly complex conversations and scenarios, while AI technology (through tools like Microsoft Cognitive Services) is causing companies to rethink how users will interact with everything from social media to supply chain management.

這些互動類型會依賴新類別的設計工具,以及平臺的基本支援。These types of interactions will rely on both a new class of design tools as well fundamental support from the platform. 建立這些工具並為裝置和服務建立此平臺,需要瞭解明天的體驗如何解決真正的有形問題。Creating these tools and building this platform for devices and services relies on understanding how tomorrow's experiences will solve real, tangible problems. 我們已找出五個機會,我們的企業合作夥伴已提供寶貴的解決方案,而且我們認為持續的投資將有助於我們為這個新的運算類別做好準備。We've identified five areas of opportunity where our enterprise partners have delivered valuable solutions and where we believe continued investment will help us prepare for this new class of computing.

商機領域Areas of opportunity

過去一年來,開發人員合作關係已發現與客戶產生共鳴並建立成功企業解決方案的機會。The past year of developer partnerships has uncovered areas of opportunity that resonate with customers and create successful enterprise solutions. 從科學家和技術人員到設計人員和用戶端,都有五個機會,也就是 Microsoft 合作夥伴發現混合現實的價值。From scientists and technicians to designers and clients, five areas of opportunity have emerged where Microsoft partners are find value with mixed reality. 這些區域已提供對未來平臺需求(例如 Windows Mixed Reality)的大規模深入解析,並可協助您瞭解這些新體驗將如何影響我們學習、共同作業、溝通及建立的方式。These areas are already providing massive insight into the future needs of platforms like Windows Mixed Reality and can help you understand how these new experiences will impact the ways we learn, collaborate, communicate, and create.

1. 建立和設計1. Creation and design

混合現實的其中一個主要機會,就是能夠以真正的大小和規模,在真實世界的設定中即時查看和操作3D 設計。One of the chief opportunities of mixed reality is the ability to see and manipulate 3D designs in real time, in a real-world setting, at true size and scale. 設計和原型設計工具已將畫面的範圍化,並回到設計領域,通常會保留給有形的實體材質,例如木材和 clay。Design and prototyping tools have escaped the confines of screens, returning to a realm of design usually reserved for tangible, physical materials like wood and clay.

Autodesk 所建立的混合現實體驗,旨在改善整個產品開發程式的共同作業。Autodesk created a mixed reality experience aimed at improving collaboration across the entire product development process. 工程師和設計人員在其環境中調查和 critique 3D 模型的能力,讓他們可以即時反復查看設計。The ability for engineers and designers to survey and critique 3D models in their environment allowed them to iterate on a design in real-time. 這不僅可以更快速地進行原型設計,還能在程式中提供更自信的決策。This not only enabled faster prototyping but gave way to more confident decisions in the process.

這類體驗強調了核心共同作業體驗的需求,讓使用者能夠查看共用物件並與其進行通訊。Experiences like this highlight the need for core collaboration experiences, the ability for users to see and communicate with shared objects. Autodesk 的目標是將產品從 design professional 和工程師擴展到數位演出者、學生和業餘人士。Autodesk's goal is to expand the product from design professional and engineers to digital artists, students, and hobbyists. 隨著使用者的3D 專業知識層級的減少,與物件互動的能力自然就變得很重要。As the level of 3D expertise of users decreases, the ability to interact with objects naturally becomes crucial.

2. 元件和製造2. Assembly and manufacturing

從工廠樓層增加的特製化數量,到供應鏈管理的快速進步,對相關資訊的無縫存取就是關鍵。From the increasing amount of specialization on factory floors to the rapid advancements of supply chain management, seamless access to relevant information is key. Mixed reality 提供合成大量資料集的能力,並提供可協助流覽和操作等區域的視覺顯示。Mixed reality offers the ability to synthesize extensive data sets and provide visual displays that aid in areas like navigation and operations. 這些通常是高度技術性的小型欄位,其中與自訂資料集和服務的整合對於降低複雜性和提供成功的體驗至關重要。These are often highly technical, niche fields where integration with custom datasets and services are crucial to reducing complexity and providing a successful experience.

電梯化製造商 ThyssenKrupp 為電梯服務技術人員建立了體驗,讓他們能夠以視覺化方式找出準備工作的問題。Elevator manufacturer ThyssenKrupp created an experience for elevator service technicians, allowing them to visualize and identify problems in preparation for a job. 由於有超過24000技術人員的團隊,所以 HoloLens 這類裝置可讓這些技術人員擁有免費的技術和專家資訊存取權。With a team spanning over 24,000 technicians, devices like HoloLens allow these technicians to have remote, hands-free access to technical and expert information.

ThyssenKrupp 強調了一個功能強大的概念,在此可以迅速將重要、內容相關的資訊傳遞給使用者。ThyssenKrupp highlights a powerful concept here that critical, contextually-relevant information can be delivered quickly to users. 當我們深入探討新的體驗時,distilling 與使用者相關的內容有大量可能的資訊,就是重要的。As we look ahead to a new class of experiences, distilling vast amounts of possible information to content that is relevant to the user will be key.

3. 訓練和開發3. Training and development

以三個維度表示物件和資訊,可提供新的方式來說明以視覺化方式和空間理解的情節。Representing objects and information in three dimensions offer new ways to explain scenarios visually and with spatial understanding. 定型成為商機的重要領域,利用數位來代表龐大、複雜的物件 (例如 jet 引擎) ,以建立定型模擬,以提供實體解決方案的一小部分成本。Training becomes a key area of opportunity, leveraging the ability to digitally represent enormous, complex objects (like jet engines) to create training simulations for a fraction of the cost of a physical solution.

日本航空公司 一直在試驗概念計畫,以提供引擎機制和航班小組的補充訓練。Japan Airlines has been experimenting with concept programs to provide supplemental training for engine mechanics and flight crews. 大型的 jet 引擎 (大小和複雜度) 可在3D 中表示,並忽略實體世界在受訓者上移動和旋轉虛擬物件的限制,讓他們可以查看航空公司元件如何即時運作。The massive jet engines (in both size and complexity) can be represented in 3D, ignoring the limitations of the physical world to move and rotate virtual objects around trainees, allowing them to see how airline components work in real-time.

使用虛擬元件進行 (,並減少昂貴、實體訓練模擬器) 的需求,是在企業案例中提供價值的主要方式。Training with virtual components (and reducing the need for expensive, physical training simulators) is a key way to deliver value in enterprise scenarios today. 隨著此案例的 (,如同我們在醫學) 等區域中所看到的一樣,電腦視覺對於辨識環境中的唯一物件很重要,請瞭解使用者的內容,並提供相關的指示。As this scenario expands (as we've seen in areas like medicine), computer vision becomes especially important to recognize unique objects in the environment, understand the context of the user, and deliver relevant instructions.

4. 溝通與理解4. Communication and understanding

兩個人之間的互動 (這兩個參與者是否位於混合現實裝置中,或者是在傳統電腦或手機上,) 可以在新環境內提供深度,或在與他人進行通訊時,有意義的存在。Interactions between two people (whether both participants are in mixed reality devices or one is on a traditional PC or phone) can provide both a sense of immersion within a new environment or a sense of presence when communicating virtually with others. 在企業案例中,這是行動團隊的 boon,有助於提高對專案的理解,並減少旅行的需求。In enterprise scenarios this sense of presence is a boon to mobile teams, helping to increase understanding of projects and lessen the need for travel.

商業技術製造商 Trimble 為架構和建築產業專業人員開發了解決方案,以在建立開發期間共同作業和審核工作。Commercial tech manufacturer Trimble developed a solution for architecture and construction industry professionals to collaborate and review work during building development. 專業人員可以從遠端 immerse 至專案中,以討論進度,或根據位置和審核計畫,在其周圍的環境中以其最終形式) (。Professionals can remotely immerse themselves into a project to discuss progress or be on location and review plans as they would look (in their final form) in the environment around them.

共用體驗是 Microsoft 的主要投資區域,其中的應用程式(例如 Skype)會探索在網際空間中呈現人類的新方式。Shared experiences are a major area of investment for Microsoft, with apps like Skype exploring new ways to represent humans in digital space. 小組正在探索體積型影片錄製、虛擬人偶和重建參與者的實體空間。Teams are exploring volumetric video recordings, avatars, and recreations of a participant's physical space.

5. 娛樂和 engagement5. Entertainment and engagement

混合現實的深度本質可能會對客戶與娛樂和行銷的互動方式產生巨大的影響。The nature of immersion in mixed reality can have a tremendous impact on the way customers engage with entertainment and marketing. 演出者和娛樂工作室以吸引人的故事行銷,讓公司探索品牌行銷的含意。Artists and entertainment studios have explored mixed reality as a compelling medium for storytelling, while companies are exploring the implications for brand marketing. 從私用 showrooms 中的產品示範,到以貿易 expos 說出的願景,內容可透過更好用、量身打造的方式來提供。From product demos in private showrooms to vision pieces told at trade expos, content can be delivered in more immersive and tailored ways.

Volvo 建立了展示其最新汽車模型的體驗, (immersing 使用者的不同色彩和設定) 同時強調 advanced 感應器的運作方式,以建立更好、更安全的駕駛體驗。Volvo created an experience for showcasing their latest car models (immersing users in different colors and configurations) while highlighting how advanced sensors work to create a better, safer driving experience. 透過引導式 showroom 體驗將客戶帶到可讓 Volvo 告知尖端汽車功能背後的故事,同時提供令人印象深刻的品牌故事圖解。Taking customers through a guided showroom experience allows Volvo to tell the story behind cutting-edge car feature while delivering a memorable portrayal of their brand story.

娛樂有許多方式可將混合現實的界限 (特別是虛擬實境) 而且在此空間中最吸引人的是,它會與上一個商機的互動方式結合:通訊和理解。Entertainment is in many ways pushing the bounds of mixed reality (especially virtual reality) and perhaps most compelling in this space is how it will interact combine with the previous area of opportunity: communication and understanding. 有了多個使用者,每個使用者都有自己的裝置和介面方法變體,您可以想像在娛樂領域中的個人化未來。With multiple users, each with their own variant of devices and interface methods, you can imagine a vast future of personalization in the realm of entertainment.

立即開始建立Start building today

我們很難說,「混合式」的未來未來會看起來像是取用者,但專注于獨特的問題、實際的硬體實際操作, 立即 試用混合現實、語音和 AI 之間的交集It's hard to say what the far future of mixed reality will look like for consumers, but focusing on unique problems, getting hands-on with real hardware, and experimenting today with the intersection between mixed reality, voice, and AI is key. Microsoft 剛開始使用 mixed reality,但從現今的企業經驗中學習,將可協助您建立未來的體驗。Microsoft is just getting started with mixed reality but learning from the successes realized by businesses today will help you create the experiences of tomorrow.

下次探索檢查點Next Discovery Checkpoint

如果您正在關注我們所配置的 探索旅程 ,您將會在探索混合現實的基本概念。If you're following the discovery journey we've laid out, you're in the midst of exploring the basics of Mixed Reality. 您可以從這裡繼續進行下一個基本主題:From here, you can proceed to the next foundational topic:

或跳至混合現實的核心體驗:Or jump into Mixed Reality core experiences:

關於作者About the author

Picture of Mark Vitazko Mark VitazkoMark Vitazko
UX 設計者 @MicrosoftUX Designer @Microsoft