案例研究-我在 HoloLens 设计团队中的第一年Case study - My first year on the HoloLens design team

当我在2016年1月加入 HoloLens 设计团队时,我从 2D flatland 到三维世界的旅程开始使用。My journey from a 2D flatland to the 3D world started when I joined the HoloLens design team in January, 2016. 在加入团队之前,我在三维设计方面的体验非常少。Before joining the team, I had very little experience in 3D design. 这就像是从单个步骤开始,这就是谚语的一段时间,只不过第一步是 leap,It was like the Chinese proverb about a journey of a thousand miles beginning with a single step, except in my case that first step was a leap!

从2D 到3D 的Taking the Leap from 2D to 3D
从2D 到3D 的Taking the leap from 2D to 3D

"我认为我一直在不知道如何推动汽车的情况下,跳转到驱动程序的座位。我曾恐惧,但却非常专注。 "“I felt as though I had jumped into the driver’s seat without knowing how to drive the car. I was overwhelmed and scared, yet very focused.”
— Hae 金— Hae Jin Lee

在过去一年中,我选择了快速的技能和知识,但仍有许多经验。During the past year, I picked up skills and knowledge as fast as I could, but I still have a lot to learn. 在这里,我使用视频教程撰写了4个观察,其中介绍了从2D 到三维交互设计器的转换。Here, I’ve written up 4 observations with a video tutorial documenting my transition from a 2D to 3D interaction designer. 我希望我的体验会使其他设计人员能够使用这一三维。I hope my experience will inspire other designers to take the leap to 3D.

令人非常再见的帧。Good-bye frame. Hello 空间/diegetic UIHello spatial / diegetic UI

无论何时设计海报、杂志、网站或应用程序屏幕,定义的帧都 (通常是每个问题的常量) 。Whenever I designed posters, magazines, websites, or app screens, a defined frame (usually a rectangle) was a constant for every problem. 除非你要在 HoloLens 或其他 VR 设备中阅读此文章,否则你将从外部通过2D 屏幕安全地在帧内 查看此 文章。Unless you are reading this post in a HoloLens or other VR device, you are looking at this from the outside through 2D screen safely guarded within a frame. 内容是您的外部内容。Content is external to you. 但是,混合现实头戴式耳机 消除了帧 ,因此你在内容空间中,并在内容空间内查看内容。However, Mixed Reality headset eliminates the frame , so you are within the content space, looking and walking through the content from inside-out.

我在概念上了解到这一点,但在开始时,我犯了一个错误,只是将2D 思维转移到了3D 空间。I understood this conceptually, but in the beginning I made the mistake of simply transferring 2D thinking into 3D space. 显然,这不是很好,因为3D 空间具有自己的唯一属性(如视图更改 (基于用户的头移动) 和 用户舒适 (的不同要求 基于设备的属性和使用它们) 的人员)。That obviously didn’t work well because 3D space has its own unique properties such as a view change (based on user’s head movement) and different requirement for user comfort (based on the properties of the devices and the humans using them). 例如,在 2D UI 设计空间中,将 UI 元素锁定到屏幕的角是一种非常常见的模式,但这种 HUD (Head 显示) 样式 UI 在 MR/VR 体验中并不自然。它阻碍用户浸入式空间,导致用户 discomfort。For example, in a 2D UI design space, locking UI elements into the corner of a screen is a very common pattern, but this HUD (Head Up Display) style UI does not feel natural in MR/VR experiences; it hinders user’s immersion into the space and causes user discomfort. 这就像在眼镜上有一个令人讨厌的灰尘粒子,因为您中断了。It’s like having an annoying dust particle on your glasses that you are dying to get rid of. 随着时间的推移,我已经了解到,在三维空间中放置内容更自然,并添加了正文锁定行为,使内容以相对固定的距离在用户关注。Over time, I learned that it feels more natural to position content in 3D space and add body-locked behavior that makes the content follow the user at a relative fixed distance.



片段:大 Diegetic UI 的示例Fragments: An example of great Diegetic UI

片段Asobo Studio for HoloLens 开发的第一人犯罪 thriller,它展示了一个很好的 Diegetic UI。Fragments, a first-person crime thriller developed by Asobo Studio for HoloLens demonstrates a great Diegetic UI. 在此游戏中,用户成为了一个主字符,这是一种试图解决神秘的侦探。In this game, the user becomes a main character, a detective who tries to solve a mystery. 在用户的物理空间中解决这个神秘的 get 分散的 pivotal 线索,通常是嵌入在虚构的对象中,而不是在自己的内部嵌入。The pivotal clues to solve this mystery get sprinkled in the user’s physical room and are often times embedded inside a fictional object rather than existing on their own. 与正文锁定的 UI 相比,此 diegetic UI 的可发现性更低,因此 Asobo 团队巧妙使用了许多提示,其中包括虚拟字符看起来、声音、光和参考线 (例如,箭头指向线索位置) 以吸引用户的注意力。This diegetic UI tends to be less discoverable than body-locked UI, so the Asobo team cleverly used many cues including virtual characters’ gaze direction, sound, light, and guides (e.g., arrow pointing the location of the clue) to grab user’s attention.

片段-Diegetic UI 示例Fragments - Diegetic UI examples
片段-Diegetic UI 示例Fragments - Diegetic UI examples

有关 diegetic UI 的观察Observations about diegetic UI

空间 UI (正文锁定的和全局锁定的) 和 diegetic UI 都具有自己的优势和劣势。Spatial UI (both body-locked and world-locked) and diegetic UI have their own strengths and weaknesses. 我鼓励设计人员尽可能多地尝试 MR/VR 应用,并为各种 UI 定位方法开发自己的了解和差异。I encourage designers to try out as many MR/VR apps as possible, and to develop their own understanding and sensibility for various UI positioning methods.

Skeuomorphism 和神奇交互的返回The return of skeuomorphism and magical interaction

Skeuomorphism,模仿现实世界对象的形状的数字接口在设计行业中的过去5–7年一直为 "uncool"。Skeuomorphism, a digital interface that mimics the shape of real world objects has been “uncool” for the last 5–7 years in the design industry. 当 Apple 最终在 iOS 7 中提供了平面设计方式时,似乎 Skeuomorphism 最终就会成为接口设计方法。When Apple finally gave way to flat design in iOS 7, it seemed like Skeuomorphism was finally dead as an interface design methodology. 不过,在市场上,新的媒体(MR/VR 耳机)到达市场,似乎 Skeuomorphism 再次返回。But then, a new medium, MR/VR headset arrived to the market and it seems like Skeuomorphism returned again. : ): )

作业模拟器: skeuomorphic VR 设计的示例Job Simulator: An example of skeuomorphic VR design

作业模拟器是 skeuomorphic VR 设计的最常见示例之一,由 Owlchemy 实验室 开发的古怪游戏。Job Simulator, a whimsical game developed by Owlchemy Labs is one of the most popular example for skeuomorphic VR design. 在此游戏中,播放机将被传输到将来,其中,机器人会将其替换为在以下四个不同作业之一中执行常见任务的内容:自动 Mechanic、Gourmet Chef、商店员工Within this game, players are transported into future where robots replace humans and humans visit a museum to experience what it feels like to perform mundane tasks at one of four different jobs: Auto Mechanic, Gourmet Chef, Store Clerk, or Office Worker.

Skeuomorphism 的好处是显而易见的。The benefit of Skeuomorphism is clear. 此游戏中熟悉的环境和对象可帮助新的 VR 用户感觉更舒适,并在虚拟空间中呈现。Familiar environments and objects within this game help new VR users feel more comfortable and present in virtual space. 它还通过将熟悉的知识和行为与对象及其相应的物理反应关联起来,使它们在控制之下。It also makes them feel like they are in control by associating familiar knowledge and behaviors with objects and their corresponding physical reactions. 例如,为了给杯杯杯杯,人们只需走到咖啡机,按下按钮,抓住杯控把手,就像在现实世界中那样倾斜。For example, to drink a cup of coffee, people simply need to walk to the coffee machine, press a button, grab the cup handle and tilt it towards their mouth as they would do in the real world.

作业模拟器Job Simulator
作业模拟器Job Simulator

因为 MR/VR 仍是开发媒介,所以必须使用一定程度的 skeuomorphism 来释义 MR/VR 技术,并将其引入世界各地更大的受众。Because MR/VR is still a developing medium, using a certain degree of skeuomorphism is necessary to demystify MR/VR technology and to introduce it to larger audiences around the world. 此外,对于特定类型的应用程序(如外科模拟或飞行模拟),使用 skeuomorphism 或现实表示可能会很有用。Additionally, using skeuomorphism or realistic representation could be beneficial for specific types of applications like surgery or flight simulation. 由于这些应用程序的目标是开发和优化可以直接应用于现实世界的特定技能,因此,对现实世界的模拟越接近,该知识就越有限制。Since the goal of these apps is to develop and refine specific skills that can be directly applied in the real world, the closer the simulation is to the real world, the more transferable the knowledge is.

请记住,skeuomorphism 只是一种方法。Remember that skeuomorphism is only one approach. MR/VR 世界的潜力远远超过这一点,设计人员应尽力创建神奇的超自然交互,这是在 MR/VR 世界中独特的新实用。The potential of the MR/VR world is far greater than that, and designers should strive to create magical hyper-natural interactions — new affordances that are uniquely possible in MR/VR world. 开始时,请考虑向普通对象添加神奇,以使用户能够满足其根本需要,包括 teleportation 和 omniscience。As a start, consider adding magical powers to ordinary objects to enable users to fulfill their fundamental desires—including teleportation and omniscience.

Doraemon 的神奇门 (左) ,Ruby slippers (权限) Doraemon’s magical door (left) and Ruby slippers(right)
Doraemon 的神奇门 (左) ,ruby slippers (权限)Doraemon’s magical door (left) and ruby slippers(right)

在 VR 中了解 skeuomorphismObservations about skeuomorphism in VR

从 Doraemon 的 "Anywhere 门" 中的 "Oz" 的向导中的 "Ruby Slippers" 到 Harry 哈里波特中的 "Maurader 地图",其中包含神奇 power abound 的普通对象示例。From “Anywhere door” in Doraemon, “Ruby Slippers” in The Wizard of Oz to “Maurader’s map” in Harry Potter, examples of ordinary objects with magical power abound in popular fiction. 这些神奇对象可帮助我们直观显示现实世界和理想之处之间的连接。These magical objects help us visualize a connection between the real-world and the fantastic, between what is and what could be. 请记住,设计神奇或 surreal 对象时,需要在功能和娱乐之间实现平衡。Keep in mind that when designing the magical or surreal object one needs to strike a balance between functionality and entertainment. 请注意,创建纯粹神奇的东西只是为了新奇。Beware of the temptation to create something purely magical just for novelty’s sake.

了解不同的输入方法Understanding different input methods

当我为2D 中型设计时,我不得不专注于输入的触摸、鼠标和键盘交互。When I designed for the 2D medium, I had to focus on touch, mouse, and keyboard interactions for inputs. 在 MR/VR 设计领域,我们的主体成为接口,用户可以使用更广泛的输入方法:包括语音、注视、手势、 6 dof 控制器和手套,它们具有更直观的与虚拟对象的直接连接。In the MR/VR design space, our body becomes the interface and users are able to use a broader selection of input methods: including speech, gaze, gesture, 6-dof controllers, and gloves that afford more intuitive and direct connection with virtual objects.

HoloLens 中的可用输入Available inputs in HoloLens
HoloLens 中的可用输入Available inputs in HoloLens

"一切都最适用于某些事情,但对于其他事情,这是最差的。"“Everything is best for something, and worst for something else.”
- 帐单 BuxtonBill Buxton

例如,在 HMD 设备上使用裸机和相机传感器进行的手势输入使用户不会拿住控制器或戴 sweaty 手套,但频繁使用会导致物理疲劳 (gorilla arm) 。For example, gesture input using bare hand and camera sensors on an HMD device frees users hand from holding controllers or wearing sweaty gloves, but frequent use can cause physical fatigue (a.k.a gorilla arm). 此外,用户必须在视线内保持其手。如果相机看不到手,则无法使用手。Also, users have to keep their hands within the line of sight; if the camera cannot see the hands, the hands cannot be used.

语音输入非常适合遍历复杂任务,因为它允许用户通过一个命令 (来剪切嵌套菜单,如 "显示 Laika studio 制作的电影"。与其他模态结合使用时,) 和也非常经济 (例如,"面部 me" 命令定向用户正在查看用户) 的全息影像。Speech input is good at traversing complex tasks because it allows users to cut through nested menus with one command (e.g., “Show me the movies made by Laika studio.”) and also very economical when coupled with other modality (e.g., “Face me” command orients the hologram a user is looking at towards the user). 但是,语音输入在干扰环境中可能无法正常工作,或者可能不适合在非常安静的空间中使用。However, speech input may not work well in noisy environment or may not appropriate in a very quiet space.

除了笔势和语音外,手动将跟踪控制器 (例如,Oculus 触控、Naopak 等。 ) 是非常流行的输入方法,因为它们易于使用、准确、利用人员的 proprioception,并提供被动 haptic 提示。不过,这些优势的代价是不能成为实际的,而是使用完全 finger 跟踪。Besides gesture and speech, hand-held tracked controllers (e.g., Oculus touch, Vive, etc.) are very popular input methods because they are easy to use, accurate, leverage people’s proprioception, and provide passive haptic cues. However, these benefits come at the cost of not being able to be bare-hands and use full finger tracking.

Senso (Left) 和 Manus VR (权限) Senso (Left) and Manus VR(Right)
Senso (Left) 和 Manus VR (权限)Senso (Left) and Manus VR (Right)

与控制器一样,与控制器相比,手套还能再次发展,这归功于尊敬的尊敬。While not as popular as controllers, gloves are gaining momentum again thanks to the MR/VR wave. 最近,大脑输入已经开始通过将 EEG 或 EMG 传感器集成到耳机 ((例如 MINDMAZE VR) )来获得虚拟环境的界面。Most recently, brain/mind input have started to gain traction as an interface for virtual environments by integrating EEG or EMG sensor to headset (e.g., MindMaze VR).

有关输入方法的观察Observations about input methods

这只是一种为 MR/VR 市场提供的输入设备示例。These are a just a sample of input devices available in the market for MR/VR. 在行业成熟并同意最佳实践之前,他们将继续增加。They will continue to proliferate until the industry matures and agrees upon best practices. 在此之前,设计人员应始终知道新的输入设备,并在特定的项目的特定输入方法中精通。Until then, designers should remain aware of new input devices and be well-versed in the specific input methods for their particular project. 设计人员需要在限制范围内寻找创造性的解决方案,同时还能在设备上寻找优势。Designers need to look for creative solutions inside of limitations, while also playing to a device’s strengths.

草拟耳机中的场景和测试Sketch the scene and test in the headset

当我在2D 中工作时,我主要只是要绘出内容。When I worked in 2D, I mostly sketched just the content. 但是,在混合现实空间中,这并不是足够的。However, in mixed reality space that wasn’t sufficient. 我必须草拟整个场景,以便更好地想象用户和虚拟对象之间的关系。I had to sketch out the entire scene to better imagine the relationships between the user and virtual objects. 为了帮助我的空间思维,我开始在 电影院 4d 中草拟场景,有时为 Maya中的原型创建简单的资产。To help my spatial thinking, I started to sketch scenes in Cinema 4D and sometimes create simple assets for prototyping in Maya. 在加入 HoloLens 团队之前,我从未使用过任何一种程序,但我仍然是新手,但使用这些三维程序的确有助于我熟悉新术语,如 着色器IK (反转运动) I had never used either program before joining the HoloLens team and I am still a newbie, but working with these 3D programs definitely helped me get comfortable with new terminology, such as shader and IK (inverse kinematics).

"无论是在三维中绘制场景的紧密关系,耳机中的实际体验几乎不会与草图相同。这就是在目标耳机中测试场景很重要的原因。 "— Hae 金“No matter how closely I sketched out the scene in 3D, the actual experience in headset was almost never the same as the sketch. That’s why it’s important to test out the scene in the target headsets.” — Hae Jin Lee

对于 HoloLens 原型编写,我尝试在 混合现实教程 中启动所有教程。For HoloLens prototyping, I tried out all the tutorials at Mixed Reality tutorials to start. 然后我开始开始与 Microsoft 为开发人员提供的 HoloToolkit ,以加速全息应用程序的开发。Then I began to play with HoloToolkit.Unity that Microsoft provides to developers to accelerate development of holographic applications. 当我停滞了某些东西后,我将我的问题发布到了 HoloLens 问题 & 答案论坛When I got stuck with something, I posted my question to HoloLens Question & Answer Forum.

在获取对 HoloLens 原型的基本了解后,我想要自行实现其他非普通人原型。After acquiring basic understanding of HoloLens prototyping, I wanted to empower other non-coders to prototype on their own. 我制作了一个视频教程,介绍如何使用 HoloLens 开发一个简单的 projectile。So I made a video tutorial that teaches how to develop a simple projectile using HoloLens. 我简要介绍了基本的概念,因此,即使您在 HoloLens 开发中没有经验,您也应该能够继续操作。I briefly explain the basic concepts, so even if you have zero experience in HoloLens development, you should be able to follow along.

我为非编程人员提供了这个简单的教程。I made this simple tutorial for non-programmers like myself.

对于 VR 原型编写,我采用了 Vr 开发学校 的课程,并在 Lynda.com 中 为虚拟现实拍摄了3d 内容For VR prototyping, I took courses at VR Dev School and also took 3D Content Creation for Virtual Reality at Lynda.com. 使用 VR 开发人员,我们可以更深入地了解如何编写代码和 Lynda 课程,为我提供了创建用于 VR 的资产的简短介绍。VR Dev school provided me more in depth knowledge in coding and the Lynda course offered me a nice short introduction to creating assets for VR.

采用 leapTake the leap

一年前,我觉得这一切都有点复杂。A year ago, I felt like all of this was a bit overwhelming. 现在,我可以告诉您此操作是否值得100%。Now I can tell you that it was 100% worth the effort. 先生/VR 仍非常年轻,有很多有趣的等待实现。MR/VR is still very young medium and there are so many interesting possibilities waiting to be realized. 我觉得有些灵感,幸运可以在设计未来的一小部分中发挥作用。I feel inspired and fortunate be able to play one small part in designing the future. 我希望将我的旅程加入三维空间!I hope you will join me on the journey into 3D space!

关于作者About the author

Picture of Hae Jin Lee Hae 金Hae Jin Lee
UX 设计器 @MicrosoftUX Designer @Microsoft