以不同的方式考虑混合现实Thinking differently for Mixed Reality

当 Microsoft 启动了 HoloLens (第一代) 为在2016的开发开发人员时,团队已经与 Microsoft 内部和外的工作室合作,以生成设备的启动体验。When Microsoft launched the HoloLens (1st gen) to eager developers in 2016, the team had already partnered with studios inside and out of Microsoft to build the device’s launch experiences. 这些团队边做边学,在混合现实设计的新领域中寻找机会和挑战。These teams learned by doing, finding both opportunities and challenges in the new field of mixed reality design.

我们将其方法和见解转变为一个课程,在我们的混合现实学院中讲授新开发人员。We turned their methods and insights into a curriculum that we teach new developers in our Mixed Reality Academy. 此课程包含我们向企业合作伙伴) 提供的周内设计研讨会。The curriculum includes week-long design workshops we offer to our enterprise partners).

下面是我们学习的所有内容的快照,旨在帮助企业合作伙伴为混合现实开发准备团队。Below is a snapshot of everything we've learned to help our enterprise partners prepare their teams for mixed reality development. 尽管许多这类方法最初面向 HoloLens 开发,但3D 中的应该构想和通信对各种混合现实体验非常重要。While many of these methods originally target HoloLens development, imagining and communicating in 3D are critical to the full spectrum of mixed reality experiences.

在设计过程中考虑空间Thinking spatially during the design process

任何设计过程或设计框架都可循环使用:要充分利用问题,若要评估这些想法并获得解决方案,请与他人进行沟通。Any design process or design framework is meant to iterate thinking: To approach a problem broadly, to share ideas with others effectively, to evaluate those ideas and reach a solution. 如今,我们提供了良好的设计和开发方法,用于在台式机、手机和平板电脑上构建体验。Today, we have well-established design and development methods for building experiences on desktops, phones, and tablets. 团队清楚地期望循环访问某一想法并将其转换为用户产品。Teams have clear expectations of what is necessary to iterate an idea and turn it into a product for users.

团队通常由开发、设计、研究和管理组成。Teams are often composed of a mix of development, design, research, and management. 所有角色都应参与设计过程。All roles should participate in the design process. 为移动应用程序提供理念的入门障碍非常简单,只是在设备屏幕上绘制矩形。The barrier to entry for contributing an idea for a mobile app is as simple as drawing a rectangle for the device’s screen. 同时,具有框和线条的草绘 UI 元素足以建立技术要求或定义潜在的用户行为。Meanwhile, sketching UI elements with boxes and lines is enough to establish technical requirements or define potential user behavior.

在混合现实中,传统的2D 设计过程开始细分。With mixed reality, the traditional 2D design process begins to break down. 对于大多数人而言,3D 中的草绘很难,而且使用笔和纸或白板等2D 工具通常可以将观点限制到这些尺寸。Sketching in 3D is difficult for most people, and using 2D tools like pen and paper or whiteboards, can often limit ideas to those dimensions. 同时,为游戏或工程设计构建的3D 工具需要高度的技能来迅速充实创意。Meanwhile, 3D tools, built for gaming or engineering, require a high degree of skill to quickly flesh out ideas. 新设备固有的技术不确定性(其中仍建立了基础交互方法)会进一步加剧了缺少轻型工具的情况。The lack of lightweight tools is compounded by the technical uncertainty inherent with new devices, where foundational interaction methods are still being established. 利用三维开发背景,这些挑战可能会限制团队成员的设计,大大降低了团队迭代的能力。These challenges can potentially limit the design contributions to team members with 3D development backgrounds — drastically reducing the team’s ability to iteration.

团队在混合现实学院中的讨论会工作Teams working during a workshop in the Mixed Reality Academy
我们研讨会中混合现实合作伙伴计划的团队Teams from the Mixed Reality Partner Program in our workshop

当我们与外部合作伙伴合作时,我们将听到团队的故事,等待开发人员完成原型,然后才能继续执行其设计过程。When we work with external partners, we hear stories of teams waiting for the developer to finish the prototype before they can continue their design process. 然后,团队的其余部分可能会被阻止,使其对产品产生有意义的进度。The rest of the team is then potentially blocked from making meaningful progress on the product. 开发人员最终承担了解决技术实现和用户体验的所有责任,因为他们试图在代码中添加大致的理念。Developers end up with all the responsibility for solving both the technical implementation and user experience as they attempt to put a rough idea into code.

用于展开设计过程的方法Techniques for expanding the design process

Microsoft 的团队提供了一系列技术来更有效地包括团队并快速循环访问复杂的设计问题。Our teams at Microsoft have a set of techniques to more effectively include their team and quickly iterate through complex design problems. 虽然这不是一种正式的过程,但这些技术旨在进行补充,而不是替换工作流。While not a formal process, these techniques are intended to supplement rather than replace your workflow. 通过这些方法,团队成员无需专门的3D 技能即可在深入研究原型阶段之前提供灵感。These methods allow team members without specialized 3D skills to offer ideas before diving into the prototyping phase. 不同的角色和学科可以是设计过程的一部分,而不只是3D 设计人员和开发人员。Different roles and disciplines can be part of the design process, not just 3D designers, and developers. 发现机会并确定可能的问题在开发后可能会发生。Uncovering opportunities and identifying possible challenges might occur later in development.

设计过程的技术

通过 bodystorming 生成创意Generating ideas with bodystorming

让团队考虑到现实世界中发生的事件,而不是传统的2D 设备,这是开发创新性的混合现实体验的关键所在。Getting your team to think about events occurring in the real world, beyond the traditional world of 2D devices, is key to developing innovative mixed reality experiences. 在 Microsoft,我们通过在实际空间中鼓励与物理属性的交互来找到最佳结果。At Microsoft, we've found the best results by encouraging interaction with physical props in a real-world space. 使用简单、廉价的构造材料,我们构建物理属性来表示数字对象、用户界面和动画,并获得建议的体验。Using simple, cheap crafting materials we build physical props to represent digital objects, user interfaces, and animations in a proposed experience. 这种技术称为 bodystorming,是一种在行业设计中对数十年的产品构思的装订。This technique is called bodystorming and has been a staple of product ideation within industrial design for decades.

团队在混合现实院校中使用材料在 bodystorming 中使用的简单、廉价的图片Teams working with materials in the Mixed Reality Academy Simple, cheap art supplies used in bodystorming

简单、物理属性级别的参与者的 "播放" 字段,允许具有不同技能集和背景的人员提供想法并发现混合现实体验固有的机会,而不是被锁定为2D 思维。Simple, physical props level the playing field for participants, allowing individuals with different skillsets and backgrounds to contribute ideas and uncover opportunities inherent to mixed reality experiences instead of being locked into 2D thinking. 虽然技术原型制作人员或高保真情节提要需要训练有素的三维开发人员或音乐家,但少数 Styrofoam 球和 cardboard 可以足以展示接口在物理空间中的弹出方式。While technical prototyping or high-fidelity storyboarding requires a skilled 3D developer or artist, a few Styrofoam balls and cardboard can be enough to showcase how an interface might unfold in physical space. 这些技术适用于使用 HoloLens 和沉浸式耳机进行混合现实开发。These techniques apply to both mixed reality development with HoloLens and the immersive headsets. 例如,一组塑料连接器可能会大致说明在 HoloLens 体验中显示的全息影像大小,或属性以在虚拟世界中处理种不可交互元素或运动设计。For example, a set of plastic connectors might roughly illustrate the size of holograms that appear in a HoloLens experience or as props to act out interactable elements or motion designs in a virtual world.

Bodystorming 是一种技术,用于快速生成创意,并评估过于含混的概念。Bodystorming is a technique used to quickly generate ideas and evaluate ideas that are too nebulous to prototype. 在 Microsoft,bodystorming 最常用于快速来审查。At Microsoft, bodystorming is most commonly used to quickly vet an idea. 如果你涉及到不熟悉混合现实开发的利益干系人,或者需要进行广泛的方案,它还可以帮助托管更深入的会话。It can also help host a more in-depth session if you involve outside stakeholders who aren't familiar with mixed reality development or need to distill broad scenarios. 请记住,bodystorming 的目的是通过鼓励参与者在空间上快速高效地进行 ideate。Remember that the purpose of bodystorming is to ideate quickly and efficiently by encouraging participants to think spatially. 在此阶段,详细的图稿或精确度量并不重要。Detailed artwork or precise measurements aren't important at this stage. 物理属性只需满足最小要求才能浏览或传达想法。Physical props need only meet the minimum requirements to explore or communicate an idea. 通过 bodystorming 介绍的想法不应完全审查,但该过程可帮助缩小在设备原型构建阶段期间进行测试的可能性。Ideas presented through bodystorming are not expected to be fully vetted, but the process can help narrow down possibilities to test later during the in-device prototyping phase. 这种情况下,bodystorming 不会取代技术原型,而是抵消在原型制作阶段解决技术和设计挑战的负担。As such, bodystorming doesn't replace technical prototyping, but rather offsets the burden of solving both technical and design challenges during the prototyping phase.

操作和专家反馈Acting and expert feedback

在现实世界中,bodystorming ideating 与物理对象结合使用后,下一步就是演练这些对象。Following the bodystorming process of ideating with physical objects in the real world, the next step is to walk through an experience with these objects. 我们称之为操作过程的这一阶段,这通常涉及到暂存用户将如何逐步完成经验或特定的交互。We call this phase of the process acting and it often involves staging how a user would move through the experience or a specific interaction.

团队在混合现实学院中的研讨会期间实施方案Teams acting out a scenario during a workshop in the Mixed Reality Academy
团队在研讨会期间担任方案Teams acting out a scenario during a workshop

借助物理属性,参与者可以体验用户的观点,同时允许外部观察者查看事件的播放方式。这为你提供了更多的时间,可为团队成员或可提供特定 "专家" 反馈的利益干系人提供更广泛的受众。Acting with physical props allows participants to experience the user's perspective, while allowing outside observers to see how events play out. This presents an ideal time to include a wider audience of team members or stakeholders who can provide specific ‘expert’ feedback. 例如,如果你要探索专为医院设计的混合现实体验,那么向医疗专业人员演出你的想法可提供宝贵的反馈。For example, if you're exploring a mixed reality experience designed for hospitals, acting out your thinking to a medical professional can provide invaluable feedback. 另一个例子是,您可能会遇到您要尝试了解的特定挑战,例如,空间音频或资产质量与性能的预算。Another example is when you might have a specific challenge you're trying to understand, like spatial audio or budgeting for asset quality vs. performance. 在这些情况下,it 人员使专家能够快速、粗略地了解体验的方式,而无需使用基于设备的原型。In these cases, acting gives experts a quick, rough concept of how the experience might unfold, without the need for a device-based prototype.

这种行为并不是正规的,无需引入专业人才,但如果我们想要与不存在的其他人分享想法,我们将录制 "场景" 交互,或包含情节提要艺术家来观察和草拟关键工作。This sort of acting isn't formalized, with no need to bring in professional talent, but in situations where we want to share the thinking with others who aren't present, we'll video record a ‘scene’ of interaction or include a storyboard artist to observe and sketch key moments. 操作也可以是轻型活动,在 bodystorming 阶段通常会原位。Acting can also be a lightweight activity, often happening in-situ during the bodystorming phase. 确定要使用哪种方法取决于需要使用哪种方法来提示适当类型的反馈。Deciding which method to use depends on the audience and the fidelity needed to prompt the appropriate type of feedback. 它最终会产生最大的影响,从而最有效地捕获团队思维。It ultimately comes down to whatever will most effectively capture your team’s thinking.

利用情节提要捕获观点Capturing ideas with storyboards

传达你的建议体验的观点和概念的最佳方法取决于你的目标受众,以及下一次迭代所需的反馈类型。The best method for conveying the ideas and concepts of your proposed experience depends on your intended audience, and the type of feedback your next iteration requires. 为团队成员提供新的想法时,bodystorming 的低保真重新 enactments 可能足以使某人的速度更快。When presenting new ideas to team members, low fidelity re-enactments of bodystorming can be enough to bring someone up to speed. 向新的利益干系人或潜在用户介绍你的体验概念时,最佳方法通常是情节提要。When introducing your experience concepts to new stakeholders or potential users, the best method is often storyboarding. 情节提要是娱乐行业的一项技术,可在电影和视频游戏开发的幕后找到。Storyboarding is a technique common to the entertainment industry, found behind the scenes in movies and video game development. 此方法可帮助同时) 在低 fidelities (的总体流,以及高 fidelities) 的美观外观 (。The technique helps convey both the overall flow of an experience (at low fidelities) and the aesthetic look and feel (at high fidelities). 与原型设计一样,了解情节提要的保真度需求是收集正确反馈和避免进行副署的关键需求。As with prototyping, understanding the fidelity needs of your storyboard is key to gathering the right feedback and avoiding counter-productive discussions.

低保真情节提要的示例Example of a low-fidelity storyboard
低保真情节提要的示例Example of a low-fidelity storyboard

低保真情节提要是快速讨论的正确保真度,尤其是在传达高级观点时。Low-fidelity storyboards are the right fidelity for quick discussions, especially when conveying high-level ideas. 它们可以像 "驾驶图" 图形和 "基元" 形状一样简单,以表示场景中的虚拟元素,或者物理和虚拟) (的交互式组件的邻近性。These can be as simple as stick-figure drawings and primitive shapes to denote virtual elements in a scene or the proximity of interactive components (both physical and virtual). 尽管这些方法对于执行的相对轻松和低技能障碍非常有用,但请记住 bodystorming 的课程:并非每个人都可以看到一种经验的2D 描述并了解三维含义。While these are useful given the relative ease and low skill barrier to execute, remember the lesson from bodystorming: Not everyone can see 2D depictions of an experience and understand the 3D implications.

高保真情节提要的示例Example of a high-fidelity storyboard
高保真情节提要的示例Example of a high-fidelity storyboard

高保真情节提要是一种功能强大的工具,当新的利益干系人或将见解从 bodystorming 会话合并到你的体验的提议美观方向时,这是一个强大的工具。High-fidelity storyboards are a powerful tool when bringing in new stakeholders or combining insights from a bodystorming session with the proposed aesthetic direction of your experience. 情节提要可以构建各种调式板来说明虚拟体验的最终外观,并捕获可能 pivotal 最终产品的关键时刻。Storyboards can build off mood boards to illustrate the final appearance of a virtual experience, and capture key moments that may be pivotal to the final product. 请记住,高保真情节提要通常需要一个艺术家,特别是团队中嵌入的一位,他们可以捕获难以理解的观点。Keep in mind that high-fidelity storyboards often require an artist, especially one embedded within the team, who can capture difficult-to-describe ideas. 在 Microsoft,我们已在游戏开发中添加了情节提要艺术家(通常带有背景),这是我们的混合现实团队,他们准备好快速捕获将在以后扩展到更高保真度原型的草图。At Microsoft, we have added storyboard artists, often with backgrounds in game development, to our mixed reality teams who attend meetings prepared to quickly capture sketches that will later be expanded into higher fidelity mockups. 这些人与技术艺术家密切合作,帮助传达最终体验中使用的资产的艺术方向。These individuals work closely with technical artists, helping to convey the artistic direction of assets used in the final experience.

设计过程Design process

如今,我们的过程从白板和不干胶便笺开始,展开为物理 bodystorming 并进行操作。Our process today starts with whiteboarding and sticky notes and expands into physical bodystorming and acting. 接下来,我们将这种想法转换为情节提要,而我们说的是 "单行概念" (是介绍我们所) 做的事情的高级句子,然后再转到原型制作和资产创建。We then translate that thinking into storyboards and something we call a 'One-Line Concept' (the high-level sentence that describes what we're making) before moving on to prototyping and asset creation. 在开发阶段之前,我们的设计工作通常是物理工作。Our design work is often physical before we get into the development phase.

设计流程1Design process flow 1
整体设计过程Overall design process

设计过程杂乱,每个团队都不同。The design process is messy and every team is different. 不过,bodystorming、操作和情节提要等方法有助于缓解迭代中的困难。However, methods like bodystorming, acting, and storyboarding can help alleviate difficulties in iteration.

设计流程2Design process flow 2
事实上,它的连续迭代阶段In reality, its continuous iterative phases

拓展团队技能Expanding your team's skills

更改进程以更好地适应混合现实体验取决于更密切地连接技术和设计角色。Altering your process to better accommodate mixed reality experiences depends on more closely connecting technical and design roles. 与 Microsoft 合作伙伴合作,我们已了解到多个团队已成功过渡到三维开发环境,我们发现,最大的优势在于,团队成员从他们的舒适区域跳出并发展技能,使他们能够在整个开发过程中获得更多的参与 (不只是将他们的技能限制为) 设计或开发。Working with partners at Microsoft, we've seen several teams successfully transition into 3D development, and we find the biggest advantage comes from team members stepping out of their comfort zone and developing skills that give them more involvement throughout the development process (not just limiting their skills to design or development).

反馈和迭代是任何成功设计的关键,并且通过混合现实体验,这种反馈通常会扭曲更多的技术概念,特别是在 nascency 混合现实的技术和工具的相关情况下。Feedback and iteration are key to any successful design, and with mixed reality experiences, this sort of feedback often skews to more technical concepts, especially given the relative nascency of mixed reality’s technology and tools. 构建设计团队在技术领域的技能,包括与 Unity 或 Unreal 等工具相关的技能,有助于这些人员更好地了解开发人员的方法,并在创建原型期间提供更有效的反馈。Building your design team’s skills in technical areas, including skills with tools like Unity or Unreal, helps those individuals better understand the approach of developers and provide more effective feedback during prototyping. 同样,了解混合现实体验的基本 UX 概念 (和潜在的设计缺陷) 的开发人员可以帮助他们在规划的早期阶段提供实现见解。Similarly, developers who understand the fundamental UX concepts of mixed reality experiences (and potential design pitfalls) can help them offer implementation insights during the early phases of planning.

我们通常会从合作伙伴那里获得一些问题,这些问题涉及到如何最好地利用三维工具发展团队和技能集。We usually get questions from partners working in mixed reality about how best to grow their team and skillsets with 3D tools. 由于行业的状态,这是一个很难回答的问题,因为团队正在使用昨天的工具构建明天的经验。This is a difficult question to answer given the state of the industry, as teams are faced with building tomorrow’s experiences with yesterday’s tools. 许多团队正在收集游戏和娱乐中具有背景的个人,其中,3D 开发已成为数十年的书。Many teams are gathering individuals with backgrounds in gaming and entertainment, where 3D development has been a staple for decades. 团队还鼓励其现有团队选取 Unity 和3D 建模软件等工具。Teams are also encouraging their existing teams to pick up tools like Unity and 3D modeling software. 尽管这两个都是满足当今混合现实经验的基准的基本需求,但很可能有大量专门的工具和技能可帮助构建明天的各种应用程序。While these are both essential needs for meeting the baseline of mixed reality experiences today, there will likely be a great number of specialized tools and skills to help build tomorrow’s ground-breaking applications.

混合现实设计的领域始终在不断变化,重点在于丰富的数据和云服务。The field of mixed reality design is always changing, with more focus on data and cloud services in holographic experiences. 下一代体验将利用高级的自然语言处理和实时计算机视觉,并且更多原本是沉浸式体验中的社交方案。This next generation of experiences will take advantage of advanced natural language processing and real-time computer vision, and more inherently social scenarios within immersive experiences. 例如,利用虚拟现实中的体验,设计人员可以构建创新的环境和空间,需要更多的技能,而不是传统的界面设计。For example, experiences in virtual reality allow designers to build imaginative environments and spaces, requiring a skill set more akin to architecture or psychology rather than traditional interface design. 或者,您可以考虑在现实世界中体验到 HoloLens,其中,用例通常涉及高度专用的字段(如医学、工程或制造),其中特定环境和实时数据是体验的重要组成部分。Or you can consider experiences in the real world with HoloLens, where the use cases often involve highly specialized fields such as medicine, engineering, or manufacturing, where specific environments and real-time data are essential elements of the experience. 跨角色工作,使用设计和专业知识,并获得了解新工具的意愿,对于混合现实中的团队而言是非常有用的。Working across roles, using both design and specialized knowledge, and having a willingness to learn new tools are invaluable skill for teams working in mixed reality.

帮助你的团队快速高效地浏览混合现实Helping your team explore mixed reality quickly and effectively

在 HoloLens 的早期阶段,这些技术是不必要的,因为设备的早期原型硬件被证明是低于理想的格式,可以快速地循环访问设计灵感。In the early days of HoloLens, these techniques came out of necessity as the device’s early prototype hardware proved to be a less than ideal format for quickly iterating through design ideas. 交互方法(包括操作系统中的基本 UI)已在 bodystorming 中探索,同时有助于我们的可用性团队了解工作区中方案的基本用户行为。Interaction methods, including fundamental UI in the operating system were explored with bodystorming, while acting helped our usability team understand fundamental user behaviors for scenarios in the workplace. 最终,团队为 HoloLens 核心 UX 确立了强大的基线,跨团队角色传达了概念,并允许操作系统开发与硬件开发并行移动。In the end, the team established a strong baseline for the HoloLens core UX, communicating concepts across team roles and allowing OS development to move quickly in parallel with hardware development.

混合现实学院中的研讨会材料

无论是 bodystorming、正在咨询专家还是情节提要,这些技术都旨在节省时间和精力。Whether it's bodystorming, acting, reviewing with experts, or storyboarding, these techniques are intended to save you time and effort. 尽管我们在 HoloLens 和虚拟现实开发的早期阶段仍非常多,但扩大混合现实的设计过程有助于您的团队花精力浏览新的、挑战的设计问题,而不是克服传达观点的难题。While we're still very much in the early days of HoloLens and virtual reality development, expanding your design process for mixed reality will help your team spend their energy exploring new and challenging design problems rather than overcoming difficulties communicating ideas.

讨论会提供示例列表Sample list of workshop supplies

简单的艺术用品为团队成员提供必要的工具来说明没有任何高级艺术技能的创意。Simple art supplies provide team members with the necessary tools to explain ideas without any advanced artistic skills. 下面是我们的团队在 bodystorming 期间通常使用的示例:Here's a sample of what our team commonly uses during bodystorming:

  • Styrofoam 光盘Styrofoam discs
  • Styrofoam 多维数据集Styrofoam cubes
  • Styrofoam 圆锥Styrofoam cones
  • Styrofoam 球体Styrofoam spheres
  • Cardboard 框Cardboard boxes
  • 木板 dowelsWooden dowels
  • 棒糖形棒Lollipop sticks
  • CardstockCardstock
  • 纸张 cupPaper cups
  • 管道磁带Duct tape
  • 屏蔽磁带Masking tape
  • 剪刀Scissors
  • PaperclipsPaperclips
  • 存档夹紧Filing clamps
  • Post-其Post-Its
  • TwineTwine
  • 铅笔Pencils
  • SharpiesSharpies

下一个发现检查点Next Discovery Checkpoint

恭喜,你已完成了混合现实研究旅程的核心!Congratulations, you've completed the core of the Mixed Reality discovery journey! 你可以 了解行业合作伙伴是如何使用混合现实探索 HoloLens,还是开始设计或开发旅程:You can either see how industry partners are using Mixed Reality, explore the HoloLens, or start your design or development journeys: