案例研究-为 HoloLens 构建 HoloSketch、空间布局和 UX 草绘应用Case study - Building HoloSketch, a spatial layout and UX sketching app for HoloLens

HoloSketch 是一种设备上的空间布局和适用于 HoloLens 的 UX 草图工具,用于帮助构建全息体验。HoloSketch is an on-device spatial layout and UX sketching tool for HoloLens to help build holographic experiences. HoloSketch 适用于配对的蓝牙键盘和鼠标以及笔势和语音命令。HoloSketch works with a paired Bluetooth keyboard and mouse as well as gesture and voice commands. HoloSketch 的目的是提供一个简单的 UX 布局工具,用于快速可视化和迭代。The purpose of HoloSketch is to provide a simple UX layout tool for quick visualization and iteration.

HoloSketch:适用于 HoloLens 的空间布局和 UX 草绘应用。HoloSketch: A spatial layout and UX sketching app for HoloLens.
HoloSketch:用于 HoloLens 的空间布局和 UX 草绘应用HoloSketch: spatial layout and UX sketching app for HoloLens

用于快速可视化和迭代的简单 UX 布局工具。A simple UX layout tool for quick visualization and iteration.
简单的 UX 布局工具,用于快速可视化和迭代A simple UX layout tool for quick visualization and iteration

功能Features

用于快速研究和规模原型构建的基元Primitives for quick studies and scale-prototyping

使用基元形状

将基元形状用于快速 massing 研究和规模原型设计。Use primitive shapes for quick massing studies and scale-prototyping.

通过 OneDrive 导入对象Import objects through OneDrive

导入对象

导入 PNG/JPG 图像或 3D FBX 对象 (需要通过 OneDrive 将 Unity) 打包到混合现实空间。Import PNG/JPG images or 3D FBX objects(requires packaging in Unity) into the mixed reality space through OneDrive.

操作对象Manipulate objects

操作对象

使用熟悉的3D 对象界面操作对象 (移动/旋转/缩放) 。Manipulate objects (move/rotate/scale) with a familiar 3D object interface.

蓝牙、鼠标/键盘、笔势和语音命令Bluetooth, mouse/keyboard, gestures and voice commands

支持蓝牙

HoloSketch 支持蓝牙鼠标/键盘、笔势和语音命令。HoloSketch supports Bluetooth mouse/keyboard, gestures and voice commands.

背景Background

在设备中体验设计的重要性Importance of experiencing your design in the device

为 HoloLens 设计内容时,在设备中体验设计很重要。When you design something for HoloLens, it is important to experience your design in the device. 混合现实应用程序设计面临的最大挑战之一是很难理解规模、位置和深度,尤其是通过传统的2D 草图。One of the biggest challenges in mixed reality app design is that it is hard to get the sense of scale, position and depth, especially through traditional 2D sketches.

基于2D 的通信的成本Cost of 2D based communication

为了有效地向他人传达 UX 流和方案,设计人员可能会花费大量时间,使用传统的2D 工具(例如 Illustrator、Photoshop 和 PowerPoint)来创建资产。To effectively communicate UX flows and scenarios to others, a designer may end up spending a lot of time creating assets using traditional 2D tools such as Illustrator, Photoshop and PowerPoint. 这些2D 设计通常需要额外的精力才能将其转换为三维。These 2D designs often require additional effort to convert them it into 3D. 从2D 到3D 的转换中,一些想法会丢失。Some ideas are lost in this translation from 2D to 3D.

复杂的部署过程Complex deployment process

由于混合现实是我们的新画布,因此它的本质涉及到许多设计迭代和试用和错误。Since mixed reality is a new canvas for us, it involves a lot of design iteration and trial and error by its nature. 对于不熟悉 Unity 和 Visual Studio 等工具的设计人员,在 HoloLens 中将某些内容组合在一起并不容易。For designer who are not familiar with tools such as Unity and Visual Studio, it is not easy to put something together in HoloLens. 通常,您必须完成以下过程才能在设备中查看您的 2D/3D 插图。Typically you have to go through the process below to see your 2D/3D artwork in the device. 这是一种非常大的障碍,设计人员能够快速循环访问创意和方案。This was a big barrier for designers iterating on ideas and scenarios quickly.

复杂的部署过程Complex deployment process
部署过程Deployment process

简化了 HoloSketch 的过程Simplified process with HoloSketch

通过 HoloSketch,我们想要简化此过程,而不涉及开发工具和设备门户配对。With HoloSketch, we wanted to simplify this process without involving development tools and device portal pairing. 使用 OneDrive,用户可以轻松地将 2D/3D 资产置于 HoloLens 中。Using OneDrive, users can easily put 2D/3D assets into HoloLens.

简化了 HoloSketch 的过程Simplified process with HoloSketch
简化了 HoloSketch 的过程Simplified process with HoloSketch

鼓励三维设计思想和解决方案Encouraging three-dimensional design thinking and solutions

我们认为,此工具可让设计人员在真正的三维空间中探索解决方案,而不是在2D 中。We thought this tool would give designers an opportunity to explore solutions in a truly three-dimensional space and not be stuck in 2D. 他们无需考虑为 UI 创建三维背景,因为在这种情况下,在 HoloLens 情况下,背景是真实世界。They don’t have to think about creating a 3D background for their UI since the background is the real world in the case of HoloLens. HoloSketch 打开了一种方法,使设计人员能够轻松地在 HoloLens 上浏览3D 设计。HoloSketch opens up a way for designers to easily explore 3D design on HoloLens.

开始使用Get Started

如何将 (JPG/PNG) 的2D 图像导入 HoloSketchHow to import 2D images (JPG/PNG) into HoloSketch

  • 将 JPG/PNG 图像上传到 OneDrive 文件夹的文档/HoloSketch。Upload JPG/PNG images to your OneDrive folder ‘Documents/HoloSketch’.
  • 在 HoloSketch 的 OneDrive 菜单中,可以选择映像并将其放置在环境中。From the OneDrive menu in HoloSketch, you will be able to select and place the image in the environment.

导入2D 图像Importing 2D images
通过 OneDrive 导入图像和3D 对象Importing images and 3D objects through OneDrive

如何将3D 对象导入 HoloSketchHow to import 3D objects into HoloSketch

在上传到 OneDrive 文件夹之前,请按照下面的步骤将3D 对象打包到 Unity 资产包。Before uploading to your OneDrive folder, please follow the steps below to package your 3D objects into a Unity asset bundle. 使用此过程可以从3D 软件(如 Maya、电影院4D 和 Microsoft 画图3D)导入 FBX/OBJ 文件。Using this process you can import your FBX/OBJ files from 3D software such as Maya, Cinema 4D and Microsoft Paint 3D.

重要

目前,Unity 版本第5.4.5 节 f1 支持创建资产包。Currently, asset bundle creation is supported with Unity version 5.4.5f1.

  1. 下载并打开 Unity 项目 "AssetBunlder_Unity"Download and open the Unity project 'AssetBunlder_Unity'. 此 Unity 项目包含捆绑资产生成的脚本。This Unity project includes the script for the bundle asset generation.

  2. 创建新的 GameObject。Create a new GameObject.

  3. 基于内容命名 GameObject。Name the GameObject based on the contents.

  4. 在检查器面板中,单击 "添加组件" 并添加 "Box 碰撞器"。In the Inspector panel, click ‘Add Component’ and add ‘Box Collider’.

    在检查器面板中,单击 "添加组件" 并添加 "Box 碰撞器"

    在检查器面板中,单击 "添加组件" 并添加 "Box 碰撞器" #2

  5. 通过将 3D FBX 文件拖到 "项目" 面板中,将其导入。Import the 3D FBX file by dragging it into the project panel.

  6. 将对象拖到 新 GameObject 下 的 "层次结构" 面板中。Drag the object into the Hierarchy panel under your new GameObject .

    将对象拖到新 GameObject 下的 "层次结构" 面板中。

  7. 如果碰撞器维度与对象不匹配,则调整它。Adjust the collider dimension if it does not match the object. 将对象旋转为面朝 Z 轴Rotate the object to face Z-axis .

    如果它与对象不匹配,则调整碰撞器维度。

  8. 将对象从 "层次结构" 面板拖到 "项目" 面板, 使其 prefabDrag the object from the Hierarchy panel to the Project panel to make it prefab .

  9. 在检查器面板的底部,单击下拉列表,创建并分配一个新的唯一名称。On the bottom of the inspector panel, click the dropdown, create and assign a new unique name. 下面的示例演示如何添加 "brownchair" 并将其分配给 AssetBundle 名称。Below example shows adding and assigning 'brownchair' for the AssetBundle Name.

    在检查器面板的底部,单击下拉列表并分配一个新的唯一名称。

  10. 为模型对象准备缩略图。Prepare a thumbnail image for the model object. 将一个图像拖到 "项目" 面板中,并分配用于该对象的名称。Drag an image into the project panel and assign the name used for the object.

  11. 在 Unity 项目的 "资产" 文件夹下创建一个名为 "Assetbundles" 的文件夹。Create a folder named ‘Assetbundles’ under the Unity project’s ‘Asset’ folder.

  12. 从 "资产" 菜单中选择 "生成 AssetBundles" 以生成文件。From the Assets menu, select ‘Build AssetBundles’ to generate file. 从 "资产" 菜单中选择 "生成 AssetBundles" 以生成文件。From the Assets menu, select ‘Build AssetBundles’ to generate file.

  13. 将生成的文件上传到 OneDrive 上的/Files/Documents/HoloSketch 文件夹。Upload the generated file to the /Files/Documents/HoloSketch folder on OneDrive. 仅上传 asset_unique_name 文件。Upload the asset_unique_name file only. 不需要上传清单文件或 AssetBundles 文件。You don’t need to upload manifest files or AssetBundles file.
    将文件添加到文件/文档/HoloSketch/文件夹在  HoloSketch 的 OneDrive 菜单中可以看到添加的3d 对象Add files to Files/Documents/HoloSketch/ folder You will see added 3D object in HoloSketch's OneDrive menu

如何操作对象How to manipulate the objects

HoloSketch 支持3D 软件广泛使用的传统接口。HoloSketch supports the traditional interface that is widely used in 3D software. 您可以使用 "移动"、"旋转"、"用手势和鼠标缩放对象"。You can use move, rotate, scale the objects with gestures and a mouse. 您可以通过语音或键盘在不同模式之间快速切换。You can quickly switch between different modes with voice or keyboard.

对象操作模式Object manipulation modes

如何操作对象How to manipulate the objects
如何操作对象How to manipulate the objects

上下文和工具皮带菜单Contextual and Tool Belt menus

使用上下文菜单Using the Contextual Menu

双击以打开上下文菜单。Double air tap to open the Contextual Menu.

菜单项:Menu items:

  • 布局图面: 这是一个三维网格系统,你可以在其中布局多个对象并将其作为一个组进行管理。Layout Surface: This is a 3D grid system where you can layout multiple objects and manage them as a group. 双击布局面以向其中添加对象。Double air-tap on the Layout Surface to add objects to it.
  • 基元: 使用多维数据集、球体、圆柱和圆锥进行 massing 研究。Primitives: Use cubes, spheres, cylinders and cones for massing studies.
  • OneDrive: 打开 OneDrive 菜单以导入对象。OneDrive: Open the OneDrive menu to import objects.
  • 帮助: 显示帮助屏幕。Help: Displays help screen.

上下文菜单Contextual menu
上下文菜单Contextual menu

使用工具皮带菜单Using the Tool Belt Menu

可从 "工具" 菜单使用 "移动"、"旋转"、"缩放"、"保存" 和 "加载" 场景。Move, Rotate, Scale, Save, and Load Scene are available from the Tool Belt Menu.

使用键盘、笔势和语音命令Using keyboard, gestures and voice commands

键盘、手势和语音命令Keyboard, Gestures and Voice Commands
键盘、笔势和语音命令Keyboard, gestures, and voice commands

下载应用Download the app

HoloSketch app icon 从 Microsoft Store 免费下载并安装 HoloSketch 应用 Download and install the HoloSketch app for free from the Microsoft Store

已知问题Known issues

  • Unity 版本第5.4.5 节 f1 支持当前创建的资产包。Currently asset bundle creation is supported with Unity version 5.4.5f1.
  • 根据 OneDrive 中的数据量,应用可能看起来就像在加载 OneDrive 内容时已停止Depending on the amount of data in your OneDrive, the app might appear as if it has stopped while loading OneDrive contents
  • 目前,保存和加载功能仅支持基元对象Currently, Save and Load feature only supports primitive objects
  • 在上下文菜单上禁用文本、声音、视频和照片菜单Text, Sound, Video and Photo menus are disabled on the contextual menu
  • 工具皮带菜单上的 "播放" 按钮将清除操作 gizmosThe Play button on the Tool Belt menu clears the manipulation gizmos

共享草图Sharing your sketches

可以通过说 "你好 Cortana,开始/停止记录" 来使用 HoloLens 中的视频录制功能。You can use the video recording feature in HoloLens by saying 'Hey Cortana, Start/Stop recording'. 同时按向上键和向下键,拍摄草图的图片。Press the volume up/down key together to take a picture of your sketch.

关于作者About the authors

Picture of Dong Yoon Park 盾 Yoon 停车场Dong Yoon Park
UX 设计器 @MicrosoftUX Designer @Microsoft
Picture of Patrick Sebring SebringPatrick Sebring
开发 @MicrosoftDeveloper @Microsoft