MR 空间 220:空间音效MR Spatial 220: Spatial sound

备注

混合现实学院教程在制作时考虑到了 HoloLens(第一代)和混合现实沉浸式头戴显示设备。The Mixed Reality Academy tutorials were designed with HoloLens (1st gen) and Mixed Reality Immersive Headsets in mind. 因此,对于仍在寻求这些设备的开发指导的开发人员而言,我们觉得很有必要保留这些教程。As such, we feel it is important to leave these tutorials in place for developers who are still looking for guidance in developing for those devices. 我们 不会 在这些教程中更新 HoloLens 2 所用的最新工具集或集成相关的内容。These tutorials will not be updated with the latest toolsets or interactions being used for HoloLens 2. 我们将维护这些教程,使之持续适用于支持的设备。They will be maintained to continue working on the supported devices. 已经为 HoloLens 2 发布了一系列新教程A new series of tutorials has been posted for HoloLens 2.

空间音效 breathes 影像,并使其在世界中存在。Spatial sound breathes life into holograms and gives them presence in our world. 全息影像同时包含光和声音,如果你发生丢失全息影像的情况,空间音效可以帮助你找到它们。Holograms are composed of both light and sound, and if you happen to lose sight of your holograms, spatial sound can help you find them. 空间音效并不像您在广播上听到的典型声音,而是位于3D 空间中的声音。Spatial sound is not like the typical sound that you would hear on the radio, it is sound that is positioned in 3D space. 利用空间音效,你可以制作出全息影像,就像你,甚至是你自己的背景上!With spatial sound, you can make holograms sound like they're behind you, next to you, or even on your head! 在本课程中,你将:In this course, you will:

  • 将你的开发环境配置为使用 Microsoft 空间音质。Configure your development environment to use Microsoft Spatial Sound.
  • 使用空间音效来增强交互。Use Spatial Sound to enhance interactions.
  • 将空间音效与 空间映射结合使用。Use Spatial Sound in conjunction with Spatial Mapping.
  • 了解合理设计和混合最佳实践。Understand sound design and mixing best practices.
  • 使用声音增强特殊效果,并使用户进入混合现实世界。Use sound to enhance special effects and bring the user into the Mixed Reality world.

设备支持Device support

课程Course HoloLensHoloLens 沉浸式头戴显示设备Immersive headsets
MR 空间 220:空间音效MR Spatial 220: Spatial sound ✔️✔️ ✔️✔️

准备工作Before you start

必备条件Prerequisites

项目文件Project files

  • 下载项目所需的 文件Download the files required by the project. 需要 Unity 2017.2 或更高版本。Requires Unity 2017.2 or later.
    • 如果仍需要 Unity 5.6 支持,请使用 此版本If you still need Unity 5.6 support, please use this release. 此版本可能不再是最新版本。This release may no longer be up-to-date.
    • 如果仍需要 Unity 5.5 支持,请使用 此版本If you still need Unity 5.5 support, please use this release. 此版本可能不再是最新版本。This release may no longer be up-to-date.
    • 如果仍需要 Unity 5.4 支持,请使用 此版本If you still need Unity 5.4 support, please use this release. 此版本可能不再是最新版本。This release may no longer be up-to-date.
  • 取消将文件存档到桌面或其他易于访问的位置。Un-archive the files to your desktop or other easy to reach location.

备注

如果要在下载之前查看源代码, 可在 GitHub 上找到。If you want to look through the source code before downloading, it's available on GitHub.

勘误表和说明Errata and Notes

  • 需要在 Visual Studio 的 "工具"-">选项->调试" 中禁用 "启用仅我的代码" (取消选中) ,以便在代码中命中断点。"Enable Just My Code" needs to be disabled (unchecked) in Visual Studio under Tools->Options->Debugging in order to hit breakpoints in your code.

第1章-Unity 设置Chapter 1 - Unity Setup

目标Objectives

  • 更改 Unity 的声音配置,以使用 Microsoft 空间音质。Change Unity's sound configuration to use Microsoft Spatial Sound.
  • 将三维声音添加到 Unity 中的对象。Add 3D sound to an object in Unity.

说明Instructions

  • 启动 Unity。Start Unity.
  • 选择“打开” 。Select Open.
  • 导航到桌面并找到以前未存档的文件夹。Navigate to your Desktop and find the folder you previously un-archived.
  • 单击 " Starting\Decibel " 文件夹,然后按 " 选择文件夹 " 按钮。Click on the Starting\Decibel folder and then press the Select Folder button.
  • 等待项目在 Unity 中加载。Wait for the project to load in Unity.
  • 在 " 项目 " 面板中,打开 Scenes\Decibel.unityIn the Project panel, open Scenes\Decibel.unity.
  • 在 " 层次结构 " 面板中,展开 " HologramCollection " 并选择 " P0LY"。In the Hierarchy panel, expand HologramCollection and select P0LY.
  • 在检查器中,展开 " AudioSource ",注意没有 " Spatialize " 复选框。In the Inspector, expand AudioSource and notice that there is no Spatialize check box.

默认情况下,Unity 不会加载 spatializer 插件。By default, Unity does not load a spatializer plugin. 以下步骤将在项目中启用空间音质。The following steps will enable Spatial Sound in the project.

  • 在 Unity 的顶部菜单中,参阅 编辑 > 音频 > 项目设置In Unity's top menu, go to Edit > Project Settings > Audio.
  • 找到 Spatializer 插件 下拉列表,并选择 " MS HRTF Spatializer"。Find the Spatializer Plugin dropdown, and select MS HRTF Spatializer.
  • 在 " 层次结构 " 面板中,选择 " HologramCollection > P0LY"。In the Hierarchy panel, select HologramCollection > P0LY.
  • 在 " 检查器 " 面板中,找到 " 音频源 " 组件。In the Inspector panel, find the Audio Source component.
  • 选中 " Spatialize " 复选框。Check the Spatialize checkbox.
  • 空间混合 滑块一直拖到 3d 上,或在编辑框中输入 1Drag the Spatial Blend slider all the way to 3D, or enter 1 in the edit box.

现在,我们将在 Unity 中生成项目,并在 Visual Studio 中配置该解决方案。We will now build the project in Unity and configure the solution in Visual Studio.

  1. 在 Unity 中,选择 " 文件 > 生成设置"。In Unity, select File > Build Settings.
  2. 单击 " 添加打开的场景 " 添加场景。Click Add Open Scenes to add the scene.
  3. 选择 "平台" 列表中的 "通用 Windows 平台",然后单击 "切换平台"。Select Universal Windows Platform in the Platform list and click Switch Platform.
  4. 如果要专门针对 HoloLens 进行开发,请将 " 目标设备 " 设置为 " hololens"。If you're specifically developing for HoloLens, set Target device to HoloLens. 否则,请将其留在 任何设备 上。Otherwise, leave it on Any device.
  5. 确保将 " 生成类型 " 设置为 " D3D ",并将 " Sdk " 设置为 " 最新安装 的 (,这应是 SDK 16299 或更高) 版本Ensure Build Type is set to D3D and SDK is set to Latest installed (which should be SDK 16299 or newer).
  6. 单击“生成”。Click Build.
  7. 创建名为 "App" 的 新文件夹Create a New Folder named "App".
  8. 单击 应用 文件夹。Single click the App folder.
  9. 按 " 选择文件夹"。Press Select Folder.

当 Unity 完成后,将显示文件资源管理器窗口。When Unity is done, a File Explorer window will appear.

  1. 打开 应用程序 文件夹。Open the App folder.
  2. 打开 " 分贝 Visual Studio 解决方案"。Open the Decibel Visual Studio Solution.

如果部署到 HoloLens:If deploying to HoloLens:

  1. 使用 Visual Studio 中的顶部工具栏,将目标从 "调试" 更改为 " 发布 ",将 "从 ARM" 更改为 " x86"。Using the top toolbar in Visual Studio, change the target from Debug to Release and from ARM to x86.
  2. 单击 "本地计算机" 按钮旁的下拉箭头,然后选择 " 远程计算机"。Click on the drop down arrow next to the Local Machine button, and select Remote Machine.
  3. 输入 HoloLens 设备 IP 地址 ,并将身份验证模式设置为 通用 (未加密协议)Enter your HoloLens device IP address and set Authentication Mode to Universal (Unencrypted Protocol). 单击“选择” 。Click Select. 如果你不知道设备 IP 地址,请在 "设置" 中查找 " > 网络 & Internet > 高级选项"。If you do not know your device IP address, look in Settings > Network & Internet > Advanced Options.
  4. 在顶部菜单栏中,单击 " 调试-> 启动而不调试 " 或按 Ctrl + F5In the top menu bar, click Debug -> Start Without debugging or press Ctrl + F5. 如果这是首次部署到设备,则需要将 其与 Visual Studio 配对If this is the first time deploying to your device, you will need to pair it with Visual Studio.

如果要部署到沉浸式耳机:If deploying to an immersive headset:

  1. 使用 Visual Studio 中的顶部工具栏,将目标从 "调试" 更改为 " 发布 ",将 "从 ARM" 更改为 " x64"。Using the top toolbar in Visual Studio, change the target from Debug to Release and from ARM to x64.
  2. 确保将部署目标设置为 " 本地计算机"。Make sure the deployment target is set to Local Machine.
  3. 在顶部菜单栏中,单击 " 调试-> 启动而不调试 " 或按 Ctrl + F5In the top menu bar, click Debug -> Start Without debugging or press Ctrl + F5.

第2章-空间音效和交互Chapter 2 - Spatial Sound and Interaction

目标Objectives

  • 使用声音提高全息图的真实感。Enhance hologram realism using sound.
  • 使用声音定向用户的注视。Direct the user's gaze using sound.
  • 使用声音提供手势反馈。Provide gesture feedback using sound.

第1部分-增强真实性Part 1 - Enhancing Realism

关键概念Key Concepts

  • Spatialize 全息影像声音。Spatialize hologram sounds.
  • 应将声音源置于全息图上的适当位置。Sound sources should be placed at an appropriate location on the hologram.

声音的适当位置将取决于全息图。The appropriate location for the sound is going to depend on the hologram. 例如,如果全息影像是人,则声音源应位于嘴附近,而不是英尺附近。For example, if the hologram is of a human, the sound source should be located near the mouth and not the feet.

说明Instructions

以下说明将 spatialized 声音附加到全息图。The following instructions will attach a spatialized sound to a hologram.

  • 在 " 层次结构 " 面板中,展开 " HologramCollection " 并选择 " P0LY"。In the Hierarchy panel, expand HologramCollection and select P0LY.
  • 在 " 检查器 " 面板的 AudioSource 中,单击 " AudioClip " 旁边的圆圈,并从弹出窗口中选择 " PolyHover "。In the Inspector panel, in the AudioSource, click the circle next to AudioClip and select PolyHover from the pop-up.
  • 单击 " 输出 " 旁边的圆圈,并从弹出窗口中选择 " SoundEffects "。Click the circle next to Output and select SoundEffects from the pop-up.

项目分贝使用 Unity AudioMixer 组件来启用调整声音组的音量级别。Project Decibel uses a Unity AudioMixer component to enable adjusting sound levels for groups of sounds. 通过以这种方式对声音进行分组,可以在保持每个声音的相对音量的同时调整总体音量。By grouping sounds this way, the overall volume can be adjusted while maintaining the relative volume of each sound.

  • AudioSource 中,展开 " 三维声音设置"。In the AudioSource, expand 3D Sound Settings.
  • Doppler 级别 设置为 0Set Doppler Level to 0.

如果将 Doppler 级别设置为零,则将禁用由 (全息图或用户) 中的运动引起的螺距更改。Setting Doppler level to zero disables changes in pitch caused by motion (either of the hologram or the user). Doppler 的典型示例是一种快速移动的汽车。A classic example of Doppler is a fast-moving car. 当汽车接近固定的侦听器时,引擎的跨度会上升。As the car approaches a stationary listener, the pitch of the engine rises. 当它通过侦听程序时,间距会降低距离。When it passes the listener, the pitch lowers with distance.

第2部分-定向用户的注视Part 2 - Directing the User's Gaze

关键概念Key Concepts

  • 使用声音吸引重要的全息影像。Use sound to call attention to important holograms.
  • 此耳有助于指导眼睛的外观。The ears help direct where the eyes should look.
  • 大脑有一些预期的预期。The brain has some learned expectations.

了解预期的一个示例就是,鸟瞰一般都是人的水平。One example of learned expectations is that birds are generally above the heads of humans. 如果用户听到了鸟瞰声,其初始反应就是查找。If a user hears a bird sound, their initial reaction is to look up. 如果在用户下放置鸟,会使其面向正确的声音方向,但无法根据需要查找的预期来找到全息影像。Placing a bird below the user can lead to them facing the correct direction of the sound, but being unable to find the hologram based on the expectation of needing to look up.

说明Instructions

以下说明允许 P0LY 隐藏在您后面,以便您可以使用声音查找全息图。The following instructions enable P0LY to hide behind you, so that you can use sound to locate the hologram.

  • 在 " 层次结构 " 面板中,选择 " 管理器"。In the Hierarchy panel, select Managers.
  • 检查器 面板中,找到 " 语音输入处理程序"。In the Inspector panel, find Speech Input Handler.
  • 语音输入处理程序 中,展开 " 转到隐藏"。In Speech Input Handler, expand Go Hide.
  • No 函数 更改为 PolyActions. GoHideChange No Function to PolyActions.GoHide.

关键字:中转隐藏

第3部分-手势反馈Part 3 - Gesture Feedback

关键概念Key Concepts

  • 使用声音为用户提供积极的手势确认Provide the user with positive gesture confirmation using sound
  • 不要严重影响用户的声音,Do not overwhelm the user - overly loud sounds get in the way
  • 微妙的声音效果最好-不要会掩盖体验Subtle sounds work best - do not overshadow the experience

说明Instructions

  • 在 " 层次结构 " 面板中,展开 " HologramCollection"。In the Hierarchy panel, expand HologramCollection.
  • 展开 " EnergyHub ",然后选择 " 基本"。Expand EnergyHub and select Base.
  • 检查器 面板中,单击 " 添加组件 " 并添加 手势声音处理程序In the Inspector panel, click Add Component and add Gesture Sound Handler.
  • 手势声音处理程序 中,单击 " 导航已启动的剪辑导航已更新的剪辑 " 旁边的圆圈,并从弹出窗口中选择 " RotateClick "。In Gesture Sound Handler, click the circle next to Navigation Started Clip and Navigation Updated Clip and select RotateClick from the pop-up for both.
  • 双击 "GestureSoundHandler" 以在 Visual Studio 中加载。Double click on "GestureSoundHandler" to load in Visual Studio.

手势声音处理程序执行以下任务:Gesture Sound Handler performs the following tasks:

  • 创建和配置 AudioSourceCreate and configure an AudioSource.
  • AudioSource 放在适当 GameObject 的位置。Place the AudioSource at the location of the appropriate GameObject.
  • 播放与该笔势关联的 AudioClipPlays the AudioClip associated with the gesture.

生成和部署Build and Deploy

  1. 在 Unity 中,选择 " 文件 > 生成设置"。In Unity, select File > Build Settings.
  2. 单击“生成”。Click Build.
  3. 单击 应用 文件夹。Single click the App folder.
  4. 按 " 选择文件夹"。Press Select Folder.

检查工具栏是否显示 "发布"、"x86" 或 "x64",以及 "远程设备"。Check that the Toolbar says "Release", "x86" or "x64", and "Remote Device". 如果不是,则这是 Visual Studio 的代码实例。If not, this is the coding instance of Visual Studio. 可能需要从应用程序文件夹重新打开解决方案。You may need to re-open the solution from the App folder.

  • 如果系统提示,请重新加载项目文件。If prompted, reload the project files.
  • 如前所述,从 Visual Studio 部署。As before, deploy from Visual Studio.

部署应用程序后:After the application is deployed:

  • 观察在 P0LY 移动时声音如何变化。Observe how the sound changes as you move around P0LY.
  • "转到隐藏" ,使 P0LY 移到你后面的位置。Say "Go Hide" to make P0LY move to a location behind you. 通过声音查找。Find it by the sound.
  • 注视能源中心的底部。Gaze at the base of the Energy Hub. 点击并向左或向右拖动以旋转全息影像,并注意单击声音如何确认手势。Tap and drag left or right to rotate the hologram and notice how the clicking sound confirms the gesture.

注意:有一个文本面板会与你一起标记。Note: There is a text panel that will tag-along with you. 这将包含可在本课程中使用的可用语音命令。This will contain the available voice commands that you can use throughout this course.

第3章-空间音质和空间映射Chapter 3 - Spatial Sound and Spatial Mapping

目标Objectives

  • 使用声音确认全息影像与真实环境之间的交互。Confirm interaction between holograms and the real world using sound.
  • 使用实物遮蔽声音。Occlude sound using the physical world.

第1部分-物理世界交互Part 1 - Physical World Interaction

关键概念Key Concepts

  • 当遇到图面或其他对象时,物理对象通常会发出声音。Physical objects generally make a sound when encountering a surface or another object.
  • 声音应在体验中是适当的上下文。Sounds should be context appropriate within the experience.

例如,对表设置杯会使声音更安静,而不是在金属片上放置 boulder。For example, setting a cup on a table should make a quieter sound than dropping a boulder on a piece of metal.

说明Instructions

  • 在 " 层次结构 " 面板中,展开 " HologramCollection"。In the Hierarchy panel, expand HologramCollection.
  • 展开 " EnergyHub",选择 " 基本"。Expand EnergyHub, select Base.
  • 在 " 检查器 " 面板中,单击 " 添加组件 ",然后单击 "添加 声音和操作"。In the Inspector panel, click Add Component and add Tap To Place With Sound and Action.
  • 在中, 单击以放置声音和操作In Tap To Place With Sound and Action:
    • 在点击时选中 "父项"Check Place Parent On Tap.
    • 设置要 放置放置声音Set Placement Sound to Place.
    • 拾取声音 设置为 装货Set Pickup Sound to Pickup.
    • 在 " 分拣" 操作"放置" 操作 中,按右下方的 "+"。Press the + in the bottom right under both On Pickup Action and On Placement Action. 将 EnergyHub 从场景拖入 None (对象) 字段。Drag EnergyHub from the scene into the None (Object) fields.
      • "分拣操作" 下,单击 "无函数 -> EnergyHubBase -> ResetAnimation"。Under On Pickup Action, click on No Function -> EnergyHubBase -> ResetAnimation.
      • "放置时" 操作 下,单击 "无函数 -> EnergyHubBase -> OnSelect"。Under On Placement Action, click on No Function -> EnergyHubBase -> OnSelect.

点击以放入声音和操作

第2部分-声音封闭Part 2 - Sound Occlusion

关键概念Key Concepts

  • 声音,如光源,可以是封闭像素。Sound, like light, can be occluded.

典型的示例是音乐会厅。A classic example is a concert hall. 当某个侦听器在大厅外且门闭合时,音乐听起来 muffled。When a listener is standing outside of the hall and the door is closed, the music sounds muffled. 通常还会减少容量。There is also typically a reduction in volume. 打开门后,就会听到真实音量的所有声音。When the door is opened, the full spectrum of the sound is heard at the actual volume. 高频声音通常会超出低频率。High frequency sounds are generally absorbed more than low frequencies.

说明Instructions

  • 在 " 层次结构 " 面板中,展开 " HologramCollection " 并选择 " P0LY"。In the Hierarchy panel, expand HologramCollection and select P0LY.
  • 检查器 面板中,单击 " 添加组件 " 并添加 音频发射器In the Inspector panel, click Add Component and add Audio Emitter.

音频发射器类提供以下功能:The Audio Emitter class provides the following features:

  • 还原对 AudioSource 卷的任何更改。Restores any changes to the volume of the AudioSource.
  • 按照 AudioEmitter 连接到的 GameObject 的方向,从用户的位置执行 RaycastNonAllocPerforms a Physics.RaycastNonAlloc from the user's position in the direction of the GameObject to which the AudioEmitter is attached.

RaycastNonAlloc 方法用作性能优化,以限制分配和返回的结果数。The RaycastNonAlloc method is used as a performance optimization to limit allocations as well as the number of results returned.

  • 对于遇到的每个 IAudioInfluencer ,将调用 ApplyEffect 方法。For each IAudioInfluencer encountered, calls the ApplyEffect method.
  • 对于不再遇到的每个以前的 IAudioInfluencer ,请调用 RemoveEffect 方法。For each previous IAudioInfluencer that is no longer encountered, call the RemoveEffect method.

请注意,AudioEmitter 对人为时间刻度的更新,而不是每个框架的更新。Note that AudioEmitter updates on human time scales, as opposed to on a per frame basis. 这样做是因为,人们通常不能以足够快的速度移动,因此,需要比每个季度或半秒钟的频率更频繁地进行更新。This is done because humans generally do not move fast enough for the effect to need to be updated more frequently than every quarter or half of a second. 从一个位置到另一个位置快速传送的全息影像可能会突破错觉。Holograms that teleport rapidly from one location to another can break the illusion.

  • 在 " 层次结构 " 面板中,展开 " HologramCollection"。In the Hierarchy panel, expand HologramCollection.
  • 展开 " EnergyHub " 并选择 " BlobOutside"。Expand EnergyHub and select BlobOutside.
  • 检查器 面板中,单击 " 添加组件 " 并添加 音频 OccluderIn the Inspector panel, click Add Component and add Audio Occluder.
  • 音频 Occluder 中,将 " 截止频率 " 设置为 1500In Audio Occluder, set Cutoff Frequency to 1500.

此设置将 AudioSource 频率限制为 1500 Hz 和更低。This setting limits the AudioSource frequencies to 1500 Hz and below.

  • Volume Pass 设置为 0.9Set Volume Pass Through to 0.9.

此设置将 AudioSource 的量降低到其当前级别的90%。This setting reduces the volume of the AudioSource to 90% of it's current level.

音频 Occluder 实现 IAudioInfluencer:Audio Occluder implements IAudioInfluencer to:

  • 使用附加到 AudioSource 托管的 AudioLowPassFilter 的封闭 效果。Apply an occlusion effect using an AudioLowPassFilter which gets attached to the AudioSource managed buy the AudioEmitter.
  • 将卷衰减应用于 AudioSource。Applies volume attenuation to the AudioSource.
  • 通过设置非特定截止频率并禁用筛选器来禁用该效果。Disables the effect by setting a neutral cutoff frequency and disabling the filter.

用于中性的频率为 22 kHz (22000 Hz) 。The frequency used as neutral is 22 kHz (22000 Hz). 选择此频率的原因是,它的最大频率为人体耳可以听到的最大公称,这不会影响声音。This frequency was chosen due to it being above the nominal maximum frequency that can be heard by the human ear, this making no discernable impact to the sound.

  • 在 " 层次结构 " 面板中,选择 " SpatialMapping"。In the Hierarchy panel, select SpatialMapping.
  • 检查器 面板中,单击 " 添加组件 " 并添加 音频 OccluderIn the Inspector panel, click Add Component and add Audio Occluder.
  • 音频 Occluder 中,将 " 截止频率 " 设置为 750In Audio Occluder, set Cutoff Frequency to 750.

如果多个 occluders 位于用户与 AudioEmitter 之间的路径中,则会将最低频率应用于筛选器。When multiple occluders are in the path between the user and the AudioEmitter, the lowest frequency is applied to the filter.

  • Volume Pass 设置为 0.75Set Volume Pass Through to 0.75.

如果多个 occluders 位于用户与 AudioEmitter 之间的路径中,则会应用添加性地。When multiple occluders are in the path between the user and the AudioEmitter, the volume pass through is applied additively.

  • 在 " 层次结构 " 面板中,选择 " 管理器"。In the Hierarchy panel, select Managers.
  • 在 " 检查器 " 面板中,展开 " 语音输入处理程序"。In the Inspector panel, expand Speech Input Handler.
  • 语音输入处理程序 中,展开 " 转到"。In Speech Input Handler, expand Go Charge.
  • No 函数 更改为 PolyActions. GoChargeChange No Function to PolyActions.GoCharge.

关键字:中转

  • 在此处 展开。Expand Come Here.
  • No 函数 更改为 PolyActions. 卷土重来Change No Function to PolyActions.ComeBack.

关键字:此处提供

生成和部署Build and Deploy

  • 如前所述,在 Unity 中生成项目,并在 Visual Studio 中部署。As before, build the project in Unity and deploy in Visual Studio.

部署应用程序后:After the application is deployed:

  • "走电" ,让 P0LY 进入能量中心。Say "Go Charge" to have P0LY enter the Energy Hub.

请注意声音的变化。Note the change in the sound. 这听起来 muffled。It should sound muffled and a little quieter. 如果你能够在你与能源中心之间定位墙壁或其他对象,你应该注意到 muffling 的声音,因为现实世界封闭了。If you are able to position yourself with a wall or other object between you and the Energy Hub, you should notice a further muffling of the sound due to the occlusion by the real world.

  • "现在 就可以",让 P0LY 离开能源中心,并将其放在您前面。Say "Come Here" to have P0LY leave the Energy Hub and position itself in front of you.

请注意,一旦 P0LY 退出能源中心,就会删除声音封闭。Note that the sound occlusion is removed once P0LY exits the Energy Hub. 如果你仍在封闭,P0LY 可能会被现实世界封闭像素。If you are still hearing occlusion, P0LY may be occluded by the real world. 尝试移动以确保对 P0LY 有清楚的视觉。Try moving to ensure you have a clear line of sight to P0LY.

第3部分-房间模型Part 3 - Room Models

关键概念Key Concepts

  • 空间大小提供了 subliminal 的队列,它们有助于进行合理的本地化。The size of the space provides subliminal queues that contribute to sound localization.
  • 房间模型是按 AudioSource 设置的。Room models are set per-AudioSource.
  • MixedRealityToolkit For Unity提供用于设置房间模型的代码。The MixedRealityToolkit for Unity provides code for setting the room model.
  • 对于混合现实体验,请选择最适合于现实世界空间的房间模型。For Mixed Reality experiences, select the room model that best fits the real world space.

如果要创建虚拟现实方案,请选择最适合虚拟环境的房间模型。If you are creating a Virtual Reality scenario, select the room model that best fits the virtual environment.

第4章-声音设计Chapter 4 - Sound Design

目标Objectives

  • 了解有效的声音设计的注意事项。Understand considerations for effective sound design.
  • 了解混合技巧和指导原则。Learn mixing techniques and guidelines.

第1部分-声音和体验设计Part 1 - Sound and Experience Design

本部分介绍关键的声音,并体验设计注意事项和指导原则。This section discusses key sound and experience design considerations and guidelines.

规范化所有声音Normalize all sounds

这样就无需使用特殊案例代码调整每个声音的音量级别,这可能非常耗时,并且限制了轻松更新声音文件的能力。This avoids the need for special case code to adjust volume levels per sound, which can be time consuming and limits the ability to easily update sound files.

设计 untethered 体验Design for an untethered experience

HoloLens 是完全包含的 untethered 全息计算机。HoloLens is a fully contained, untethered holographic computer. 你的用户可以在移动时使用你的体验。Your users can and will use your experiences while moving. 务必通过浏览来测试您的音频组合。Be sure to test your audio mix by walking around.

从逻辑位置在全息影像上发出声音Emit sound from logical locations on your holograms

在现实生活中,狗不会吠其尾部,人类的语音不会来自其英尺。In the real world, a dog does not bark from its tail and a human's voice does not come from his/her feet. 避免在全息影像的意外部分发出声音。Avoid having your sounds emit from unexpected portions of your holograms.

对于小全息影像,从几何图形中心开始发出声音是合理的。For small holograms, it is reasonable to have sound emit from the center of the geometry.

熟悉的声音是最可本地化的Familiar sounds are most localizable

人为语音和音乐非常易于本地化。The human voice and music are very easy to localize. 如果有人调用了您的姓名,则可以从语音的方向和距离。If someone calls your name, you are able to very accurately determine from what direction the voice came and from how far away. 简短,不熟悉的声音更难本地化。Short, unfamiliar sounds are harder to localize.

Cognizant 用户期望Be cognizant of user expectations

生活经验使我们能够确定声音位置。Life experience plays a part in our ability to identify the location of a sound. 这就是人们语音特别容易本地化的原因之一。This is one reason why the human voice is particularly easy to localize. 发出声音时,请注意用户的预期预期。It is important to be aware of your user's learned expectations when placing your sounds.

例如,当有人听到了他们通常会查找的鸟瞰歌曲时,就像鸟瞰 (飞出或) 树中一样。For example, when someone hears a bird song they generally look up, as birds tend to be above the line of sight (flying or in a tree). 用户打开正确的声音方向并不是很常见,而是在无法找到全息影像时在错误的垂直方向上看起来不确定的。It is not uncommon for a user to turn in the correct direction of a sound, but look in the wrong vertical direction and become confused or frustrated when they are unable to find the hologram.

避免隐藏发射器Avoid hidden emitters

在实际情况下,如果听到声音,通常可以识别出发出声音的对象。In the real world, if we hear a sound, we can generally identify the object that is emitting the sound. 在您的体验中,这也应为 true。This should also hold true in your experiences. 用户听听声音非常不安,知道声音出自何处,看不到对象。It can be very disconcerting for users to hear a sound, know from where the sound originates and be unable to see an object.

此准则有一些例外情况。There are some exceptions to this guideline. 例如,字段中的 crickets 等环境声音无需可见。For example, ambient sounds such as crickets in a field need not be visible. 生活经验为我们熟悉这些声音的来源,无需查看。Life experience gives us familiarity with the source of these sounds without the need to see it.

第2部分-混音Part 2 - Sound Mixing

在 HoloLens 上将混合目标设定为70% 的卷Target your mix for 70% volume on the HoloLens

混合现实体验允许在现实世界中查看全息影像。Mixed Reality experiences allow holograms to be seen in the real world. 它们还应该允许听真实世界声音。They should also allow real world sounds to be heard. 利用70% 的音量目标,用户可以在他们周围倾听世界,并获得你的体验。A 70% volume target enables the user to hear the world around them along with the sound of your experience.

100% 卷上的 HoloLens 应 drown 外部声音HoloLens at 100% volume should drown out external sounds

100% 的卷级别与虚拟现实体验类似。A volume level of 100% is akin to a Virtual Reality experience. 用户在视觉上传输到不同的世界。Visually, the user is transported to a different world. 相同的呼叫时应为 true。The same should hold true audibly.

使用 Unity AudioMixer 调整声音类别Use the Unity AudioMixer to adjust categories of sounds

设计你的组合时,创建声音类别并能够将其音量增加或减少为一个单元通常很有帮助。When designing your mix, it is often helpful to create sound categories and have the ability to increase or decrease their volume as a unit. 这会保留每个声音的相对级别,同时使整体组合的快速而简单的更改。This retains the relative levels of each sound while enabling quick and easy changes to the overall mix. 常见类别包括:声音效果、环境、语音转移和背景音乐。Common categories include; sound effects, ambience, voice overs and background music.

基于用户的注视混合声音Mix sounds based on the user's gaze

根据用户 (的位置或不) 查找,通常可以根据用户的工作情况更改声音混合。It can often be useful to change the sound mix in your experience based on where a user is (or is not) looking. 此方法的一个常见用途是降低全息帧之外的全息影像的音量级别,使用户能够更轻松地将重点放在其前面的信息上。One common use for this technique are to reduce the volume level for holograms that are outside of the Holographic Frame to make it easier for the user to focus on the information in front of them. 另一种用途是增加声音量,以吸引用户关注重要事件。Another use is to increase the volume of a sound to draw the user's attention to an important event.

构建混合Building your mix

在构建组合时,建议从体验背景音频开始,并根据重要性添加层。When building your mix, it is recommended to start with your experience's background audio and add layers based on importance. 通常,这会导致每个层都大于上一个层。Often, this results in each layer being louder than the previous.

应该构想你的混合为反转漏斗,其中最不重要的 (,通常 quietest 的声音在底部) ,建议采用类似于下图的方式来构建组合。Imagining your mix as an inverted funnel, with the least important (and generally quietest sounds) at the bottom, it is recommended to structure your mix similar to the following diagram.

声音混合结构

语音转移是一个有趣的方案。Voice overs are an interesting scenario. 根据你所创建的体验,你可能希望使用立体声 (未) 声音或 spatialize 你的语音转移。Based on the experience you are creating you may wish to have a stereo (not localized) sound or to spatialize your voice overs. 两个 Microsoft 发布的体验演示了每个方案的极佳示例。Two Microsoft published experiences illustrate excellent examples of each scenario.

HoloTour 使用立体声声音。HoloTour uses a stereo voice over. 当讲述人描述正在查看的位置时,声音是一致的,并且不会根据用户的位置而变化。When the narrator is describing the location being viewed, the sound is consistent and does not vary based on the user's position. 这使讲述人可以在不离开环境 spatialized 声音的情况下描述场景。This enables the narrator to describe the scene without taking away from the spatialized sounds of the environment.

片段 使用 spatialized 的语音。Fragments utilizes a spatialized voice over in the form of a detective. 侦探的声音用于帮助用户关注重要的线索,就像实际人在房间里。The detective's voice is used to help bring the user's attention to an important clue as if an actual human was in the room. 这样,就可以更好地浸入式解决谜的体验。This enables an even greater sense of immersion into the experience of solving the mystery.

第3部分-性能Part 3 -Performance

CPU 使用率CPU usage

使用空间音效时,10-12 发射器会消耗约12% 的 CPU。When using Spatial Sound, 10 - 12 emitters will consume approximately 12% of the CPU.

流式传输长音频文件Stream long audio files

音频数据可能很大,尤其是在 (44.1 和 48 kHz) 的常见采样速率。Audio data can be large, especially at common sample rates (44.1 and 48 kHz). 一般规则是,应流式传输超过 5-10 秒的音频文件,以降低应用程序内存使用量。A general rule is that audio files longer than 5 - 10 seconds should be streamed to reduce application memory usage.

在 Unity 中,可以在文件的导入设置中标记音频文件以进行流式传输。In Unity, you can mark an audio file for streaming in the file's import settings.

音频导入设置

第5章-特殊效果Chapter 5 - Special Effects

目标Objectives

  • 将深度添加到 "幻窗口"。Add depth to "Magic Windows".
  • 将用户带入虚拟世界。Bring the user into the virtual world.

魔术窗口Magic Windows

关键概念Key Concepts

  • 将视图创建到隐藏世界中,这在视觉上非常引人注目。Creating views into a hidden world, is visually compelling.
  • 当全息图或用户处于隐藏世界附近时,通过添加音频效果来增强真实性。Enhance realism by adding audio effects when a hologram or the user is near the hidden world.

说明Instructions

  • 在 " 层次结构 " 面板中,展开 " HologramCollection " 并选择 " Underworld"。In the Hierarchy panel, expand HologramCollection and select Underworld.
  • 展开 " Underworld " 并选择 " VoiceSource"。Expand Underworld and select VoiceSource.
  • 检查器 面板中,单击 " 添加组件 " 并添加 " 用户语音效果"。In the Inspector panel, click Add Component and add User Voice Effect.

AudioSource 组件将添加到 VoiceSource 中。An AudioSource component will be added to VoiceSource.

  • AudioSource 中,将 " 输出 " 设置为 UserVoice (混音器)In AudioSource, set Output to UserVoice (Mixer).
  • 选中 " Spatialize " 复选框。Check the Spatialize checkbox.
  • 空间混合 滑块一直拖到 3d 上,或在编辑框中输入 1Drag the Spatial Blend slider all the way to 3D, or enter 1 in the edit box.
  • 展开 " 三维声音设置"。Expand 3D Sound Settings.
  • Doppler 级别 设置为 0Set Doppler Level to 0.
  • 在 " 用户语音效果" 中,将 父对象 从场景中设置为 UnderworldIn User Voice Effect, set Parent Object to the Underworld from the scene.
  • 最大距离 设置为 1Set Max Distance to 1.

设置 最大距离 会告诉 用户语音效果 在启用此效果之前,用户必须先到父对象。Setting Max Distance tells User Voice Effect how close the user must be to the parent object before the effect is enabled.

  • 在 " 用户语音效果" 中,展开 " Chorus 参数"。In User Voice Effect, expand Chorus Parameters.
  • 深度 设置为 0.1Set Depth to 0.1.
  • 依次点击 " 1"、" 2 卷 " 和 " 3 卷0.8"。Set Tap 1 Volume, Tap 2 Volume and Tap 3 Volume to 0.8.
  • 原始 音量设置为 0.5Set Original Sound Volume to 0.5.

以前的设置配置用于向用户语音添加丰富的 Unity AudioChorusFilter 的参数。The previous settings configure the parameters of the Unity AudioChorusFilter used to add richness to the user's voice.

  • 在 " 用户语音效果" 中,展开 " Echo Parameters"。In User Voice Effect, expand Echo Parameters.
  • 延迟 设置为 300Set Delay to 300
  • 衰减比率 设置为 0.2Set Decay Ratio to 0.2.
  • 原始 音量设置为 0Set Original Sound Volume to 0.

以前的设置配置用于使用户的语音回显的 Unity AudioEchoFilter 的参数。The previous settings configure the parameters of the Unity AudioEchoFilter used to cause the user's voice to echo.

用户语音效果脚本负责:The User Voice Effect script is responsible for:

  • 测量用户与脚本附加到的 GameObject 之间的距离。Measuring the distance between the user and the GameObject to which the script is attached.
  • 确定用户是否面向 GameObjectDetermining whether or not the user is facing the GameObject.

对于启用的效果,用户必须面向 GameObject,而不考虑距离。The user must be facing the GameObject, regardless of distance, for the effect to be enabled.

  • AudioChorusFilterAudioEchoFilter 应用和配置到 AudioSourceApplying and configuring an AudioChorusFilter and an AudioEchoFilter to the AudioSource.
  • 禁用筛选器以禁用该效果。Disabling the effect by disabling the filters.

用户语音效果使用 MixedRealityToolkit For Unity中的 Mic Stream 选择器组件选择高质量的语音流并将其路由到 Unity 的音频系统。User Voice Effect uses the Mic Stream Selector component, from the MixedRealityToolkit for Unity, to select the high quality voice stream and route it into Unity's audio system.

  • 在 " 层次结构 " 面板中,选择 " 管理器"。In the Hierarchy panel, select Managers.
  • 在 " 检查器 " 面板中,展开 " 语音输入处理程序"。In the Inspector panel, expand Speech Input Handler.
  • 语音输入处理程序 中,展开 Show UnderworldIn Speech Input Handler, expand Show Underworld.
  • No 函数 更改为 UnderworldBase. OnEnableChange No Function to UnderworldBase.OnEnable.

关键字: Show Underworld

  • 展开 " 隐藏 Underworld"。Expand Hide Underworld.
  • No 函数 更改为 UnderworldBase. OnDisableChange No Function to UnderworldBase.OnDisable.

关键字:隐藏 Underworld

生成和部署Build and Deploy

  • 如前所述,在 Unity 中生成项目,并在 Visual Studio 中部署。As before, build the project in Unity and deploy in Visual Studio.

部署应用程序后:After the application is deployed:

  • 面部 (墙壁、楼层、桌子) 并说 "显示 Underworld"Face a surface (wall, floor, table) and say "Show Underworld".

将显示 underworld,所有其他全息影像都将隐藏。The underworld will be shown and all other holograms will be hidden. 如果看不到 underworld,请确保您面对的是实际的表面。If you do not see the underworld, ensure that you are facing a real-world surface.

  • Underworld 全息图1米内的方法,开始谈话。Approach within 1 meter of the underworld hologram and start talking.

现在音频效果适用于你的语音!There are now audio effects applied to your voice!

  • 退出 underworld 并注意如何不再应用该效果。Turn away from the underworld and notice how the effect is no longer applied.
  • "隐藏 Underworld" 以隐藏 Underworld。Say "Hide Underworld" to hide the underworld.

Underworld 将隐藏,并且以前隐藏的全息影像会重新出现。The underworld will be hidden and the previously hidden holograms will reappear.

结束The End

恭喜!Congratulations! 你现在已经完成了 MR 空间220:空间音质You have now completed MR Spatial 220: Spatial sound.

听世界,让你的体验生活生动!Listen to the world and bring your experiences to life with sound!