MR Spatial 220:空間音效MR Spatial 220: Spatial sound

注意

混合實境學院教學課程的設計是以 HoloLens (第 1 代) 和混合實境沉浸式頭戴裝置為準。The Mixed Reality Academy tutorials were designed with HoloLens (1st gen) and Mixed Reality Immersive Headsets in mind. 因此,對於仍在尋找這些裝置開發指引的開發人員而言,我們覺得這些教學課程很重要。As such, we feel it is important to leave these tutorials in place for developers who are still looking for guidance in developing for those devices. 這些教學課程 不會 使用用於 HoloLens 2 的最新工具組或互動進行更新。These tutorials will not be updated with the latest toolsets or interactions being used for HoloLens 2. 系統會保留這些資訊,以繼續在支援的裝置上運作。They will be maintained to continue working on the supported devices. 已針對 HoloLens 2 公佈一系列新的教學課程A new series of tutorials has been posted for HoloLens 2.

空間音效 breathes 生活進入全像全球,並讓他們在世界各地都存在。Spatial sound breathes life into holograms and gives them presence in our world. 全像投影是由光線和音效所組成,如果您似乎無法看見您的全像影像,空間音效可以協助您找到它們。Holograms are composed of both light and sound, and if you happen to lose sight of your holograms, spatial sound can help you find them. 空間音效與您在無線電上聽到的一般音效不一樣,它是位於3D 空間的音效。Spatial sound is not like the typical sound that you would hear on the radio, it is sound that is positioned in 3D space. 有了空間音效,您就可以讓像是在您後方、您或甚至是在您的頭上的全像投影一樣:With spatial sound, you can make holograms sound like they're behind you, next to you, or even on your head! 在此課程中,您將會:In this course, you will:

  • 設定您的開發環境以使用 Microsoft 空間音效。Configure your development environment to use Microsoft Spatial Sound.
  • 使用空間音效來增強互動。Use Spatial Sound to enhance interactions.
  • 使用空間音效搭配 空間對應Use Spatial Sound in conjunction with Spatial Mapping.
  • 瞭解音效設計和混合最佳做法。Understand sound design and mixing best practices.
  • 使用音效強化特殊效果,並讓使用者進入混合現實世界。Use sound to enhance special effects and bring the user into the Mixed Reality world.

裝置支援Device support

課程Course HoloLensHoloLens 沉浸式頭戴裝置Immersive headsets
MR Spatial 220:空間音效MR Spatial 220: Spatial sound ✔️✔️ ✔️✔️

在您開始使用 Intune 之前Before you start

必要條件Prerequisites

專案檔Project files

  • 下載專案 所需的 檔案。Download the files required by the project. 需要 Unity 2017.2 或更新版本。Requires Unity 2017.2 or later.
    • 如果您仍然需要 Unity 5.6 支援,請使用 此版本If you still need Unity 5.6 support, please use this release. 此版本可能不再是最新版本。This release may no longer be up-to-date.
    • 如果您仍然需要 Unity 5.5 支援,請使用 此版本If you still need Unity 5.5 support, please use this release. 此版本可能不再是最新版本。This release may no longer be up-to-date.
    • 如果您仍然需要 Unity 5.4 支援,請使用 此版本If you still need Unity 5.4 support, please use this release. 此版本可能不再是最新版本。This release may no longer be up-to-date.
  • 取消將檔案封存到您的桌面或其他易於觸及的位置。Un-archive the files to your desktop or other easy to reach location.

注意

如果您想要在下載之前查看原始程式碼, 可在 GitHub 上取得。If you want to look through the source code before downloading, it's available on GitHub.

勘誤表和記事Errata and Notes

  • 您必須在 [工具]-[>選項] 下的 Visual Studio 中停用 [啟用 Just My Code] ( 核取) ,才能在程式碼中叫用中斷點。"Enable Just My Code" needs to be disabled (unchecked) in Visual Studio under Tools->Options->Debugging in order to hit breakpoints in your code.

第1章-Unity 設定Chapter 1 - Unity Setup

目標Objectives

  • 變更 Unity 的音效設定以使用 Microsoft 空間音效。Change Unity's sound configuration to use Microsoft Spatial Sound.
  • 將3D 音效新增至 Unity 中的物件。Add 3D sound to an object in Unity.

指示Instructions

  • 啟動 Unity。Start Unity.
  • 選取 [開啟] 。Select Open.
  • 流覽至您的桌面,並尋找您先前取消封存的資料夾。Navigate to your Desktop and find the folder you previously un-archived.
  • 按一下 [ Starting\Decibel ] 資料夾,然後按下 [ 選取資料夾 ] 按鈕。Click on the Starting\Decibel folder and then press the Select Folder button.
  • 等候專案在 Unity 中載入。Wait for the project to load in Unity.
  • 在 [ 專案 ] 面板中,開啟 [ Scenes\Decibel.unity]。In the Project panel, open Scenes\Decibel.unity.
  • 在 [階層] 面板中,展開 [ HologramCollection ],然後選取 [ P0LY]。In the Hierarchy panel, expand HologramCollection and select P0LY.
  • 在 [偵測器] 中,展開 [ spatialize ],並注意 [沒有 Spatialize ] 核取方塊。In the Inspector, expand AudioSource and notice that there is no Spatialize check box.

根據預設,Unity 不會載入空間定位器外掛程式。By default, Unity does not load a spatializer plugin. 下列步驟將會在專案中啟用空間音效。The following steps will enable Spatial Sound in the project.

  • 在 Unity 的頂端功能表中,移至 [ 編輯 > 專案設定 > 音訊]。In Unity's top menu, go to Edit > Project Settings > Audio.
  • 尋找 [ 空間定位器外掛程式 ] 下拉式清單,然後選取 [ MS HRTF 空間定位器]。Find the Spatializer Plugin dropdown, and select MS HRTF Spatializer.
  • 在 [階層] 面板中,選取 [ HologramCollection > P0LY]。In the Hierarchy panel, select HologramCollection > P0LY.
  • 在 [偵測 ] 面板中,尋找 音訊來源 元件。In the Inspector panel, find the Audio Source component.
  • 核取 [ Spatialize ] 核取方塊。Check the Spatialize checkbox.
  • 空間 Blend 滑杆拖曳至 3d,或在編輯方塊中輸入 1Drag the Spatial Blend slider all the way to 3D, or enter 1 in the edit box.

我們現在會在 Unity 中建立專案,並在 Visual Studio 中設定解決方案。We will now build the project in Unity and configure the solution in Visual Studio.

  1. 在 Unity 中,選取 [ File > Build Settings]。In Unity, select File > Build Settings.
  2. 按一下 [ 新增開啟場景 ] 以加入場景。Click Add Open Scenes to add the scene.
  3. 在 [平臺] 清單中選取 通用 Windows 平臺,然後按一下 [切換平臺]。Select Universal Windows Platform in the Platform list and click Switch Platform.
  4. 如果您是特別針對 HoloLens 進行開發,請將 目標裝置 設定為 hololensIf you're specifically developing for HoloLens, set Target device to HoloLens. 否則,請將它保留在 任何裝置 上。Otherwise, leave it on Any device.
  5. 請確定 組建類型 設定為 [ D3D ],並將 [ Sdk ] 設定為 [ 最新安裝 的 (,其應為 SDK 16299 或更新版本的) 。Ensure Build Type is set to D3D and SDK is set to Latest installed (which should be SDK 16299 or newer).
  6. 按一下 [建置]。Click Build.
  7. 建立名為 "App" 的 新資料夾Create a New Folder named "App".
  8. 按一下 應用程式 資料夾。Single click the App folder.
  9. 按下 [ 選取資料夾]。Press Select Folder.

當 Unity 完成時,將會出現檔案總管視窗。When Unity is done, a File Explorer window will appear.

  1. 開啟 應用程式 資料夾。Open the App folder.
  2. 開啟 [ 分貝 Visual Studio] 解決方案Open the Decibel Visual Studio Solution.

如果要部署到 HoloLens:If deploying to HoloLens:

  1. 使用 Visual Studio 中的頂端工具列,將目標從 Debug 變更為 Release ,以及從 ARM 變更為 x86Using the top toolbar in Visual Studio, change the target from Debug to Release and from ARM to x86.
  2. 按一下 [本機電腦] 按鈕旁的下拉箭號,然後選取 [ 遠端電腦]。Click on the drop down arrow next to the Local Machine button, and select Remote Machine.
  3. 輸入 您的 HoloLens 裝置 IP 位址 ,並將驗證模式設定為 通用 (未加密的通訊協定)Enter your HoloLens device IP address and set Authentication Mode to Universal (Unencrypted Protocol). 按一下 [選取]。Click Select. 如果您不知道您的裝置 IP 位址,請查看 [ 設定] > Network & Internet > Advanced 選項If you do not know your device IP address, look in Settings > Network & Internet > Advanced Options.
  4. 在頂端功能表列中,按一下 [ Debug-> 啟動但不進行調試 ],或按 Ctrl + F5In the top menu bar, click Debug -> Start Without debugging or press Ctrl + F5. 如果這是您第一次部署至您的裝置,您必須將 它與 Visual Studio 配對If this is the first time deploying to your device, you will need to pair it with Visual Studio.

如果部署到沉浸式耳機:If deploying to an immersive headset:

  1. 使用 Visual Studio 中的頂端工具列,將目標從 Debug 變更為 Release ,以及從 ARM 變更為 x64Using the top toolbar in Visual Studio, change the target from Debug to Release and from ARM to x64.
  2. 請確定部署目標設定為 [ 本機電腦]。Make sure the deployment target is set to Local Machine.
  3. 在頂端功能表列中,按一下 [ Debug-> 啟動但不進行調試 ],或按 Ctrl + F5In the top menu bar, click Debug -> Start Without debugging or press Ctrl + F5.

第2章-空間音效和互動Chapter 2 - Spatial Sound and Interaction

目標Objectives

  • 使用音效增強全像配音。Enhance hologram realism using sound.
  • 使用音效指示使用者的注視。Direct the user's gaze using sound.
  • 使用音效提供手勢意見反應。Provide gesture feedback using sound.

第1部分-增強真實感Part 1 - Enhancing Realism

重要概念Key Concepts

  • Spatialize 全像音效。Spatialize hologram sounds.
  • 音效來源應放置於全息圖的適當位置。Sound sources should be placed at an appropriate location on the hologram.

音效的適當位置將會取決於全像影像。The appropriate location for the sound is going to depend on the hologram. 比方說,如果全息圖是人類的,則音效來源應該位於嘴附近,而不是距離。For example, if the hologram is of a human, the sound source should be located near the mouth and not the feet.

指示Instructions

下列指示會將 hrtf 音效附加至全像影像。The following instructions will attach a spatialized sound to a hologram.

  • 在 [階層] 面板中,展開 [ HologramCollection ],然後選取 [ P0LY]。In the Hierarchy panel, expand HologramCollection and select P0LY.
  • 在 [偵測 ] 面板的 [ spatialize] 中,按一下 [ AudioClip ] 旁的圓形,然後從快顯視窗中選取 [ PolyHover ]。In the Inspector panel, in the AudioSource, click the circle next to AudioClip and select PolyHover from the pop-up.
  • 按一下 [ 輸出 ] 旁的圓形,然後從快顯視窗中選取 [ SoundEffects ]。Click the circle next to Output and select SoundEffects from the pop-up.

專案分貝會使用 Unity AudioMixer 元件來調整音效群組的音效等級。Project Decibel uses a Unity AudioMixer component to enable adjusting sound levels for groups of sounds. 藉由以這種方式將聲音分組,可以調整整體磁片區,同時維持每個音效的相對音量。By grouping sounds this way, the overall volume can be adjusted while maintaining the relative volume of each sound.

  • spatialize 中,展開 [ 3d 音效設定]。In the AudioSource, expand 3D Sound Settings.
  • Doppler 層級 設定為 0Set Doppler Level to 0.

將 Doppler 層級設定為 [零] 會停用 (由全像影像或使用者) 的移動所造成的音調變化。Setting Doppler level to zero disables changes in pitch caused by motion (either of the hologram or the user). Doppler 的典型範例是快速移動的汽車。A classic example of Doppler is a fast-moving car. 當汽車採用固定的接聽程式時,引擎的音調也會上升。As the car approaches a stationary listener, the pitch of the engine rises. 當它通過接聽程式時,會降低距離。When it passes the listener, the pitch lowers with distance.

第2部分-引導使用者的注視Part 2 - Directing the User's Gaze

重要概念Key Concepts

  • 使用音效呼叫重要的全像投影。Use sound to call attention to important holograms.
  • 此耳可協助引導眼睛的外觀。The ears help direct where the eyes should look.
  • 大腦有一些學習的期望。The brain has some learned expectations.

瞭解預期的其中一個範例是,鳥通常會高於人類的標題。One example of learned expectations is that birds are generally above the heads of humans. 如果使用者聽到鳥音效,他們的初始反應就是查閱。If a user hears a bird sound, their initial reaction is to look up. 將鳥放在使用者的下方,可能會讓他們面對正確的音效方向,但根據預期需要查閱,找不到影像。Placing a bird below the user can lead to them facing the correct direction of the sound, but being unable to find the hologram based on the expectation of needing to look up.

指示Instructions

下列指示可讓 P0LY 在您後方隱藏,讓您可以使用音效找出全像影像。The following instructions enable P0LY to hide behind you, so that you can use sound to locate the hologram.

  • 在 [階層] 面板中,選取 [管理員]。In the Hierarchy panel, select Managers.
  • 在 [偵測 ] 面板中,尋找 [ 語音輸入處理常式]。In the Inspector panel, find Speech Input Handler.
  • 在 [ 語音輸入處理常式] 中,展開 [ 移至隱藏]。In Speech Input Handler, expand Go Hide.
  • 函數 變更為 PolyActions. GoHideChange No Function to PolyActions.GoHide.

關鍵字:移至隱藏

第3部分-手勢意見反應Part 3 - Gesture Feedback

重要概念Key Concepts

  • 使用音效為使用者提供正面的手勢確認Provide the user with positive gesture confirmation using sound
  • 不要讓使用者過高的聲音變得太過Do not overwhelm the user - overly loud sounds get in the way
  • 微妙的音效效果最佳-請勿失色經驗Subtle sounds work best - do not overshadow the experience

指示Instructions

  • 在 [階層] 面板中,展開 [ HologramCollection]。In the Hierarchy panel, expand HologramCollection.
  • 展開 [ EnergyHub ],然後選取 [ 基底]。Expand EnergyHub and select Base.
  • 在 [偵測 ] 面板中,按一下 [ 新增元件 ] 並加入 手勢音效處理常式In the Inspector panel, click Add Component and add Gesture Sound Handler.
  • 在 [ 手勢音效處理常式] 中,按一下 [ 導覽開始的剪輯 ] 和 [ 導覽更新的剪輯 ] 旁的圓形,然後從快顯視窗中選取 [ RotateClick ]。In Gesture Sound Handler, click the circle next to Navigation Started Clip and Navigation Updated Clip and select RotateClick from the pop-up for both.
  • 按兩下 [GestureSoundHandler] 以 Visual Studio 載入。Double click on "GestureSoundHandler" to load in Visual Studio.

手勢音效處理常式會執行下列工作:Gesture Sound Handler performs the following tasks:

  • 建立並設定 spatializeCreate and configure an AudioSource.
  • spatialize 放在適當 GameObject 的位置。Place the AudioSource at the location of the appropriate GameObject.
  • 播放與手勢相關聯的 AudioClipPlays the AudioClip associated with the gesture.

建置和部署Build and Deploy

  1. 在 Unity 中,選取 [ File > Build Settings]。In Unity, select File > Build Settings.
  2. 按一下 [建置]。Click Build.
  3. 按一下 應用程式 資料夾。Single click the App folder.
  4. 按下 [ 選取資料夾]。Press Select Folder.

確認工具列中顯示的是「發行」、「x86」、「x64」和「遠端裝置」。Check that the Toolbar says "Release", "x86" or "x64", and "Remote Device". 如果沒有,這就是 Visual Studio 的編碼實例。If not, this is the coding instance of Visual Studio. 您可能需要從應用程式資料夾重新開啟方案。You may need to re-open the solution from the App folder.

  • 出現提示時,請重載專案檔。If prompted, reload the project files.
  • 同樣地,從 Visual Studio 部署。As before, deploy from Visual Studio.

部署應用程式之後:After the application is deployed:

  • 觀察當您在 P0LY 上移動時的音效變化。Observe how the sound changes as you move around P0LY.
  • 說「 Go 隱藏 」讓 P0LY 移至您後方的位置。Say "Go Hide" to make P0LY move to a location behind you. 依音效找出它。Find it by the sound.
  • 注視能源中樞的基底。Gaze at the base of the Energy Hub. 您可以用滑鼠左鍵或向右拖曳來旋轉全像投影,然後留意一下按一下音效如何確認手勢。Tap and drag left or right to rotate the hologram and notice how the clicking sound confirms the gesture.

注意:有一個文字面板會與您一起標記。Note: There is a text panel that will tag-along with you. 這將包含可在整個課程中使用的語音命令。This will contain the available voice commands that you can use throughout this course.

第3章-空間音效和空間對應Chapter 3 - Spatial Sound and Spatial Mapping

目標Objectives

  • 使用音效確認全像全息和真實世界之間的互動。Confirm interaction between holograms and the real world using sound.
  • 使用實體世界遮蔽音效。Occlude sound using the physical world.

第1部分-實體世界互動Part 1 - Physical World Interaction

重要概念Key Concepts

  • 實體物件通常會在遇到表面或另一個物件時產生音效。Physical objects generally make a sound when encountering a surface or another object.
  • 音效應該在體驗中是適當的內容。Sounds should be context appropriate within the experience.

例如,在資料表上設定杯的音效應該會比將科羅拉多放在金屬片上的音效更安靜。For example, setting a cup on a table should make a quieter sound than dropping a boulder on a piece of metal.

指示Instructions

  • 在 [階層] 面板中,展開 [ HologramCollection]。In the Hierarchy panel, expand HologramCollection.
  • 展開 [ EnergyHub],選取 [ 基底]。Expand EnergyHub, select Base.
  • 在 [偵測 ] 面板中,按一下 [ 新增元件 ],然後 以音效和動作 新增點一下。In the Inspector panel, click Add Component and add Tap To Place With Sound and Action.
  • 利用 音效和動作來放置In Tap To Place With Sound and Action:
    • 選取 [在點處放置父系]。Check Place Parent On Tap.
    • 設定放置 音效放置Set Placement Sound to Place.
    • 取貨音效 設定為 收取Set Pickup Sound to Pickup.
    • 在 [ 取貨動作 ] 和 [ 放置] 動作 下,按右下方的 +。Press the + in the bottom right under both On Pickup Action and On Placement Action. 將 EnergyHub 從場景拖曳至 [ 無] (物件) 欄位。Drag EnergyHub from the scene into the None (Object) fields.
      • 在 [裝貨] 動作 底下,按一下 [沒有函數 -> EnergyHubBase -> ResetAnimation]。Under On Pickup Action, click on No Function -> EnergyHubBase -> ResetAnimation.
      • 在 [放置動作] 下方,按一下 [沒有函數 -> EnergyHubBase -> >onselect]。Under On Placement Action, click on No Function -> EnergyHubBase -> OnSelect.

利用音效和動作來放置

第2部分-音效遮蔽Part 2 - Sound Occlusion

重要概念Key Concepts

  • 像光一樣的音效可以是 pixels occluded。Sound, like light, can be occluded.

典型的範例是音樂會廳。A classic example is a concert hall. 當接聽程式位於大廳之外且門已關閉時,音樂音效會 muffled。When a listener is standing outside of the hall and the door is closed, the music sounds muffled. 通常磁片區也會減少。There is also typically a reduction in volume. 當大門開啟時,就會在實際的音量聽到音效的整個範圍。When the door is opened, the full spectrum of the sound is heard at the actual volume. 高頻率音效通常會吸收較低的頻率。High frequency sounds are generally absorbed more than low frequencies.

指示Instructions

  • 在 [階層] 面板中,展開 [ HologramCollection ],然後選取 [ P0LY]。In the Hierarchy panel, expand HologramCollection and select P0LY.
  • 在 [偵測 ] 面板中,按一下 [ 新增元件 ],然後新增 音訊發射器In the Inspector panel, click Add Component and add Audio Emitter.

音訊發射器類別提供下列功能:The Audio Emitter class provides the following features:

  • 還原 spatialize 磁片區的任何變更。Restores any changes to the volume of the AudioSource.
  • 從使用者的位置,以 AudioEmitter 附加的 GameObject 方向來執行 RaycastNonAllocPerforms a Physics.RaycastNonAlloc from the user's position in the direction of the GameObject to which the AudioEmitter is attached.

RaycastNonAlloc 方法會用來做為效能優化,以限制配置以及傳回的結果數目。The RaycastNonAlloc method is used as a performance optimization to limit allocations as well as the number of results returned.

  • 針對每個遇到的 IAudioInfluencer ,會呼叫 ApplyEffect 方法。For each IAudioInfluencer encountered, calls the ApplyEffect method.
  • 針對不再遇到的每個先前 IAudioInfluencer ,請呼叫 RemoveEffect 方法。For each previous IAudioInfluencer that is no longer encountered, call the RemoveEffect method.

請注意,AudioEmitter 會更新人為時間刻度,而不是每個畫面上的。Note that AudioEmitter updates on human time scales, as opposed to on a per frame basis. 這樣做的原因是因為人們通常不會移動到足夠的速度,因此效果必須比每一季或半秒更頻繁地更新。This is done because humans generally do not move fast enough for the effect to need to be updated more frequently than every quarter or half of a second. 從某個位置快速傳送到另一個位置的全像影像,可能會破壞假像。Holograms that teleport rapidly from one location to another can break the illusion.

  • 在 [階層] 面板中,展開 [ HologramCollection]。In the Hierarchy panel, expand HologramCollection.
  • 展開 [ EnergyHub ],然後選取 [ BlobOutside]。Expand EnergyHub and select BlobOutside.
  • 在 [偵測 ] 面板中,按一下 [ 新增元件 ],然後新增 音訊 OccluderIn the Inspector panel, click Add Component and add Audio Occluder.
  • 在 [ 音訊 Occluder] 中,將 截止頻率 設定為 1500In Audio Occluder, set Cutoff Frequency to 1500.

此設定會將 Spatialize 頻率限制為 1500 Hz 和以下。This setting limits the AudioSource frequencies to 1500 Hz and below.

  • 磁片區傳遞0.9Set Volume Pass Through to 0.9.

這項設定會將 Spatialize 的磁片區縮減為其目前層級的90%。This setting reduces the volume of the AudioSource to 90% of it's current level.

音訊 Occluder 會將 IAudioInfluencer 實作為:Audio Occluder implements IAudioInfluencer to:

  • 使用附加至 spatialize 受控購買 AudioEmitterAudioLowPassFilter 來套用遮蔽效果。Apply an occlusion effect using an AudioLowPassFilter which gets attached to the AudioSource managed buy the AudioEmitter.
  • 將磁片區衰減套用至 Spatialize。Applies volume attenuation to the AudioSource.
  • 藉由設定中性截止頻率和停用篩選來停用效果。Disables the effect by setting a neutral cutoff frequency and disabling the filter.

使用中性的頻率是 22 kHz (22000 Hz) 。The frequency used as neutral is 22 kHz (22000 Hz). 選擇此頻率的原因,是因為它高於可由人類 ear 聽到的最大頻率,所以不會對音效造成明顯影響。This frequency was chosen due to it being above the nominal maximum frequency that can be heard by the human ear, this making no discernable impact to the sound.

  • 在 [階層] 面板中,選取 [ SpatialMapping]。In the Hierarchy panel, select SpatialMapping.
  • 在 [偵測 ] 面板中,按一下 [ 新增元件 ],然後新增 音訊 OccluderIn the Inspector panel, click Add Component and add Audio Occluder.
  • 在 [ 音訊 Occluder] 中,將 截止頻率 設定為 750In Audio Occluder, set Cutoff Frequency to 750.

當使用者與 AudioEmitter 之間的路徑中有多個阻隔器時,會將最低頻率套用至篩選準則。When multiple occluders are in the path between the user and the AudioEmitter, the lowest frequency is applied to the filter.

  • 磁片區傳遞0.75Set Volume Pass Through to 0.75.

當使用者與 AudioEmitter 之間的路徑中有多個阻隔器時,會套用額外的大量傳遞。When multiple occluders are in the path between the user and the AudioEmitter, the volume pass through is applied additively.

  • 在 [階層] 面板中,選取 [管理員]。In the Hierarchy panel, select Managers.
  • 在 [偵測 ] 面板中,展開 [ 語音輸入處理常式]。In the Inspector panel, expand Speech Input Handler.
  • 在 [ 語音輸入處理常式] 中,展開 [ 繼續收費]。In Speech Input Handler, expand Go Charge.
  • 函數 變更為 PolyActions. GoChargeChange No Function to PolyActions.GoCharge.

關鍵字: Go 收費

  • 展開 這裡Expand Come Here.
  • 函數 變更為 PolyActions. 撰寫復活Change No Function to PolyActions.ComeBack.

關鍵字:這裡

建置和部署Build and Deploy

  • 同樣地,在 Unity 中建立專案,並在 Visual Studio 中部署。As before, build the project in Unity and deploy in Visual Studio.

部署應用程式之後:After the application is deployed:

  • 說「 繼續收費 」以 P0LY 輸入能源中樞。Say "Go Charge" to have P0LY enter the Energy Hub.

請注意音效中的變更。Note the change in the sound. 它應該聽起來 muffled,而且有點安靜。It should sound muffled and a little quieter. 如果您可以將自己放在您和能源中樞之間的牆或其他物件,您應該會注意到因為真實世界遮蔽而 muffling 的音效。If you are able to position yourself with a wall or other object between you and the Energy Hub, you should notice a further muffling of the sound due to the occlusion by the real world.

  • 說「 來到這裡 」讓 P0LY 離開能源中樞,並將它放在您的前方。Say "Come Here" to have P0LY leave the Energy Hub and position itself in front of you.

請注意,P0LY 結束能源中樞後,就會移除音效遮蔽。Note that the sound occlusion is removed once P0LY exits the Energy Hub. 如果您仍在聽到遮蔽,P0LY 可能會受到真實世界的 pixels occluded。If you are still hearing occlusion, P0LY may be occluded by the real world. 請嘗試移動以確保您有清楚的 P0LY。Try moving to ensure you have a clear line of sight to P0LY.

第3部分-房間模型Part 3 - Room Models

重要概念Key Concepts

  • 空間大小提供可提供音效當地語系化的 subliminal 佇列。The size of the space provides subliminal queues that contribute to sound localization.
  • 每個 spatialize 都會設定房間模型。Room models are set per-AudioSource.
  • MixedRealityToolkit For Unity提供用來設定房間模型的程式碼。The MixedRealityToolkit for Unity provides code for setting the room model.
  • 針對混合現實體驗,請選取最適合真實世界空間的房間模型。For Mixed Reality experiences, select the room model that best fits the real world space.

如果您要建立虛擬實境案例,請選取最適合虛擬環境的房間模型。If you are creating a Virtual Reality scenario, select the room model that best fits the virtual environment.

第4章-音效設計Chapter 4 - Sound Design

目標Objectives

  • 瞭解有效音效設計的考慮。Understand considerations for effective sound design.
  • 學習混合技術與指導方針。Learn mixing techniques and guidelines.

第1部分-音效和體驗設計Part 1 - Sound and Experience Design

本節將討論主要音效,以及體驗設計考慮與指導方針。This section discusses key sound and experience design considerations and guidelines.

標準化所有音效Normalize all sounds

如此一來,就不需要特殊案例的程式碼來調整每個音效的音量層級,這可能相當耗時,而且會限制輕鬆更新音效檔的能力。This avoids the need for special case code to adjust volume levels per sound, which can be time consuming and limits the ability to easily update sound files.

非網路共用體驗的設計Design for an untethered experience

HoloLens 是完全獨立的非網路共用全像電腦。HoloLens is a fully contained, untethered holographic computer. 在移動時,您的使用者可以和將會使用您的體驗。Your users can and will use your experiences while moving. 請務必四處流覽,以測試您的音訊組合。Be sure to test your audio mix by walking around.

從您的全像位置的邏輯位置發出音效Emit sound from logical locations on your holograms

在真實世界中,狗不會從其尾端吠,而人類的語音也不會來自于其腳。In the real world, a dog does not bark from its tail and a human's voice does not come from his/her feet. 避免讓您的音效從非預期的全息部分發出。Avoid having your sounds emit from unexpected portions of your holograms.

針對小型全像影像,從幾何中央開始發出音效是合理的。For small holograms, it is reasonable to have sound emit from the center of the geometry.

熟悉的音效最能當地語系化Familiar sounds are most localizable

人類心聲和音樂很容易當地語系化。The human voice and music are very easy to localize. 如果有人呼叫您的名字,您可以非常精確地判斷聲音的出現方向和距離。If someone calls your name, you are able to very accurately determine from what direction the voice came and from how far away. 簡短的是,不熟悉的音效也難以當地語系化。Short, unfamiliar sounds are harder to localize.

Cognizant 使用者期望Be cognizant of user expectations

生命體驗讓我們能夠找出音效的位置。Life experience plays a part in our ability to identify the location of a sound. 這是人們的聲音特別容易當地語系化的原因之一。This is one reason why the human voice is particularly easy to localize. 在放置音效時,請務必留意使用者的預期期望。It is important to be aware of your user's learned expectations when placing your sounds.

例如,當有人聽到鳥的歌曲時,通常會進行查閱,因為鳥通常會在 (飛出或在樹狀結構) 上。For example, when someone hears a bird song they generally look up, as birds tend to be above the line of sight (flying or in a tree). 使用者開啟正確的音效方向並不常見,但請查看錯誤的垂直方向,並在找不到全像時感到混淆或感到挫折。It is not uncommon for a user to turn in the correct direction of a sound, but look in the wrong vertical direction and become confused or frustrated when they are unable to find the hologram.

避免隱藏的發射器Avoid hidden emitters

在真實世界中,如果聽到音效,我們通常可以找出發出音效的物件。In the real world, if we hear a sound, we can generally identify the object that is emitting the sound. 這也應該在您的體驗中成立。This should also hold true in your experiences. 這可能很令人不安,讓使用者聽聽音效,從音效的來源,到無法查看物件。It can be very disconcerting for users to hear a sound, know from where the sound originates and be unable to see an object.

此指導方針有一些例外。There are some exceptions to this guideline. 例如,欄位中的 crickets 等環境音效不需要可見。For example, ambient sounds such as crickets in a field need not be visible. 生命體驗讓我們熟悉這些音效的來源,而不需要加以查看。Life experience gives us familiarity with the source of these sounds without the need to see it.

第2部分-音效混合Part 2 - Sound Mixing

將您在 HoloLens 上的70% 磁片區混合設為目標Target your mix for 70% volume on the HoloLens

混合現實體驗可讓您在真實世界中看到全像影像。Mixed Reality experiences allow holograms to be seen in the real world. 它們也應該讓真實的聲音聽起來。They should also allow real world sounds to be heard. 70% 的磁片區目標可讓使用者在世界各地聆聽您的體驗。A 70% volume target enables the user to hear the world around them along with the sound of your experience.

100% 磁片區的 HoloLens 應下拉式清單外部聲音HoloLens at 100% volume should drown out external sounds

磁片區層級100% 類似于虛擬實境體驗。A volume level of 100% is akin to a Virtual Reality experience. 使用者會以視覺化方式傳送到不同的世界。Visually, the user is transported to a different world. 相同的語音也是如此。The same should hold true audibly.

使用 Unity AudioMixer 來調整聲音的類別Use the Unity AudioMixer to adjust categories of sounds

設計混合時,建立音效類別以及能夠將其磁片區增加或減少為一個單位通常會很有説明。When designing your mix, it is often helpful to create sound categories and have the ability to increase or decrease their volume as a unit. 這會保留每個音效的相對層級,同時讓整體混合的快速且輕鬆地進行變更。This retains the relative levels of each sound while enabling quick and easy changes to the overall mix. 常見類別包括:音效、環境、語音轉移和背景音樂。Common categories include; sound effects, ambience, voice overs and background music.

根據使用者的注視來混合音效Mix sounds based on the user's gaze

根據使用者在 (或不) 的情況下,變更您體驗的音效通常會很有用。It can often be useful to change the sound mix in your experience based on where a user is (or is not) looking. 這項技術的其中一種常見用途是減少全像全像攝影框架外的全息體音量層級,讓使用者更容易專注于他們之前的資訊。One common use for this technique are to reduce the volume level for holograms that are outside of the Holographic Frame to make it easier for the user to focus on the information in front of them. 另一種用途是增加音效的音量,將使用者的注意力吸引到重要的活動。Another use is to increase the volume of a sound to draw the user's attention to an important event.

打造您的混合Building your mix

建立混合時,建議您從經驗的背景音訊著手,並根據重要性新增圖層。When building your mix, it is recommended to start with your experience's background audio and add layers based on importance. 這通常會導致每個圖層比上一個層更大。Often, this results in each layer being louder than the previous.

將您的混合想像為反向漏斗圖,其中最重要的 (,且通常會在底部) quietest 音效,建議您以類似下圖的方式來結構您的混合。Imagining your mix as an inverted funnel, with the least important (and generally quietest sounds) at the bottom, it is recommended to structure your mix similar to the following diagram.

音效混合結構

語音轉移是一種有趣的案例。Voice overs are an interesting scenario. 根據您所建立的體驗,您可能會想要有身歷聲 (未當地語系化) 音效或 spatialize 語音轉移。Based on the experience you are creating you may wish to have a stereo (not localized) sound or to spatialize your voice overs. 兩個 Microsoft 發佈的經驗說明每個案例的絕佳範例。Two Microsoft published experiences illustrate excellent examples of each scenario.

HoloTour 使用身歷聲語音。HoloTour uses a stereo voice over. 當朗讀程式描述所要查看的位置時,音效是一致的,而且不會根據使用者的位置而有所不同。When the narrator is describing the location being viewed, the sound is consistent and does not vary based on the user's position. 這可讓朗讀程式描述場景,而不需要離開環境的 hrtf 音效。This enables the narrator to describe the scene without taking away from the spatialized sounds of the environment.

片段 會利用 hrtf 的語音,以偵探的形式來使用。Fragments utilizes a spatialized voice over in the form of a detective. 您可以使用偵探的聲音來協助使用者關注重要的線索,就像實際的人在房間裡一樣。The detective's voice is used to help bring the user's attention to an important clue as if an actual human was in the room. 這可讓您更清楚地深度解決謎的體驗。This enables an even greater sense of immersion into the experience of solving the mystery.

第3部分-效能Part 3 -Performance

CPU 使用量CPU usage

使用空間音效時,10-12 發射器將耗用大約12% 的 CPU。When using Spatial Sound, 10 - 12 emitters will consume approximately 12% of the CPU.

串流長音訊檔案Stream long audio files

音訊資料可能很大,尤其是常見的取樣率 (44.1 和 48 kHz) 。Audio data can be large, especially at common sample rates (44.1 and 48 kHz). 一般規則是,應串流處理超過 5-10 秒的音訊檔案,以減少應用程式記憶體使用量。A general rule is that audio files longer than 5 - 10 seconds should be streamed to reduce application memory usage.

在 Unity 中,您可以在檔案的匯入設定中標示要串流處理的音訊檔案。In Unity, you can mark an audio file for streaming in the file's import settings.

音訊匯入設定

第5章-特殊效果Chapter 5 - Special Effects

目標Objectives

  • 將深度新增至「魔術視窗」。Add depth to "Magic Windows".
  • 將使用者帶入虛擬世界。Bring the user into the virtual world.

魔術視窗Magic Windows

重要概念Key Concepts

  • 以視覺效果吸引人,將視圖建立到隱藏的世界中。Creating views into a hidden world, is visually compelling.
  • 藉由在隱藏體或使用者接近隱藏的世界時新增音訊效果,以增強真實感。Enhance realism by adding audio effects when a hologram or the user is near the hidden world.

指示Instructions

  • 在 [階層] 面板中,展開 [ HologramCollection ],然後選取 [ Underworld]。In the Hierarchy panel, expand HologramCollection and select Underworld.
  • 展開 [ Underworld ],然後選取 [ VoiceSource]。Expand Underworld and select VoiceSource.
  • 在 [偵測 ] 面板中,按一下 [ 新增元件 ] 和 [新增 使用者語音效果]。In the Inspector panel, click Add Component and add User Voice Effect.

Spatialize 元件將會新增至 VoiceSourceAn AudioSource component will be added to VoiceSource.

  • spatialize 中,將 [ 輸出 ] 設定為 UserVoice (混音器)In AudioSource, set Output to UserVoice (Mixer).
  • 核取 [ Spatialize ] 核取方塊。Check the Spatialize checkbox.
  • 空間 Blend 滑杆拖曳至 3d,或在編輯方塊中輸入 1Drag the Spatial Blend slider all the way to 3D, or enter 1 in the edit box.
  • 展開 [ 3D 音效設定]。Expand 3D Sound Settings.
  • Doppler 層級 設定為 0Set Doppler Level to 0.
  • 在 [ 使用者心聲效果] 中,將 父物件 從場景設定為 UnderworldIn User Voice Effect, set Parent Object to the Underworld from the scene.
  • 最大距離 設定為 1Set Max Distance to 1.

設定 [ 最大距離 ] 會告知 使用者語音效果 在啟用效果之前,使用者必須在父物件中的距離。Setting Max Distance tells User Voice Effect how close the user must be to the parent object before the effect is enabled.

  • 在 [ 使用者語音效果] 中,展開 [ 一首哀歌參數]。In User Voice Effect, expand Chorus Parameters.
  • 深度 設定為 0.1Set Depth to 0.1.
  • 設定一下 1 個磁片 區,再按2個 音量 ,然後將3個 磁片區分0.8Set Tap 1 Volume, Tap 2 Volume and Tap 3 Volume to 0.8.
  • 原始音效磁片 區設定為 0.5Set Original Sound Volume to 0.5.

先前的設定會設定 Unity AudioChorusFilter 的參數,這些參數是用來為使用者的語音新增豐富的聲音。The previous settings configure the parameters of the Unity AudioChorusFilter used to add richness to the user's voice.

  • 在 [ 使用者語音效果] 中,展開 [ Echo 參數]。In User Voice Effect, expand Echo Parameters.
  • 延遲 設定為 300Set Delay to 300
  • 衰減比例 設定為 0.2Set Decay Ratio to 0.2.
  • 原始音效音量 設定為 0Set Original Sound Volume to 0.

先前的設定會設定用來讓使用者的語音回應的 Unity AudioEchoFilter 參數。The previous settings configure the parameters of the Unity AudioEchoFilter used to cause the user's voice to echo.

使用者心聲效果腳本負責:The User Voice Effect script is responsible for:

  • 測量使用者與附加腳本的 GameObject 之間的距離。Measuring the distance between the user and the GameObject to which the script is attached.
  • 判斷使用者是否遇到 GameObjectDetermining whether or not the user is facing the GameObject.

無論距離為何,使用者都必須面對 GameObject,才能啟用效果。The user must be facing the GameObject, regardless of distance, for the effect to be enabled.

  • AudioChorusFilterAudioEchoFilter 套用至 spatializeApplying and configuring an AudioChorusFilter and an AudioEchoFilter to the AudioSource.
  • 停用篩選以停用效果。Disabling the effect by disabling the filters.

使用者心聲效果會使用來自 MixedRealityToolkit For Unity的 Mic 串流選取器元件,來選取高品質的語音串流,並將其路由傳送至 Unity 的音訊系統。User Voice Effect uses the Mic Stream Selector component, from the MixedRealityToolkit for Unity, to select the high quality voice stream and route it into Unity's audio system.

  • 在 [階層] 面板中,選取 [管理員]。In the Hierarchy panel, select Managers.
  • 在 [偵測 ] 面板中,展開 [ 語音輸入處理常式]。In the Inspector panel, expand Speech Input Handler.
  • 在 [ 語音輸入處理常式] 中,展開 [ 顯示 Underworld]。In Speech Input Handler, expand Show Underworld.
  • 函數 變更為 UnderworldBase. OnEnableChange No Function to UnderworldBase.OnEnable.

關鍵字:顯示 Underworld

  • 展開 [ 隱藏 Underworld]。Expand Hide Underworld.
  • 函數 變更為 UnderworldBase. OnDisableChange No Function to UnderworldBase.OnDisable.

關鍵字:隱藏 Underworld

建置和部署Build and Deploy

  • 同樣地,在 Unity 中建立專案,並在 Visual Studio 中部署。As before, build the project in Unity and deploy in Visual Studio.

部署應用程式之後:After the application is deployed:

  • 臉部 (牆壁、樓層、表格) ,然後說出 "Show Underworld"Face a surface (wall, floor, table) and say "Show Underworld".

將會顯示 underworld,並隱藏所有其他的全像投影。The underworld will be shown and all other holograms will be hidden. 如果看不到 underworld,請確定您面對的是真實世界表面。If you do not see the underworld, ensure that you are facing a real-world surface.

  • Underworld 全像1計量中的方法,並開始討論。Approach within 1 meter of the underworld hologram and start talking.

現在音訊效果已套用至您的語音!There are now audio effects applied to your voice!

  • 關閉 underworld,並注意效果不再適用的方式。Turn away from the underworld and notice how the effect is no longer applied.
  • 說「 隱藏 Underworld 」以隱藏 Underworld。Say "Hide Underworld" to hide the underworld.

Underworld 將會隱藏,而先前隱藏的全像全像投影會再度出現。The underworld will be hidden and the previously hidden holograms will reappear.

結束The End

恭喜!Congratulations! 您現在已完成 MR 空間220:空間音效You have now completed MR Spatial 220: Spatial sound.

聽聽全世界的音效,利用音效讓您的生活體驗!Listen to the world and bring your experiences to life with sound!