如何在混合現實應用程式中使用音效How to use sound in mixed-reality applications

您可以使用音效來通知和強化使用者的應用程式狀態的精神模型。You can use sound to inform and reinforce the user's mental model of application state. 適當時,請使用 spatialization 將聲音放在混合現實世界中。Use spatialization, when appropriate, to place sounds in the mixed-reality world. 當您以這種方式連接聽覺和視覺效果時,您可以加深互動的直覺本質,並提高使用者的信心。When you connect the auditory and the visual in this way, you deepen the intuitive nature of interactions and increase user confidence.

新增聲音的時機When to add sounds

由於應用程式缺乏 tactile 介面,因此混合現實應用程式通常比2D 應用程式更需要音效。Mixed-reality applications often have a greater need for sound than 2D apps, because of their lack of a tactile interface. 當聲音通知使用者或強化互動時,請新增音效。Add sounds when they inform the user or reinforce interactions.

通知和強化Inform and reinforce

  • 針對不是由使用者所起始的事件(例如通知),請使用音效通知使用者發生變更。For events that aren't initiated by the user, such as notifications, use sound to inform the user that a change occurred.
  • 互動可能有數個階段。Interactions may have several stages. 使用音效強化階段轉換。Use sound to reinforce stage transitions.

請參閱下列互動、事件和建議的音效特性範例。See the following examples of interactions, events, and suggested sound characteristics.

克制Exercise restraint

使用者沒有無限容量的音訊資訊。Users don't have an unlimited capacity for audio information.

  • 每個音效都應該傳達特定、寶貴的資訊。Each sound should communicate specific, valuable information.
  • 當您的應用程式播放音效來通知使用者時,請暫時減少其他音效的音量。When your app plays a sound to inform the user, temporarily reduce the volume of other sounds.
  • 針對按鈕停留音效 (查看下列資訊) ,加入時間延遲以防止過多的音效觸發。For button hover sounds (see the following information), add a time delay to prevent excessive sound triggering.

不要完全依賴聲音Don't rely solely on sounds

使用起來很重要的音效對您的使用者很有説明。Sounds that are used well are valuable to your users. 但請確定您的應用程式即使已關閉音效也可供使用。But make sure your application is usable even with the sound turned off.

  • 使用者可能會被聽力受損。Users may be hearing impaired.
  • 您的應用程式可能會在較高的環境中使用。Your application may be used in a loud environment.
  • 使用者可能會有隱私權疑慮或其他停用裝置音訊的原因。Users may have privacy concerns or other reasons to disable device audio.

如何 sonify 互動How to sonify interactions

混合現實中的互動類型包括筆勢、直接操作和語音。Interaction types in mixed reality include gesture, direct manipulation, and voice. 使用下列建議的特性來選取或設計這些互動的音效。Use the following suggested characteristics to select or design sounds for these interactions.

手勢互動Gesture interactions

在混合的現實情況下,使用者可以使用滑鼠與按鈕互動。In mixed reality, users may interact with buttons by using a mouse. 按鈕動作通常會在使用者釋放時,而不是按下按鈕來讓使用者有機會取消互動。Button actions generally occur when the user releases rather than presses the button to give the user a chance to cancel the interaction. 使用音效來強化這些階段。Use sounds to reinforce these stages. 若要協助使用者以較遠的按鈕為目標,也請考慮使用指標停留音效。To assist users in targeting distant buttons, also consider using a pointer-hover sound.

  • 按鈕-按下聲音應該是簡短的 tactile 「按一下」。Button-press sounds should be a short, tactile "click."
    範例: MRTK_ButtonPress .wavExample: MRTK_ButtonPress.wav
  • 按鈕-"unpress" 音效應該有類似的 tactile 風格。Button-"unpress" sounds should have a similar tactile feel. 比按音效更高的音調會強調完成的意義。A higher pitch than the press sound reinforces the sense of completion.
    範例: MRTK_ButtonUnpress .wavExample: MRTK_ButtonUnpress.wav
  • 若為暫止音效,請考慮使用微妙且不危險的音效,例如低頻率的 thud 或凹凸效果。For hover sounds, consider using a subtle and non-threatening sound, such as a low-frequency thud or bump.

直接操作Direct manipulation

在 HoloLens 2 上,可說的手追蹤支援直接操作使用者介面元素。On HoloLens 2, articulated hand tracking supports direct manipulation of user-interface elements. 當沒有其他實體意見反應時,音效就很重要。Sounds are important when there's no other physical feedback.

按鈕的按 音效很重要,因為當使用者到達按鍵筆劃的底部時,不會收到任何其他指示。A button press sound is important because the user doesn't get any other indication when they reach the bottom of the key stroke. 關鍵旅遊的音效指標可能很小、微妙且 pixels occluded。Sound indicators of key travel can be small, subtle, and occluded. 如同手勢互動,按下按鈕應該會得到短暫的 tactile 音效,像是按一下。As with gesture interactions, button presses should get a short, tactile sound like a click. Unpresses 應該會有類似的點擊音效,但卻有凸起的音調。Unpresses should have a similar click sound but with raised pitch.

很難以視覺化方式確認抓取或發行動作。It's difficult to visually confirm a grab or release action. 使用者的手通常會是任何視覺效果,而主體的物件缺乏真實的視覺類比「抓取」。The user's hand will often be in the way of any visual effect, and hard-bodied objects lack a real-world visual analog of "grabbing." 音效可以有效地傳達成功的抓取和發行互動。Sounds can effectively communicate successful grab and release interactions.

  • 抓取動作應該有簡短、有點 muffled 的 tactile 音效,以提示在物件周圍關閉手指的概念。Grab actions should have a short, somewhat-muffled tactile sound that prompts the idea of fingers closing around an object. 有時候也有一個「竊竊私語」音效,會導致抓取音效傳達手中的動作。Sometimes there's also a "whoosh" sound that leads up to the grabbing sound to communicate the motion of the hand.
    範例: MRTK_Move_Start .wavExample: MRTK_Move_Start.wav
  • 發行動作應該會得到類似的簡短和 tactile 音效。Release actions should get a similarly short and tactile sound. 它通常會比抓取音效更低,而反向順序則會有影響,然後是「竊竊私語」,以傳達物件正在進行的程式。It's usually lower pitched than the grab sound and in reverse order, with an impact and then a "whoosh" to communicate that the object is settling into place.
    範例: MRTK_Move_End .wavExample: MRTK_Move_End.wav

繪圖 互動應該會以使用者的移動量來取得持續的迴圈音效。A drawing interaction should get a persistent, looping sound with volume determined by the user's hand movement. 當使用者的手仍在移動時,當手 loudest 時,它應該是無訊息的。It should be silent when the user's hand is still and loudest when the hand is moving quickly.

語音互動Voice interactions

語音互動通常有細微的視覺元素。Voice interactions often have subtle visual elements. 使用音效強化互動階段。Use sounds to reinforce interaction stages. 您可能會想要使用更音效的音效來區別它們與手勢和直接操作音效。You may want to use more-tonal sounds to distinguish them from gesture and direct-manipulation sounds.

  • 使用發音語氣來 確認 語音命令。Use a positive-sounding tone for voice command confirmations. 逐漸增加的聲音和主要的樂譜間隔都有效。Rising tones and major musical intervals are effective.
  • 使用較短且較不正面發音的語音命令 失敗 語氣。Use a shorter, less-positive-sounding tone for voice command failures. 避免負聲音。Avoid negative sounds. 相反地,請使用更 percussive 的中性音效來與應用程式從互動中移動的通訊。Instead, use a more percussive, neutral sound to communicate that the application is moving on from the interaction.
  • 如果您的應用程式具有喚醒字,則在裝置 開始接聽 時,請使用短暫、溫和的語氣。If your application has a wake word, use a short, gentle tone when the device starts listening. 應用程式接聽時使用 微妙的迴圈音效。Use a subtle looping sound while the application is listening.

通知Notifications

通知會通知應用程式狀態變更,以及使用者未起始的其他事件。Notifications signal application-state changes and other events the user didn't initiate. 狀態變更可能包括處理常式完成、訊息和通話。State changes can include process completions, messages, and phone calls.

在混合的現實情況下,物件有時會移出使用者的觀點。In mixed reality, objects sometimes move out of the user's field of view. 將移動 動畫物件 與 hrtf 音效(取決於物件類型和動作速度)配對。Pair moving animated objects with a spatialized sound that depends on the object type and speed of motion.

  • 它有助於在動畫結束時播放 hrtf 音效,以通知使用者物件的新位置。It helps to play a spatialized sound at the end of an animation to inform the user of the object's new position.
  • 針對漸進式移動,移動期間的「竊竊私語」音效可協助使用者追蹤物件。For gradual movements, a "whoosh" sound during movement helps the user track the object.

訊息通知 音效可能會重複聽到,有時會快速地連續播放。Message notification sounds may be heard repeatedly, sometimes in quick succession. 這一點很重要,It's important they don't stand out or sound harsh. 中等範圍正面的音效很有效率。Mid-range positive tonal sounds are effective.

  • 來電的音效應該與行動電話鈴聲具有類似的品質。Incoming-call sounds should have similar qualities to a cell phone ringtone. 這些音效都是迴圈播放的音樂片語,直到使用者接聽電話為止。These sounds are looping musical phrases that play until the user answers the call.
  • 語音通訊連線和中斷連接應該有短暫的音效音效。Voice communication connection and disconnection should have a short, tonal sound. 連接音效應該是正面的語氣,表示連接成功。The connection sound should be a positive tone to indicate a successful connection. 中斷連線的音效應該是中性的音效,表示已完成通話。The disconnection sound should be a neutral sound to indicate completion of the call.

處理 spatializationHandle spatialization

Spatialization 使用身歷聲耳機或喇叭在混合現實世界中放置音效。Spatialization uses stereo headphones or speakers to place sounds in the mixed-reality world.

要 spatialize 的音效Which sounds to spatialize

當音效與具有空間位置的事件相關聯時,就應該 hrtf 音效。A sound should be spatialized when it's associated with an event that has a spatial location. 這包括 UI、所涵蓋的 AI 語音和視覺指標。This includes UI, embodied AI voices, and visual indicators.

Spatialize 使用者介面 元素,藉由限制發音音效的數目來協助清理使用者的「空間」。Spatialize user interface elements to help declutter the user's sonic "space" by limiting the number of stereo sounds that they hear. 在 hrtf 音訊意見反應時,操作互動(例如觸控、抓取及放開)感覺更自然。Manipulation interactions such as touching, grabbing, and releasing feel more natural when audio feedback is spatialized. 請考慮下列有關這些元素的距離衰減資訊。Consider the following information about distance attenuation for these elements.

Spatialize 視覺 指標和所指的 AI 聲音 ,以直覺方式通知使用者這些專案位於視野之外。Spatialize visual indicators and embodied AI voices to intuitively inform users when these things are outside the field of view.

相反地,請避免使用 spatialization 來 FACELESS AI 語音 ,以及其他缺少妥善定義空間位置的元素。In contrast, avoid spatialization for faceless AI voices and other elements that lack a well-defined spatial location. Spatialization 如果沒有相關的視覺元素,可能會讓使用者在思考有一個視覺元素找不到。Spatialization without a related visual element can distract users into thinking there's a visual element that they can't find.

Spatialization 會帶來一些 CPU 成本。Spatialization does come with some CPU cost. 許多應用程式都有兩個同時播放的音效。Many applications have at most two sounds playing simultaneously. 在該情況下,spatialization 的成本可能會是微不足道。The cost of spatialization in that case is likely negligible. 您可以使用 MRTK 畫面播放速率監視器來判斷新增 spatialization 的影響。You can use the MRTK frame rate monitor to judge the impact of adding spatialization.

何時及如何套用以距離為基礎的衰減When and how to apply distance-based attenuation

在真實世界中,更不會的音效會更安靜。In the physical world, sounds that are farther away are quieter. 您的音訊引擎可以根據來源距離來建立此衰減的模型。Your audio engine can model this attenuation based on the source distance. 使用以距離為基礎的衰減,以傳達相關資訊。Use distance-based attenuation when it communicates relevant information.

視覺 指標、 動畫影像 和其他資訊音效之間的距離,與使用者相關。The distances to visual indicators, animated holograms, and other informative sounds are relevant to the user. 使用以距離為基礎的衰減,以直覺的形式提供提示。Use distance-based attenuation to intuitively provide cues.

調整每個來源的衰減曲線,以符合您的混合現實世界空間大小。Adjust the attenuation curve for each source to fit the size of your mixed-reality world's spaces. 音訊引擎的預設曲線通常適用于大型的 (,最多可達半平方公里) 空間。Your audio engine's default curve is often meant for large (up to half-kilometer) spaces.

強化按鈕動作和其他互動 之漸進階段 的音效不應套用衰減。Sounds that reinforce the progressive stages of button actions and other interactions shouldn't get attenuation applied. 這些音效的強化效果比傳達按鈕的距離更重要。The reinforcing effects of these sounds are more important than communicating the distance to the button. 變化可能會受到干擾,尤其是鍵盤,可能會連續聽到許多按鈕點擊。Variations can be distracting, especially with keyboards, when many button clicks may be heard in succession.

要使用的 spatialization 技術Which spatialization technology to use

使用耳機或 HoloLens 喇叭, (以) 為基礎的 spatialization 技術,請使用前端相關的轉移功能。With headphones or the HoloLens speakers, use head-related transfer function (HRTF)-based spatialization technologies. 這些技術會針對實體世界中的標頭,建立音效傳播的模型。These technologies model the sound propagation around the head in the physical world. 即使音效來源位於某一端的一端,音效也會隨著衰減和延遲而散佈到遙遠的 ear。Even when a sound source is on the far side of one's head, sound propagates to the distant ear with some attenuation and delay. 說話者的移動只依賴衰減,並且在當音效位於右邊時,在左方的 ear 中套用總衰減。Speaker panning relies only on attenuation and applies total attenuation in the left ear when sounds are on the right side, and the other way around. 這項技術對於「一般聽力」接聽程式很不舒服,而且在一個 ear 中有聽力障礙的接聽程式無法存取。This technique can be uncomfortable for "normal hearing" listeners and inaccessible for listeners who have hearing impairment in one ear.

後續步驟Next steps