轉譯Rendering

全像影像呈現可讓您的應用程式在世界各地的精確位置繪製一個全像,無論是精確地放在實體世界或您所建立的虛擬領域中。Holographic rendering enables your application to draw a hologram at a precise location in the world around the user, whether it's precisely placed in the physical world or within a virtual realm you've created. 像是音效和光線所組成的物件。Holograms are objects made of sound and light. 呈現可讓您的應用程式新增燈光。Rendering enables your application to add the light.

裝置支援Device support

功能Feature HoloLens (第一代) HoloLens (first gen) HoloLens 2HoloLens 2 沉浸式頭戴裝置Immersive headsets
轉譯Rendering ✔️✔️ ✔️✔️ ✔️✔️

全像攝影的呈現Holographic rendering

全像攝影轉譯的關鍵在於瞭解正在使用的裝置類型。Key to holographic rendering is knowing what kind of device is being used. 具有 觀看顯示 的裝置(例如 HoloLens)會將燈光新增至世界。Devices with see-through displays, such as HoloLens, add light to the world. 黑色圖元是完全透明的,而較亮的圖元則逐漸不透明。Black pixels are fully transparent, while brighter pixels are increasingly opaque. 由於顯示器的光線是從真實世界的光線加入,因此白色圖元是半透明的。Because the light from the displays is added to the light from the real world, white pixels are translucent.

雖然 stereoscopic 轉譯針對您的全像投影提供一個深度的提示,但新增 接地效果 可協助使用者更輕鬆地看到全息圖的表面。While stereoscopic rendering provides one depth cue for your holograms, adding grounding effects can help users see more easily what surface a hologram is near. 其中一個接地技巧是在附近表面上的全像投影周圍加上發光,然後針對這項發光呈現陰影。One grounding technique is to add a glow around a hologram on the nearby surface, and then render a shadow against this glow. 如此一來,您的陰影就可以從環境中減去燈光。In this way, your shadow appears to subtract light from the environment. 空間音效 是另一個重要的深度提示,可讓使用者知道全像影像距離和相對位置的原因。Spatial sound is another important depth cue, letting users reason about the distance and relative location of a hologram.

具有不 透明顯示 的裝置(例如 Windows Mixed Reality 沉浸式耳機)會封鎖世界。Devices with opaque displays, like Windows Mixed Reality immersive headsets, block out the world. 黑色圖元是實心的黑色,而任何其他色彩會顯示為使用者的色彩。Black pixels are solid black, and any other color appears as that color to the user. 您的應用程式會負責呈現使用者所看到的所有內容。Your application is responsible for rendering everything the user sees. 這可讓您更重要的是維護持續的重新整理頻率,讓使用者擁有舒適的體驗。This makes it even more important to maintain a constant refresh rate so that users have a comfortable experience.

預測的呈現參數Predicted rendering parameters

(HoloLens 和沉浸式耳機的混合現實耳機) 會持續追蹤使用者的標題相對於其周圍的位置和方向。Mixed reality headsets (both HoloLens and immersive headsets) continually track the position and orientation of the user's head relative to their surroundings. 當您的應用程式開始準備它的下一個畫面時,系統會預測使用者在該畫面上顯示的確切時間。As your application begins preparing its next frame, the system predicts where the user's head will be in the future at the exact moment that the frame shows up on the displays. 根據這種預測,系統會計算要用於該框架的視圖和投射轉換。Based on this prediction, the system calculates the view and the projection transforms to use for that frame. 您的應用程式 必須使用這些轉換來產生正確的結果Your application must use these transforms to produce correct results. 如果未使用系統提供的轉換,則產生的影像不會與真實世界一致,進而導致使用者不適感。If system-supplied transforms aren't used, the resulting image won't align with the real world, leading to user discomfort.

注意

為了精確地預測新框架何時會到達顯示器,系統會不斷地測量應用程式轉譯管線的有效端對端延遲。To accurately predict when a new frame will reach the displays, the system is constantly measuring the effective end-to-end latency of your application's rendering pipeline. 雖然系統會調整為轉譯管線的長度,但您可以將該管線保持在最短的時間,藉以改善全像影像的穩定性。While the system adjusts to the length of your rendering pipeline, you can improve hologram stability by keeping that pipeline as short as possible.

使用先進技術來加強系統預測的應用程式,可以覆寫系統檢視和投影轉換。Applications that use advanced techniques to augment the system prediction can override the system view and projection transforms. 這些應用程式仍然必須使用系統提供的轉換,作為其自訂轉換的基礎,以產生有意義的結果。These applications must still use system-supplied transforms as a basis for their custom transforms to produce meaningful results.

其他呈現參數Other rendering parameters

當轉譯框架時,系統會指定應用程式應繪製的後置緩衝區區。When rendering a frame, the system specifies the back-buffer viewport in which your application should draw. 此區通常小於框架緩衝區的完整大小。This viewport is often smaller than the full size of the frame buffer. 無論是什麼區大小,一旦應用程式呈現框架之後,系統就會 upscales 影像以填滿整個顯示器。Whatever the viewport size, once the frame is rendered by the application, the system upscales the image to fill the entirety of the displays.

如果應用程式無法以所需的重新整理頻率轉譯, 系統轉譯參數可以設定 為減少記憶體壓力和轉譯成本,代價是增加圖元別名。For applications that find themselves unable to render at the required refresh rate, system rendering parameters can be configured to reduce memory pressure and rendering cost at the cost of increased pixel aliasing. 您也可以變更背景緩衝區格式,讓某些應用程式可以協助改善記憶體頻寬和圖元輸送量。The back buffer format can also be changed, which for some apps can help to improve memory bandwidth and pixel throughput.

要求您的應用程式轉譯時,轉譯的圖格、解析度和畫面播放速率可能也會從框架變更為框架,而且可能會在左右眼睛之間有差異。The rendering frustum, resolution, and framerate in which your app is asked to render might also change from frame to frame, and might differ across the left and right eye. 例如,當 mixed reality capture (MRC) 處於作用中狀態,且未加入宣告 相片/攝影機視圖 設定時,可能會以較大的 FOV 或解析度來呈現一眼。For example, when mixed reality capture (MRC) is active and the photo/video camera view configuration isn't opted-into, one eye might be rendered with a larger FOV or resolution.

針對任何指定的框架,您的應用程式 必須 使用系統所提供的視圖轉換、投影轉換和視口解析來呈現。For any given frame, your app must render using the view transform, projection transform, and viewport resolution provided by the system. 此外,您的應用程式絕對不能假設任何轉譯或 view 參數都保持從框架到框架固定。Additionally, your application must never assume that any rendering or view parameter remains fixed from frame-to-frame. Unity 等引擎會在自己的攝影機物件中為您處理所有的轉換,如此一來,使用者的實際移動和系統的狀態一律會被遵守。Engines like Unity handle all these transforms for you in their own camera objects so that the physical movement of your users and the state of the system is always respected. 如果您的應用程式允許使用者透過全球進行虛擬移動 (例如,使用遊戲台上的操縱杆) ,您可以在相機上方新增上層的 rig 物件來四處移動。If your application allows for virtual movement of the user through the world (e.g. using the thumbstick on a gamepad), you can add a parent rig object above the camera that moves it around. 這會導致相機反映使用者的虛擬和實際動作。This causes the camera to reflect both the user's virtual and physical motion. 如果您的應用程式修改系統所提供的視圖轉換、投影轉換或視口維度,它必須呼叫適當的覆 寫 API來通知系統。If your application modifies the view transform, projection transform, or viewport dimension provided by the system, it must inform the system by calling the appropriate override API.

為了增強您的全像轉譯的穩定性,您的應用程式應該提供 Windows 每個畫面格用於轉譯的深度緩衝區。To enhance the stability of your holographic rendering, your app should provide to Windows each frame the depth buffer it used for rendering. 如果您的應用程式確實提供深度緩衝區,它應該會有一致的深度值,並以從相機的計量深度表示。If your app does provide a depth buffer, it should have coherent depth values, with depth expressed in meters from the camera. 這可讓系統使用您的每個圖元深度資料,在使用者的前端與預測的位置稍微稍微位移時,使用您的每個圖元深度資料來提供更佳的穩定內容。This enables the system to use your per-pixel depth data to better stabilize content if the user's head ends up slightly offset from the predicted location. 如果您無法提供深度緩衝區,您可以提供焦點點和正常,定義可剪下大部分內容的平面。If you aren't able to provide your depth buffer, you can provide a focus point and normal, defining a plane that cuts through most of your content. 如果同時提供深度緩衝區和焦點平面,則系統可能會使用這兩者。If both the depth buffer and a focus plane are provided, the system might use both. 尤其是當您的應用程式顯示正在移動的全息影像時,提供深度緩衝區和焦點點(包含速度向量)會很有説明。In particular, it's helpful to provide both the depth buffer and a focus point that includes a velocity vector when your application displays holograms that are in motion.

如需有關本主題的低層級詳細資料,請參閱 DirectX 文章中 的轉譯。Refer to Rendering in DirectX article for low-level details about this topic.

全像攝影機Holographic cameras

Windows Mixed Reality 引進了 全息攝影機 的概念。Windows Mixed Reality introduces the concept of a holographic camera. 全像是在3D 圖形文字中找到的繁體攝影機,它們會定義外建的 (位置和方向) 和內建相機屬性。Holographic cameras are similar to the traditional camera found in 3D graphics texts; they define both the extrinsic (position and orientation) and intrinsic camera properties. (例如,[視圖] 會用來查看虛擬3D 場景。 ) 不同于傳統3D 相機,應用程式無法控制攝影機的位置、方向和內建屬性。(For example, field-of-view is used to view a virtual 3D scene.) Unlike traditional 3D cameras, the application isn't in control of the position, orientation, and intrinsic properties of the camera. 而是由使用者的移動隱含控制全像攝影相機的位置和方向。Rather, the position and orientation of the holographic camera is implicitly controlled by the user's movement. 使用者的移動會透過「視圖」轉換,依畫面格逐一轉送至應用程式。The user's movement is relayed to the application on a frame-by-frame basis via a view transform. 同樣地,相機的內建屬性是由裝置的校正型器件所定義,並透過投影轉換依畫面轉送。Likewise, the camera's intrinsic properties are defined by the device's calibrated optics and relayed frame-by-frame via the projection transform.

一般情況下,您的應用程式會針對單一身歷聲攝影機轉譯。In general, your application will render for a single stereo camera. 健全的轉譯迴圈將會支援多個相機,並同時支援 mono 和身歷聲攝影機。A robust rendering loop will support multiple cameras, and will support both mono and stereo cameras. 例如,系統可能會要求您的應用程式在使用者啟動像是「 混合現實 」 () 的功能時,根據耳機圖形,從替代的觀點來呈現。For example, the system might ask your application to render from an alternate perspective when the user activates a feature like mixed reality capture (MRC), depending on the headset shape. 可支援多個相機的應用程式會選擇其可支援的相機類型來取得它們。Applications that can support multiple cameras get them by opting-in to the kind of cameras they can support.

磁碟區呈現Volume rendering

在3D 中轉譯醫療/Mri 或工程磁片區時,通常會使用 磁片 區轉譯技術。When rendering medical MRIs or engineering volumes in 3D, volume rendering techniques are often used. 這些技術在混合的現實中可能很有趣,因為使用者可以藉由移動其頭部,自然地從主要角度來查看這類音量。These techniques can be interesting in mixed reality, where users can naturally view such a volume from key angles, simply by moving their head.

HoloLens (第一代) 支援的解決方案Supported resolutions on HoloLens (first gen)

  • 最大的區數大小是 HolographicDisplay的屬性。The max viewport size is a property of the HolographicDisplay. HoloLens 預設會設定為最大的最大區大小,也就是 720p (1268x720) 。HoloLens is set to the maximum viewport size, which is 720p (1268x720), by default.
  • 您可以藉由在 HolographicCamera 上設定 ViewportScaleFactor 來變更此區大小。The viewport size can be changed by setting the ViewportScaleFactor on the HolographicCamera. 此比例因數的範圍介於0到1之間。This scale factor is in the range of 0 to 1.
  • HoloLens (第一代) 的最低支援的區大小是720p 的50%,也就是 360p (634x360) 。The lowest supported viewport size on HoloLens (first gen) is 50% of 720p, which is 360p (634x360). 這是0.5 的 ViewportScaleFactor。This is a ViewportScaleFactor of 0.5.
  • 由於視覺效果降低,因此不建議使用低於540p 的任何元素,但是可以用來識別圖元填滿率的瓶頸。Anything lower than 540p isn't recommended because of visual degradation, but can be used to identify bottlenecks in pixel fill rate.

HoloLens 2 支援的解決方案Supported resolutions on HoloLens 2

  • 目前和最大支援的轉譯目標大小是 view設定的屬性。The current and maximum supported render target sizes are properties of the view configuration. 預設會將 HoloLens 2 設定為最大轉譯目標大小(預設為1440x936)。HoloLens 2 is set to the maximum render target size, which is 1440x936, by default.
  • 應用程式可以藉由呼叫 RequestRenderTargetSize 方法來要求新的呈現目標大小,藉以變更轉譯目標緩衝區的大小。Apps can change the size of render target buffers by calling the RequestRenderTargetSize method to request a new render target size. 將會選擇新的轉譯目標大小,以符合或超過要求的呈現目標大小。A new render target size will be chosen, which meets or exceeds the requested render target size. 此 API 會變更呈現目標緩衝區的大小,這需要在 GPU 上重新配置記憶體。This API changes the size of the render target buffer, which requires memory reallocation on the GPU. 這包括:轉譯目標大小可相應減少以降低 GPU 上的記憶體壓力,而且不應以較高的頻率呼叫這個方法。The implications of this include: The render target size can be scaled down to reduce memory pressure on the GPU, and this method shouldn't be called at high frequency.
  • 應用程式仍然可以用與 HoloLens 1 相同的方式變更其區大小。Apps can still change the viewport size in the same way that they did for HoloLens 1. GPU 上沒有額外的記憶體重新配置,因此可以在較高的頻率進行變更,但無法用來降低 GPU 上的記憶體壓力。There's no added memory reallocation on the GPU, so it can be changed at high frequency, but it cannot be used to reduce memory pressure on the GPU.
  • HoloLens 2 上最低支援的區大小為634x412,當預設轉譯目標大小正在使用中時,ViewportScaleFactor 約為0.44。The lowest supported viewport size on HoloLens 2 is 634x412, a ViewportScaleFactor of approximately 0.44 when the default render target size is in use.
  • 如果提供的轉譯目標大小小於最低支援的區大小,則會忽略此區縮放比例。If a render target size is provided which is smaller than the lowest supported viewport size, the viewport scale factor will be ignored.
  • 由於視覺效果降低,因此不建議使用低於540p 的任何元素,但是可以用來識別圖元填滿率的瓶頸。Anything lower than 540p isn't recommended because of visual degradation, but can be used to identify bottlenecks in pixel fill rate.

另請參閱See also