渲染Rendering

全息渲染使你的应用程序可以在世界各地的确切位置绘制一个全息影像,无论是精确地放入到物理领域还是在创建的虚拟领域内。Holographic rendering enables your application to draw a hologram at a precise location in the world around the user, whether it's precisely placed in the physical world or within a virtual realm you've created. 全息影像 是发出声音和光的对象。Holograms are objects made of sound and light. 呈现使应用程序能够添加光。Rendering enables your application to add the light.

设备支持Device support

功能Feature HoloLens (第一代) HoloLens (first gen) HoloLens 2HoloLens 2 沉浸式头戴显示设备Immersive headsets
渲染Rendering ✔️✔️ ✔️✔️ ✔️✔️

全息渲染Holographic rendering

用于全息呈现的关键是了解所使用的设备类型。Key to holographic rendering is knowing what kind of device is being used. 具有 查看显示 的设备,如 HoloLens,将灯光添加到世界。Devices with see-through displays, such as HoloLens, add light to the world. 黑色像素是完全透明的,而较亮的像素则越来越不透明。Black pixels are fully transparent, while brighter pixels are increasingly opaque. 由于显示器中的光线被添加到现实世界的灯光,因此白色像素是半透明的。Because the light from the displays is added to the light from the real world, white pixels are translucent.

尽管 stereoscopic 呈现为你的全息影像提供了一个深度提示,但增加了 接地效果 可帮助用户更轻松地了解到全息图附近的图面。While stereoscopic rendering provides one depth cue for your holograms, adding grounding effects can help users see more easily what surface a hologram is near. 一种接地方法是围绕附近图面上的全息图添加发光,然后针对此发光呈现阴影。One grounding technique is to add a glow around a hologram on the nearby surface, and then render a shadow against this glow. 这样一来,您的阴影就会显得比环境少一些。In this way, your shadow appears to subtract light from the environment. 空间音效 是另一个重要的深度提示,可让用户了解全息图的距离和相对位置。Spatial sound is another important depth cue, letting users reason about the distance and relative location of a hologram.

显示不 透明 的设备,如 Windows Mixed Reality 沉浸式耳机,会阻止世界。Devices with opaque displays, like Windows Mixed Reality immersive headsets, block out the world. 黑色像素为纯色,而任何其他颜色则作为该颜色显示给用户。Black pixels are solid black, and any other color appears as that color to the user. 应用程序负责呈现用户看到的所有内容。Your application is responsible for rendering everything the user sees. 这使得更重要的是保持持续的刷新率,使用户有舒适的体验。This makes it even more important to maintain a constant refresh rate so that users have a comfortable experience.

预测呈现参数Predicted rendering parameters

混合现实耳机 (HoloLens 和沉浸式耳机) 持续跟踪用户相对于其周围的位置和方向。Mixed reality headsets (both HoloLens and immersive headsets) continually track the position and orientation of the user's head relative to their surroundings. 当您的应用程序开始准备下一帧时,系统将预测用户在显示器上的未来的哪一段时间。As your application begins preparing its next frame, the system predicts where the user's head will be in the future at the exact moment that the frame shows up on the displays. 根据此预测,系统将计算用于该帧的视图和投影转换。Based on this prediction, the system calculates the view and the projection transforms to use for that frame. 应用程序 必须使用这些转换才能生成正确的结果Your application must use these transforms to produce correct results. 如果未使用系统提供的转换,则生成的图像不会与真实环境保持一致,从而导致用户 discomfort。If system-supplied transforms aren't used, the resulting image won't align with the real world, leading to user discomfort.

备注

为了准确预测新帧何时会到达显示器,系统会不断地测量应用程序渲染管道的端到端延迟。To accurately predict when a new frame will reach the displays, the system is constantly measuring the effective end-to-end latency of your application's rendering pipeline. 当系统调整到您的渲染管道的长度时,您可以通过使该管道尽可能简短来改善全息影像的稳定性。While the system adjusts to the length of your rendering pipeline, you can improve hologram stability by keeping that pipeline as short as possible.

使用高级技术来增强系统预测的应用程序可能会覆盖系统视图和投影转换。Applications that use advanced techniques to augment the system prediction can override the system view and projection transforms. 这些应用程序仍然必须使用系统提供的转换,以使其自定义转换产生有意义的结果。These applications must still use system-supplied transforms as a basis for their custom transforms to produce meaningful results.

其他呈现参数Other rendering parameters

呈现帧时,系统将指定应用程序应在其中进行绘制的后台缓冲区视区。When rendering a frame, the system specifies the back-buffer viewport in which your application should draw. 此视区通常小于帧缓冲区的完整大小。This viewport is often smaller than the full size of the frame buffer. 无论什么是视区大小,应用程序呈现一帧后,系统都将 upscales 图像以填充整个显示器。Whatever the viewport size, once the frame is rendered by the application, the system upscales the image to fill the entirety of the displays.

对于自行找不到的应用程序,无法按所需的刷新速率进行呈现, 系统呈现参数可以配置 为减少内存压力,并以增加像素别名的成本呈现成本。For applications that find themselves unable to render at the required refresh rate, system rendering parameters can be configured to reduce memory pressure and rendering cost at the cost of increased pixel aliasing. 还可以更改后台缓冲区格式,对于某些应用,这可以帮助提高内存带宽和像素吞吐量。The back buffer format can also be changed, which for some apps can help to improve memory bandwidth and pixel throughput.

呈现拆分器、分辨率和应用程序在其中呈现的帧速率可能也会随帧之间的变化而变化,并且可能会在左右眼睛之间有所不同。The rendering frustum, resolution, and framerate in which your app is asked to render might also change from frame to frame, and might differ across the left and right eye. 例如,当 混合现实捕获 (MRC) 处于活动状态,并且未选择 " 照片/视频相机" 视图配置 时,可能会使用较大的 FOV 或分辨率来呈现一种眼睛。For example, when mixed reality capture (MRC) is active and the photo/video camera view configuration isn't opted-into, one eye might be rendered with a larger FOV or resolution.

对于任何给定的帧,应用 必须 使用系统提供的视图转换、投影转换和视区分辨率进行呈现。For any given frame, your app must render using the view transform, projection transform, and viewport resolution provided by the system. 此外,应用程序决不能假设任何呈现或视图参数都保持从帧到帧的固定。Additionally, your application must never assume that any rendering or view parameter remains fixed from frame-to-frame. 类似于 Unity 的引擎会在其自己的相机对象中处理所有这些转换,以便始终遵循用户的物理移动和系统的状态。Engines like Unity handle all these transforms for you in their own camera objects so that the physical movement of your users and the state of the system is always respected. 如果你的应用程序允许用户通过世界来虚拟移动 (例如,在游戏板上使用操纵杆) ,你可以在移动它的相机上方添加一个父远程测试机组对象。If your application allows for virtual movement of the user through the world (e.g. using the thumbstick on a gamepad), you can add a parent rig object above the camera that moves it around. 这将导致照相机反映用户的虚拟动作和物理动作。This causes the camera to reflect both the user's virtual and physical motion. 如果你的应用程序修改系统提供的视图转换、投影转换或视区维度,则它必须通过调用相应的 重写 API来通知系统。If your application modifies the view transform, projection transform, or viewport dimension provided by the system, it must inform the system by calling the appropriate override API.

为了增强全息呈现的稳定性,你的应用程序应向 Windows 提供它用于呈现的深度缓冲区。To enhance the stability of your holographic rendering, your app should provide to Windows each frame the depth buffer it used for rendering. 如果你的应用程序提供了深度缓冲区,则它应具有连贯的深度值,并以米为单位从相机中表示深度。If your app does provide a depth buffer, it should have coherent depth values, with depth expressed in meters from the camera. 这使系统可以使用每像素深度数据来更好地实现稳定的内容,前提是用户的头与预测的位置有略微的偏移量。This enables the system to use your per-pixel depth data to better stabilize content if the user's head ends up slightly offset from the predicted location. 如果无法提供深度缓冲区,则可以提供焦点,并定义一个用于剪切大部分内容的平面。If you aren't able to provide your depth buffer, you can provide a focus point and normal, defining a plane that cuts through most of your content. 如果深度缓冲区和焦点平面都提供,则系统可能会同时使用这两者。If both the depth buffer and a focus plane are provided, the system might use both. 特别是,提供深度缓冲区和焦点(当应用程序显示动态影像时包含速度矢量)会很有帮助。In particular, it's helpful to provide both the depth buffer and a focus point that includes a velocity vector when your application displays holograms that are in motion.

有关本主题的详细信息,请参阅 DirectX 文章中的呈现Refer to Rendering in DirectX article for low-level details about this topic.

全息相机Holographic cameras

Windows Mixed Reality 引入了 全息相机 的概念。Windows Mixed Reality introduces the concept of a holographic camera. 全息相机类似于3D 图形文本中的传统相机;它们同时定义外部 (位置和方向) 和内部照相机属性。Holographic cameras are similar to the traditional camera found in 3D graphics texts; they define both the extrinsic (position and orientation) and intrinsic camera properties. (例如,视图字段用于查看虚拟三维场景。 ) 不同于传统的3D 相机,应用程序不控制相机的位置、方向和内部属性。(For example, field-of-view is used to view a virtual 3D scene.) Unlike traditional 3D cameras, the application isn't in control of the position, orientation, and intrinsic properties of the camera. 而是通过用户的移动隐式控制全息相机的位置和方向。Rather, the position and orientation of the holographic camera is implicitly controlled by the user's movement. 用户的移动通过视图转换逐帧地中继到应用程序。The user's movement is relayed to the application on a frame-by-frame basis via a view transform. 同样,照相机的内部属性由设备的校准光学,并通过投影转换按帧中继。Likewise, the camera's intrinsic properties are defined by the device's calibrated optics and relayed frame-by-frame via the projection transform.

通常,应用程序将针对单个立体声相机进行呈现。In general, your application will render for a single stereo camera. 可靠的呈现循环将支持多个照相机,同时支持 mono 和立体声相机。A robust rendering loop will support multiple cameras, and will support both mono and stereo cameras. 例如,当用户激活 混合现实捕获 (MRC) 等功能时,系统可能会要求你的应用程序从备用角度进行呈现,具体取决于头戴式耳机形状。For example, the system might ask your application to render from an alternate perspective when the user activates a feature like mixed reality capture (MRC), depending on the headset shape. 可以支持多个照相机的应用 程序通过选择 他们可以支持的相机 类型 来获取它们。Applications that can support multiple cameras get them by opting-in to the kind of cameras they can support.

立体渲染Volume rendering

在三维中呈现医学面向 mri 或工程量时,通常使用 卷渲染 技术。When rendering medical MRIs or engineering volumes in 3D, volume rendering techniques are often used. 这些技术在混合现实中非常有趣,用户可以自然地从不同的角度观看此类卷,只需移动其头。These techniques can be interesting in mixed reality, where users can naturally view such a volume from key angles, simply by moving their head.

HoloLens (第一代) 支持的解决方法Supported resolutions on HoloLens (first gen)

  • 最大视区大小是 HolographicDisplay的属性。The max viewport size is a property of the HolographicDisplay. 默认情况下,HoloLens 设置为最大视区大小,默认情况下) (为HoloLens is set to the maximum viewport size, which is 720p (1268x720), by default.
  • 可以通过在 HolographicCamera 上设置 ViewportScaleFactor 来更改视区大小。The viewport size can be changed by setting the ViewportScaleFactor on the HolographicCamera. 此比例因子在0到1的范围内。This scale factor is in the range of 0 to 1.
  • HoloLens (第一代) 上支持的最小视区大小为720p 的50%,这是 360p (634x360) 。The lowest supported viewport size on HoloLens (first gen) is 50% of 720p, which is 360p (634x360). 这是0.5 的 ViewportScaleFactor。This is a ViewportScaleFactor of 0.5.
  • 由于视觉对象降级,不建议使用小于540p 的任何内容,但可用于识别像素填充率的瓶颈。Anything lower than 540p isn't recommended because of visual degradation, but can be used to identify bottlenecks in pixel fill rate.

HoloLens 2 上支持的解决方法Supported resolutions on HoloLens 2

  • 支持的当前和最大呈现器目标大小是 视图配置的属性。The current and maximum supported render target sizes are properties of the view configuration. 默认情况下,HoloLens 2 设置为1440x936 的最大呈现目标大小。HoloLens 2 is set to the maximum render target size, which is 1440x936, by default.
  • 应用可以通过调用 RequestRenderTargetSize 方法来更改呈现器目标缓冲区的大小,以请求新的呈现目标大小。Apps can change the size of render target buffers by calling the RequestRenderTargetSize method to request a new render target size. 将选择一个新的呈现目标大小,该大小满足或超过请求的呈现目标大小。A new render target size will be chosen, which meets or exceeds the requested render target size. 此 API 更改呈现器目标缓冲区的大小,这需要在 GPU 上重新分配内存。This API changes the size of the render target buffer, which requires memory reallocation on the GPU. 这包括:呈现目标大小可缩小以减少 GPU 上的内存压力,不应以高频率调用此方法。The implications of this include: The render target size can be scaled down to reduce memory pressure on the GPU, and this method shouldn't be called at high frequency.
  • 应用仍可更改视区大小,其方式与为 HoloLens 1 的操作方式相同。Apps can still change the viewport size in the same way that they did for HoloLens 1. 不会在 GPU 上增加内存重新分配,因此它可以以较高的频率进行更改,但不能用于减少 GPU 上的内存压力。There's no added memory reallocation on the GPU, so it can be changed at high frequency, but it cannot be used to reduce memory pressure on the GPU.
  • HoloLens 2 上支持的最小视区大小为634x412,当使用默认呈现器目标大小时,ViewportScaleFactor 约为0.44。The lowest supported viewport size on HoloLens 2 is 634x412, a ViewportScaleFactor of approximately 0.44 when the default render target size is in use.
  • 如果提供的呈现目标大小小于受支持的最小视区大小,则会忽略视区缩放系数。If a render target size is provided which is smaller than the lowest supported viewport size, the viewport scale factor will be ignored.
  • 由于视觉对象降级,不建议使用小于540p 的任何内容,但可用于识别像素填充率的瓶颈。Anything lower than 540p isn't recommended because of visual degradation, but can be used to identify bottlenecks in pixel fill rate.

另请参阅See also