Unity 中的混合现实原生对象Mixed Reality native objects in Unity

每个混合现实应用都在开始接收相机数据和渲染帧之前 获得 HolographicSpaceEvery Mixed Reality app gets a HolographicSpace before it starts receiving camera data and rendering frames. 在 Unity 中,引擎负责处理这些步骤,处理全息对象和内部更新作为其呈现循环的一部分。In Unity, the engine takes care of those steps for you, handling Holographic objects and internally updating as part of its render loop.

但是,在高级方案中,可能需要访问基础本机对象,例如 HolographicCamera 和 current HolographicFrameHowever, in advanced scenarios you may need to get access to the underlying native objects, such as the HolographicCamera and current HolographicFrame. UnityEngine. XR. XRDevice 提供对这些本机对象的访问权限。UnityEngine.XR.XRDevice is what provides access to these native objects.


命名空间: UnityEngine. XRNamespace: UnityEngine.XR
类型: XRDeviceType: XRDevice

XRDevice 类型允许使用 GetNativePtr方法获取对基础本机对象的访问权限。The XRDevice type allows you to get access to underlying native objects using the GetNativePtr method. 不同平台之间的 GetNativePtr 返回内容有所不同。What GetNativePtr returns varies between different platforms. 在通用 Windows 平台上,当面向 Windows Mixed Reality XR SDK 时,XRDevice 会返回) 到以下结构的指针 (IntPtr:On the Universal Windows Platform, when targeting the Windows Mixed Reality XR SDK, XRDevice.GetNativePtr returns a pointer (IntPtr) to the following structure:

using System;
using System.Runtime.InteropServices;

struct HolographicFrameNativeData
    public uint VersionNumber;
    public uint MaxNumberOfCameras;
    public IntPtr ISpatialCoordinateSystemPtr; // Windows::Perception::Spatial::ISpatialCoordinateSystem
    public IntPtr IHolographicFramePtr; // Windows::Graphics::Holographic::IHolographicFrame 
    public IntPtr IHolographicCameraPtr; // // Windows::Graphics::Holographic::IHolographicCamera

可以使用 Marshal.ptrtostructure 方法将其转换为 HolographicFrameNativeData:You can convert it to HolographicFrameNativeData using Marshal.PtrToStructure method:

var nativePtr = UnityEngine.XR.XRDevice.GetNativePtr();
HolographicFrameNativeData hfd = Marshal.PtrToStructure<HolographicFrameNativeData>(nativePtr);

IHolographicCameraPtr 是一个作为 unmanagedtype.bool 进行封送的 IntPtr 数组,其长度等于 MaxNumberOfCamerasIHolographicCameraPtr is an array of IntPtr marshaled as UnmanagedType.ByValArray with a length equal to MaxNumberOfCameras

取消封送本机指针Unmarshaling native pointers

如果你使用的是 MixedReality,则可以使用方法从本机指针构造托管对象 FromNativePtr()If you are using Microsoft.Windows.MixedReality.DotNetWinRT, you can construct a managed object from a native pointer using the FromNativePtr() method:

var worldOrigin = Microsoft.Windows.Perception.Spatial.SpatialCoordinateSystem.FromNativePtr(hfd.ISpatialCoordinateSystemPtr);

否则,请使用 Marshal.GetObjectForIUnknown() 并强制转换为所需的类型:Otherwise, use Marshal.GetObjectForIUnknown() and cast to the type you want:

var worldOrigin = (Windows.Perception.Spatial.SpatialCoordinateSystem)Marshal.GetObjectForIUnknown(hfd.ISpatialCoordinateSystemPtr);

坐标系统之间的转换Converting between coordinate systems

Unity 使用左手坐标系,而 Windows 感知 Api 使用右手坐标系。Unity uses a left-handed coordinate system, while the Windows Perception APIs use right-handed coordinate systems. 若要在这两种约定之间进行转换,可以使用以下帮助器:To convert between these two conventions, you can use the following helpers:

namespace NumericsConversion
    public static class NumericsConversionExtensions
        public static UnityEngine.Vector3 ToUnity(this System.Numerics.Vector3 v) => new UnityEngine.Vector3(v.X, v.Y, -v.Z);
        public static UnityEngine.Quaternion ToUnity(this System.Numerics.Quaternion q) => new UnityEngine.Quaternion(-q.X, -q.Y, q.Z, q.W);
        public static UnityEngine.Matrix4x4 ToUnity(this System.Numerics.Matrix4x4 m) => new UnityEngine.Matrix4x4(
            new Vector4( m.M11,  m.M12, -m.M13,  m.M14),
            new Vector4( m.M21,  m.M22, -m.M23,  m.M24),
            new Vector4(-m.M31, -m.M32,  m.M33, -m.M34),
            new Vector4( m.M41,  m.M42, -m.M43,  m.M44));

        public static System.Numerics.Vector3 ToSystem(this UnityEngine.Vector3 v) => new System.Numerics.Vector3(v.x, v.y, -v.z);
        public static System.Numerics.Quaternion ToSystem(this UnityEngine.Quaternion q) => new System.Numerics.Quaternion(-q.x, -q.y, q.z, q.w);
        public static System.Numerics.Matrix4x4 ToSystem(this UnityEngine.Matrix4x4 m) => new System.Numerics.Matrix4x4(
            m.m00,  m.m10, -m.m20,  m.m30,
            m.m01,  m.m11, -m.m21,  m.m31,
           -m.m02, -m.m12,  m.m22, -m.m32,
            m.m03,  m.m13, -m.m23,  m.m33);

使用 HolographicFrame 本机数据Using HolographicFrame native data


更改通过 HolographicFrameNativeData 接收的本机对象的状态可能会导致不可预知的行为和呈现项目,特别是当 Unity 也导致相同状态时。Changing the state of the native objects received via HolographicFrameNativeData may cause unpredictable behaviour and rendering artifacts, especially if Unity also reasons about that same state. 例如,您不应调用 HolographicFrame. UpdateCurrentPrediction,否则 Unity 呈现与该框架的姿势预测将与 Windows 所需的姿势不同步,这将减少全息图的 稳定性For example, you should not call HolographicFrame.UpdateCurrentPrediction, or else the pose prediction that Unity renders with that frame will be out of sync with the pose that Windows is expecting, which will reduce hologram stability.

如果需要访问本机接口以进行呈现或调试,请在本机插件或 c # 代码中使用 HolographicFrameNativeData 中的数据。If you need access to native interfaces for rendering or debugging purposes, use data from HolographicFrameNativeData in your native plugins or C# code.

下面的示例演示如何使用 HolographicFrameNativeData 获取当前帧预测的 photon 时间。Here's an example of how you can use HolographicFrameNativeData to get the current frame's prediction for photon time.

using System;
using System.Runtime.InteropServices;

public static bool GetCurrentFrameDateTime(out DateTime frameDateTime)
    IntPtr nativeStruct = UnityEngine.XR.XRDevice.GetNativePtr();

    if (nativeStruct != IntPtr.Zero)
        HolographicFrameNativeData hfd = Marshal.PtrToStructure<HolographicFrameNativeData>(nativeStruct);

        if (hfd.IHolographicFramePtr != IntPtr.Zero)
            var holographicFrame = (Windows.Graphics.Holographic.HolographicFrame)Marshal.GetObjectForIUnknown(hfd.IHolographicFramePtr);
            frameDateTime = holographicFrame.CurrentPrediction.Timestamp.TargetTime.DateTime;
            return true;


    frameDateTime = DateTime.MinValue;
    return false;

另请参阅See Also