混合現實 OpenXR Unity 中支援的功能Mixed Reality OpenXR supported features in Unity

Mixed Reality OpenXR 外掛程式 套件是 Unity OpenXR 外掛程式 的延伸模組,並支援 HoloLens 2 和 Windows Mixed Reality 耳機的功能套件。The Mixed Reality OpenXR Plugin package is an extension of Unity's OpenXR Plugin and supports a suite of features for HoloLens 2 and Windows Mixed Reality headsets. 繼續之前,請確定您已安裝 unity 2020.2 或更新版本、 OpenXR 外掛程式版本 0.1.2 或更新版本,且您的 Unity 專案已 設定 OpenXRBefore continuing, make sure that you've installed Unity 2020.2 or later, OpenXR Plugin version 0.1.2 or later, and your Unity project is configured for OpenXR.

支援的項目What's supported

目前支援下列功能:The following features are currently supported:

  • 支援 HoloLens 2 的 UWP 應用程式,並針對 HoloLens 2 應用程式模型進行優化。Supports UWP applications for HoloLens 2, and optimize for HoloLens 2 application model.
  • 支援適用于 Windows Mixed Reality 耳機的 Win32 VR 應用程式,具有最新的控制器設定檔和全像應用程式遠端。Supports Win32 VR applications for Windows Mixed Reality headset with latest controller profiles and holographic app remoting.
  • 使用錨點和未系結空間的世界規模追蹤。World scale tracking using Anchors and Unbounded space.
  • 錨點儲存體 API,以將錨點保存 到 HoloLens 2 的本機儲存體。Anchor storage API to persist anchors to HoloLens 2 local storage.
  • 移動控制器和手互動,包括新的 HP 回音卡控制器。Motion controller and hand interactions, including the new HP Reverb G2 controller.
  • 使用26個接點和聯合半徑輸入,以明確表述的手勢進行追蹤。Articulated hand tracking using 26 joints and joint radius inputs.
  • HoloLens 2 上的眼睛注視互動。Eye gaze interaction on HoloLens 2.
  • HoloLens 2 上尋找 (PV) 攝影機的相片/影片。Locating photo/video (PV) camera on HoloLens 2.
  • 混合實境擷取使用透過 PV 攝影機的第三種眼睛呈現。Mixed Reality Capture using 3rd eye rendering through PV camera.
  • 支援「播放」以使用全像「 遠端處理」應用程式 HoloLens 2,讓開發人員不需要建立及部署到裝置,即可將腳本進行偵錯工具。Supports "Play" to HoloLens 2 with the Holographic Remoting app, allowing developers to debug scripts without building and deploying to the device.
  • 透過 MRTK OpenXR 提供者支援,與 MRTK Unity 2.5.3 和更新版本相容。Compatible with MRTK Unity 2.5.3 and newer through MRTK OpenXR provider support.
  • 與 Unity ARFoundation 4.0 或更新版本相容Compatible with Unity ARFoundation 4.0 or later

Unity 編輯器 play 模式的全像遠端功能Holographic Remoting in Unity Editor play mode

在 Visual Studio 專案中建立 UWP Unity 專案,然後將它封裝並部署到 HoloLens 2 裝置,可能需要一些時間。Building a UWP Unity project in Visual Studio project and then packaging and deploying it to a HoloLens 2 device can take some time. 其中一個解決方法是啟用全像「全像」編輯器遠端處理,讓您透過網路直接使用「播放」模式來將 c # 腳本進行 HoloLens 2 裝置的偵測。One solution is to enable the Holographic Editor Remoting, which lets you debug your C# script using “Play” mode directly to a HoloLens 2 device over your network. 此案例可避免建立 UWP 套件並將其部署至遠端裝置的額外負荷。This scenario avoids the overhead of building and deploying a UWP package to remote device.

  1. 首先,您必須從存放區的 HoloLens 2 安裝全像 遠端播放機應用程式First, you need to install the Holographic Remoting Player app from Store on your HoloLens 2

  2. 在 HoloLens 2 上執行全像遠端播放播放程式應用程式,您會看到要連接的版本號碼和 IP 位址Run the Holographic Remoting Player app on HoloLens 2 and you'll see the version number and IP address to connect to

    • 您將需要2.4 或更新版本才能使用 OpenXR 外掛程式You'll need v2.4 or later to work with the OpenXR plugin

    HoloLens 中執行的全像 Remoting 播放程式的螢幕擷取畫面

  3. 開啟 [ 編輯-> 專案設定],流覽至 XR 外掛程式管理,然後選取 [ Windows Mixed Reality 功能集 ] 方塊:Open Edit -> Project Settings, navigate to XR plug-in Management, and check the Windows Mixed Reality feature set box:

    醒目提示 XR 外掛程式管理的 Unity 編輯器中開啟之 [專案設定] 面板的螢幕擷取畫面

  4. 展開 [ OpenXR ] 底下的 [功能] 區段,然後選取 [全部顯示]Expand the Features section under OpenXR and select Show All

  5. 核取 [全像攝影 編輯器遠端處理 ] 核取方塊,然後輸入您從「全像」遠端應用程式取得的 IP 位址Check the Holographic Editor Remoting checkbox and input the IP address you get from the Holographic Remoting app:

    在 Unity 編輯器中開啟專案設定面板的螢幕擷取畫面,其中已醒目提示功能

現在您可以按一下 [播放] 按鈕,以在 HoloLens 上的全像遠端應用程式中播放 Unity 應用程式。Now you can click the “Play” button to play your Unity app into the Holographic Remoting app on your HoloLens. 您也可以 將 Visual Studio 附加至 Unity ,以在播放模式中進行 c # 腳本的調試。You can also attach Visual Studio to Unity to debug C# scripts in the play mode.

注意

從0.1.0 版起,全像「全像」遠端執行時間不支援錨點,而 ARAnchorManager 功能將無法透過遠端處理。As of version 0.1.0, the Holographic Remoting runtime doesn’t support Anchors, and ARAnchorManager functionalities will not work through remoting. 這項功能會在未來的版本中推出。This feature is coming in future releases.

錨點和錨點持續性Anchors and Anchor Persistence

Mixed Reality OpenXR 外掛程式透過 Unity 的 ARFoundation ARAnchorManager 的執行,提供基本錨定功能。The Mixed Reality OpenXR Plugin supplies basic anchor functionality through an implementation of Unity’s ARFoundation ARAnchorManager. 若要瞭解 ARFoundation 中 ARAnchor 的基本概念,請造訪 AR 錨點管理員的 ARFoundation 手冊To learn the basics on ARAnchor s in ARFoundation, visit the ARFoundation Manual for AR Anchor Manager. 從版本0.1.0,此外掛程式除了建立附加至平面的錨點(即將在未來版本中)之外,也支援所有 ARAnchorManager 功能。As of version 0.1.0, this plugin supports all ARAnchorManager functionality except creating anchors attached to a plane, which is coming in a future release.

錨點持續性和 XRAnchorStoreAnchor Persistence and the XRAnchorStore

另一個稱為 XRAnchorStore 的 API 可讓錨點在會話之間保存。An additional API called the XRAnchorStore enables anchors to be persisted between sessions. XRAnchorStore 是您裝置上已儲存錨點的標記法。The XRAnchorStore is a representation of the saved anchors on your device. 錨點可以從 Unity 場景中的 ARAnchors 保存、從儲存體載入至新的 ARAnchors,或從儲存體中刪除。Anchors can be persisted from ARAnchors in the Unity scene, loaded from storage into new ARAnchors, or deleted from storage.

注意

這些錨點會儲存並載入至相同的裝置。These anchors are to be saved and loaded on the same device. 未來版本中的 Azure 空間錨點將會支援跨裝置錨定儲存體。Cross-device anchor storage will be supported through Azure Spatial Anchors in a future release.

public class Microsoft.MixedReality.ARSubsystems.XRAnchorStore
{
    // A list of all persisted anchors, which can be loaded.
    public IReadOnlyList<string> PersistedAnchorNames { get; }

    // Clear all persisted anchors
    public void Clear();

    // Load a single persisted anchor by name. The ARAnchorManager will create this new anchor and report it in
    // the ARAnchorManager.anchorsChanged event. The TrackableId returned here is the same TrackableId the
    // ARAnchor will have when it is instantiated.
    public TrackableId LoadAnchor(string name);

    // Attempts to persist an existing ARAnchor with the given TrackableId to the local store. Returns true if
    // the storage is successful, false otherwise.
    public bool TryPersistAnchor(string name, TrackableId trackableId);

    // Removes a single persisted anchor from the anchor store. This will not affect any ARAnchors in the Unity
    // scene, only the anchors in storage.
    public void UnpersistAnchor(string name);
}

為了載入 XRAnchorStore,外掛程式會在 XRAnchorSubsystem (ARAnchorManager 的子系統)上提供擴充方法:To load the XRAnchorStore, the plugin provides an extension method on the XRAnchorSubsystem, the subsystem of an ARAnchorManager:

public static Task<XRAnchorStore> LoadAnchorStoreAsync(this XRAnchorSubsystem anchorSubsystem)

若要使用此擴充方法,請從 ARAnchorManager 的子系統進行存取,如下所示:To use this extension method, access it from an ARAnchorManager's subsystem as follows:

ARAnchorManager arAnchorManager = GetComponent<ARAnchorManager>();
XRAnchorStore anchorStore = await arAnchorManager.subsystem.LoadAnchorStoreAsync();

若要查看保存/unpersisting 錨點的完整範例,請查看 混合現實 OpenXR 外掛程式範例場景中的錨點 > 錨點範例 GameObject 和 AnchorsSample.cs 腳本:To see a full example of persisting / unpersisting anchors, check out the Anchors -> Anchors Sample GameObject and AnchorsSample.cs script in the Mixed Reality OpenXR Plugin Sample Scene:

在 Unity 編輯器中開啟之 [階層] 面板的螢幕擷取畫面,其中已醒目提示錨點範例

在 Unity 編輯器中開啟的 [偵測器] 面板的螢幕擷取畫面,其中已醒目提示錨點範例腳本

移動控制器和手互動Motion controller and hand interactions

若要瞭解 Unity 中混合現實互動的基本概念,請流覽 unity Manual For UNITY XR 輸入To learn the basics about mixed reality interactions in Unity, visit the Unity Manual for Unity XR Input. 這項 Unity 檔涵蓋從控制器專屬輸入到更一般化 InputFeatureUsage 的對應、如何識別和分類可用的 XR 輸入、如何從這些輸入讀取資料等等。This Unity documentation covers the mappings from controller-specific inputs to more generalizable InputFeatureUsage s, how available XR inputs can be identified and categorized, how to read data from these inputs, and more.

Mixed Reality OpenXR 外掛程式提供額外的輸入互動設定檔,對應至標準 InputFeatureUsage s,如下所述:The Mixed Reality OpenXR Plugin provides additional input interaction profiles, mapped to standard InputFeatureUsage s as detailed below:

InputFeatureUsageInputFeatureUsage HP 回音 (OpenXR) HP Reverb G2 Controller (OpenXR) HoloLens (OpenXR) HoloLens Hand (OpenXR)
primary2DAxisprimary2DAxis 操縱 杆Joystick
primary2DAxisClickprimary2DAxisClick 搖桿-按一下Joystick - Click
觸發程序 (trigger)trigger 觸發程序Trigger
grip Grip 點擊或擠壓Air tap or squeeze
primaryButtonprimaryButton [X/A]-按下[X/A] - Press 空中點選Air tap
secondaryButtonsecondaryButton [Y/B]-按下[Y/B] - Press
gripButtongripButton 握住-按Grip - Press
triggerButtontriggerButton 觸發程式-按下Trigger - Press
menuButtonmenuButton 功能表Menu

目標和底姿勢Aim and Grip Poses

您可以透過 OpenXR 輸入互動存取兩組姿勢:You have access to two sets of poses through OpenXR input interactions:

  • 抓手中呈現物件的底框The grip poses for rendering objects in the hand
  • 目標是指向世界。The aim poses for pointing into the world.

有關此設計的詳細資訊,以及這兩個姿勢之間的差異,請參閱 OpenXR 規格-輸入子路徑More information on this design and the differences between the two poses can be found in the OpenXR Specification - Input Subpaths.

InputFeatureUsages DevicePositionDeviceRotationDeviceVelocityDeviceAngularVelocity 提供 的姿勢全都代表 OpenXR 的 把手姿勢。Poses supplied by the InputFeatureUsages DevicePosition, DeviceRotation, DeviceVelocity, and DeviceAngularVelocity all represent the OpenXR grip pose. 與框姿勢相關的 InputFeatureUsages 是在 Unity 的 CommonUsages中定義。InputFeatureUsages related to grip poses are defined in Unity’s CommonUsages.

InputFeatureUsages PointerPositionPointerRotationPointerVelocityPointerAngularVelocity 提供的姿勢全都代表 OpenXR 的 目標Poses supplied by the InputFeatureUsages PointerPosition, PointerRotation, PointerVelocity, and PointerAngularVelocity all represent the OpenXR aim pose. 這些 InputFeatureUsages 不會定義在任何包含的 c # 檔案中,因此您必須定義您自己的 InputFeatureUsages,如下所示:These InputFeatureUsages aren't defined in any included C# files, so you'll need to define your own InputFeatureUsages as follows:

public static readonly InputFeatureUsage<Vector3> PointerPosition = new InputFeatureUsage<Vector3>("PointerPosition");

HapticsHaptics

如需在 Unity 的 XR 輸入系統中使用 haptics 的相關資訊,請參閱 unity Manual For UNITY XR 輸入-haptics中的檔。For information on using haptics in Unity’s XR Input system, documentation can be found at the Unity Manual for Unity XR Input - Haptics.

即將推出的內容What's coming soon

下列問題和遺漏的功能都是已知的混合現實 OpenXR 外掛程式 版本 0.1.0The following issues and missing features are known with Mixed Reality OpenXR plugin version 0.1.0. 我們正在處理這些問題,並將在即將推出的版本中發行修正程式和新功能。We're working on these and will release fixes and new features in upcoming releases.

  • 尚不支援 ARPlaneSubsystemARPlaneSubsystem is not supported yet. HoloLens 2 也不支援 ARPlaneManagerARRaycastManager 和相關的 API,例如 ARAnchorManager. AttachAnchorARPlaneManager, ARRaycastManager, and related API like ARAnchorManager.AttachAnchor are also not supported on HoloLens 2.
  • 目前尚未支援錨點,但不久的未來將會推出 錨點Anchor isn't supported by Holographic Remoting yet, but it's coming in the near future.
  • 目前尚不支援手上的 網格 追蹤、 QR 代碼XRMeshSubsystemHand Mesh tracking, QR Codes, and XRMeshSubsystem aren't supported yet.
  • Azure 空間錨點 支援即將在未來版本中推出。Azure Spatial Anchors support is coming in a future release.
  • ARM64 是 HoloLens 2 應用程式唯一支援的平臺。ARM64 is the only supported platform for HoloLens 2 apps. ARM 平臺即將在未來版本中推出。The ARM platform is coming in a future release.

疑難排解Troubleshooting

當您在 HoloLens 2 上暫停和繼續 Unity 應用程式時,應用程式無法正確地繼續,這會導致 HoloLens 視圖中有4個旋轉點。When you suspend and resume a Unity app on HoloLens 2, the app can't correctly resume, which leads to 4 spinning dots in the HoloLens view.

  • 將 OpenXR 專案設定中的 深度提交模式 設定為 [ ] 作為因應措施Set Depth submission Mode to None in the OpenXR project settings as a workaround