您现在访问的是微软AZURE全球版技术文档网站,若需要访问由世纪互联运营的MICROSOFT AZURE中国区技术文档网站,请访问 https://docs.azure.cn.

教程:接口和自定义模型Tutorial: Interfaces and custom models

在本教程中,你将了解如何执行以下操作:In this tutorial, you learn how to:

  • 将混合现实工具包添加到项目Add Mixed Reality Toolkit to the project
  • 管理模型状态Manage model state
  • 为 Azure Blob 存储配置模型转换Configure Azure Blob Storage for model ingestion
  • 上传和处理模型以实现渲染Upload and process models for rendering

先决条件Prerequisites

混合现实工具包 (MRTK) 入门Get started with the Mixed Reality Toolkit (MRTK)

混合现实工具包 (MRTK) 是用于构建混合现实体验的跨平台工具包。The Mixed Reality Toolkit (MRTK) is a cross-platform toolkit for building mixed reality experiences. 我们将使用 MRTK 2.5.1 实现其交互和可视化功能。We'll use MRTK 2.5.1 for its interaction and visualization features.

若要添加 MRTK,请按照 MRTK 安装指南中列出的所需步骤进行操作。To add MRTK, follow the Required steps listed in the MRTK Installation Guide.

这些步骤如下:Those steps are:

导入本教程要使用的资产Import assets used by this tutorial

从本章开始,我们将为涉及到的大部分材料实现一种简单的“模型-视图-控制器”模式Starting in this chapter, we'll implement a simple model-view-controller pattern for much of the material covered. 该模式的“模型”部分是特定于 Azure 远程渲染的代码以及与 Azure 远程渲染相关的状态管理。The model part of the pattern is the Azure Remote Rendering specific code and the state management related to Azure Remote Rendering. 该模式的“视图”和“控制器”部分是使用 MRTK 资产和某些自定义脚本实现的 。The view and controller parts of the pattern are implemented using MRTK assets and some custom scripts. 在本教程中,可以只使用模型,而无需实现“视图-控制器” 。It is possible to use the model in this tutorial without the view-controller implemented here. 通过这种分离,你可以将本教程中的代码轻松集成到自己的应用程序中,在那里,代码将接管设计模式中的“视图-控制器”部分。This separation allows you to easily integrate the code found in this tutorial into your own application where it will take over the view-controller part of the design pattern.

通过引入 MRTK,现在可以将许多脚本、预制项和资产添加到项目中,用于支持交互和视觉反馈。With the introduction of MRTK, there are a number of scripts, prefabs, and assets that can now be added to the project to support interactions and visual feedback. 这些“教程资产”会捆绑到 Azure 远程渲染 GitHub 中“\Unity\TutorialAssets\TutorialAssets.unitypackage”路径下的一个 Unity 资产包中These assets, referred to as the Tutorial Assets, are bundled into a Unity Asset Package, which is included in the Azure Remote Rendering GitHub in '\Unity\TutorialAssets\TutorialAssets.unitypackage'.

  1. 如果下载操作会将 zip 提取到已知位置,请克隆或下载 git 存储库 Azure 远程渲染Clone or download the git repository Azure Remote Rendering, if downloading extract the zip to a known location.
  2. 在 Unity 项目中,选择“资产”->“导入包”->“自定义包”。In your Unity project, choose Assets -> Import Package -> Custom Package.
  3. 在文件资源管理器中,导航到克隆或解压缩的 Azure 远程渲染存储库的目录,然后选择“Unity”->“TutorialAssets”->“TutorialAssets.unitypackage”中的 .unitypackageIn the file explorer, navigate to the directory where you cloned or unzipped the Azure Remote Rendering repository, then select the .unitypackage found in Unity -> TutorialAssets -> TutorialAssets.unitypackage
  4. 选择“导入”按钮,将包内容导入到项目中。Select the Import button to import the contents of the package into your project.
  5. 在 Unity 编辑器中,从顶部菜单栏选择“混合现实工具包”->“实用工具”->“升级轻量级渲染管道的 MRTK 标准着色器”,并按照提示升级着色器。In the Unity Editor, select Mixed Reality Toolkit -> Utilities -> Upgrade MRTK Standard Shader for Lightweight Render Pipeline from the top menu bar and follow the prompts to upgrade the shader.

在项目中包括 MRTK 和教程资产后,我们会将 MRTK 配置文件切换到一个更适合教程的配置文件。Once MRTK and the Tutorial Assets are included in the project, we'll switch the MRTK profile to one more suitable for the tutorial.

  1. 选择场景层次结构中的 MixedRealityToolkit GameObject。Select the MixedRealityToolkit GameObject in the scene hierarchy.
  2. 在检查器的 MixedRealityToolkit 组件下,将配置文件切换为 ARRMixedRealityToolkitConfigurationProfile。In the Inspector, under the MixedRealityToolkit component, switch the configuration profile to ARRMixedRealityToolkitConfigurationProfile.
  3. 按 Ctrl+S 保存所做的更改。Press Ctrl+S to save your changes.

随即将主要使用默认的 HoloLens 2 配置文件配置 MRTK。This will configure MRTK, primarily, with the default HoloLens 2 profiles. 提供的配置文件按以下方式预配置:The provided profiles are pre-configured in the following ways:

  • 关闭探查器(在设备上可按 9 打开/关闭,或说“显示/隐藏探查器”)。Turn off the profiler (Press 9 to toggle it on/off, or say "Show/Hide Profiler" on device).
  • 关闭“眼睛凝视”光标。Turn off the eye gaze cursor.
  • 启用 Unity 鼠标单击,这样可使用鼠标而不是模拟手势来单击 MRTK UI 元素。Enable Unity mouse clicks, so you can click MRTK UI elements with the mouse instead of the simulated hand.

添加应用菜单Add the App Menu

本教程中的大部分视图控制器操作是针对抽象基类而不是具体基类。Most of the view controllers in this tutorial operate against abstract base classes instead of against concrete classes. 此模式提供了更大的灵活性,使我们能够为你提供视图控制器,同时还有助于你了解 Azure 远程渲染代码。This pattern provides more flexibility and allows us to provide the view controllers for you, while still helping you learn the Azure Remote Rendering code. 为简单起见,RemoteRenderingCoordinator 类未提供抽象类,并且其视图控制器直接针对具体类进行操作。For simplicity, the RemoteRenderingCoordinator class does not have an abstract class provided and its view controller operates directly against the concrete class.

现在可以将预制项 AppMenu 添加到场景中来获取当前会话状态的视觉反馈了。You can now add the prefab AppMenu to the scene, for visual feedback of the current session state. 此视图控制器将在实现更多 ARR 功能并将其集成到场景中时“解锁”更多子菜单视图控制器。This view controller will "unlock" more sub menu view controllers as we implement and integrate more ARR features into the scene. 现在,AppMenu 将以可视方式指示 ARR 状态,并显示用户用于授权应用程序连接 ARR 的模式面板。For now, the AppMenu will have a visual indication of the ARR state and present the modal panel that the user uses to authorize the application to connect to ARR.

  1. 在 Assets/RemoteRenderingTutorial/Prefabs/AppMenu 中找到 AppMenu 预制项Locate the AppMenu prefab in Assets/RemoteRenderingTutorial/Prefabs/AppMenu
  2. 将 AppMenu 预制项拖到场景中。Drag the AppMenu prefab into the scene.
  3. 你可能会看到“TMP 导入工具”对话框,因为这是第一次在场景中加入“文本网格 Pro”资产。You'll likely see a dialog for TMP Importer, since this is the first time we're including Text Mesh Pro assets in the scene. 按照“导入 TMP 基本要素”的提示操作。Follow the prompts to Import TMP Essentials. 然后关闭导入工具对话框,这里不需要示例和其他功能。Then close the importer dialog, the examples and extras are not needed.
  4. AppMenu 配置为自动挂钩,并提供“同意连接到会话”的模式,以便能够删除之前放置的旁路。The AppMenu is configured to automatically hook up and provide the modal for consenting to connecting to a Session, so we can remove the bypass placed earlier. 在 RemoteRenderingCoordinator GameObject 上,按“请求授权时”事件上的“-”按钮,删除先前实现的授权旁路 。On the RemoteRenderingCoordinator GameObject, remove the bypass for authorization we implemented previously, by pressing the '-' button on the On Requesting Authorization event. 删除旁路Remove bypass.
  5. 在 Unity 编辑器中按“播放”来测试视图控制器。Test the view controller by pressing Play in the Unity Editor.
  6. 在编辑器中,已配置了 MRTK,现在可以使用 WASD 键更改视图的位置,然后按住鼠标右键同时移动鼠标来更改视图方向。In the Editor, now that MRTK is configured, you can use the WASD keys to change the position your view and holding the right mouse button + moving the mouse to change your view direction. 尝试在各方向上稍微移动场景,感受控制的感觉。Try "driving" around the scene a bit to get a feel for the controls.
  7. 在设备上,可以通过举起手掌来召唤 AppMenu,在 Unity 编辑器中,可以使用热键“M”来实现此目的。On device, you can raise your palm up to summon the AppMenu, in the Unity Editor, use the hotkey 'M'.
  8. 如果看不到菜单,可按“M”键召唤菜单。If you've lost sight of the menu, press the 'M' key to summon the menu. 菜单将位于相机附近,便于交互。The menu will be placed near the camera for easy interaction.
  9. 授权现在显示为 AppMenu右侧的请求,从现在开始,使用授权应用管理远程渲染会话。The authorization will now show as a request to the right of the AppMenu, from now on, you'll use this to authorize the app to manage remote rendering sessions. UI 授权UI authorize
  10. 让 Unity 停止播放,以继续学习本教程。Stop Unity from playing to continue with the tutorial.

管理模型状态Manage model state

现在,我们将实现一个新的脚本 RemoteRenderedModel,用于跟踪状态、响应事件、引发事件和进行配置。Now we'll implement a new script, RemoteRenderedModel that is for tracking state, responding to events, firing events, and configuration. 实质上,RemoteRenderedModel 在 modelPath 中存储模型数据的远程路径。Essentially, RemoteRenderedModel stores the remote path for the model data in modelPath. 它侦听 RemoteRenderingCoordinator 中的状态更改,以确认自己是否应自动加载或卸载定义的模型。It will listen for state changes in the RemoteRenderingCoordinator to see if it should automatically load or unload the model it defines. 附加了 RemoteRenderedModel 的 GameObject 是远程内容的本地父级。The GameObject that has the RemoteRenderedModel attached to it will be the local parent for the remote content.

请注意,RemoteRenderedModel 脚本实现所包含的、教程资产中的 BaseRemoteRenderedModel 。Notice that the RemoteRenderedModel script implements BaseRemoteRenderedModel, included from Tutorial Assets. 这使远程模型视图控制器能够与脚本绑定。This will allow the remote model view controller to bind with your script.

  1. 在与 RemoteRenderingCoordinator 相同的文件夹中创建一个名为 RemoteRenderedModel 的新脚本 。Create a new script named RemoteRenderedModel in the same folder as RemoteRenderingCoordinator. 将整个内容替换为以下代码:Replace the entire contents with the following code:

    // Copyright (c) Microsoft Corporation. All rights reserved.
    // Licensed under the MIT License. See LICENSE in the project root for license information.
    
    using Microsoft.Azure.RemoteRendering;
    using Microsoft.Azure.RemoteRendering.Unity;
    using System;
    using UnityEngine;
    using UnityEngine.Events;
    
    public class RemoteRenderedModel : BaseRemoteRenderedModel
    {
        public bool AutomaticallyLoad = true;
    
        private ModelState currentModelState = ModelState.NotReady;
    
        [SerializeField]
        [Tooltip("The friendly name for this model")]
        private string modelDisplayName;
        public override string ModelDisplayName { get => modelDisplayName; set => modelDisplayName = value; }
    
        [SerializeField]
        [Tooltip("The URI for this model")]
        private string modelPath;
        public override string ModelPath
        {
            get => modelPath.Trim();
            set => modelPath = value;
        }
    
        public override ModelState CurrentModelState
        {
            get => currentModelState;
            protected set
            {
                if (currentModelState != value)
                {
                    currentModelState = value;
                    ModelStateChange?.Invoke(value);
                }
            }
        }
    
        public override event Action<ModelState> ModelStateChange;
        public override event Action<float> LoadProgress;
        public override Entity ModelEntity { get; protected set; }
    
        public UnityEvent OnModelNotReady = new UnityEvent();
        public UnityEvent OnModelReady = new UnityEvent();
        public UnityEvent OnStartLoading = new UnityEvent();
        public UnityEvent OnModelLoaded = new UnityEvent();
        public UnityEvent OnModelUnloading = new UnityEvent();
    
        public UnityFloatEvent OnLoadProgress = new UnityFloatEvent();
    
        public void Awake()
        {
            // Hook up the event to the Unity event
            LoadProgress += (progress) => OnLoadProgress?.Invoke(progress);
    
            ModelStateChange += HandleUnityStateEvents;
        }
    
        private void HandleUnityStateEvents(ModelState modelState)
        {
            switch (modelState)
            {
                case ModelState.NotReady:  OnModelNotReady?.Invoke();  break;
                case ModelState.Ready:     OnModelReady?.Invoke();     break;
                case ModelState.Loading:   OnStartLoading?.Invoke();   break;
                case ModelState.Loaded:    OnModelLoaded?.Invoke();    break;
                case ModelState.Unloading: OnModelUnloading?.Invoke(); break;
            }
        }
    
        private void Start()
        {
            //Attach to and initialize current state (in case we're attaching late)
            RemoteRenderingCoordinator.CoordinatorStateChange += Instance_CoordinatorStateChange;
            Instance_CoordinatorStateChange(RemoteRenderingCoordinator.instance.CurrentCoordinatorState);
        }
    
        /// <summary>
        /// Listen for state changes on the coordinator, clean up this model's remote objects if we're no longer connected.
        /// Automatically load if required
        /// </summary>
        private void Instance_CoordinatorStateChange(RemoteRenderingCoordinator.RemoteRenderingState state)
        {
            switch (state)
            {
                case RemoteRenderingCoordinator.RemoteRenderingState.RuntimeConnected:
                    CurrentModelState = ModelState.Ready;
                    if (AutomaticallyLoad)
                        LoadModel();
                    break;
                default:
                    UnloadModel();
                    break;
            }
        }
    
        private void OnDestroy()
        {
            RemoteRenderingCoordinator.CoordinatorStateChange -= Instance_CoordinatorStateChange;
            UnloadModel();
        }
    
        /// <summary>
        /// Asks the coordinator to create a model entity and listens for coordinator state changes
        /// </summary>
        [ContextMenu("Load Model")]
        public override async void LoadModel()
        {
            if (CurrentModelState != ModelState.Ready)
                return; //We're already loaded, currently loading, or not ready to load
    
            CurrentModelState = ModelState.Loading;
    
            ModelEntity = await RemoteRenderingCoordinator.instance?.LoadModel(ModelPath, this.transform, SetLoadingProgress);
    
            if (ModelEntity != null)
                CurrentModelState = ModelState.Loaded;
            else
                CurrentModelState = ModelState.Error;
        }
    
        /// <summary>
        /// Clean up the local model instances
        /// </summary>
        [ContextMenu("Unload Model")]
        public override void UnloadModel()
        {
            CurrentModelState = ModelState.Unloading;
    
            if (ModelEntity != null)
            {
                var modelGameObject = ModelEntity.GetOrCreateGameObject(UnityCreationMode.DoNotCreateUnityComponents);
                Destroy(modelGameObject);
                ModelEntity.Destroy();
                ModelEntity = null;
            }
    
            if (RemoteRenderingCoordinator.instance.CurrentCoordinatorState == RemoteRenderingCoordinator.RemoteRenderingState.RuntimeConnected)
                CurrentModelState = ModelState.Ready;
            else
                CurrentModelState = ModelState.NotReady;
        }
    
        /// <summary>
        /// Update the Unity progress event
        /// </summary>
        /// <param name="progressValue"></param>
        public override void SetLoadingProgress(float progressValue)
        {
            LoadProgress?.Invoke(progressValue);
        }
    }
    

在最基本的条件下,RemoteRenderedModel 保存模型(在此例中为 SAS 或 builtin:// URI)加载所需的数据并跟踪远程模型状态。In the most basic terms, RemoteRenderedModel holds the data needed to load a model (in this case the SAS or builtin:// URI) and tracks the remote model state. 在加载时,在 RemoteRenderingCoordinator 上调用 LoadModel 方法,并返回包含模型的实体以供引用和卸载。When it's time to load, the LoadModel method is called on RemoteRenderingCoordinator and the Entity containing the model is returned for reference and unloading.

加载测试模型Load the Test Model

现在通过再次加载测试模型来测试新脚本。Let's test the new script by loading the test model again. 我们将创建一个包含脚本的游戏对象,并将其作为测试模型的父级。We'll create a Game Object to contain the script and be a parent to the test model.

  1. 在场景中创建新的空游戏对象并将其命名为 TestModel。Create a new empty Game Object in the scene and name it TestModel.
  2. 将 RemoteRenderedModel 脚本添加到 TestModel。Add the RemoteRenderedModel script to TestModel. 添加 RemoteRenderedModel 组件Add RemoteRenderedModel component
  3. 分别用“TestModel”和“builtin://Engine”填写 Model Display NameModel PathFill in the Model Display Name and the Model Path with "TestModel" and "builtin://Engine" respectively. 指定模型详细信息Specify model details
  4. 将 TestModel 对象放置在相机前面,位置为 x = 0, y = 0, z = 3 。Position the TestModel object in front of the camera, at position x = 0, y = 0, z = 3. 位置对象Position object
  5. 确保启用 AutomaticallyLoad。Ensure AutomaticallyLoad is turned on.
  6. 在 Unity 编辑器中按“播放”来测试应用程序。Press Play in the Unity Editor to test the application.
  7. 单击“连接”按钮授予权限,允许应用创建会话,然后它会连接到会话并自动加载模型。Grant authorization by clicking the Connect button to allow the app to create a session and it will connect to a Session and automatically load the model.

当应用程序通过其状态进行监视时,请监视控制台。Watch the Console as the application progresses through its states. 请记住,某些状态可能需要一些时间才能完成,并且不会显示进度。Keep in mind, some states may take some time to complete, and won't show progress. 最终,你会看到模型加载中的日志,然后场景中会呈现测试模型。Eventually, you'll see the logs from the model loading and then the test model will be rendered in the scene.

尝试在检查器或场景视图中通过转换功能移动和旋转 TestModel GameObject。Try moving and rotating the TestModel GameObject via the Transform in the Inspector, or in the Scene view. 你会看到模型在游戏视图中移动和旋转。You'll see the model move and rotate it in the Game view.

Unity 日志

预配 Azure 中的 Blob 存储和自定义模型引入Provision Blob Storage in Azure and custom model ingestion

现在,我们可以尝试加载自己的模型。Now we can try loading your own model. 要执行此操作,你需要配置 Blob 存储,并在 Azure 上上传和转换模型,然后我们会使用 RemoteRenderedModel 脚本加载模型。To do that, you'll need to configure Blob Storage and on Azure, upload and convert a model, then we'll load the model using the RemoteRenderedModel script. 如果此时你没有要加载的自己的模型,则可以安全地跳过自定义模型加载步骤。The custom model loading steps can be safely skipped if you don't have your own model to load at this time.

按照 快速入门:转换要渲染的模型中指定的步骤进行操作。Follow the steps specified in the Quickstart: Convert a model for rendering. 对于本教程的目的,请跳过“将新模型插入快速入门示例应用”部分。Skip the Insert new model into Quickstart Sample App section for the purpose of this tutorial. 引入模型的共享访问签名 (SAS) URI 后,请继续执行下面的下一步。Once you have your ingested model's Shared Access Signature (SAS) URI, continue to the next step below.

加载和渲染自定义模型Load and rendering a custom model

  1. 在场景中创建新的空 GameObject,并将其命名为类似于自定义模型。Create a new empty GameObject in the scene and name it similar to your custom model.

  2. 将 RemoteRenderedModel 脚本添加到新创建的 GameObject。Add the RemoteRenderedModel script to the newly created GameObject. 添加 RemoteRenderedModel 组件Add RemoteRenderedModel component

  3. 使用适当的模型名称填充 Model Display NameFill in the Model Display Name with an appropriate name for your model.

  4. 使用在上面的引入步骤中创建的模型共享访问签名 (SAS) URI 填充 Model PathFill in the Model Path with the model's Shared Access Signature (SAS) URI you created in the ingestion steps above.

  5. 将 GameObject 放置在相机前面,位置为 x = 0, y = 0, z = 3。Position the GameObject in front of the camera, at position x = 0, y = 0, z = 3.

  6. 确保启用 AutomaticallyLoad。Ensure AutomaticallyLoad is turned on.

  7. 在 Unity 编辑器中按“播放”来测试应用程序。Press Play in the Unity Editor to test the application.

    你会看到,控制台开始填充当前状态,最后会看到模型加载进度消息。You will see the Console begin to populate with the current state, and eventually, model loading progress messages. 然后,场景中载入自定义模型。Your custom model will then load into the scene.

  8. 从场景中删除自定义模型对象。Remove your custom model object from the scene. 本教程中的最佳实践将使用测试模型。The best experience for this tutorial will be using the test model. 虽然 ARR 中支持多个模型,但在按照本教程操作时,最好一次支持一个远程模型。While multiple models are certainly supported in ARR, this tutorial was written to best support a single remote model at a time.

后续步骤Next steps

现在你可以将自己的模型加载到 Azure 远程渲染并在应用程序中进行查看了!You can now load your own models into Azure Remote Rendering and view them in your application! 接下来介绍如何操作模型。Next, we'll guide you through manipulating your models.