Azure Remote Rendering client library for Java - version 1.1.28

  • Article

Azure Remote Rendering (ARR) is a service that enables you to render high-quality, interactive 3D content in the cloud and stream it in real time to devices, such as the HoloLens 2.

This SDK offers functionality to convert assets to the format expected by the runtime, and also to manage the lifetime of remote rendering sessions.

NOTE: Once a session is running, a client application will connect to it using one of the "runtime SDKs". These SDKs are designed to best support the needs of an interactive application doing 3d rendering. They are available in .NET and C++.

Source code | API reference documentation | Product Documentation

Getting started

Prerequisites

Include the package

Include the BOM file

Please include the azure-sdk-bom to your project to take dependency on the General Availability (GA) version of the library. In the following snippet, replace the {bom_version_to_target} placeholder with the version number. To learn more about the BOM, see the AZURE SDK BOM README.

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>com.azure</groupId>
            <artifactId>azure-sdk-bom</artifactId>
            <version>{bom_version_to_target}</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

and then include the direct dependency in the dependencies section without the version tag as shown below.

<dependencies>
  <dependency>
    <groupId>com.azure</groupId>
    <artifactId>azure-mixedreality-remoterendering</artifactId>
  </dependency>
</dependencies>

Include direct dependency

Note: This version targets Azure Remote Rendering service API version v2021-01-01.

Add the following Maven dependency:

<dependency>
    <groupId>com.azure</groupId>
    <artifactId>azure-mixedreality-remoterendering</artifactId>
    <version>1.1.28</version>
</dependency>

Authenticate the client

Constructing a remote rendering client requires an authenticated account, and a remote rendering endpoint. An account consists of its accountId and an account domain. For an account created in the eastus region, the account domain will have the form "eastus.mixedreality.azure.com". There are several forms of authentication:

  • Account Key authentication
    • Account keys enable you to get started quickly with using Azure Remote Rendering. But before you deploy your application to production, we recommend that you update your app to use Azure AD authentication.
  • Azure Active Directory (AD) token authentication
    • If you're building an enterprise application and your company is using Azure AD as its identity system, you can use user-based Azure AD authentication in your app. You then grant access to your Azure Remote Rendering accounts by using your existing Azure AD security groups. You can also grant access directly to users in your organization.
    • Otherwise, we recommend that you obtain Azure AD tokens from a web service that supports your app. We recommend this method for production applications because it allows you to avoid embedding the credentials for access to Azure Spatial Anchors in your client application.

See here for detailed instructions and information.

In all the following examples, the client is constructed with a RemoteRenderingClientBuilder object. The parameters are always the same, except for the credential object, which is explained in each example. The remoteRenderingEndpoint parameter is a URL that determines the region in which the service performs its work. An example is https://remoterendering.eastus2.mixedreality.azure.com.

NOTE: For converting assets, it is preferable to pick a region close to the storage containing the assets.

NOTE: For rendering, it is strongly recommended that you pick the closest region to the devices using the service. The time taken to communicate with the server impacts the quality of the experience.

Authenticating with account key authentication

Use the AzureKeyCredential object to use an account identifier and account key to authenticate:

AzureKeyCredential credential = new AzureKeyCredential(environment.getAccountKey());

RemoteRenderingClient client = new RemoteRenderingClientBuilder()
    .accountId(environment.getAccountId())
    .accountDomain(environment.getAccountDomain())
    .endpoint(environment.getServiceEndpoint())
    .credential(credential)
    .buildClient();

Authenticating with an AAD client secret

Use the ClientSecretCredential object to perform client secret authentication.

ClientSecretCredential credential = new ClientSecretCredentialBuilder()
    .tenantId(environment.getTenantId())
    .clientId(environment.getClientId())
    .clientSecret(environment.getClientSecret())
    .authorityHost("https://login.microsoftonline.com/" + environment.getTenantId())
    .build();

RemoteRenderingClient client = new RemoteRenderingClientBuilder()
    .accountId(environment.getAccountId())
    .accountDomain(environment.getAccountDomain())
    .endpoint(environment.getServiceEndpoint())
    .credential(credential)
    .buildClient();

Authenticating a user using device code authentication

Use the DeviceCodeCredential object to perform device code authentication.

DeviceCodeCredential credential = new DeviceCodeCredentialBuilder()
    .challengeConsumer((DeviceCodeInfo deviceCodeInfo) -> {
        logger.info(deviceCodeInfo.getMessage());
    })
    .clientId(environment.getClientId())
    .tenantId(environment.getTenantId())
    .authorityHost("https://login.microsoftonline.com/" + environment.getTenantId())
    .build();

RemoteRenderingClient client = new RemoteRenderingClientBuilder()
    .accountId(environment.getAccountId())
    .accountDomain(environment.getAccountDomain())
    .endpoint(environment.getServiceEndpoint())
    .credential(credential)
    .buildClient();

See here for more information about using device code authentication flow.

Interactive authentication with DefaultAzureCredential

Use the DefaultAzureCredential object:

DefaultAzureCredential credential = new DefaultAzureCredentialBuilder().build();

RemoteRenderingClient client = new RemoteRenderingClientBuilder()
    .accountId(environment.getAccountId())
    .accountDomain(environment.getAccountDomain())
    .endpoint(environment.getServiceEndpoint())
    .credential(credential)
    .buildClient();

Authenticating with a static access token

You can pass a Mixed Reality access token as an AccessToken previously retrieved from the Mixed Reality STS service to be used with a Mixed Reality client library:

// GetMixedRealityAccessTokenFromWebService is a hypothetical method that retrieves
// a Mixed Reality access token from a web service. The web service would use the
// MixedRealityStsClient and credentials to obtain an access token to be returned
// to the client.
AccessToken accessToken = getMixedRealityAccessTokenFromWebService();

RemoteRenderingClient client = new RemoteRenderingClientBuilder()
    .accountId(environment.getAccountId())
    .accountDomain(environment.getAccountDomain())
    .endpoint(environment.getServiceEndpoint())
    .accessToken(accessToken)
    .buildClient();

Key concepts

RemoteRenderingClient

The RemoteRenderingClient is the client library used to access the RemoteRenderingService. It provides methods to create and manage asset conversions and rendering sessions.

Examples

Convert a simple asset

We assume that a RemoteRenderingClient has been constructed as described in the Authenticate the Client section. The following snippet describes how to request that "box.fbx", found at the root of the blob container at the given URL, gets converted.

AssetConversionOptions conversionOptions = new AssetConversionOptions()
    .setInputStorageContainerUrl(getStorageURL())
    .setInputRelativeAssetPath("box.fbx")
    .setOutputStorageContainerUrl(getStorageURL());

// A randomly generated UUID is a good choice for a conversionId.
String conversionId = UUID.randomUUID().toString();

SyncPoller<AssetConversion, AssetConversion> conversionOperation = client.beginConversion(conversionId, conversionOptions);

The output files will be placed beside the input asset.

Convert a more complex asset

Assets can reference other files, and blob containers can contain files belonging to many different assets. In this example, we show how prefixes can be used to organize your blobs and how to convert an asset to take account of that organization. Assume that the blob container at inputStorageURL contains many files, including "Bicycle/bicycle.gltf", "Bicycle/bicycle.bin" and "Bicycle/saddleTexture.jpg". (So the prefix "Bicycle" is acting very like a folder.) We want to convert the gltf so that it has access to the other files which share the prefix, without requiring the conversion service to access any other files. To keep things tidy, we also want the output files to be written to a different storage container and given a common prefix: "ConvertedBicycle". The code is as follows:

AssetConversionOptions conversionOptions = new AssetConversionOptions()
    .setInputStorageContainerUrl(inputStorageURL)
    .setInputRelativeAssetPath("bicycle.gltf")
    .setInputBlobPrefix("Bicycle")
    .setOutputStorageContainerUrl(outputStorageURL)
    .setOutputBlobPrefix("ConvertedBicycle");

String conversionId = UUID.randomUUID().toString();

SyncPoller<AssetConversion, AssetConversion> conversionOperation = client.beginConversion(conversionId, conversionOptions);

NOTE: when a prefix is given in the input options, then the input file parameter is assumed to be relative to that prefix. The same applies to the output file parameter in output options.

Get the output when an asset conversion has finished

Converting an asset can take anywhere from seconds to hours. This code uses an existing conversionOperation and polls regularly until the conversion has finished or failed. The default polling period is 10 seconds. Note that a conversionOperation can be constructed from the conversionId of an existing conversion and a client.

AssetConversion conversion = conversionOperation.getFinalResult();
if (conversion.getStatus() == AssetConversionStatus.SUCCEEDED) {
    logger.info("Conversion succeeded: Output written to {}", conversion.getOutputAssetUrl());
} else if (conversion.getStatus() == AssetConversionStatus.FAILED) {
    logger.error("Conversion failed: {} {}", conversion.getError().getCode(), conversion.getError().getMessage());
} else {
    logger.error("Unexpected conversion status: {}", conversion.getStatus());
}

List conversions

You can get information about your conversions using the listConversions method. This method may return conversions which have yet to start, conversions which are running and conversions which have finished. In this example, we just list the output URLs of successful conversions started in the last day.

for (AssetConversion conversion : client.listConversions()) {
    if ((conversion.getStatus() == AssetConversionStatus.SUCCEEDED)
        && (conversion.getCreationTime().isAfter(OffsetDateTime.now().minusDays(1)))) {
        logger.info("Output Asset URL: {}", conversion.getOutputAssetUrl());
    }
}

Create a rendering session

We assume that a RemoteRenderingClient has been constructed as described in the Authenticate the Client section. The following snippet describes how to request that a new rendering session be started.

BeginSessionOptions options = new BeginSessionOptions()
    .setMaxLeaseTime(Duration.ofMinutes(30))
    .setSize(RenderingSessionSize.STANDARD);

// A randomly generated GUID is a good choice for a sessionId.
String sessionId = UUID.randomUUID().toString();

SyncPoller<RenderingSession, RenderingSession> startSessionOperation = client.beginSession(sessionId, options);

Extend the lease time of a session

If a session is approaching its maximum lease time, but you want to keep it alive, you will need to make a call to increase its maximum lease time. This example shows how to query the current properties and then extend the lease if it will expire soon.

NOTE: The runtime SDKs also offer this functionality, and in many typical scenarios, you would use them to extend the session lease.

RenderingSession currentSession = client.getSession(sessionId);

Duration sessionTimeAlive = Duration.between(OffsetDateTime.now(), currentSession.getCreationTime()).abs();
if (currentSession.getMaxLeaseTime().minus(sessionTimeAlive).toMinutes() < 2) {
    Duration newLeaseTime = currentSession.getMaxLeaseTime().plus(Duration.ofMinutes(30));
    UpdateSessionOptions longerLeaseOptions = new UpdateSessionOptions().maxLeaseTime(newLeaseTime);
    client.updateSession(sessionId, longerLeaseOptions);
}

List rendering sessions

You can get information about your sessions using the listSessions method. This method may return sessions which have yet to start and sessions which are ready.

for (RenderingSession session : client.listSessions()) {
    if (session.getStatus() == RenderingSessionStatus.STARTING) {
        logger.info("Session {} is starting.");
    } else if (session.getStatus() == RenderingSessionStatus.READY) {
        logger.info("Session {} is ready at host {}", session.getId(), session.getHostname());
    } else if (session.getStatus() == RenderingSessionStatus.ERROR) {
        logger.error("Session {} encountered an error: {} {}", session.getId(), session.getError().getCode(), session.getError().getMessage());
    } else {
        logger.error("Session {} has unexpected status {}", session.getId(), session.getStatus());
    }
}

Stop a session

The following code will stop a running session with given id.

client.endSession(sessionId);

Troubleshooting

For general troubleshooting advice concerning Azure Remote Rendering, see the Troubleshoot page for remote rendering at docs.microsoft.com.

The client methods will throw exceptions if the request cannot be made. However, in the case of both conversions and sessions, the requests can succeed but the requested operation may not be successful. In this case, no exception will be thrown, but the returned objects can be inspected to understand what happened.

If the asset in a conversion is invalid, the conversion operation will return an AssetConversion object with a Failed status and carrying a RemoteRenderingServiceError with details. Once the conversion service is able to process the file, an <assetName>.result.json file will be written to the output container. If the input asset is invalid, then that file will contain a more detailed description of the problem.

Similarly, sometimes when a session is requested, the session ends up in an error state. The startSessionOperation method will return a RenderingSession object, but that object will have an Error status and carry a RemoteRenderingServiceError with details.

Next steps

  • Read the Product documentation
  • Learn about the runtime SDKs:
    • .NET: /dotnet/api/microsoft.azure.remoterendering
    • C++: /cpp/api/remote-rendering/

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Impressions