Upload usage data, metrics, and logs to Azure Monitor

Periodically, you can export out usage information for billing purposes, monitoring metrics, and logs and then upload it to Azure. The export and upload of any of these three types of data will also create and update the data controller, SQL managed instance, and PostgreSQL Hyperscale server group resources in Azure.

Note

During the preview period, there is no cost for using Azure Arc enabled data services.

Note

As a preview feature, the technology presented in this article is subject to Supplemental Terms of Use for Microsoft Azure Previews.

The latest updates are available in the release notes.

For the current update, deployment of Arc data controller in direct connectivity mode is only supported from Azure portal.

For additional limitations and more details, check the known issues.

Before you can upload usage data, metrics, or logs you need to:

Install tools

The required tools include:

  • Azure CLI (az)
  • Azure Data CLI (azdata)

See Install tools.

Register the resource provider

Prior to uploading metrics or user data to Azure, you need to ensure that your Azure subscription has the Microsoft.AzureArcData resource provider registered.

To verify the resource provider, run the following command:

az provider show -n Microsoft.AzureArcData -o table

If the resource provider is not currently registered in your subscription, you can register it. To register it, run the following command. This command may take a minute or two to complete.

az provider register -n Microsoft.AzureArcData --wait

Create service principal

The service principal is used to upload usage and metrics data.

Follow these commands to create your metrics upload service principal:

Note

Creating a service principal requires certain permissions in Azure.

To create a service principal, update the following example. Replace <ServicePrincipalName>, SubscriptionId and resourcegroup with your values and run the command:

az ad sp create-for-rbac --name <ServicePrincipalName> --role Contributor --scopes /subscriptions/{SubscriptionId}/resourceGroups/{resourcegroup}

If you created the service principal earlier, and just need to get the current credentials, run the following command to reset the credential.

az ad sp credential reset --name <ServicePrincipalName>

For example, to create a service principal named azure-arc-metrics, run the following command

az ad sp create-for-rbac --name azure-arc-metrics --role Contributor --scopes /subscriptions/a345c178a-845a-6a5g-56a9-ff1b456123z2/resourceGroups/myresourcegroup

Example output:

"appId": "2e72adbf-de57-4c25-b90d-2f73f126e123",
"displayName": "azure-arc-metrics",
"name": "http://azure-arc-metrics",
"password": "5039d676-23f9-416c-9534-3bd6afc78123",
"tenant": "72f988bf-85f1-41af-91ab-2d7cd01ad1234"

Save the appId, password, and tenant values in an environment variable for use later.

SET SPN_CLIENT_ID=<appId>
SET SPN_CLIENT_SECRET=<password>
SET SPN_TENANT_ID=<tenant>
export SPN_CLIENT_ID='<appId>'
export SPN_CLIENT_SECRET='<password>'
export SPN_TENANT_ID='<tenant>'
$Env:SPN_CLIENT_ID="<appId>"
$Env:SPN_CLIENT_SECRET="<password>"
$Env:SPN_TENANT_ID="<tenant>"

After you have created the service principal, assign the service principal to the appropriate role.

Assign roles to the service principal

Run this command to assign the service principal to the Monitoring Metrics Publisher role on the subscription where your database instance resources are located:

Note

You need to use double quotes for role names when running from a Windows environment.

az role assignment create --assignee <appId> --role "Monitoring Metrics Publisher" --scope subscriptions/{SubscriptionID}/resourceGroups/{resourcegroup}

az role assignment create --assignee <appId> --role 'Monitoring Metrics Publisher' --scope subscriptions/{SubscriptionID}/resourceGroups/{resourcegroup}
az role assignment create --assignee <appId> --role 'Monitoring Metrics Publisher' --scope subscriptions/{SubscriptionID}/resourceGroups/{resourcegroup}

Example output:

{
  "canDelegate": null,
  "id": "/subscriptions/<Subscription ID>/providers/Microsoft.Authorization/roleAssignments/f82b7dc6-17bd-4e78-93a1-3fb733b912d",
  "name": "f82b7dc6-17bd-4e78-93a1-3fb733b9d123",
  "principalId": "5901025f-0353-4e33-aeb1-d814dbc5d123",
  "principalType": "ServicePrincipal",
  "roleDefinitionId": "/subscriptions/<Subscription ID>/providers/Microsoft.Authorization/roleDefinitions/3913510d-42f4-4e42-8a64-420c39005123",
  "scope": "/subscriptions/<Subscription ID>",
  "type": "Microsoft.Authorization/roleAssignments"
}

With the service principal assigned to the appropriate role, you can proceed to upload metrics, or user data.

Upload logs, metrics, or user data

The specific steps for uploading logs, metrics, or user data vary depending about the type of information you are uploading.

Upload logs to Azure Monitor

Upload metrics to Azure Monitor

Upload usage data to Azure Monitor

General guidance on exporting and uploading usage, metrics

Create, read, update, and delete (CRUD) operations on Azure Arc enabled data services are logged for billing and monitoring purposes. There are background services that monitor for these CRUD operations and calculate the consumption appropriately. The actual calculation of usage or consumption happens on a scheduled basis and is done in the background.

During preview, this process happens nightly. The general guidance is to upload the usage only once per day. When usage information is exported and uploaded multiple times within the same 24 hour period, only the resource inventory is updated in Azure portal but not the resource usage.

For uploading metrics, Azure monitor only accepts the last 30 minutes of data (Learn more). The guidance for uploading metrics is to upload the metrics immediately after creating the export file so you can view the entire data set in Azure portal. For instance, if you exported the metrics at 2:00 PM and ran the upload command at 2:50 PM. Since Azure Monitor only accepts data for the last 30 minutes, you may not see any data in the portal.

Next steps

Learn about service principals

Upload billing data to Azure and view it in the Azure portal

View Azure Arc data controller resource in Azure portal