Edit

Upload usage data, metrics, and logs to Azure

  • Article

Periodically, you can export out usage information for billing purposes, monitoring metrics, and logs and then upload it to Azure. The export and upload of any of these three types of data will also create and update the data controller, and SQL managed instance resources in Azure.

Before you can upload usage data, metrics, or logs you need to:

Examples in this article use angle brackets < ... > to identify values that you need to replace before you run the script. Replace the brackets and the values inside the brackets.

Install tools

The required tools include:

  • Azure CLI (az)
  • arcdata extension

See Install tools.

Register the resource provider

Prior to uploading metrics or user data to Azure, you need to ensure that your Azure subscription has the Microsoft.AzureArcData resource provider registered.

To verify the resource provider, run the following command:

az provider show -n Microsoft.AzureArcData -o table

If the resource provider is not currently registered in your subscription, you can register it. To register it, run the following command. This command may take a minute or two to complete.

az provider register -n Microsoft.AzureArcData --wait

Create service principal

The service principal is used to upload usage and metrics data.

Follow these commands to create your metrics upload service principal:

Note

Creating a service principal requires certain permissions in Azure.

To create a service principal, update the following example. Replace <ServicePrincipalName>, SubscriptionId and resourcegroup with your values and run the command:

az ad sp create-for-rbac --name <ServicePrincipalName> --role Contributor --scopes /subscriptions/<SubscriptionId>/resourceGroups/<resourcegroup>

If you created the service principal earlier, and just need to get the current credentials, run the following command to reset the credential.

az ad sp credential reset --name <ServicePrincipalName>

For example, to create a service principal named azure-arc-metrics, run the following command

az ad sp create-for-rbac --name azure-arc-metrics --role Contributor --scopes /subscriptions/<SubscriptionId>/resourceGroups/myresourcegroup

Example output:

"appId": "<appId>",
"displayName": "azure-arc-metrics",
"name": "http://azure-arc-metrics",
"password": "<password>",
"tenant": "<tenant>"

Save the appId, password, and tenant values in an environment variable for use later. These values are in the form of globally unique identifier (GUID).

SET SPN_CLIENT_ID=<appId>
SET SPN_CLIENT_SECRET=<password>
SET SPN_TENANT_ID=<tenant>

After you have created the service principal, assign the service principal to the appropriate role.

Assign roles to the service principal

Run this command to assign the service principal to the Monitoring Metrics Publisher role on the subscription where your database instance resources are located:

Note

You need to use double quotes for role names when running from a Windows environment.

az role assignment create --assignee <appId> --role "Monitoring Metrics Publisher" --scope subscriptions/<SubscriptionID>/resourceGroups/<resourcegroup>

Example output:

{
  "canDelegate": null,
  "id": "/subscriptions/<Subscription ID>/providers/Microsoft.Authorization/roleAssignments/<globally unique identifier>",
  "name": "<globally unique identifier>",
  "principalId": "<principal id>",
  "principalType": "ServicePrincipal",
  "roleDefinitionId": "/subscriptions/<Subscription ID>/providers/Microsoft.Authorization/roleDefinitions/<globally unique identifier>",
  "scope": "/subscriptions/<Subscription ID>",
  "type": "Microsoft.Authorization/roleAssignments"
}

Verify service principal role

az role assignment list --scope subscriptions/<SubscriptionID>/resourceGroups/<resourcegroup> -o table

With the service principal assigned to the appropriate role, you can proceed to upload metrics, or user data.

Upload logs, metrics, or usage data

The specific steps for uploading logs, metrics, or usage data vary depending about the type of information you are uploading.

Upload logs to Azure Monitor

Upload metrics to Azure Monitor

Upload usage data to Azure

General guidance on exporting and uploading usage, and metrics

Create, read, update, and delete (CRUD) operations on Azure Arc-enabled data services are logged for billing and monitoring purposes. There are background services that monitor for these CRUD operations and calculate the consumption appropriately. The actual calculation of usage or consumption happens on a scheduled basis and is done in the background.

Upload the usage only once per day. When usage information is exported and uploaded multiple times within the same 24 hour period, only the resource inventory is updated in Azure portal but not the resource usage.

Note

Note that usage data is automatically uploaded for Azure Arc data controller deployed in direct connected mode.

For uploading metrics, Azure monitor only accepts the last 30 minutes of data (Learn more). The guidance for uploading metrics is to upload the metrics immediately after creating the export file so you can view the entire data set in Azure portal. For instance, if you exported the metrics at 2:00 PM and ran the upload command at 2:50 PM. Since Azure Monitor only accepts data for the last 30 minutes, you may not see any data in the portal.

Related content

Learn about service principals

Upload billing data to Azure and view it in the Azure portal

View Azure Arc data controller resource in Azure portal