Quickstart: Export data from an IoT Central application

This quickstart shows you how to continuously export data from your Azure IoT Central application to another cloud service. To get you set up quickly, this quickstart uses Azure Data Explorer, a fully managed data analytics service for real-time analysis. Azure Data Explorer lets you store, query, and process the telemetry from devices such as the IoT Plug and Play smartphone app.

In this quickstart, you:

• Use the data export feature in IoT Central to export the telemetry sent by the smartphone app to an Azure Data Explorer database.
• Use Azure Data Explorer to run queries on the telemetry.

Prerequisites

• Before you begin, you should complete the first quickstart Create an Azure IoT Central application. The second quickstart, Configure rules and actions for your device, is optional.
• You need the IoT Central application URL prefix that you chose in the first quickstart Create an Azure IoT Central application.

• Use the Bash environment in Azure Cloud Shell. For more information, see Azure Cloud Shell Quickstart - Bash.

  

• If you prefer to run CLI reference commands locally, install the Azure CLI. If you're running on Windows or macOS, consider running Azure CLI in a Docker container. For more information, see How to run the Azure CLI in a Docker container.

  • If you're using a local installation, sign in to the Azure CLI by using the az login command. To finish the authentication process, follow the steps displayed in your terminal. For other sign-in options, see Sign in with the Azure CLI.

  • When you're prompted, install the Azure CLI extension on first use. For more information about extensions, see Use extensions with the Azure CLI.

  • Run az version to find the version and dependent libraries that are installed. To upgrade to the latest version, run az upgrade.

Install Azure services

Before you can export data from your IoT Central application, you need an Azure Data Explorer cluster and database. In this quickstart, you use the bash environment in the Azure Cloud Shell to create and configure them.

Run the following script in the Azure Cloud Shell. Replace the clustername value with a unique name for your cluster before you run the script. The cluster name can contain only lowercase letters and numbers. Replace the centralurlprefix value with the URL prefix you chose in the first quickstart:

Important

The script can take 20 to 30 minutes to run.

# The cluster name can contain only lowercase letters and numbers.
# It must contain from 4 to 22 characters.
clustername="<A unique name for your cluster>"

centralurlprefix="<The URL prefix of your IoT Central application>"

databasename="phonedata"
location="eastus"
resourcegroup="IoTCentralExportData"

az extension add -n kusto

# Create a resource group for the Azure Data Explorer cluster
az group create --location $location \
    --name $resourcegroup

# Create the Azure Data Explorer cluster
# This command takes at least 10 minutes to run
az kusto cluster create --cluster-name $clustername \
    --sku name="Standard_D11_v2"  tier="Standard" \
    --enable-streaming-ingest=true \
    --enable-auto-stop=true \
    --resource-group $resourcegroup --location $location

# Create a database in the cluster
az kusto database create --cluster-name $clustername \
    --database-name $databasename \
    --read-write-database location=$location soft-delete-period=P365D hot-cache-period=P31D \
    --resource-group $resourcegroup

# Create and assign a managed identity to use
# when authenticating from IoT Central.
# This assumes your IoT Central was created in the default
# IOTC resource group.
MI_JSON=$(az iot central app identity assign --name $centralurlprefix \
    --resource-group IOTC --system-assigned)

## Assign the managed identity permissions to use the database.
az kusto database-principal-assignment create --cluster-name $clustername \
                                              --database-name $databasename \
                                              --principal-id $(jq -r .principalId <<< $MI_JSON) \
                                              --principal-assignment-name $centralurlprefix \
                                              --resource-group $resourcegroup \
                                              --principal-type App \
                                              --tenant-id $(jq -r .tenantId <<< $MI_JSON) \
                                              --role Admin

echo "Azure Data Explorer URL: $(az kusto cluster show --name $clustername --resource-group $resourcegroup --query uri -o tsv)" 

Make a note of the Azure Data Explorer URL. You use this value later in the quickstart.

Configure the database

To add a table in the database to store the accelerometer data from the IoT Plug and Play smartphone app:

1. Use the Azure Data Explorer URL from the previous section to navigate to your Azure Data Explorer environment.

2. Expand the cluster node and select the phonedata database. The cope of the query window changes to Scope:yourclustername.eastus/phonedata.

3. Paste the following Kusto script into the query editor and select Run:

   .create table acceleration (
     EnqueuedTime:datetime,
     Device:string,
     X:real,
     Y:real,
     Z:real
   );
   

   The result looks like the following screenshot:

   

4. In the Azure Data Explorer, open a new tab and paste in the following Kusto script. The script enables streaming ingress for the acceleration table:

   .alter table acceleration policy streamingingestion enable;
   

Keep the Azure Data Explorer page open in your browser. After you configure the data export in your IoT Central application, you'll run a query to view the accelerometer telemetry stored in the acceleration table.

Configure data export

To configure the data export destination from IoT Central:

1. Navigate to the Data export page in your IoT Central application.
2. Select the Destinations tab and then Add a destination.
3. Enter Azure Data Explorer as the destination name. Select Azure Data Explorer as the destination type.
4. In Cluster URL, enter the Azure Data Explorer URL you made a note of previously.
5. In Database name, enter phonedata.
6. In Table name, enter acceleration.
7. In Authorization, select System-assigned managed identity.
8. Select Save.

To configure the data export:

1. On the Data export page, select the Exports tab, and then Add an export.

2. Enter Phone accelerometer as the export name.

3. Select Telemetry as the type of data to export.

4. Use the information in the following table to add two filters:

   Name	Operator	Value
   Device template	Equals	IoT Plug and Play mobile
   Sensors/Acceleration/X	Exists	N/A

   Make sure that the option to export the data if all of the conditions are true is set.

5. Add Azure Data Explorer as a destination.

6. Add a data transformation to the destination. Add the following query in the 2. Build transformation query field on the Data transformation page:

   import "iotc" as iotc;
   {
       Device: .device.id,
       EnqueuedTime: .enqueuedTime,
       X: .telemetry | iotc::find(.name == "accelerometer").value.x,
       Y: .telemetry | iotc::find(.name == "accelerometer").value.y,
       Z: .telemetry | iotc::find(.name == "accelerometer").value.z
   }
   

   

   If you want to see how the transformation works and experiment with the query, paste the following sample telemetry message into 1. Add your input message:

   {
     "messageProperties": {},
     "device": {
       "id": "8hltz8xa7n",
       "properties": {
         "reported": []
       },
       "approved": true,
       "types": [],
       "name": "8hltz8xa7n",
       "simulated": false,
       "provisioned": true,
       "modules": [],
       "templateId": "urn:modelDefinition:vlcd3zvzdm:y425jkkpqzeu",
       "templateName": "IoT Plug and Play mobile",
       "organizations": [],
       "cloudProperties": [],
       "blocked": false
     },
     "component": "sensors",
     "applicationId": "40a97c91-50cc-44f0-9f63-71386613facc",
     "messageSource": "telemetry",
     "telemetry": [
       {
         "id": "dtmi:azureiot:PhoneSensors:__accelerometer;1",
         "name": "accelerometer",
         "value": {
           "x": 0.09960123896598816,
           "y": 0.09541380405426025,
           "z": 9.907781600952148
         }
       }
     ],
     "enqueuedTime": "2021-11-12T10:01:30.588Z",
     "enrichments": {}
   }
   

7. Save the transformation. Then save the data export definition.

Wait until the export status shows Healthy:

Query exported data

In Azure Data Explorer, open a new tab and paste in the following Kusto query and then select Run to plot the accelerometer telemetry:

['acceleration'] 
    | project EnqueuedTime, Device, X, Y, Z
    | render timechart 

You may need to wait for several minutes to collect enough data. Try holding your phone in different orientations to see the telemetry values change:

Clean up resources

If you don't plan to complete any further IoT Central quickstarts or tutorials, you can delete your IoT Central application:

1. In your IoT Central application, navigate to Application > Management.
2. Select Delete and then confirm your action.

To remove the Azure Data Explorer instance from your subscription and avoid being billed unnecessarily, delete the IoTCentralExportData resource group from the Azure portal or run the following command in the Azure Cloud Shell:

az group delete --name IoTCentralExportData

Next steps

In this quickstart, you learned how to continuously export data from IoT Central to another Azure service.

Now that you know now to export your data, the suggested next step is to:

Build and manage a device template.