Tutorial: Send device data to Azure Storage using IoT Hub message routing

Use message routing in Azure IoT Hub to send telemetry data from your IoT devices to Azure services such as blob storage, Service Bus Queues, Service Bus Topics, and Event Hubs.

Every IoT hub has a default built-in endpoint that is compatible with Event Hubs. You can also create custom endpoints and route messages to other Azure services by defining routing queries. Each message that arrives at the IoT hub is routed to all endpoints whose routing queries it matches. If a message doesn't match any of the defined routing queries, it is routed to the default endpoint.

In this tutorial, you perform the following tasks:

  • Create an IoT hub and send device messages to it.
  • Create a storage account.
  • Create a custom endpoint for the storage account and route messages to it from the IoT hub.
  • View device messages in the storage account blob.

Prerequisites

  • An Azure subscription. If you don't have an Azure subscription, create a free account before you begin.

  • An IoT hub in your Azure subscription. If you don't have a hub yet, you can follow the steps in Create an IoT hub.

  • This tutorial uses sample code from Azure IoT samples for C#.

    • Download or clone the samples repo to your development machine.
    • Have .NET Core 3.0.0 or greater on your development machine. Check your version by running dotnet --version and Download .NET if necessary.
  • Make sure that port 8883 is open in your firewall. The sample in this tutorial uses MQTT protocol, which communicates over port 8883. This port may be blocked in some corporate and educational network environments. For more information and ways to work around this issue, see Connecting to IoT Hub (MQTT).

  • Optionally, install Azure IoT Explorer. This tool helps you observe the messages as they arrive at your IoT hub.

There are no other prerequisites for the Azure portal.

Register a device and send messages to IoT Hub

Register a new device in your IoT hub.

  1. Sign in to the Azure portal and navigate to your IoT hub.

  2. Select Devices from the Device management section of the menu.

  3. Select Add device.

    Add a new device in the Azure portal.

  4. Provide a device ID and select Save.

  5. The new device should be in the list of devices now. If it's not, refresh the page. Select the device ID to open the device details page.

  6. Copy one of the device keys and save it. You'll use this value to configure the sample code that generates simulated device telemetry messages.

    Copy the primary key from the device details page.

Now that you have a device ID and key, use the sample code to start sending device telemetry messages to IoT Hub.

Tip

If you're following the Azure CLI steps for this tutorial, run the sample code in a separate session. That way, you can allow the sample code to continue running while you follow the rest of the CLI steps.

  1. If you didn't as part of the prerequisites, download or clone the Azure IoT samples for C# repo from GitHub now.

  2. In the sample folder, navigate to the /iot-hub/Tutorials/Routing/SimulatedDevice/ folder.

  3. In an editor of your choice, open the Program.cs file.

  4. Find the variable definitions at the top of the Program class. Update the following variables with your own information:

    • s_myDeviceId: The device ID that you assigned when registering the device.
    • s_iotHubUri: The hostname of your IoT hub, which takes the format IOTHUB_NAME.azure-devices.net.
    • s_deviceKey: The device key that you copied from the device identity information.
  5. Save and close the file.

  6. Install the Azure IoT C# SDK and necessary dependencies as specified in the SimulatedDevice.csproj file:

    dotnet restore
    
  7. Run the sample code:

    dotnet run
    
  8. You should start to see messages printed to output as they are sent to IoT Hub. Leave this program running for the duration of the tutorial.

Configure IoT Explorer to view messages

Configure IoT Explorer to connect to your IoT hub and read messages as they arrive at the built-in endpoint.

First, retrieve the connection string for your IoT hub.

  1. In the Azure portal, navigate to your IoT hub.

  2. Select Shared access policies from the Security settings section of the menu.

  3. Select the iothubowner policy.

    Open the iothubowner shared access policy.

  4. Copy the Primary connection string.

    Copy the iothubowner primary connection string.

Now, use that connection string to configure IoT Explorer for your IoT hub.

  1. Open IoT Explorer on your development machine.

  2. Select Add connection.

    Add IoT hub connection in IoT Explorer.

  3. Paste your hub's connection string into the text box.

  4. Select Save.

  5. Once you connect to your IoT hub, you should see a list of devices. Select the device ID that you created for this tutorial.

  6. Select Telemetry.

  7. Select Start.

    Start monitoring device telemetry in IoT Explorer.

  8. You should see the messages arriving from your device, with the most recent displayed at the top.

    View messages arriving at IoT hub on the built-in endpoint.

Watch the incoming messages for a few moments to verify that you see three different types of messages: normal, storage, and critical.

These messages are all arriving at the default built-in endpoint for your IoT hub. In the next sections, we're going to create a custom endpoint and route some of these messages to storage based on the message properties. Those messages will stop appearing in IoT Explorer because messages only go to the built-in endpoint when they don't match any other routes in IoT hub.

Set up message routing

You're going to route messages to different resources based on properties attached to the message by the simulated device. Messages that aren't custom routed are sent to the default endpoint (messages/events).

The sample app for this tutorial assigns a level property to each message it sends to IoT hub. Each message is randomly assigned a level of normal, storage, or critical.

The first step is to set up the endpoint to which the data will be routed. The second step is to set up the message route that uses that endpoint. After setting up the routing, you can view endpoints and message routes in the portal.

Create a storage account

Create an Azure Storage account and a container within that account, which will hold the device messages that are routed to it.

  1. In the Azure portal, search for Storage accounts.

  2. Select Create.

  3. Provide the following values for your storage account:

    Parameter Value
    Subscription Select the same subscription that contains your IoT hub.
    Resource group Select the same resource group that contains your IoT hub.
    Storage account name Provide a globally unique name for your storage account.
    Performance Accept the default Standard value.

    Create a storage account.

  4. You can accept all the other default values by selecting Review + create.

  5. After validation completes, select Create.

  6. After the deployment is complete, select Go to resource.

  7. In the storage account menu, select Containers from the Data storage section.

  8. Select Container to create a new container.

    Create a storage container

  9. Provide a name for your container and select Create.

Route to a storage account

Now set up the routing for the storage account. In this section you define a new endpoint that points to the storage account you created. Then, create a route that filters for messages where the level property is set to storage, and route those to the storage endpoint.

Note

The data can be written to blob storage in either the Apache Avro format, which is the default, or JSON.

The encoding format can be only set at the time the blob storage endpoint is configured. The format cannot be changed for an endpoint that has already been set up. When using JSON encoding, you must set the contentType to JSON and the contentEncoding to UTF-8 in the message system properties.

For more detailed information about using a blob storage endpoint, please see guidance on routing to storage.

  1. In the Azure portal, navigate to your IoT hub.

  2. Select Message Routing from the Hub settings section of the menu.

  3. In the Routes tab, select Add.

    Add a new message route.

  4. Select Add endpoint next to the Endpoint field, then select Storage from the dropdown menu.

    Add a new endpoint for a route.

  5. Provide the following information for the new storage endpoint:

    Parameter Value
    Endpoint name Create a name for this endpoint.
    Azure Storage container Select Pick a container, which takes you to a list of storage accounts. Choose the storage account that you created in the previous section, then choose the container that you created in that account. Select Select.
    Encoding Select JSON. If this field is greyed out, then your storage account region doesn't support JSON. In that case, continue with the default AVRO.

    Pick a container.

  6. Accept the default values for the rest of the parameters and select Create.

  7. Continue creating the new route, now that you've added the storage endpoint. Provide the following information for the new route:

    Parameter Value
    Name Create a name for your route.
    Data source Verify that Device Telemetry Messages is selected from the dropdown list.
    Enable route Verify that this field is set to enabled.
    Routing query Enter level="storage" as the query string.

    Save the routing query information

  8. Select Save.

View routed messages

Once the route is created in IoT Hub and enabled, it will immediately start routing messages that meet its query condition to the storage endpoint.

Monitor the built-in endpoint with IoT Explorer

Return to the IoT Explorer session on your development machine. Recall that the IoT Explorer monitors the built-in endpoint for your IoT hub. That means that now you should be seeing only the messages that are not being routed by the custom route we created. Watch the incoming messages for a few moments and you should only see messages where level is set to normal or critical.

View messages in the storage container

Verify that the messages are arriving in the storage container.

  1. In the Azure portal, navigate to your storage account.

  2. Select Containers from the Data storage section of the menu.

  3. Select the container that you created for this tutorial.

  4. There should be a folder with the name of your IoT hub. Drill down through the file structure until you get to a .json file.

    Find routed messages in storage.

  5. Download the JSON file and confirm that it contains messages from your device that have the level property set to storage.

Clean up resources

If you want to remove all of the Azure resources you used for this tutorial, delete the resource group. This action deletes all resources contained within the group. If you don't want to delete the entire resource group, use the Azure portal to locate and delete the individual resources.

Tip

If you intend to complete Tutorial: Use Azure IoT Hub message enrichments, be sure to maintain the resources you created here.

  1. In the Azure portal, navigate to the resource group that contains the IoT hub and storage account for this tutorial.
  2. Review all the resources that are in the resource group to determine which ones you want to clean up.
    • If you want to delete all the resource, select Delete resource group.
    • If you only want to delete certain resource, use the check boxes next to each resource name to select the ones you want to delete. Then select Delete.

Next steps

In this tutorial you learned how to create a custom endpoint for an Azure resource and then create a route to send device messages to that endpoint. Continue to the next tutorial to learn how to enrich messages with extra data that can be used to simplify downstream processing