Tutorial: Use the Azure CLI to configure IoT Hub message routing

Message routing enables sending telemetry data from your IoT devices to built-in Event Hub-compatible endpoints or custom endpoints such as blob storage, Service Bus Queues, Service Bus Topics, and Event Hubs. To configure custom message routing, you create routing queries to customize the route that matches a certain condition. Once set up, the incoming data is automatically routed to the endpoints by the IoT Hub. If a message doesn't match any of the defined routing queries, it is routed to the default endpoint.

In this 2-part tutorial, you learn how to set up and use these custom routing queries with IoT Hub. You route messages from an IoT device to one of multiple endpoints, including blob storage and a Service Bus queue. Messages to the Service Bus queue are picked up by a Logic App and sent via e-mail. Messages that do not have custom message routing defined are sent to the default endpoint, then picked up by Azure Stream Analytics and viewed in a Power BI visualization.

To complete Parts 1 and 2 of this tutorial, you perform the following tasks:

Part I: Create resources, set up message routing

  • Create the resources -- an IoT hub, a storage account, a Service Bus queue, and a simulated device. This can be done using the Azure portal, an Azure Resource Manager template, the Azure CLI, or Azure PowerShell.
  • Configure the endpoints and message routes in IoT Hub for the storage account and Service Bus queue.

Part II: Send messages to the hub, view routed results

  • Create a Logic App that is triggered and sends e-mail when a message is added to the Service Bus queue.
  • Download and run an app that simulates an IoT Device sending messages to the hub for the different routing options.
  • Create a Power BI visualization for data sent to the default endpoint.
  • View the results ...
  • ...in the Service Bus queue and e-mails.
  • ...in the storage account.
  • ...in the Power BI visualization.

Prerequisites

  • For Part 1 of this tutorial:

    • You must have an Azure subscription. If you don't have an Azure subscription, create a free account before you begin.
  • For Part 2 of this tutorial:

    • You must have completed Part 1 of this tutorial, and have the resources still available.
    • Install Visual Studio.
    • Have access to a Power BI account to analyze the default endpoint's stream analytics. (Try Power BI for free.)
    • Have a work or school account for sending notification e-mails.
    • Make sure that port 8883 is open in your firewall. The sample in this tutorial uses MQTT protocol, which communicates over port 8883. This port may be blocked in some corporate and educational network environments. For more information and ways to work around this issue, see Connecting to IoT Hub (MQTT).

Use Azure Cloud Shell

Azure hosts Azure Cloud Shell, an interactive shell environment that you can use through your browser. You can use either Bash or PowerShell with Cloud Shell to work with Azure services. You can use the Cloud Shell preinstalled commands to run the code in this article without having to install anything on your local environment.

To start Azure Cloud Shell:

Option Example/Link
Select Try It in the upper-right corner of a code block. Selecting Try It doesn't automatically copy the code to Cloud Shell. Example of Try It for Azure Cloud Shell
Go to https://shell.azure.com, or select the Launch Cloud Shell button to open Cloud Shell in your browser. Launch Cloud Shell in a new window
Select the Cloud Shell button on the menu bar at the upper right in the Azure portal. Cloud Shell button in the Azure portal

To run the code in this article in Azure Cloud Shell:

  1. Start Cloud Shell.

  2. Select the Copy button on a code block to copy the code.

  3. Paste the code into the Cloud Shell session by selecting Ctrl+Shift+V on Windows and Linux or by selecting Cmd+Shift+V on macOS.

  4. Select Enter to run the code.

Create base resources

Before you can configure the message routing, you need to create an IoT hub, a storage account, and a Service Bus queue. These resources can be created using one of the four articles that are available for Part 1 of this tutorial: the Azure portal, an Azure Resource Manager template, the Azure CLI, or Azure PowerShell.

Use the same resource group and location for all of the resources. Then at the end, you can remove all of the resources in one step by deleting the resource group.

Below is a summary of the steps to be performed in the following sections:

  1. Create a resource group.

  2. Create an IoT hub in the S1 tier. Add a consumer group to your IoT hub. The consumer group is used by the Azure Stream Analytics when retrieving data.

    Note

    You must use an Iot hub in a paid tier to complete this tutorial. The free tier only allows you to set up one endpoint, and this tutorial requires multiple endpoints.

  3. Create a standard V1 storage account with Standard_LRS replication.

  4. Create a Service Bus namespace and queue.

  5. Create a device identity for the simulated device that sends messages to your hub. Save the key for the testing phase. (If creating a Resource Manager template, this is done after deploying the template.)

Download the script (optional)

For the second part of this tutorial, you download and run a Visual Studio application to send messages to the IoT Hub. There is a folder in the download that contains the Azure Resource Manager template and parameters file, as well as the Azure CLI and PowerShell scripts.

If you want to view the finished script, download the Azure IoT C# Samples. Unzip the master.zip file. The Azure CLI script is in /iot-hub/Tutorials/Routing/SimulatedDevice/resources/ as iothub_routing_cli.azcli.

Use the Azure CLI to create your resources

Copy and paste the script below into Cloud Shell and press Enter. It runs the script one line at a time. This first section of the script will create the base resources for this tutorial, including the storage account, IoT Hub, Service Bus Namespace, and Service Bus queue. As you go through the rest of the tutorial, copy each block of script and paste it into Cloud Shell to run it.

Tip

A tip about debugging: this script uses the continuation symbol (the backslash \) to make the script more readable. If you have a problem running the script, make sure your Cloud Shell session is running bash and that there are no spaces after any of the backslashes.

There are several resource names that must be globally unique, such as the IoT Hub name and the storage account name. To make this easier, those resource names are appended with a random alphanumeric value called randomValue. The randomValue is generated once at the top of the script and appended to the resource names as needed throughout the script. If you don't want it to be random, you can set it to an empty string or to a specific value.

Important

The variables set in the initial script are also used by the routing script, so run all of the script in the same Cloud Shell session. If you open a new session to run the script for setting up the routing, several of the variables will be missing values.

# This command retrieves the subscription id of the current Azure account. 
# This field is used when setting up the routing queries.
subscriptionID=$(az account show --query id -o tsv)

# Concatenate this number onto the resources that have to be globally unique.
# You can set this to "" or to a specific value if you don't want it to be random.
# This retrieves a random value.
randomValue=$RANDOM

# This command installs the IOT Extension for Azure CLI.
# You only need to install this the first time.
# You need it to create the device identity. 
az extension add --name azure-iot

# Set the values for the resource names that 
#   don't have to be globally unique.
location=westus
resourceGroup=ContosoResources
iotHubConsumerGroup=ContosoConsumers
containerName=contosoresults
iotDeviceName=Contoso-Test-Device

# Create the resource group to be used
#   for all the resources for this tutorial.
az group create --name $resourceGroup \
    --location $location

# The IoT hub name must be globally unique, 
#   so add a random value to the end.
iotHubName=ContosoTestHub$randomValue 
echo "IoT hub name = " $iotHubName

# Create the IoT hub.
az iot hub create --name $iotHubName \
    --resource-group $resourceGroup \
    --sku S1 --location $location

# Add a consumer group to the IoT hub for the 'events' endpoint.
az iot hub consumer-group create --hub-name $iotHubName \
    --name $iotHubConsumerGroup

# The storage account name must be globally unique, 
#   so add a random value to the end.
storageAccountName=contosostorage$randomValue
echo "Storage account name = " $storageAccountName

# Create the storage account to be used as a routing destination.
az storage account create --name $storageAccountName \
    --resource-group $resourceGroup \
    --location $location \
    --sku Standard_LRS

# Get the primary storage account key. 
#    You need this to create the container.
storageAccountKey=$(az storage account keys list \
    --resource-group $resourceGroup \
    --account-name $storageAccountName \
    --query "[0].value" | tr -d '"') 

# See the value of the storage account key.
echo "storage account key = " $storageAccountKey

# Create the container in the storage account. 
az storage container create --name $containerName \
    --account-name $storageAccountName \
    --account-key $storageAccountKey \
    --public-access off

# The Service Bus namespace must be globally unique, 
#   so add a random value to the end.
sbNamespace=ContosoSBNamespace$randomValue
echo "Service Bus namespace = " $sbNamespace

# Create the Service Bus namespace.
az servicebus namespace create --resource-group $resourceGroup \
    --name $sbNamespace \
    --location $location

# The Service Bus queue name must be globally unique, 
#   so add a random value to the end.
sbQueueName=ContosoSBQueue$randomValue
echo "Service Bus queue name = " $sbQueueName

# Create the Service Bus queue to be used as a routing destination.
az servicebus queue create --name $sbQueueName \
    --namespace-name $sbNamespace \
    --resource-group $resourceGroup

# Create the IoT device identity to be used for testing.
az iot hub device-identity create --device-id $iotDeviceName \
    --hub-name $iotHubName

# Retrieve the information about the device identity, then copy the primary key to
#   Notepad. You need this to run the device simulation during the testing phase.
az iot hub device-identity show --device-id $iotDeviceName \
    --hub-name $iotHubName

Now that the base resources are set up, you can configure the message routing.

Set up message routing

You are going to route messages to different resources based on properties attached to the message by the simulated device. Messages that are not custom routed are sent to the default endpoint (messages/events). In the next tutorial, you send messages to the IoT Hub and see them routed to the different destinations.

Value Result
level="storage" Write to Azure Storage.
level="critical" Write to a Service Bus queue. A Logic App retrieves the message from the queue and uses Office 365 to e-mail the message.
default Display this data using Power BI.

The first step is to set up the endpoint to which the data will be routed. The second step is to set up the message route that uses that endpoint. After setting up the routing, you can view the endpoints and message routes in the portal.

To create a routing endpoint, use az iot hub routing-endpoint create. To create the message route for the endpoint, use az iot hub route create.

Route to a storage account

Note

The data can be written to blob storage in either the Apache Avro format, which is the default, or JSON (preview).

The capability to encode JSON format is in preview in all regions in which IoT Hub is available, except East US, West US and West Europe. The encoding format can be only set at the time the blob storage endpoint is configured. The format cannot be changed for an endpoint that has already been set up. When using JSON encoding, you must set the contentType to JSON and the contentEncoding to UTF-8 in the message system properties.

For more detailed information about using a blob storage endpoint, please see guidance on routing to storage.

First, set up the endpoint for the storage account, then set up the route.

These are the variables used by the script that must be set within your Cloud Shell session:

storageConnectionString: This value is retrieved from the storage account set up in the previous script. It is used by the message routing to access the storage account.

resourceGroup: There are two occurrences of resource group -- set them to your resource group.

endpoint subscriptionID: This field is set to the Azure subscriptionID for the endpoint.

endpointType: This field is the type of endpoint. This value must be set to azurestoragecontainer, eventhub, servicebusqueue, or servicebustopic. For your purposes here, set it to azurestoragecontainer.

iotHubName: This field is the name of the hub that will do the routing.

containerName: This field is the name of the container in the storage account to which data will be written.

encoding: This field will be either avro or json. This denotes the format of the stored data.

routeName: This field is the name of the route you are setting up.

endpointName: This field is the name identifying the endpoint.

enabled: This field defaults to true, indicating that the message route should be enabled after being created.

condition: This field is the query used to filter for the messages sent to this endpoint. The query condition for the messages being routed to storage is level="storage".

Copy this script and paste it into your Cloud Shell window and run it.

##### ROUTING FOR STORAGE ##### 

endpointName="ContosoStorageEndpoint"
endpointType="azurestoragecontainer"
routeName="ContosoStorageRoute"
condition='level="storage"'

# Get the connection string for the storage account.
# Adding the "-o tsv" makes it be returned without the default double quotes around it.
storageConnectionString=$(az storage account show-connection-string \
  --name $storageAccountName --query connectionString -o tsv)

The next step is to create the routing endpoint for the storage account. You also specify the container in which the results will be stored. The container was created previously when the storage account was created.

# Create the routing endpoint for storage.
az iot hub routing-endpoint create \
  --connection-string $storageConnectionString \
  --endpoint-name $endpointName \
  --endpoint-resource-group $resourceGroup \
  --endpoint-subscription-id $subscriptionID \
  --endpoint-type $endpointType \
  --hub-name $iotHubName \
  --container $containerName \
  --resource-group $resourceGroup \
  --encoding avro

Next, create the route for the storage endpoint. The message route designates where to send the messages that meet the query specification.

# Create the route for the storage endpoint.
az iot hub route create \
  --name $routeName \
  --hub-name $iotHubName \
  --source devicemessages \
  --resource-group $resourceGroup \
  --endpoint-name $endpointName \
  --enabled \
  --condition $condition

Route to a Service Bus queue

Now set up the routing for the Service Bus queue. To retrieve the connection string for the Service Bus queue, you must create an authorization rule that has the correct rights defined. The following script creates an authorization rule for the Service Bus queue called sbauthrule, and sets the rights to Listen Manage Send. Once this authorization rule is defined, you can use it to retrieve the connection string for the queue.

# Create the authorization rule for the Service Bus queue.
az servicebus queue authorization-rule create \
  --name "sbauthrule" \
  --namespace-name $sbNamespace \
  --queue-name $sbQueueName \
  --resource-group $resourceGroup \
  --rights Listen Manage Send \
  --subscription $subscriptionID

Now use the authorization rule to retrieve the connection string to the Service Bus queue.

# Get the Service Bus queue connection string.
# The "-o tsv" ensures it is returned without the default double-quotes.
sbqConnectionString=$(az servicebus queue authorization-rule keys list \
  --name "sbauthrule" \
  --namespace-name $sbNamespace \
  --queue-name $sbQueueName \
  --resource-group $resourceGroup \
  --subscription $subscriptionID \
  --query primaryConnectionString -o tsv)

# Show the Service Bus queue connection string.
echo "service bus queue connection string = " $sbqConnectionString

Now set up the routing endpoint and the message route for the Service Bus queue. These are the variables used by the script that must be set within your Cloud Shell session:

endpointName: This field is the name identifying the endpoint.

endpointType: This field is the type of endpoint. This value must be set to azurestoragecontainer, eventhub, servicebusqueue, or servicebustopic. For your purposes here, set it to servicebusqueue.

routeName: This field is the name of the route you are setting up.

condition: This field is the query used to filter for the messages sent to this endpoint. The query condition for the messages being routed to the Service Bus queue is level="critical".

Here is the Azure CLI for the routing endpoint and the message route for the Service Bus queue.

endpointName="ContosoSBQueueEndpoint"
endpointType="ServiceBusQueue"
routeName="ContosoSBQueueRoute"
condition='level="critical"'

# Set up the routing endpoint for the Service Bus queue.
# This uses the Service Bus queue connection string.
az iot hub routing-endpoint create \
  --connection-string $sbqConnectionString \
  --endpoint-name $endpointName \
  --endpoint-resource-group $resourceGroup \
  --endpoint-subscription-id $subscriptionID \
  --endpoint-type $endpointType \
  --hub-name $iotHubName \
  --resource-group $resourceGroup 

# Set up the message route for the Service Bus queue endpoint.
az iot hub route create --name $routeName \
  --hub-name $iotHubName \
  --source-type devicemessages \
  --resource-group $resourceGroup \
  --endpoint-name $endpointName \
  --enabled \
  --condition $condition

View message routing in the portal

Now that your endpoints and message routes are set up, you can view their configuration in the portal. Sign in to the Azure portal and go to Resource Groups. Next, select your resource group, then select your hub (the hub name starts with ContosoTestHub in this tutorial). You see the IoT Hub pane.

IoT Hub properties screen

In the options for the IoT Hub, select Message Routing. The routes you have set up successfully are displayed.

The routes you set up

On the Message routing screen, select Custom Endpoints to see the endpoints you have defined for the routes.

The endpoints set up for the routes

Next steps

Now that you have the resources set up and the message routes configured, advance to the next tutorial to learn how to send messages to the IoT hub and see them be routed to the different destinations.