Build a real-time dashboard in Microsoft Fabric using MQTT data from Azure IoT MQ Preview

Article
01/05/2024

Important

Azure IoT Operations Preview – enabled by Azure Arc is currently in PREVIEW. You shouldn't use this preview software in production environments.

See the Supplemental Terms of Use for Microsoft Azure Previews for legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.

In this walkthrough, you build a real-time Power BI dashboard in Microsoft Fabric using simulated MQTT data that's published to Azure IoT MQ Preview. The architecture uses the IoT MQ's Kafka connector to deliver messages to an Event Hubs namespace. Messages are then streamed to a Kusto database in Microsoft Fabric using an eventstream and visualized in a Power BI dashboard.

Azure IoT Operation Preview - enabled by Azure Arc can be deployed with the Azure CLI, Azure portal or with infrastructure-as-code (IaC) tools. This tutorial uses the IaC method using the Bicep language.

Prepare your Kubernetes cluster

This walkthrough uses a virtual Kubernetes environment hosted in a GitHub Codespace to help you get going quickly. If you want to use a different environment, all the artifacts are available in the explore-iot-operations GitHub repository so you can easily follow along.

Create the Codespace, optionally entering your Azure details to store them as environment variables for the terminal.
Once the Codespace is ready, select the menu button at the top left, then select Open in VS Code Desktop.
Connect the cluster to Azure Arc.

Deploy edge and cloud Azure resources

The MQTT broker and north-bound cloud connector components can be deployed as regular Azure resources as they have Azure Resource Provider (RPs) implementations. A single Bicep template file from the explore-iot-operations repository deploys all the required edge and cloud resources and Azure role-based access assignments. Run this command in your Codespace terminal:

CLUSTER_NAME=<arc-connected-cluster-name>
TEMPLATE_FILE_NAME='tutorials/mq-realtime-fabric-dashboard/deployEdgeAndCloudResources.bicep'

 az deployment group create \ 
    --name az-resources \
    --resource-group $RESOURCE_GROUP \
    --template-file $TEMPLATE_FILE_NAME \
    --parameters clusterName=$CLUSTER_NAME

Important

The deployment configuration is for demonstration or development purposes only. It's not suitable for production environments.

The resources deployed by the template include:

Send test MQTT data and confirm cloud delivery

Simulate test data by deploying a Kubernetes workload. The pod simulates a sensor by sending sample temperature, vibration, and pressure readings periodically to the MQ broker using an MQTT client. Execute the following command in the Codespace terminal:
```
kubectl apply -f tutorials/mq-realtime-fabric-dashboard/simulate-data.yaml
```
The Kafka north-bound connector is preconfigured in the deployment to pick up messages from the MQTT topic where messages are being published to Event Hubs in the cloud.
After about a minute, confirm the message delivery in Event Hubs metrics.

Create and configure Microsoft Fabric event streams

Create a KQL Database.
Create an eventstream in Microsoft Fabric.
1. Add the Event Hubs namespace created in the previous section as a source.
2. Add the KQL Database created in the previous step as a destination.
In the wizard's Inspect step, add a New table called sensor_readings, enter a Data connection name and select Next.
In the Preview data tab, select the JSON format and select Finish.

In a few seconds, you should see the data being ingested into KQL Database.

Create Power BI report

From the KQL Database, right-click on the sensor-readings table and select Build Power BI report.
Drag the ∑ Temperature onto the canvas and change the visualization to a line graph. Drag the EventEnqueuedUtcTime column onto the visual and save the report.
Open the Power BI report to see the real-time dashboard, you can refresh the dashboard with latest sensor reading using the button on the top right.

In this walkthrough, you learned how to build a real-time dashboard in Microsoft Fabric using simulated MQTT data that is published to IoT MQ.

Next steps

Upload MQTT data to Microsoft Fabric lakehouse

Share via