question

AnuragShelar-8594 avatar image
0 Votes"
AnuragShelar-8594 asked SandervandeVelde42 answered

How to handle data coming on IOT Hub which has multiple JSON formats?

Hello,
I am working on a use case wherein I am sending data from az3166 iot device to IOT hub. Further I am running a stream analytics query to store data onto blob storage.
The json for the same is
{"messageId":1,"temperature":28.399999618530273,"humidity":45.799999237060547,"EventProcessedUtcTime":"2021-04-27T06:30:10.6759169Z","PartitionId":3,"EventEnqueuedUtcTime":"2021-04-27T06:28:55.0330000Z","IoTHub":{"MessageId":null,"CorrelationId":null,"ConnectionDeviceId":"az3166","ConnectionDeviceGenerationId":"637540017300317851","EnqueuedTime":"2021-04-27T06:28:55.0290000Z"}}

Also , I am Simulating OPC UA server and OPC publisher on IOT Edge and it is sending data to IOT hub in following format.
{"NodeId":"http://www.prosysopc.com/OPCUA/SimulationNodes/#i=1004","ApplicationUri":"urn:host.docker.internal:OPCUA:SimulationServer","DisplayName":"Temperature","Value":{"Value":1.902113,"SourceTimestamp":"2021-04-28T10:47:06Z"},"EventProcessedUtcTime":"2021-04-28T10:48:27.0652993Z","PartitionId":1,"EventEnqueuedUtcTime":"2021-04-28T10:47:15.5030000Z","IoTHub":{"MessageId":null,"CorrelationId":null,"ConnectionDeviceId":"opauavm","ConnectionDeviceGenerationId":"637551916393812820","EnqueuedTime":"2021-04-28T10:47:15.5110000Z"}}

Now if we look at both the json file it is structured differently. I want to write a single Stream Analytics Job that will help me store both the json in blob storage.

PS: I am looking to store Temperature and Pressure parameters from both az3166 and OPC Publisher module.

Hi @SandervandeVelde42 . Needed help in this.

azure-iot-hubazure-iot-edgeazure-stream-analytics
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

SandervandeVelde42 avatar image
0 Votes"
SandervandeVelde42 answered

Hello @AnuragShelar-8594,

So multiple types of messages arrive at the IoT Hub and you want to split messages using routing.

There are two main solutions:

  1. Routing on the IoT Hub

  2. Routing on Azure Stream analytics

Regarding Routing on the IoT Hub, I prefer to do this in a non-functional way. You can add message routes using the content of desired properties but I like to keep it simple and only split on tags, desired properties or the application properties send with the message as context of why the message is sent.

Azure Stream Analytics is great for splitting, and joining, messages based on the content.

In ASA, a split between multiple messages can be as simple as testing for a JSON element only existing in one of the messages.

Check out the ASA query language and IoT Hub routing.


5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.