question

CharlesElwood-6092 avatar image
1 Vote"
CharlesElwood-6092 asked CharlesElwood-6092 commented

Stream Analytics on the Edge with Azure Percept - Debugging route

I was able to follow these directions to get simulated temp sensor data through azure stream analytics on the edge.
https://docs.microsoft.com/en-us/azure/iot-edge/tutorial-deploy-stream-analytics?view=iotedge-2020-11

I am now trying to modify the code to put azure eye data through stream analytics to a cloud storage container.
Need some help debugging.

Steps to reproduce :
1) I created an output in the stream analytics editor
2) I modified the query to select from the input stream from the azure eye and then place into the cloud storage container.
SELECT

INTO
[name of cloud storage container]
FROM
[name of input - Edge Hub]
3) In the IOT Hub I added in Edge Devices, set module, routes the following route
FROM /messages/modules/azureeyemodule/outputs/* INTO BrokeredEndpoint("/modules/streamonedgeversion2/inputs/[name of input to ASA]")
4) In Percept Studio I then verify via view Telemetry that the Percept is at least sending data from AzureEye and the temp simulator up to cloud.
5) I opened Azure Storage Explorer and took a look at the Storage container and I don't see the new container being populated with new data from this query.

@mimcco @AmiraYoussef-8715

azure-iot-hubazure-iotazure-stream-analyticsazure-percept
· 5
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @CharlesElwood-6092 Community SME's on this topic or our team will review your scenario and circle back at the possible earliest time.

1 Vote 1 ·

Hello @CharlesElwood-6092 Did you tired the other way with simulated data or without the Azure Percept to ingest telemetry data? Such as simulated sensor data from an IoT-Edge or web simulator for quick testing?

Also, you will need to get the raw message from IoT Hub and check if the payload is in valid JSON format. This error is usually caused by two readers are reading data from the same consumer group, just-in-case better to check in this direction. Ref

Please also read this StackOverflow post: https://stackoverflow.com/a/37277134/13174899

1 Vote 1 ·

@SatishBoddu-MSFT @mimcco @AmiraYoussef-8715
Apologies for the delay. I didn't realize there were responses.
I'll check into this and get back to you tomorrow.

I l had tried the simulated temperature module before I used the actual data from the Percept but I don't think I kept records.
I'll do this again. Are there any particular logs from particular modules that I should be saving a copy of?

0 Votes 0 ·

@mimcco I was able to go into IOT Hub -> View modules -> click on this module and click on troubleshoot. Error message excerpt below from the log.

{\"Source\":\"temp\",\"Type\":\"DataError\",\"DataErrorType\":\"InputDeserializerError.MissingColumns\",\"BriefMessage\":\"Could not deserialize the input event(s) from resource 'azureeyemodule' as Json. Some possible reasons: 1) Malformed events 2) Input source configured with incorrect serialization format\",\"ErrorCode\":\"InputDeserializationError\",\"ErrorCategory\":\"DataError\",\"Message\":\"Error while deserializing input message Id: azureeyemodule. Hit following error: Missing fields specified in query or in create table.\nFields expected: TIMECREATED. \nFields found: NEURAL_NETWORK. \nMissing fields: TIMECREATED\",\"ExampleEvents\":\"[\\"{\\\\"NEURAL_NETWORK\\\\": [{\\\\"bbox\\\\": [0.161, 0.481, 0.550, 0.990],\\\\"label\\\\": \\\\"person\\\\", \\\\"confidence\\\\": \\\\"0.924805\\\\", \\\\"timestamp\\\\": \\\\"1627051174312572427\\\\"}, {\\\\"bbox\\\\": [0.157, 0.521, 0.321, 0.755],\\\\"label\\\\": \\\\"potted plant\\\\", \\\\"confidence\\\\": \\\\"0.806152\\\\", \\\\"timestamp\\\\": \\\\"1627051174312572427\\\\"}]}\\"]\",\"FromTimestamp\":\"2021-07-23T14:39:36.1965051Z\",\"ToTimestamp\":\"2021-07-23T14:39:36.2107964Z\",\"EventCount\":1}"]

0 Votes 0 ·

@mimcco @JussiNiemela-7425 @AmiraYoussef-8715 @HimanshuSinha-MSFT
I went and dug into this error a little bit

https://docs.microsoft.com/en-us/azure/stream-analytics/data-errors

The query is just to push * into the output.

117641-image.png

It looks like it can't find certain query parameters, but that doesn't make sense with the "*".

117440-image.png

I'm wondering now if my stream analytics modification to the query isn't being picked up and deployed?
If I modify the query, should the new query automatically get saved to the container storage and then will the Percept pick up the update automatically?
I'll try creating a completely new Stream Analytics module, delete the old one, and deploy the new one to test.
Now that I can find the error logs in the deployment section of the IOT Hub, that should help me debug this further.

0 Votes 0 ·
image.png (20.1 KiB)
image.png (209.3 KiB)

1 Answer

CharlesElwood-6092 avatar image
1 Vote"
CharlesElwood-6092 answered CharlesElwood-6092 commented

@SatishBoddu-MSFT @AmiraYoussef-8715

I was able to follow the following directions and now have ASA running and outputting simulated sensor data on the edge through stream analytics and routed up through IoT Hub and can view telemetry data in Percept Studio and also see the screen shot that ASA is processing the data as shown in attached screenshot.

Next step, see if I can route Percept Data through the same path.

https://docs.microsoft.com/en-us/azure/iot-edge/tutorial-deploy-stream-analytics?view=iotedge-2020-11

123595-image.png



image.png (82.7 KiB)
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Just tried to change everything over to Percept data and I am getting the same error : @SatishBoddu-MSFT @AmiraYousif-2257
123580-image.png


0 Votes 0 ·
image.png (121.7 KiB)