question

KumarParvendra-0260 avatar image
0 Votes"
KumarParvendra-0260 asked PRADEEPCHEEKATLA-MSFT commented

How to check all failed message data in Stream Analytics log monitoring? Currently I can see only 5 example events in logs but I want to see all.

I am using a Stream analytics job to receive message from event hub, parse it and then insert it to Synapse table. Sometimes, there is some data conversion issue due to incorrect input data. I want to log all these input message which have some data conversion issue. I configured log analytics workspace to achieve this. But it is only showing the max 5 example events for failure messages. How I can see the rest of input data having errors?

azure-monitorazure-stream-analytics
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

PRADEEPCHEEKATLA-MSFT avatar image
0 Votes"
PRADEEPCHEEKATLA-MSFT answered PRADEEPCHEEKATLA-MSFT commented

Hello @KumarParvendra-0260,

Thanks for the question and using MS Q&A platform.

Stream Analytics provides pre-defined queries that allows you to easily search for the logs that you are interested in. You can select any pre-defined queries on the left pane and then select Run. You will see the results of the query in the bottom pane.

129031-image.png

For more details, refer to Send diagnostics to Azure Monitor logs.

Hope this helps. Do let us know if you any further queries.


Please "Accept the answer" if the information helped you. This will help us and others in the community as well.


image.png (105.2 KiB)
· 4
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Thank you @PRADEEPCHEEKATLA-MSFT for your response.

Here my problem is different. My stream analytics jobs takes input from event hub and insert it to a database. Sometimes, Our SA jobs gives some error due to incorrect length of a field in input message. We want to see all these failed input message string somewhere. To achieve this we configured a log analytics here. But when we are running query for error records then it's just showing 5 example events not all.

For example, there are 32 input records those have some incorrect values for a past date. When I am running below query in log analytics, it's just shows me 5 example records as a property of "properties_s" field. How I can check the rest 27 records. Please see attached screenshots. Could you please suggest on this.

AzureDiagnostics
| where ResourceProvider == "MICROSOFT.STREAMANALYTICS" and parse_json(properties_s).Type == "DataError"
| project TimeGenerated, Resource, parse_json(properties_s).Message, properties_s

128950-1.png128969-2.png


0 Votes 0 ·
1.png (74.9 KiB)

Hello @KumarParvendra-0260,

This is strange, could you please verify the same from the Monitoring => metrics tab?

0 Votes 0 ·

Yes, same issue is there(Monitoring = >Logs)129643-2.png


0 Votes 0 ·
2.png (84.6 KiB)
Show more comments