question

yjay-4307 avatar image
0 Votes"
yjay-4307 asked yjay-4307 commented

Separate data in data explorer and use as datastore

Hello,

We are sending data from IoT Central to Event Hubs and then to Data Explorer, with the hopes of then sending the data to Azure Machine Learning.

In order to send data from Event Hubs to Data Explorer it needs a data ingestion into a table on data explorer.

For this data ingestion, it needs a json mapping.

We could ingest the data, but the message from the iot central data goes to event hubs that goes to data explorer carries the telemetry data as a dynamic type (a json inside a json).

 ("telemetry":{"Temp:"37","Vol":"97"})

90018-data.jpg
We want to separate the telemetry data in different columns.

So Temp will have one column and Vol another.

I am wondering how that can be done?

And additionally, since we would like to send the data to ML, can data explorer be used as a datastore in ML?

Thanks!!


azure-machine-learningazure-event-hubsazure-iot-centralazure-data-explorer
data.jpg (30.6 KiB)
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

PRADEEPCHEEKATLA-MSFT avatar image
1 Vote"
PRADEEPCHEEKATLA-MSFT answered yjay-4307 commented

Hello @yjay-4307,

You can use parse operator - Evaluates a string expression and parses its value into one or more calculated columns. The calculated columns will have nulls, for unsuccessfully parsed strings.

For more details, refer SO thread addressing similar issue.

Unfortuantely, Azure Data Explorer is not a supported storage solution with Azure Machine Learning.

Datastores currently support storing connection information to the storage services listed in the following matrix.

90159-image.png

For unsupported storage solutions, and to save data egress cost during ML experiments, move your data to a supported Azure storage solution.

Reference: Connect to storage services on Azure - Azure Machine Learning.

Hope this helps. Do let us know if you any further queries.


Please don’t forget to Accept Answer and Up-Vote wherever the information provided helps you, this can be beneficial to other community members.


image.png (55.9 KiB)
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Thanks so much, this is very helpful!

0 Votes 0 ·