Filter blob storage data and send to Machine Learning

Question

Hi,

As a kind of continuation of my previous question: Send data from IoT central to Azure Machine Learning Resource
I am wondering how I can filter my data to send to Machine Learning. I have multiple devices that are exporting data from IoT Central to Azure Blob Storage, I am wondering how I can filter the data for each device so I can send to a Machine Learning model?

Additionally, do I need to create multiple ML models for each device or can I send all filtered data to the same model? (For ex. If I have 5 devices collecting temperature data and I want to predict if the device user has a cold, I want to use the same model that predicts colds but I want to predict separately for each user based on their data)

Thanks so much!

Answer 1

@yjay

I am wondering how I can filter my data to send to Machine Learning. I have multiple devices that are exporting data from IoT Central to Azure Blob Storage, I am wondering how I can filter the data for each device so I can send to a Machine Learning model?

You can add filters to reduce the amount of data exported. There are different types of filter available for each data export type:

To filter telemetry, you can:

• Filter the exported stream to only contain telemetry from devices that match the device name, device ID, and device template filter condition.
• Filter over capabilities: If you choose a telemetry item in the Name dropdown, the exported stream only contains telemetry that meets the filter condition. If you choose a device or cloud property item in the Name dropdown, the exported stream only contains telemetry from devices with properties matching the filter condition.
• Message property filter: Devices that use the device SDKs can send message properties or application properties on each telemetry message. The properties are a bag of key-value pairs that tag the message with custom identifiers. To create a message property filter, enter the message property key you're looking for, and specify a condition. Only telemetry messages with properties that match the specified filter condition are exported. The following string comparison operators are supported: equals, does not equal, contains, does not contain, exists, does not exist. Learn more about application properties from IoT Hub docs.

Additionally, do I need to create multiple ML models for each device or can I send all filtered data to the same model? (For ex. If I have 5 devices collecting temperature data and I want to predict if the device user has a cold, I want to use the same model that predicts colds but I want to predict separately for each user based on their data)

I am enquiring at the moment and will update you with my findings.