Hello. I monitor telemetry data with IoT Central. I want to flow the telemetry data I am receiving in IoT Central to Cosmos DB in real time. Is there any way to do this? Thanks in advance.

Hello @Hinata Katsunori , welcome to this moderated Azure community forum. Is Azure CosmosDB a hard requirement or could any database be used? If not, check out the tutorial for Azure Data Explorer . Azure Data Explorer is a database optimized for time-series data; this means time related facts, observation, telemetry. It's great for querying over large amounts of data within seconds or less. For querying telemetry, it gives a better performance than CosmosDB. Perhaps the standard prise of a Azure Data Explorer cluster, in that case check out the free tier. You can make use of the free tier by exporting live data to an Event Hub first. From there, check out this tutorial about ingesting streaming Event Hub data into the free Azure Data Explorer. Compare this with CosmosDB and see what is more suitable for your solution. If the response helped, do "Accept Answer". If it doesn't work, please let us know the progress. All community members with similar issues will benefit by doing so. Your contribution is highly appreciated.

Yes, Azure Cosmos DB is the proposed database for the IOT scenarios, you can check this reference architecture https://learn.microsoft.com/en-us/azure/architecture/solution-ideas/articles/iot-using-cosmos-db

I want to store data from IoT Central to CosmosDB in real time.

Accepted answer

Sander van de Velde 28,706 Reputation points MVP

2024-04-16T06:14:42.9966667+00:00

Hello @Hinata Katsunori ,

welcome to this moderated Azure community forum.

Is Azure CosmosDB a hard requirement or could any database be used?

If not, check out the tutorial for Azure Data Explorer.

Azure Data Explorer is a database optimized for time-series data; this means time related facts, observation, telemetry.

It's great for querying over large amounts of data within seconds or less. For querying telemetry, it gives a better performance than CosmosDB.

Perhaps the standard prise of a Azure Data Explorer cluster, in that case check out the free tier.

You can make use of the free tier by exporting live data to an Event Hub first.

From there, check out this tutorial about ingesting streaming Event Hub data into the free Azure Data Explorer.

Compare this with CosmosDB and see what is more suitable for your solution.

If the response helped, do "Accept Answer". If it doesn't work, please let us know the progress. All community members with similar issues will benefit by doing so. Your contribution is highly appreciated.
Please sign in to rate this answer.
Hinata Katsunori 536 Reputation points

2024-04-16T14:40:36.3266667+00:00

@Sander van de Velde
Thank you for the information.

Is Azure CosmosDB a hard requirement or could any database be used?

First of all, we want to simply store the data and we are thinking of linking to Cosmos DB. We may also look at using Data Explorer in the future. Thanks for the information.

I have a question on the tutorial description you gave us.

The following statement is made about exporting telemetry.

'Export telemetry messages from devices in near-real time.'

This is the feature I was hoping for, but I am concerned about the way it is written 'near-real time'.

Does this mean that the IoT Central export feature here will be slower than, for example, flowing telemetry data from the IoT Hub to Cosmos DB?

If it is slower, do you know what the delay is?

Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.

Sander van de Velde 28,706 Reputation points MVP

2024-04-16T21:00:20.3066667+00:00

Hello @Hinata Katsunori ,

Timing telemetry is a hard challenge due to the many services and connections between sensor and dashboard.

If you expect sub-second responses, this is a promise you can only expect on the edge, on the local network.

Still cloud solution can deliver near real-time insights within seconds, depending on the setup and available resources (read: compute power).

Personally, I expect that the ingestion of Azure Data Explorer will take more time due the batching mechanism (although there is also a smart streaming ingestion alternative).

On the other hand, the actual querying (thus displaying data in a real-time dashboard) is much faster.

I recommend just testing both database solutions and see what is the best option for you. Setting up a proof of concept could be done in a couple of hours (check the tutorial link above) and you have a good indication of what to expect.

Hinata Katsunori 536 Reputation points

2024-04-17T06:15:51.97+00:00

@Sander van de Velde
Thank you.

It was very helpful.

I will try about the real-time nature.

Hinata Katsunori 536 Reputation points

2024-04-17T06:17:08.78+00:00

@Sander van de Velde
Thank you.
It was very helpful.
I will try about the real-time nature.
Sign in to comment

I want to store data from IoT Central to CosmosDB in real time.

1 additional answer