Hi Suppose I have 8 databases on premise and if i want to implement the lambda architecture then How can i ETL the data from these on-premise databases into cloud databases/storage in realtime. Can i use the Event hub for this? Is there any better approach to do this do i need to write producer app too for event hub to stream realtime on-premise relational data to cloud

Realtime time data from on premise databases

Kuldeep Chitrakar 1

Suppose I have 8 databases on premise and if i want to implement the lambda architecture then How can i ETL the data from these on-premise databases into cloud databases/storage in realtime.

Can i use the Event hub for this?
Is there any better approach to do this
do i need to write producer app too for event hub to stream realtime on-premise relational data to cloud

MartinJaffer-MSFT 26,031 Reputation points

2020-09-22T22:32:50.873+00:00

Hello @Kuldeep Chitrakar and welcome to Microsoft Q&A.

Before I can give you a good answer, I need to know more about your situation and goals.
ETL stands for Extranct Transform Load. What transformations are you looking to do, or do you just want to move/copy the data to cloud? Also, what sort of sink are you planning to use?

While Event hub is one approach,The approach depends upon the details of your needs.

If your source databases are SQL, there are two features you should look into. "Data sync" and "Transactional replication".
MartinJaffer-MSFT 26,031 Reputation points

2020-09-24T19:40:49.637+00:00

@Kuldeep Chitrakar did you get a chance to review my suggestion or get more details on your requirements?
MartinJaffer-MSFT 26,031 Reputation points

2020-09-25T19:14:42.01+00:00

I haven't heard back from you, @Kuldeep Chitrakar . Are you still in need of assistance?
If you found your own solution, could you please share it here to benefit the community?

Thank you.