question

cloudnativeuser avatar image
0 Votes"
cloudnativeuser asked JayaC-MSFT answered

Is Azure Durable function are efficent/suitable choice to dump around 1 billion records from cloud based Data warehouse to RDBMS?.

We want to create once a day activity to copy Data from Cloud based Dataware house to Azure based RDBMS.
Record count we are expecting daily is around 1 billion record size is small. Azure durable functions will be a good choice as compared to a microservice ? Or there are other suitable cloud native solutions ?

azure-functionsazure-cloud-services
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @cloudnativeuser, Welcome to Microsoft QnA. Thank you for your question. We will review and update at the earliest.

0 Votes 0 ·

1 Answer

JayaC-MSFT avatar image
0 Votes"
JayaC-MSFT answered

Hello @cloudnativeuser, In this scenario, you should consider Azure Data factory

Azure Data Factory is the platform that solves such data scenarios. It is the cloud-based ETL and data integration service that allows you to create data-driven workflows for orchestrating data movement and transforming data at scale. Using Azure Data Factory, you can create and schedule data-driven workflows (called pipelines) that can ingest data from disparate data stores. You can build complex ETL processes that transform data visually with data flows or by using compute services such as Azure HDInsight Hadoop, Azure Databricks, and Azure SQL Database.

https://azure.microsoft.com/en-in/services/data-factory/#features

https://docs.microsoft.com/en-us/azure/data-factory/connector-azure-sql-database


Please let me know if this helps.




5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.