question

AzureUser-9588 avatar image
0 Votes"
AzureUser-9588 asked WilliamDam-6254 edited

Azure data factory failures and disaster recovery

How can I configure my Azure data factory for failover and disaster recovery?

Say I have a pipeline expected to run for 2 hours and after 1 hour if the entire Azure region is down in which Azure Integration Runtime is running [or] after 1 hour if the Azure Integration Runtime got crashed (I think ADF has 99.9% SLA). What is the expected behavior in these scenarios?

Is there any way I can configure my pipeline resilient to these kind of failures? How I can configure the DR to tackle them?

azure-data-factory
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

HarithaMaddi-MSFT avatar image
0 Votes"
HarithaMaddi-MSFT answered HarithaMaddi-MSFT edited

Hi @AzureUser-9588,

Welcome to Microsoft Q&A Platform. Thanks for posting the query.

  • Azure Data Factory has BCDR support. In case of disaster factory will be failed over to the paired region . The customer need not do anything if the outage is decided/declared by ADF team.
    In case of disaster there may be some metadata loss. RPO - 24 Hrs.
    It may take upto 24Hrs for fail over to complete. RTO - 24 Hrs.
    Runs may fail during the disaster time and before the fail over happens.
    Once the fail over completes, triggers will be restarted and data restored.
    No change needed from customer side, once the fail over completes

Please refer understanding-rpo-and-rto-considerations-of-azure-solutions for RPO and RTO definitions.

  • When using Azure IR, If, say, your source and your sink both failed over to another region during a regional outage. If the azure IR is marked auto-resolved as below, then ADF will detect the data source and sink region when the new pipeline job starts.

35397-image.png

  • In case of SHIR (Self-Hosted Integration Runtime), the documentation suggests creating multiple nodes for IR to ensure high availability during disasters.

Additional Reference: azure-data-factory-disaster-recovery

Hope this helps! Please suggest for further queries and we will be glad to assist.



  • Please accept an answer if correct. Original posters help the community find answers faster by identifying the correct answer. Here is how.

  • Want a reminder to come back and check responses? Here is how to subscribe to a notification.




image.png (88.6 KiB)
· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @AzureUser-9588,

We have not received a response from you. Are you still facing the issue? If you found a solution, would you please share it here with the community? Otherwise, let us know and we will continue to engage with you on the issue.

Please do consider to click on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members

0 Votes 0 ·

Hi @AzureUser-9588,

We still have not heard back from you. Following up to check if above answer helped in resolving the issue. Let us know for further queries and we will be glad to assist.

Please do consider to click on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members

0 Votes 0 ·
AnthonyAlbertoObandoCarrasco-9792 avatar image
0 Votes"
AnthonyAlbertoObandoCarrasco-9792 answered

Hello HarithaMaddi:

Thank you so much for clarifying the whole answer. I have a question: You pointed in case Region is marked as "Auto Resolve", but , what happens if a selected Region (let's say Central US) goes down, what would happen with source and sink?

Thank you!

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.