question

BHATSCShivaranjini-5230 avatar image
0 Votes"
BHATSCShivaranjini-5230 asked BHATSCShivaranjini-5230 commented

ADF pipeline taks more than 12 hrs to complete

The same pipeline completed by 3-4 hrs max but now its takes more than 12 hrs to complete it

azure-data-factory
· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @BHATSCShivaranjini-5230,

Thanks for reaching out. Could you please confirm if this is happening consistently or intermittently? Please confirm what is the source and sink? What is the IR being used in both source and sink? If you are using SHIR older version, could you please give a try upgrading to latest version and see if it helps.

If you are using Copy activity, firstly I would recommend you to please review the Troubleshoot copy activity performance document to check if you can implement performance tuning on the pipeline.

Also have you noticed any difference in the amount of data movement before and now? If the above info didn't help please provide pipeline and activity runID for both scenarios (slow run as well as normal run)

Thank you





0 Votes 0 ·

Hi @BHATSCShivaranjini-5230,

We still have not heard back from you. Just wanted to check if you are still facing the issue or need assistance on this? In case If you already found a solution, would you please share it here with the community? Otherwise, let us know and we will continue to engage with you on the issue.

Thank you

0 Votes 0 ·

Hello Karthik,
The issue got resolved by using the statistics and indexes and hash distribution in some of the tables.
Thanks

1 Vote 1 ·

0 Answers