question

KatherineBrock-8747 avatar image
0 Votes"
KatherineBrock-8747 asked KranthiPakala-MSFT commented

ADF Copy Data Activity using Dynamics Connector - Slow Performance

I am using the ADF Connector for Dynamics. I have created a Linked Service and Dataset and using that in a copy data activity in my pipeline. It took 6.5 minutes to process <200MB of data (throughout ~500KB/s). This is not sustainable for our production pipelines. I am using the default Azure IR. Any suggestions for performance improvement would be greatly appreciated.

azure-data-factory
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

KranthiPakala-MSFT avatar image
0 Votes"
KranthiPakala-MSFT answered KranthiPakala-MSFT commented

Hi @katherinebrock-8747,

Welcome to Microsoft Q&A forum and thanks for reaching out. If you use an Azure Integration Runtime that is in a not optimal region resulting in slow read/write, it is suggested to configure an Azure IR in another region closest to your data source. Could you please try IR in different region and see if that helps.

Also please refer to guidance mentioned in this document to boost copy activity performance running on Azure IR : Troubleshoot copy activity on Azure IR

Do let us know if this helps.



Please don’t forget to Accept Answer and Up-Vote wherever the information provided helps you, this can be beneficial to other community members.


· 4
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @katherinebrock-8747,

Following up to see if you have got a chance to review the Azure IR performance tunings steps described in the document shared in my previous comment. Do let us know if you have further query.

Thanks

0 Votes 0 ·

Our Dynamics instance in East US and our ADF IR is set to auto-resolve region and I see that it is also running in East US. The biggest bottleneck is on Transfer - reading from source. Is there any recommended ways to enhancement the performance of our fetch XML queries?

0 Votes 0 ·

Hi @katherinebrock-8747, thanks for your response. Glad to know that you were able to narrow down the issue to query performance. As Dynamics is beyond my area of expertise, I would recommend you to reach out in Dynamics 365 community forum which is better suited forum for this ask, where we have dynamics experts who can better assist you on how to improve the fetchXML query performance.

Thank you


1 Vote 1 ·
Show more comments