Memory Leak in Self-Hosted IR Affects Data Copy Tasks in Azure Data Factory

Question

Hello,

I am encountering a serious issue with the Azure Self-Hosted Integration Runtime where a process named diawp.exe is consuming all available memory. This is impacting the performance of data copy tasks in Azure Data Factory, where jobs remain pending due to lack of resources.

Issue Description: The problem started last week. Despite no active data factory pipelines running, about 5-10 minutes after rebooting the VM, the diawp.exe process appears and uses up all the available memory.

This process also seems to trigger randomly, and I am unable to identify the root cause. I have to manually kill this task each time to ensure that the Data Factory pipelines can proceed.

Has anyone else experienced this issue or can provide insights into what might be causing this memory consumption? Any suggestions on how to prevent this from recurring would be greatly appreciated.

Thank you in advance for any assistance!

Answer 1

Hi Coby Lin,

Thanks for reaching out to Microsoft Q&A.

Instead of duplicating, I am quoting an answer by BhargavaGunnam-MSFT for a similar issue which I believe sums up the best way. I suggest you go through the same (both answer and all the comments) and make sure if this is all checked and good. If incase the issue still persists, I advise you open a case with support for a faster resolution.

https://learn.microsoft.com/en-us/answers/questions/935059/self-hosted-integration-runtime-consuming-memory-a

Please 'Upvote'(Thumbs-up) and 'Accept' as an answer if the reply was helpful. This will benefit other community members who face the same issue.