It is observed that performance of the data factory pipeline that is using Azure IR to copy data from a ADLS Gen2 Storage account to another ADLS Gen2 Storage account is slowing down after 12hours. When restarted, the speed is picking up. Please share any suggestions to ensure consistent performance or improve the performance in such scenarios.
Pipeline Run ID - 1fa60a85-4349-4f12-9884-92ece1db6fb8
It took ~3hours to copy 4TB of data as below (Throughput - 423MBPS)
But other activity which started after this is completed is taking more time as below (Throughput - 35 mbps, very less compared to above one that is causing it to run for longer duration)