question

Hello @Vinay5-0499,
Thanks for the ask and using the Microsoft Q&A platform .

1.loading data between two dates. (Ex - 2018-10-10 to 2018-12-10).

Since the dates in you are case are known , I will try to create a array and add the date in that ( I used excel , but you can use any text editor) , then use a for each loop ( FE loop ) and inside that I will add a copy activity with a parameterized dataset . The parameter which we will pass is the is the date element from the array .
Below animation and snapshot should help .

1. When you say 'Delta loading " , I am assuming that you want to copy the files every day , on a fixed schedule . If that case then you use the same pipeline , remove array and FE each and create a the date like 2021-03-18 when dynamic expression with the date expression . You will have to use the dataset paramterized .

   
   
   

Please do let me know how it goes .
Thanks
Himanshu
Please do consider to click on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members

Hi Himanshu,

Thanks for your answer. While I was waiting for the reply to my question, I tried the logic given in the below link and it worked.

https://docs.microsoft.com/en-us/answers/questions/162895/adf-pipeline-to-increment-the-date-back-to-six-mon.html

However, I got stuck at delta loading.
The files in S3 bucket gets added randomly everyday and I have to copy them to Azure as in when they arrive.
Could you please elaborate on how to achieve this requires.