question

VikasTiwari-2263 avatar image
0 Votes"
VikasTiwari-2263 asked ShreyasiSrivastava-1468 commented

ADF For each with until activity

I have 6 different API details as rows in sql watermark table and using foreach to iterates over these 6 rows and copy data from source to destination using ADF pipeline.

Now, I need to split data into different files and each file will have 2000 records, for this I have tried until activity and its working fine standalone but unable to find way where I can use until activity inside foreach loop.

Basically I am looking to find a way, foreach -> 1st iteration (call 1st API)-> get the data and split using until activity -> 2nd iteration (call 2nd API)->get the data and split using until activity .... upto 6 iteration.

Is it possible to use until activity under foreach loop?

azure-data-factory
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi Vikas,
Could you please share the snapshot of your pipelines for for each and until activity pipelines, I have the same issue and I am not able to understand how to pass parameter as value to the datasets in different pipelines

0 Votes 0 ·

1 Answer

NandanHegde-7720 avatar image
0 Votes"
NandanHegde-7720 answered ShreyasiSrivastava-1468 commented

Hey,
You can call excute pipeline activity within for each activity.
So you can add the untill logic in another pipeline and call that pipeline within foreach

· 4
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Thanks @NandanHegde-7720. Do you also if we can pass variable across the pipelines?

I have tried to set variable in main pipeline but its not showing in child pipeline to use it.

0 Votes 0 ·

Hey,
you need to pass the variable value of 1st pipeline as parameter to another pipeline in order to use it..

https://docs.microsoft.com/en-us/answers/questions/175485/passing-parameters-in-the-execute-pipeline-activit.html

Please accept it as answer in case if helped as it might benefit other community members as well

0 Votes 0 ·

Thanks it worked as expected.

0 Votes 0 ·

Hi if I have pipeline parameter set in for each activity named relative url which has array of relative url as value that I need to pass as value for dataset parameter inside copydata which is the part of until activity, how can I achieve that?
199297-image.png


This is inside until activity.

0 Votes 0 ·
image.png (58.1 KiB)