'Execute Pipeline' activity fails to pass a parameter of array type to child pipeline

ThierryL 141 Reputation points
2022-03-21T07:52:28.58+00:00

Hello,

I have designed a pipeline which executes a ForEach activity on items defined in the same pipeline's parameter 'source_tables' of type Array.
The default value for this parameter is defined as follows:

[{"source":{"table":"tableA"},"destination":{"fileName":"tableA.parquet","datasetName":"tableA"}},  
 {"source":{"table":"tableB"},"destination":{"fileName":"tableB.parquet","datasetName":"tableB"}},  
  *other items...*   
]  

The pipeline works well when run by itself.
What I'm trying to do is to reuse this pipeline in another pipeline with an Execute Pipeline activity.
When I drag and drop the existing pipeline (child) into the new pipeline (parent), I can see that the default value for the 'source_tables' parameter of the child pipeline is well reflected and visible from the parent pipeline when I click on the Execute Pipeline activity.
However, when I execute the parent pipeline, it fails with a message:

Operation on target ForEach_xxx failed: The execution of template action 'MainForEach_xxx' failed: the result of the evaluation of 'foreach' expression '@pipeline().parameters.source_tables' is of type 'String'. The result must be a valid array.  

'@pipeline().parameters.source_tables' is the dynamic expression I use in the ForEach activity of the child pipeline to retrieve the value of the parameter of the child pipeline.
Again, the child pipeline works well on its own, but when it is called from another pipeline the Array somehow becomes a string...

185050-pipeline.jpg

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,525 questions
0 comments No comments
{count} votes

Accepted answer
  1. AnnuKumari-MSFT 30,601 Reputation points Microsoft Employee
    2022-03-21T15:22:26.027+00:00

    Hi @ThierryL-3166 ,
    Thankyou for using Microsoft Q&A platform and posting your query.
    Let me summarize your issue , you are using execute pipeline activity to run the child pipeline having pipeline parameters with some default values. When you are executing the master pipeline, you are expecting the child pipeline should run with the default values provided. Please let me know if my understanding is not correct.

    You need to pass the parameter value from the master pipeline. As the default value is set for child pipeline , however , as you are calling it from master pipeline you need to provide the values from master pipeline itself. Kindly try creating the parameter in master pipeline with the same value and call that pipeline parameter inside the source_tables value using dynamic content.

    Here is the video for your reference: 185286-executechildpipeline.gif

    Please refer to following video for more details : https://www.youtube.com/watch?v=nc4IFKkkfXM&t=35s

    Hope this will help. Please let us know if any further queries.

    ------------------------------

    • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you.
      Original posters help the community find answers faster by identifying the correct answer. Here is how
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification
    • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

3 additional answers

Sort by: Most helpful
  1. MADHUSUDAN PANWAR 86 Reputation points
    2023-02-23T18:10:30.66+00:00

    I faced the same problem but the above accepted solution didn't work for me.

    I edited the JSON code and put the array in the way it's mentioned in the below link


  2. MADHUSUDAN PANWAR 86 Reputation points
    2023-02-23T18:10:51.99+00:00

    I faced the same problem but the above accepted solution didn't work for me.

    I edited the Json code and put the array in the way it's mentioned in the below link .

    0 comments No comments

  3. Todd Chittenden 21 Reputation points
    2024-01-17T20:17:44.18+00:00

    I, too, am using a nested loop setup. My outer pipeline has a ForEach loop that iterates over the first array. Inside, it calls the inner pipeline and passes it a second array as a parameter. The inner pipeline has a second ForEach task that iterates over the second array. In both the outer and inner pipelines, the parameter is defined as an array. And the inner pipeline fails. My solution was to change the parameter type on the outer pipeline from array to string. I guess an array type parameter cannot accept an array as input, go figure. But it CAN accept a string input.