Hi,
I have created a data copy task in Azure Synapse . that basically copies 1000 records from Onprem SQL to Azure Storage - in .csv format. ( dynamic name - Filename_UTCNOW_MMHHSS.csv)
In the next phase of pipeline, I want to use the created file in datacopy as a source for the DataFlow.
How can i use the dynamically created file in the dataflow for further processing.
Thanks,
Parikshit.
Just checking in to see if the below answer helped. If this answers your query, do click 
and upvote 
for the same. And, if you have any further query do let us know.
Thank you for posting query in Microsoft Q&A Platform.
We can achieve this by using a variable to store dynamic file name and then pass that variable as value to dataflow. In data flow we should have a parameter to take variable as value and then in source transformation we should use wild card path to write expression to point file.
Please check below high-level implementation steps to get better understanding.
Step1: Create a variable in pipeline to hold dynamic file name and use set variable activity to set value to it.
Step2: Copy activity - to copy data from on-prem SQL to csv file. Here under sink we will be using variable from step1 as value for file to generate. Sink dataset will be parameterized on to dynamically take file name.
Step3: Have parameter created in Dataflow and use expression inside source transformation to dynamically point to same created file.
Hope this helps. Please let us know if any further queries.
Please consider hitting Accept Answer button. Accepted answers help community as well.
11 people are following this question.
