Configuring sink data set in azure data factory

Question

0

I am trying to copy multiple folders with their files (.dat and .csv ) from ftp to Azure storage account , so I am using a get metadata for each and copy activity. My problem is that when setting the file path in the output data set I am not sure how to set the filename so it picks up all files in my folder.

I added a filename parameter in the data set and in the copydata sink I set it as @item().name but it's not working instead of copying the files, it copies the folder. the second try is that I dont set the filename in the directory, and it does copy the files but it adds the extension.txt to the files instead of keeping their original format.

Thank you for your help

Answer 1

Hi @Anonymous ,

Thank you for posting query in Microsoft Q&A Platform.

It seems your GetMetaData activity is giving you Folders names array and you are passing that folder names array to ForEach activity to iterate over each folder.

So, here inside Copy Activity source tab use "wildcard file path" and use patterns to copy .csv and .dat files as shown in below screenshot.

To store files in desired extension you can leverage Copy activity --> Sink tab --> File extension field

Hope this will help. Please let us know if any further queries. Thank you.

-----------------

• Please accept an answer if correct. Original posters help the community find answers faster by identifying the correct answer. Here is how.
• Want a reminder to come back and check responses? Here is how to subscribe to a notification.