The question is resolved. needs to refer to a file with some dummy values instead of an empty file.
azure-data-factory-cannot-save-the-output-of-set-variable-into-file-database
Azure Data Factory: Cannot save the output of Set Variable into file/Database.
I'm trying to store a list of file names within an Azure Blob container into a SQL db. The pipeline run successfully, but after running the pipeline, it cannot output the values (file names) into the sink database and the table is still empty after the pipeline completed. Followings are the steps I went through to implement the pipeline. I wonder which steps I made mistake.
I have followed the solutions given in the following links:
https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-overview#add-additional-columns-during-copy
Steps: Validating File Exists, Get Files metadata and child items, Iterate the files through a foreach.
1-
2- Variable defined at the pipeline level to hold the filenames
3- parameter defined to dynamically specify the dataset directory name
4- Get Metadata configurations
5- Foreach settings
@activity('MetaGetFileNames').output.childItems
6 - Foreach Activity overview. A set Variable to set the each filename into the defined variable 'files'. Copy Activity to store the set value into db.
7- set variable configuration
8- Copy Activity source configuration. Excel Dataset refers to an empty excel file in azure blob container.
9- Copy Activity sink configuration
10-Copy Activity: mapping configuration
1 answer
Sort by: Most helpful
-
Alireza Tabebordbar 6 Reputation points
2022-04-24T02:00:42.517+00:00