Hi All,
I am looking for a document that shows how to load multiple files from a files system into ADLS Gen2. Please advice.
Thanks
Hi All,
I am looking for a document that shows how to load multiple files from a files system into ADLS Gen2. Please advice.
Thanks
@NandanHegde-7720
Thanks i came across the process but not able to figure it out, please see the below screen shots, do i need to pass parameters ?
![129967-capture97a.png][2]![129888-capture97b.png][3]
Sorry a quick update i entered the childitems in the field list, now i was able to copy 13 out of the 17 files, please see the below screen shot, it shows as failed but does not shows the error message
Another update the four files that were failing with the below message
Error details
Error code2200
Troubleshooting guide
Failure typeUser configuration issue
DetailsFailure happened on 'Sink' side. ErrorCode=DelimitedTextMoreColumnsThanDefined,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Error found when processing 'Csv/Tsv Format Text' source 'PartyData.txt' with row number 332834: found more columns than expected column count 31.,Source=Microsoft.DataTransfer.Common,'
I looked at the source file and there is a column (comment) and it has commas in it and they are valid, so how to overcome this issue without removing these kind of rows.
Please advice.
Thanks
Hey,
Based on the error message it seems there is a delimiter value within one of the column values in your files.
Since you are copying the files, I would say use Binary dataset rather than delimited .
https://docs.microsoft.com/en-us/azure/data-factory/format-binary
IIn case of binary dataset, the file is copied as is. There is no validation of delimitter data or file format etc.
It would be a case of just copy paste scenario for binary dataset. So would say shift to Binary dataset
Thanks @NandanHegde-7720 yes i looked at used double quotes as escape character (")
That resolved the issue.
Glad that worked out.
But ideally in case of just copying files from 1 location to another, it would be preferred to use binary datasets :)
Can you please accept it as answer as it might be beneficial for other community members as well .
Hey,
The below flow can help :
1) Set up SHIR and Create a linked service with file system
2) Then via GetMetaData activity, get the list of files
3) Via Foreach iteratively copy the files from the file system into ADLS
4 people are following this question.