question

Nicholas-3747 avatar image
0 Votes"
Nicholas-3747 asked ·

ADF copy activity logging changing order of CSV columns in OperationItem

Trying to export all failed (incompatible) rows for an input CSV file and was hoping I could use the output found in OperationItem, but I have found that in my example file the last two columns are swapped and there is an extra comma at the end. I'm using the correct double quote escaping, it just looks like ADF is changing the order of the CSV columns. Is this a bug?

The setting are here:
73489-image.png


And I found it was the last two columns (could be others too).
Source file:
73509-image.png

Output from the logging when it skips incompatible rows:
73552-image.png

NOTE: The error is because a field in the CSV exceeds the size of a SQL column.


azure-data-factory
image.png (39.1 KiB)
image.png (12.8 KiB)
image.png (12.8 KiB)
· 8
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

@nicholas-3747 Thanks for using Microsoft Q&A !!
I tried to reproduce the same scenario but not seeing the behavior which you are seeing. What's the escape character and quote character you are using on csv source dataset. Also, how many columns you have in csv ? Is CSV is having a header row ?

0 Votes 0 ·
Nicholas-3747 avatar image Nicholas-3747 SaurabhSharma-msft ·

So to read the data quote and escape are both double quotes. But the output in the text file as as shown in this question I.E before I read the log file produced by the copy activity. I can upload an example CSV if you like?

0 Votes 0 ·

@nicholas-3747 Thanks. Yes can you please upload the example csv. Also share table schema please.

0 Votes 0 ·
Show more comments

0 Answers