question

UtsavChanda-0290 avatar image
0 Votes"
UtsavChanda-0290 asked SathyamoorthyVijayakumar-MSFT answered

Handle comma(,) in column value in comma separated delimited text file

I have comma separated delimited file as input and using ADF copy activity to copy it to a .CSV file
Now if a field has a comma(,) in its value, obviously it treats the rest of the field value as the next column and the copy aborts since it is having one extra column than expected.

How to escape the comma(,) in between the field value?

azure-data-factory
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

SathyamoorthyVijayakumar-MSFT avatar image
1 Vote"
SathyamoorthyVijayakumar-MSFT answered

Hello @UtsavChanda-0290,

Thanks for the question and using Microsoft Q&A.

One thing I observed, when I had used MS Excel to create an csv through Excel. I saw that when the comma (,) was present in the FieldValue. There were quotes ("") between the text that contained comma (,).

138420-image.png

The ADF was able to process the above CSV file.

If the field values that contained comma(,) are not quoted (""), then I encounter same error message mentioned above.

I can think of 2 option at our end:

Option 1 :

you could configure the source that is generating the comma separated files to quote either all field values or the field values that contain comma(,).

 TestText,"V,S",29

OR

 "TestText","V,S","29"

This is to differentiate between comma in column separation and comma in field value.

Alternatively, you could check whether you could change the delimiting character at the source.


Option 2 :

If the above is not viable option for you, you could choose to skip rows that are incompatible (i.e. field values with comma) with the below configuration under the setting section.

138464-image.png

This is skips all the rows that are incompatible and you would also get a log stating which row in the file has been skipped. The copy activity is carried out sucessfully for the rest of the rows.

138485-image.png

Hope this will help. Please let us know if any further queries.


  • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how

  • Want a reminder to come back and check responses? Here is how to subscribe to a notification

  • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators



image.png (6.3 KiB)
image.png (17.1 KiB)
image.png (9.8 KiB)
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.