question

MoS-2887 avatar image
0 Votes"
MoS-2887 asked KranthiPakala-MSFT commented

Why does the log generated by the Copy Data command contain an extra \r character in the Message field for Warning lines?

The log file data generated by the Copy Data activity in an Azure Synapse Pipeline appears to contain an extra \r character embedded within the Message field for Warning level lines, (while Info level lines do not exhibit this behavior.) We are aiming to ingest the log file data into a table, but the extra embedded carriage return character is causing the ingestion of the log data to fail.
Is there any work-around for how the log data from a Copy Data activity is written out?
A few sample lines from the logfile created by the Copy Activity are below. Note the carriage return preceding the date is correct "\r2021-08-19 15:47:41, while the carriage return embedded in the following string causes an additional line break, "gary"",""1"",""2\r"",","Column \'Prop"

Here are some sample logfile lines:
Timestamp,Level,OperationName,OperationItem,Message\r2021-08-19 15:47:41.1313523,Warning,TabularRowSkip,"""2"",""gary"",""1"",""2\r"",","Column \'Prop_2\' contains an invalid value \'1\'. Cannot convert \'1\' to type \'DateTime\'."\r2021-08-19 15:47:41.1313523,Warning,TabularRowSkip,"""3"",""gary"",""2"",""3\r"",","Column \'Prop_2\' contains an invalid value \'2\'. Cannot convert \'2\' to type \'DateTime\'."\r')]

azure-data-factory
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Are we talking: pushing the data to a MySql or SQL log table? If so, can't you just write a string cleanup routine to remove the carriage return?

0 Votes 0 ·

1 Answer

KranthiPakala-MSFT avatar image
0 Votes"
KranthiPakala-MSFT answered KranthiPakala-MSFT commented

Hi @MoS-2887,

Welcome to Microsoft Q&A forum and thanks for posting your query.

This is an expected behavior with the JSON output format in ADF. You will have to replace those or do escape those characters while writing to your desired SQL table/column.

Could you please confirm what is the service you are using to read this JSON data and write to your sql table?

Thanks

· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @MoS-2887,

Just checking to see if you have got a chance to see my previous comment, if so could you please confirm the details requested.

Thanks

0 Votes 0 ·

Hi there,

We still have not heard back from you. Just wanted to check if the above suggestion was helpful? If it answers your query, please do click “Accept Answer” and/or Up-Vote, as it might be beneficial to other community members reading this thread. And, if you have any further query do let us know.

0 Votes 0 ·