question

SUNGMINKIM-8008 avatar image
0 Votes"
SUNGMINKIM-8008 asked ·

What is this pipeline error in Azure Synapse Analytics?

Run ID : 248db404-306f-4067-bfbc-01743d64212f

It works well until 2021.02.22 07:30 by daily trigger. But I triggered manually on 2021.02.22 18:00, and occurred an error this time.

What is problem with it?
Data type or Blob setting?


Here is Error Detail.

{
"errorCode": "2200",
"message": "ErrorCode=FailedDbOperation,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Please make sure SQL DW has access to ADLS Gen2 account,Source=Microsoft.DataTransfer.ClientLibrary,''Type=System.Data.SqlClient.SqlException,Message=HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: HadoopExecutionException: Not enough columns in this line.,Source=.Net SqlClient Data Provider,SqlErrorNumber=107090,Class=16,ErrorCode=-2146232060,State=1,Errors=[{Class=16,Number=107090,State=1,Message=HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: HadoopExecutionException: Not enough columns in this line.,},],'",
"failureType": "UserError",
"target": "----------------------------------",
"details": []
}

azure-synapse-analyticsazure-blob-storage
· 6
10 |1000 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @SUNGMINKIM-8008,

Welcome to Microsoft Q&A and thanks for your query.

Could you please confirm if you are noticing the error consistently when you try to run the pipeline with manual trigger? If so, please confirm if you were able to manual trigger this pipeline without any issue previously or is this the first time you tried manual trigger?

Are you experiencing this issue only with manual trigger or do you also observe the same behavior with daily automated trigger?

Also please do share the complete error message which your are seeing.

We look forward to your response.

Thank you

0 Votes 0 ·

Our customer uses Oracle DB for source. And we import Data with azure.
They only did insert data in the DB between Daily automated trigger and Manual trigger.
A Manual trigger was performed after their INSERT operation.

It occurs both manual and automated trigger.


Sorry, but Do you mean full target table name?
I wrote complete error message and also Run ID.

Can't I know the reason type with that message?

0 Votes 0 ·

Hi @SUNGMINKIM-8008 ,

Thanks for the details. We are looking into it and will get back to you as soon as we have an update.

Thank you for your patience

0 Votes 0 ·
KranthiPakala-MSFT avatar image KranthiPakala-MSFT SUNGMINKIM-8008 ·

Hi @SUNGMINKIM-8008,

Apologizes for the delay in my response. Your issues seems to be exactly the same as this issue : https://docs.microsoft.com/answers/questions/201623/not-enough-columns-in-this-line.html

Could you please confirm what is your source and sink? Also please please confirm if you are using Polybase? Are you noticing this issue consistently or only with a particular set of data?

As discussed in other thread the error message could be misleading but the actual issue might be with the data which was processed during this run failure.

Please let us know the above information so that we can escalate to internal team for further analysis.

Thank you

0 Votes 0 ·
Show more comments

0 Answers