question

alexandreg avatar image
0 Votes"
alexandreg asked PRADEEPCHEEKATLA-MSFT edited

Errors while downloading files from SFTP

Hello,

I'm encountering an error since a few days from a pipeline that has been working fine for months, on the same SFTP server, with no changes recently.

The pipeline look up for files in a SFTP server (Get Metadata activity), then download each file to ADLS (Copy activity).

This is the error I get :

Failure happened on 'Sink' side. ErrorCode=UserErrorFailedToReadStream,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Failed to read data from Sftp server 'sftpserver.com', file path '/path/myfile.zip', offset '16777216', length: '16777216'.,Source=Microsoft.DataTransfer.ClientLibrary.SftpConnector,''Type=Renci.SshNet.Common.SshException,Message=Failed to read file.,Source=Renci.SshNet,'

What could be the cause of this? It has been working fine for months.

Thanks


azure-data-factory
· 5
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @alexandreg,

Thanks for the question and using MS Q&A platform.

Could you please confirm, is SFTP linked service test connection is working fine?

Also, if test connection is successful, kindly check the permissions of identity which is used in linked service having proper access to perform Read.

0 Votes 0 ·
alexandreg avatar image alexandreg PRADEEPCHEEKATLA-MSFT ·

Hello! Yes the test connection is working fine. I have read and execute (r-x) rights on the SFTP. I can see the files. ADF starts to download them, but after around 16MB downloaded, fail with the above mentioned error message.

0 Votes 0 ·

Hello @alexandreg,

Thanks for the details.

I'm reaching out to the internal team to get help on this issue and will update you as soon as we have a response.

Thank you for your patience.

0 Votes 0 ·
Show more comments

1 Answer

PRADEEPCHEEKATLA-MSFT avatar image
1 Vote"
PRADEEPCHEEKATLA-MSFT answered PRADEEPCHEEKATLA-MSFT edited

Hello @alexandreg,

Thanks for posting your question and using MS Q&A platform.

The error message indicates that the SFTP server not support read from non-zero offset. And our service has no changes recently, so I suspect there is behavior change in your SFTP server side.

Besides, I would suggest you to try this: in copy source, set max concurrent connections to 1, and see how the result.

102280-image.png

Investigating with the provided activity id:

We suspect there are some changes on server side. I would suggest you reach out to the SFTP server vendor.

Hope this helps. Do let us know if you any further queries.


Please "Accept the answer" if the information helped you. This will help us and others in the community as well.


image.png (324.5 KiB)
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.