question

KothaiRamanathan-7384 avatar image
1 Vote"
KothaiRamanathan-7384 asked FahadHameed-3841 commented

SFTP Connection Issue

I am trying to copy from file share to sftp and getting the below error.

Message=Meet network issue when connect to Sftp server 'xxx', SocketErrorCode:
'TimedOut'.,Source=Microsoft.DataTransfer.ClientLibrary.SftpConnector,''Type=System.Net.Sockets.SocketException,
Message=A connection attempt failed because the connected party did not properly respond after a period of time,
or established connection failed because connected host has failed to respond,Source=Renci.SshNet,'"

However, the same pipeline works in a different environment (to the same sftp server). I have compared the linked service, dataset and the pipeline code between the two environments are they are the same.

When I test the connection in the linked service, it is successful. Any pointers on the above error will be helpful, thanks.

azure-data-factory
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

KranthiPakala-MSFT avatar image
0 Votes"
KranthiPakala-MSFT answered FahadHameed-3841 commented

Hi @KothaiRamanathan-7384 ,

Thanks for your query. This could be because of either of the below 2 reasons:

a. It could be related to a firewall issue, if you are using SHIR on either source or sink and if the access is restricted to IPs that are whitelisted in the firewall rules of your SFTP, then could you please make sure to whitelist the IP address of the machine that hosts your SHIR. If you are using Azure IR, then you can choose to add Azure Integration Runtime IPs into the allow list.

Here is the download link: Azure IP Ranges and Service Tags – Public Cloud

b. Another possible reason I could think of is the number of Max concurrent connections to your SFTP server. Could you please check out what is the maximum concurrent connection limit to your sftp server? Based on that please set your sink Max concurrent connection less than the SFTP concurrent connection limit set by SFTP admin.

26177-image.png


Hope this information helps. If you still experience the issue, please do share the failed pipeline runID for further investigation.

Thank you



Please do consider to click on "Accept Answer" and "Upvote" on the post that helps you, as it can be beneficial to other community members.



image.png (22.2 KiB)
· 4
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

@KothaiRamanathan-7384 - Glad to know your issue was resolved. Thanks for confirming it's a SHIR machine network issue.

Here is few additional info why you were able to test connection of SFTP but fails when copy pipeline is triggered.
The reason is when you did test connection to your sink SFTP, Azure IR is being used to test the connection, but when you actually trigger run the pipeline, your copy is executed on SHIR since you source (Fileshare) is using SHIR and sink SFTP is using Azure IR.

Please see below document for better understanding of this behavior: Copy activity determining which IR to use

26630-image.png

Hope this additional information will be helpful for other community members who read this thread.

Thank you & Stay safe

2 Votes 2 ·
image.png (49.7 KiB)

Hi @KranthiPakala-MSFT, I am using Copy activity to copy file from source hosted by SHIR and target in Azure Runtime and facing the issue you mentioned above. Can you please guide what can be a possible solution to it?
Thanks in advance.

0 Votes 0 ·

Thank you @KranthiPakala-MSFT for the details.

a. I guess it is not a IP issue, as when I test the connection, I am able to browse the folders in the SFTP server.
b. I am only transferring 3 small files, and this works perfectly fine from my dev environment. Hence dont this is an issue either.

This is one run id that has failed. e05fdf72-449d-4c0d-9a5d-301ac5836147

Any pointers to resolve this issue will be helpful.

Thanks,
Kothai.

0 Votes 0 ·

Thank you @KranthiPakala-MSFT for further details. Yes, it was a case of IP whitelisting.

0 Votes 0 ·
atulsahu-3476 avatar image
0 Votes"
atulsahu-3476 answered

Hi Team,

Is there a set of standard-setting (like timeout value ), I need to share with the SFTP server admin?

My pipeline keeps failing on the linked service test, 2/5 times.

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.