Uploading many files with FTP service

Question

question

JORGEMALDONADOBARRERA-7687 asked Oct 26, '20 JoyQiao-MSFT answered Oct 27, '20

Uploading many files with FTP service

Hi,

I have a Windows Server 2019 cloud server and I need to transfer about 2 million documents (PDF, XML, text). My cloud server has no limit in the amount of data transfer. I have performed several tests and it always fails at some point. The connection drops and my PC Internet service also breaks and I need to restart it. The procedure I follow to upload files is as follows:

1) Make a connection to the FTP service as an administrator.
2) Select local folders to be uploaded.
3) Drag and drop selected local files to target.
4) Go to step 2

I do this and, suddenly, the connection drops and my PC has no longer access to the Internet.
I have also tried to close and open the FTP connection each time I select source folders to upload without success.

I also have a Lynux CentOS cloud server in which I also made tests without problem.

I have asked for help in other forum and I was suggested this solution but it did not work (https://social.msdn.microsoft.com/Forums/en-US/62c12a7a-8b3b-468b-9be3-c23396616b9e/ftp-server-connection-times-out-when-uploading-multiple-files?forum=winserver8gen)

I have tried FileZilla and WinSCP clients.

Is this a limitation of the Windows Server FTP service?

Best regards,
Jorge Maldonado

windows-server-2019

5 |1600 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Answer 1 · 2020-10-26T18:55:58.853Z

SethWH answered Oct 26, '20

I personally wouldn't use Windows IIS FTP services over the web. Is that what your using or WinSCP SFTP host service? If you can manage to upload the files to your linux cloud server (assuming the Windows Server is with the same host provider), you can grab the files from there if your server had an SFTP client on the Windows Server.

5 |1600 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Answer 2 · 2020-10-26T20:16:57.137Z

MotoX80 answered Oct 26, '20 MotoX80 edited Oct 26, '20

Is Defender or another AV product scanning the files that you are uploading? It sounds like something is keeping network connections open. RDP to the server and watch what processes are using cpu with task manager.

I would suggest using the Site Manager within the Filezilla client and in the Transfer Settings tab, set a limit on the number of simultaneous connections. Use FZ to upload the directories

With an admin command prompt run "netstat -aon" and watch how many connections you have between the client and the FTP server.

Start with 2 or 3 connections. If netstat shows that the number of connections is more less steady, then gradually increase the number. But watch network utilization too. Try not to overwhelm any resource (cpu/disk/network).

5 |1600 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Answer 3 · 2020-10-27T09:25:43.437Z

JoyQiao-MSFT answered Oct 27, '20

Hi,

According to your description, we think your issue is more related with IIS. So we recommend to create a thread on IIS forum with the link below:

https://forums.iis.net/

Thank you for your understanding.

============================================
If the Answer is helpful, please click "Accept Answer" and upvote it.
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

5 |1600 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

question