question

PiyushMeshram-2335 avatar image
0 Votes"
PiyushMeshram-2335 asked MotoX80 answered

Load tests are failing due to System.Net.Sockets.SocketException

I need some help with this error:

System.Net.Sockets.SocketException A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond example.com:80


I have hosted a website in IIS on port 80 only.

5 other sub web services are hosted and all are using the same port 80 and same app pool.

I need to understand few things as follows:

  1. As I have several web application (a website and 5 Web Services) running, will it cause socket connection errors? if so how can I keep track of the limit of the available sockets IIS/System can handle.

  2. Web applications will be hosted on IIS 64bit of windows server 2019 8Gib RAM, Azure D2 V2 VM. Is it a good idea?

  3. I don't want the system to have socket exceptions because of too many concurrent connections. How to solve this problem? may be with each app in separate app pool or something?

  4. Test Case Scenario: Website will have max 5 users only but web services will be used by 30 users and each user can fire up to 90,000 requests in a day (sometimes up to 20 concurrent request each user might fire) for a single web API. I need a way to monitor/know how many concurrent requests are possible?

  5. Each web service call will take less than 2 seconds to complete and code is pretty good and it runs smoothly for 1000 concurrent threads no problem. I am using SOAP UI to load test the application from a PC with normal specs (dual core, 8gb ram) and if I increase the load the socket connection failures as mentioned above occurs. Where is the problem code, IIS Server level or client side?

  6. I think the above few points will help to understand the environment these applications are running in. I would like to know some best practices to follow in such cases.

windows-serverazure-virtual-networkwindows-server-iis-general
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

MotoX80 avatar image
1 Vote"
MotoX80 answered PiyushMeshram-2335 commented

Use the IIS request monitor to see active requests.

https://inthetechpit.com/2019/11/10/check-incoming-requests-iis-with-request-monitor/

Check the HTTP error logs for anything "bad". C:\Windows\System32\LogFiles\HTTPERR

Run the netstat command from an admin command prompt to see how many sockets are open.

 netstat -aon


Use LogParser to analyze your web sites log file. Look for long response times.

https://www.microsoft.com/en-us/download/details.aspx?id=24659

 logparser "SELECT  TO_LOCALTIME(TO_TIMESTAMP(date,time)) as [Local Time], c-ip, cs-username, sc-status, cs-uri-stem, cs-uri-query,  time-taken FROM c:\inetpub\logs\logfiles\w3svc1\u_ex*.log order by [Local Time] " -rtp:-1 -recurse 


· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Excellent! I have tried these steps you mentioned @MotoX80, following them was very helpful, but I have to come up with a number as to how many concurrent requests a system can handle. The problem is, two Azure VMs with the same configuration are behaving differently when it comes to LOAD testing with more than 1000 concurrent threads running locally and simultaneously. The total number of request failures are always different and the "difference margin" so is big that it is practically not possible to determine the average score.

for example Load test with 3000 users. 2500 users made the successful request on system A, while on system B (Azure VM with same configurations) only 1700 successful concurrent requests were made. This is one scenario, the ratio differs every time I do the testing. There must be something I am doing wrong here I just don't get it what? Also, the results vary too much more than this if the requests are made remotely.


What could I be doing wrong here?
Isn't there a way, I can say for sure the system with these configurations can handle these many users?

please note that the errors are only socket exceptions, no other errors so far.

Is it fair to assume the Azure system with a higher pricing tier and specs can handle more users?

Can I set a limit on the number of concurrent users in IIS even if a system is more capable?

Note: Netstat commands runs so slow that it takes forever to print active connections while the tests are running, I am sure you are aware of this.



0 Votes 0 ·

Please mention if different app pool for different webservices (an app pool for a published webservice project bin) will improve the performance or it will behave same?


Example: WebUI and "Web services app bin A" in one app pool
Web services app Bin B in app pool 2
WebServices app bin C in app pool 3 and so on.

Is it going to better the performance? or it will behave the same as when all these applications are in a single app pool?

0 Votes 0 ·
MotoX80 avatar image
0 Votes"
MotoX80 answered

Decades ago we were testing a new system and the company paid users overtime to come in on a weekend to test it. They all hit enter at the same time and the system crashed.

Then they asked me to try to simulate the users with some software that they purchased. So we tried to think through how a real user would access the system. They might come in to the office sign in, do a few things, and go get a cup of coffee or maybe go to the bathroom. Then come back to their desk and work at a somewhat randomized pace depending on when a customer might call for information, or the user might get interrupted with a phone call or something.

We had a number of discussions on "how many concurrent order entry users", "how many concurrent order inquiry users", "how many users accessing other parts of the application", "how may users logged in but doing nothing at any given time", etc. Plus our user base was US centric but spread out from east to west coast. So when west coast users were signing in, east coast users might be going to lunch.

So for whatever load test tool you are using, your "script" should include "think time" and spread out the user logon timeframe over whatever period you expect your users to sign in. Don't just throw thousands of users at your site, you need to consider how an actual human might use the site.

Based on the numbers that you have given, I would suggest starting small, lets say 500 users, and let it run for a while and try to reach some normalization point where you can determine that "this many users" need "this much cpu and memory". Watch the cpu/memory/disk utilization on both the web and database servers.

Then ramp up, go to 750 users. But each time you increment, you need to have your web and database admins look at the performance. What requests are taking the longest time? How efficient is the SQL call? Is the disk subsystem able to keep up with the I/O? How is memory utilization, CPU utilization, where is the bottleneck?

Did you use the '-aon' switches when you ran netstat?


5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.