How are the metrics "Requests in Application Queue" and "HTTP queue length" measured?

John K 6

Question 1:

At the App Service Plan level, there is a metric called "HTTP queue length". And on the App Service level there is a different metric called "Requests in Application Queue".

I haven't been able to find any official information about what the difference between them are. Are they just the same thing under two different names?

Question 2:

Exactly how is this being measured. Based on my understanding, the queueing of HTTP requests is implemented by each web server, right? So if I run my app service in two instances, both instance will maintain it´s own HTTP request queue. So does that mean the App Service are monitoring these values on each instance and summing them up?

From this page, I can see that Kestrel offers this metric as well. Is that what ends up being used by Azure, or is it getting measured at a different point as well?

JasonPan - MSFT 4,376 Reputation points Microsoft Vendor

2022-06-29T05:20:01.82+00:00

Hi @John K

Ryan has answered this question, and you also can check below posts, hope it will useful to you.

HTTP queue length

Requests in Application Queue

Best regards,
Jason

1 answer

Ryan Hill 26,056 Reputation points Microsoft Employee

2022-06-29T03:16:29.557+00:00

The doc you're looking for @John K is Monitor apps in Azure App Service > Understand metrics. There a table lists the definitions of the of two metrics.

Requests In Application Queue: The number of requests in the application request queue.
Http Queue Length: The average number of HTTP requests that had to sit on the queue before being fulfilled. A high or increasing HTTP Queue length is a symptom of a plan under heavy load.

Requests Queue gives you the current number of requests the app service is processing whereas the HTTP Queue Length gives the average of those requests being held in the queue on the service plan.
Please sign in to rate this answer.

1 person found this answer helpful.
John K 6 Reputation points

2022-06-29T07:12:06.513+00:00

I've already seen that page and it doesn't provide much information, but your comment adds some insight.

Regarding my other question: is HTTP Queue Length measured by the App Service Plan or is it measured by the web servers (like Kestrel) and then reported back to App Service?

Ryan Hill 26,056 Reputation points Microsoft Employee

2022-06-29T13:36:47.59+00:00

HTTP Queue Length is measured on the app service plan. I did update my answer and there's a caption right above the table in the tables under Understand Metrics.

John K 6 Reputation points

2022-06-29T19:24:45.62+00:00

Thanks! What I don't understand though is how the App Service plan can be aware of this, because the queueing itself is implemented by each web server (Kestrel in my case), or does the App Service plan also implement a queueing system upstream from the web server? And if so, how does it know when the web server won't be able to handle more requests?

Ryan Hill 26,056 Reputation points Microsoft Employee

2022-07-01T00:41:47.937+00:00

The App Service Plan is your compute resources that your app service runs on, that's how HTTP Queue length is known at the plan level. The best analogy I can think of is that the plan is IIS and the server it's running on while the app service is the site running inside IIS.

does the App Service plan also implement a queueing system upstream from the web server?

There isn't any queueing system on the app service. Queueing is handled by the front end which is a Layer 7 load balancer that forwards the request to correct web app in a round robin fashion. To read more about it, see this blog post.

John K 6 Reputation points

2022-07-01T08:49:20.817+00:00

That was a really insightful article, thanks for that! However I can't find anything about queueing in it. But I think I understand it now anyway, based on your last reply.

So have I got this right: the HTTP request queueing feature of Kestrel never comes into play, because the load balancer that sits in front of all the instances will start queueing up requests way before the instances become overwhelmed. Is that right? If so: how does the load balancer know when it's time to start putting requests in the queue rather than letting them through to the instances? Is it just based on some constant number like "maximum concurrent requests" or does it actually instrument the instances and check for CPU/memory usage and base the decision on that?

Ryan Hill 26,056 Reputation points Microsoft Employee

2022-07-08T21:09:36.48+00:00

Apologies for the delayed reply. I don't know the answers to what technically the settings/configuration of the load balancer are with regards to request queueing.

I also wouldn't say HTTP request queueing never comes into play, as the worker (what your app code sits on) is the thing effectively processing the request. The primary purpose of load balancer sitting in front is to forward requests to the correct worker as instances are either scaled up or down.

In a nutshell, think of HTTP queue length as how well your app code is getting through requests. As it grows, the application requests queue will start to grow (however, I don't know at what rate). Depending on how it grows, you will need to decide to either optimize the requests taking the longest amount of time, adding additional compute resources to the app service plan, and/or scaling out additional instances of workers.
Sign in to comment