question

VishalSharma-7527 avatar image
0 Votes"
VishalSharma-7527 asked GitaraniSharmaMSFT-4262 commented

Azure Application Gateway: web socket connections with backend terminates due to 504 gateway timeout error

Setup: I have deployed an azure application gateway in front of my AKS cluster service.
1- App Gw shows the backend pools are healthy.
2- AKS cluster service is running a WebSocket server.
3- The App gw TCP timeout is changed from the default value of 4 to 30 minutes.
4- BE request timeout is changed from the default value of 30 seconds to 15 minutes.
Test Case:
I am trying 10,000 web socket connections with my test clients. All the 10,000 connections get connected in a few seconds.

Issue:
In a one-minute window, the connection start dropping, and the client starts complaining 504: timeout error

Note: When the same WebSocket client tried with AKS service ( WebSocket server) it connects well and also does not show any disconnection.

Could you please help me with what i am missing in the application gateway?





azure-application-gateway
· 3
5 |1600 characters needed characters left characters exceeded
Toggle Comment visibility. Current Visibility: Visible to all users

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

GitaraniSharmaMSFT-4262 avatar image GitaraniSharmaMSFT-4262 ·

Hello @VishalSharma-7527 ,

Welcome to Microsoft Q&A Platform. Thank you for reaching out & hope you are doing well.

The default timeout value for a response is 30 seconds (this is configurable via BackendHttpSetting). If the Application Gateway v1 does not receive a response in 30 seconds, the user will receive a 502 error.
The behavior for Application Gateway v2 is slightly different in timeout scenarios, and v2 will return a 504 error.
Solution : The backends configured to handle the given URL take too long to respond. Use request-timeout annotation to set a higher timeout.
Refer : https://github.com/Azure/application-gateway-kubernetes-ingress/blob/master/docs/annotations.md#request-timeout

But you mentioned that you have already increased the request time-out value in Application gateway and the backend.
In this case, we may need to look into the backend logs and would need access to your resources for further investigation. Hence, if you have a support plan, I request you file a support ticket, else please do let us know, we will try and help you get a one-time free technical support.

Regards,
Gita


0 Votes 0 ·
GitaraniSharmaMSFT-4262 avatar image GitaraniSharmaMSFT-4262 ·

Hello @VishalSharma-7527 ,

I'm following up on my above comment. Could you please let us know if you are able to file a support ticket, else please do let us know, we will try and help you get a one-time free technical support?

Regards,
Gita

0 Votes 0 ·
GitaraniSharmaMSFT-4262 avatar image GitaraniSharmaMSFT-4262 ·

Hello @VishalSharma-7527 , do you have any updates on this issue?

0 Votes 0 ·

0 Answers