Angular universal Web App with SSR randomly slow (15s per request)

Remy Villain 16

Hi,

We are trying to publish an SSR (Server side rendering) Angular App on an Azure Web App running on Node 14.
Everything is working very well, pages are rendered server side nicely but sometimes we have very slow requests but without errors or logs. Just slow requests.
The problem seems to happen only when the SSR render function needs to execute external http requests.

We tried different versions of Angular, different project templates but every time SSR is slow randomly on pages with external http requests.

So pages are rendered in less than 1 second most of the time, and sometimes they are in ~15 seconds
Here a request duration distribution of a test page:

We also try many plan app service level and many external apis with no effect.
Have you got any tips to configure Azure Web App for Angular Universal in production mode ?

Thanks

Ryan Hill 25,661 Reputation points Microsoft Employee

2021-03-12T19:30:51.87+00:00

Hi @Remy Villain , are there any additional details you can provide on GET /external-api? Does it surpass 15s when it's certain type of data in the request body? Did you enable application insights on your application? The latency could be caused by performance of a downstream dependency. The Troubleshoot performance degradation doc is good place to get started in analyzing what could be causing that spike. Should you need any further assistance, feel free to comment down below.

Regards,
Ryan
S.Bushmakin 1 Reputation point

2021-03-19T08:32:50.413+00:00

Hi @Remy Villain , please tell me how deploy Angular universal on IIS?
When developing use spa.UseAngularCliServer(npmScript: "dev:ssr"); into Startup.cs
How app net.core starting SSR angular for prodaction IIS?
Remy Villain 16 Reputation points

2021-03-19T08:57:08.877+00:00

Thanks for your help,

Yes we have enabled application insights, APIs are very simple ones. Just returning some json objects in an array. It surpass 15s when there is a request body or not, a large response array or not.

Application insights reveal a very short high CPU usage on each request. No errors.

We decided to replace the "code" based web app to a docker based web app and this problem totally disappear.
I don't know why but Azure create a docker container that doesn't work well with SSR.
Creating a container ourselves solves the problem.
Remy Villain 16 Reputation points

2021-03-19T08:58:55.217+00:00

Hi,
We don't use IIS, our SSR app is running on a Node.js docker container.
S.Bushmakin 1 Reputation point

2021-03-19T10:02:56.32+00:00

@Remy Villain , your backend JS( for Node.js)?
If i use net core for backend my solution direct SSR app is running on a Node.js from NET?

Best Regards
Ryan Hill 25,661 Reputation points Microsoft Employee

2021-03-19T19:03:09.4+00:00

@Remy Villain , was this deployed to Windows or Linux? Which region?
Ryan Hill 25,661 Reputation points Microsoft Employee

2021-03-19T19:59:56.163+00:00

@S.Bushmakin you're correct. DotNetCore will spin up a node.js environment, see https://learn.microsoft.com/en-us/aspnet/core/client-side/spa/angular?view=aspnetcore-5.0&tabs=netcore-cli#drawbacks-of-ssr.

Regards,
Ryan
Remy Villain 16 Reputation points

2021-03-22T08:32:26.997+00:00

Linux, switzerland north.
Martin Christensen 1 Reputation point

2021-06-15T09:50:57.463+00:00

Hi @Remy Villain are you willing to share your dockerfile? I'm curious about which image you started from and also if you added additional stuff aside from the code.

Remy Villain 16

Hi @Martin Christensen , here my DockerFile :

FROM node:14 as buildContainer  
ARG ANGULAR_ENV  
WORKDIR /app  
COPY ClientApp/package.json ClientApp/package-lock.json ./  
RUN npm install  
RUN npm install -g @angular/cli  
COPY ClientApp .  
RUN npm run build:ssr:${ANGULAR_ENV}  
  
FROM node:14-alpine  
WORKDIR /app  
RUN npm i eventsource request ws  
COPY --from=buildContainer /app/dist/MyProject .  
  
# Expose the port the app runs in  
EXPOSE 4000  
  
# Serve the app  
CMD ["node", "/app/server/main.js"]

04821556 31 Reputation points

2021-06-29T12:02:01.907+00:00

Hi @Remy Villain ,

did you find the root cause of your problem ? I have the same issue , I am using angular SSR with node.js server by default created with angular cli. for the loading home page I have only 4 api calls which I can see that happing between 400-600ms, abut anyway it randomly takes 15s to load the page.
Remy Villain 16 Reputation points

2021-06-29T13:20:41.073+00:00

No, but with our docker image (see the dockerfile above), we have no problem with api calls
04821556 31 Reputation points

2021-06-29T14:02:35.39+00:00

I have tried with the same docker file as you posted here, but with no luck. okay, thank you anyway.
Alvin Ho 11 Reputation points

2021-07-07T20:57:10.93+00:00

@04821556 I am experiencing the same 15 second issue on one of my environments that have custom domains.

Do you have a custom domain attached to the web app? I wonder if that causes the issue.
04821556 31 Reputation points

2021-07-07T20:59:57.89+00:00

Yes. We have custom domain attached and I was wondering about that also, but no way to test that cause the app is in live and we can't play with it.
Ryan Hill 25,661 Reputation points Microsoft Employee

2021-07-08T19:43:58.81+00:00

Are you still able to reproduce this problem @04821556 ?
04821556 31 Reputation points

2021-07-08T19:45:30.943+00:00

@Ryan Hill yes, if I do not turn off API calls during server side rendering, otherwise everything is fine.
04821556 31 Reputation points

2021-07-08T19:47:14.207+00:00

Also, I have another environement were it's not reproducible at all. The only difference between two envs is custom domain and very little difference in user count
Ryan Hill 25,661 Reputation points Microsoft Employee

2021-07-08T19:47:39.71+00:00

Shoot me an email to AzCommunity[at]microsoft[dot]com @04821556 so I can work more closely with you in this matter. Use ATTN: Ryan in the subject line and include your app name and subscription id.
04821556 31 Reputation points

2021-07-08T19:59:30.973+00:00

thank you, I have sent an email.
Alvin Ho 11 Reputation points

2021-07-08T20:04:31.807+00:00

Same here. We have a QA environment with the same exact setup (but without custom domains) as the production environment with the 15 seconds issue and it isn't reproducible there.

We just ended up pointing the custom domains used in the production environment to the QA environment and it seems to be fine. Please let me know if you find out the reason or a better solution. @04821556 @Ryan Hill
julian 1 Reputation point

2021-07-08T20:37:19.07+00:00
Hi All,

I am experiencing the same problem. I have filed a support ticket with Azure support and they are escalating the case to the Azure Angular team.

Steps to create a bare bones demo Angular 12 Universal app and repro the bug:

ng new universal-demo

ng add @nguniversal/express-engine

Make any http request using HttpClientModule from AppComponent

Deploy to a an Azure App Service (Windows or Linux) running Node 14 LTS.

Keep on refreshing the app until on a random attempt you get a delay of approx 15 seconds. It is not possible to repro this bug if the http request code is removed.

Angular CLI: 12.1.1
Node: 14.17.1
Package Manager: npm 6.14.13
OS: win32 x64

Angular: 12.1.1
... animations, cli, common, compiler, compiler-cli, core, forms
... platform-browser, platform-browser-dynamic, platform-server
... router

Package Version

@angular-devkit/architect 0.1201.1
@angular-devkit/build-angular 12.1.1
@angular-devkit/core 12.1.1
@angular-devkit/schematics 12.1.1
@nguniversal/builders 12.1.0
@nguniversal/express-engine 12.1.0
@schematics/angular 12.1.1
rxjs 6.6.7
typescript 4.3.5
julian 1 Reputation point

2021-07-08T20:51:44.03+00:00
@04821556 @Ryan Hill @Alvin Ho

HI All,

I am experiencing the same problem. I have filed a support ticket with Azure support and they are escalating the case to the Azure Angular team.

Steps to create a bare bones demo Angular 12 Universal app and repro the bug:

ng new universal-demo

ng add @nguniversal/express-engine

Make any http request using HttpClientModule from AppComponent

Deploy to a an Azure App Service (Windows or Linux) running Node 14 LTS.

Keep on refreshing the app until on a random attempt you get a delay of approx 15 seconds. It is not possible to repro this bug if the http request code is removed.

Angular CLI: 12.1.1
Node: 14.17.1
Package Manager: npm 6.14.13
OS: win32 x64

Angular: 12.1.1
... animations, cli, common, compiler, compiler-cli, core, forms
... platform-browser, platform-browser-dynamic, platform-server
... router

Package Version

---------------------------------------------------------

@angular-devkit/architect 0.1201.1
@angular-devkit/build-angular 12.1.1
@angular-devkit/core 12.1.1
@angular-devkit/schematics 12.1.1
@nguniversal/builders 12.1.0
@nguniversal/express-engine 12.1.0
@schematics/angular 12.1.1
rxjs 6.6.7
typescript 4.3.5
04821556 31 Reputation points

2021-07-08T20:55:01.503+00:00

@julian Exactly the same setup (just with more functionality cause it's a real app) and the same problem. Will be glad to hear back from you if support will be able to find the issue.
julian 1 Reputation point

2021-07-08T21:06:11.127+00:00

@04821556
I'll definitely report back to this post what I learn from Azure support. This problem is blocking my dedicated servers to Azure migration path at the moment.

FYI my demo app I described has this problem without any custom domain i.e. it is running on {my-demo-app-name}.azurewebsites.net

@Ryan Hill @Alvin Ho

I
04821556 31 Reputation points

2021-07-08T21:11:37.413+00:00

@julian interesting to know that it's not related to custom domains. we were also thinking about the SNAT ports exhaustion because we had some alerts sometimes related to that. But after some testing, we found out that probably that is not the case also.
julian 1 Reputation point

2021-07-09T16:39:40.953+00:00

I have added the source code for the bare bones Angular 12 Universal app with a simple HttpClient request which randomly gets a loading delay of around 15 seconds when deployed to a Windows or Linux Azure App Service as Code.

https://github.com/juliansamuel/universal-demo
Shubham Anand 36 Reputation points

2021-08-09T08:22:40.67+00:00

Hi @Remy Villain ,

Can you check the resolution, which I have provided in the comment below & do let me know ,if it has solved the issue of slow requests (~15 seconds), which were faced by you, executing external http requests

4 answers

Shubham Anand 36 Reputation points

2021-08-09T08:12:21.563+00:00

Hi Everyone ( @04821556 , @julian ,@Remy Villain ,@Alvin Ho & others who are facing similar issue),

I have debugged the reported issue with Universal Angular Web App with SSR & making HTTP Requests where you have reported random slowness & after thorough testing, we have identified that the issue is only happening when using Codeless integration of Application insights.

The reason for same is that for Nodejs SDK for Application Insights (when using SSR in Angular, it is used by default), the value of maxBatchIntervalMs is specified as 15000ms (15 seconds) by default, if no other value is specified as shown below:

maxBatchIntervalMs: The maximum amount of time to wait to for a payload to reach maxBatchSize (Default 15000 ms or 15 seconds).

Please refer to following link for more information:

Monitor Node.js services with Azure Application Insights - Azure Monitor | Microsoft Learn

So,unless the payload size reaches maxBatchSize the http requests are blocked to be sent to telemetry end point (Azure). This causes intermittent delays.

Resolution:

The issue is resolved after specifying a custom value for maxBatchIntervalMs (example 100 ms) as shown in the code sample below:

https://github.com/shuanand/angularuniversalSSRApplicationInsight/blob/main/server.ts

To configure the same, firstly disable the Application Insights from the Azure portal of the Webapp after copying Instrumentation key from the App Insights Dashboard by clicking on Overview blade as shown below:

You can then update the instrumentation key for your application in server.ts file as shown below:

appInsights.setup("PLEASE_ENTER_YOUR_APPLICATION_INSIGHTS_INSTRUMENTATION_KEY_HERE")

Once, it’s done, there is no random delay even after using the Application Insights as shown below and all the metrics are available in Application Insights Dashboard:

Angular universal App using SSR & Http Requests on Linux App Service:

angularssrdemo.azurewebsites.net

The code is working on Windows & Container App Service too after making the mentioned changes.

Thanks to @julian for sharing the sample code to quick do a repro for the issue.

Please comment, if you are still facing the issue with sample code and I will check the same.
Please sign in to rate this answer.

6 people found this answer helpful.
04821556 31 Reputation points

2021-08-09T14:34:51.147+00:00

@Anonymous

Seems the issue is fixed with your resolution. I am monitoring and will continue, but after deployment of your fix to my env didn't receive any spikes.

Alvin Ho 11 Reputation points

2021-08-09T14:53:03.323+00:00

Thank you @Anonymous for the solution, appreciate it.

We haven't faced the issue on a different App Service instance so we've been using that instead and removed the problematic one, insights is disabled for this one, so it might be the same reason as yours. Good to know this was most likely the cause. Thanks!

Shubham Anand 36 Reputation points

2021-08-10T03:02:53.05+00:00

@04821556 , Thanks for prompt confirmation, I am glad to know that issue is now resolved at your end.

Shubham Anand 36 Reputation points

2021-08-10T03:09:10.95+00:00

@Alvin Ho

Yes Correct. That's why you did not experienced the issue in different App service instance. If you are planning to use Application Insights in current instance , you can still use the Node.js SDK integration & Azure Monitor to monitor the response times by referring to below sample & tweaking value of maxBatchIntervalMs :

https://github.com/shuanand/angularuniversalSSRApplicationInsight/blob/main/server.ts

Thanks for the confirmation!

Ryan Hill 25,661 Reputation points Microsoft Employee

2021-08-11T18:25:42.537+00:00

@04821556 @Alvin Ho can you upvote @Anonymous answer to benefit the greater community.

04821556 31 Reputation points

2021-08-11T19:16:49.767+00:00

Thank you @Anonymous , you did a great job there. Everything still looks good after your solution.

Alvin Ho 11 Reputation points

2021-08-12T14:01:05.8+00:00

done!

Ivan Smok 1 Reputation point

2022-11-02T10:23:40.95+00:00

1 year later and Angular 14.2 version, still works. Thanks a lot

Neeraj Chaurasia 1 Reputation point

2022-12-05T20:04:11.043+00:00

worked for me, thanks

Morten Felumb 0 Reputation points

2023-01-24T20:33:24.2533333+00:00

Excellent work, Shubham Anand. The important thing was to know the reason for these 15 second delays. Would never have guessed without this post!
Sign in to comment
04821556 31 Reputation points

2021-06-29T13:17:57.513+00:00

We also tried creating our Docker container, but it didn't help
Please sign in to rate this answer.

0 comments No comments
Sign in to comment
Deleted

This answer has been deleted due to a violation of our Code of Conduct. The answer was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.

Comments have been turned off. Learn more
88082004 0 Reputation points

2023-11-04T21:11:20.7933333+00:00

Hi Everyone, are you aware if do we still need this workaround ?
Please sign in to rate this answer.

0 comments No comments
Sign in to comment

Angular universal Web App with SSR randomly slow (15s per request)

Package Version

4 answers