question

jamieg avatar image
1 Vote"
jamieg asked RutgerStorm answered

Azure App Service LInux temp storage 24gB

We are currently running into a problem in our Linux App Service resources.


The first thing that we have noticed is that for a brand new, near empty WebAPI deployment – into a fresh WebApp, App Service and Resource group – we have an immediate temp storage usage of +/- 22GB. When we compare this with Windows App Services (+/- 200MB) this seems excessive. Is this normal, or do we have an anomaly?

azure-webapps-performance
· 7
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

@Jamieg

According to your description, your issue is not in Teams for Linux scope. So I will remove teams-for-linux tag from your thread. Thanks for your understanding and patience!


1 Vote 1 ·

Hi @jamieg,

This is abnormal. What web stack are you using? What are you deploying? How are you deploying?

Regards,
Ryan

0 Votes 0 ·

Hi @ryanchill,

Stack .Net
Major version .Net Core 3
Minor version .Net Core 3.1 (LTS)

We are deploying a small app service.

Deployment is done via Azure Devops, but the same happens if we directly deploy from Visual studio.

Regards,
Jamie

0 Votes 0 ·

What region are you deploying to? I'll try to repo to see if I get the same result.

0 Votes 0 ·

In west Europe we are deploying this.

0 Votes 0 ·

Hi @jamieg,

Sorry for the late reply. I tried to repro but my app measured up to 35.21MiB (pictured below) on an S1 plan. When you browse the temp storage, what folder is showing a lot of space? What do the contents look like?

86461-image.png


0 Votes 0 ·
image.png (310.5 KiB)

Hi @ryanchill,

Thanks for your reply, sorry for my late reply.

We are looking in different metrics. This is at the best practices of the webapp. The metrics you showed are indeed almost the same, and the issue is not there.

If we go to the app service, then the 'Diagnose and solve problems'. In the app diagnostic you can search for 'Best Practices for Availability & Performance'. In here is an item 'Temp File Usage On Workers'. This metric shows this huge amount of storage while the webapp is just started. We can not see this metrics anywhere else, but after a while this starts growing. Do you have any idea?

87229-image024.png


2 Votes 2 ·
image024.png (60.5 KiB)
ryanchill avatar image
0 Votes"
ryanchill answered ryanchill converted comment to answer

Hi @jamieg,

I've been able to confirm that the high temp file usage is expected on Linux app service plans. It contains the blessed docker images of the various supported runtimes.

The Linux - Host Disk Space Usage elaborates on this a bit further and provides insights on what to do should your usage increase.

Regards,
Ryan

· 5
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @ryanchill ,

Okay thanks for looking into, and where can we find the Linux - Host Disk Space Usage?

And is it normal that this storage grows?

We had an issue with one of our app services, that this Disk space was filling up without any usage.

Regards,
Jamie

0 Votes 0 ·

It's under Diagnose and solve problems blade.

88279-image.png

You will see two different lines

  • /mnt/data-diskUsage

  • /mnt/data/docker/images-diskUsage

/mnt/data/docker/images is the default docker storage mount for your docker images and can increase as you add more apps under the app service plan. /mnt/data is used for monitoring and shouldn't be an impact. If you do happen to see that usage steadily increasing, let me know down below.




0 Votes 0 ·
image.png (129.4 KiB)

Hi @ryanchill

We ran into the problem again today. It seems "/mnt/data/docker/images-diskUsage" is growing. Any ideas how we can find out why? We haven't added any app services under the serviceplan. What we do in our CD process is we first deploy to a blue slot and then swap. Can that have anything to do with this?

And here are the screenshots of the app service.

89478-image.png


89548-image.png


0 Votes 0 ·
image.png (94.4 KiB)
image.png (100.2 KiB)
Show more comments
RutgerStorm avatar image
1 Vote"
RutgerStorm answered

Hi @LukaszAndrzejak-5527 we ended up working together with Microsoft/Azure Support in identifying what caused the temp files with us to fill up. In the end for us it was the ApplicationInsights profiler which a developer added and forgot to remove when debugging a problem in the past. This caused the temp folder to fill up with dumps etc. We also tuned our app service plans better so they have a little bit more breathing room.

· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.