question

balazsorban avatar image
0 Votes"
balazsorban asked FranzOmbler-4425 commented

Azure CDN should respect origin cache-control

So we have a Next.js (https://nextjs.org/) and in front of it sits an Azure CDN service.

In the past, I have shot myself in the foot by assuming that the Azure CDN service will follow the cache-control instructions from the origin server, and not cache anything unless it is told to. I accidentally cached my credentials at the CDN, meaning anyone logging in used my credentials on our site. More info on this here: https://github.com/vercel/next.js/discussions/14136, but in short, the main problem was that the origin (Next.js) server sent a request through the CDN WITHOUT a cache-control header, but Azure CDN decided to cache it anyway.

This is a past problem that I resolved by creating a rule in the rules engine, that tells the CDN to bypass caching for any request under a certain path. (/api https://nextjs.org/docs/api-routes/introduction).

In my opinion, this is flawed default behavior of the CDN service, and there should be a configurable option to tell the CDN service "if cache-control headers are present in the origin response, use them. if not, just forward the response".

The CDN should be configurable to not make assumptions on what should and should not be cached but strictly follow cache-control header instruction by the origin server. As far as I can tell, it is currently not possible to act upon a response from the origin server in the rules engine, you can only respond to incoming requests from the user.

Am I missing something?

azure-cdn
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

SaiKishor-MSFT avatar image
0 Votes"
SaiKishor-MSFT answered SaiKishor-MSFT commented

@balazsorban Thank you for reaching out to Microsoft Q&A. We apologize for the delay in response to your issue.

I understand that you are looking for CDN to be able to honor the cache control header instruction by the origin server. This should be possible with all Azure CDN products except for Verizon DSA and Akamai DSA, please refer to this document for more details.

"Similar to how caching is implemented in a web browser, you can control how caching is performed in a CDN by sending cache-directive headers. Cache-directive headers are HTTP headers, which are typically added by the origin server. Although most of these headers were originally designed to address caching in client browsers, they are now also used by all intermediate caches, such as CDNs." As given here in this document.

Please let us know if you have any further questions and we will be glad to assist you further. Thank you!

Remember:

Please accept an answer if correct. Original posters help the community find answers faster by identifying the correct answer. Here is how.

Want a reminder to come back and check responses? Here is how to subscribe to a notification.



· 12
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Thank you for answering! So far it's the best answer I have gotten, but still needs a bit more help. Next.js sets the s-maxage cache control header, as (according to my understanding) it is the one type meant for shared/public caches (like a CDN) as opposed to the documentation you linked to, where I am supposed to use maxage, which is meant for private (eg. users' browsers) caches. It feels like Azure should be supporting s-maxage here. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control#expiration

Also you mention that honoring the origin server cache control should be available for most of the tiers. We have "Microsoft" but unfortunately assets without cache control headers from the origin are still getting cached! Any configuration I am unaware of?

0 Votes 0 ·

Please refer to this discussion for more information https://github.com/vercel/next.js/discussions/14136#discussioncomment-25630.

Putting an alternative CDN in front of the origin worked as expected, but wish to use Azure if possible!

0 Votes 0 ·

I attach a picture of the options I get when adding a condition in Rules Engine.

81251-image.png

I also attach a picture that is showing what pricing tier our CDN profile is set to.
81169-image.png


Still, I have no clue how I should configure the CDN to honor the origin server. Which again, I am pretty sure should be the default behavior!

0 Votes 0 ·
image.png (10.9 KiB)
image.png (2.4 KiB)
Show more comments
SaiKishor-MSFT avatar image
0 Votes"
SaiKishor-MSFT answered SaiKishor-MSFT commented

@balazsorban Thank you for your patience while we were investigating this issue further.

So to summarize, you want Azure CDN to not cache traffic whenever the cache directive headers are missing in the response. However, upon further investigation, this is not possible with Azure CDN i.e., you cannot set up rules engine rules based on response headers. Absence of Cache-control header leads to CDN caching the traffic.

The only solution possible through Azure CDN is to know which specific url paths can end up in this situation and set up rules accordingly in the rules engine.

Regarding s-maxage, this is still true:

Azure CDN will give preference to s-maxage over maxage provided both are non-zero values. Having maxage=0 or s-maxage=0 will make the resource uncacheable.

And we are working via the Github thread that you created to include this in our documentation as well. Hope this dissolves your doubts.

If you still have further questions, please feel free to let us know and we will gladly discuss further. Thank you so much for your patience.

Remember:

Please accept an answer if correct. Original posters help the community find answers faster by identifying the correct answer. Here is how.

Want a reminder to come back and check responses? Here is how to subscribe to a notification.




· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Thank you for investigating. Unfortunately I cannot mark anything here as an answer because apparently there is none for my problem. Where can I create a feature request for this issue? Because it is definitely something a modern CDN should consider in my opinion. I don't see the proper reasoning behind default caching without no instructions. As mentioned earlier this was a big foot gun where I ended up exposing sensitive information to the world by caching something was never meant to be cached (therefore I never instructed the CDN to do so, but it did it anyway)

0 Votes 0 ·

@balazsorban I understand the issues you are running into with the current setup, however, the solutions are only as suggested at the moment. Here is the page for Feedback- https://feedback.azure.com/forums/34192--general-feedback

0 Votes 0 ·
balazsorban avatar image
0 Votes"
balazsorban answered FranzOmbler-4425 commented

@SaiKishor-MSFT I do have a follow-up question though! I was wondering what the support state of stale-while-revalidate cache directive was? From my results, it really looks like it is ignored/unsupported as well, and my research can confirm that: https://www.ctrl.blog/entry/cdn-rfc5861-support.html

Here is the spec: https://tools.ietf.org/html/rfc5861#section-3

The part I am missing is the async nature of revalidating a resource in the CDN cache, meaning if the s-maxage value indicates expired content, STILL serve a stale version to the user, and re-fetch the resource from the origin WITHOUT blocking the user response.

I tried to use it in the above-mentioned app, and it looks like when there is a TCP_MISS, the client will WAIT for the origin to respond to the CDN even when this directive is present, instead of receiving a stale version of the content and the CDN updating the content in the background, without blocking for the user.

Is this possible/supported/planned?


· 8
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

@balazsorban Currently we do not support stale and re validate feature and there are no plans to support it yet. Please let me know if you have any further questions. Thank you!

0 Votes 0 ·

Sorry to hear that Azure CDN lags behind on this and has no plan to change. It is unfortunately a dealbreaker for us. Thank you for your time, but I now have to look for better alternatives than Azure CDN.

0 Votes 0 ·

@balazsorban I understand that your requirements does not meet the Azure CDN available features yet. Please feel free to add on to the feature requests in the Azure Feature Request Page so we our team can consider this in the future. Thank you!


0 Votes 0 ·
Show more comments

Thank you for asking this question, Balazs. You might like to upvote this feature request:
https://feedback.azure.com/d365community/idea/1b9e3901-9fa1-ec11-a81c-6045bd7d1bee

0 Votes 0 ·