question

Alastair-6698 avatar image
3 Votes"
Alastair-6698 asked Alastair-6698 commented

Intermittant issue with app service slot autoswaps and warmup

I am experiencing an intermittent issue with app service deployment slots and warm-ups.

Every day we create a new app service plan, app service, and a deployment slot in North Europe region. The slot is configured with the auto swap feature enabled. Although we have not changed our code, a couple of weeks ago we started experiencing an intermittent issue suggesting something may have changed within app service as it was working reliably beforehand.

We create the infrastructure and then deploy to the slot using zip deploy. Autoswap starts and we poll the status API. Occasionally like today we are given an app service plan where all app services will fail to slot swap because the swap warmup always fails no matter what we do. The slot is configured with app settings WEBSITE_SWAP_WARMUP_PING_PATH = "/api/healthcheck" and WEBSITE_SWAP_WARMUP_PING_STATUSES="200" and even though that API is returning a 200 code the warm up always fails for all app services on that plan.

The error is
"Failed swapping site. Error: Cannot swap slots for site XXX because the application initialization in 'XXX' slot either took too long or failed. Please check AppInit module configuration or try using swap with preview if application initialization time is very long. "

However, we are certain the warm up returned a 200 status and no sites on that plan will succeed for a slot swap if WEBSITE_SWAP_WARMUP_PING_PATH is configured. If you remove the WEBSITE_SWAP_WARMUP_PING_PATH setting then the slot swap will succeed, (but the app does not get warmed up!)

The only workaround is to destroy the app service plan and recreate everything, whereby usually the deploy and slot swap with warmup will start working again.

Is it possible there is a bug in app service?

azure-webappsazure-webapps-development
· 5
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @Alastair-6698 I noticed you created a support ticket. I will monitor but I do want to inform you that you don't need to delete the app service plan when performing a slot swap. Doing so removes your web worker, the underlying hardware, of your app service. Deleting the resource will also make it more difficult to troubleshoot.

1 Vote 1 ·

Thanks @ryanchill - yes have filed support ticket. Reason for deleting of app service plan as it is an easy way for us to get a fresh new instance which does not have the bug that fails the slot swaps

0 Votes 0 ·

I'm attempting to setup a warmup and I'm running into the same error.

Based on our app insights logs, I believe it's failing because it's trying to hit "/", opposed to the actual value we've specified in the config.

0 Votes 0 ·

I'm still seeing the same issue described by OP.
Has the fix mentioned by @Alistair-6698 been rolled out to West Europe do you know?

0 Votes 0 ·

We were instructed to check the platform version to verify. Go to the app service WebApp Kudu Console > Environment and check the value of variable WEBSITE_PLATFORM_VERSION. 92 or later is supposed to have the fix for the slot swap issue in it.

0 Votes 0 ·
Alastair-6698 avatar image
1 Vote"
Alastair-6698 answered Alastair-6698 commented

This has been acknowledged as a bug in App Service by MS support and "will be rolled out in next release cycle (tentatively)."

· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Latests from our MS support engineer three months later:

As per the latest update we have received Engineering team, they are diligently working on the fix. Once we have the update on the fix deployment or any further updates I will keep you posted.

0 Votes 0 ·

Today's update from support:

As per the recent update from Engineering team the fix will deployed during next platform upgrade release and the current ETA for the platform upgrade is February, 2021(Tentatively).

0 Votes 0 ·

MS confirmed the fix for this has been rolled out to our region (North Europe) and it seems to have fixed the issue.

1 Vote 1 ·
EmanueleDeCupis-8666 avatar image
0 Votes"
EmanueleDeCupis-8666 answered

Hi all, any update on this bug? I'm experiencing the same, too.

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

JDB-6115 avatar image
0 Votes"
JDB-6115 answered

I suddenly experience it today.

 Failed to complete swap between slot 'Bridge' and slot 'production'. Error: {"_body":"{\"Code\":\"BadRequest\",\"Message\":\"Cannot swap slots for site 'qompro' because the application initialization in 'Bridge' slot either took too long or failed. Please check AppInit module configuration or try using swap with preview if application initialization time is very long.\",\"Target\":null,\"Details\":[{\"Message\":\"Cannot swap slots for site 'qompro' because the application initialization in 'Bridge' slot either took too long or failed. Please check AppInit module configuration or try using swap with preview if application initialization time is very long.\"},{\"Code\":\"BadRequest\"},{\"ErrorEntity\":{\"ExtendedCode\":\"04903\",\"MessageTemplate\":\"Cannot swap slots for site '{0}' because the application initialization in '{1}' slot either took too long or failed. Please check AppInit module configuration or try using swap with preview if application initialization time is very long.\",\"Parameters\":[],\"InnerErrors\":[],\"Code\":\"BadRequest\",\"Message\":\"Cannot swap slots for site 'qompro' because the application initialization in 'Bridge' slot either took too long or failed. Please check AppInit module configuration or try using swap with preview if application initialization time is very long.\"}}],\"Innererror\":null}","status":400,"ok":false,"statusText":"OK","headers":{"cache-control":["no-cache"],"content-length":["1217"],"content-type":["application/json; charset=utf-8"],"date":["Mon"," 23 Nov 2020 07:55:23 GMT"],"expires":["-1"],"pragma":["no-cache"],"server":["Microsoft-IIS/10.0"],"strict-transport-security":["max-age=31536000; includeSubDomains"],"x-aspnet-version":["4.0.30319"],"x-ms-correlation-request-id":["fa54dc51-8ea4-4a44-a0e7-8ce6d7851695"],"x-ms-ratelimit-remaining-subscription-reads":["11952"],"x-ms-request-id":["fa54dc51-8ea4-4a44-a0e7-8ce6d7851695"],"x-ms-routing-request-id":["UAENORTH:20201123T075523Z:fa54dc51-8ea4-4a44-a0e7-8ce6d7851695"],"x-powered-by":["ASP.NET"]},"type":2,"url":"https://management.azure.com/subscriptions/80ca208d-8a79-4774-8633-6bd5d5e9febe/resourceGroups/sic/providers/Microsoft.Web/sites/qompro/slots/Bridge/operationresults/bf7a15dc-0bd1-4125-a8c6-cc29e395fca3?api-version=2018-11-01"}
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

MarkBowler-7155 avatar image
0 Votes"
MarkBowler-7155 answered

We get the same problem when using the WARMUP_PING_PATH, the slot swap intermittently fails with same error. In our case, repeating the slot swap normally succeeds.

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.