Azure Container Instance stuck in "Waiting" state.

Question

Hello - We have started noticing an issue where are ACIs are stuck in "Waiting" state. We have started noticing this post April 26th. We create number of containers in an hour and almost all of them are stuck in "Waiting" state. We create the containers programmatically and the complete architecture was working fine till last week. Our application is heavily dependent on Azure Containers and this is affecting us pretty badly.

I read other similar issues and tried following recommendations. First I tried stopping and restarting containers(which in our case is not very feasible) and 1 out of 10 times it works and starts the ACI. Later I tried retrieving container logs via Azure CLI using -

az container logs --resource-group myResourceGroup --name mycontainer

And the output was null as there were no events. The only information I could grab was from the JSON View on Azure Portal -

"osType": "Windows",
        "instanceView": {
            "events": [
                {
                    "count": 1,
                    "firstTimestamp": "2024-04-29T22:50:57.2464356Z",
                    "lastTimestamp": "2024-04-29T22:50:57.2464356Z",
                    "name": "DeploymentTimeout",
                    "message": "Subscription deployment didn't reach a successful provisioning state after '00:30:00'.",
                    "type": "Warning"
                }
            ],
            "state": "Running"
        }

We prefer to create our ACIs in region - South Central US. I would highly appreciate if someone from Support team help us at earliest and provide some direction to resolve this issue. Thank you.

Answer 1

Hello Arpan Mehetre,

Welcome to microsoft Q&A , Thankyou for posting your query here.

Verify if there are any ongoing issues with Azure services, especially in the South Central US region where you deploy your ACIs.

Azure Service Health will provide updates on outages or other issues that might be affecting container services.

Ensure that you haven't hit any limits or quotas that could be affecting the deployment of new containers like number of ACIs, CPU, memory resources, and network resources in your subscription.

For instance, Check that your ACIs aren't waiting on any external resources that aren't available.

check if there were any changes to your Azure environment, application code, or related dependencies around that time.

If an answer has been helpful, please consider accepting the answer to help increase visibility of this question for other members of the Microsoft Q&A community. If not, please let us know what is still needed in the comments so the question can be answered. Thank you for helping to improve Microsoft Q&A!

Answer 2

So turns out it was an issue from Azure side. Our subscription is support enabled and we opened a support ticket with Microsoft; Initially we got to know that due to some bug in Azure backend it is unable to assign a node when the container instance spins up. Not sure what they have done from the backend but I see our ACIs are back to normal since yesterday.

In case if anyone facing this issue does not have a support enabled subscription then send an email to AzCommunity@Microsoft.com along with your subscription id and Microsoft will enable a one time Technical Support.