question

DaLi-2142 avatar image
2 Votes"
DaLi-2142 asked sd-evo answered

Azure ML real-time inference endpoint deloyment stuck - with deployment state as Transitioning for over 2 hours.

Hi,

I was deploying a real-time inference pipeline into an AKS compute in East US region today. The endpoint deployment state was stuck at Transitioning for over 2 hours and never finished and I had to delete it. A separate deployment to region East US 2 got stuck as well. I was able to deploy the same pipeline to East US the day before yesterday.

I wonder if this is likely an error related to my account/resources or a system wide issue? Did anyone else encounter the similar issue?

thanks in advance!

azure-machine-learningazure-kubernetes-serviceazure-webapps-content-deployment
· 6
5 |1600 characters needed characters left characters exceeded
Toggle Comment visibility. Current Visibility: Visible to all users

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

65821898 avatar image 65821898 ·

I am in the same situation!
problem continues for over 20 hours!!! :-(

0 Votes 0 ·
marvol avatar image marvol ·

I have the same problem. I tried to deploy to the United Kingdom - South and Germany - Midwest regions.

0 Votes 0 ·
SalilBapat-1766 avatar image SalilBapat-1766 ·

Thanks for reporting this problem. AzureML team is investigating this. Will update this thread as we figure out a resolution.

0 Votes 0 ·
gnodeb avatar image gnodeb ·

Same service can be deployed to local machine or ACI, but it stucks at Transitioning for AKS (DevTest, STANDARD_B2S, Australia-East).

0 Votes 0 ·
gnodeb avatar image gnodeb gnodeb ·

I've tried today again and it worked.

1 Vote 1 ·
AlexanderAlmengor-4493 avatar image AlexanderAlmengor-4493 ·

Hi everyone, i have the same error :(

0 Votes 0 ·

2 Answers

romungi-MSFT avatar image
1 Vote"
romungi-MSFT answered Rodrigo-2521 edited

Hello All,

We have deployed a fix now to all regions and this should be fixed. Could you please retry and let us know if there are any issues.

-Rohit

· 2
5 |1600 characters needed characters left characters exceeded
Toggle Comment visibility. Current Visibility: Visible to all users

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

marvol avatar image marvol ·

Hi, it seems to be fine and it works. I already have an endpoint in Healthy state. Thank you for the fix!

1 Vote 1 ·
DaLi-2142 avatar image DaLi-2142 ·

Rohit,

Yes, it did fix the problem for me. Thanks!

-DaLi

1 Vote 1 ·
sd-evo avatar image
0 Votes"
sd-evo answered

A "real-time endpoint deployment" that "failed" cannot be deleted

I also can't update the deployment to a working version.

I tried deleting using "Azure CLI", "Azure Portal", "Azure ML Studio" and Python SDK.

Now I'm trying to update the deployment with "bicep" but it's been running for 20 minutes now so I think it will fail again.

I also can't delete the endpoint that contains the deployment.

Now that this is day 3, my next try is to delete the entire "Azure ML"

5 |1600 characters needed characters left characters exceeded
Toggle Comment visibility. Current Visibility: Visible to all users

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.