This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(240, 98%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ETV%3C/text%3E%3C/svg%3E)
We are using a private Artifactory registry for our Docker images and have added a ContainerRegistry connection to the Azure ML workspace to authenticate.
This connection is working fine to run Jobs, but when creating online endpoint deployments we get the following error message:
{"level":"[ERROR]","ts":"Jun 13 13:30:39","logger":"VMAgent.ImageFetcher","caller":"imagefetcher/image_fetcher.go:641","msg":"ctr pull image failed","imageURL":"xxx.jfrog.io/xxx/xxx:v0.1.0","error":"exit status 1","stderr":"time=\"2023-06-13T13:30:39Z\" level=info msg=\"trying next host\" error=\"failed to authorize: failed to fetch anonymous token: unexpected status: 401 \" host=xxx.jfrog.io\nctr: failed to resolve reference \"xxx.jfrog.io/xxx/xxx:v0.1.0\": failed to authorize: failed to fetch anonymous token: unexpected status: 401 \n","stacktrace":"goms.io/azureml/mir/mir-vmagent/pkg/imagefetcher.(*ctrCommandExecutor).PullImage\n\t/mnt/vss/_work/1/s/pkg/imagefetcher/image_fetcher.go:641\ngoms.io/azureml/mir/mir-vmagent/pkg/imagefetcher.(*ImageFetcher).PullAllImages.func1.1\n\t/mnt/vss/_work/1/s/pkg/imagefetcher/image_fetcher.go:271\ngoms.io/azureml/mir/mir-vmagent/pkg/common.RetryWithCondition\n\t/mnt/vss/_work/1/s/pkg/common/code_util.go:29\ngoms.io/azureml/mir/mir-vmagent/pkg/common.Retry\n\t/mnt/vss/_work/1/s/pkg/common/code_util.go:22\ngoms.io/azureml/mir/mir-vmagent/pkg/imagefetcher.(*ImageFetcher).PullAllImages.func1\n\t/mnt/vss/_work/1/s/pkg/imagefetcher/image_fetcher.go:269\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/opt/hostedtoolcache/go/1.18.5/x64/pkg/mod/golang.org/x/sync@v0.1.0/errgroup/errgroup.go:75"}
Is there a reason why the connection does not seem to work for deployments? How can this be fixed?
@Tomas Van Pottelbergh Since the connections feature is still in preview I think that this could be an issue or bug while creating online endpoints. Since the error seems to be with authentication and online endpoints need owner or contributor role for the Azure Machine Learning workspace, or a custom role allowing Microsoft.MachineLearningServices/workspaces/onlineEndpoints/* while deploying them. Do you think adding a contributor role to your connection settting properties will help?
Thanks for the answer @romungi-MSFT . I understand that the connections feature is in preview, but it is working fine for Jobs, so we'd hope it works similarly for Deployments.
I don't understand exactly what permissions I should add. The Service Principal deploying the endpoint has the Contributor role. Would that be enough or does the managed identity of the online endpoint also need certain permissions?
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(172.8, 33%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGL%3C/text%3E%3C/svg%3E)
Hi, did you manage to figure out the solution?
I'm also experiencing problems when using images from an Artifactory Docker registry and Microsoft documentation is very lacking in that regard.
Microsoft Support confirmed that this feature is not supported for endpoints, so we had to migrate to Azure Container Registry.