question

CollinsNick-5124 avatar image
3 Votes"
CollinsNick-5124 asked CollinsNick-5124 commented

Unable to update service principle

I am currently unable to update my service principle for my AKS cluster. I am trying to update the credentials by following this guide: https://docs.microsoft.com/en-us/azure/aks/update-credentials

Whenever I run the az aks update-credentials step I get the following error:

Deployment failed. Correlation ID: de7d345a-009b-40f2-95c7-466b9157e481. Category: ClientError; SubCode: InvalidResourceReference; Dependency: Azure Resource Manager; OrginalError Code="InvalidResourceReference" Message="Resource /subscriptions/xxxx/resourceGroups/ICMUS-STAGING/providers/Microsoft.Network/virtualNetworks/ICMUS-STAGING referenced by resource /subscriptions/xxxx/resourceGroups/MC_ICMUS-STAGING-CLUSTER_ICMUS-STAGING-CLUSTER_WESTUS2/providers/Microsoft.Compute/virtualMachineScaleSets/aks-default-41547229-vmss was not found. Please make sure that the referenced resource exists, and that both resources are in the same region." Details=[{"code":"NotFound","message":"Resource /subscriptions/xxxx/resourceGroups/ICMUS-STAGING/providers/Microsoft.Network/virtualNetworks/ICMUS-STAGING not found."}]

I find this error message to be particularly odd

If I go into the Resource Explorer on the portal and navigate to the VMSS referenced in the error message, there is no reference to the missing virtualNetworks. This may be because I moved the VNET to a new resource group. The Resource Explorer in the portal reflects this change, but the CLI is not.

Any clue on how to fix this issue? There seems to be some disconnect with the Resource Explorer.

azure-kubernetes-serviceazure-ad-app-registrationazure-ad-enterpriseapps
· 4
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

@CollinsNick-5124 , thank you for reaching out to us. Can you please check if your AKS cluster has Private Cluster enabled? :

85690-95.png


Also, do check if your Cluster's API server address is visible in VNET Overview section or not:


85667-96.png




0 Votes 0 ·
95.png (57.2 KiB)
96.png (47.7 KiB)

Private Cluster is Not Enabled and the API server address is not in the vNet overview.

I did a little more investigation and I think I know what happened. When I moved the vNet & associated subnets to a new resource group it looks like it did not update the vnetSubnetID in the default Nodepool/agentPoolProfiles of the cluster but it did update in the VMSS.

If I go to the Resource Explorer and navigate to my cluster I can see that in properties.agentPoolProfiles for my default Nodepool/agentPoolProfiles the vnetSubnetID wasn't updated to point at the resource group that the vnet & subnet were moved to. I recreated the resource group and moved the vNet back and the cluster is in the healthy state again.

0 Votes 0 ·

@CollinsNick-5124 , if you want to change Resource Group/ vnetSubnetID, you can do so by using Managed Clusters - Create Or Update. Let me know if this helps.




0 Votes 0 ·
Show more comments
singhh-msft avatar image
0 Votes"
singhh-msft answered CollinsNick-5124 commented

@CollinsNick-5124 , thank you for the information. I got in touch with PG on this issue, you can't change AKS resources to different resource groups (as it is not supported by AKS currently). You will have to open a support ticket so the support team can fix it. Please use the steps here to raise a support ticket. If you do not have a support plan, please send an email with subject line “Attn:Harshita” to AzCommunity[at]Microsoft[dot]com referencing subscription ID and a link to this thread (for context) and we will gladly assist you further.


Please "Accept the answer" if the information helped you. This will help us and others in the community as well.

· 4
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

I appreciate the support and I will shoot you an email!

Real quick, my understanding was that vNets and Subnets are not AKS resources?

https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/move-support-resources#microsoftnetwork

This says that you're allowed to move Resource Groups for virtualnetworks. Once you associate a vNet with an AKS cluster does it then become an AKS resource?


0 Votes 0 ·

@CollinsNick-5124 , yes you are right - when you associate a subnet to AKS cluster, it becomes AKS resource. And if you move VNET, the subnet moves along with it. That is why, you see that error.

0 Votes 0 ·

@CollinsNick-5124 , if the above answer helps, can you mark it as an answer to help other community members?

0 Votes 0 ·

Yes! Marked as an answer and sent the email!

Thanks again :)

0 Votes 0 ·
CollinsNick-5124 avatar image
1 Vote"
CollinsNick-5124 answered

I also tried to use the REST API to create a new NodePool in the correct Subnet: https://docs.microsoft.com/en-us/rest/api/aks/agentpools/createorupdate#linuxosconfig

I get the following error:

 {
   "code": "NotFound",
   "message": "Failed to get a VNet: icmus-staging."
 }

This is with setting the vnetSubnetID field to the ID of the vNet that I changed resource groups


5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.