question

crsreddy1447 avatar image
0 Votes"
crsreddy1447 asked singhh-msft commented

AKS not able to access Keyvault using Managed identity when Autoscaling the Nodes.

Issue: We have deployed AKS from ARM template with managed identity by using default Kubnet. Auto scaling is disabled while deploying. we have deployed cluster with two node configuration.. Recently one of our Node utilisation reached 95%. We have given a Pod replica as 1. So pod is present on a single node only. To over come this problem we tried to enable autoscaling of node and kept Node min as2 and max as 5. All these autoscaling is enabled from portal. Main Issue is when we enable autoscaling the agentpool identity system assigned managed identity is getting disabled Refer pic 1. Due to which we are not able to connect with key vault.

When ever the Node gets scaled the ASK pool identity system managed is getting disabled. We are using AKS pool System Managed Identity and adding in keyvault access policy. Then we can get values from Keyvault. But when the node is auto scaling system managed identity is getting disabled. So keyvault connection is getting break. Is there any way to connect automatically even autoscales the node?

azure-kubernetes-service
· 5
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

@crsreddy1447, thank you for reaching out to us. Can you please attach "pic 1" in the question to help you further? Also, pls do share screenshots of Scale settings of your AKS cluster.

0 Votes 0 ·

89796-aksproperties.jpg89841-aks-system-assigned.jpg89823-aksscaling.jpg




Actually we are connecting the keyvault using System Managed Identity. But when enabling AKS auto scaling the System Managed Identity is getting Off. So App is not fetching Keyvault values. We have to again manually enable the System Managed Identity.

Is there any way to access keyvault using User managed Identity in AKS with using Yaml file. If you have the flow. Can you give all the steps.
Thanks

0 Votes 0 ·
aksproperties.jpg (46.2 KiB)
aksscaling.jpg (50.3 KiB)

@crsreddy1447 , just checking in to see if you got a chance to check my previous questions.

0 Votes 0 ·

@crsreddy1447 , just checking in to see if you got a chance to check my previous questions.

0 Votes 0 ·

@crsreddy1447 , just checking in to see if you got a chance to check my answer.

0 Votes 0 ·

1 Answer

singhh-msft avatar image
0 Votes"
singhh-msft answered singhh-msft commented

@crsreddy1447, thank you for reaching out to us. I tried reproducing your issue by creating a Managed Identity-enabled AKS cluster and scaling it. In this attempt, I did not face the issue that you are facing. By looking at the screenshots you have shared, we notice that you are not using AKS managed Identity for your cluster, instead you seem to have enabled Managed Identity manually in associated VMSS. Since, manual changes are not supported in AKS currently, they will be removed by any operation done on AKS.

Key summary points:

Please "Accept the answer" if the information helped you. This will help us and others in the community as well.






· 5
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

[90860-aksarmtemplate.txt][1]

In this below arm template we are using identity-type": "SystemAssigned"
[1]: /answers/storage/attachments/90860-aksarmtemplate.txt

We have used AKS managed Identity given by you
https://docs.microsoft.com/en-us/azure/aks/use-managed-identity

But still we are facing same issue.

Issue :
Aks Vmss Identity changed to SystemManaged Identity manually. When we enable auto scaling this Vmss System managed identity is getting disabled. That feature has to be same even after autoscaling is enabled.

If this is not possible. Can you tell me How to access keyvault from AKS using useridentity method.
In this scenario we have deployed AKS with identity type as "User Assigned"

We are not interested to use AAD pod and CSI Drivers method. Please help how to do Using user managed Identity. If possible can you send step by step screen shots or any video.




0 Votes 0 ·
aksarmtemplate.txt (5.3 KiB)

@crsreddy1447 , I understand your requirements, but unfortunately there are only the above ways to access Key Vault secrets from AKS. Check out this thread on similar question on Stack Overflow.

I would recommend you to refer to below articles (using Managed identity):
- CSI Driver: Tutorial: Configure and run the Azure Key Vault provider for the Secrets Store CSI driver on Kubernetes
- Pod Identity: Access KeyVault from Azure Kubernetes Service (AKS) with an ASP.NET Core application using a Managed Identity

0 Votes 0 ·

@crsreddy1447 , just checking in to see if you got a chance to look at my previous response.

0 Votes 0 ·

@crsreddy1447 , just checking in to see if you got a chance to look at my previous response.

0 Votes 0 ·

@crsreddy1447 , just checking in to see if you got a chance to look at my previous response.

0 Votes 0 ·