SpringBoot app fails to start: Connection timeout when attempting to get token from IMDS endpoint

Octavian Cismasu 21 Reputation points
2020-08-07T17:52:51.817+00:00

Hi,

I am trying to deploy a SpringBoot app in a container instance (ACI) to a virtual network. The application startup fails on startup. From the error message, I concluded that the connection to the MSI endpoint fails during the initialization of the azure.keyvault... properties in the Spring environment.

The application.properties of my app bundle contain the following configuration for Azure KeyVault (with appropriate values in the '<>' tokens below):

azure.keyvault:
  enabled: true
  uri: <keyvault-uri>
  client-id: <client-id>
  tenant-id: <tenant-id>

The subnet to which the container instance is deployed has the Microsoft.KeyVault service endpoint enabled.

Now, I would expect that this configuration is enough to establish the connection to my KeyVault.

To start the container, I use the following az container create command:

az container create \
       --name aci-$APP_NAME-nonprod-cc \
       --resource-group $ACI_RES_GROUP \
       --image $ACR_LOGIN_SERVER/$APP_GROUP/$APP_NAME:$APP_VERSION \
       --assign-identity $resourceID \
       --registry-login-server $ACR_LOGIN_SERVER \
       --registry-username $(az keyvault secret show --vault-name $AKV_NAME -n $ACR_NAME-pull-usr --query value -o tsv) \
       --registry-password $(az keyvault secret show --vault-name $AKV_NAME -n $ACR_NAME-pull-pwd --query value -o tsv) \
       --vnet $(az network vnet show --resource-group $APP_RES_GROUP --name $VNET_NAME --query id -o tsv) \
       --subnet $(az network vnet subnet show --resource-group $APP_RES_GROUP --vnet-name $VNET_NAME --name $ACI_SNET_NAME --query id -o tsv) \
       --ports 8080 \
       --query ipAddress.ip

The container is created and started, but the SpringBoot app hangs for a while on startup and finally gets terminated with the following exception:

Caused by: java.lang.RuntimeException: java.net.ConnectException: Connection timed out (Connection timed out)
    at com.microsoft.azure.credentials.MSICredentials.getTokenFromIMDSEndpoint(MSICredentials.java:210) ~[azure-client-authentication-1.6.13.jar:na]
    at com.microsoft.azure.credentials.MSICredentials.getToken(MSICredentials.java:146) ~[azure-client-authentication-1.6.13.jar:na]
    at com.microsoft.azure.keyvault.spring.AzureKeyVaultMSICredential.doAuthenticate(AzureKeyVaultMSICredential.java:33) ~[azure-spring-boot-2.2.0.jar:na]
    at com.microsoft.azure.keyvault.authentication.KeyVaultCredentials.doAuthenticate(KeyVaultCredentials.java:420) ~[azure-keyvault-1.2.2.jar:1.2.2]
    at com.microsoft.azure.keyvault.authentication.KeyVaultCredentials.getAuthenticationCredentials(KeyVaultCredentials.java:224) ~[azure-keyvault-1.2.2.jar:1.2.2]
    at com.microsoft.azure.keyvault.authentication.KeyVaultCredentials.buildAuthenticatedRequest(KeyVaultCredentials.java:123) ~[azure-keyvault-1.2.2.jar:1.2.2]

If I create the container from the same image, same command except the vnet and subnet arguments, the Spring app starts OK and everything is fine.
My constraint is that I have to deploy this container instance to a virtual network.

Could anyone point to me what I might be missing and a possible reason I'm getting the exception above?

Thank you so much.

Azure Key Vault
Azure Key Vault
An Azure service that is used to manage and protect cryptographic keys and other secrets used by cloud apps and services.
1,135 questions
Azure Container Instances
Azure Container Instances
An Azure service that provides customers with a serverless container experience.
645 questions
Azure Spring Apps
Azure Spring Apps
An Azure platform as a service for running Spring Boot applications at cloud scale. Previously known as Azure Spring Cloud.
110 questions
Accepted answer
  1. prmanhas-MSFT 17,891 Reputation points Microsoft Employee
    2020-08-17T17:46:22.177+00:00

    @Octavian Cismasu Apologies for the delay in response and thank you for your patience over the matter.

    I reached out to our internal team and got the confirmation that currently you can’t use a managed identity with ACI in a VNet. This might be part of the problem.

    Same has been Documented here as well which you already mentioned in your previous comment.

    Do let me know in case of any more queries.

    Please 'Accept as answer' if it helped, so that it can help others in the community looking for help on similar topics

    Thanks

1 additional answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. SonalBK843 1 Reputation point
    2021-03-10T12:17:43.427+00:00

    I am facing exact same issue... only thing is i am not using private links... instead i am having Vnet integrated app service with service endpoints to Azure pass services like Azure keyvault, Azure Blobstore and Azure Service Bus ... But not able to connect to any of them.
    Note : I am using a spring boot application to connect.
    Any leads?

    0 comments