Issue with authenticating API management with Azure OpenAI services using managed identity

Question

Hello,

We are trying to connect API management services with one or multiple model deployments on Azure OpenAI services. Basically we are trying to do some version of what is shown in this repo (and this blog post).

Following the instructions, we have model deployments in multiple regions (a mix of gpt models) with multiple endpoints and multiple keys. We are trying to use API management as an unique endpoint. We are also leveraging the subscription capabilities of API management in order to handle one unique API key.

The APIs are configured in API management for each model, with the appropriate openapi specification file from the Azure reference repository. The selection of the backend service in API management is defined by a policy that seems to be working, based on the query logs. The requests are forwarded to the appropriate Azure OpenAI endpoint.

The problem is with the authentication using managed identity between API management and Azure OpenAI services. We created an user assigned identity, assigned the roles "Cognitive Services User" and "Cognitive Services OpenAI User" to that identity. This identity is then assigned to the API management instance we created. In the API management policy, we are using the "authentication-managed-identity" directive in order to retrieve an access token for that identity for the resource "https://cognitiveservices.azure.com", passing the identity's client id in order to ensure we are using Entra ID.

The resulting token seems valid but when forwarded to the Azure OpenAI service, the backend request fails with a status 400 response :

{
	"error": {
		"code": "Request is badly formated",
		"message": "Resource Id is badly formed: NA"
	}
}

We tried using that token directly with Azure OpenAI service without going through API management, and it also breaks, whereas using the "api-key" header as normal works for the same request.

Due to the lack of additional information on why the request failed, we need some help. Is there anything we forgot ? Or is there some sort of incompatibility between the managed identity and Azure OpenAI services ?

Thank you for your help

Answer 1

Solution:

The problem lies in the use of regional endpoints instead of custom endpoints as the backend url in API Management. Please ensure that when you specify the backend URL, it is not a regional endpoint but a custom endpoint.

Example:

Wrong backend url endpoint: https://<region>.api.cognitive.microsoft.com

Correct backend url endpoint: https://<custom-name>.openai.azure.com

Note:

To find the Azure OpenAI service endpoint, you can follow these steps:

1. Go to your Azure openAI service resource in the Azure portal.
2. In the Keys & Endpoint section, copy your endpoint.
3. You can also find the value in the Azure OpenAI Studio > Playground > Code View.