ML online deployment InternalServerError

Question

I've been trying to deploy a ML online endpoint for inference but I'm getting the following error:

Code: None

Message: InternalServerError: Internal error. Please see troubleshooting guide, available here: https://aka.ms/oe-tsg#error-internalservererror

Exception Details: (None) InternalServerError: Internal error. Please see troubleshooting guide, available here: https://aka.ms/oe-tsg#error-internalservererror

Unfortunately there's not sufficient logs available to troubleshoot and I cannot make the deployment succeed.
Here you can see the reference deployment files:

Environment reference file

name: inference-env
channels:
  - conda-forge
dependencies:
  - python=3.11
  - pip
  - pip:
      - joblib
      - azure-identity
      - azure-keyvault-secrets
      - utils

score.py (simplified version omitting some postgres statements)

import json
import os
from azure.identity import DefaultAzureCredential
from azure.keyvault.secrets import SecretClient
from azureml.core.model import Model


def init():
    global model
    global conn

    # Load the model from the Azure ML model registry
    model_path = Model.get_model_path("my-model")

    # Retrieve database credentials from Azure Key Vault
    key_vault_name = os.environ["AZURE_KEY_VAULT_NAME"]
    kv_uri = f"https://{key_vault_name}.vault.azure.net"

    credential = DefaultAzureCredential()
    client = SecretClient(vault_url=kv_uri, credential=credential)


def run(raw_data):
    data = json.loads(raw_data)
    description = data["description"]

    return {"status": "success", "embedding": description}

Answer 1

Hello @Giuseppe , Thanks for using Microsoft Q&A Platform.

I request you to check the logs/debug the endpoint: https://learn.microsoft.com/en-us/azure/machine-learning/how-to-debug-managed-online-endpoints-visual-studio-code?view=azureml-api-2&tabs=cli

If you still face issues, with support plan, please submit a customer support ticket in the Azure portal with all the related information. If you don't have a plan, we can help you in creating a one-time free support ticket.I hope this helps.

Regards,

Vasavi

-Please kindly accept the answer and vote 'yes' if you feel helpful to support the community, thanks.