Tutorial: Access resources with managed online endpoints and identity (preview)

In this tutorial, you learn how to securely access Azure resources from your scoring script with a managed online endpoint and a system-assigned managed identity.

Important

This feature is currently in public preview. This preview version is provided without a service-level agreement, and it's not recommended for production workloads. Certain features might not be supported or might have constrained capabilities. For more information, see Supplemental Terms of Use for Microsoft Azure Previews.

This tutorial demonstrates how to take the following actions with the Azure CLI and its ML extension:

  • Set the default values for the Azure CLI to use
  • Configure the variables to be used with your managed online endpoint
  • Create a blob storage account and blob container
  • Create a managed online endpoint
  • Give the system assigned managed identity permission to access storage
  • Create a deployment associated with managed endpoint

Prerequisites

  • To use Azure machine learning, you must have an Azure subscription. If you don't have an Azure subscription, create a free account before you begin. Try the free or paid version of Azure Machine Learning today.

  • You must install and configure the Azure CLI and ML extension. For more information, see Install, set up, and use the CLI (v2) (preview).

  • You must have an Azure Resource group, in which you (or the service principal you use) need to have Contributor access. You'll have such a resource group if you configured your ML extension per the above article.

  • You must have an Azure Machine Learning workspace. You'll have such a workspace if you configured your ML extension per the above article.

  • A trained machine learning model ready for scoring and deployment.

Set the defaults for Azure CLI

To ensure the correct resources are used throughout this tutorial, set the default values for the Azure subscription ID, Azure Machine Learning workspace, and resource group you want to use. Doing so allows you to avoid having to repeatedly pass in the values every time you call an Azure CLI command.

Important

Ensure your user account has "User Access Administrator" role assigned to resource group.

az account set --subscription <subscription id>
az configure --defaults workspace=<azureml workspace name> group=<resource group>

Define the configuration YAML file for your deployment

To deploy a managed endpoint, you first need to define the configuration for your endpoint in a YAML file.

The following code example creates a managed endpoint that,

  • Shows the YAML files from endpoints/online/managed/managed-identities/ directory.
  • Defines the name by which you want to refer to the endpoint, my-sai-endpoint
  • Specifies the type of authorization to use to access the endpoint, auth-mode: key
  • Specifies that the type of endpoint you want to create is an online endpoint.
  • Indicates that the endpoint has an associated deployment called blue
  • Configures the details of the deployment such as, which model to deploy and which environment and scoring script to use.
# yaml-language-server: $schema=https://azuremlsdk2.blob.core.windows.net/latest/managedOnlineEndpoint.schema.json
name: my-sai-endpoint
type: online
auth_mode: key
traffic:
  blue: 100

deployments:
  - name: blue
    model:
      name: model-1
      version: 1
      local_path: ../../model-1/model/sklearn_regression_model.pkl
    code_configuration:
      code: 
        local_path: ../../model-1/onlinescoring/
      scoring_script: score_managedidentity.py
    environment: 
      name: env-model1
      version: 1
      path: .
      conda_file: file:../../model-1/environment/conda.yml
      docker:
        image: mcr.microsoft.com/azureml/openmpi3.1.2-ubuntu18.04:20210727.v1
    instance_type: Standard_F2s_v2
    scale_settings:
      scale_type: manual
      instance_count: 1
      min_instances: 1
      max_instances: 2
    environment_variables:
      STORAGE_ACCOUNT_NAME: "storage_place_holder"
      STORAGE_CONTAINER_NAME: "container_place_holder"
      FILE_NAME: "file_place_holder"

For a reference to the YAML, see Managed online endpoints (preview) YAML reference.

Configure variables for your deployment

Configure the variable names for the workspace, workspace location, and the endpoint you want to create. The following code exports these values as environment variables in your endpoint:

export WORKSPACE="<WORKSPACE_NAME>"
export LOCATION="<WORKSPACE_LOCATION>"
export ENDPOINT_NAME="<ENDPOINT_NAME>"

Next, specify what you want to name your blob storage account, blob container, and file. These variable names are defined here, and are referred to in az storage account create and az storage container create commands in the next section.

The following code exports those values as environment variables:

export STORAGE_ACCOUNT_NAME="<BLOB_STORAGE_TO_ACCESS>"
export STORAGE_CONTAINER_NAME="<CONTAINER_TO_ACCESS>"
export FILE_NAME="<FILE_TO_ACCESS>"

After these variables are exported, create a text file locally. When the endpoint is deployed, the scoring script will access this text file using the system assigned managed identity that's generated upon endpoint creation.

Create blob storage and container

For this example, you create a blob storage account and blob container, and then upload the previously created text file to the blob container.

First, create a storage account.

az storage account create --name $STORAGE_ACCOUNT_NAME --location $LOCATION

Next, create the blob container in storage account.

az storage container create --account-name $STORAGE_ACCOUNT_NAME --name $STORAGE_CONTAINER_NAME

Then, upload your text file to the blob container.

az storage blob upload --account-name $STORAGE_ACCOUNT_NAME --container-name $STORAGE_CONTAINER_NAME --name $FILE_NAME --file endpoints/online/managed/managed-identities/hello.txt

Create a managed online endpoint

The following code creates a managed online endpoint without specifying a deployment. Deployment creation is done later in the tutorial.

When you create a managed endpoint, a system-assigned managed identity is created for the endpoint by default.

Important

System assigned managed identities are immutable and can't be changed once created.

az ml endpoint create --name $ENDPOINT_NAME -f endpoints/online/managed/managed-identities/1-sai-create-endpoint.yml

Check the status of the endpoint with the following.

az ml endpoint show --name $ENDPOINT_NAME

If you encounter any issues, see Troubleshooting managed online endpoints deployment and scoring (preview).

Give storage permission to system-assigned managed identity

You can allow the managed endpoint permission to access your storage via its system assigned managed identity.

Retrieve the system- assigned managed identity was created for your endpoint.

system_identity=`az ml endpoint show --name $ENDPOINT_NAME --query "identity.principal_id" -o tsv`

From here, you can give the system-assigned managed identity permission to access your storage.

az role assignment create --assignee $system_identity --role "Storage Blob Data Reader" --scope $storage_id

Scoring script to access Azure resource

Refer to the following scoring script, to understand how to use system-assigned managed identity token to access Azure resources. In this scenario, the Azure resource is the storage account created in previous sections.

import os
import logging
import json
import numpy
import joblib
import requests


def get_token():
    access_token = None
    msi_endpoint = os.environ.get("MSI_ENDPOINT", None)
    msi_secret = os.environ.get("MSI_SECRET", None)

    # If UAI_CLIENT_ID is provided then assume that endpoint was created with user assigned identity,
    # # otherwise system assigned identity deployment.
    client_id = os.environ.get("UAI_CLIENT_ID", None)
    if client_id is not None:
        token_url = (
            msi_endpoint + f"?clientid={client_id}&resource=https://storage.azure.com/"
        )
    else:
        token_url = msi_endpoint + f"?resource=https://storage.azure.com/"

    logging.info("Trying to get identity token...")
    headers = {"secret": msi_secret, "Metadata": "true"}
    resp = requests.get(token_url, headers=headers)
    resp.raise_for_status()
    access_token = resp.json()["access_token"]
    logging.info("Retrieved token successfully.")
    return access_token


def access_blob_storage():
    logging.info("Trying to access blob storage...")
    storage_account = os.environ.get("STORAGE_ACCOUNT_NAME")
    storage_container = os.environ.get("STORAGE_CONTAINER_NAME")
    file_name = os.environ.get("FILE_NAME")
    logging.info(
        f"storage_account: {storage_account}, container: {storage_container}, filename: {file_name}"
    )
    token = get_token()

    blob_url = f"https://{storage_account}.blob.core.windows.net/{storage_container}/{file_name}?api-version=2019-04-01"
    auth_headers = {
        "Authorization": f"Bearer {token}",
        "x-ms-blob-type": "BlockBlob",
        "x-ms-version": "2019-02-02",
    }
    resp = requests.get(blob_url, headers=auth_headers)
    resp.raise_for_status()
    logging.info(f"Blob containts: {resp.text}")


def init():
    global model
    # AZUREML_MODEL_DIR is an environment variable created during deployment.
    # It is the path to the model folder (./azureml-models/$MODEL_NAME/$VERSION)
    # For multiple models, it points to the folder containing all deployed models (./azureml-models)
    model_path = os.path.join(
        os.getenv("AZUREML_MODEL_DIR"), "sklearn_regression_model.pkl"
    )
    # deserialize the model file back into a sklearn model
    model = joblib.load(model_path)
    logging.info("Model loaded")

    # Access Azure resource (Blob storage) using system assigned identity token
    access_blob_storage()

    logging.info("Init complete")


# note you can pass in multiple rows for scoring
def run(raw_data):
    logging.info("Request received")
    data = json.loads(raw_data)["data"]
    data = numpy.array(data)
    result = model.predict(data)
    logging.info("Request processed")
    return result.tolist()

Create a deployment using your configuration

Create a deployment that's associated with the managed endpoint.

This deployment can take approximately 8-14 minutes depending on whether the underlying environment/image is being built for the first time. Subsequent deployments using the same environment will go quicker.

az ml endpoint update --name $ENDPOINT_NAME --deployment blue --file endpoints/online/managed/managed-identities/2-sai-deployment.yml --set deployments[0].environment_variables.STORAGE_ACCOUNT_NAME=$STORAGE_ACCOUNT_NAME deployments[0].environment_variables.STORAGE_CONTAINER_NAME=$STORAGE_CONTAINER_NAME deployments[0].environment_variables.FILE_NAME=$FILE_NAME

Check the status of the deployment.

az ml endpoint show --name $ENDPOINT_NAME

Note

The init method in the scoring script reads the file from your storage account using the system assigned managed identity token.

To check the init method output, see the deployment log with the following code.

# Check deployment logs to confirm blob storage file contents read operation success.
az ml endpoint get-logs --name $ENDPOINT_NAME --deployment blue

Once this command completes, you will have registered the model, the environment, and the endpoint in your Azure Machine Learning workspace.

Confirm your endpoint deployed successfully

Once your endpoint is deployed, confirm its operation. Details of inferencing vary from model to model. For this tutorial, the JSON query parameters look like:

{"data": [
    [1,2,3,4,5,6,7,8,9,10], 
    [10,9,8,7,6,5,4,3,2,1]
]}

To call your endpoint, run:

az ml endpoint invoke --name $ENDPOINT_NAME --request-file endpoints/online/model-1/sample-request.json

Delete the endpoint and storage account

If you don't plan to continue using the deployed endpoint and storage, delete them to reduce costs. When you delete the endpoint, all of its associated deployments are deleted as well.

az ml endpoint delete --name $ENDPOINT_NAME --yes
az storage account delete --name $STORAGE_ACCOUNT_NAME --yes

Next steps

In this Azure Machine Learning tutorial, you used the machine learning CLI for the following tasks:

  • Set the default values for the Azure CLI to use
  • Configure the variables to be used with your endpoint
  • Create a blob storage account and Blob container
  • Create a managed endpoint
  • Give the system assigned managed identity permission to access storage
  • Create a deployment associated with managed endpoint