Gridwich Media Services setup and scaling

Functions
Key Vault
Media Services

Gridwich uses the Azure Media Services Platform as a Service (PaaS) for media processing. Depending on the type of operation, the Gridwich application uses one of two methods to access Azure Media Services.

Azure Media Services V2

To perform the encoding of sprite sheets, or to create thumbnails during media processing, Gridwich uses the Azure Media Services V2 API via REST.

The MediaServicesV2EncodeCreateHandler initiates work by calling the MediaServicesV2RestEncodeService, which in turn uses the MediaServicesV2RestWrapper.

Within the MediaServicesV2RestWrapper, the function ConfigureRestClient sets up authentication via an Azure.Core.TokenCredential object:

var amsAccessToken = _tokenCredential.GetToken(
    new TokenRequestContext(
        scopes: new[] { "https://rest.media.azure.net/.default" },
        parentRequestId: null),
    default);

This code presents the identity of the TokenCredential and requests authorization at the REST API scope.

When running locally, the TokenCredential prompts the developer to sign in. That identity is then presented when requesting access to the scope. For successful authentication, the developer must be a contributor on the resource, and the correct environment variables must be in the local settings file.

Use the Terraform file functions/main.tf to configure a system-assigned managed identity for the Azure Functions App, with:

resource "azurerm_function_app" "fxn" {
  name                       = format("%s-%s-fxn-%s", var.appname, var.domainprefix, var.environment)
  location                   = var.location
  resource_group_name        = var.resource_group_name
  app_service_plan_id        = azurerm_app_service_plan.fxnapp.id
  storage_account_name       = azurerm_storage_account.fxnstor.name
  storage_account_access_key = azurerm_storage_account.fxnstor.primary_access_key
  version                    = "~3"
  https_only                 = true

  identity {
    type = "SystemAssigned"
  }

  lifecycle {
    ignore_changes = [
      app_settings
    ]
  }
}

Use the Terraform bashscriptgenerator/templates/ams_sp.sh script to authorize the Azure Functions service principal on the Azure Media Services account:

for id in ${mediaServicesAccountResourceId}
{
    echo "Granting fxn access to $id"
    az role assignment create --role "Contributor" --assignee-object-id ${functionPrincipalId} --scope $id
}

Azure Media Services V3

The Azure Media Services V3 SDK doesn't support managed identity. Instead, the ams_sp.sh script creates an explicit service principal to use with the Media Services V3 SDK, by using the az ams account sp create command:

# Ref: https://docs.microsoft.com/azure/media-services/latest/access-api-cli-how-to

echo 'Creating service principal for Azure Media Services'
AZOUT=$(az ams account sp create --account-name ${mediaServicesName} --resource-group ${mediaServicesResourceGroupName} | jq '{AadClientId: .AadClientId, AadSecret:.AadSecret}')

The script then places the credentials in a key vault for app settings to consume:

echo 'Adding access policy in KeyVault'
USER_PRINCIPAL_NAME=$(az ad signed-in-user show | jq -r '.userPrincipalName')
az keyvault set-policy --name ${keyVaultName} --upn $USER_PRINCIPAL_NAME --secret-permissions set get list delete > /dev/null
echo 'Updating ams-sp-client-id and ams-sp-client-secret in KeyVault'
az keyvault secret set --vault-name ${keyVaultName} --name 'ams-sp-client-id' --value $(echo $AZOUT | jq -r '.AadClientId') > /dev/null
az keyvault secret set --vault-name ${keyVaultName} --name 'ams-sp-client-secret' --value $(echo $AZOUT | jq -r '.AadSecret')  > /dev/null
echo 'Revoking access policy in KeyVault'
az keyvault delete-policy --name ${keyVaultName} --upn $USER_PRINCIPAL_NAME > /dev/null
echo 'Done.'

The Function App settings use a reference to the Azure Key Vault. The script creates those and other settings in the Terraform functions/main.tf file:

    {
      name        = "AmsAadClientId"
      value       = format("@Microsoft.KeyVault(SecretUri=https://%s.vault.azure.net/secrets/%s/)", var.key_vault_name, "ams-sp-client-id")
      slotSetting = false
    },
    {
      name        = "AmsAadClientSecret"
      value       = format("@Microsoft.KeyVault(SecretUri=https://%s.vault.azure.net/secrets/%s/)", var.key_vault_name, "ams-sp-client-secret")
      slotSetting = false
    },

Scale Media Services resources

The Azure Media Services account owner can scale media processing resources to perform the expected work by calling the Azure command-line interface (Azure CLI) within a YAML pipeline step.

The script is in azcli-last-steps-template.yml.

To set the Media Services reserved encoding infrastructure scale, run:

- task: AzureCLI@1
  displayName: 'Set the scale of Azure Media Services reserved encoding infrastructure.'
  inputs:
    azureSubscription: '${{parameters.serviceConnection}}'
    scriptLocation: inlineScript
    inlineScript: |
      set -eu
      echo Set the scale of Azure Media Services reserved encoding infrastructure for $AZURERM_MEDIA_SERVICES_ACCOUNT_RESOURCE_ID
      # Configurable values:
      amsMediaReservedUnitCountDesired=1
      amsMediaReservedUnitTypeDesired=S3
      #
      # Setup
      amsMediaReservedUnitJson=$(az ams account mru show --ids $AZURERM_MEDIA_SERVICES_ACCOUNT_RESOURCE_ID)
      amsMediaReservedUnitCountActual=$(echo $amsMediaReservedUnitJson | jq -r '.count')
      amsMediaReservedUnitTypeActual=$(echo $amsMediaReservedUnitJson | jq -r '.type')
      #
      # Validate count and type, set if needed.
      if [[ $amsMediaReservedUnitCountDesired -eq $amsMediaReservedUnitCountActual && $amsMediaReservedUnitTypeDesired == $amsMediaReservedUnitTypeActual ]]
      then
          echo Azure Media Services reserved encoding infrastructure does not required scaling, $AZURERM_MEDIA_SERVICES_ACCOUNT_RESOURCE_ID.
      else
          echo Azure Media Services reserved encoding infrastructure requires scaling from $amsMediaReservedUnitCountActual to $amsMediaReservedUnitCountDesired and/or $amsMediaReservedUnitTypeDesired to $amsMediaReservedUnitTypeActual.
          echo az ams account mru set --count $amsMediaReservedUnitCountDesired --type $amsMediaReservedUnitTypeDesired --ids $AZURERM_MEDIA_SERVICES_ACCOUNT_RESOURCE_ID
          az ams account mru set --count $amsMediaReservedUnitCountDesired --type $amsMediaReservedUnitTypeDesired --ids $AZURERM_MEDIA_SERVICES_ACCOUNT_RESOURCE_ID
      fi
    addSpnToEnvironment: true

To set the Media Services streaming endpoint infrastructure scale, run:

- task: AzureCLI@1
  displayName: 'Set the scale of Azure Media Services streaming endpoint infrastructure.'
  inputs:
    azureSubscription: '${{parameters.serviceConnection}}'
    scriptLocation: inlineScript
    inlineScript: |
      set -eu
      echo Set the scale of Azure Media Services streaming endpoint infrastructure for $AZURERM_MEDIA_SERVICES_ACCOUNT_RESOURCE_ID
      # Configurable values:
      amsStreamingEndpointScaleUnitsDesired=0
      #
      # Setup
      amsStreamingEndpointName=${{ parameters.applicationName }}amsse01${{ parameters.environment }}
      amsStreamingEndpointName=$(echo $amsStreamingEndpointName | tr '[:upper:]' '[:lower:]')
      amsaccount=$(az ams account show --ids $AZURERM_MEDIA_SERVICES_ACCOUNT_RESOURCE_ID)
      amsAccountName=$(echo $amsaccount | jq -r '.name')
      amsAccountResourceGroupName=$(echo $amsaccount | jq -r '.resourceGroup')
      amsStreamingEndpointListJson=$(az ams streaming-endpoint list --resource-group $amsAccountResourceGroupName --account-name $amsAccountName)
      amsStreamingEndpointJson=$(echo $amsStreamingEndpointListJson | jq --arg amssename $amsStreamingEndpointName -r '.[] | select(.name == $amssename)')
      amsDefaultJson=$(echo $amsStreamingEndpointListJson | jq -r '.[] | select(.name == "default")')
      #
      # If there is an endpoint, check and update the scale and run-state, otherwise create it.
      if [[ -n "$amsStreamingEndpointJson" ]]
      then
        scaleUnitsActual=$(echo $amsStreamingEndpointJson | jq -r '.scaleUnits')
        if [[ $scaleUnitsActual -ne $amsStreamingEndpointScaleUnitsDesired ]]
        then
          echo Azure Media Services streaming endpoint $amsStreamingEndpointName will be scaled
          echo az ams streaming-endpoint scale --resource-group $amsAccountResourceGroupName --account-name $amsAccountName --name $amsStreamingEndpointName --scale-units $amsStreamingEndpointScaleUnitsDesired --no-wait
          az ams streaming-endpoint scale --resource-group $amsAccountResourceGroupName --account-name $amsAccountName --name $amsStreamingEndpointName --scale-units $amsStreamingEndpointScaleUnitsDesired --no-wait
        fi
        resourceStateActual=$(echo $amsStreamingEndpointJson | jq -r '.resourceState')
        if [[ $resourceStateActual == "Stopped"  ]]
        then
          echo Starting the $amsStreamingEndpointName endpoint
          echo az ams streaming-endpoint start --resource-group $amsAccountResourceGroupName --account-name $amsAccountName --name $amsStreamingEndpointName --no-wait
          az ams streaming-endpoint start --resource-group $amsAccountResourceGroupName --account-name $amsAccountName --name $amsStreamingEndpointName --no-wait
        fi
      else
        echo Azure Media Services streaming endpoint $amsStreamingEndpointName will be created
        echo az ams streaming-endpoint create --resource-group $amsAccountResourceGroupName --account-name $amsAccountName --name $amsStreamingEndpointName --auto-start --scale-units $amsStreamingEndpointScaleUnitsDesired --no-wait
        az ams streaming-endpoint create --resource-group $amsAccountResourceGroupName --account-name $amsAccountName --name $amsStreamingEndpointName --auto-start --scale-units $amsStreamingEndpointScaleUnitsDesired --no-wait
      fi
      #
      # If there is a default endpoint, stop it.
      if [[ -n "$amsDefaultJson" ]]
      then
        resourceStateActual=$(echo $amsDefaultJson | jq -r '.resourceState')
        if [[ $resourceStateActual != "Stopped"  ]]
        then
          echo Stopping the default endpoint
          echo az ams streaming-endpoint stop --resource-group $amsAccountResourceGroupName --account-name $amsAccountName --name default --no-wait
          az ams streaming-endpoint stop --resource-group $amsAccountResourceGroupName --account-name $amsAccountName --name default --no-wait
        fi
      fi
    addSpnToEnvironment: true

Next steps

Product documentation:

Microsoft Learn modules: