Monitor your Azure Machine Learning models with Application Insights

In this article, you learn how to set up Azure Application Insights for Azure Machine Learning. Application Insights gives you the opportunity to monitor:

  • Request rates, response times, and failure rates.
  • Dependency rates, response times, and failure rates.
  • Exceptions.

Learn more about Application Insights.

Prerequisites

Use SDK to configure

Update a deployed service

  1. Identify the service in your workspace. The value for ws is the name of your workspace.

    from azureml.core.webservice import Webservice
    aks_service= Webservice(ws, "my-service-name")
    
  2. Update your service and enable Application Insights.

    aks_service.update(enable_app_insights=True)
    

Log custom traces in your service

If you want to log custom traces, follow the standard deployment process for AKS or ACI in the How to deploy and where document. Then use the following steps:

  1. Update the scoring file by adding print statements.

    print ("model initialized" + time.strftime("%H:%M:%S"))
    
  2. Update the service configuration.

    config = Webservice.deploy_configuration(enable_app_insights=True)
    
  3. Build an image and deploy it on AKS or ACI.

Disable tracking in Python

To disable Application Insights, use the following code:

## replace <service_name> with the name of the web service
<service_name>.update(enable_app_insights=False)

Use portal to configure

You can enable and disable Application Insights in the Azure portal.

  1. In the Azure portal, open your workspace.

  2. On the Deployments tab, select the service where you want to enable Application Insights.

    List of services on the Deployments tab

  3. Select Edit.

    Edit button

  4. In Advanced Settings, select the Enable AppInsights diagnostics check box.

    Selected check box for enabling diagnostics

  5. Select Update at the bottom of the screen to apply the changes.

Disable

  1. In the Azure portal, open your workspace.

  2. Select Deployments, select the service, and select Edit.

    Use the edit button

  3. In Advanced Settings, clear the Enable AppInsights diagnostics check box.

    Cleared check box for enabling diagnostics

  4. Select Update at the bottom of the screen to apply the changes.

Evaluate data

Your service's data is stored in your Application Insights account, within the same resource group as Azure Machine Learning. To view it:

  1. Go to your Machine Learning service workspace in the Azure portal and click on Application Insights link.

    AppInsightsLoc

  2. Select the Overview tab to see a basic set of metrics for your service.

    Overview

  3. To look into your custom traces, select Analytics.

  4. In the schema section, select Traces. Then select Run to run your query. Data should appear in a table format and should map to your custom calls in your scoring file.

    Custom traces

To learn more about how to use Application Insights, see What is Application Insights?.

Example notebook

The enable-app-insights-in-production-service.ipynb notebook demonstrates concepts in this article.

Learn how to run notebooks by following the article, Use Jupyter notebooks to explore this service.

Next steps

You can also collect data on your models in production. Read the article Collect data for models in production.

Also read Azure Monitor for containers.