Add telemetry to your bot

APPLIES TO: yesSDK v4 no SDK v3

Telemetry logging was added to version 4.2 of the Bot Framework SDK. This enables bot applications to send event data to telemetry services such as Application Insights. Telemetry offers insights into your bot by showing which features are used the most, detects unwanted behavior and offers visibility into availability, performance, and usage.

In this article you will learn how to implement telemetry into your bot using Application Insights:

  • The code required to wire up telemetry in your bot and connect to Application Insights

  • Enabling telemetry in your bots Dialogs

  • Enabling telemetry to capture usage data from other services like LUIS and QnA Maker.

  • Visualizing your telemetry data in Application Insights

Prerequisites

Note

The Application Insights sample code was built on top of the CoreBot sample code. This article will step you through modifying the CoreBot sample code to incorporate telemetry. If you are following along in Visual Studio you will have the Application Insights sample code by the time we are finished.

Wiring up telemetry in your bot

We will start with the CoreBot sample app and add the code required to integrate telemetry into any bot. This will enable Application Insights to begin tracking requests.

  1. Open the CoreBot sample app in Visual Studio

  2. Add the Microsoft.Bot.Builder.Integration.ApplicationInsights.Core NuGet package. For more information on using NuGet, see Install and manage packages in Visual Studio:

  3. Include the following statements in Startup.cs:

    using Microsoft.ApplicationInsights.Extensibility;
    using Microsoft.Bot.Builder.ApplicationInsights;
    using Microsoft.Bot.Builder.Integration.ApplicationInsights.Core;
    using Microsoft.Bot.Builder.Integration.AspNet.Core;
    

    Note: If you're following along by updating the CoreBot sample code you will notice that the using statement for Microsoft.Bot.Builder.Integration.AspNet.Core already exists in the CoreBot sample.

  4. Include the following code in the ConfigureServices() method in Startup.cs. This will make telemetry services available to your bot via dependency injection (DI):

    // This method gets called by the runtime. Use this method to add services to the container.
    public void ConfigureServices(IServiceCollection services)
    {
        ...
    
        // Create the Bot Framework Adapter with error handling enabled.
        services.AddSingleton<IBotFrameworkHttpAdapter, AdapterWithErrorHandler>();
    
        // Add Application Insights services into service collection
        services.AddApplicationInsightsTelemetry();
    
        // Create the telemetry client.
        services.AddSingleton<IBotTelemetryClient, BotTelemetryClient>();
    
        // Add ASP middleware to store the http body mapped with bot activity key in the httpcontext.items. This will be picked by the TelemetryBotIdInitializer
        services.AddTransient<TelemetrySaveBodyASPMiddleware>();
    
        // Add telemetry initializer that will set the correlation context for all telemetry items.
        services.AddSingleton<ITelemetryInitializer, OperationCorrelationTelemetryInitializer>();
    
        // Add telemetry initializer that sets the user ID and session ID (in addition to other bot-specific properties such as activity ID)
        services.AddSingleton<ITelemetryInitializer, TelemetryBotIdInitializer>();
    
        // Create the telemetry middleware to track conversation events
        services.AddSingleton<IMiddleware, TelemetryLoggerMiddleware>();
    
        ...
    }
    

    Note: If your following along by updating the CoreBot sample code you will notice that services.AddSingleton<IBotFrameworkHttpAdapter, AdapterWithErrorHandler>(); already exists

  5. Add the UseBotApplicationInsights() method call in the Configure() method in Startup.cs. This enables your bot to store the required bot-specific properties in the HTTP context so it can be retrieved by the telemetry initializer when an event is tracked:

    // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
    public void Configure(IApplicationBuilder app, IHostingEnvironment env)
    {
        ...
    
        app.UseBotApplicationInsights();
    }
    
  6. Instruct the adapter to use the middleware code that was added to the ConfigureServices() method. You do this in AdapterWithErrorHandler.cs with the parameter IMiddleware middleware in the constructors parameter list, and the Use(middleware); statement in the contructor as shown here:

    public AdapterWithErrorHandler(ICredentialProvider credentialProvider, ILogger<BotFrameworkHttpAdapter> logger, IMiddleware middleware, ConversationState conversationState = null)
            : base(credentialProvider)
    {
        ...
    
        Use(middleware);
    }
    
  7. Add the Application Insights instrumentation key in your appsettings.json file The appsettings.json file contains metadata about external services the Bot uses while running. For example, CosmosDB, Application Insights and the Language Understanding (LUIS) service connection and metadata is stored there. The addition to your appsettings.json file must be in this format:

    {
        "MicrosoftAppId": "",
        "MicrosoftAppPassword": "",
        "ApplicationInsights": {
            "InstrumentationKey": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
        }
    }
    

    Note: Details on getting the Application Insights instrumentation key can be found in the article Application Insights keys.

At this point the preliminary work to enable telemetry using Application Insights is done. You can run your bot locally using the bot emulator and then go into Application Insights to see what is being logged such as response time, overall app health, and general running information.

Next we will see what needs to be included to add telemetry functionality to the dialogs. This will enable you to get additional information such as what dialogs run, and statistics about each one.

Enabling telemetry in your bots Dialogs

To get the built-in telemetry information about your dialogs, you need to add the telemetry client to every dialog. Follow the steps below to update your CoreBot example:

  1. In MainDialog.cs you need to add a new TelemetryClient field to the MainDialog class then update the constructors parameter list to include an IBotTelemetryClient parameter, then pass that to each call to the AddDialog() method.

    • Add the parameter IBotTelemetryClient telemetryClient to the MainDialog class constructor, then assign it to the TelemetryClient field:

      public MainDialog(IConfiguration configuration, ILogger<MainDialog> logger, IBotTelemetryClient telemetryClient)
          : base(nameof(MainDialog))
      {
          TelemetryClient = telemetryClient;
      
          ...
      
      }
      
    • When adding each dialog using the AddDialog method, set the TelemetryClient value:

      public MainDialog(IConfiguration configuration, ILogger<MainDialog> logger, IBotTelemetryClient telemetryClient)
          : base(nameof(MainDialog))
      {
          ...
      
          AddDialog(new TextPrompt(nameof(TextPrompt))
          {
              TelemetryClient = telemetryClient,
          });
      
          ...
      }
      
    • Waterfall Dialogs generate events independently of other dialogs. Set the TelemetryClient property after the list of Waterfall Steps:

      // The IBotTelemetryClient to the WaterfallDialog
      AddDialog(new WaterfallDialog(
          nameof(WaterfallDialog),
          new WaterfallStep[]
      {
          IntroStepAsync,
          ActStepAsync,
          FinalStepAsync,
      })
      {
          TelemetryClient = telemetryClient,
      });
      
      
  2. In DialogExtensions.cs you need to set the TelemetryClient property of the dialogSet object in the Run() method:

    public static async Task Run(this Dialog dialog, ITurnContext turnContext, IStatePropertyAccessor<DialogState> accessor, CancellationToken cancellationToken = default(CancellationToken))
    {
        ...
    
        dialogSet.TelemetryClient = dialog.TelemetryClient;
    
        ...
    
    }
    
    
  3. In BookingDialog.cs use the same process used when updating MainDialog.cs to enable telemetry in the four dialogs added in this file. Don't forget to add the TelemetryClient field to the BookingDialog class and the IBotTelemetryClient telemetryClient parameter to the BookingDialog constructor.

Tip

If you are following along and updating the CoreBot sample code, you can refer to the Application Insights sample code if you run into any problems.

That's all there is to adding telemetry to your bots dialogs, at this point if you ran your bot you should see things being logged in Application Insights, however if you have any integrated technology such as LUIS and QnA Maker you will need to add the TelemetryClient to that code as well.

Enabling telemetry to capture usage data from other services like LUIS and QnA Maker

We will next implement telemetry functionality in your LUIS service. The LUIS service has built-in telemetry logging available so there is very little you need to do to start getting telemetry data from LUIS.

For this sample, we just need to provide the telemetry client in a similar way to how we did for dialogs.

  1. The IBotTelemetryClient telemetryClient parameter is required in the ExecuteLuisQuery() method in LuisHelper.cs:

    public static async Task<BookingDetails> ExecuteLuisQuery(IBotTelemetryClient telemetryClient, IConfiguration configuration, ILogger logger, ITurnContext turnContext, CancellationToken cancellationToken)
    
  2. The LuisPredictionOptions Class enables you to provide optional parameters for a LUIS prediction request. To enable telemetry, you will need to set the TelemetryClient parameter when creating the luisPredictionOptions object in LuisHelper.cs:

    var luisPredictionOptions = new LuisPredictionOptions()
    {
        TelemetryClient = telemetryClient,
    };
    
    var recognizer = new LuisRecognizer(luisApplication, luisPredictionOptions);
    
  3. The last step is to pass in the telemetry client to the call to ExecuteLuisQuery in the ActStepAsync() method of MainDialog.cs:

    await LuisHelper.ExecuteLuisQuery(TelemetryClient, Configuration, Logger, stepContext.Context, cancellationToken)
    

That's it, you should have a functional bot that logs telemetry data into Application insights. You can use the Bot Framework Emulator to run your bot locally. You shouldn't see any changes in the bot's behavior, but it will be logging information into Application Insights. Interact with the bot by sending multiple messages and in the next section we will review the telemetry results in Application Insights.

For information on testing and debugging your bot, you can refer to the following articles:

Visualizing your telemetry data in Application Insights

Application Insights monitors the availability, performance, and usage of your bot application whether it's hosted in the cloud or on-premises. It leverages the powerful data analysis platform in Azure Monitor to provide you with deep insights into your application's operations and diagnose errors without waiting for a user to report them. There are a few ways to see the telemetry data collected by Application Insights, two of the primary ways are through queries and the dashboard.

Querying your telemetry data in Application Insights using Kusto Queries

Use this section as a starting point to learn how to use log queries in Application Insights. It demonstrates two useful queries and provides links to other documentation with additional information.

To query your data

  1. Go to the Azure portal

  2. Navigate to your Application Insights. Easiest way to do so is click on Monitor > Applications and find it there.

  3. Once in your Application Insights, you can click on Logs (Analytics) on the navigation bar.

    Logs (Analytics)

  4. This will bring up the Query window. Enter the following query and select Run:

    customEvents
    | where name=="WaterfallStart"
    | extend DialogId = customDimensions['DialogId']
    | extend InstanceId = tostring(customDimensions['InstanceId'])
    | join kind=leftouter (customEvents | where name=="WaterfallComplete" | extend InstanceId = tostring(customDimensions['InstanceId'])) on InstanceId    
    | summarize starts=countif(name=='WaterfallStart'), completes=countif(name1=='WaterfallComplete') by bin(timestamp, 1d), tostring(DialogId)
    | project Percentage=max_of(0.0, completes * 1.0 / starts), timestamp, tostring(DialogId) 
    | render timechart
    
  5. This will return the percentage of waterfall dialogs that run to completion.

    Logs (Analytics)

Tip

You can pin any query to your Application Insights dashboard by selecting the button on the top right of the Logs (Analytics) blade. Just select the dashboard you want it pinned to, and it will be available next time you visit that dashboard.

The Application Insights dashboard

Anytime you create an Application Insights resource in Azure, a new dashboard will automatically be created and associated with it. You can see that dashboard by selecting the button at the top of your Application Insights blade, labeled Application Dashboard.

Application Dashboard Link

Alternatively, to view the data, go to the Azure portal. Click Dashboard on the left, then select the dashboard you want from the drop-down.

There, you'll see some default information about your bot performance and any additional queries that you've pinned to your dashboard.

Additional Information