Use multiple LUIS and QnA models

APPLIES TO: yesSDK v4 no SDK v3

If a bot uses multiple LUIS models and QnA Maker knowledge bases (knowledge bases), you can use Dispatch tool to determine which LUIS model or QnA Maker knowledge base best matches the user input. The dispatch tool does this by creating a single LUIS app to route user input to the correct model. For more information about the Dispatch, including the CLI commands, refer to the README.

Prerequisites

About this sample

This sample is based on a predefined set of LUIS and QnA Maker Apps.

Code sample logic flow

OnMessageActivityAsync is called for each user input received. This module finds the top scoring user intent and passes that result on to DispatchToTopIntentAsync. DispatchToTopIntentAsync, in turn, calls the appropriate app handler

  • ProcessSampleQnAAsync - for bot faq questions.
  • ProcessWeatherAsync - for weather queries.
  • ProcessHomeAutomationAsync - for home lighting commands.

The handler calls the LUIS or QnA Maker service and returns the generated result back to the user.

Create LUIS apps and QnA knowledge base

Before you can create the dispatch model, you'll need to have your LUIS apps and QnA knowledge bases created and published. In this article, we'll publish the following models that are included with the NLP With Dispatch sample in the \CognitiveModels folder:

Name Description
HomeAutomation A LUIS app that recognizes a home automation intent with associated entity data.
Weather A LUIS app that recognizes weather-related intents with location data.
QnAMaker A QnA Maker knowledge base that provides answers to simple questions about the bot.

Create LUIS apps

  1. Log into the LUIS web portal. Under the My apps section, select the Tab Import new app. The following Dialog Box will appear:

    Import LUIS json file

  2. Select the button Choose app file, navigate to the CognitiveModel folder of your sample code and select the file 'HomeAutomation.json'. Leave the optional name field blank.

  3. Select Done.

  4. Once LUIS opens up your Home Automation app, select the Train button. This will train your app using the set of utterances you just imported using the 'home-automation.json' file.

  5. When training is complete, select the Publish button. The following Dialog Box will appear:

    Publish LUIS app

  6. Choose the 'production' environment and then select the Publish button.

  7. Once your new LUIS app has been published, select the MANAGE Tab. From the 'Application Information' page, record the values Application ID as "app-id-for-app" and Display name as "name-of-app". From the 'Key and Endpoints' page, record the values Authoring Key as "your-luis-authoring-key" and Region as "your-region". These values will later be used within your 'appsetting.json' file.

  8. Once completed, Train and Publish both your LUIS Home Automation app and your LUIS Weather app by repeating the above steps for 'Weather.json' file.

Create QnA Maker knowledge base

The first step to setting up a QnA Maker knowledge base is to set up a QnA Maker service in Azure. To do that, follow the step-by-step instructions found here.

Once your QnA Maker Service has been created in Azure, you need to record the Cognitive Services Key 1 provided for your QnA Maker service. This will be used as <azure-qna-service-key1> when adding the QnA Maker app to your dispatch application.

Learn more about the two different types of keys used with QnA Maker.

The following steps provide you with this key:

Select Cognitive Service

  1. From within your Azure portal, select your QnA Maker cognitive service.

    Select Cognitive Service Keys

  2. Select the Keys icon found under the Resource Management section on the left-hand menu.

    Select Cognitive Service Key1

  3. Copy the value of Key 1 to your clipboard and save this locally. this will later be used for the (-k) key value <azure-qna-service-key1> when adding the QnA Maker app to your dispatch application.

  4. Now sign in to the QnAMaker web portal.

  5. At step 2, select the following:

    • Your Azure AD account.
    • Your Azure subscription name.
    • The name you created for your QnA Maker service. (If your Azure QnA service does not initially appear in this pull down list, try refreshing the page.)

    Create QnA Step 2

  6. At step 3, provide a name for your QnA Maker knowledge base. For this example use the name 'sample-qna'.

    Create QnA Step 3

  7. At step 4, select the option + Add File, navigate to the CognitiveModel folder of your sample code, and select the file 'QnAMaker.tsv'. There is an additional selection to add a Chit-chat personality to your knowledge base but our example does not include this option.

    Create QnA Step 4

  8. At step 5, select Create your knowledge base.

  9. Once the knowledge base is created from your uploaded file, select Save and train and when finished, select the PUBLISH Tab and publish your app.

  10. Once your QnA Maker app is published, select the SETTINGS Tab, and scroll down to 'Deployment details'. Record the following values from the Postman Sample HTTP request.

    POST /knowledge bases/<knowledge-base-id>/generateAnswer
    Host: <your-hostname>  // NOTE - this is a URL.
    Authorization: EndpointKey <qna-maker-resource-key>
    

    The full URL string for your hostname will look like "https://.azure.net/qnamaker". These values will later be used within your appsettings.json or .env file.

Dispatch app needs read access to existing apps

The dispatch tool needs authoring access to read the existing LUIS and QnA Maker apps in order to create a new parent LUIS app that dispatches to the LUIS and QnA Maker apps. This access is provided with the app IDs and authoring keys.

Service authoring keys

The authoring key is only used for creating and editing the models. You need an ID and key for each of the two LUIS apps and the QnA Maker app.

App Location of information
LUIS App ID - found in the LUIS portal for each app, Manage -> Application Information
Authoring Key - found in the LUIS portal, top-right corner, select your own User, then Settings.
QnA Maker App ID - found in the QnA Maker portal on the Settings page after you publish the app. This is the ID found in first part of the POST command after the knowledgebase. An example of where to find the app ID is POST /knowledgebases/<APP-ID>/generateAnswer.
Authoring Key - found in the Azure portal, for the QnA Maker resource, under the Keys. You only need one of the keys.

The authoring key is not used to get a prediction score or confidence score from the published application. You need the endpoint keys for this action. The endpoint keys are found and used later in this tutorial.

Learn more about the two different types of keys used with QnA Maker.

Create the dispatch model

The CLI interface for the dispatch tool creates the model for dispatching to the correct LUIS or QnA Maker app.

  1. Open a command prompt or terminal window, and change directories to the CognitiveModels directory

  2. Make sure you have the current version of npm and the Dispatch tool.

    npm i -g npm
    npm i -g botdispatch
    
  3. Use dispatch init to initialize create a .dispatch file for your dispatch model. Create this using a filename you will recognize.

    dispatch init -n <filename-to-create> --luisAuthoringKey "<your-luis-authoring-key>" --luisAuthoringRegion <your-region>
    
  4. Use dispatch add to add your LUIS apps and QnA Maker knowledge bases to the .dispatch file.

    dispatch add -t luis -i "<app-id-for-weather-app>" -n "<name-of-weather-app>" -v <app-version-number> -k "<your-luis-authoring-key>" --intentName l_Weather
    dispatch add -t luis -i "<app-id-for-home-automation-app>" -n "<name-of-home-automation-app>" -v <app-version-number> -k "<your-luis-authoring-key>" --intentName l_HomeAutomation
    dispatch add -t qna -i "<knowledge-base-id>" -n "<knowledge-base-name>" -k "<azure-qna-service-key1>" --intentName q_sample-qna
    
  5. Use dispatch create to generate a dispatch model from the .dispatch file.

    dispatch create
    
  6. Publish the dispatch LUIS app, just created.

Use the dispatch LUIS app

The generated LUIS app defines intents for each of the child apps and the knowledge base, as well as a none intent for when the utterance doesn't have a good fit.

  • l_HomeAutomation
  • l_Weather
  • None
  • q_sample-qna

These services need to be published under the correct names for the bot to run properly. The bot needs information about the published services, so that it can access those services.

Service endpoint keys

The bot needs the query prediction endpoints for the three LUIS apps (dispatch, weather, and home automation) and the single QnA Maker knowledge base. Use the following table to find the endpoint keys:

App Query endpoint key location
LUIS In the LUIS portal, for each LUIS app, in the Manage section, select Keys and Endpoint settings to find the keys associated with each app. If you are following this tutorial, the endpoint key is the same key as the <your-luis-authoring-key>. The authoring key allows for 1000 endpoint hits then expires.
QnA Maker In the QnA Maker portal, for the knowledge base, in the Manage settings, use the key value shows in the Postman settings for the Authorization header, without the text of EndpointKey.

These values are used in the appsettings.json for C# and the .env file for javascript.

Installing packages

Prior to running this app for the first time ensure that several NuGet packages are installed:

  • Microsoft.Bot.Builder
  • Microsoft.Bot.Builder.AI.Luis
  • Microsoft.Bot.Builder.AI.QnA

Manually update your appsettings.json file

Once all of your service apps are created, the information for each needs to be added into your 'appsettings.json' file. The initial C# Sample code contains an empty appsettings.json file:

appsettings.json

 "MicrosoftAppId": "",
 "MicrosoftAppPassword": "",

 "QnAKnowledgebaseId": "",
 "QnAEndpointKey": "",
 "QnAEndpointHostName": "",

 "LuisAppId": "",
 "LuisAPIKey": "",
 "LuisAPIHostName": "",

For each of the entities shown below, add the values you recorded earlier in these instructions:

appsettings.json

"MicrosoftAppId": "",
"MicrosoftAppPassword": "",
  
"QnAKnowledgebaseId": "<knowledge-base-id>",
"QnAEndpointKey": "<qna-maker-resource-key>",
"QnAEndpointHostName": "<your-hostname>",

"LuisAppId": "<app-id-for-dispatch-app>",
"LuisAPIKey": "<your-luis-endpoint-key>",
"LuisAPIHostName": "<your-dispatch-app-region>",

When all changes are complete, save this file.

Connect to the services from your bot

To connect to the Dispatch, LUIS, and QnA Maker services, your bot pulls information from the settings file (either the appsettings.json or the .env file).

In BotServices.cs, the information contained within configuration file appsettings.json is used to connect your dispatch bot to the Dispatch and SampleQnA services. The constructors use the values you provided to connect to these services.

BotServices.cs

// Read the setting for cognitive services (LUIS, QnA) from the appsettings.json
// If includeApiResults is set to true, the full response from the LUIS api (LuisResult)
// will be made available in the properties collection of the RecognizerResult
Dispatch = new LuisRecognizer(new LuisApplication(
    configuration["LuisAppId"],
    configuration["LuisAPIKey"],
    $"https://{configuration["LuisAPIHostName"]}.api.cognitive.microsoft.com"),
    new LuisPredictionOptions { IncludeAllIntents = true, IncludeInstanceData = true },
    includeApiResults: true);

SampleQnA = new QnAMaker(new QnAMakerEndpoint
{
    KnowledgeBaseId = configuration["QnAKnowledgebaseId"],
    EndpointKey = configuration["QnAEndpointKey"],
    Host = configuration["QnAEndpointHostName"]
});

Note

By default the includeApiResults parameter is set to false, meaning the recognizer will only return basic information about entities / intents. If you require the full response from LUIS (such as the ConnectedServiceResult used later in this tutorial), then set this parameter to true. This will then add the full response from the LUIS service into the Properties collection on the RecognizerResult.

Call the services from your bot

For each input from your user, the bot logic checks user input against the combined Dispatch model, finds the top returned intent, and uses that information to call the appropriate service for the input.

In the DispatchBot.cs file whenever the OnMessageActivityAsync method is called, we check the incoming user message against the Dispatch model. We then pass the Dispatch Model's topIntent and recognizerResult on to the correct method to call the service and return the result.

bots\DispatchBot.cs

protected override async Task OnMessageActivityAsync(ITurnContext<IMessageActivity> turnContext, CancellationToken cancellationToken)
{
    // First, we use the dispatch model to determine which cognitive service (LUIS or QnA) to use.
    var recognizerResult = await _botServices.Dispatch.RecognizeAsync(turnContext, cancellationToken);
    
    // Top intent tell us which cognitive service to use.
    var topIntent = recognizerResult.GetTopScoringIntent();
    
    // Next, we call the dispatcher with the top intent.
    await DispatchToTopIntentAsync(turnContext, topIntent.intent, recognizerResult, cancellationToken);
}

Work with the recognition results

When the model produces a result, it indicates which service can most appropriately process the utterance. The code in this bot routes the request to the corresponding service, and then summarizes the response from the called service. Depending on the intent returned from Dispatch, this code uses the returned intent to route to the correct LUIS model or QnA service.

bots\DispatchBot.cs

private async Task DispatchToTopIntentAsync(ITurnContext<IMessageActivity> turnContext, string intent, RecognizerResult recognizerResult, CancellationToken cancellationToken)
{
    switch (intent)
    {
        case "l_HomeAutomation":
            await ProcessHomeAutomationAsync(turnContext, recognizerResult.Properties["luisResult"] as LuisResult, cancellationToken);
            break;
        case "l_Weather":
            await ProcessWeatherAsync(turnContext, recognizerResult.Properties["luisResult"] as LuisResult, cancellationToken);
            break;
        case "q_sample-qna":
            await ProcessSampleQnAAsync(turnContext, cancellationToken);
            break;
        default:
            _logger.LogInformation($"Dispatch unrecognized intent: {intent}.");
            await turnContext.SendActivityAsync(MessageFactory.Text($"Dispatch unrecognized intent: {intent}."), cancellationToken);
            break;
    }
}

If method ProcessHomeAutomationAsync or ProcessWeatherAsync are invoked, they are passed the results from the dispatch model within luisResult.ConnectedServiceResult. The specified method then provides user feedback showing the dispatch model top intent, plus a ranked listing of all intents and entities that were detected.

If method q_sample-qna is invoked, it uses the user input contained within the turnContext to generate an answer from the knowledge base and display that result to the user.

Note

If this were a production application, this is where the selected LUIS methods would connect to their specified service, pass in the user input, and process the returned LUIS intent and entity data.

Test your bot

  1. Using your development environment, start the sample code. Note the localhost address shown in the address bar of the browser window opened by your App: "https://localhost:<Port_Number>".

  2. Open your Bot Framework Emulator, then select Create a new bot configuration. A .bot file enables you to use the Inspector in the bot emulator to see the JSON returned from LUIS and QnA Maker.

  3. In the New bot configuration dialog box, enter your bot name, and your endpoint URL, such as http://localhost:3978/api/messages. Save the file at the root of your bot sample code project.

  4. Open the bot file and add sections for your LUIS and QnA Maker apps. Use this example file as a template for settings. Save the changes.

  5. Select the bot name in the My Bots list to access your running bot. For your reference, here are some of the questions and commands that are covered by the services built for your bot:

    • QnA Maker
      • hi, good morning
      • what are you, what do you do
    • LUIS (home automation)
      • turn on bedroom light
      • turn off bedroom light
      • make some coffee
    • LUIS (weather)
      • whats the weather in redmond washington
      • what's the forecast for london
      • show me the forecast for nebraska

Dispatch for user utterance to QnA Maker

  1. In the bot emulator, enter the text hi and submit the utterance. The bot submits this query to the dispatch LUIS app and gets back a response indicating which child app should get this utterance for further processing.

  2. By selecting the LUIS Trace line in the log, you can see the LUIS response in the bot emulator . The LUIS result from the dispatch LUIS app displays in the Inspector.

    {
      "luisResponse": {
        "entities": [],
        "intents": [
          {
            "intent": "q_sample-qna",
            "score": 0.9489713
          },
          {
            "intent": "l_HomeAutomation",
            "score": 0.0612499453
          },
          {
            "intent": "None",
            "score": 0.008567564
          },
          {
            "intent": "l_Weather",
            "score": 0.0025761195
          }
        ],
        "query": "Hi",
        "topScoringIntent": {
          "intent": "q_sample-qna",
          "score": 0.9489713
        }
      }
    }
    

    Because the utterance, hi, is part of the dispatch LUIS app's q_sample-qna intent, and is selected as the topScoringIntent, the bot will make a second request, this time to the QnA Maker app, with the same utterance.

  3. Select the QnAMaker Trace line in the bot emulator log. The QnA Maker result displays in the Inspector.

{
    "questions": [
        "hi",
        "greetings",
        "good morning",
        "good evening"
    ],
    "answer": "Hello!",
    "score": 1,
    "id": 96,
    "source": "QnAMaker.tsv",
    "metadata": [],
    "context": {
        "isContextOnly": false,
        "prompts": []
    }
}

Resolving incorrect top intent from Dispatch

Once your bot is running, it is possible to improve the bot's performance by removing similar or overlapping utterances between the dispatched apps.

You can use the Dispatch command-line tool to test and evaluate your dispatch model.

To update or create a new LUIS model

This sample is based on a preconfigured LUIS model. Additional information to help you update this model, or create a new LUIS model, can be found here.

After updating the underlying models (QnA or LUIS) run dispatch refresh to update your Dispatch LUIS app. dispatch refresh is basically the same command as dispatch create except no new LUIS app ID is created.

Note that utterances that were added directly in LUIS will not be retained when running dispatch refresh. To keep those extra utterances in the Dispatch app add those utterances in a text file (one utterance per line), and then add the file to Dispatch by running the command:

dispatch add -t file -f <file path> --intentName <target intent name, ie l_General>

Once the file with extra utterances is added to Dispatch the utterances will stay with every refresh.

To delete resources

This sample creates a number of applications and resources that you can delete using the steps listed below, but you should not delete resources that any other apps or services rely on.

To delete LUIS resources:

  1. Sign in to the luis.ai portal.
  2. Go to the My Apps page.
  3. Select the apps created by this sample.
    • Home Automation
    • Weather
    • NLP-With-Dispatch-BotDispatch
  4. Click Delete, and click Ok to confirm.

To delete QnA Maker resources:

  1. Sign in to the qnamaker.ai portal.
  2. Go to the My knowledge bases page.
  3. Click the delete button for the Sample QnA knowledge base, and click Delete to confirm.

Best practice

To improve services used in this sample, refer to best practice for LUIS, and QnA Maker.