Use multiple LUIS and QnA models
If a bot uses multiple LUIS models and QnA Maker knowledge bases (knowledge bases), you can use Dispatch tool to determine which LUIS model or QnA Maker knowledge base best matches the user input. The dispatch tool does this by creating a single LUIS app to route user input to the correct model. For more information about the Dispatch, including the CLI commands, refer to the dispatch README.
Prerequisites
- A luis.ai account to publish LUIS apps.
- A QnA Maker account to publish the QnA knowledge base.
- A copy of the NLP with Dispatch sample in C#, JavaScript, or Python.
- Knowledge of bot basics, LUIS, and QnA Maker.
- The command-line Dispatch tool
About this sample
This sample is based on a predefined set of LUIS and QnA Maker apps.
OnMessageActivityAsync
is called for each user input received. This module finds the top scoring user intent and passes that result on to DispatchToTopIntentAsync
. DispatchToTopIntentAsync, in turn, calls the appropriate app handler.
ProcessSampleQnAAsync
- for bot faq questions.ProcessWeatherAsync
- for weather queries.ProcessHomeAutomationAsync
- for home lighting commands.
The handler calls the LUIS or QnA Maker service and returns the generated result back to the user.
Create LUIS apps and QnA knowledge base
Before you can create the dispatch model, you'll need to have your LUIS apps and QnA knowledge bases created and published. In this article, we'll publish the following models that are included with the NLP With Dispatch sample in the \CognitiveModels
folder:
Name | Description |
---|---|
HomeAutomation | A LUIS app that recognizes a home automation intent with associated entity data. |
Weather | A LUIS app that recognizes weather-related intents with location data. |
QnAMaker | A QnA Maker knowledge base that provides answers to simple questions about the bot. |
Create the LUIS apps
- Create a LUIS app from the home automation JSON file in the cognitive models directory of the sample.
- Train and publish the app to the production environment.
- Record the application ID, display name, authoring key, and location.
- Repeat these steps for the weather JSON file.
For more information, see how to Create a LUIS app in the LUIS portal and Obtain values to connect to your LUIS app in Add natural language understanding to your bot and the LUIS documentation on how to train and publish an app to the production environment.
Create the QnA Maker knowledge base
The first step to setting up a QnA Maker knowledge base is to set up a QnA Maker service in Azure. To do that, follow the step-by-step instructions found the set up qnamaker service article.
Once your QnA Maker Service has been created in Azure, you need to record the Cognitive Services Key 1 provided for your QnA Maker service. This will be used as <azure-qna-service-key1> when adding the QnA Maker app to your dispatch application.
Learn more about the two different types of keys used with QnA Maker.
The following steps provide you with this key:
From within your Azure portal, select your QnA Maker cognitive service.
Select the Keys icon found under the Resource Management section on the left-hand menu.
Copy the value of Key 1 to your clipboard and save this locally. this will later be used for the (-k) key value <azure-qna-service-key1> when adding the QnA Maker app to your dispatch application.
Now sign in to the QnAMaker web portal.
At step 2, select the following:
- Your Azure AD account.
- Your Azure subscription name.
- The name you created for your QnA Maker service. (If your Azure QnA service does not initially appear in this pull down list, try refreshing the page.)
At step 3, provide a name for your QnA Maker knowledge base. For this example use the name 'sample-qna'.
At step 4, select the option + Add File, navigate to the CognitiveModel folder of your sample code, and select the file 'QnAMaker.tsv'. There is an additional selection to add a Chit-chat personality to your knowledge base but our example does not include this option.
At step 5, select Create your knowledge base.
Once the knowledge base is created from your uploaded file, select Save and train and when finished, select the PUBLISH Tab and publish your app.
Once your QnA Maker app is published, select the SETTINGS Tab, and scroll down to 'Deployment details'. Record the following values from the Postman Sample HTTP request.
POST /knowledge bases/<knowledge-base-id>/generateAnswer Host: <your-hostname> // NOTE - this is a URL. Authorization: EndpointKey <qna-maker-resource-key>
The full URL string for your hostname will look like "https://
.azure.net/qnamaker". These values will later be used within your appsettings.json
or.env
file.
Dispatch app needs read access to existing apps
The dispatch tool needs authoring access to read the existing LUIS and QnA Maker apps in order to create a new parent LUIS app that dispatches to the LUIS and QnA Maker apps. This access is provided with the app IDs and authoring keys.
Service authoring keys
The authoring key is only used for creating and editing the models. You need an ID and key for each of the two LUIS apps and the QnA Maker app.
- for LUIS
- The App ID is found in the LUIS portal for each app, Manage > Settings > Application Settings
- The Authoring Key is found in the LUIS portal, top-right corner, select your own User, then Settings.
- for Qna Maker
- The App ID is found in the QnA Maker portal on the Settings page after you publish the app. This is the ID found in first part of the POST command after the knowledgebase. An example of where to find the app ID is
POST /knowledgebases/<APP-ID>/generateAnswer
. - The Authoring Key is found in the Azure portal, for the QnA Maker resource, under the Keys. You only need one of the keys.
- The App ID is found in the QnA Maker portal on the Settings page after you publish the app. This is the ID found in first part of the POST command after the knowledgebase. An example of where to find the app ID is
The authoring key is not used to get a prediction score or confidence score from the published application. You need the endpoint keys for this action. The endpoint keys are found and used later in this tutorial.
Learn more about the two different types of keys used with QnA Maker.
Create the dispatch model
The CLI interface for the dispatch tool creates the model for dispatching to the correct LUIS or QnA Maker app.
Open a command prompt or terminal window, and change directories to the CognitiveModels directory
Make sure you have the current version of npm and the Dispatch tool.
npm i -g npm npm i -g botdispatch
Use
dispatch init
to initialize create a.dispatch
file for your dispatch model. Create this using a filename you will recognize.dispatch init -n <filename-to-create> --luisAuthoringKey "<your-luis-authoring-key>" --luisAuthoringRegion <your-region>
Use
dispatch add
to add your LUIS apps and QnA Maker knowledge bases to the.dispatch
file.dispatch add -t luis -i "<app-id-for-weather-app>" -n "<name-of-weather-app>" -v <app-version-number> -k "<your-luis-authoring-key>" --intentName l_Weather dispatch add -t luis -i "<app-id-for-home-automation-app>" -n "<name-of-home-automation-app>" -v <app-version-number> -k "<your-luis-authoring-key>" --intentName l_HomeAutomation dispatch add -t qna -i "<knowledge-base-id>" -n "<knowledge-base-name>" -k "<azure-qna-service-key1>" --intentName q_sample-qna
Use
dispatch create
to generate a dispatch model from the.dispatch
file.dispatch create
Publish the dispatch LUIS app, just created.
Use the dispatch LUIS app
The generated LUIS app defines intents for each of the child apps and the knowledge base, as well as a none intent for when the utterance doesn't have a good fit.
l_HomeAutomation
l_Weather
None
q_sample-qna
These services need to be published under the correct names for the bot to run properly. The bot needs information about the published services, so that it can access those services.
Service endpoint keys
The bot needs the query prediction endpoints for the three LUIS apps (dispatch, weather, and home automation) and the single QnA Maker knowledge base. Find the endpoint keys in the LUIS and QnA Maker portals:
- In the LUIS portal, for each LUIS app, in the Manage section, select Keys and Endpoint settings to find the keys associated with each app. If you are following this tutorial, the endpoint key is the same key as the
<your-luis-authoring-key>
. The authoring key allows for 1000 endpoint hits then expires. - In the QnA Maker portal, for the knowledge base, in the Manage settings, use the key value shows in the Postman settings for the Authorization header, without the text of
EndpointKey
.
These values are used in the sample's configuration file: appsettings.json (C#), .env (JavaScript), or config.py (Python).
Installing packages
Prior to running this app for the first time ensure that several NuGet packages are installed:
- Microsoft.Bot.Builder
- Microsoft.Bot.Builder.AI.Luis
- Microsoft.Bot.Builder.AI.QnA
Manually update your appsettings.json file
Once all of your service apps are created, the information for each needs to be added into your 'appsettings.json' file. The initial C# Sample code contains an empty appsettings.json file:
appsettings.json
"MicrosoftAppId": "",
"MicrosoftAppPassword": "",
"QnAKnowledgebaseId": "",
"QnAEndpointKey": "",
"QnAEndpointHostName": "",
"LuisAppId": "",
"LuisAPIKey": "",
"LuisAPIHostName": "",
For each of the entities shown below, add the values you recorded earlier in these instructions:
appsettings.json
"MicrosoftAppId": "",
"MicrosoftAppPassword": "",
"QnAKnowledgebaseId": "<knowledge-base-id>",
"QnAEndpointKey": "<qna-maker-resource-key>",
"QnAEndpointHostName": "<your-hostname>",
"LuisAppId": "<app-id-for-dispatch-app>",
"LuisAPIKey": "<your-luis-endpoint-key>",
"LuisAPIHostName": "<your-dispatch-app-region>",
When all changes are complete, save this file.
Connect to the services from your bot
To connect to the Dispatch, LUIS, and QnA Maker services, your bot pulls information from the settings file.
In BotServices.cs, the information contained within configuration file appsettings.json is used to connect your dispatch bot to the Dispatch
and SampleQnA
services. The constructors use the values you provided to connect to these services.
BotServices.cs
public class BotServices : IBotServices
{
public BotServices(IConfiguration configuration)
{
// Read the setting for cognitive services (LUIS, QnA) from the appsettings.json
// If includeApiResults is set to true, the full response from the LUIS api (LuisResult)
// will be made available in the properties collection of the RecognizerResult
var luisApplication = new LuisApplication(
configuration["LuisAppId"],
configuration["LuisAPIKey"],
$"https://{configuration["LuisAPIHostName"]}.api.cognitive.microsoft.com");
// Set the recognizer options depending on which endpoint version you want to use.
// More details can be found in https://docs.microsoft.com/en-gb/azure/cognitive-services/luis/luis-migration-api-v3
var recognizerOptions = new LuisRecognizerOptionsV2(luisApplication)
{
IncludeAPIResults = true,
PredictionOptions = new LuisPredictionOptions()
{
IncludeAllIntents = true,
IncludeInstanceData = true
}
};
Dispatch = new LuisRecognizer(recognizerOptions);
SampleQnA = new QnAMaker(new QnAMakerEndpoint
{
KnowledgeBaseId = configuration["QnAKnowledgebaseId"],
EndpointKey = configuration["QnAEndpointKey"],
Host = configuration["QnAEndpointHostName"]
});
}
public LuisRecognizer Dispatch { get; private set; }
public QnAMaker SampleQnA { get; private set; }
}
Note
By default the includeApiResults
parameter is set to false, meaning the recognizer will only return basic information about entities / intents. If you require the full response from LUIS (such as the ConnectedServiceResult
used later in this tutorial), then set this parameter to true. This will then add the full response from the LUIS service into the Properties collection on the RecognizerResult
.
Call the services from your bot
For each input from your user, the bot logic checks user input against the combined Dispatch model, finds the top returned intent, and uses that information to call the appropriate service for the input.
In the DispatchBot.cs file whenever the OnMessageActivityAsync
method is called, we check the incoming user message against the Dispatch model. We then pass the Dispatch Model's topIntent
and recognizerResult
on to the correct method to call the service and return the result.
bots\DispatchBot.cs
protected override async Task OnMessageActivityAsync(ITurnContext<IMessageActivity> turnContext, CancellationToken cancellationToken)
{
// First, we use the dispatch model to determine which cognitive service (LUIS or QnA) to use.
var recognizerResult = await _botServices.Dispatch.RecognizeAsync(turnContext, cancellationToken);
// Top intent tell us which cognitive service to use.
var topIntent = recognizerResult.GetTopScoringIntent();
// Next, we call the dispatcher with the top intent.
await DispatchToTopIntentAsync(turnContext, topIntent.intent, recognizerResult, cancellationToken);
}
Work with the recognition results
When the model produces a result, it indicates which service can most appropriately process the utterance. The code in this bot routes the request to the corresponding service, and then summarizes the response from the called service. Depending on the intent returned from Dispatch, this code uses the returned intent to route to the correct LUIS model or QnA service.
bots\DispatchBot.cs
private async Task DispatchToTopIntentAsync(ITurnContext<IMessageActivity> turnContext, string intent, RecognizerResult recognizerResult, CancellationToken cancellationToken)
{
switch (intent)
{
case "l_HomeAutomation":
await ProcessHomeAutomationAsync(turnContext, recognizerResult.Properties["luisResult"] as LuisResult, cancellationToken);
break;
case "l_Weather":
await ProcessWeatherAsync(turnContext, recognizerResult.Properties["luisResult"] as LuisResult, cancellationToken);
break;
case "q_sample-qna":
await ProcessSampleQnAAsync(turnContext, cancellationToken);
break;
default:
_logger.LogInformation($"Dispatch unrecognized intent: {intent}.");
await turnContext.SendActivityAsync(MessageFactory.Text($"Dispatch unrecognized intent: {intent}."), cancellationToken);
break;
}
}
If method ProcessHomeAutomationAsync
or ProcessWeatherAsync
are invoked, they are passed the results from the dispatch model within luisResult.ConnectedServiceResult. The specified method then provides user feedback showing the dispatch model top intent, plus a ranked listing of all intents and entities that were detected.
If method q_sample-qna
is invoked, it uses the user input contained within the turnContext to generate an answer from the knowledge base and display that result to the user.
Note
If this were a production application, this is where the selected LUIS methods would connect to their specified service, pass in the user input, and process the returned LUIS intent and entity data.
Test your bot
Using your development environment, start the sample code. Note the localhost address shown in the address bar of the browser window opened by your App: "https://localhost:<Port_Number>".
Open Bot Framework Emulator, then select
Create a new bot configuration
. A.bot
file enables you to use the Inspector in the Emulator to see the JSON returned from LUIS and QnA Maker.In the New bot configuration dialog box, enter your bot name, and your endpoint URL, such as
http://localhost:3978/api/messages
. Save the file at the root of your bot sample code project.Open the bot file and add sections for your LUIS and QnA Maker apps. Use this example file as a template for settings. Save the changes.
Select the bot name in the My Bots list to access your running bot. For your reference, here are some of the questions and commands that are covered by the services built for your bot:
- QnA Maker
hi
,good morning
what are you
,what do you do
- LUIS (home automation)
turn on bedroom light
turn off bedroom light
make some coffee
- LUIS (weather)
whats the weather in redmond washington
what's the forecast for london
show me the forecast for nebraska
- QnA Maker
Dispatch for user utterance to QnA Maker
In the bot Emulator, enter the text
hi
and submit the utterance. The bot submits this query to the dispatch LUIS app and gets back a response indicating which child app should get this utterance for further processing.By selecting the
LUIS Trace
line in the log, you can see the LUIS response in the bot Emulator . The LUIS result from the dispatch LUIS app displays in the Inspector.{ "luisResponse": { "entities": [], "intents": [ { "intent": "q_sample-qna", "score": 0.9489713 }, { "intent": "l_HomeAutomation", "score": 0.0612499453 }, { "intent": "None", "score": 0.008567564 }, { "intent": "l_Weather", "score": 0.0025761195 } ], "query": "Hi", "topScoringIntent": { "intent": "q_sample-qna", "score": 0.9489713 } } }
Because the utterance,
hi
, is part of the dispatch LUIS app's q_sample-qna intent, and is selected as thetopScoringIntent
, the bot will make a second request, this time to the QnA Maker app, with the same utterance.Select the
QnAMaker Trace
line in the bot Emulator log. The QnA Maker result displays in the Inspector.
{
"questions": [
"hi",
"greetings",
"good morning",
"good evening"
],
"answer": "Hello!",
"score": 1,
"id": 96,
"source": "QnAMaker.tsv",
"metadata": [],
"context": {
"isContextOnly": false,
"prompts": []
}
}
Resolving incorrect top intent from Dispatch
Once your bot is running, it is possible to improve the bot's performance by removing similar or overlapping utterances between the dispatched apps.
You can use the Dispatch command-line tool to test and evaluate your dispatch model.
To update or create a new LUIS model
This sample is based on a preconfigured LUIS model. Additional information to help you update this model, or create a new LUIS model, can be found in the article Iterative app design for LUIS.
After updating the underlying models (QnA or LUIS) run dispatch refresh
to update your Dispatch LUIS app. dispatch refresh
is basically the same command as dispatch create
except no new LUIS app ID is created.
Note that utterances that were added directly in LUIS will not be retained when running dispatch refresh
. To keep those extra utterances in the Dispatch app add those utterances in a text file (one utterance per line), and then add the file to Dispatch by running the command:
dispatch add -t file -f <file path> --intentName <target intent name, ie l_General>
Once the file with extra utterances is added to Dispatch the utterances will stay with every refresh.
To delete resources
This sample creates a number of applications and resources that you can delete using the steps listed below, but you should not delete resources that any other apps or services rely on.
To delete LUIS resources:
- Sign in to the luis.ai portal.
- Go to the My Apps page.
- Select the apps created by this sample.
Home Automation
Weather
NLP-With-Dispatch-BotDispatch
- Click Delete, and click Ok to confirm.
To delete QnA Maker resources:
- Sign in to the qnamaker.ai portal.
- Go to the My knowledge bases page.
- Click the delete button for the
Sample QnA
knowledge base, and click Delete to confirm.
Best practice
To improve services used in this sample, refer to best practice for LUIS, and QnA Maker.