Review Endpoint Utterances

The breakthrough feature of LUIS is active learning. Once your LUIS has endpoint queries, LUIS uses active learning to improve the quality of the results. In the active learning process, LUIS examines all the endpoint utterances, and selects utterances that it is unsure of. If you label these utterances, train, and publish, then LUIS identifies utterances more accurately.

Endpoint utterances are taken from end-user queries on the application’s HTTP endpoint. If your app is not published or has not received hits yet, you do not have any utterances to review. If no endpoint hits are received aligned with a specific intent or entity, you do not have utterances to review that contain them.

Filter utterances

  1. Open your app (for example, TravelAgent) by clicking its name on My Apps page, then click Build in the top bar.

  2. Under the Improve app performance, select Review endpoint utterances.

    Review utterances

  3. On the Review endpoint utterances page, click in the Filter list by intent or entity text box. This drop-down list includes all intents under INTENTS and all entities under ENTITIES.

    Utterances filter

  4. Select a category (intents or entities) in the drop-down list and review the utterances.

    Intent utterances

Label entities

LUIS replaces entity tokens (words) with entity names highlighted in blue. If an utterance has unlabeled entities, you can label them in the utterance.

  1. Click on the word(s) in the utterance.

  2. Select an entity from the list.

    Label entity

Align single utterance

Each utterance has a suggested intent displayed in the Aligned intent column.

  1. If you agree with that suggestion, click on the check mark.

    Keep aligned intent

  2. If you disagree with the suggestion, select the correct intent from the aligned intent drop-down list, then click on the check mark to the right of the aligned intent.

    Align intent

  3. After you click on the check mark, the utterance is removed from the list.

Align several utterances

To align several utterances, check the box to the left of the utterances, then click on the Add selected button.

Align several

Verify aligned intent

You can verify the utterance was aligned with the correct intent by going to the Intents page, select the intent name, and reviewing the utterances. The utterance from Review endpoint utterances is in the list.

Delete utterance

Each utterance can be deleted from the review list. Once deleted, it will not appear in the list again. This is true even if the user enters the same utterance from the endpoint.

If you are unsure if you should delete the utterance, either move it to the None intent, or create a new intent such as "miscellaneous" and move the utterance to that intent.

Delete several utterances

To delete several utterances, select each item and click on the trash bin to the right of the Add selected button.

Delete several

Next steps

To test how performance improves after you label suggested utterances, you can access the test console by clicking Test in the top panel. For instructions on how to test your app using the test console, see Train and test your app.