Install and run LUIS docker containers

The Language Understanding (LUIS) container loads your trained or published Language Understanding model, also know as a LUIS app, into a docker container and provides access to the query predictions from the container's API endpoints. You can collect query logs from the container and upload these back to the Language Understanding app to improve the app's prediction accuracy.

The following video demonstrates using this container.

Container demonstration for Cognitive Services

If you don't have an Azure subscription, create a free account before you begin.

Prerequisites

In order to run the LUIS container, you must have the following:

Required Purpose
Docker Engine You need the Docker Engine installed on a host computer. Docker provides packages that configure the Docker environment on macOS, Windows, and Linux. For a primer on Docker and container basics, see the Docker overview.

Docker must be configured to allow the containers to connect with and send billing data to Azure.

On Windows, Docker must also be configured to support Linux containers.

Familiarity with Docker You should have a basic understanding of Docker concepts, like registries, repositories, containers, and container images, as well as knowledge of basic docker commands.
Azure Cognitive Services resource and LUIS packaged app file In order to use the container, you must have:

* A Cognitive Services Azure resource and the associated billing key the billing endpoint URI. Both values are available on the Overview and Keys pages for the resource and are required to start the container. You need to add the luis/v2.0 routing to the endpoint URI as shown in the following BILLING_ENDPOINT_URI example.
* A trained or published app packaged as a mounted input to the container with its associated App ID. You can get the packaged file from the LUIS portal or the Authoring APIs. If you are getting LUIS packaged app from the authoring APIs, you will also need your Authoring Key.

These requirements are used to pass command-line arguments to the following variables:

{AUTHORING_KEY}: This key is used to get the packaged app from the LUIS service in the cloud and upload the query logs back to the cloud. The format is xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.

{APPLICATION_ID}: This ID is used to select the App. The format is xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.

{API_KEY}: This key is used to start the container. You can find the endpoint key in two places. The first is the Azure portal within the Cognitive Services resource's keys list. The endpoint key is also available in the LUIS portal on the Keys and Endpoint settings page. Do not use the starter key.

{ENDPOINT_URI}: The endpoint as provided on the Overview page.

The authoring key and endpoint key have different purposes. Do not use them interchangeably.

Authoring APIs for package file

Authoring APIs for packaged apps:

The host computer

The host is a x64-based computer that runs the Docker container. It can be a computer on your premises or a Docker hosting service in Azure, such as:

Container requirements and recommendations

This container supports minimum and recommended values for the settings:

Container Minimum Recommended TPS
(Minimum, Maximum)
LUIS 1 core, 2-GB memory 1 core, 4-GB memory 20,40
  • Each core must be at least 2.6 gigahertz (GHz) or faster.
  • TPS - transactions per second

Core and memory correspond to the --cpus and --memory settings, which are used as part of the docker run command.

Get the container image with docker pull

Use the docker pull command to download a container image from the mcr.microsoft.com/azure-cognitive-services/luis repository:

docker pull mcr.microsoft.com/azure-cognitive-services/luis:latest

Use the docker pull command to download a container image.

For a full description of available tags, such as latest used in the preceding command, see LUIS on Docker Hub.

Tip

You can use the docker images command to list your downloaded container images. For example, the following command lists the ID, repository, and tag of each downloaded container image, formatted as a table:

docker images --format "table {{.ID}}\t{{.Repository}}\t{{.Tag}}"

IMAGE ID         REPOSITORY                TAG
<image-id>       <repository-path/name>    <tag-name>

How to use the container

Once the container is on the host computer, use the following process to work with the container.

Process for using Language Understanding (LUIS) container

  1. Export package for container from LUIS portal or LUIS APIs.
  2. Move package file into the required input directory on the host computer. Do not rename, alter, overwrite, or decompress LUIS package file.
  3. Run the container, with the required input mount and billing settings. More examples of the docker run command are available.
  4. Querying the container's prediction endpoint.
  5. When you are done with the container, import the endpoint logs from the output mount in the LUIS portal and stop the container.
  6. Use LUIS portal's active learning on the Review endpoint utterances page to improve the app.

The app running in the container can't be altered. In order the change the app in the container, you need to change the app in the LUIS service using the LUIS portal or use the LUIS authoring APIs. Then train and/or publish, then download a new package and run the container again.

The LUIS app inside the container can't be exported back to the LUIS service. Only the query logs can be uploaded.

Export packaged app from LUIS

The LUIS container requires a trained or published LUIS app to answer prediction queries of user utterances. In order to get the LUIS app, use either the trained or published package API.

The default location is the input subdirectory in relation to where you run the docker run command.

Place the package file in a directory and reference this directory as the input mount when you run the docker container.

Package types

The input mount directory can contain the Production, Staging, and Trained versions of the app simultaneously. All the packages are mounted.

Package Type Query Endpoint API Query availability Package filename format
Trained Get, Post Container only {APPLICATION_ID}_v{APPLICATION_VERSION}.gz
Staging Get, Post Azure and container {APPLICATION_ID}_STAGING.gz
Production Get, Post Azure and container {APPLICATION_ID}_PRODUCTION.gz

Important

Do not rename, alter, overwrite, or decompress the LUIS package files.

Packaging prerequisites

Before packaging a LUIS application, you must have the following:

Packaging Requirements Details
Azure Cognitive Services resource instance Supported regions include

West US (westus)
West Europe (westeurope)
Australia East (australiaeast)
Trained or published LUIS app With no unsupported dependencies.
Access to the host computer's file system The host computer must allow an input mount.

Export app package from LUIS portal

The LUIS portal provides the ability to export the trained or published app's package.

Export published app's package from LUIS portal

The published app's package is available from the My Apps list page.

  1. Sign on to the LUIS portal.
  2. Select the checkbox to the left of the app name in the list.
  3. Select the Export item from the contextual toolbar above the list.
  4. Select Export for container (GZIP).
  5. Select the environment of Production slot or Staging slot.
  6. The package is downloaded from the browser.

Export the published package for the container from the App page's Export menu

Export trained app's package from LUIS portal

The trained app's package is available from the Versions list page.

  1. Sign on to the LUIS portal.
  2. Select the app in the list.
  3. Select Manage in the app's navigation bar.
  4. Select Versions in the left navigation bar.
  5. Select the checkbox to the left of the version name in the list.
  6. Select the Export item from the contextual toolbar above the list.
  7. Select Export for container (GZIP).
  8. The package is downloaded from the browser.

Export the trained package for the container from the Versions page's Export menu

Export published app's package from API

Use the following REST API method, to package a LUIS app that you've already published. Substituting your own appropriate values for the placeholders in the API call, using the table below the HTTP specification.

GET /luis/api/v2.0/package/{APPLICATION_ID}/slot/{APPLICATION_ENVIRONMENT}/gzip HTTP/1.1
Host: {AZURE_REGION}.api.cognitive.microsoft.com
Ocp-Apim-Subscription-Key: {AUTHORING_KEY}
Placeholder Value
{APPLICATION_ID} The application ID of the published LUIS app.
{APPLICATION_ENVIRONMENT} The environment of the published LUIS app. Use one of the following values:
PRODUCTION
STAGING
{AUTHORING_KEY} The authoring key of the LUIS account for the published LUIS app.
You can get your authoring key from the User Settings page on the LUIS portal.
{AZURE_REGION} The appropriate Azure region:

westus - West US
westeurope - West Europe
australiaeast - Australia East

To download the published package, please refer to the API documentation here. If successfully downloaded, the response is a LUIS package file. Save the file in the storage location specified for the input mount of the container.

Export trained app's package from API

Use the following REST API method, to package a LUIS application that you've already trained. Substituting your own appropriate values for the placeholders in the API call, using the table below the HTTP specification.

GET /luis/api/v2.0/package/{APPLICATION_ID}/versions/{APPLICATION_VERSION}/gzip HTTP/1.1
Host: {AZURE_REGION}.api.cognitive.microsoft.com
Ocp-Apim-Subscription-Key: {AUTHORING_KEY}
Placeholder Value
{APPLICATION_ID} The application ID of the trained LUIS application.
{APPLICATION_VERSION} The application version of the trained LUIS application.
{AUTHORING_KEY} The authoring key of the LUIS account for the published LUIS app.
You can get your authoring key from the User Settings page on the LUIS portal.
{AZURE_REGION} The appropriate Azure region:

westus - West US
westeurope - West Europe
australiaeast - Australia East

To download the trained package, please refer to the API documentation here. If successfully downloaded, the response is a LUIS package file. Save the file in the storage location specified for the input mount of the container.

Run the container with docker run

Use the docker run command to run the container. The command uses the following parameters:

Placeholder Value
{API_KEY} This key is used to start the container. Do not use the starter key.
{ENDPOINT_URI} The endpoint value is available on the Azure portal's Cognitive Services Overview page.

Replace these parameters with your own values in the following example docker run command. Run the command in the Windows console.

docker run --rm -it -p 5000:5000 ^
--memory 4g ^
--cpus 2 ^
--mount type=bind,src=c:\input,target=/input ^
--mount type=bind,src=c:\output\,target=/output ^
mcr.microsoft.com/azure-cognitive-services/luis ^
Eula=accept ^
Billing={ENDPOINT_URI} ^
ApiKey={API_KEY}
  • This example uses the directory off the C: drive to avoid any permission conflicts on Windows. If you need to use a specific directory as the input directory, you may need to grant the docker service permission.
  • Do not change the order of the arguments unless you are familiar with docker containers.
  • If you are using a different operating system, use the correct console/terminal, folder syntax for mounts, and line continuation character for your system. These examples assume a Windows console with a line continuation character ^. Because the container is a Linux operating system, the target mount uses a Linux-style folder syntax.

This command:

  • Runs a container from the LUIS container image
  • Loads LUIS app from input mount at c:\input, located on container host
  • Allocates two CPU cores and 4 gigabytes (GB) of memory
  • Exposes TCP port 5000 and allocates a pseudo-TTY for the container
  • Saves container and LUIS logs to output mount at c:\output, located on container host
  • Automatically removes the container after it exits. The container image is still available on the host computer.

More examples of the docker run command are available.

Important

The Eula, Billing, and ApiKey options must be specified to run the container; otherwise, the container won't start. For more information, see Billing. The ApiKey value is the Key from the Keys and Endpoints page in the LUIS portal and is also available on the Azure Cognitive Services resource keys page.

Run multiple containers on the same host

If you intend to run multiple containers with exposed ports, make sure to run each container with a different exposed port. For example, run the first container on port 5000 and the second container on port 5001.

You can have this container and a different Azure Cognitive Services container running on the HOST together. You also can have multiple containers of the same Cognitive Services container running.

Endpoint APIs supported by the container

Both V2 and V3 (Preview) versions of the API are available with the container.

Query the container's prediction endpoint

The container provides REST-based query prediction endpoint APIs. Endpoints for published (staging or production) apps have a different route than endpoints for trained apps.

Use the host, https://localhost:5000, for container APIs.

Package type Method Route Query parameters
Published Get, Post /luis/v2.0/apps/{appId}? q={q}
&staging
[&timezoneOffset]
[&verbose]
[&log]
Trained Get, Post /luis/v2.0/apps/{appId}/versions/{versionId}? q={q}
[&timezoneOffset]
[&verbose]
[&log]

The query parameters configure how and what is returned in the query response:

Query parameter Type Purpose
q string The user's utterance.
timezoneOffset number The timezoneOffset allows you to change the timezone used by the prebuilt entity datetimeV2.
verbose boolean Returns all intents and their scores when set to true. Default is false, which returns only the top intent.
staging boolean Returns query from staging environment results if set to true.
log boolean Logs queries, which can be used later for active learning. Default is true.

Query published app

An example CURL command for querying the container for a published app is:

curl -X GET \
"http://localhost:5000/luis/v2.0/apps/{APPLICATION_ID}?q=turn%20on%20the%20lights&staging=false&timezoneOffset=0&verbose=false&log=true" \
-H "accept: application/json"

To make queries to the Staging environment, change the staging query string parameter value to true:

staging=true

Query trained app

An example CURL command for querying the container for a trained app is:

curl -X GET \
"http://localhost:5000/luis/v2.0/apps/{APPLICATION_ID}/versions/{APPLICATION_VERSION}?q=turn%20on%20the%20lights&timezoneOffset=0&verbose=false&log=true" \
-H "accept: application/json"

The version name has a maximum of 10 characters and contains only characters allowed in a URL.

Import the endpoint logs for active learning

If an output mount is specified for the LUIS container, app query log files are saved in the output directory, where {INSTANCE_ID} is the container ID. The app query log contains the query, response, and timestamps for each prediction query submitted to the LUIS container.

The following location shows the nested directory structure for the container's log files.

/output/luis/{INSTANCE_ID}/

From the LUIS portal, select your app, then select Import endpoint logs to upload these logs.

Import container's log files for active learning

After the log is uploaded, review the endpoint utterances in the LUIS portal.

Validate that a container is running

There are several ways to validate that the container is running.

Request Purpose
http://localhost:5000/ The container provides a home page.
http://localhost:5000/status Requested with GET, to validate that the container is running without causing an endpoint query. This request can be used for Kubernetes liveness and readiness probes.
http://localhost:5000/swagger The container provides a full set of documentation for the endpoints and a Try it now feature. With this feature, you can enter your settings into a web-based HTML form and make the query without having to write any code. After the query returns, an example CURL command is provided to demonstrate the HTTP headers and body format that's required.

Container's home page

Stop the container

To shut down the container, in the command-line environment where the container is running, press Ctrl+C.

Troubleshooting

If you run the container with an output mount and logging enabled, the container generates log files that are helpful to troubleshoot issues that happen while starting or running the container.

Billing

The LUIS container sends billing information to Azure, using a Cognitive Services resource on your Azure account.

Queries to the container are billed at the pricing tier of the Azure resource that's used for the <ApiKey>.

Azure Cognitive Services containers aren't licensed to run without being connected to the billing endpoint for metering. You must enable the containers to communicate billing information with the billing endpoint at all times. Cognitive Services containers don't send customer data, such as the image or text that's being analyzed, to Microsoft.

Connect to Azure

The container needs the billing argument values to run. These values allow the container to connect to the billing endpoint. The container reports usage about every 10 to 15 minutes. If the container doesn't connect to Azure within the allowed time window, the container continues to run but doesn't serve queries until the billing endpoint is restored. The connection is attempted 10 times at the same time interval of 10 to 15 minutes. If it can't connect to the billing endpoint within the 10 tries, the container stops running.

Billing arguments

For the docker run command to start the container, all three of the following options must be specified with valid values:

Option Description
ApiKey The API key of the Cognitive Services resource that's used to track billing information.
The value of this option must be set to an API key for the provisioned resource that's specified in Billing.
Billing The endpoint of the Cognitive Services resource that's used to track billing information.
The value of this option must be set to the endpoint URI of a provisioned Azure resource.
Eula Indicates that you accepted the license for the container.
The value of this option must be set to accept.

For more information about these options, see Configure containers.

Supported dependencies for latest container

The latest container, released at 2019 //Build, will support:

  • Bing spell check: requests to the query prediction endpoint with the &spellCheck=true&bing-spell-check-subscription-key={bingKey} query string parameters. Use the Bing Spell Check v7 tutorial to learn more. If this feature is used, the container sends the utterance to your Bing Spell Check V7 resource.
  • New prebuilt domains: these enterprise-focused domains include entities, example utterances, and patterns. Extend these domains for your own use.

Unsupported dependencies for latest container

If your LUIS app has unsupported dependencies, you won't be able to export for container until you remove the unsupported features. When you attempt to export for container, the LUIS portal reports the unsupported features you need to remove.

You can use a LUIS application if it doesn't include any of the following dependencies:

Unsupported app configurations Details
Unsupported container cultures Dutch (nl-NL)
Japanese (ja-JP)
German is only supported with the 1.0.2 tokenizer.
Unsupported entities for all cultures KeyPhrase prebuilt entity for all cultures
Unsupported entities for English (en-US) culture GeographyV2 prebuilt entities
Speech priming External dependencies are not supported in the container.
Sentiment analysis External dependencies are not supported in the container.

Blog posts

Developer samples

Developer samples are available at our GitHub repository.

View webinar

Join the webinar to learn about:

  • How to deploy Cognitive Services to any machine using Docker
  • How to deploy Cognitive Services to AKS

Summary

In this article, you learned concepts and workflow for downloading, installing, and running Language Understanding (LUIS) containers. In summary:

  • Language Understanding (LUIS) provides one Linux container for Docker providing endpoint query predictions of utterances.
  • Container images are downloaded from the Microsoft Container Registry (MCR).
  • Container images run in Docker.
  • You can use REST API to query the container endpoints by specifying the host URI of the container.
  • You must specify billing information when instantiating a container.

Important

Cognitive Services containers are not licensed to run without being connected to Azure for metering. Customers need to enable the containers to communicate billing information with the metering service at all times. Cognitive Services containers do not send customer data (for example, the image or text that is being analyzed) to Microsoft.

Next steps