Edit

Continuous Integration and Continuous Delivery workflows for LUIS DevOps

  • Article

Important

LUIS will be retired on October 1st 2025 and starting April 1st 2023 you will not be able to create new LUIS resources. We recommend migrating your LUIS applications to conversational language understanding to benefit from continued product support and multilingual capabilities.

Software engineers who are developing a Language Understanding (LUIS) app can apply DevOps practices around source control, automated builds, testing, and release management. This article describes concepts for implementing automated builds for LUIS.

Build automation workflows for LUIS

CI workflows

In your source code management (SCM) system, configure automated build pipelines to run at the following events:

  1. PR workflow triggered when a pull request (PR) is raised. This workflow validates the contents of the PR before the updates get merged into the main branch.
  2. CI/CD workflow triggered when updates are pushed to the main branch, for example upon merging the changes from a PR. This workflow ensures the quality of all updates to the main branch.

The CI/CD workflow combines two complementary development processes:

  • Continuous Integration (CI) is the engineering practice of frequently committing code in a shared repository, and performing an automated build on it. Paired with an automated testing approach, continuous integration allows us to verify that for each update, the LUDown source is still valid and can be imported into a LUIS app, but also that it passes a group of tests that verify the trained app can recognize the intents and entities required for your solution.

  • Continuous Delivery (CD) takes the Continuous Integration concept further to automatically deploy the application to an environment where you can do more in-depth testing. CD enables us to learn early about any unforeseen issues that arise from our changes as quickly as possible, and also to learn about gaps in our test coverage.

The goal of continuous integration and continuous delivery is to ensure that "main is always shippable,". For a LUIS app, this means that we could, if we needed to, take any version from the main branch LUIS app and ship it on production.

Tools for building automation workflows for LUIS

Tip

You can find a complete solution for implementing DevOps in the LUIS DevOps template repo.

There are different build automation technologies available to create build automation workflows. All of them require that you can script steps using a command-line interface (CLI) or REST calls so that they can execute on a build server.

Use the following tools for building automation workflows for LUIS:

The PR workflow

As mentioned, you configure this workflow to run when a developer raises a PR to propose changes to be merged from a feature branch into the main branch. Its purpose is to verify the quality of the changes in the PR before they're merged to the main branch.

This workflow should:

  • Create a temporary LUIS app by importing the .lu source in the PR.
  • Train and publish the LUIS app version.
  • Run all the unit tests against it.
  • Pass the workflow if all the tests pass, otherwise fail it.
  • Clean up and delete the temporary app.

If supported by your SCM, configure branch protection rules so that this workflow must complete successfully before the PR can be completed.

The main branch CI/CD workflow

Configure this workflow to run after the updates in the PR have been merged into the main branch. Its purpose is to keep the quality bar for your main branch high by testing the updates. If the updates meet the quality bar, this workflow deploys the new LUIS app version to an environment where you can do more in-depth testing.

This workflow should:

  • Build a new version in your primary LUIS app (the app you maintain for the main branch) using the updated source code.

  • Train and publish the LUIS app version.

    Note

    As explained in Running tests in an automated build workflow you must publish the LUIS app version under test so that tools such as NLU.DevOps can access it. LUIS only supports two named publication slots, staging and production for a LUIS app, but you can also publish a version directly and query by version. Use direct version publishing in your automation workflows to avoid being limited to using the named publishing slots.

  • Run all the unit tests.

  • Optionally run batch tests to measure the quality and accuracy of the LUIS app version and compare it to some baseline.

  • If the tests complete successfully:

    • Tag the source in the repo.
    • Run the Continuous Delivery (CD) job to deploy the LUIS app version to environments for further testing.

Continuous delivery (CD)

The CD job in a CI/CD workflow runs conditionally on success of the build and automated unit tests. Its job is to automatically deploy the LUIS application to an environment where you can do more testing.

There's no one recommended solution on how best to deploy your LUIS app, and you must implement the process that is appropriate for your project. The LUIS DevOps template repo implements a simple solution for this which is to publish the new LUIS app version to the production publishing slot. This is fine for a simple setup. However, if you need to support a number of different production environments at the same time, such as development, staging and UAT, then the limit of two named publishing slots per app will prove insufficient.

Other options for deploying an app version include:

  • Leave the app version published to the direct version endpoint and implement a process to configure downstream production environments with the direct version endpoint as required.
  • Maintain different LUIS apps for each production environments and write automation steps to import the .lu into a new version in the LUIS app for the target production environment, to train, and publish it.
  • Export the tested LUIS app version into a LUIS docker container and deploy the LUIS container to Azure Container instances.

Release management

Generally we recommend that you do continuous delivery only to your non-production environments, such as to development and staging. Most teams require a manual review and approval process for deployment to a production environment. For a production deployment, you might want to make sure it happens when key people on the development team are available for support, or during low-traffic periods.

Apply DevOps to LUIS app development using GitHub Actions

Go to the LUIS DevOps template repo for a complete solution that implements DevOps and software engineering best practices for LUIS. You can use this template repo to create your own repository with built-in support for CI/CD workflows and practices that enable source control, automated builds, testing, and release management with LUIS for your own project.

The LUIS DevOps template repo walks through how to:

Next steps