Translate user input to make your bot multilingual

[This topic is pre-release documentation and is subject to change.]

Your bot can use Microsoft Translator to automatically translate messages to the language your bot understands, and optionally translate the bot's replies back to the user's language. Adding translation to your bot allows it to reach a larger audience without changing significant parts of your bot's core programming.

In this tutorial we'll be using Microsoft Translator service for the translation, then adding it to a simple bot to show how it works.

Get a Text Services key

First, you'll need a key for using the Microsoft Translator service. You can get a free trial key in the Azure portal.

Installing Packages

Make sure you have the packages necessary to add translation to your bot.

Add a reference to the prerelease version of the following NuGet packages:

  • Microsoft.Bot.Builder.Integration.AspNet.Core
  • Microsoft.Bot.Builder.Ai.Translation (required for translation)

If you're going to combine translation with Language Understanding (LUIS), also add a reference to:

  • Microsoft.Bot.Builder.Ai.Luis (required for LUIS)

Configure translation

Next you can configure your bot to call the translator for every message received from a user, simply by adding it to your bot's middleware stack. The middleware uses the translation result to modify the user's message using the context object.

Start with the EchoBot sample in the v4 SDK, and update the ConfigureServices method in your Startup.cs file to add TranslationMiddleware to the bot. This configures your bot to translate every message received from a user.

  • Update your using statements.

  • Update your ConfigureServices method to include the translation middleware.

    The snippet here has been simplified by removing most of the comments and removing the exception handling middleware.

Startup.cs

using System;
using System.Collections.Generic;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.Bot.Builder.Ai.Translation;
using Microsoft.Bot.Builder.BotFramework;
using Microsoft.Bot.Builder.Core.Extensions;
using Microsoft.Bot.Builder.Integration.AspNet.Core;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
// This method gets called by the runtime. Use this method to add services to the container.
public void ConfigureServices(IServiceCollection services)
{
    services.AddBot<EchoBot>(options =>
    {
        options.CredentialProvider = new ConfigurationCredentialProvider(Configuration);
        var middleware = options.Middleware;
        // Add translation middleware
        // The first parameter is a list of languages the bot recognizes
        middleware.Add(new TranslationMiddleware(new string[] { "en" }, "<YOUR MICROSOFT TRANSLATOR API KEY>", false));

    });
}


Tip

The BotBuilder SDK automatically detects the user's language based on the message they just submitted. To override this functionality, you can supply additional callback parameters to add your own logic for detecting and changing the user's language.

Take a look at the code in EchoBot.cs, where it sends "You sent" followed by what the user says:

using Microsoft.Bot.Builder;
using Microsoft.Bot.Schema;
using System.Threading.Tasks;

namespace Microsoft.Bot.Samples
{
    public class EchoBot : IBot
    {
        public EchoBot() { }

        public async Task OnTurn(ITurnContext context)
        {
            switch (context.Activity.Type)
            {
                case ActivityTypes.Message:
                // echo back the user's input.
                    await context.SendActivity($"You sent '{context.Activity.Text}'");


                case ActivityTypes.ConversationUpdate:
                    foreach (var newMember in context.Activity.MembersAdded)
                    {
                        if (newMember.Id != context.Activity.Recipient.Id)
                        {
                            await context.SendActivity("Hello and welcome to the echo bot.");
                        }
                    }
                    break;
            }
        }
        
    }
}

When you add translation middleware, an optional parameter specifies whether to translate replies back to the user's language. In Startup.cs, we specified false to just translate the user messages to the bot's language.

// The first parameter is a list of languages the bot recognizes
// The second parameter is your API key
// The third parameter specifies whether to translate the bot's replies back to the user's language
middleware.Add(new TranslationMiddleware(new string[] { "en" }, "<YOUR MICROSOFT TRANSLATOR API KEY>", false));

Run the bot and see translated input

Run the bot, and type in a few messages in other languages. You'll see that the bot translated the user message and indicates the translation in its response.

bot detects language and translates input

Invoke logic in the bot's native language

Now, add logic that checks for English words. If the user says "help" or "cancel" in another language, the bot translates it into English and the logic that checks for the english words "help" or "cancel" is invoked.

In EchoBot.cs, update the case statement for message activities in your bot's OnTurn method.

case ActivityTypes.Message:
    // check the message text before calling context.SendActivity
    var messagetext = context.Activity.Text.Trim().ToLower();
    if (string.Equals("help", messagetext))
    {
        await context.SendActivity("You asked for help.");
    }

bot detects help in French

Translate replies back to the user's language

You can also translate replies back to the user's language, by setting the last constructor parameter to true.

In Startup.cs, update the following line of the ConfigureServices method.

// Use language recognition to detect the user's language from their message, instead of providing helper callbacks.
// Last parameter indicates that we'll translate replies back to the user's language
middleware.Add(new TranslationMiddleware(new string[] { "en" }, "TRANSLATION-SUBSCRIPTION-KEY", true));

Run the bot to see replies in the user's language

Run the bot, and type in a few messages in other languages. You'll see that it detects the user's language and translates the response.

bot detects language and translates response

Adding logic for detecting or changing the user language

Instead of letting the Botbuilder SDK automatically detect the user's language, you can provide callbacks to add your own logic for determining the user's language, or determining when the user's language has changed.

Tip

See the Translation sample in the SDK for a sample bot that implements callbacks for detecting and changing the user language.

In the following example, the CheckUserChangedLanguage callback checks for a specific user message to change the language.

In Startup.cs, add a callback to the translation middleware.

middleware.Add(new TranslationMiddleware(new string[] { "en" },
     "<YOUR MICROSOFT TRANSLATOR API KEY>", null,
     TranslatorLocaleHelper.GetActiveLanguage,
     TranslatorLocaleHelper.CheckUserChangedLanguage));

Add a TranslatorLocalHelper.cs file and add the following to the TranslatorLocalHelper class definition.

    class CurrentUserState
    {
        public string Language { get; set; }
    }

    public static void SetLanguage(ITurnContext context, string language) =>
        context.GetConversationState<CurrentUserState>().Language = language;

    public static bool IsSupportedLanguage(string language) =>
        _supportedLanguages.Contains(language);

    public static async Task<bool> CheckUserChangedLanguage(ITurnContext context)
    {
        bool changeLang = false;
        
        // use a specific message from user to change language
        if (context.Activity.Type == ActivityTypes.Message)
        {
            var messageActivity = context.Activity.AsMessageActivity();
            if (messageActivity.Text.ToLower().StartsWith("set my language to"))
            {
                changeLang = true;
            }
            if (changeLang)
            {
                var newLang = messageActivity.Text.ToLower().Replace("set my language to", "").Trim();
                if (!string.IsNullOrWhiteSpace(newLang)
                        && IsSupportedLanguage(newLang))
                {
                    SetLanguage(context, newLang);
                    await context.SendActivity($@"Changing your language to {newLang}");
                }
                else
                {
                    await context.SendActivity($@"{newLang} is not a supported language.");
                }
                // return true to intercept message from further processesing
                return true;
            }
        }

        return false;
    }

    public static string GetActiveLanguage(ITurnContext context)
    {

        if (context.Activity.Type == ActivityTypes.Message
            && context.GetConversationState<CurrentUserState>() != null
            && context.GetConversationState<CurrentUserState>().Language != null)
        {
            return context.GetConversationState<CurrentUserState>().Language;
        }

        return "en";
    }

Combining LUIS or QnA with translation

If you're combining translation with other services in your bot, like LUIS or QnA maker, add the translation middleware first, so that the messages are translated before passing them to other middleware that expects the bot's native language.

public void ConfigureServices(IServiceCollection services)
{
    services.AddSingleton(_ => Configuration);
    services.AddBot<AiBot>(options =>
    {
        options.CredentialProvider = new ConfigurationCredentialProvider(Configuration);
        var middleware = options.Middleware;

        // Add translation middleware
        // The first parameter is a list of languages the bot recognizes. Input in these language won't be translated.
        middleware.Add(new TranslationMiddleware(new string[] { "en" }, "<YOUR MICROSOFT TRANSLATOR API KEY>", false));

        // Add LUIS middleware after translation middleware to pass translated messages to LUIS
        middleware.Add(
            new LuisRecognizerMiddleware(
                new LuisModel("luisAppId", "subscriptionId", new Uri("luisModelBaseUrl"))));
    });
}

In your bot code, the LUIS results are based on input that has already been translated to the bot's native language. Try modifying the bot code to check the results of a LUIS app:

public async Task OnTurn(ITurnContext context)
{
    switch (context.Activity.Type)
    {
        case ActivityTypes.Message:
            // check LUIS results
            var luisResult = context.Services.Get<RecognizerResult>(LuisRecognizerMiddleware.LuisRecognizerResultKey);
            (string key, double score) topItem = luisResult.GetTopScoringIntent();
            await context.SendActivity($"The top intent was: '{topItem.key}', with score {topItem.score}");

            await context.SendActivity($"You sent '{context.Activity.Text}'");

        case ActivityTypes.ConversationUpdate:
            foreach (var newMember in context.Activity.MembersAdded)
            {
                if (newMember.Id != context.Activity.Recipient.Id)
                {
                    await context.SendActivity("Hello and welcome to the echo bot.");
                }
            }
            break;
        }
    }  
}          

Bypass translation for specified patterns

There may be certain words you don't want your bot to translate, such as proper names. You can provide regular expressions to indicate patterns that shouldn't be translated. For example, if the user says "My name is ..." in a non-native language for your bot, and you want to avoid translating their name, you can use a pattern to specify that.

In Startup.cs

// Pattern representing input to not translate
Dictionary<string, List<string>> patterns = new Dictionary<string, List<string>>();
// single pattern for fr language, to avoid translating what follows "my name is"
patterns.Add("fr", new List<string> { "mon nom est (.+)" });

middleware.Add(new TranslationMiddleware(new string[] { "en" },
    "<YOUR API KEY>", patterns,
    TranslatorLocaleHelper.GetActiveLanguage,
    TranslatorLocaleHelper.CheckUserChangedLanguage));

bot bypasses translation for pattern

Localize dates

If need to localize dates, you can add LocaleConverterMiddleware. For example, if you know that your bot expects dates in the format MM/DD/YYYY, and users in other locales might enter dates in the format DD/MM/YYYY, the locale converter middleware can automatically convert dates to the format your bot expects.

Note

The locale converter middleware is intended to convert only dates. It has no knowledge of the results of translation middleware. If you're using translation middleware, be careful how you combine it with locale converter. Translation middleware will translate some dates that are in textual format along with other text input, but it doesn't translate dates

For example, the following image shows a bot that echos back the user input, after translating from English to French. It uses TranslationMiddleware without using LocaleConverterMiddleware.

bot translating dates without date conversion

The following shows the same bot if the LocaleConverterMiddleware is added.

bot translating dates without date conversion

Locale converters can support English, French, German, and Chinese locales.

In Startup.cs

// Add locale converter middleware
middleware.Add(new LocaleConverterMiddleware(
    TranslatorLocaleHelper.GetActiveLocale,
    TranslatorLocaleHelper.CheckUserChangedLocale,
     "en-us", LocaleConverter.Converter));

In TranslatorLocaleHelper.cs

// TranslatorLocaleHelper.cs
public static async Task<bool> CheckUserChangedLocale(ITurnContext context)
{
    bool changeLocale = false;//logic implemented by developper to make a signal for language changing 
    //use a specific message from user to change language
    if (context.Activity.Type == ActivityTypes.Message)
    {
        var messageActivity = context.Activity.AsMessageActivity();
        if (messageActivity.Text.ToLower().StartsWith("set my locale to"))
        {
            changeLocale = true;
        }
        if (changeLocale)
        {
            var newLocale = messageActivity.Text.ToLower().Replace("set my locale to", "").Trim(); //extracted by the user using user state 
            if (!string.IsNullOrWhiteSpace(newLocale)
                    && IsSupportedLanguage(newLocale))
            {
                SetLocale(context, newLocale);
                await context.SendActivity($@"Changing your language to {newLocale}");
            }
            else
            {
                await context.SendActivity($@"{newLocale} is not a supported locale.");
            }
            //intercepts message
            return true;
        }
    }

    return false;
}
public static string GetActiveLocale(ITurnContext context)
{
    if (currentLocale != null)
    {
        //the user has specified a different locale so update the bot state
        if (context.GetConversationState<CurrentUserState>() != null
            && currentLocale != context.GetConversationState<CurrentUserState>().Locale)
        {
            SetLocale(context, currentLocale);
        }
    }
    if (context.Activity.Type == ActivityTypes.Message
        && context.GetConversationState<CurrentUserState>() != null && context.GetConversationState<CurrentUserState>().Locale != null)
    {
        return context.GetConversationState<CurrentUserState>().Locale;
    }

    return "en-us";
}