Quickstart: Recognize speech, intents, and entities with Language Understanding (LUIS)

In this quickstart, you'll use the Speech SDK and the Language Understanding (LUIS) service to recognize intents from audio data captured from a microphone. Specifically, you'll use the Speech SDK to capture speech, and a prebuilt domain from LUIS to identify intents for home automation, like turning on and off a light.

After satisfying a few prerequisites, recognizing speech and identifying intents from a microphone only takes a few steps:

  • Create a SpeechConfig object from your subscription key and region.
  • Create an IntentRecognizer object using the SpeechConfig object from above.
  • Using the IntentRecognizer object, start the recognition process for a single utterance.
  • Inspect the IntentRecognitionResult returned.

If you prefer to jump right in, view or download all Speech SDK C# Samples on GitHub. Otherwise, let's get started.

Prerequisites

Before you get started:

Create a LUIS app for intent recognition

To complete the intent recognition quickstart, you'll need to create a LUIS account and a project using the LUIS preview portal. This quickstart only requires a LUIS subscription. A Speech service subscription isn't required.

The first thing you'll need to do is create a LUIS account and app using the LUIS preview portal. The LUIS app that you create will use a prebuilt domain for home automation, which provides intents, entities, and example utterances. When you're finished, you'll have a LUIS endpoint running in the cloud that you can call using the Speech SDK.

Follow these instructions to create your LUIS app:

When you're done, you'll need three things:

  • Your LUIS key
  • Your LUIS region
  • Your LUIS app ID

Here's where you can find this information in the LUIS preview portal:

  1. From the LUIS preview portal, select Manage, then select Azure Resources. On this page, you'll find your LUIS key and location (sometimes referred to as region).

    LUIS key and location

  2. After you've got your key and location, you'll need the app ID. Select Application Settings -- your app ID is available on this page.

    LUIS app ID

Open your project in Visual Studio

Next, open your project in Visual Studio.

  1. Launch Visual Studio 2019.
  2. Load your project and open Program.cs.

Start with some boilerplate code

Let's add some code that works as a skeleton for our project. Make note that you've created an async method called RecognizeIntentAsync().

using System;
using System.Threading.Tasks;
using Microsoft.CognitiveServices.Speech;
using Microsoft.CognitiveServices.Speech.Intent;

namespace helloworld
{
    class Program
    {
        public static async Task RecognizeIntentAsync()
        {
        }

        static void Main()
        {
            RecognizeIntentAsync().Wait();
            Console.WriteLine("Please press <Return> to continue.");
            Console.ReadLine();
        }
    }
}

Create a Speech configuration

Before you can initialize an IntentRecognizer object, you need to create a configuration that uses the key and location for your LUIS prediction resource.

Important

Your starter key and authoring keys will not work. You must use your prediction key and location that you created earlier. For more information, see Create a LUIS app for intent recognition.

Insert this code in the RecognizeIntentAsync() method. Make sure you update these values:

  • Replace "YourLanguageUnderstandingSubscriptionKey" with your LUIS prediction key.
  • Replace "YourLanguageUnderstandingServiceRegion" with your LUIS location.

Tip

If you need help finding these values, see Create a LUIS app for intent recognition.

var config = SpeechConfig.FromSubscription("YourLanguageUnderstandingSubscriptionKey", "YourLanguageUnderstandingServiceRegion");

This sample uses the FromSubscription() method to build the SpeechConfig. For a full list of available methods, see SpeechConfig Class.

The Speech SDK will default to recognizing using en-us for the language, see Specify source language for speech to text for information on choosing the source language.

Initialize an IntentRecognizer

Now, let's create an IntentRecognizer. This object is created inside of a using statement to ensure the proper release of unmanaged resources. Insert this code in the RecognizeIntentAsync() method, right below your Speech configuration.

// Creates an intent recognizer using microphone as audio input.
using (var recognizer = new IntentRecognizer(config))
{
}

Add a LanguageUnderstandingModel and intents

You need to associate a LanguageUnderstandingModel with the intent recognizer, and add the intents that you want recognized. We're going to use intents from the prebuilt domain for home automation. Insert this code in the using statement from the previous section. Make sure that you replace "YourLanguageUnderstandingAppId" with your LUIS app ID.

Tip

If you need help finding this value, see Create a LUIS app for intent recognition.

// Creates a Language Understanding model using the app id, and adds specific intents from your model
var model = LanguageUnderstandingModel.FromAppId("YourLanguageUnderstandingAppId");
recognizer.AddIntent(model, "YourLanguageUnderstandingIntentName1", "id1");
recognizer.AddIntent(model, "YourLanguageUnderstandingIntentName2", "id2");
recognizer.AddIntent(model, "YourLanguageUnderstandingIntentName3", "any-IntentId-here");

Recognize an intent

From the IntentRecognizer object, you're going to call the RecognizeOnceAsync() method. This method lets the Speech service know that you're sending a single phrase for recognition, and that once the phrase is identified to stop recognizing speech.

Inside the using statement, add this code below your model:

var result = await recognizer.RecognizeOnceAsync().ConfigureAwait(false);

Display recognition results (or errors)

When the recognition result is returned by the Speech service, you'll want to do something with it. We're going to keep it simple and print the results to console.

Inside the using statement, below RecognizeOnceAsync(), add this code:

// Checks result.
if (result.Reason == ResultReason.RecognizedIntent)
{
    Console.WriteLine($"RECOGNIZED: Text={result.Text}");
    Console.WriteLine($"    Intent Id: {result.IntentId}.");
    Console.WriteLine($"    Language Understanding JSON: {result.Properties.GetProperty(PropertyId.LanguageUnderstandingServiceResponse_JsonResult)}.");
}
else if (result.Reason == ResultReason.RecognizedSpeech)
{
    Console.WriteLine($"RECOGNIZED: Text={result.Text}");
    Console.WriteLine($"    Intent not recognized.");
}
else if (result.Reason == ResultReason.NoMatch)
{
    Console.WriteLine($"NOMATCH: Speech could not be recognized.");
}
else if (result.Reason == ResultReason.Canceled)
{
    var cancellation = CancellationDetails.FromResult(result);
    Console.WriteLine($"CANCELED: Reason={cancellation.Reason}");

    if (cancellation.Reason == CancellationReason.Error)
    {
        Console.WriteLine($"CANCELED: ErrorCode={cancellation.ErrorCode}");
        Console.WriteLine($"CANCELED: ErrorDetails={cancellation.ErrorDetails}");
        Console.WriteLine($"CANCELED: Did you update the subscription info?");
    }
}

Check your code

At this point, your code should look like this:

Note

We've added some comments to this version.


// <code>
using System;
using System.Threading.Tasks;
using Microsoft.CognitiveServices.Speech;
using Microsoft.CognitiveServices.Speech.Intent;

namespace helloworld
{
    class Program
    {
        public static async Task RecognizeIntentAsync()
        {
            // Creates an instance of a speech config with specified subscription key
            // and service region. Note that in contrast to other services supported by
            // the Cognitive Services Speech SDK, the Language Understanding service
            // requires a specific subscription key from https://www.luis.ai/.
            // The Language Understanding service calls the required key 'endpoint key'.
            // Once you've obtained it, replace with below with your own Language Understanding subscription key
            // and service region (e.g., "westus").
            // The default language is "en-us".
            var config = SpeechConfig.FromSubscription("YourLanguageUnderstandingSubscriptionKey", "YourLanguageUnderstandingServiceRegion");

            // Creates an intent recognizer using microphone as audio input.
            using (var recognizer = new IntentRecognizer(config))
            {
                // Creates a Language Understanding model using the app id, and adds specific intents from your model
                var model = LanguageUnderstandingModel.FromAppId("YourLanguageUnderstandingAppId");
                recognizer.AddIntent(model, "YourLanguageUnderstandingIntentName1", "id1");
                recognizer.AddIntent(model, "YourLanguageUnderstandingIntentName2", "id2");
                recognizer.AddIntent(model, "YourLanguageUnderstandingIntentName3", "any-IntentId-here");

                // Starts recognizing.
                Console.WriteLine("Say something...");

                // Starts intent recognition, and returns after a single utterance is recognized. The end of a
                // single utterance is determined by listening for silence at the end or until a maximum of 15
                // seconds of audio is processed.  The task returns the recognition text as result. 
                // Note: Since RecognizeOnceAsync() returns only a single utterance, it is suitable only for single
                // shot recognition like command or query. 
                // For long-running multi-utterance recognition, use StartContinuousRecognitionAsync() instead.
                var result = await recognizer.RecognizeOnceAsync().ConfigureAwait(false);

                // Checks result.
                if (result.Reason == ResultReason.RecognizedIntent)
                {
                    Console.WriteLine($"RECOGNIZED: Text={result.Text}");
                    Console.WriteLine($"    Intent Id: {result.IntentId}.");
                    Console.WriteLine($"    Language Understanding JSON: {result.Properties.GetProperty(PropertyId.LanguageUnderstandingServiceResponse_JsonResult)}.");
                }
                else if (result.Reason == ResultReason.RecognizedSpeech)
                {
                    Console.WriteLine($"RECOGNIZED: Text={result.Text}");
                    Console.WriteLine($"    Intent not recognized.");
                }
                else if (result.Reason == ResultReason.NoMatch)
                {
                    Console.WriteLine($"NOMATCH: Speech could not be recognized.");
                }
                else if (result.Reason == ResultReason.Canceled)
                {
                    var cancellation = CancellationDetails.FromResult(result);
                    Console.WriteLine($"CANCELED: Reason={cancellation.Reason}");

                    if (cancellation.Reason == CancellationReason.Error)
                    {
                        Console.WriteLine($"CANCELED: ErrorCode={cancellation.ErrorCode}");
                        Console.WriteLine($"CANCELED: ErrorDetails={cancellation.ErrorDetails}");
                        Console.WriteLine($"CANCELED: Did you update the subscription info?");
                    }
                }
            }
        }

        static void Main()
        {
            RecognizeIntentAsync().Wait();
            Console.WriteLine("Please press <Return> to continue.");
            Console.ReadLine();
        }
    }
}

Build and run your app

Now you're ready to build your app and test our speech recognition using the Speech service.

  1. Compile the code - From the menu bar of Visual Studio, choose Build > Build Solution.
  2. Start your app - From the menu bar, choose Debug > Start Debugging or press F5.
  3. Start recognition - It'll prompt you to speak a phrase in English. Your speech is sent to the Speech service, transcribed as text, and rendered in the console.

Next steps

In this quickstart, you'll use the Speech SDK and the Language Understanding (LUIS) service to recognize intents from audio data captured from a microphone. Specifically, you'll use the Speech SDK to capture speech, and a prebuilt domain from LUIS to identify intents for home automation, like turning on and off a light.

After satisfying a few prerequisites, recognizing speech and identifying intents from a microphone only takes a few steps:

  • Create a SpeechConfig object from your subscription key and region.
  • Create an IntentRecognizer object using the SpeechConfig object from above.
  • Using the IntentRecognizer object, start the recognition process for a single utterance.
  • Inspect the IntentRecognitionResult returned.

If you prefer to jump right in, view or download all Speech SDK C++ Samples on GitHub. Otherwise, let's get started.

Prerequisites

Before you get started:

Create a LUIS app for intent recognition

To complete the intent recognition quickstart, you'll need to create a LUIS account and a project using the LUIS preview portal. This quickstart only requires a LUIS subscription. A Speech service subscription isn't required.

The first thing you'll need to do is create a LUIS account and app using the LUIS preview portal. The LUIS app that you create will use a prebuilt domain for home automation, which provides intents, entities, and example utterances. When you're finished, you'll have a LUIS endpoint running in the cloud that you can call using the Speech SDK.

Follow these instructions to create your LUIS app:

When you're done, you'll need three things:

  • Your LUIS key
  • Your LUIS region
  • Your LUIS app ID

Here's where you can find this information in the LUIS preview portal:

  1. From the LUIS preview portal, select Manage, then select Azure Resources. On this page, you'll find your LUIS key and location (sometimes referred to as region).

    LUIS key and location

  2. After you've got your key and location, you'll need the app ID. Select Application Settings -- your app ID is available on this page.

    LUIS app ID

Open your project in Visual Studio

Next, open your project in Visual Studio.

  1. Launch Visual Studio 2019.
  2. Load your project and open helloworld.cpp.

Start with some boilerplate code

Let's add some code that works as a skeleton for our project. Make note that you've created an async method called recognizeIntent().

#include "stdafx.h"
// <code>
#include <iostream>
#include <speechapi_cxx.h>

using namespace std;
using namespace Microsoft::CognitiveServices::Speech;
using namespace Microsoft::CognitiveServices::Speech::Intent;

void recognizeIntent()
{
}

int wmain()
{
    recognizeIntent();
    cout << "Please press a key to continue.\n";
    cin.get();
    return 0;
}

Create a Speech configuration

Before you can initialize an IntentRecognizer object, you need to create a configuration that uses the key and location for your LUIS prediction resource.

Important

Your starter key and authoring keys will not work. You must use your prediction key and location that you created earlier. For more information, see Create a LUIS app for intent recognition.

Insert this code in the recognizeIntent() method. Make sure you update these values:

  • Replace "YourLanguageUnderstandingSubscriptionKey" with your LUIS prediction key.
  • Replace "YourLanguageUnderstandingServiceRegion" with your LUIS location.

Tip

If you need help finding these values, see Create a LUIS app for intent recognition.

auto config = SpeechConfig::FromSubscription("YourLanguageUnderstandingSubscriptionKey", "YourLanguageUnderstandingServiceRegion");

This sample uses the FromSubscription() method to build the SpeechConfig. For a full list of available methods, see SpeechConfig Class.

The Speech SDK will default to recognizing using en-us for the language, see Specify source language for speech to text for information on choosing the source language.

Initialize an IntentRecognizer

Now, let's create an IntentRecognizer. Insert this code in the recognizeIntent() method, right below your Speech configuration.

auto recognizer = IntentRecognizer::FromConfig(config);

Add a LanguageUnderstandingModel and Intents

You need to associate a LanguageUnderstandingModel with the intent recognizer, and add the intents you want recognized. We're going to use intents from the prebuilt domain for home automation.

Insert this code below your IntentRecognizer. Make sure that you replace "YourLanguageUnderstandingAppId" with your LUIS app ID.

Tip

If you need help finding this value, see Create a LUIS app for intent recognition.

auto model = LanguageUnderstandingModel::FromAppId("YourLanguageUnderstandingAppId");
recognizer->AddIntent(model, "YourLanguageUnderstandingIntentName1", "id1");
recognizer->AddIntent(model, "YourLanguageUnderstandingIntentName2", "id2");
recognizer->AddIntent(model, "YourLanguageUnderstandingIntentName3", "any-IntentId-here");

Recognize an intent

From the IntentRecognizer object, you're going to call the RecognizeOnceAsync() method. This method lets the Speech service know that you're sending a single phrase for recognition, and that once the phrase is identified to stop recognizing speech. For simplicity we'll wait on the future returned to complete.

Insert this code below your model:

auto result = recognizer->RecognizeOnceAsync().get();

Display the recognition results (or errors)

When the recognition result is returned by the Speech service, you'll want to do something with it. We're going to keep it simple and print the result to console.

Insert this code below auto result = recognizer->RecognizeOnceAsync().get();:

if (result->Reason == ResultReason::RecognizedIntent)
{
    cout << "RECOGNIZED: Text=" << result->Text << std::endl;
    cout << "  Intent Id: " << result->IntentId << std::endl;
    cout << "  Intent Service JSON: " << result->Properties.GetProperty(PropertyId::LanguageUnderstandingServiceResponse_JsonResult) << std::endl;
}
else if (result->Reason == ResultReason::RecognizedSpeech)
{
    cout << "RECOGNIZED: Text=" << result->Text << " (intent could not be recognized)" << std::endl;
}
else if (result->Reason == ResultReason::NoMatch)
{
    cout << "NOMATCH: Speech could not be recognized." << std::endl;
}
else if (result->Reason == ResultReason::Canceled)
{
    auto cancellation = CancellationDetails::FromResult(result);
    cout << "CANCELED: Reason=" << (int)cancellation->Reason << std::endl;

    if (cancellation->Reason == CancellationReason::Error)
    {
        cout << "CANCELED: ErrorCode=" << (int)cancellation->ErrorCode << std::endl;
        cout << "CANCELED: ErrorDetails=" << cancellation->ErrorDetails << std::endl;
        cout << "CANCELED: Did you update the subscription info?" << std::endl;
    }
}

Check your code

At this point, your code should look like this:

Note

We've added some comments to this version.

#include "stdafx.h"
// <code>
#include <iostream>
#include <speechapi_cxx.h>

using namespace std;
using namespace Microsoft::CognitiveServices::Speech;
using namespace Microsoft::CognitiveServices::Speech::Intent;

void recognizeIntent()
{
    // Creates an instance of a speech config with specified subscription key
    // and service region. Note that in contrast to other services supported by
    // the Cognitive Services Speech SDK, the Language Understanding service
    // requires a specific subscription key from https://www.luis.ai/.
    // The Language Understanding service calls the required key 'endpoint key'.
    // Once you've obtained it, replace with below with your own Language Understanding subscription key
    // and service region (e.g., "westus").
    // The default recognition language is "en-us".
    auto config = SpeechConfig::FromSubscription("YourLanguageUnderstandingSubscriptionKey", "YourLanguageUnderstandingServiceRegion");
    
    // Creates an intent recognizer using microphone as audio input.
    auto recognizer = IntentRecognizer::FromConfig(config);

    // Creates a Language Understanding model using the app id, and adds specific intents from your model
    auto model = LanguageUnderstandingModel::FromAppId("YourLanguageUnderstandingAppId");
    recognizer->AddIntent(model, "YourLanguageUnderstandingIntentName1", "id1");
    recognizer->AddIntent(model, "YourLanguageUnderstandingIntentName2", "id2");
    recognizer->AddIntent(model, "YourLanguageUnderstandingIntentName3", "any-IntentId-here");

    cout << "Say something...\n";

    // Starts intent recognition, and returns after a single utterance is recognized. The end of a
    // single utterance is determined by listening for silence at the end or until a maximum of 15
    // seconds of audio is processed.  The task returns the recognition text as result. 
    // Note: Since RecognizeOnceAsync() returns only a single utterance, it is suitable only for single
    // shot recognition like command or query. 
    // For long-running multi-utterance recognition, use StartContinuousRecognitionAsync() instead.
    auto result = recognizer->RecognizeOnceAsync().get();

    // Checks result.
    if (result->Reason == ResultReason::RecognizedIntent)
    {
        cout << "RECOGNIZED: Text=" << result->Text << std::endl;
        cout << "  Intent Id: " << result->IntentId << std::endl;
        cout << "  Intent Service JSON: " << result->Properties.GetProperty(PropertyId::LanguageUnderstandingServiceResponse_JsonResult) << std::endl;
    }
    else if (result->Reason == ResultReason::RecognizedSpeech)
    {
        cout << "RECOGNIZED: Text=" << result->Text << " (intent could not be recognized)" << std::endl;
    }
    else if (result->Reason == ResultReason::NoMatch)
    {
        cout << "NOMATCH: Speech could not be recognized." << std::endl;
    }
    else if (result->Reason == ResultReason::Canceled)
    {
        auto cancellation = CancellationDetails::FromResult(result);
        cout << "CANCELED: Reason=" << (int)cancellation->Reason << std::endl;

        if (cancellation->Reason == CancellationReason::Error)
        {
            cout << "CANCELED: ErrorCode=" << (int)cancellation->ErrorCode << std::endl;
            cout << "CANCELED: ErrorDetails=" << cancellation->ErrorDetails << std::endl;
            cout << "CANCELED: Did you update the subscription info?" << std::endl;
        }
    }
}

int wmain()
{
    recognizeIntent();
    cout << "Please press a key to continue.\n";
    cin.get();
    return 0;
}

Build and run your app

Now you're ready to build your app and test our speech recognition using the Speech service.

  1. Compile the code - From the menu bar of Visual Studio, choose Build > Build Solution.
  2. Start your app - From the menu bar, choose Debug > Start Debugging or press F5.
  3. Start recognition - It'll prompt you to speak a phrase in English. Your speech is sent to the Speech service, transcribed as text, and rendered in the console.

Next steps


In this quickstart, you'll use the Speech SDK and the Language Understanding (LUIS) service to recognize intents from audio data captured from a microphone. Specifically, you'll use the Speech SDK to capture speech, and a prebuilt domain from LUIS to identify intents for home automation, like turning on and off a light.

After satisfying a few prerequisites, recognizing speech and identifying intents from a microphone only takes a few steps:

  • Create a SpeechConfig object from your subscription key and region.
  • Create an IntentRecognizer object using the SpeechConfig object from above.
  • Using the IntentRecognizer object, start the recognition process for a single utterance.
  • Inspect the IntentRecognitionResult returned.

If you prefer to jump right in, view or download all Speech SDK Java Samples on GitHub. Otherwise, let's get started.

Prerequisites

Before you get started:

Create a LUIS app for intent recognition

To complete the intent recognition quickstart, you'll need to create a LUIS account and a project using the LUIS preview portal. This quickstart only requires a LUIS subscription. A Speech service subscription isn't required.

The first thing you'll need to do is create a LUIS account and app using the LUIS preview portal. The LUIS app that you create will use a prebuilt domain for home automation, which provides intents, entities, and example utterances. When you're finished, you'll have a LUIS endpoint running in the cloud that you can call using the Speech SDK.

Follow these instructions to create your LUIS app:

When you're done, you'll need three things:

  • Your LUIS key
  • Your LUIS region
  • Your LUIS app ID

Here's where you can find this information in the LUIS preview portal:

  1. From the LUIS preview portal, select Manage, then select Azure Resources. On this page, you'll find your LUIS key and location (sometimes referred to as region).

    LUIS key and location

  2. After you've got your key and location, you'll need the app ID. Select Application Settings -- your app ID is available on this page.

    LUIS app ID

Open your project

  1. Open your preferred IDE.
  2. Load your project and open Main.java.

Start with some boilerplate code

Let's add some code that works as a skeleton for our project.

package speechsdk.quickstart;

import com.microsoft.cognitiveservices.speech.*;
import com.microsoft.cognitiveservices.speech.intent.*;

/**
 * Quickstart: recognize speech using the Speech SDK for Java.
 */
public class Main {

    /**
     * @param args Arguments are ignored in this sample.
     */
    public static void main(String[] args) {
        try {
        } catch (Exception ex) {
            System.out.println("Unexpected exception: " + ex.getMessage());

            assert(false);
            System.exit(1);
        } 
    }
}

Create a Speech configuration

Before you can initialize an IntentRecognizer object, you need to create a configuration that uses the key and location for your LUIS prediction resource.

Insert this code in the try / catch block in main(). Make sure you update these values:

  • Replace "YourLanguageUnderstandingSubscriptionKey" with your LUIS prediction key.
  • Replace "YourLanguageUnderstandingServiceRegion" with your LUIS location.

Tip

If you need help finding these values, see Create a LUIS app for intent recognition.

SpeechConfig config = SpeechConfig.fromSubscription("YourLanguageUnderstandingSubscriptionKey", "YourLanguageUnderstandingServiceRegion");

This sample uses the FromSubscription() method to build the SpeechConfig. For a full list of available methods, see SpeechConfig Class.

The Speech SDK will default to recognizing using en-us for the language, see Specify source language for speech to text for information on choosing the source language.

Initialize an IntentRecognizer

Now, let's create an IntentRecognizer. Insert this code right below your Speech configuration.

IntentRecognizer recognizer = new IntentRecognizer(config);

Add a LanguageUnderstandingModel and Intents

You need to associate a LanguageUnderstandingModel with the intent recognizer, and add the intents you want recognized. We're going to use intents from the prebuilt domain for home automation.

Insert this code below your IntentRecognizer. Make sure that you replace "YourLanguageUnderstandingAppId" with your LUIS app ID.

Tip

If you need help finding this value, see Create a LUIS app for intent recognition.

LanguageUnderstandingModel model = LanguageUnderstandingModel.fromAppId("YourLanguageUnderstandingAppId");
recognizer.addIntent(model, "YourLanguageUnderstandingIntentName1", "id1");
recognizer.addIntent(model, "YourLanguageUnderstandingIntentName2", "id2");
recognizer.addIntent(model, "YourLanguageUnderstandingIntentName3", "any-IntentId-here");

Recognize an intent

From the IntentRecognizer object, you're going to call the recognizeOnceAsync() method. This method lets the Speech service know that you're sending a single phrase for recognition, and that once the phrase is identified to stop recognizing speech.

Insert this code below your model:

IntentRecognitionResult result = recognizer.recognizeOnceAsync().get();

Display the recognition results (or errors)

When the recognition result is returned by the Speech service, you'll want to do something with it. We're going to keep it simple and print the result to console.

Insert this code below your call to recognizeOnceAsync():

if (result.getReason() == ResultReason.RecognizedIntent) {
    System.out.println("RECOGNIZED: Text=" + result.getText());
    System.out.println("    Intent Id: " + result.getIntentId());
    System.out.println("    Intent Service JSON: " + result.getProperties().getProperty(PropertyId.LanguageUnderstandingServiceResponse_JsonResult));
}
else if (result.getReason() == ResultReason.RecognizedSpeech) {
    System.out.println("RECOGNIZED: Text=" + result.getText());
    System.out.println("    Intent not recognized.");
}
else if (result.getReason() == ResultReason.NoMatch) {
    System.out.println("NOMATCH: Speech could not be recognized.");
}
else if (result.getReason() == ResultReason.Canceled) {
    CancellationDetails cancellation = CancellationDetails.fromResult(result);
    System.out.println("CANCELED: Reason=" + cancellation.getReason());

    if (cancellation.getReason() == CancellationReason.Error) {
        System.out.println("CANCELED: ErrorCode=" + cancellation.getErrorCode());
        System.out.println("CANCELED: ErrorDetails=" + cancellation.getErrorDetails());
        System.out.println("CANCELED: Did you update the subscription info?");
    }
}

Release Resources

It's important to release the speech resources when you're done using them. Insert this code at the end of the try / catch block:

result.close();
recognizer.close();

Check your code

At this point, your code should look like this:

Note

We've added some comments to this version.

package speechsdk.quickstart;

import com.microsoft.cognitiveservices.speech.*;
import com.microsoft.cognitiveservices.speech.intent.*;

/**
 * Quickstart: recognize speech using the Speech SDK for Java.
 */
public class Main {

    /**
     * @param args Arguments are ignored in this sample.
     */
    public static void main(String[] args) {
        try {
            // <IntentRecognitionWithMicrophone>
            // Creates an instance of a speech config with specified
            // subscription key (called 'endpoint key' by the Language Understanding service)
            // and service region. Replace with your own subscription (endpoint) key
            // and service region (e.g., "westus2").
            // The default language is "en-us".
            SpeechConfig config = SpeechConfig.fromSubscription("YourLanguageUnderstandingSubscriptionKey", "YourLanguageUnderstandingServiceRegion");

            // Creates an intent recognizer using microphone as audio input.
            IntentRecognizer recognizer = new IntentRecognizer(config);

            // Creates a language understanding model using the app id, and adds specific intents from your model
            LanguageUnderstandingModel model = LanguageUnderstandingModel.fromAppId("YourLanguageUnderstandingAppId");
            recognizer.addIntent(model, "YourLanguageUnderstandingIntentName1", "id1");
            recognizer.addIntent(model, "YourLanguageUnderstandingIntentName2", "id2");
            recognizer.addIntent(model, "YourLanguageUnderstandingIntentName3", "any-IntentId-here");

            System.out.println("Say something...");

            // Starts recognition. It returns when the first utterance has been recognized.
            IntentRecognitionResult result = recognizer.recognizeOnceAsync().get();

            // Checks result.
            if (result.getReason() == ResultReason.RecognizedIntent) {
                System.out.println("RECOGNIZED: Text=" + result.getText());
                System.out.println("    Intent Id: " + result.getIntentId());
                System.out.println("    Intent Service JSON: " + result.getProperties().getProperty(PropertyId.LanguageUnderstandingServiceResponse_JsonResult));
            }
            else if (result.getReason() == ResultReason.RecognizedSpeech) {
                System.out.println("RECOGNIZED: Text=" + result.getText());
                System.out.println("    Intent not recognized.");
            }
            else if (result.getReason() == ResultReason.NoMatch) {
                System.out.println("NOMATCH: Speech could not be recognized.");
            }
            else if (result.getReason() == ResultReason.Canceled) {
                CancellationDetails cancellation = CancellationDetails.fromResult(result);
                System.out.println("CANCELED: Reason=" + cancellation.getReason());

                if (cancellation.getReason() == CancellationReason.Error) {
                    System.out.println("CANCELED: ErrorCode=" + cancellation.getErrorCode());
                    System.out.println("CANCELED: ErrorDetails=" + cancellation.getErrorDetails());
                    System.out.println("CANCELED: Did you update the subscription info?");
                }
            }
            
            result.close();
            recognizer.close();
        } catch (Exception ex) {
            System.out.println("Unexpected exception: " + ex.getMessage());

            assert(false);
            System.exit(1);
        } 
    }
}

Build and run your app

Press F11, or select Run > Debug. The next 15 seconds of speech input from your microphone will be recognized and logged in the console window.

Next steps

In this quickstart, you'll use the Speech SDK and the Language Understanding (LUIS) service to recognize intents from audio data captured from a microphone. Specifically, you'll use the Speech SDK to capture speech, and a prebuilt domain from LUIS to identify intents for home automation, like turning on and off a light.

After satisfying a few prerequisites, recognizing speech and identifying intents from a microphone only takes a few steps:

  • Create a SpeechConfig object from your subscription key and region.
  • Create an IntentRecognizer object using the SpeechConfig object from above.
  • Using the IntentRecognizer object, start the recognition process for a single utterance.
  • Inspect the IntentRecognitionResult returned.

If you prefer to jump right in, view or download all Speech SDK Python Samples on GitHub. Otherwise, let's get started.

Prerequisites

Before you get started:

Create a LUIS app for intent recognition

To complete the intent recognition quickstart, you'll need to create a LUIS account and a project using the LUIS preview portal. This quickstart only requires a LUIS subscription. A Speech service subscription isn't required.

The first thing you'll need to do is create a LUIS account and app using the LUIS preview portal. The LUIS app that you create will use a prebuilt domain for home automation, which provides intents, entities, and example utterances. When you're finished, you'll have a LUIS endpoint running in the cloud that you can call using the Speech SDK.

Follow these instructions to create your LUIS app:

When you're done, you'll need three things:

  • Your LUIS key
  • Your LUIS region
  • Your LUIS app ID

Here's where you can find this information in the LUIS preview portal:

  1. From the LUIS preview portal, select Manage, then select Azure Resources. On this page, you'll find your LUIS key and location (sometimes referred to as region).

    LUIS key and location

  2. After you've got your key and location, you'll need the app ID. Select Application Settings -- your app ID is available on this page.

    LUIS app ID

Open your project

  1. Open your preferred IDE.
  2. Create a new project and create file called quickstart.py, then open it.

Start with some boilerplate code

Let's add some code that works as a skeleton for our project.

import azure.cognitiveservices.speech as speechsdk

print("Say something...")

Create a Speech configuration

Before you can initialize an IntentRecognizer object, you need to create a configuration that uses the key and location for your LUIS prediction resource.

Insert this code in quickstart.py. Make sure you update these values:

  • Replace "YourLanguageUnderstandingSubscriptionKey" with your LUIS prediction key.
  • Replace "YourLanguageUnderstandingServiceRegion" with your LUIS location.

Tip

If you need help finding these values, see Create a LUIS app for intent recognition.

intent_config = speechsdk.SpeechConfig(subscription="YourLanguageUnderstandingSubscriptionKey", region="YourLanguageUnderstandingServiceRegion")

This sample constructs the SpeechConfig object using LUIS key and region. For a full list of available methods, see SpeechConfig Class.

The Speech SDK will default to recognizing using en-us for the language, see Specify source language for speech to text for information on choosing the source language.

Initialize an IntentRecognizer

Now, let's create an IntentRecognizer. Insert this code right below your Speech configuration.

intent_recognizer = speechsdk.intent.IntentRecognizer(speech_config=intent_config)

Add a LanguageUnderstandingModel and Intents

You need to associate a LanguageUnderstandingModel with the intent recognizer and add the intents you want recognized. We're going to use intents from the prebuilt domain for home automation.

Insert this code below your IntentRecognizer. Make sure that you replace "YourLanguageUnderstandingAppId" with your LUIS app ID.

Tip

If you need help finding this value, see Create a LUIS app for intent recognition.

model = speechsdk.intent.LanguageUnderstandingModel(app_id="YourLanguageUnderstandingAppId")
intents = [
    (model, "HomeAutomation.TurnOn"),
    (model, "HomeAutomation.TurnOff"),
    ("This is a test.", "test"),
    ("Switch to channel 34.", "34"),
    ("what's the weather like", "weather"),
]
intent_recognizer.add_intents(intents)

Recognize an intent

From the IntentRecognizer object, you're going to call the recognize_once() method. This method lets the Speech service know that you're sending a single phrase for recognition, and that once the phrase is identified to stop recognizing speech.

Insert this code below your model:

intent_result = intent_recognizer.recognize_once()

Display the recognition results (or errors)

When the recognition result is returned by the Speech service, you'll want to do something with it. We're going to keep it simple and print the result to console.

Below your call to recognize_once(), add this code:

if intent_result.reason == speechsdk.ResultReason.RecognizedIntent:
    print("Recognized: \"{}\" with intent id `{}`".format(intent_result.text, intent_result.intent_id))
elif intent_result.reason == speechsdk.ResultReason.RecognizedSpeech:
    print("Recognized: {}".format(intent_result.text))
elif intent_result.reason == speechsdk.ResultReason.NoMatch:
    print("No speech could be recognized: {}".format(intent_result.no_match_details))
elif intent_result.reason == speechsdk.ResultReason.Canceled:
    print("Intent recognition canceled: {}".format(intent_result.cancellation_details.reason))
    if intent_result.cancellation_details.reason == speechsdk.CancellationReason.Error:
        print("Error details: {}".format(intent_result.cancellation_details.error_details))

Check your code

At this point, your code should look like this:

Note

We've added some comments to this version.

import azure.cognitiveservices.speech as speechsdk

print("Say something...")

"""performs one-shot intent recognition from input from the default microphone"""
# <IntentRecognitionOnceWithMic>
# Set up the config for the intent recognizer (remember that this uses the Language Understanding key, not the Speech Services key)!
intent_config = speechsdk.SpeechConfig(subscription="YourLanguageUnderstandingSubscriptionKey", region="YourLanguageUnderstandingServiceRegion")

# Set up the intent recognizer
intent_recognizer = speechsdk.intent.IntentRecognizer(speech_config=intent_config)

# set up the intents that are to be recognized. These can be a mix of simple phrases and
# intents specified through a LanguageUnderstanding Model.
model = speechsdk.intent.LanguageUnderstandingModel(app_id="YourLanguageUnderstandingAppId")
intents = [
    (model, "HomeAutomation.TurnOn"),
    (model, "HomeAutomation.TurnOff"),
    ("This is a test.", "test"),
    ("Switch to channel 34.", "34"),
    ("what's the weather like", "weather"),
]
intent_recognizer.add_intents(intents)

# Starts intent recognition, and returns after a single utterance is recognized. The end of a
# single utterance is determined by listening for silence at the end or until a maximum of 15
# seconds of audio is processed. It returns the recognition text as result.
# Note: Since recognize_once() returns only a single utterance, it is suitable only for single
# shot recognition like command or query.
# For long-running multi-utterance recognition, use start_continuous_recognition() instead.
intent_result = intent_recognizer.recognize_once()

# Check the results
if intent_result.reason == speechsdk.ResultReason.RecognizedIntent:
    print("Recognized: \"{}\" with intent id `{}`".format(intent_result.text, intent_result.intent_id))
elif intent_result.reason == speechsdk.ResultReason.RecognizedSpeech:
    print("Recognized: {}".format(intent_result.text))
elif intent_result.reason == speechsdk.ResultReason.NoMatch:
    print("No speech could be recognized: {}".format(intent_result.no_match_details))
elif intent_result.reason == speechsdk.ResultReason.Canceled:
    print("Intent recognition canceled: {}".format(intent_result.cancellation_details.reason))
    if intent_result.cancellation_details.reason == speechsdk.CancellationReason.Error:
        print("Error details: {}".format(intent_result.cancellation_details.error_details))

Build and run your app

Run the sample from the console or in your IDE:

python quickstart.py

The next 15 seconds of speech input from your microphone will be recognized and logged in the console window.

Next steps

View or download all Speech SDK Samples on GitHub.

Additional language and platform support

If you've clicked this tab, you probably didn't see a quickstart in your favorite programming language. Don't worry, we have additional quickstart materials and code samples available on GitHub. Use the table to find the right sample for your programming language and platform/OS combination.

Language Additional Quickstarts Code samples
C++ Quickstarts, Samples
C# .NET Framework, .NET Core, UWP, Unity, Xamarin
Java Android, JRE
Javascript Browser
Node.js Windows, Linux, macOS
Objective-C macOs, iOS iOS, macOS
Python Windows, Linux, macOS
Swift macOs, iOS iOS, macOS