How to manage issues with audio input (XAML)

Learn how to manage issues with speech-recognition accuracy caused by audio-input quality.

What you need to know

Technologies

Prerequisites

This topic builds on Quickstart: Speech recognition.

To complete this tutorial, have a look through these topics to get familiar with the technologies discussed here:

Install Microsoft Visual Studio.
Get a developer license. For instructions, see Develop using Visual Studio 2013.
Create your first Windows Store app using C# or Visual Basic.
Roadmap for Windows Store apps using C# or Visual Basic
Learn about events with Events and routed events overview
See Speech design guidelines for helpful tips on designing a useful and engaging speech-enabled app.

Instructions

Step 1: Assess audio-input quality

When speech recognition is active, use the RecognitionQualityDegrading event of your speech recognizer to determine whether one or more audio issues might be interfering with speech input. The event argument (SpeechRecognitionQualityDegradingEventArgs) provides the Problem property, which describes the issues detected with the audio input.

Recognition can be affected by too much background noise, a muted microphone, and the volume or speed of the speaker.

Here, we configure a speech recognizer and start listening for the RecognitionQualityDegrading event.

private async void WeatherSearch_Click(object sender, RoutedEventArgs e)
{
    // Create an instance of SpeechRecognizer.
    var speechRecognizer = new Windows.Media.SpeechRecognition.SpeechRecognizer();

    // Listen for audio input issues.
    speechRecognizer.RecognitionQualityDegrading += speechRecognizer_RecognitionQualityDegrading;

    // Add a web search grammar to the recognizer.
    var webSearchGrammar = new Windows.Media.SpeechRecognition.SpeechRecognitionTopicConstraint(Windows.Media.SpeechRecognition.SpeechRecognitionScenario.WebSearch, "webSearch");


    speechRecognizer.UIOptions.AudiblePrompt = "Say what you want to search for...";
    speechRecognizer.UIOptions.ExampleText = @"Ex. 'weather for London'";
    speechRecognizer.Constraints.Add(webSearchGrammar);

    // Compile the constraint.
    await speechRecognizer.CompileConstraintsAsync();

    // Start recognition.
    Windows.Media.SpeechRecognition.SpeechRecognitionResult speechRecognitionResult = await speechRecognizer.RecognizeWithUIAsync();
    //await speechRecognizer.RecognizeWithUIAsync();

    // Do something with the recognition result.
    var messageDialog = new Windows.UI.Popups.MessageDialog(speechRecognitionResult.Text, "Text spoken");
    await messageDialog.ShowAsync();
}

Step 2: Manage the speech-recognition experience

Use the description provided by the Problem property to help the user improve conditions for recognition.

Here, we create a handler for the RecognitionQualityDegrading event that checks for a low volume level. We then use a SpeechSynthesizer object to suggest that the user try speaking louder.

private async void speechRecognizer_RecognitionQualityDegrading(
    Windows.Media.SpeechRecognition.SpeechRecognizer sender,
    Windows.Media.SpeechRecognition.SpeechRecognitionQualityDegradingEventArgs args)
{
    // Create an instance of a speech synthesis engine (voice).
    var speechSynthesizer =
        new Windows.Media.SpeechSynthesis.SpeechSynthesizer();

    // If input speech is too quiet, prompt the user to speak louder.
    if (args.Problem == Windows.Media.SpeechRecognition.SpeechRecognitionAudioProblem.TooQuiet)
    {
        // Generate the audio stream from plain text.
        Windows.Media.SpeechSynthesis.SpeechSynthesisStream stream;
        try
        {
            stream = await speechSynthesizer.SynthesizeTextToStreamAsync("Try speaking louder");
            stream.Seek(0);
        }
        catch (Exception)
        {
            stream = null;
        }

        // Send the stream to the MediaElement declared in XAML.
        await CoreApplication.MainView.CoreWindow.Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.High, () =>
        {
            this.media.SetSource(stream, stream.ContentType);
        });
    }
}

Responding to speech interactions

Designers

Speech design guidelines