What is a voice assistant?

Voice assistants using the Speech service empowers developers to create natural, human-like conversational interfaces for their applications and experiences.

The voice assistant service provides fast, reliable interaction between a device and an assistant implementation that uses either (1) Direct Line Speech (via Azure Bot Service) for adding voice capabilities to your bots, or, (2) Custom Commands for voice commanding scenarios.

Choosing an assistant solution

The first step to creating a voice assistant is to decide what it should do. The Speech service provides multiple, complementary solutions for crafting your assistant interactions. You can add voice in and voice out capabilities to your flexible and versatile bot built using Azure Bot Service with the Direct Line Speech channel, or leverage the simplicity of authoring a Custom Commands app for straightforward voice commanding scenarios.

If you want... Then consider... For example...
Open-ended conversation with robust skills integration and full deployment control Azure Bot Service bot with Direct Line Speech channel
  • "I need to go to Seattle"
  • "What kind of pizza can I order?"
Voice commanding or simple task-oriented conversations with simplified authoring and hosting Custom Commands
  • "Turn on the overhead light"
  • "Make it 5 degrees warmer"
  • Other samples available here

We recommend Direct Line Speech as the best default choice if you aren't yet sure what you'd like your assistant to handle. It offers integration with a rich set of tools and authoring aids such as the Virtual Assistant Solution and Enterprise Template and the QnA Maker service to build on common patterns and use your existing knowledge sources.

Custom Commands makes it easy to build rich voice commanding apps optimized for voice-first interaction experiences. It provides a unified authoring experience, an automatic hosting model, and relatively lower complexity, helping you focus on building the best solution for your voice commanding scenarios.

Comparison of assistant solutions

Reference Architecture for building a voice assistant using the Speech SDK

Conceptual diagram of the voice assistant orchestration service flow

Core features

Whether you choose Direct Line Speech or Custom Commands to create your assistant interactions, you can use a rich set of customization features to customize your assistant to your brand, product, and personality.

Category Features
Custom keyword Users can start conversations with assistants with a custom keyword like "Hey Contoso." An app does this with a custom keyword engine in the Speech SDK, which can be configured with a custom keyword that you can generate here. Voice assistants can use service-side keyword verification to improve the accuracy of the keyword activation (versus the device alone).
Speech to text Voice assistants convert real-time audio into recognized text using Speech-to-text from the Speech service. This text is available, as it's transcribed, to both your assistant implementation and your client application.
Text to speech Textual responses from your assistant are synthesized using Text-to-speech from the Speech service. This synthesis is then made available to your client application as an audio stream. Microsoft offers the ability to build your own custom, high-quality Neural TTS voice that gives a voice to your brand. To learn more, contact us.

Getting started with voice assistants

We offer quickstarts designed to have you running code in less than 10 minutes. This table includes a list of voice assistant quickstarts, organized by language.

Sample code and Tutorials

Sample code for creating a voice assistant is available on GitHub. These samples cover the client application for connecting to your assistant in several popular programming languages.

Customization

Voice assistants built using Azure Speech services can use the full range of customization options.

Note

Customization options vary by language/locale (see Supported languages).

Next steps