Solution Idea
If you'd like to see us expand this article with more information, such as potential use cases, alternative services, implementation considerations, or pricing guidance, let us know with GitHub Feedback!
Create seamless conversational interfaces with all of your internet-accessible devices-from your connected television or fridge to devices in a connected power plant. By combining Azure Speech Service, Language Understanding Service (LUIS) and Azure Bot Framework, developers can create natural, human-like conversational interfaces to control smart devices using Azure IoT Hub.
Architecture
Data flow
- Using voice, the user asks the voice assistant app to turn on the exterior house lights.
- Using the Speech SDK, the app connects to Direct Line Speech. If keywords are confirmed by Keyword Verification, the speech is transcribed to text and sent to the Bot Service.
- The Bot Service connects to Language Understanding service (LUIS). LUIS allows an application to understand what a person wants in their own words. The intent of the user's request (example: TurnOnLight) is returned to the Bot Service.
- The request is relayed to the device.
- If the device is connected to Azure IoT Hub, Bot Service connects to Azure IoT Hub Service API and sends the command to the device using either a Direct Method, an update to the device twin's Desired Property, or a Cloud to Device message.
- If the device is connected to a third party IoT cloud, Bot Service connects to the third-party service API and sends a command to the device.
- The Bot returns the results of the command to the user by generating a response that includes the text to speak.
- The response is turned into audio using the Text-to-speech service and passed back to the voice assistant app by Direct Line Speech.
- Application Insights gathers runtime telemetry to help development with bot performance and usage
- Azure App Service hosts the Bot Service application.
Components
- Voice assistants documentation
- Tutorial: Voice-enable your bot using the Speech SDK
- What is Direct Line Speech
- Azure Bot Service
- Speech to Text
- Text to Speech
- Custom Keywords
- Language Understanding Service (LUIS)
- Azure IoT Hub
Next steps
- To learn how to create a LUIS app and add intents to your app, see these articles:
- Learn more about adding a LUIS app to a bot, including:
- How to create a bot by using the Azure Bot Service
- How to add natural language understanding (LUIS) to a bot
- Learn about various methods you can use to send commands to an IoT device by using Azure IoT Hub, including:
- Sending cloud-to-device commands
- Using device twins
- Using direct methods
- Learn how to build an enterprise-grade conversational bot by using the Azure Bot Service.
- To learn more about developing solutions by using IoT Hub, see the Azure Iot Hub developer guide.