您现在访问的是微软AZURE全球版技术文档网站,若需要访问由世纪互联运营的MICROSOFT AZURE中国区技术文档网站,请访问 https://docs.azure.cn.

教程:使用语音 SDK 实现机器人的语音Tutorial: Voice-enable your bot using the Speech SDK

可以使用语音服务来语音启用聊天机器人。You can use the Speech service to voice-enable a chat bot.

在本教程中,您将创建一个可重复您所说的操作的机器人。In this tutorial, you'll create a bot that repeats what you say to it. 你将使用 Microsoft Bot Framework 创建机器人,并将其部署到 Azure,并将其注册到机器人 Framework 直通 Line 语音通道。You will create your bot using Microsoft Bot Framework, deploy it to Azure, and register it with the Bot Framework Direct Line Speech channel. 然后,你将配置一个适用于 Windows 的示例客户端应用程序,该应用程序可让你与机器人进行对话,并倾听你的反馈。Then you will configure a sample client app for Windows that lets you speak to your bot and hear it speak back to you.

本教程适用于不熟悉 Azure、Bot Framework bot、直行语音或语音 SDK 的开发人员,并且想要使用有限的编码快速构建工作系统。This tutorial is designed for developers who are new to Azure, Bot Framework bots, Direct Line Speech, or the Speech SDK, and want to quickly build a working system with limited coding. 不需要经验或对这些服务的熟悉。No experience or familiarity with these services is needed.

在本教程中,已启用语音的聊天机器人将遵循以下步骤:The voice-enabled chat bot you make in this tutorial follows these steps:

  1. 示例客户端应用程序已配置为连接到直接线路语音通道和 Echo 机器人。The sample client application is configured to connect to Direct Line Speech channel and the Echo Bot.
  2. 当用户按下按钮时,将从麦克风播放语音音频。When the user presses a button, voice audio streams from the microphone. 使用自定义关键字时,将连续录制 (或音频。 ) (Or audio is continuously recorded when a custom keyword is used.)
  3. 如果使用了自定义关键字,关键字检测会在本地设备上进行,并将音频流传送到云。If a custom keyword is used, keyword detection happens on the local device, gating audio streaming to the cloud.
  4. 使用语音 SDK 时,示例客户端应用程序连接到直接线路语音通道并流式传输音频。Using Speech SDK, the sample client application connects to Direct Line Speech channel and streams audio.
  5. 或者,在服务上进行更高的准确性关键字验证。Optionally, higher accuracy keyword verification happens on the service.
  6. 音频会传递到语音识别服务,并转录为文本。The audio is passed to the speech recognition service and transcribed to text.
  7. 识别的文本将作为 Bot 框架活动传递到回显机器人。The recognized text is passed to the Echo Bot as a Bot Framework Activity.
  8. 文本到语音 (TTS) 服务将响应文本转为音频,并流式传输回客户端应用程序进行播放。The response text is turned into audio by the Text-to-Speech (TTS) service, and streamed back to the client application for playback.

关系图-标记diagram-tag

备注

本教程中的步骤不需要付费服务。The steps in this tutorial do not require a paid service. 作为新的 Azure 用户,你将能够使用免费的 Azure 试用版订阅和语音服务的免费层来完成本教程。As a new Azure user, you'll be able to use credits from your free Azure trial subscription and the free tier of the Speech service to complete this tutorial.

本教程的内容:Here's what this tutorial covers:

  • 创建新的 Azure 资源Create new Azure resources
  • 生成、测试 Echo 机器人示例并将其部署到 Azure App ServiceBuild, test, and deploy the Echo Bot sample to an Azure App Service
  • 向直接线路语音频道注册机器人Register your bot with Direct Line Speech channel
  • 生成并运行 Windows 语音助手客户端以与 Echo Bot 交互Build and run the Windows Voice Assistant Client to interact with your Echo Bot
  • 添加自定义关键字激活Add custom keyword activation
  • 了解如何更改识别语音的语言Learn to change the language of the recognized and spoken speech

先决条件Prerequisites

下面是完成本教程所需的操作:Here's what you'll need to complete this tutorial:

创建资源组Create a resource group

你将在本教程中创建的客户端应用使用少量的 Azure 服务。The client app that you'll create in this tutorial uses a handful of Azure services. 若要减少来自机器人的响应的往返时间,需要确保这些服务位于同一 Azure 区域。To reduce the round-trip time for responses from your bot, you'll want to make sure that these services are located in the same Azure region. 在本部分中,你将在 美国西部 区域创建一个资源组。In this section, you'll create a resource group in the West US region. 此资源组将在为 Bot 框架、直接连线语音通道和语音服务创建单个资源时使用。This resource group will be used when creating individual resources for the Bot Framework, the Direct Line Speech channel, and the Speech service.

  1. 创建资源组 Create a resource group
  2. 系统将提示你提供一些信息:You'll be prompted to provide some information:
    • 将 " 订阅 " 设置为 " 免费试用 (你也可以使用现有的订阅) 。Set Subscription to Free Trial (you can also use an existing subscription).
    • 输入 资源组 的名称。Enter a name for your Resource group. 建议 SpeechEchoBotTutorialWe recommend SpeechEchoBotTutorial-ResourceGroup.
    • 从 " 区域 " 下拉箭头中,选择 " 美国西部"。From the Region drop-down, select West US.
  3. 单击“审阅并创建”。Click Review and create. 应该会看到一个横幅,其中显示了已 通过的验证You should see a banner that read Validation passed.
  4. 单击“创建”。 Click Create. 创建资源组可能需要几分钟的时间。It may take a few minutes to create the resource group.
  5. 与稍后在本教程中创建的资源一样,最好将此资源组固定到仪表板以便于访问。As with the resources you'll create later in this tutorial, it's a good idea to pin this resource group to your dashboard for easy access. 如果要固定此资源组,请单击资源组名称右侧的固定图标。If you'd like to pin this resource group, click the pin icon right of the resource group name.

选择 Azure 区域Choosing an Azure region

如果要在本教程中使用其他区域,这些因素可能会限制你的选择:If you'd like to use a different region for this tutorial these factors may limit your choices:

有关区域的详细信息,请参阅 Azure 位置For more information about regions, see Azure locations.

创建资源Create resources

现在,在受支持的区域中拥有资源组,下一步是为要在本教程中使用的每个服务创建单独的资源。Now that you have a resource group in a supported region, the next step is to create individual resources for each service that you'll use in this tutorial.

创建语音服务资源Create a Speech service resource

按照以下说明创建语音资源:Follow these instructions to create a Speech resource:

  1. 创建语音服务资源 Create a Speech service resource
  2. 系统将提示你提供一些信息:You'll be prompted to provide some information:
    • 为资源指定 名称Give your resource a Name. 建议 SpeechEchoBotTutorialWe recommend SpeechEchoBotTutorial-Speech
    • 对于 " 订阅",请确保选择 " 免费试用 "。For Subscription, make sure Free Trial is selected.
    • 对于 " 位置",请选择 " 美国西部"。For Location, select West US.
    • 对于 " 定价层",请选择 F0For Pricing tier, select F0. 这是免费级别。This is the free tier.
    • 对于 " 资源组",选择 " SpeechEchoBotTutorial"。For Resource group, select SpeechEchoBotTutorial-ResourceGroup.
  3. 输入所有所需信息后,单击 " 创建"。After you've entered all required information, click Create. 创建资源可能需要几分钟时间。It may take a few minutes to create your resource.
  4. 稍后在本教程中,你将需要此服务的订阅密钥。Later in this tutorial you'll need subscription keys for this service. 你可以随时从资源 概述 (管理密钥) 或 密钥 访问这些密钥。You can access these keys at any time from your resource's Overview (Manage keys) or Keys.

此时,请检查资源组 (SpeechEchoBotTutorial-ResourceGroup) 是否有语音资源:At this point, check that your resource group (SpeechEchoBotTutorial-ResourceGroup) has a Speech resource:

名称Name 类型Type 位置Location
SpeechEchoBotTutorial-SpeechSpeechEchoBotTutorial-Speech 认知服务Cognitive Services 美国西部West US

创建 Azure 应用服务计划Create an Azure App Service plan

下一步是创建应用服务计划。The next step is to create an App Service Plan. 应用服务计划为要运行的 Web 应用定义一组计算资源。An App Service plan defines a set of compute resources for a web app to run.

  1. 创建 Azure App Service 计划 Create an Azure App Service plan
  2. 系统将提示你提供一些信息:You'll be prompted to provide some information:
    • 将 " 订阅 " 设置为 " 免费试用 (你也可以使用现有的订阅) 。Set Subscription to Free Trial (you can also use an existing subscription).
    • 对于 " 资源组",选择 " SpeechEchoBotTutorial"。For Resource group, select SpeechEchoBotTutorial-ResourceGroup.
    • 为资源指定 名称Give your resource a Name. 建议 SpeechEchoBotTutorial-AppServicePlanWe recommend SpeechEchoBotTutorial-AppServicePlan
    • 对于 " 操作系统",选择 " Windows"。For Operating System, select Windows.
    • 对于 " 区域",选择 " 美国西部"。For Region, select West US.
    • 对于 " 定价层",请确保选择 " 标准 S1 "。For Pricing Tier, make sure that Standard S1 is selected. 这应该是默认值。This should be the default value. 如果不是这样,请确保将 操作系统 设置为 Windows ,如上所述。If it isn't, make sure you set the Operating System to Windows as described above.
  3. 单击“审阅并创建”。Click Review and create. 应该会看到一个横幅,其中显示了已 通过的验证You should see a banner that read Validation passed.
  4. 单击“创建”。 Click Create. 创建资源组可能需要几分钟的时间。It may take a few minutes to create the resource group.

此时,请检查资源组 (SpeechEchoBotTutorial-ResourceGroup) 是否有两个资源:At this point, check that your resource group (SpeechEchoBotTutorial-ResourceGroup) has two resources:

名称Name 类型Type 位置Location
SpeechEchoBotTutorial-AppServicePlanSpeechEchoBotTutorial-AppServicePlan 应用服务计划App Service Plan 美国西部West US
SpeechEchoBotTutorial-SpeechSpeechEchoBotTutorial-Speech 认知服务Cognitive Services 美国西部West US

构建 Echo BotBuild an Echo Bot

现在,你已创建了一些资源,接下来让我们来构建机器人。Now that you've created some resources, let's build a bot. 接下来,我们将开始介绍回响机器人示例,如名称所示,将输入的文本回显为其响应。We're going to start with the Echo Bot sample, which as the name implies, echoes the text that you've entered as its response. 不用担心,示例代码已准备就绪,无需进行任何更改。Don't worry, the sample code is ready for you to use without any changes. 它配置为使用直接线路语音频道,我们将在我们将机器人部署到 Azure 后进行连接。It's configured to work with the Direct Line Speech channel, which we'll connect after we've deployed the bot to Azure.

备注

GitHub 上的示例自述文件中提供了后面的说明以及有关 Echo Bot 的其他信息。The instructions that follow, as well as additional information about the Echo Bot are available in the sample's README on GitHub.

在计算机上运行机器人示例Run the bot sample on your machine

  1. 克隆示例存储库:Clone the samples repository:

    git clone https://github.com/Microsoft/botbuilder-samples.git
    
  2. 启动 Visual Studio。Launch Visual Studio.

  3. 从工具栏中,选择 "文件" " > 打开 > 项目/解决方案",然后打开 Echo 机器人项目解决方案:From the toolbar, select File > Open > Project/Solution, and open the Echo Bot project solution:

    samples\csharp_dotnetcore\02.echo-bot\EchoBot.sln
    
  4. 加载项目后,按 F5 生成并运行项目。After the project is loaded, press F5 to build and run the project.

  5. 浏览器将启动,您将看到类似于下面的屏幕。A browser will launch and you'll see a screen similar to this.

    屏幕截图显示 "EchoBot" 页,其中显示了机器人已准备就绪!Screenshot shows the EchoBot page with the message Your bot is ready!

用 Bot Framework 模拟器测试机器人示例Test the bot sample with the Bot Framework Emulator

机器人框架模拟器是一个桌面应用程序,它允许机器人开发人员在本地 (或通过隧道) 远程测试和调试其 bot。The Bot Framework Emulator is a desktop app that lets bot developers test and debug their bots locally (or remotely through a tunnel). 仿真器接受键入的文本作为输入 (不是语音) 。The Emulator accepts typed text as the input (not voice). 机器人还会响应文本。The bot will also respond with text. 按照以下步骤操作,使用机器人框架模拟器测试在本地运行的、带有文本输入和文本输出的回显机器人。Follow these steps to use the Bot Framework Emulator to test your Echo Bot running locally, with text input and text output. 将机器人部署到 Azure 后,我们将通过语音输入和语音输出对其进行测试。After we deploy the bot to Azure we will test it with voice input and voice output.

  1. 安装 机器人 Framework 模拟器 版本4.3.0 或更高版本Install the Bot Framework Emulator version 4.3.0 or greater

  2. 启动 Bot Framework 模拟器并打开机器人:Launch the Bot Framework Emulator and open your bot:

    • 文件 -> 打开机器人File -> Open Bot.
  3. 输入机器人的 URL。Enter the URL for your bot. 例如:For example:

    http://localhost:3978/api/messages
    

    然后按 "连接"。and press "Connect".

  4. 机器人应 fibonacci 你的 "Hello and 欢迎!"The bot should greet you with "Hello and welcome!" 消息作为响应。message. 键入任何短信,确认从机器人获得响应。Type in any text message and confirm you get a response from the bot.

  5. 与 Echo Bot 实例通信的交换方式如下: 屏幕截图显示机器人框架模拟器。This is what an exchange of communication with an Echo Bot instance might look like: Screenshot shows the Bot Framework Emulator.

将机器人部署到 Azure App ServiceDeploy your bot to an Azure App Service

下一步是将回显机器人部署到 Azure。The next step is to deploy the Echo Bot to Azure. 可以通过多种方法来部署机器人,但在本教程中,我们将重点介绍如何直接从 Visual Studio 进行发布。There are a few ways to deploy a bot, but in this tutorial we'll focus on publishing directly from Visual Studio.

备注

或者,你可以使用 Azure CLI部署模板部署机器人。Alternatively, you can deploy a bot using the Azure CLI and deployment templates.

备注

如果在执行以下步骤时未显示 " 发布 ... ",请使用 Visual Studio 安装程序来添加 ASP.NET 和 web 开发 工作负荷。If Publish... doesn't appear as you perform the following steps, use Visual Studio Installer to add the ASP.NET and web development workload.

  1. 在 Visual Studio 中,打开已配置为与直接线路语音通道一起使用的回响机器人:From Visual Studio, open the Echo Bot that's been configured for use with Direct Line Speech channel:

    samples\csharp_dotnetcore\02.echo-bot\EchoBot.sln
    
  2. 解决方案资源管理器 中,右键单击 EchoBot 项目,然后选择 "发布 ... "In the Solution Explorer, right-click the EchoBot project and select Publish...

  3. 此时会打开一个名为 Publish 的新窗口。A new window titled Publish opens.

  4. 选择 " Azure",单击 " 下一步",选择 Azure App Service (Windows),单击 " 下一步",然后单击 " 创建新的 Azure App Service ... "。Select Azure, click Next, select Azure App Service (Windows), click Next, and click Create a new Azure App Service... by the green plus sign.

  5. 当 " 应用服务 (Windows) " 窗口出现时:When the App Service (Windows) window appears:

    • 单击 " 添加帐户",然后使用 Azure 帐户凭据登录。Click Add an account, and sign in with your Azure account credentials. 如果已登录,请从下拉列表中选择所需的帐户。If you're already signed in, select the account you want from the drop-down list.
    • 对于 " 名称",需要输入机器人的全局唯一名称。For the Name, you'll need to enter a globally unique name for your Bot. 此名称用于创建唯一的机器人 URL。This name is used to create a unique bot URL. 将填充默认值,包括日期和时间 (例如: "EchoBot20190805125647" ) 。A default value will be populated including the date and time (For example: "EchoBot20190805125647"). 您可以使用本教程的默认名称。You can use the default name for this tutorial.
    • 对于 "订阅",将其设置为 "免费试用"For Subscription, set it to Free Trial
    • 对于 " 资源组",选择 SpeechEchoBotTutorialFor Resource Group, select SpeechEchoBotTutorial-ResourceGroup
    • 对于 托管计划,请选择 SpeechEchoBotTutorial-AppServicePlanFor Hosting Plan, select SpeechEchoBotTutorial-AppServicePlan
  6. 单击“创建”。 Click Create. 在向导的最后一个屏幕上,单击 " 完成"。On the final wizard screen, click Finish.

  7. 单击 "发布" 屏幕右侧的 " 发布 "。Click Publish on the right side of the Publish screen. Visual Studio 将机器人部署到 Azure。Visual Studio deploys the bot to Azure.

  8. 在 Visual Studio 的输出窗口中应会看到如下所示的成功消息:You should see a success message in Visual Studio's output window that looks like this:

    Publish Succeeded.
    Web App was published successfully https://EchoBot20190805125647.azurewebsites.net/
    
  9. 默认浏览器应打开并显示一个页面,其中显示 "你的机器人已就绪!"。Your default browser should open and display a page that reads: "Your bot is ready!".

  10. 此时,请查看 Azure 门户中的资源组 SpeechEchoBotTutorial- 资源组,并确认这三个资源:At this point, check your Resource Group SpeechEchoBotTutorial-ResourceGroup in the Azure portal, and confirm these three resources:

名称Name 类型Type 位置Location
EchoBot20190805125647EchoBot20190805125647 应用服务App Service 美国西部West US
SpeechEchoBotTutorial-AppServicePlanSpeechEchoBotTutorial-AppServicePlan 应用服务计划App Service plan 美国西部West US
SpeechEchoBotTutorial-SpeechSpeechEchoBotTutorial-Speech 认知服务Cognitive Services 美国西部West US

启用 Web 套接字Enable web sockets

需要进行较小的配置更改,使机器人能够使用 web 套接字与直接线路语音通道通信。You'll need to make a small configuration change so that your bot can communicate with the Direct Line Speech channel using web sockets. 按照以下步骤启用 web 套接字:Follow these steps to enable web sockets:

  1. 导航到 " Azure 门户",然后单击 "应用服务"。Navigate to the Azure portal, and click your App Service. 资源应命名为类似于 EchoBot20190805125647 (唯一的应用名称) 。The resource should be named similar to EchoBot20190805125647 (your unique app name).
  2. 在左侧导航窗格中的 " 设置" 下,单击 " 配置"。In the left navigation pane, under Settings, click Configuration.
  3. 选择 " 常规设置 " 选项卡。Select the General settings tab.
  4. 找到 " Web 套接字 " 的切换,并将其设置为 "开"。Locate the toggle for Web sockets and set it to On.
  5. 单击“ 保存”。Click Save.

提示

您可以使用 Azure App Service 页顶部的控件来停止或重新启动该服务。You can use the controls at the top of your Azure App Service page to stop or restart the service. 故障排除时,这可能很方便。This may come in handy when troubleshooting.

创建频道注册Create a channel registration

创建用于托管机器人的 Azure App Service 后,下一步就是创建 机器人频道注册Now that you've created an Azure App Service to host your bot, the next step is to create a Bot Channels Registration. 创建频道注册是将机器人注册到机器人框架通道(包括直接线路语音通道)的先决条件。Creating a channel registration is a prerequisite for registering your bot with Bot Framework channels, including Direct Line Speech channel. 如果要了解有关 bot 如何使用频道的详细信息,请参阅 将 Bot 连接到通道If you'd like to learn more about how bots use channels, see Connect a bot to channels.

  1. 创建 Azure 机器人频道注册 Create an Azure Bot Channels Registration
  2. 系统将提示你提供一些信息:You'll be prompted to provide some information:
    • 对于 机器人句柄,输入 SpeechEchoBotTutorial-BotRegistration-# # # #,并将替换为你 #### 选择的数字。For Bot handle, enter SpeechEchoBotTutorial-BotRegistration-#### and replace #### with your a number of your choice. 请注意,Bot 句柄必须全局唯一。Note that Bot handle must be globally unique. 如果输入机器人句柄,但收到错误消息,则 请求的机器人 ID 不可用,请选择其他数字。If you enter a Bot handle but get the error message The requested bot ID is not available, then pick a different number. 在下面的示例中,我们使用了8726In the examples below we used 8726
    • 对于 " 订阅",请选择 " 免费试用"。For Subscription, select Free Trial.
    • 对于 " 资源组",选择 " SpeechEchoBotTutorial"。For Resource group, select SpeechEchoBotTutorial-ResourceGroup.
    • 对于 " 位置",请选择 " 美国西部"。For Location, select West US.
      • 对于 " 定价层",请选择 F0For Pricing tier, select F0.
      • 对于 " 消息传递终结点",请输入 web 应用的 URL, /api/messages 路径后面追加了路径。For Messaging endpoint, enter the URL for your web app with the /api/messages path appended at the end. 例如:如果全局唯一的应用名称为 EchoBot20190805125647,则消息传递终结点将为: https://EchoBot20190805125647.azurewebsites.net/api/messages/For example: if your globally unique App Name was EchoBot20190805125647, your messaging endpoint would be: https://EchoBot20190805125647.azurewebsites.net/api/messages/.
      • 对于 application insights,你可以将其设置为 OffFor Application insights, you can set this to Off. 有关详细信息,请参阅 机器人分析For more information, see Bot analytics.
      • 忽略 自动创建应用 ID 和密码Ignore Auto create App ID and password.
  3. 在 " 机器人频道注册 " 边栏选项卡底部,单击 " 创建"。At the bottom of the Bot Channels Registration blade, click Create.

此时,请查看 Azure 门户中的资源组 SpeechEchoBotTutorialAt this point, check your Resource Group SpeechEchoBotTutorial-ResourceGroup in the Azure portal. 它现在应显示至少四个资源:It should now show at least four resources:

名称Name 类型Type 位置Location
EchoBot20190805125647EchoBot20190805125647 应用服务App Service 美国西部West US
SpeechEchoBotTutorial-AppServicePlanSpeechEchoBotTutorial-AppServicePlan 应用服务计划App Service plan 美国西部West US
SpeechEchoBotTutorial-BotRegistration-8726SpeechEchoBotTutorial-BotRegistration-8726 机器人频道注册Bot Channels Registration 全球Global
SpeechEchoBotTutorial-SpeechSpeechEchoBotTutorial-Speech 认知服务Cognitive Services 美国西部West US

重要

即使你选择了 "美国西部",Bot 通道注册资源也会显示全局区域。The Bot Channels Registration resource will show the Global region even though you selected West US. 这是正常情况。This is expected.

可选:在 web 聊天中测试Optional: Test in web chat

"Azure 机器人频道注册" 页在 "机器人管理" 下有一个 " Web 聊天" 选项。The Azure Bot Channels Registration page has a Test in Web Chat option under Bot Management. 默认情况下,它不会与机器人一起使用,因为 web 聊天需要针对机器人进行身份验证。It will not work by default with your bot, since web chat needs to authenticate against your bot. 如果要测试已部署的机器人并提供文本输入,请遵循以下步骤。If you would like to test your deployed bot with text input, follow the steps below. 请注意,这些步骤是可选的,无需执行本教程的后续步骤。Note that these steps are optional, and are not required in order to continue with the next steps of the tutorial.

  1. Azure 门户中找到并打开 EchoBotTutorial-BotRegistration-# # # # 资源Locate and open your EchoBotTutorial-BotRegistration-#### resource in the Azure portal
  2. 机器人管理 导航中,选择 " 设置"。From the Bot management navigation, select Settings. 复制 Microsoft 应用 ID 下的值Copy the value under Microsoft App ID
  3. 打开 Visual Studio EchoBot 解决方案。Open the Visual Studio EchoBot solution. 在 "解决方案资源管理器" 中,找到并双击 appsettings.jsIn the solution explorer, locate and double click on appsettings.json
  4. 将 JSON 文件中 " MicrosoftAppId " 旁边的空字符串替换为复制的 ID 值Replace the empty string next to MicrosoftAppId in the JSON file with the copied ID value
  5. 返回到 Azure 门户,在 机器人管理 导航中,选择 "设置",然后单击 " Microsoft 应用 ID " 旁的 (管理)Got back to the Azure portal, in Bot management navigation, select Settings, and click on (Manage) next to Microsoft App ID
  6. 单击 " 新建客户端密钥"。Click on New client secret. 添加 "说明" (例如 "web chat" ) 并单击 " 添加"。Add a description (e.g. "web chat") and click Add. 复制新机密Copy the new secret
  7. 将 JSON 文件中 MicrosoftAppPassword 旁边的空字符串替换为复制的机密值Replace the empty string next to MicrosoftAppPassword in the JSON file with the copied secret value
  8. 保存 JSON 文件。Save the JSON file. 结果应如下所示:It should look something like this:
{
  "MicrosoftAppId": "3be0abc2-ca07-475e-b6c3-90c4476c4370",
  "MicrosoftAppPassword": "-zRhJZ~1cnc7ZIlj4Qozs_eKN.8Cq~U38G"
}
  1. 重新发布应用 (右键单击 Visual Studio 解决方案资源管理器中的 EchoBot 项目,选择 " 发布 ... ",然后单击 " 发布 " 按钮) Re-publish the app (right-click on EchoBot project in Visual Studio solution explorer, select Publish... and click on the Publish button)
  2. 现在,你已准备好在 web 聊天中测试机器人!Now you are ready to test the bot in web chat!

注册直接连线语音通道Register the Direct Line Speech channel

现在是时候将机器人注册到了直接线路语音通道。Now it's time to register your bot with the Direct Line Speech channel. 此通道在你的机器人和使用语音 SDK 编译的客户端应用之间创建连接。This channel creates a connection between your bot and a client app compiled with the Speech SDK.

  1. Azure 门户中找到并打开 SpeechEchoBotTutorial-BotRegistration-# # # # 资源。Locate and open your SpeechEchoBotTutorial-BotRegistration-#### resource in the Azure portal.

  2. 机器人管理 导航中,选择 " 通道"。From the Bot management navigation, select Channels.

    • 在 " 更多频道" 下,单击 " 直行语音"。Under More channels, click Direct Line Speech.
    • 查看标题为 " 配置直接线路语音" 的页面上的文本,然后展开 " 认知服务帐户 " 下拉菜单。Review the text on the page titled Configure Direct line Speech, then expand the Cognitive service account drop-down menu.
    • 选择之前创建的语音资源 (例如,从菜单中选择 " SpeechEchoBotTutorial ") ,将机器人关联到语音订阅密钥。Select the speech resource you created earlier (e.g., SpeechEchoBotTutorial-Speech) from the menu to associate your bot to your speech subscription key.
    • 忽略其余的可选字段。Ignore the rest of the optional fields.
    • 单击“ 保存”。Click Save.
  3. 机器人管理 导航中,单击 " 设置"。From the Bot management navigation, click Settings.

    • 选中标记为 启用流式处理终结点 的框。Check the box labeled Enable Streaming Endpoint. 这是在机器人和直接线路语音通道之间创建在 web 套接字上构建的通信协议所必需的。This is needed to create a communication protocol built on web sockets between your bot and the Direct Line Speech channel.
    • 单击“ 保存”。Click Save.

提示

若要了解详细信息,请参阅 连接机器人以定向语音If you'd like to learn more, see Connect a bot to Direct Line Speech. 此页包含其他信息和已知问题。This page includes additional information and known issues.

运行 Windows 语音助手客户端Run the Windows Voice Assistant Client

在此步骤中,你将运行 Windows 语音助手客户端。In this step, you will run the Windows Voice Assistant Client. 客户端是 c # 中的一种 Windows Presentation Foundation (WPF) 应用程序,该应用程序使用 语音 SDK 来管理与机器人的通信。The client is a Windows Presentation Foundation (WPF) app in C# that uses the Speech SDK to manage communication with your bot using the Direct Line Speech channel. 在编写自定义客户端应用之前,可以使用它来与机器人进行交互和测试。Use it to interact with and test your bot before writing a custom client app. 它是开源的,因此您可以下载并运行可执行文件,也可以自行生成。It's open source, so you can either download the executable and run it, or build it yourself.

Windows 语音助手客户端具有一个简单的 UI,可用于配置与机器人的连接、查看文本对话、以 JSON 格式查看 Bot 框架活动,以及显示自适应卡。The Windows Voice Assistant Client has a simple UI that allows you to configure the connection to your bot, view the text conversation, view Bot Framework activities in JSON format, and display adaptive cards. 它还支持使用自定义关键字。It also supports the use of custom keywords. 你将使用此客户端与机器人对话,并收到语音回复。You'll use this client to speak with your bot and receive a voice response.

备注

此时,请确认麦克风和扬声器已启用且正常工作。At this point, confirm that your microphone and speakers are enabled and working.

  1. 导航至 Windows 语音助手客户端的 GitHub 存储库。Navigate to the GitHub repository for the Windows Voice Assistant Client.

  2. 按照此处提供的说明进行操作Follow the instructions provided there to either

    • 下载 ZIP 包中要运行的预生成的可执行文件,或者download a pre-built executable in a ZIP package to run, or
    • 通过克隆存储库和生成项目来自行生成可执行文件。build the executable yourself, by cloning the repository and building the project.
  3. 按照 VoiceAssistantClient.exe GitHub 存储库中的说明,启动客户端应用程序并将其配置为连接到你的机器人。Launch the VoiceAssistantClient.exe client application and configure it to connect to your bot, by following the instructions in the GitHub repository.

  4. 单击 " 重新连接 ",确保看到 " 新会话已启动" 的消息-键入或按下麦克风按钮Click Reconnect and make sure you see the message New conversation started - type or press the microphone button.

  5. 让我们对它进行测试。单击麦克风按钮,并口述几个英语单词。Let's test it out. Click the microphone button, and speak a few words in English. 将显示已识别的文本。The recognized text will appear as you speak. 完成此操作后,bot 将以自己的语音进行回复,并显示 "echo" 后跟识别的单词。When you're done speaking, the bot will reply in its own voice, saying "echo" followed by the recognized words.

  6. 你还可以使用文本与机器人通信。You can also use text to communicate with the bot. 只需在底部栏中键入文本。Just type in the text at the bottom bar.

排查 Windows 语音助手客户端中的错误Troubleshooting errors in Windows Voice Assistant Client

如果在主应用窗口中收到错误消息,请使用此表来确定并解决该错误:If you get an error message in your main app window, use this table to identify and troubleshoot the error:

错误Error 你应该如何操作?What should you do?
AuthenticationFailure) (错误: WebSocket 升级失败,出现身份验证错误 (401) 。Error (AuthenticationFailure) : WebSocket Upgrade failed with an authentication error (401). 检查正确的订阅密钥 (或授权令牌) 和区域名称Check for correct subscription key (or authorization token) and region name 在应用的 "设置" 页中,确保已正确输入语音订阅密钥及其区域。In the Settings page of the app, make sure you entered the Speech Subscription key and its region correctly.
请确保正确输入了语音密钥和密钥区域。Make sure your speech key and key region were entered correctly.
ConnectionFailure) (错误:连接已被远程主机关闭。Error (ConnectionFailure) : Connection was closed by the remote host. 错误代码:1011。Error code: 1011. 错误详细信息:无法在发送消息之前连接到机器人Error details: We could not connect to the bot before sending a message 请确保 已选中 "启用流式处理终结点"框和/或将 Web 套接字 切换到 "打开"。Make sure you checked the "Enable Streaming Endpoint" box and/or toggled Web sockets to On.
请确保 Azure App Service 正在运行。Make sure your Azure App Service is running. 如果已启动,请尝试重新启动应用服务。If it is, try restarting your App Service.
ConnectionFailure) (错误:连接已被远程主机关闭。Error (ConnectionFailure) : Connection was closed by the remote host. 错误代码:1002。Error code: 1002. 错误详细信息:当需要状态代码 "101" 时,服务器返回了状态代码 "503"Error details: The server returned status code '503' when status code '101' was expected 请确保 已选中 "启用流式处理终结点"框和/或将 Web 套接字 切换到 "打开"。Make sure you checked the "Enable Streaming Endpoint" box and/or toggled Web sockets to On.
请确保 Azure App Service 正在运行。Make sure your Azure App Service is running. 如果已启动,请尝试重新启动应用服务。If it is, try restarting your App Service.
ConnectionFailure) (错误:连接已被远程主机关闭。Error (ConnectionFailure) : Connection was closed by the remote host. 错误代码:1011。Error code: 1011. 错误详细信息:响应状态代码不指示成功: 500 (InternalServerError) Error details: Response status code does not indicate success: 500 (InternalServerError) 机器人在其输出活动的 " 朗读 " 字段中指定了一个神经声音,但与您的语音订阅密钥关联的 Azure 区域不支持神经语音。Your bot specified a neural voice in its output Activity Speak field, but the Azure region associated with your Speech subscription key does not support neural voices. 请参阅 标准和神经声音See Standard and neural voices.

如果未在表中解决问题,请参阅 语音助手:常见问题解答If your issue isn't addressed in the table, see Voice assistants: Frequently asked questions. 如果在执行本教程中的所有步骤后仍无法解决问题,请在 " 语音助手 GitHub" 页中输入新问题。If your are still not able to resolve your issue after following all the steps in this tutorial, please enter a new issue in the Voice Assistant GitHub page.

有关连接超时的说明A note on connection time out

如果你已连接到 bot,而在过去5分钟内未发生任何活动,则该服务将使用客户端和机器人自动关闭 websocket 连接。If you are connected to a bot and no activity happened in the last 5 minutes, the service will automatically close the websocket connection with the client and with the bot. 这是设计的结果。This is by design. 底部栏中会出现一条消息: "活动连接已超时,但可以按需重新连接"A message will appear in the bottom bar: "Active connection timed out but ready to reconnect on demand". 不需要按 "重新连接" 按钮,只需按下麦克风按钮并开始谈话,键入短信,或者说关键字 (如果启用了) 。You do not need to press the "Reconnect" button - simply press the microphone button and start talking, type in a text message, or say the keyword (if one is enabled). 将自动重新建立连接。The connection will automatically be reestablished.

查看机器人活动View bot activities

每个机器人发送和接收 活动 消息。Every bot sends and receives Activity messages. 在 Windows 语音助手客户端的 " 活动日志 " 窗口中,你将看到带时间戳的日志,其中包含客户端从机器人接收的每个活动。In the Activity Log window of Windows Voice Assistant Client, you'll see timestamped logs with each activity that the client has received from the bot. 还可以查看客户端使用方法发送到机器人的活动 DialogServiceConnector.SendActivityAsyncYou can also see the activities that the client sent to the bot using the DialogServiceConnector.SendActivityAsync method. 选择日志项时,它会将关联活动的详细信息显示为 JSON。When you select a log item, it will show the details of the associated activity as JSON.

下面是客户端收到的活动的示例 json:Here's a sample json of an Activity the client received:

{
    "attachments":[],
    "channelData":{
        "conversationalAiData":{
             "requestInfo":{
                 "interactionId":"8d5cb416-73c3-476b-95fd-9358cbfaebfa",
                 "version":"0.2"
             }
         }
    },
    "channelId":"directlinespeech",
    "conversation":{
        "id":"129ebffe-772b-47f0-9812-7c5bfd4aca79",
        "isGroup":false
    },
    "entities":[],
    "from":{
        "id":"SpeechEchoBotTutorial-BotRegistration-8726"
    },
    "id":"89841b4d-46ce-42de-9960-4fe4070c70cc",
    "inputHint":"acceptingInput",
    "recipient":{
        "id":"129ebffe-772b-47f0-9812-7c5bfd4aca79|0000"
    },
    "replyToId":"67c823b4-4c7a-4828-9d6e-0b84fd052869",
    "serviceUrl":"urn:botframework:websocket:directlinespeech",
    "speak":"<speak version='1.0' xmlns='https://www.w3.org/2001/10/synthesis' xml:lang='en-US'><voice name='Microsoft Server Speech Text to Speech Voice (en-US, AriaRUS)'>Echo: Hello and welcome.</voice></speak>",
    "text":"Echo: Hello and welcome.",
    "timestamp":"2019-07-19T20:03:51.1939097Z",
    "type":"message"
}

若要详细了解 JSON 输出中返回的内容,请参阅 活动中的字段To learn more about what's returned in the JSON output, see fields in the Activity. 出于本教程的目的,您可以将重点放在 文本讲述 字段上。For the purpose of this tutorial, you can focus on the Text and Speak fields.

查看调用语音 SDK 的客户端源代码View client source code for calls to the Speech SDK

Windows 语音助手客户端使用包含语音 SDK 的 NuGet 包cognitiveservices account。The Windows Voice Assistant Client uses the NuGet package Microsoft.CognitiveServices.Speech, which contains the Speech SDK. 开始查看示例代码的好地方是 InitSpeechConnector 文件中 # A1 ( 方法 VoiceAssistantClient\MainWindow.xaml.cs ,这会创建两个语音 SDK 对象:A good place to start reviewing the sample code is the method InitSpeechConnector() in file VoiceAssistantClient\MainWindow.xaml.cs, which creates these two Speech SDK objects:

  • DialogServiceConfig -对于配置设置 (例如,语音订阅密钥、密钥区域) DialogServiceConfig - For configuration settings (e.g., speech subscription key, key region)
  • DialogServiceConnector -管理通道连接和客户端订阅事件,以便处理识别的语音和机器人响应。DialogServiceConnector - To manage the channel connection and client subscription events for handling recognized speech and bot responses.

添加自定义关键字激活Add custom keyword activation

语音 SDK 支持自定义关键字激活。The Speech SDK supports custom keyword activation. 与 Microsoft 助手的 "你好 Cortana" 类似,你可以编写一个应用,该应用会持续侦听你选择的关键字。Similar to "Hey Cortana" for Microsoft's Assistant, you can write an app that will continuously listen for a keyword of your choice. 请记住,关键字可以是单个词,也可以是多单词短语。Keep in mind that a keyword can be single word or a multi-word phrase.

备注

术语 关键字 通常与字词 唤醒词 互换使用,你可能会看到在 Microsoft 文档中使用了这两项。The term keyword is often used interchangeably with the term wake word, and you may see both used in Microsoft documentation.

关键字检测在客户端应用上完成。Keyword detection is done on the client app. 如果使用关键字,则只有在检测到关键字时才会将音频流式传输到直接语音通道。If using a keyword, audio is only streamed to the Direct Line Speech channel if the keyword is detected. 直接连线语音频道包括一个名为 关键字验证 (KWV) 的组件,它在云中执行更复杂的处理,以验证所选关键字是否位于音频流的开头。The Direct Line Speech channel includes a component called keyword verification (KWV), which does more complex processing in the cloud to verify that the keyword you've chosen is at the start of the audio stream. 如果关键字 word 验证成功,则通道将与机器人通信。If key word verification succeeds, then the channel will communicate with the bot.

按照以下步骤创建关键字模型,将 Windows 语音助手客户端配置为使用此模型,最后使用机器人进行测试。Follow these steps to create a keyword model, configure the Windows Voice Assistant Client to use this model, and finally, test it with your bot.

  1. 按照这些说明 使用语音服务创建自定义关键字Follow these instructions to create a custom keyword by using the Speech service.
  2. 解压缩在上一步中下载的模型文件。Unzip the model file that you downloaded in the previous step. 应为关键字命名。It should be named for your keyword. 正在查找名为的文件 kws.tableYou're looking for a file named kws.table.
  3. 在 Windows 语音助手客户端中,找到 " 设置 " 菜单 (在右上) 查找齿轮图标。In the Windows Voice Assistant Client, locate the Settings menu (look for the gear icon in the top right). 查找 " 模型文件路径 " 并输入 kws.table 步骤2中的文件的完整路径名称。Locate Model file path and enter the full path name for the kws.table file from step 2.
  4. 确保选中标记为 " 已启用" 的框。Make sure to check the box labeled Enabled. 在复选框旁边应会看到此消息: "下次连接时将侦听关键字"。You should see this message next to the check box: "Will listen for the keyword upon next connection". 如果提供了错误的文件或路径无效,应会看到一条错误消息。If you've provided the wrong file or an invalid path, you should see an error message.
  5. 输入语音 订阅密钥订阅密钥区域,然后单击 "确定" 以关闭 " 设置 " 菜单。Enter your speech subscription key, subscription key region, and then click OK to close the Settings menu.
  6. 单击 " 重新连接"。Click Reconnect. 应该会看到一条消息,其中显示: "新会话已启动-类型,按下麦克风按钮,或说关键字为"。You should see a message that reads: "New conversation started - type, press the microphone button, or say the keyword". 应用现在正在持续侦听。The app is now continuously listening.
  7. 说任何以关键字开头的短语。Speak any phrase that starts with your keyword. 例如: "{关键字},它的时间是多少?"。For example: "{your keyword}, what time is it?". Uttering 关键字后,不需要暂停。You don't need to pause after uttering the keyword. 完成后,会发生两种情况:When finished, two things happen:
    • 你会看到你的辐射You'll see a transcription of what you spoke
    • 不久之后,你会听到机器人的响应Shortly after, you'll hear the bot's response
  8. 继续试验机器人支持的三个输入类型:Continue to experiment with the three input types that your bot supports:
    • 键入文本在底部栏中Typed-text in the bottom bar
    • 按下麦克风图标和讲话Pressing the microphone icon and speaking
    • 说以关键字开头的短语Saying a phrase that starts with your keyword

查看启用关键字的源代码View the source code that enables keyword

在 Windows 语音助手客户端源代码中,查看以下文件以查看用于启用关键字检测的代码:In the Windows Voice Assistant Client source code, take a look at these files to review the code that's used to enable keyword detection:

  1. VoiceAssistantClient\Models.cs 包括对语音 SDK 方法的调用 KeywordRecognitionModel.fromFile() ,该方法用于实例化磁盘上的本地文件中的模型。VoiceAssistantClient\Models.cs includes a call to the Speech SDK method KeywordRecognitionModel.fromFile(), which is used to instantiate the model from a local file on disk.
  2. VoiceAssistantClient\MainWindow.xaml.cs 包括对语音 SDK 方法的调用 DialogServiceConnector.StartKeywordRecognitionAsync() ,该方法激活连续关键字检测。VoiceAssistantClient\MainWindow.xaml.cs includes a call to Speech SDK method DialogServiceConnector.StartKeywordRecognitionAsync(), which activates continuous keyword detection.

(可选) 更改语言和机器人声音(Optional) Change the language and bot voice

你创建的机器人将侦听并响应英语,其中默认为美国英语文本语音。The bot that you've created will listen for and respond in English, with a default US English text-to-speech voice. 但是,你并不局限于使用英语或默认语音。However, you're not limited to using English, or a default voice. 在本部分中,你将了解如何更改机器人侦听和响应的语言。In this section, you'll learn how to change the language that your bot will listen for and respond in. 你还将了解如何为该语言选择其他语音。You will also learn how to select a different voice for that language.

更改语言Change the language

你可以从 " 语音到文本 " 表中所述的任何一种语言中进行选择。You can chose from any one of the languages mentioned in the speech-to-text table. 在下面的示例中,我们将语言更改为德语。In the example below, we will change the language to German.

  1. 打开 Windows 语音助手客户端应用程序,单击 "设置" 按钮 (右上方的齿轮图标) ,并 de-de 在 "语言" 字段中输入 (这是 " 语音到文本 " 表) 中提到的区域设置值。Open the Windows Voice Assistant Client app, click on the settings button (upper-right gear icon), and enter de-de in the Language field (this is the Locale value mentioned in the speech-to-text table). 这将设置要识别的口述语言,并覆盖默认语言 en-usThis sets the spoken language to be recognized, overriding the default en-us. 这也指示 "直接线路语音频道" 使用默认的德语语音进行机器人回复。This also instructs Direct Line Speech channel to use a default German voice for the Bot reply.
  2. 关闭 "设置" 页,然后单击 "重新连接" 按钮,建立与 echo bot 的新连接。Close the settings page, and click on the Reconnect button to establish a new connection to your echo bot.
  3. 单击麦克风按钮,并口述德语短语。Click on the microphone button, and say a phrase in German. 你将看到已识别的文本,并且 echo bot 将用默认的德语语音进行回复。You will see the recognized text and the echo bot replying with the default German voice.

更改默认机器人语音Change the default bot voice

如果机器人以 语音合成标记语言 ( (SSML) ,而不是简单文本)的形式指定答复,则可以选择文本到语音声音并控制发音。Selecting the text-to-speech voice and controlling pronunciation can be done if the Bot specifies the reply in the form of a Speech Synthesis Markup Language (SSML) instead of simple text. Echo bot 不使用 SSML,但我们可以轻松修改代码来执行此操作。The echo bot does not use SSML, but we can easily modify the code to do that. 在下面的示例中,我们将 SSML 添加到回响机器人回复,以便使用 Stefan) (Apollo 的德语语音,而不是使用默认的女性声音。In the example below we add SSML to the echo bot reply, such that the German voice Stefan Apollo (a male voice) will be used instead of the default female voice. 请参阅你的语言支持的 标准语音神经声音 列表。See list of Standard Voices and Neural Voices supported for your language.

  1. 首先打开 samples\csharp_dotnetcore\02.echo-bot\echo-bot.csLet's start by opening samples\csharp_dotnetcore\02.echo-bot\echo-bot.cs.
  2. 找到以下两行:Locate these two lines:
    var replyText = $"Echo: {turnContext.Activity.Text}";
    await turnContext.SendActivityAsync(MessageFactory.Text(replyText, replyText), cancellationToken);
    
  3. 将它们替换为:Replace them with:
    var replyText = $"Echo: {turnContext.Activity.Text}";
    var replySpeak = @"<speak version='1.0' xmlns='https://www.w3.org/2001/10/synthesis' xml:lang='de-DE'>
                    <voice name='Microsoft Server Speech Text to Speech Voice (de-DE, Stefan, Apollo)'>" +
                    $"{replyText}" + "</voice></speak>";
    await turnContext.SendActivityAsync(MessageFactory.Text(replyText, replySpeak), cancellationToken);
    
  4. 在 Visual Studio 中生成解决方案并修复任何生成错误。Build your solution in Visual Studio and fix any build errors.

方法 "MessageFactory" 中的第二个参数设置了机器人回复中的 活动讲述字段The second argument in the method 'MessageFactory.Text' sets the Activity speak field in the bot reply. 进行上述更改后,已将其从简单文本替换为 SSML,以指定非默认的德语语音。With the above change, it has been replaced from simple text to SSML in order to specify a non-default German voice.

重新部署机器人Redeploy your bot

完成机器人的必要更改后,下一步就是将其重新发布到 Azure App Service 并试用:Now that you've made the necessary change to the bot, the next step is to republish it to your Azure App Service and try it out:

  1. 在 "解决方案资源管理器" 窗口中,右键单击 EchoBot 项目,然后选择 " 发布"。In the Solution Explorer window, right-click on the EchoBot project and select Publish.
  2. 以前的部署配置已作为默认值加载。Your previous deployment configuration has already been loaded as the default. 只需单击 "EchoBot20190805125647" 旁边 Web 部署的"发布"。Simply click Publish next to EchoBot20190805125647 - Web Deploy.
  3. " 发布成功 " 消息将显示在 Visual Studio 的 "输出" 窗口中,并且网页将启动并显示消息 "机器人已就绪!"。The Publish Succeeded message will appear in the Visual Studio output window, and a web page will launch with the message "Your bot is ready!".
  4. 打开 Windows 语音助手客户端应用程序,单击 "设置" 按钮 (右上方的齿轮图标) ,并确保 de-de 在 "语言" 字段中仍有。Open the Windows Voice Assistant Client app, click on the settings button (upper-right gear icon), and make sure you still have de-de in the Language field.
  5. 按照 运行 Windows 语音助手客户端 中的说明重新连接到你的新部署的机器人,并以新语言讲述,并听到你用新的语音回复该语言的机器人。Follow the instructions in Run the Windows Voice Assistant Client to reconnect with your newly deployed bot, speak in the new language and hear you bot reply in that language with the new voice.

清理资源Clean up resources

如果不打算继续使用本教程中部署的 echo 机器人,只需删除 Azure 资源组 SpeechEchoBotTutorial,即可将其删除及其关联的所有 Azure 资源。If you're not going to continue using the echo-bot deployed in this tutorial, you can remove it and all its associated Azure resources by simply deleting the Azure Resource group SpeechEchoBotTutorial-ResourceGroup.

  1. 从 " Azure 门户中,单击" Azure 服务"导航中的"资源组"。From the Azure portal, click on Resource Groups from the Azure services navigation.
  2. 找到名为 " SpeechEchoBotTutorial" 的资源组。Find the resource group named: SpeechEchoBotTutorial-ResourceGroup. 单击 ) ( 三个点。Click the three dots (...).
  3. 选择“删除资源组”。Select Delete resource group.

后续步骤Next steps

另请参阅See also