Tutorial: Build a Flask app with Azure Cognitive Services

In this tutorial, you'll build a Flask web app that uses Azure Cognitive Services to translate text, analyze sentiment, and synthesize translated text into speech. Our focus is on the Python code and Flask routes that enable our application, however, we will help you out with the HTML and Javascript that pulls the app together. If you run into any issues let us know using the feedback button below.

Here's what this tutorial covers:

  • Get Azure subscription keys
  • Set up your development environment and install dependencies
  • Create a Flask app
  • Use the Translator Text API to translate text
  • Use Text Analytics to analyze positive/negative sentiment of input text and translations
  • Use Speech Services to convert translated text into synthesized speech
  • Run your Flask app locally

Tip

If you'd like to skip ahead and see all the code at once, the entire sample, along with build instructions are available on GitHub.

What is Flask?

Flask is a microframework for creating web applications. This means Flask provides you with tools, libraries, and technologies that allow you to build a web application. This web application can be some web pages, a blog, a wiki or go as substantive as a web-based calendar application or a commercial website.

For those of you who want to deep dive after this tutorial here are a few helpful links:

Prerequisites

Let's review the software and subscription keys that you'll need for this tutorial.

Create an account and subscribe to resources

As previously mentioned, you're going to need three subscription keys for this tutorial. This means that you need to create a resource within your Azure account for:

  • Translator Text
  • Text Analytics
  • Speech Services

Use Create a Cognitive Services Account in the Azure portal for step-by-step instructions to create resources.

Important

For this tutorial, please create your resources in the West US region. If using a different region, you'll need to adjust the base URL in each of your Python files.

Set up your dev environment

Before you build your Flask web app, you'll need to create a working directory for your project and install a few Python packages.

Create a working directory

  1. Open command line (Windows) or terminal (macOS/Linux). Then, create a working directory and sub directories for your project:

    mkdir -p flask-cog-services/static/scripts && mkdir flask-cog-services/templates
    
  2. Change to your project's working directory:

    cd flask-cog-services
    

Create and activate your virtual environment with virtualenv

Let's create a virtual environment for our Flask app using virtualenv. Using a virtual environment ensures that you have a clean environment to work from.

  1. In your working directory, run this command to create a virtual environment: macOS/Linux:

    virtualenv venv --python=python3
    

    We've explicitly declared that the virtual environment should use Python 3. This ensures that users with multiple Python installations are using the correct version.

    Windows CMD / Windows Bash:

    virtualenv venv
    

    To keep things simple, we're naming your virtual environment venv.

  2. The commands to activate your virtual environment will vary depending on your platform/shell:

    Platform Shell Command
    macOS/Linux bash/zsh source venv/bin/activate
    Windows bash source venv/Scripts/activate
    Command Line venv\Scripts\activate.bat
    PowerShell venv\Scripts\Activate.ps1

    After running this command, your command line or terminal session should be prefaced with venv.

  3. You can deactivate the session at any time by typing this into the command line or terminal: deactivate.

Note

Python has extensive documentation for creating and managing virtual environments, see virtualenv.

Install requests

Requests is a popular module that is used to send HTTP 1.1 requests. There’s no need to manually add query strings to your URLs, or to form-encode your POST data.

  1. To install requests, run:

    pip install requests
    

Note

If you'd like to learn more about requests, see Requests: HTTP for Humans.

Install and configure Flask

Next we need to install Flask. Flask handles the routing for our web app, and allows us to make server-to-server calls that hide our subscription keys from the end user.

  1. To install Flask, run:

    pip install Flask
    

    Let's make sure Flask was installed. Run:

    flask --version
    

    The version should be printed to terminal. Anything else means something went wrong.

  2. To run the Flask app, you can either use the flask command or Python’s -m switch with Flask. Before you can do that you need to tell your terminal which app to work with by exporting the FLASK_APP environment variable:

    macOS/Linux:

    export FLASK_APP=app.py
    

    Windows:

    set FLASK_APP=app.py
    

Create your Flask app

In this section, you're going to create a barebones Flask app that returns an HTML file when users hit the root of your app. Don't spend too much time trying to pick apart the code, we'll come back to update this file later.

What is a Flask route?

Let's take a minute to talk about "routes". Routing is used to bind a URL to a specific function. Flask uses route decorators to register functions to specific URLs. For example, when a user navigates to the root (/) of our web app, index.html is rendered.

@app.route('/')
def index():
    return render_template('index.html')

Let's take a look at one more example to hammer this home.

@app.route('/about')
def about():
    return render_template('about.html')

This code ensures that when a user navigates to http://your-web-app.com/about that the about.html file is rendered.

While these samples illustrate how to render html pages for a user, routes can also be used to call APIs when a button is pressed, or take any number of actions without having to navigate away from the homepage. You'll see this in action when you create routes for translation, sentiment, and speech synthesis.

Get started

  1. Open the project in your IDE, then create a file named app.py in the root of your working directory. Next, copy this code into app.py and save:

    from flask import Flask, render_template, url_for, jsonify, request
    
    app = Flask(__name__)
    app.config['JSON_AS_ASCII'] = False
    
    @app.route('/')
    def index():
        return render_template('index.html')
    

    This code block tells the app to display index.html whenever a user navigates to the root of your web app (/).

  2. Next, let's create the front-end for our web app. Create a file named index.html in the templates directory. Then copy this code into templates/index.html.

    <!doctype html>
    <html lang="en">
      <head>
        <!-- Required metadata tags -->
        <meta charset="utf-8">
        <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
        <meta name="description" content="Translate and analyze text with Azure Cognitive Services.">
        <!-- Bootstrap CSS -->
        <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0/css/bootstrap.min.css" integrity="sha384-Gn5384xqQ1aoWXA+058RXPxPg6fy4IWvTNh0E263XmFcJlSAwiGgFAW/dAiS6JXm" crossorigin="anonymous">
        <title>Translate and analyze text with Azure Cognitive Services</title>
      </head>
      <body>
        <div class="container">
          <h1>Translate, synthesize, and analyze text with Azure</h1>
          <p>This simple web app uses Azure for text translation, text-to-speech conversion, and sentiment analysis of input text and translations. Learn more about <a href="https://docs.microsoft.com/azure/cognitive-services/">Azure Cognitive Services</a>.
         </p>
         <!-- HTML provided in the following sections goes here. -->
    
         <!-- End -->
        </div>
    
        <!-- Required Javascript for this tutorial -->
        <script src="https://code.jquery.com/jquery-3.2.1.slim.min.js" integrity="sha384-KJ3o2DKtIkvYIK3UENzmM7KCkRr/rE9/Qpg6aAZGJwFDMVNA/GpGFF93hXpG5KkN" crossorigin="anonymous"></script>
        <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
        <script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.12.9/umd/popper.min.js" integrity="sha384-ApNbgh9B+Y1QKtv3Rn7W3mgPxhU9K/ScQsAP7hUibX39j7fakFPskvXusvfa0b4Q" crossorigin="anonymous"></script>
        <script src="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0/js/bootstrap.min.js" integrity="sha384-JZR6Spejh4U02d8jOt6vLEHfe/JQGiRRSQQxSfFWpi1MquVdAyjUar5+76PVCmYl" crossorigin="anonymous"></script>
        <script type = "text/javascript" src ="static/scripts/main.js"></script>
      </body>
    </html>
    
  3. Let's test the Flask app. From the terminal, run:

    flask run
    
  4. Open a browser and navigate to the URL provided. You should see your single page app. Press Ctrl + c to kill the app.

Translate text

Now that you have an idea of how a simple Flask app works, let's:

  • Write some Python to call the Translator Text API and return a response
  • Create a Flask route to call your Python code
  • Update the HTML with an area for text input and translation, a language selector, and translate button
  • Write Javascript that allows users to interact with your Flask app from the HTML

Call the Translator Text API

The first thing you need to do is write a function to call the Translator Text API. This function will take two arguments: text_input and language_output. This function is called whenever a user presses the translate button in your app. The text area in the HTML is sent as the text_input, and the language selection value in the HTML is sent as language_output.

  1. Let's start by creating a file called translate.py in the root of your working directory.
  2. Next, add this code to translate.py. This function takes two arguments: text_input and language_output.
    import os, requests, uuid, json
    
    # Don't forget to replace with your Cog Services subscription key!
    # If you prefer to use environment variables, see Extra Credit for more info.
    subscription_key = 'YOUR_TRANSLATOR_TEXT_SUBSCRIPTION_KEY'
    
    # Our Flask route will supply two arguments: text_input and language_output.
    # When the translate text button is pressed in our Flask app, the Ajax request
    # will grab these values from our web app, and use them in the request.
    # See main.js for Ajax calls.
    def get_translation(text_input, language_output):
        base_url = 'https://api.cognitive.microsofttranslator.com'
        path = '/translate?api-version=3.0'
        params = '&to=' + language_output
        constructed_url = base_url + path + params
    
        headers = {
            'Ocp-Apim-Subscription-Key': subscription_key,
            'Content-type': 'application/json',
            'X-ClientTraceId': str(uuid.uuid4())
        }
    
        # You can pass more than one object in body.
        body = [{
            'text' : text_input
        }]
        response = requests.post(constructed_url, headers=headers, json=body)
        return response.json()
    
  3. Add your Translator Text subscription key and save.

Add a route to app.py

Next, you'll need to create a route in your Flask app that calls translate.py. This route will be called each time a user presses the translate button in your app.

For this app, your route is going to accept POST requests. This is because the function expects the text to translate and an output language for the translation.

Flask provides helper functions to help you parse and manage each request. In the code provided, get_json() returns the data from the POST request as JSON. Then using data['text'] and data['to'], the text and output language values are passed to get_translation() function available from translate.py. The last step is to return the response as JSON, since you'll need to display this data in your web app.

In the following sections, you'll repeat this process as you create routes for sentiment analysis and speech synthesis.

  1. Open app.py and locate the import statement at the top of app.py and add the following line:

    import translate
    

    Now our Flask app can use the method available via translate.py.

  2. Copy this code to the end of app.py and save:

    @app.route('/translate-text', methods=['POST'])
    def translate_text():
        data = request.get_json()
        text_input = data['text']
        translation_output = data['to']
        response = translate.get_translation(text_input, translation_output)
        return jsonify(response)
    

Update index.html

Now that you have a function to translate text, and a route in your Flask app to call it, the next step is to start building the HTML for your app. The HTML below does a few things:

  • Provides a text area where users can input text to translate.
  • Includes a language selector.
  • Includes HTML elements to render the detected language and confidence scores returned during translation.
  • Provides a read-only text area where the translation output is displayed.
  • Includes placeholders for sentiment analysis and speech synthesis code that you'll add to this file later in the tutorial.

Let's update index.html.

  1. Open index.html and locate these code comments:

    <!-- HTML provided in the following sections goes here. -->
    
    <!-- End -->
    
  2. Replace the code comments with this HTML block:

    <div class="row">
      <div class="col">
        <form>
          <!-- Enter text to translate. -->
          <div class="form-group">
            <label for="text-to-translate"><strong>Enter the text you'd like to translate:</strong></label>
            <textarea class="form-control" id="text-to-translate" rows="5"></textarea>
          </div>
          <!-- Select output language. -->
          <div class="form-group">
            <label for="select-language"><strong>Translate to:</strong></label>
            <select class="form-control" id="select-language">
              <option value="ar">Arabic</option>
              <option value="ca">Catalan</option>
              <option value="zh-Hans">Chinese (Simplified)</option>
              <option value="zh-Hant">Chinese (Traditional)</option>
              <option value="hr">Croatian</option>
              <option value="en">English</option>
              <option value="fr">French</option>
              <option value="de">German</option>
              <option value="el">Greek</option>
              <option value="he">Hebrew</option>
              <option value="hi">Hindi</option>
              <option value="it">Italian</option>
              <option value="ja">Japanese</option>
              <option value="ko">Korean</option>
              <option value="pt">Portuguese</option>
              <option value="ru">Russian</option>
              <option value="es">Spanish</option>
              <option value="th">Thai</option>
              <option value="tr">Turkish</option>
              <option value="vi">Vietnamese</option>
            </select>
          </div>
          <button type="submit" class="btn btn-primary mb-2" id="translate">Translate text</button></br>
          <div id="detected-language" style="display: none">
            <strong>Detected language:</strong> <span id="detected-language-result"></span><br />
            <strong>Detection confidence:</strong> <span id="confidence"></span><br /><br />
          </div>
    
          <!-- Start sentiment code-->
    
          <!-- End sentiment code -->
    
        </form>
      </div>
      <div class="col">
        <!-- Translated text returned by the Translate API is rendered here. -->
        <form>
          <div class="form-group" id="translator-text-response">
            <label for="translation-result"><strong>Translated text:</strong></label>
            <textarea readonly class="form-control" id="translation-result" rows="5"></textarea>
          </div>
    
          <!-- Start voice font selection code -->
    
          <!-- End voice font selection code -->
    
        </form>
    
        <!-- Add Speech Synthesis button and audio element -->
    
        <!-- End Speech Synthesis button -->
    
      </div>
    </div>
    

The next step is to write some Javascript. This is the bridge between your HTML and Flask route.

Create main.js

The main.js file is the bridge between your HTML and Flask route. Your app will use a combination of jQuery, Ajax, and XMLHttpRequest to render content, and make POST requests to your Flask routes.

In the code below, content from the HTML is used to construct a request to your Flask route. Specifically, the contents of the text area and the language selector are assigned to variables, and then passed along in the request to translate-text.

The code then iterates through the response, and updates the HTML with the translation, detected language, and confidence score.

  1. From your IDE, create a file named main.js in the static/scripts directory.
  2. Copy this code into static/scripts/main.js:
    //Initiate jQuery on load.
    $(function() {
      //Translate text with flask route
      $("#translate").on("click", function(e) {
        e.preventDefault();
        var translateVal = document.getElementById("text-to-translate").value;
        var languageVal = document.getElementById("select-language").value;
        var translateRequest = { 'text': translateVal, 'to': languageVal }
    
        if (translateVal !== "") {
          $.ajax({
            url: '/translate-text',
            method: 'POST',
            headers: {
                'Content-Type':'application/json'
            },
            dataType: 'json',
            data: JSON.stringify(translateRequest),
            success: function(data) {
              for (var i = 0; i < data.length; i++) {
                document.getElementById("translation-result").textContent = data[i].translations[0].text;
                document.getElementById("detected-language-result").textContent = data[i].detectedLanguage.language;
                if (document.getElementById("detected-language-result").textContent !== ""){
                  document.getElementById("detected-language").style.display = "block";
                }
                document.getElementById("confidence").textContent = data[i].detectedLanguage.score;
              }
            }
          });
        };
      });
      // In the following sections, you'll add code for sentiment analysis and
      // speech synthesis here.
    })
    

Test translation

Let's test translation in the app.

flask run

Navigate to the provided server address. Type text into the input area, select a language, and press translate. You should get a translation. If it doesn't work, make sure that you've added your subscription key.

Tip

If the changes you've made aren't showing up, or the app doesn't work the way you expect it to, try clearing your cache or opening a private/incognito window.

Press CTRL + c to kill the app, then head to the next section.

Analyze sentiment

The Text Analytics API can be used to perform sentiment analysis, extract key phrases from text, or detect the source language. In this app, we're going to use sentiment analysis to determine if the provided text is positive, neutral, or negative. The API returns a numeric score between 0 and 1. Scores close to 1 indicate positive sentiment, and scores close to 0 indicate negative sentiment.

In this section, you're going to do a few things:

  • Write some Python to call the Text Analytics API to perform sentiment analysis and return a response
  • Create a Flask route to call your Python code
  • Update the HTML with an area for sentiment scores, and a button to perform analysis
  • Write Javascript that allows users to interact with your Flask app from the HTML

Call the Text Analytics API

Let's write a function to call the Text Analytics API. This function will take four arguments: input_text, input_language, output_text, and output_language. This function is called whenever a user presses the run sentiment analysis button in your app. Data provided by the user from the text area and language selector, as well as the detected language and translation output are provided with each request. The response object includes sentiment scores for the source and translation. In the following sections, you're going to write some Javascript to parse the response and use it in your app. For now, let's focus on call the Text Analytics API.

  1. Let's create a file called sentiment.py in the root of your working directory.
  2. Next, add this code to sentiment.py.
    import os, requests, uuid, json
    
    # Don't forget to replace with your Cog Services subscription key!
    subscription_key = 'YOUR_TEXT_ANALYTICS_SUBSCRIPTION_KEY'
    
    # Our Flask route will supply four arguments: input_text, input_language,
    # output_text, output_language.
    # When the run sentiment analysis button is pressed in our Flask app,
    # the Ajax request will grab these values from our web app, and use them
    # in the request. See main.js for Ajax calls.
    
    def get_sentiment(input_text, input_language, output_text, output_language):
        base_url = 'https://westus.api.cognitive.microsoft.com/text/analytics'
        path = '/v2.0/sentiment'
        constructed_url = base_url + path
    
        headers = {
            'Ocp-Apim-Subscription-Key': subscription_key,
            'Content-type': 'application/json',
            'X-ClientTraceId': str(uuid.uuid4())
        }
    
        # You can pass more than one object in body.
        body = {
            'documents': [
                {
                    'language': input_language,
                    'id': '1',
                    'text': input_text
                },
                {
                    'language': output_language,
                    'id': '2',
                    'text': output_text
                }
            ]
        }
        response = requests.post(constructed_url, headers=headers, json=body)
        return response.json()
    
  3. Add your Text Analytics subscription key and save.

Add a route to app.py

Let's create a route in your Flask app that calls sentiment.py. This route will be called each time a user presses the run sentiment analysis button in your app. Like the route for translation, this route is going to accept POST requests since the function expects arguments.

  1. Open app.py and locate the import statement at the top of app.py and update it:

    import translate, sentiment
    

    Now our Flask app can use the method available via sentiment.py.

  2. Copy this code to the end of app.py and save:

    @app.route('/sentiment-analysis', methods=['POST'])
    def sentiment_analysis():
        data = request.get_json()
        input_text = data['inputText']
        input_lang = data['inputLanguage']
        output_text = data['outputText']
        output_lang =  data['outputLanguage']
        response = sentiment.get_sentiment(input_text, input_lang, output_text, output_lang)
        return jsonify(response)
    

Update index.html

Now that you have a function to run sentiment analysis, and a route in your Flask app to call it, the next step is to start writing the HTML for your app. The HTML below does a few things:

  • Adds a button to your app to run sentiment analysis
  • Adds an element that explains sentiment scoring
  • Adds an element to display the sentiment scores
  1. Open index.html and locate these code comments:

    <!-- Start sentiment code-->
    
    <!-- End sentiment code -->
    
  2. Replace the code comments with this HTML block:

    <button type="submit" class="btn btn-primary mb-2" id="sentiment-analysis">Run sentiment analysis</button></br>
    <div id="sentiment" style="display: none">
       <p>Sentiment scores are provided on a 1 point scale. The closer the sentiment score is to 1, indicates positive sentiment. The closer it is to 0, indicates negative sentiment.</p>
       <strong>Sentiment score for input:</strong> <span id="input-sentiment"></span><br />
       <strong>Sentiment score for translation:</strong> <span id="translation-sentiment"></span>
    </div>
    

Update main.js

In the code below, content from the HTML is used to construct a request to your Flask route. Specifically, the contents of the text area and the language selector are assigned to variables, and then passed along in the request to the sentiment-analysis route.

The code then iterates through the response, and updates the HTML with the sentiment scores.

  1. From your IDE, create a file named main.js in the static directory.

  2. Copy this code into static/scripts/main.js:

    //Run sentinment analysis on input and translation.
    $("#sentiment-analysis").on("click", function(e) {
      e.preventDefault();
      var inputText = document.getElementById("text-to-translate").value;
      var inputLanguage = document.getElementById("detected-language-result").innerHTML;
      var outputText = document.getElementById("translation-result").value;
      var outputLanguage = document.getElementById("select-language").value;
    
      var sentimentRequest = { "inputText": inputText, "inputLanguage": inputLanguage, "outputText": outputText,  "outputLanguage": outputLanguage };
    
      if (inputText !== "") {
        $.ajax({
          url: "/sentiment-analysis",
          method: "POST",
          headers: {
              "Content-Type":"application/json"
          },
          dataType: "json",
          data: JSON.stringify(sentimentRequest),
          success: function(data) {
            for (var i = 0; i < data.documents.length; i++) {
              if (typeof data.documents[i] !== "undefined"){
                if (data.documents[i].id === "1") {
                  document.getElementById("input-sentiment").textContent = data.documents[i].score;
                }
                if (data.documents[i].id === "2") {
                  document.getElementById("translation-sentiment").textContent = data.documents[i].score;
                }
              }
            }
            for (var i = 0; i < data.errors.length; i++) {
              if (typeof data.errors[i] !== "undefined"){
                if (data.errors[i].id === "1") {
                  document.getElementById("input-sentiment").textContent = data.errors[i].message;
                }
                if (data.errors[i].id === "2") {
                  document.getElementById("translation-sentiment").textContent = data.errors[i].message;
                }
              }
            }
            if (document.getElementById("input-sentiment").textContent !== '' && document.getElementById("translation-sentiment").textContent !== ""){
              document.getElementById("sentiment").style.display = "block";
            }
          }
        });
      }
    });
    // In the next section, you'll add code for speech synthesis here.
    

Test sentiment analysis

Let's test sentiment analysis in the app.

flask run

Navigate to the provided server address. Type text into the input area, select a language, and press translate. You should get a translation. Next, press the run sentiment analysis button. You should see two scores. If it doesn't work, make sure that you've added your subscription key.

Tip

If the changes you've made aren't showing up, or the app doesn't work the way you expect it to, try clearing your cache or opening a private/incognito window.

Press CTRL + c to kill the app, then head to the next section.

Convert text-to-speech

The Text-to-speech API enables your app to convert text into natural human-like synthesized speech. The service supports standard, neural, and custom voices. Our sample app uses a handful of the available voices, for a full list, see supported languages.

In this section, you're going to do a few things:

  • Write some Python to convert text-to-speech with the Text-to-speech API
  • Create a Flask route to call your Python code
  • Update the HTML with a button to convert text-to-speech, and an element for audio playback
  • Write Javascript that allows users to interact with your Flask app

Call the Text-to-Speech API

Let's write a function to convert text-to-speech. This function will take two arguments: input_text and voice_font. This function is called whenever a user presses the convert text-to-speech button in your app. input_text is the translation output returned by the call to translate text, voice_font is the value from the voice font selector in the HTML.

  1. Let's create a file called synthesize.py in the root of your working directory.

  2. Next, add this code to synthesize.py.

    import os, requests, time
    from xml.etree import ElementTree
    
    class TextToSpeech(object):
        def __init__(self, input_text, voice_font):
            subscription_key = 'YOUR_SPEECH_SERVICES_SUBSCRIPTION_KEY'
            self.subscription_key = subscription_key
            self.input_text = input_text
            self.voice_font = voice_font
            self.timestr = time.strftime('%Y%m%d-%H%M')
            self.access_token = None
    
        # This function performs the token exchange.
        def get_token(self):
            fetch_token_url = 'https://westus.api.cognitive.microsoft.com/sts/v1.0/issueToken'
            headers = {
                'Ocp-Apim-Subscription-Key': self.subscription_key
            }
            response = requests.post(fetch_token_url, headers=headers)
            self.access_token = str(response.text)
    
        # This function calls the TTS endpoint with the access token.
        def save_audio(self):
            base_url = 'https://westus.tts.speech.microsoft.com/'
            path = 'cognitiveservices/v1'
            constructed_url = base_url + path
            headers = {
                'Authorization': 'Bearer ' + self.access_token,
                'Content-Type': 'application/ssml+xml',
                'X-Microsoft-OutputFormat': 'riff-24khz-16bit-mono-pcm',
                'User-Agent': 'YOUR_RESOURCE_NAME',
            }
            # Build the SSML request with ElementTree
            xml_body = ElementTree.Element('speak', version='1.0')
            xml_body.set('{http://www.w3.org/XML/1998/namespace}lang', 'en-us')
            voice = ElementTree.SubElement(xml_body, 'voice')
            voice.set('{http://www.w3.org/XML/1998/namespace}lang', 'en-US')
            voice.set('name', 'Microsoft Server Speech Text to Speech Voice {}'.format(self.voice_font))
            voice.text = self.input_text
            # The body must be encoded as UTF-8 to handle non-ascii characters.
            body = ElementTree.tostring(xml_body, encoding="utf-8")
    
            #Send the request
            response = requests.post(constructed_url, headers=headers, data=body)
    
            # Write the response as a wav file for playback. The file is located
            # in the same directory where this sample is run.
            return response.content
    
  3. Add your Speech Services subscription key and save.

Add a route to app.py

Let's create a route in your Flask app that calls synthesize.py. This route will be called each time a user presses the convert text-to-speech button in your app. Like the routes for translation and sentiment analysis, this route is going to accept POST requests since the function expects two arguments: the text to synthesize, and the voice font for playback.

  1. Open app.py and locate the import statement at the top of app.py and update it:

    import translate, sentiment, synthesize
    

    Now our Flask app can use the method available via synthesize.py.

  2. Copy this code to the end of app.py and save:

    @app.route('/text-to-speech', methods=['POST'])
    def text_to_speech():
        data = request.get_json()
        text_input = data['text']
        voice_font = data['voice']
        tts = synthesize.TextToSpeech(text_input, voice_font)
        tts.get_token()
        audio_response = tts.save_audio()
        return audio_response
    

Update index.html

Now that you have a function to convert text-to-speech, and a route in your Flask app to call it, the next step is to start writing the HTML for your app. The HTML below does a few things:

  • Provides a voice selection drop-down
  • Adds a button to convert text-to-speech
  • Adds an audio element, which is used to play back the synthesized speech
  1. Open index.html and locate these code comments:

    <!-- Start voice font selection code -->
    
    <!-- End voice font selection code -->
    
  2. Replace the code comments with this HTML block:

    <div class="form-group">
      <label for="select-voice"><strong>Select voice font:</strong></label>
      <select class="form-control" id="select-voice">
        <option value="(ar-SA, Naayf)">Arabic | Male | Naayf</option>
        <option value="(ca-ES, HerenaRUS)">Catalan | Female | HerenaRUS</option>
        <option value="(zh-CN, HuihuiRUS)">Chinese (Mainland) | Female | HuihuiRUS</option>
        <option value="(zh-CN, Kangkang, Apollo)">Chinese (Mainland) | Male | Kangkang, Apollo</option>
        <option value="(zh-HK, Tracy, Apollo)">Chinese (Hong Kong)| Female | Tracy, Apollo</option>
        <option value="(zh-HK, Danny, Apollo)">Chinese (Hong Kong) | Male | Danny, Apollo</option>
        <option value="(zh-TW, Yating, Apollo)">Chinese (Taiwan)| Female | Yaiting, Apollo</option>
        <option value="(zh-TW, Zhiwei, Apollo)">Chinese (Taiwan) | Male | Zhiwei, Apollo</option>
        <option value="(hr-HR, Matej)">Croatian | Male | Matej</option>
        <option value="(en-US, Jessa24kRUS)">English (US) | Female | Jessa24kRUS</option>
        <option value="(en-US, Guy24kRUS)">English (US) | Male | Guy24kRUS</option>
        <option value="(en-IE, Sean)">English (IE) | Male | Sean</option>
        <option value="(fr-FR, Julie, Apollo)">French | Female | Julie, Apollo</option>
        <option value="(fr-FR, HortenseRUS)">French | Female | Julie, HortenseRUS</option>
        <option value="(fr-FR, Paul, Apollo)">French | Male | Paul, Apollo</option>
        <option value="(de-DE, Hedda)">German | Female | Hedda</option>
        <option value="(de-DE, HeddaRUS)">German | Female | HeddaRUS</option>
        <option value="(de-DE, Stefan, Apollo)">German | Male | Apollo</option>
        <option value="(el-GR, Stefanos)">Greek | Male | Stefanos</option>
        <option value="(he-IL, Asaf)">Hebrew (Isreal) | Male | Asaf</option>
        <option value="(hi-IN, Kalpana, Apollo)">Hindi | Female | Kalpana, Apollo</option>
        <option value="(hi-IN, Hemant)">Hindi | Male | Hemant</option>
        <option value="(it-IT, LuciaRUS)">Italian | Female | LuciaRUS</option>
        <option value="(it-IT, Cosimo, Apollo)">Italian | Male | Cosimo, Apollo</option>
        <option value="(ja-JP, Ichiro, Apollo)">Japanese | Male | Ichiro</option>
        <option value="(ja-JP, HarukaRUS)">Japanese | Female | HarukaRUS</option>
        <option value="(ko-KR, HeamiRUS)">Korean | Female | Haemi</option>
        <option value="(pt-BR, HeloisaRUS)">Portuguese (Brazil) | Female | HeloisaRUS</option>
        <option value="(pt-BR, Daniel, Apollo)">Portuguese (Brazil) | Male | Daniel, Apollo</option>
        <option value="(pt-PT, HeliaRUS)">Portuguese (Portugal) | Female | HeliaRUS</option>
        <option value="(ru-RU, Irina, Apollo)">Russian | Female | Irina, Apollo</option>
        <option value="(ru-RU, Pavel, Apollo)">Russian | Male | Pavel, Apollo</option>
        <option value="(ru-RU, EkaterinaRUS)">Russian | Female | EkaterinaRUS</option>
        <option value="(es-ES, Laura, Apollo)">Spanish | Female | Laura, Apollo</option>
        <option value="(es-ES, HelenaRUS)">Spanish | Female | HelenaRUS</option>
        <option value="(es-ES, Pablo, Apollo)">Spanish | Male | Pablo, Apollo</option>
        <option value="(th-TH, Pattara)">Thai | Male | Pattara</option>
        <option value="(tr-TR, SedaRUS)">Turkish | Female | SedaRUS</option>
        <option value="(vi-VN, An)">Vietnamese | Male | An</option>
      </select>
    </div>
    
  3. Next, locate these code comments:

    <!-- Add Speech Synthesis button and audio element -->
    
    <!-- End Speech Synthesis button -->
    
  4. Replace the code comments with this HTML block:

<button type="submit" class="btn btn-primary mb-2" id="text-to-speech">Convert text-to-speech</button>
<div id="audio-playback">
  <audio id="audio" controls>
    <source id="audio-source" type="audio/mpeg" />
  </audio>
</div>
  1. Make sure to save your work.

Update main.js

In the code below, content from the HTML is used to construct a request to your Flask route. Specifically, the translation and the voice font are assigned to variables, and then passed along in the request to the text-to-speech route.

The code then iterates through the response, and updates the HTML with the sentiment scores.

  1. From your IDE, create a file named main.js in the static directory.
  2. Copy this code into static/scripts/main.js:
    // Convert text-to-speech
    $("#text-to-speech").on("click", function(e) {
      e.preventDefault();
      var ttsInput = document.getElementById("translation-result").value;
      var ttsVoice = document.getElementById("select-voice").value;
      var ttsRequest = { 'text': ttsInput, 'voice': ttsVoice }
    
      var xhr = new XMLHttpRequest();
      xhr.open("post", "/text-to-speech", true);
      xhr.setRequestHeader("Content-Type", "application/json");
      xhr.responseType = "blob";
      xhr.onload = function(evt){
        if (xhr.status === 200) {
          audioBlob = new Blob([xhr.response], {type: "audio/mpeg"});
          audioURL = URL.createObjectURL(audioBlob);
          if (audioURL.length > 5){
            var audio = document.getElementById("audio");
            var source = document.getElementById("audio-source");
            source.src = audioURL;
            audio.load();
            audio.play();
          }else{
            console.log("An error occurred getting and playing the audio.")
          }
        }
      }
      xhr.send(JSON.stringify(ttsRequest));
    });
    // Code for automatic language selection goes here.
    
  3. You're almost done. The last thing you're going to do is add some code to main.js to automatically select a voice font based on the language selected for translation. Add this code block to main.js:
    // Automatic voice font selection based on translation output.
    $('select[id="select-language"]').change(function(e) {
      if ($(this).val() == "ar"){
        document.getElementById("select-voice").value = "(ar-SA, Naayf)";
      }
      if ($(this).val() == "ca"){
        document.getElementById("select-voice").value = "(ca-ES, HerenaRUS)";
      }
      if ($(this).val() == "zh-Hans"){
        document.getElementById("select-voice").value = "(zh-HK, Tracy, Apollo)";
      }
      if ($(this).val() == "zh-Hant"){
        document.getElementById("select-voice").value = "(zh-HK, Tracy, Apollo)";
      }
      if ($(this).val() == "hr"){
        document.getElementById("select-voice").value = "(hr-HR, Matej)";
      }
      if ($(this).val() == "en"){
        document.getElementById("select-voice").value = "(en-US, Jessa24kRUS)";
      }
      if ($(this).val() == "fr"){
        document.getElementById("select-voice").value = "(fr-FR, HortenseRUS)";
      }
      if ($(this).val() == "de"){
        document.getElementById("select-voice").value = "(de-DE, HeddaRUS)";
      }
      if ($(this).val() == "el"){
        document.getElementById("select-voice").value = "(el-GR, Stefanos)";
      }
      if ($(this).val() == "he"){
        document.getElementById("select-voice").value = "(he-IL, Asaf)";
      }
      if ($(this).val() == "hi"){
        document.getElementById("select-voice").value = "(hi-IN, Kalpana, Apollo)";
      }
      if ($(this).val() == "it"){
        document.getElementById("select-voice").value = "(it-IT, LuciaRUS)";
      }
      if ($(this).val() == "ja"){
        document.getElementById("select-voice").value = "(ja-JP, HarukaRUS)";
      }
      if ($(this).val() == "ko"){
        document.getElementById("select-voice").value = "(ko-KR, HeamiRUS)";
      }
      if ($(this).val() == "pt"){
        document.getElementById("select-voice").value = "(pt-BR, HeloisaRUS)";
      }
      if ($(this).val() == "ru"){
        document.getElementById("select-voice").value = "(ru-RU, EkaterinaRUS)";
      }
      if ($(this).val() == "es"){
        document.getElementById("select-voice").value = "(es-ES, HelenaRUS)";
      }
      if ($(this).val() == "th"){
        document.getElementById("select-voice").value = "(th-TH, Pattara)";
      }
      if ($(this).val() == "tr"){
        document.getElementById("select-voice").value = "(tr-TR, SedaRUS)";
      }
      if ($(this).val() == "vi"){
        document.getElementById("select-voice").value = "(vi-VN, An)";
      }
    });
    

Test your app

Let's test speech synthesis in the app.

flask run

Navigate to the provided server address. Type text into the input area, select a language, and press translate. You should get a translation. Next, select a voice, then press the convert text-to-speech button. the translation should be played back as synthesized speech. If it doesn't work, make sure that you've added your subscription key.

Tip

If the changes you've made aren't showing up, or the app doesn't work the way you expect it to, try clearing your cache or opening a private/incognito window.

That's it, you have a working app that performs translations, analyzes sentiment, and synthesized speech. Press CTRL + c to kill the app. Be sure to check out the other Azure Cognitive Services.

Get the source code

The source code for this project is available on GitHub.

Next steps