Quickstart: question answering

Get started with the custom question answering client library. Follow these steps to install the package and try out the example code for basic tasks.

You can create a question answering project from your own content, such as FAQs or product manuals. This article includes an example of creating a question answering project from a product manual, to answer questions.

Prerequisites

  • If you don't have an Azure subscription, create a free account before you begin.
  • A language resource with the custom question answering feature enabled. Remember your Azure Active Directory ID, Subscription, language resource name you selected when you created the resource.

Create your first question answering project

  1. Sign in to the Language Studio with your Azure credentials.

  2. Scroll down to the Answer questions section and select Open custom question answering.

    Open custom question answering

  3. If your resource is not yet connected to Azure Search select Connect to Azure Search. This will open a new browser tab to Features pane of your resource in the Azure portal.

    Connect to Azure Search

  4. Select Enable custom question answering, choose the Azure Search resource to link to, and then select Apply.

    Enable custom question answering

  5. Return to the Language Studio tab. You may need to refresh this page for it to register the change to your resource. Select Create new project.

  6. Choose the option I want to set the language for all projects created in this resource > select English > Select Next.

  7. Enter a project name of Sample-project, a description of My first question answering project, and leave the default answer with a setting of No answer found.

  8. Review your choices and select Create project

  9. From the Manage sources page select Add source > URLS.

  10. Select Add url enter the following values and then select Add all:

    URL Name URL Value
    Surface Book User Guide https://download.microsoft.com/download/7/B/1/7B10C82E-F520-4080-8516-5CF0D803EEE0/surface-book-user-guide-EN.pdf

    The extraction process takes a few moments to read the document and identify questions and answers.

    After successfully adding the source, you can then edit the source contents to add more custom question answer sets.

Test your project

  1. Select the link to your source, this will open the edit knowledge base page.

  2. Select Test from the menu bar > Enter the question How do I setup my surface book?. An answer will be generated based on the question answer pairs that were automatically identified and extracted from your source URL:

    Test question chat interface

    If you check the box for include short answer response you will also see a precise answer, if available, along with the answer passage in the test pane when you ask a question.

  3. Select Inspect to examine the response in more detail. The test window is used to test your changes to your project before deploying your project.

    See the confidence interval

    From the Inspect interface, you can see the level of confidence that this response will answer the question and directly edit a given question and answer response pair.

Deploy your project

  1. Select the Deploy knowledge base icon to enter the deploy knowledge base menu.

    Deploy knowledge base

    When you deploy a project, the contents of your project move from the test index to a prod index in Azure Search.

  2. Select Deploy > and then when prompted select Deploy again.

    Successful deployment

    Your project is now successfully deployed. You can use the endpoint to answer questions in your own custom application to answer or in a bot.

Prerequisites

  • The current version of cURL. Several command-line switches are used in the quickstarts, which are noted in the cURL documentation.
  • Azure subscription - Create one for free
  • Question answering, requires a Language resource with the custom question answering feature enabled to generate an API key and endpoint.
    • After your Language resource deploys, select Go to resource. You will need the key and endpoint from the resource you create to connect to the API. Paste your key and endpoint into the code below later in the quickstart.
  • To create a Language resource with Azure CLI provide the following additional properties during resource creation configure Custom Question Answering with your Language resource --api-properties qnaAzureSearchEndpointId=/subscriptions/<azure-subscription-id>/resourceGroups/<resource-group-name>/providers/Microsoft.Search/searchServices/<azure-search-service-name> qnaAzureSearchEndpointKey=<azure-search-service-auth-key>
  • An existing knowledge base to query. If you have not setup a knowledge base, you can follow the instructions in the Language Studio quickstart. Or add a knowledge base that uses this Surface User Guide URL as a data source.

Query a knowledge base

Generate an answer from a knowledge base

To query a question answering project/knowledge base with the REST APIs and cURL, you need the following information:

Variable name Value
Endpoint This value can be found in the Keys & Endpoint section when examining your resource from the Azure portal. Alternatively you can find the value in Language Studio > question answering > Deploy knowledge base > Get prediction URL. An example endpoint is: https://southcentralus.api.cognitive.microsoft.com/
API-Key This value can be found in the Keys & Endpoint section when examining your resource from the Azure portal. You can use either Key1 or Key2. Always having two valid keys always for secure key rotation with zero downtime. Alternatively you can find the value in Language Studio > question answering > Deploy knowledge base > Get prediction URL. The key value is part of the sample request.
Project The name of your question answering project.
Deployment There are two possible values: test, and production. production is dependent on you having deployed your knowledge base from Language Studio > question answering > Deploy knowledge base.

The cURL command is executed from a BASH shell. Edit this command with your own resource name, resource key, and JSON values and size of JSON.

curl -X POST -H "Ocp-Apim-Subscription-Key: {YOUR_API_KEY}" -H "Content-Type: application/json" -d '{
  "question": "How much battery life do I have left?"
  }'  'https://{YOUR_ENDPOINT}.api.cognitive.microsoft.com/language/:query-knowledgebases?projectName={YOUR_PROJECT_NAME}&api-version=2021-10-01&deploymentName={DEPLOYMENT_NAME}'

When you run the code above, if you are using the data source from the prerequisites you will get an answer that looks as follows:

{
"answers": [
    {
      "questions": [
        "Check battery level"
      ],
      "answer": "If you want to see how much battery you have left, go to **Start  **> **Settings  **> **Devices  **> **Bluetooth & other devices  **, then find your pen. The current battery level will appear under the battery icon.",
      "confidenceScore": 0.9185,
      "id": 101,
      "source": "https://support.microsoft.com/en-us/surface/how-to-use-your-surface-pen-8a403519-cd1f-15b2-c9df-faa5aa924e98",
      "metadata": {},
      "dialog": {
        "isContextOnly": false,
        "prompts": []
      }
    }
  ]
}

The confidenceScore returns a value between 0 and 1. You can think of this like a percentage and multiply by 100 so a confidence score of 0.9185 means question answering is 91.85% confident this is the correct answer to the question based on the knowledge base.

If you want to exclude answers where the confidence score falls below a certain threshold, you can add the confidenceScoreThreshold parameter.

curl -X POST -H "Ocp-Apim-Subscription-Key: {YOUR_API_KEY}" -H "Content-Type: application/json" -d '{
  "question": "How much battery life do I have left?",
  "confidenceScoreThreshold": "0.95",
  }'  'https://{YOUR_ENDPOINT}.api.cognitive.microsoft.com//language/:query-knowledgebases?projectName=Sample-project&api-version=2021-10-01&deploymentName={DEPLOYMENT_NAME}'

Since we know from our previous execution of the code that our confidence score is: .9185 setting the threshold to .95 will result in the default answer being returned.

{
  "answers": [
    {
      "questions": [],
      "answer": "No good match found in KB",
      "confidenceScore": 0.0,
      "id": -1,
      "metadata": {}
    }
  ]
}

Query text without a knowledge base

You can also use question answering without a knowledge base with the prebuilt question answering REST API, which is called via query-text. In this case, you provide question answering with both a question and the associated text records you would like to search for an answer at the time the request is sent.

For this example, you only need to modify the variables for API KEY and ENDPOINT.

curl -X POST -H "Ocp-Apim-Subscription-Key: {YOUR_API_KEY}" -H "Content-Type: application/json" -d '{
"question":"How long does it takes to charge a surface?",
"records":[
{"id":"doc1","text":"Power and charging.It takes two to four hours to charge the Surface Pro 4 battery fully from an empty state. It can take longer if you\u0027re using your Surface for power-intensive activities like gaming or video streaming while you\u0027re charging it"},
{"id":"doc2","text":"You can use the USB port on your Surface Pro 4 power supply to charge other devices, like a phone, while your Surface charges. The USB port on the power supply is only for charging, not for data transfer. If you want to use a USB device, plug it into the USB port on your Surface."}],
"language":"en",
"stringIndexType":"Utf16CodeUnit"
}'  'https://{YOUR_ENDPOINT}.api.cognitive.microsoft.com/language/:query-text?&api-version=2021-10-01'

This example will return a result of:

{  
"answers": [
    {
      "answer": "Power and charging.It takes two to four hours to charge the Surface Pro 4 battery fully from an empty state. It can take longer if you're using your Surface for power-intensive activities like gaming or video streaming while you're charging it",
      "confidenceScore": 0.9118788838386536,
      "id": "doc1",
      "answerSpan": {
        "text": "two to four hours",
        "confidenceScore": 0.9850527,
        "offset": 27,
        "length": 18
      },
      "offset": 0,
      "length": 243
    },
    {
      "answer": "It can take longer if you're using your Surface for power-intensive activities like gaming or video streaming while you're charging it",
      "confidenceScore": 0.052793052047491074,
      "id": "doc1",
      "answerSpan": {
        "text": "longer",
        "confidenceScore": 0.6694634,
        "offset": 11,
        "length": 7
      },
      "offset": 109,
      "length": 134
    },
    {
      "answer": "You can use the USB port on your Surface Pro 4 power supply to charge other devices, like a phone, while your Surface charges. The USB port on the power supply is only for charging, not for data transfer. If you want to use a USB device, plug it into the USB port on your Surface.",
      "confidenceScore": 0.017600709572434425,
      "id": "doc2",
      "answerSpan": {
        "text": "USB port on your Surface Pro 4 power supply to charge other devices, like a phone, while your Surface charges. The USB port on the power supply is only for charging",
        "confidenceScore": 0.1544854,
        "offset": 15,
        "length": 165
      },
      "offset": 0,
      "length": 280
    }
  ]
}

Use this quickstart for the question answering client library for .NET to:

  • Get an answer from a knowledge base.
  • Get an answer from a body of text that you send along with your question.
  • Get the confidence score for the answer to your question.

API reference documentation|Source code | Package (NuGet) | Samples |

Prerequisites

  • Azure subscription - Create one for free
  • The Visual Studio IDE or current version of .NET Core.
  • Question answering, requires a Language resource with the custom question answering feature enabled to generate an API key and endpoint.
    • After your Language resource deploys, select Go to resource. You will need the key and endpoint from the resource you create to connect to the API. Paste your key and endpoint into the code below later in the quickstart.
  • To create a Language resource with Azure CLI provide the following additional properties during resource creation configure Custom Question Answering with your Language resource --api-properties qnaAzureSearchEndpointId=/subscriptions/<azure-subscription-id>/resourceGroups/<resource-group-name>/providers/Microsoft.Search/searchServices/<azure-search-service-name> qnaAzureSearchEndpointKey=<azure-search-service-auth-key>
  • An existing knowledge base to query. If you have not setup a knowledge base, you can follow the instructions in the Language Studio quickstart. Or add a knowledge base that uses this Surface User Guide URL as a data source.

Setting up

CLI

In a console window (such as cmd, PowerShell, or Bash), use the dotnet new command to create a new console app with the name question-answering-quickstart. This command creates a simple "Hello World" C# project with a single source file: program.cs.

dotnet new console -n question-answering-quickstart

Change your directory to the newly created app folder. You can build the application with:

dotnet build

The build output should contain no warnings or errors.

...
Build succeeded.
 0 Warning(s)
 0 Error(s)
...

Within the application directory, install the custom question answering client library for .NET with the following command:

dotnet add package Azure.AI.Language.QuestionAnswering

Query a knowledge base

Generate an answer from a knowledge base

The example below will allow you to query a knowledge base using GetAnswers to get an answer to your question.

You will need to update the code below and provide your own values for the following variables.

Variable name Value
endpoint This value can be found in the Keys & Endpoint section when examining your resource from the Azure portal. Alternatively you can find the value in Language Studio > question answering > Deploy knowledge base > Get prediction URL. An example endpoint is: https://southcentralus.api.cognitive.microsoft.com/
credential This value can be found in the Keys & Endpoint section when examining your resource from the Azure portal. You can use either Key1 or Key2. Always having two valid keys always for secure key rotation with zero downtime. Alternatively you can find the value in Language Studio > question answering > Deploy knowledge base > Get prediction URL. The key value is part of the sample request.
projectName The name of your question answering project.
deploymentName There are two possible values: test, and production. production is dependent on you having deployed your knowledge base from Language Studio > question answering > Deploy knowledge base.

From the project directory, open the program.cs file and replace with the following code:

using Azure;
using Azure.AI.Language.QuestionAnswering;
using System;

namespace question_answering
{
    class Program
    {
        static void Main(string[] args)
        {

            Uri endpoint = new Uri("https://{YOUR-ENDPOINT}.api.cognitive.microsoft.com/");
            AzureKeyCredential credential = new AzureKeyCredential("{YOUR-LANGUAGE-RESOURCE-KEY}");
            string projectName = "{YOUR-PROJECT-NAME}";
            string deploymentName = "production";

            string question = "How long should my Surface battery last?";

            QuestionAnsweringClient client = new QuestionAnsweringClient(endpoint, credential);
            QuestionAnsweringProject project = new QuestionAnsweringProject(projectName, deploymentName);

            Response<AnswersResult> response = client.GetAnswers(question, project);

            foreach (KnowledgeBaseAnswer answer in response.Value.Answers)
            {
                Console.WriteLine($"Q:{question}");
                Console.WriteLine($"A:{answer.Answer}");
            }
        }
    }
}

While we are hard coding the variables for our example. For production, consider using a secure way of storing and accessing your credentials. For example, Azure key vault provides secure key storage.

After updating Program.cs with the code above and substituting in the correct variable values. Run the application with the dotnet run command from your application directory.

dotnet run

The response will look as follows:

Q: How much battery life do I have left?
A: If you want to see how much battery you have left, go to **Start  **> **Settings  **> **Devices  **> **Bluetooth & other devices  **, then find your pen. The current battery level will appear under the battery icon.

For information on how confident question answering is that this is the correct response add an additional print statement underneath the existing print statements:

Console.WriteLine($"Q:{question}");
Console.WriteLine($"A:{answer.Answer}");
Console.WriteLine($"({answer.Confidence})"); // add this line

If you execute dotnet run again, you will now receive a result with a confidence score:

Q:How much battery life do I have left?
A:If you want to see how much battery you have left, go to **Start  **> **Settings  **> **Devices  **> **Bluetooth & other devices  **, then find your pen. The current battery level will appear under the battery icon.
(0.9185)

The confidence score returns a value between 0 and 1. You can think of this like a percentage and multiply by 100 so a confidence score of 0.9185 means question answering is 91.85% confident this is the correct answer to the question based on the knowledge base.

If you want to exclude answers where the confidence score falls below a certain threshold, you use AnswerOptions to add the ConfidenceScoreThreshold property.

QuestionAnsweringClient client = new QuestionAnsweringClient(endpoint, credential);
QuestionAnsweringProject project = new QuestionAnsweringProject(projectName, deploymentName);
AnswersOptions options = new AnswersOptions(); //Add this line
options.ConfidenceThreshold = 0.95; //Add this line

Response<AnswersResult> response = client.GetAnswers(question, project, options); //Add the additional options parameter

Since we know from our previous execution of the code that our confidence score is: .9185 setting the threshold to .95 will result in the default answer being returned.

Q:How much battery life do I have left?
A:No good match found in KB
(0)

Query text without a knowledge base

You can also use question answering without a knowledge base with GetAnswersFromText. In this case, you provide question answering with both a question and the associated text records you would like to search for an answer at the time the request is sent.

For this example, you only need to modify the variables for endpoint and credential.

using Azure;
using Azure.AI.Language.QuestionAnswering;
using System;
using System.Collections.Generic;


namespace questionansweringcsharp
{
    class Program
    {
        static void Main(string[] args)
        {

            Uri endpoint = new Uri("https://{YOUR-ENDPOINT}.api.cognitive.microsoft.com/");
            AzureKeyCredential credential = new AzureKeyCredential("YOUR-LANGUAGE-RESOURCE-KEY");
            QuestionAnsweringClient client = new QuestionAnsweringClient(endpoint, credential);

            IEnumerable<TextDocument> records = new[]
            {
                new TextDocument("doc1", "Power and charging.It takes two to four hours to charge the Surface Pro 4 battery fully from an empty state. " +
                         "It can take longer if you're using your Surface for power-intensive activities like gaming or video streaming while you're charging it"),
                new TextDocument("doc2", "You can use the USB port on your Surface Pro 4 power supply to charge other devices, like a phone, while your Surface charges. " +
                         "The USB port on the power supply is only for charging, not for data transfer. If you want to use a USB device, plug it into the USB port on your Surface."),
            };

            AnswersFromTextOptions options = new AnswersFromTextOptions("How long does it takes to charge a surface?", records);
            Response<AnswersFromTextResult> response = client.GetAnswersFromText(options);

           foreach (TextAnswer answer in response.Value.Answers)
            {
                if (answer.Confidence > .9)
                {
                    string BestAnswer = response.Value.Answers[0].Answer;

                    Console.WriteLine($"Q:{options.Question}");
                    Console.WriteLine($"A:{BestAnswer}");
                    Console.WriteLine($"Confidence Score: ({response.Value.Answers[0].Confidence:P2})"); //:P2 converts the result to a percentage with 2 decimals of accuracy. 
                    break;
                }
                else
                {
                    Console.WriteLine($"Q:{options.Question}");
                    Console.WriteLine("No answers met the requested confidence score.");
                    break;
                }
            }

        }
    }
}

To run the code above, replace the Program.cs with the contents of the script block above and modify the endpoint and credential variables to correspond to the language resource you created as part of the prerequisites.

In this case, we iterate through all responses and only return the response with the highest confidence score that is greater than 0.9. To understand more about the options available with GetAnswersFromText.

Use this quickstart for the question answering client library for Python to:

  • Get an answer from a knowledge base.
  • Get an answer from a body of text that you send along with your question.
  • Get the confidence score for the answer to your question.

API reference documentation | Source code | Package (PyPI) | Python Samples |

Prerequisites

  • Azure subscription - Create one for free
  • Python 3.x
  • Question answering, requires a Language resource with the custom question answering feature enabled to generate an API key and endpoint.
    • After your Language resource deploys, select Go to resource. You will need the key and endpoint from the resource you create to connect to the API. Paste your key and endpoint into the code below later in the quickstart.
  • To create a Language resource with Azure CLI provide the following additional properties during resource creation configure Custom Question Answering with your Language resource --api-properties qnaAzureSearchEndpointId=/subscriptions/<azure-subscription-id>/resourceGroups/<resource-group-name>/providers/Microsoft.Search/searchServices/<azure-search-service-name> qnaAzureSearchEndpointKey=<azure-search-service-auth-key>
  • An existing knowledge base to query. If you have not setup a knowledge base, you can follow the instructions in the Language Studio quickstart. Or add a knowledge base that uses this Surface User Guide URL as a data source.

Setting up

Install the client library

After installing Python, you can install the client library with:

pip install azure-ai-language-questionanswering

Query a knowledge base

Generate an answer from a knowledge base

The example below will allow you to query a knowledge base using get_answers to get an answer to your question. You can copy this code into a dedicated .py file or into a cell in Jupyter Notebook/Lab.

You will need to update the code below and provide your own values for the following variables.

Variable name Value
endpoint This value can be found in the Keys & Endpoint section when examining your resource from the Azure portal. Alternatively you can find the value in Language Studio > question answering > Deploy knowledge base > Get prediction URL. An example endpoint is: https://southcentralus.api.cognitive.microsoft.com/
credential This value can be found in the Keys & Endpoint section when examining your resource from the Azure portal. You can use either Key1 or Key2. Always having two valid keys always for secure key rotation with zero downtime. Alternatively you can find the value in Language Studio > question answering > Deploy knowledge base > Get prediction URL. The key value is part of the sample request.
knowledge_base_project The name of your question answering project.
deployment There are two possible values: test, and production. production is dependent on you having deployed your knowledge base from Language Studio > question answering > Deploy knowledge base.
from azure.core.credentials import AzureKeyCredential
from azure.ai.language.questionanswering import QuestionAnsweringClient

endpoint = "https://{YOUR-ENDPOINT}.api.cognitive.microsoft.com/"
credential = AzureKeyCredential("{YOUR-LANGUAGE-RESOURCE-KEY}")
knowledge_base_project = "{YOUR-PROJECT-NAME}"
deployment = "production"

def main():
    client = QuestionAnsweringClient(endpoint, credential)
    with client:
        question="How much battery life do I have left?"
        output = client.get_answers(
            question = question,
            project_name=knowledge_base_project,
            deployment_name=deployment
        )
    print("Q: {}".format(question))
    print("A: {}".format(output.answers[0].answer))

if __name__ == '__main__':
    main()

While we are hard coding the variables for our example. For production, consider using a secure way of storing and accessing your credentials. For example, Azure key vault provides secure key storage.

When you run the code above, if you are using the data source from the prerequisites you will get an answer that looks as follows:

Q: How much battery life do I have left?
A: If you want to see how much battery you have left, go to **Start  **> **Settings  **> **Devices  **> **Bluetooth & other devices  **, then find your pen. The current battery level will appear under the battery icon.

For information on how confident question answering is that this is the correct response add an additional print statement underneath the existing print statements:

print("Q: {}".format(question))
print("A: {}".format(output.answers[0].answer))
print("Confidence Score: {}".format(output.answers[0].confidence_score)) # add this line 

You will now receive a result with a confidence score:

Q: How much battery life do I have left?
A: If you want to see how much battery you have left, go to **Start  **> **Settings  **> **Devices  **> **Bluetooth & other devices  **, then find your pen. The current battery level will appear under the battery icon.
Confidence Score: 0.9185

The confidence score returns a value between 0 and 1. You can think of this like a percentage and multiply by 100 so a confidence score of 0.9185 means question answering is 91.85% confident this is the correct answer to the question based on the knowledge base.

If you want to exclude answers where the confidence score falls below a certain threshold, you can modify the AnswerOptions to add the confidence_threshold parameter.

        output = client.get_answers(
            confidence_threshold = 0.95, #add this line
            question = question,
            project_name=knowledge_base_project,
            deployment_name=deployment
        )

Since we know from our previous execution of the code that our confidence score is: .9185 setting the threshold to .95 will result in the default answer being returned.

Q: How much battery life do I have left?
A: No good match found in KB
Confidence Score: 0.0

Query text without a knowledge base

You can also use question answering without a knowledge base with get_answers_from_text. In this case, you provide question answering with both a question and the associated text records you would like to search for an answer at the time the request is sent.

For this example, you only need to modify the variables for endpoint and credential.

import os
from azure.core.credentials import AzureKeyCredential
from azure.ai.language.questionanswering import QuestionAnsweringClient
from azure.ai.language.questionanswering import models as qna

endpoint = "https://{YOUR-ENDPOINT}.api.cognitive.microsoft.com/"
credential = AzureKeyCredential("YOUR-LANGUAGE-RESOURCE-KEY")

def main():
    client = QuestionAnsweringClient(endpoint, credential)
    with client:
        question="How long does it takes to charge a surface?"
        input = qna.AnswersFromTextOptions(
            question=question,
            text_documents=[
                "Power and charging. It takes two to four hours to charge the Surface Pro 4 battery fully from an empty state. " +
                "It can take longer if you're using your Surface for power-intensive activities like gaming or video streaming while you're charging it.",
                "You can use the USB port on your Surface Pro 4 power supply to charge other devices, like a phone, while your Surface charges. " +
                "The USB port on the power supply is only for charging, not for data transfer. If you want to use a USB device, plug it into the USB port on your Surface.",
            ]
        )


        output = client.get_answers_from_text(input)

    best_answer = [a for a in output.answers if a.confidence > 0.9][0]
    print(u"Q: {}".format(input.question))
    print(u"A: {}".format(best_answer.answer))
    print("Confidence Score: {}".format(output.answers[0].confidence))

if __name__ == '__main__':
    main()

You can copy this code into a dedicated .py file or into a new cell in Jupyter Notebook/Lab. This example will return a result of:

Q: How long does it takes to charge surface?
A: Power and charging. It takes two to four hours to charge the Surface Pro 4 battery fully from an empty state. It can take longer if you're using your Surface for power-intensive activities like gaming or video streaming while you're charging it.
Confidence Score: 0.9254655838012695

In this case, we iterate through all responses and only return the response with the highest confidence score that is greater than 0.9. To understand more about the options available with get_answers_from_text, review the AnswersFromTextOptions parameters.

Clean up resources

If you want to clean up and remove a Cognitive Services subscription, you can delete the resource or resource group. Deleting the resource group also deletes any other resources associated with it.

Explore the REST API

To learn about automating your question answering pipeline consult the REST API documentation. Currently authoring functionality is only available via REST API:

Next steps