Add utterances to a LUIS app using Ruby

Programmatically add utterances to your Language Understanding (LUIS) app and train it using the command line. For more information, refer to the technical documentation for the add utterance, train, and training status APIs.

Prerequisites

  • Ruby
  • Your LUIS authoring key. You can find this key under Account Settings in the LUIS website.
  • Your existing LUIS application ID. The application ID is shown in the application dashboard. The LUIS application with the intents and entities used in the utterances.json file must exist prior to running the code in add-utterances.rb. The code in this article does not create the intents and entities. It only adds the utterances for existing intents and entities.
  • The version ID within the application that receives the utterances. The default ID is "0.1"
  • Create a new file named add-utterances.rb project in VSCode.

Note

The complete add-utterances.cs file and an example utterances.json file are available from the LUIS-Samples Github repository.

Write the Ruby code

Add the dependencies to the file.

require 'json'
require 'net/https'
require 'uri'

# **********************************************
# *** Update or verify the following values. ***
# **********************************************

# NOTE: Replace this with LUIS application ID.
appID = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"

# NOTE: Replace this example LUIS application version number with the version number of your LUIS application.
appVersion = "0.1"

# NOTE: Replace this with LUIS AUTHORING KEY.
$key = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

host = "https://westus.api.cognitive.microsoft.com"
path = "/luis/api/v2.0/apps/" + appID + "/versions/" + appVersion + "/"
$uri = host + path;

usage = "Usage:
add-utterances <input file>
add-utterances -train <input file>
add-utterances -status

The contents of <input file> must be in the format described at: https://aka.ms/add-utterance-json-format
"

Add the GET request used for training status.

def SendGet(uri)
	uri = URI(uri)
	request = Net::HTTP::Get.new(uri)
	request['Ocp-Apim-Subscription-Key'] = $key

	response = Net::HTTP.start(uri.host, uri.port, :use_ssl => uri.scheme == 'https') do |http|
		http.request (request)
	end

	return response.body
end

Add the POST request used to create utterances or start training.

def SendPost(uri, body)
	uri = URI(uri)
	request = Net::HTTP::Post.new(uri)
	request['Content-type'] = 'text/json'
	request['Ocp-Apim-Subscription-Key'] = $key
	request.body = body

	response = Net::HTTP.start(uri.host, uri.port, :use_ssl => uri.scheme == 'https') do |http|
		http.request (request)
	end

	return response.body
end

Add the AddUtterances function.

def AddUtterances(input_file)
	content = File.read(input_file)
	puts "Added utterances."
	result = SendPost($uri+"examples",content)
	puts JSON.pretty_generate(JSON(result))
end

Add the Train function.

def Train(input_file)
	content = File.read(input_file)
	puts "Sent training request."
	result = SendPost($uri+"train",content)
    puts JSON.pretty_generate(JSON(result))
	Status()
end

Add the Status function.

def Status()
	puts "Requested training status."
	result = SendGet($uri+"train")
    puts JSON.pretty_generate(JSON(result))
end

To manage arguments, add the main code.

args = ARGV

if (args.length < 1) then
	puts(usage)
else
	if (0 == args[0].casecmp("-train"))
		if (args.length > 1) then
			Train(args[1])
		else
			puts(usage)
		end
	elsif (0 == args[0].casecmp("-status")) then
		Status()
	else
		AddUtterances(args[0])
	end
end

Specify utterances to add

Create and edit the file utterances.json to specify the array of utterances you want to add to the LUIS app. The intent and entities must already be in the LUIS app.

Note

The LUIS application with the intents and entities used in the utterances.json file must exist prior to running the code in add-utterances.rb. The code in this article does not create the intents and entities. It only adds the utterances for existing intents and entities.

The text field contains the text of the utterance. The intentName field must correspond to the name of an intent in the LUIS app. The entityLabels field is required. If you don't want to label any entities, provide an empty list as shown in the following example:

If the entityLabels list is not empty, the startCharIndex and endCharIndex need to mark the entity referred to in the entityName field. Both indexes are zero-based counts meaning 6 in the top example refers to the "S" of Seattle and not the space before the capital S.

[
    {
        "text": "go to Seattle",
        "intentName": "BookFlight",
        "entityLabels": [
            {
                "entityName": "Location::LocationTo",
                "startCharIndex": 6,
                "endCharIndex": 12
            }
        ]
    },
    {
        "text": "book a flight",
        "intentName": "BookFlight",
        "entityLabels": []
    }
]

Add an utterance from the command line

Run the application from a command line with Ruby.

Calling add-utterances.rb with only the utterance.json as an argument adds but does not train LUIS on the new utterances.

> ruby add-utterances.rb ./utterances.json

This result displays the results from calling the add utterances API. The response field is in this format for utterances that was added. The hasError is false, indicating the utterance was added.

    "response": [
        {
            "value": {
                "UtteranceText": "go to seattle",
                "ExampleId": -5123383
            },
            "hasError": false
        },
        {
            "value": {
                "UtteranceText": "book a flight",
                "ExampleId": -169157
            },
            "hasError": false
        }
    ]

Add an utterance and train from the command line

Call add-utterance with the -train argument to send a request to train.

> ruby add-utterances.rb ./utterances.json -train

Note

Duplicate utterances aren't added again, but don't cause an error. The response contains the ID of the original utterance.

The following shows the result of a successful request to train:

{
    "request": null,
    "response": {
        "statusId": 9,
        "status": "Queued"
    }
}

After the request to train is queued, it can take a moment to complete training.

Get training status from the command line

Call the sample with the -status argument to check the training status.

> ruby add-utterances.rb ./utterances.json -status
Requested training status.
[
   {
      "modelId": "eb2f117c-e10a-463e-90ea-1a0176660acc",
      "details": {
         "statusId": 0,
         "status": "Success",
         "exampleCount": 33,
         "trainingDateTime": "2017-11-20T18:09:11Z"
      }
   },
   {
      "modelId": "c1bdfbfc-e110-402e-b0cc-2af4112289fb",
      "details": {
         "statusId": 0,
         "status": "Success",
         "exampleCount": 33,
         "trainingDateTime": "2017-11-20T18:09:11Z"
      }
   },
   {
      "modelId": "863023ec-2c96-4d68-9c44-34c1cbde8bc9",
      "details": {
         "statusId": 0,
         "status": "Success",
         "exampleCount": 33,
         "trainingDateTime": "2017-11-20T18:09:11Z"
      }
   },
   {
      "modelId": "82702162-73ba-4ae9-a6f6-517b5244c555",
      "details": {
         "statusId": 0,
         "status": "Success",
         "exampleCount": 33,
         "trainingDateTime": "2017-11-20T18:09:11Z"
      }
   },
   {
      "modelId": "37121f4c-4853-467f-a9f3-6dfc8cad2763",
      "details": {
         "statusId": 0,
         "status": "Success",
         "exampleCount": 33,
         "trainingDateTime": "2017-11-20T18:09:11Z"
      }
   },
   {
      "modelId": "de421482-753e-42f5-a765-ad0a60f50d69",
      "details": {
         "statusId": 0,
         "status": "Success",
         "exampleCount": 33,
         "trainingDateTime": "2017-11-20T18:09:11Z"
      }
   },
   {
      "modelId": "80f58a45-86f2-4e18-be3d-b60a2c88312e",
      "details": {
         "statusId": 0,
         "status": "Success",
         "exampleCount": 33,
         "trainingDateTime": "2017-11-20T18:09:11Z"
      }
   },
   {
      "modelId": "c9eb9772-3b18-4d5f-a1e6-e0c31f91b390",
      "details": {
         "statusId": 0,
         "status": "Success",
         "exampleCount": 33,
         "trainingDateTime": "2017-11-20T18:09:11Z"
      }
   },
   {
      "modelId": "2afec2ff-7c01-4423-bb0e-e5f6935afae8",
      "details": {
         "statusId": 0,
         "status": "Success",
         "exampleCount": 33,
         "trainingDateTime": "2017-11-20T18:09:11Z"
      }
   },
   {
      "modelId": "95a81c87-0d7b-4251-8e07-f28d180886a1",
      "details": {
         "statusId": 0,
         "status": "Success",
         "exampleCount": 33,
         "trainingDateTime": "2017-11-20T18:09:11Z"
      }
   }
]

Next steps