Using AdaptiveCards in your Cortana skill

Cards are interface elements that you can use to enhance the user experience in your Cortana skill. An adaptive card is the most versatile display card. It's customizable, and can include any combination of text, speech, images, buttons, and input fields.

As with all cards, you can only use them when Cortana is running on a device with a display. See Determine Cortana's device type for details on how to get device information.

AdaptiveCards

Adaptive cards provide the following options.

Input controls Add input controls for text, date, number, time, toggle switch, and choice set.
Richer text Text in your card is not limited to title, subtitle, and text fixed formats. Use a variety of font sizes, formats, and colors.
A single open card exchange format Use your existing cards in a common and consistent way and extend your cards with rich controls using a common schema.

AdaptiveCards use the open card exchange format. This format enables you to specify user interface content for all cards in your skill in a common and consistent way. You describe the content as a simple JSON object. The JSON content is natively displayed by the skill and automatically adapts to the look and feel of your skill.

Note

Cortana currently supports AdaptiveCards version 1.0.

AdaptiveCards include elements, containers, actions, and inputs. A basic adaptive card includes:

  • an adaptive card root object,
  • an adaptive card body, which includes the elements of your card, and
  • actions for your adaptive card, which are typically displayed in an action bar at the bottom of your card.

AdaptiveCards Designer

The AdaptiveCards Designer provides an interactive card builder where you can see the resulting card JSON data.

Create an AdaptiveCard

You can create adaptive cards using proxy pattern helper classes in the SDK, or by directly using JSON (check out the Schema Explorer for details).

Important

  1. The speak object of an adaptive card must be copied to the Message for Cortana to speak the text.
  2. The speak object text must be wrapped in SSML <speak> tags. (See the Speech Synthesis Markup Language (SSML) reference.)

Create using .NET

  1. Install the AdaptiveCards NuGet package.
  2. Specify the elements of your card in code.
  3. Add the card to your Cortana skill as an attachment.

The following code adds an adaptive card to a Cortana skill response for Bot Framework version 4.

// make a response
var response = turnContext.Activity.CreateReply();

// create a Card and add elements
AdaptiveCard card = new AdaptiveCard();
card.Body.Add(new AdaptiveTextBlock()
    {
        Text = "This is a test",
        Weight = TextWeight.Bolder
        Size = TextSize.Medium,
    }
);

// add card as attachment
response.Attachments.Add(card.ToAttachment());

// send the response
await turnContext.SendActivityAsync(response, cancellationToken: cancellationToken);

Respond to AdaptiveCards

If you look at the Calendar reminder example on the AdaptiveCards website, you'll note that it has two Action.Submit buttons, one for the Snooze response, and one for the I'll be Late response. For Snooze, there is a Input.ChoiceSet with standard values of 5, 15, and 30 minutes.

In our example, Cortana will say, "Your meeting 'Adaptive Card design session' is starting at 12:30pm. Do you want to snooze or do you want to send a late notification to the attendees?" At the same time, she will display this card:

Sample card

Cortana skills should be designed for voice first, so consider this code that supports spoken replies like

  • I'll be late
  • Snooze for 10 minutes

based on simple keyword search (matches in bold).

You will receive a JSON value attached to the response message with the data properties of the Action.Submit buttons, along with any Input fields with their id and value fields. Given this example, if the user clicks the Snooze button, they'll see:

 "value": {
      "x": "snooze",  <-- the Action.Submit data
      "snooze": "5"   <-- the Input.ChoiceSet id and selected value
    }

Important

Cortana responds differently depending on how the user responds. If the user clicks the button, Cortana returns the value property. If the user speaks the response, Cortana sends the text property.

The example returns data in the JSON value because the user pressed the button on the card. If the response is spoken, there will be a text response, but no value.

You should ignore the text property on the message if a value is present.

Your code must handle both cases in order to handle conversations correctly.

Respond in C#

...
        // match 1 or 2 digits and white space and "minute"
        private static Regex regexMinutes = new Regex(@"(\d{1,2})\s+minute", RegexOptions.IgnoreCase);
...
                var message = turnContext.Activity;
                var response = turnContext.Activity.CreateReply(); 

                string sValue = "unknown";

                if (message.Value != null)
                {
                    // Got an Action Submit
                    dynamic value = message.Value;
                    string xValue = (string)value.SelectToken("x");
                    if (xValue.Equals("snooze"))
                    {
                        string snoozeValue = (string)value.SelectToken("snooze");
                        response.Text = "You clicked \"Snooze\" for " + snoozeValue + " minutes.";
                        response.Speak = response.Text;
                    }
                    else if (xValue.Equals("late"))
                    {
                        response.Text = "You clicked \"I'll be late.\"";
                        response.Speak = response.Text;
                    }
                    else
                    {
                        response.Text = "Unsupported input detected.";
                        response.Speak = response.Text;

                        sValue = value.ToString();
                        Trace.WriteLine(sValue);
                    }
                }
                else
                {
                    // Got a Voice/Text response - check for keywords
                    string sText = message.Text.ToUpper(); // as Contains doesn't allow case insensitive
                    if (sText.Contains("SNOOZE"))
                    {
                        Match m = regexMinutes.Match(sText);
                        if (m.Success)
                        {
                            Group g = m.Groups[1];
                            string snoozeValue = g.Value;
                            response.Text = "You said \"Snooze\" for " + snoozeValue + " minutes.";
                            response.Speak = response.Text;
                        }
                        else
                        {
                            response.Text = "You said \"Snooze\". I'll snooze for the default 5 minutes.";
                            response.Speak = response.Text;
                        }
                    }
                    else if (sText.Contains("LATE"))
                    {
                        response.Text = "You said \"I'll be late.\"";
                        response.Speak = response.Text;
                    }
                    else
                    {
                        response.Text = "I am not sure what you mean.";
                        response.Speak = response.Text;
                    }
                }

                await turnContext.SendActivityAsync(response, cancellationToken: cancellationToken);

More information