Add media to messages

APPLIES TO: yesSDK v4 no SDK v3

Messages exchanged between user and bot can contain media attachments, such as images, video, audio, and files. The Bot Framework SDK supports the task of sending rich messages to the user. To determine the type of rich messages a channel (Facebook, Skype, Slack, etc.) supports, consult the channel's documentation for information about limitations.

Prerequisites

Send attachments

To send the user content like an image or a video, you can add an attachment or list of attachments to a message.

See design user experience for examples of available cards.

The Attachments property of the Activity object contains an array of Attachment objects that represent the media attachments and rich cards attached to the message. To add a media attachment to a message, create an Attachment object for the reply activity (that was created off the activity with CreateReply()) and set the ContentType, ContentUrl, and Name properties.

The source code shown here is based on the Handling Attachments sample.

To create the reply message, define the text and then set up the attachments. Assigning the attachments to the reply is the same for each attachment type, however the various attachments are set up and defined differently, as seen in the following snippets. The code below is setting up the reply for an inline attachment:

Bots/AttachmentsBot.cs

reply = MessageFactory.Text("This is an inline attachment.");
reply.Attachments = new List<Attachment>() { GetInlineAttachment() };

Next, we look at the types of attachments. First is an inline attachment:

Bots/AttachmentsBot.cs

private static Attachment GetInlineAttachment()
{
    var imagePath = Path.Combine(Environment.CurrentDirectory, @"Resources\architecture-resize.png");
    var imageData = Convert.ToBase64String(File.ReadAllBytes(imagePath));

    return new Attachment
    {
        Name = @"Resources\architecture-resize.png",
        ContentType = "image/png",
        ContentUrl = $"data:image/png;base64,{imageData}",
    };
}

Then, an uploaded attachment:

Bots/AttachmentsBot.cs

private static async Task<Attachment> GetUploadedAttachmentAsync(ITurnContext turnContext, string serviceUrl, string conversationId)
{
    if (string.IsNullOrWhiteSpace(serviceUrl))
    {
        throw new ArgumentNullException(nameof(serviceUrl));
    }

    if (string.IsNullOrWhiteSpace(conversationId))
    {
        throw new ArgumentNullException(nameof(conversationId));
    }

    var imagePath = Path.Combine(Environment.CurrentDirectory, @"Resources\architecture-resize.png");

    var connector = turnContext.TurnState.Get<IConnectorClient>() as ConnectorClient;
    var attachments = new Attachments(connector);
    var response = await attachments.Client.Conversations.UploadAttachmentAsync(
        conversationId,
        new AttachmentData
        {
            Name = @"Resources\architecture-resize.png",
            OriginalBase64 = File.ReadAllBytes(imagePath),
            Type = "image/png",
        });

    var attachmentUri = attachments.GetAttachmentUri(response.Id);

    return new Attachment
    {
        Name = @"Resources\architecture-resize.png",
        ContentType = "image/png",
        ContentUrl = attachmentUri,
    };
}

Lastly, an internet attachment:

Bots/AttachmentsBot.cs

private static Attachment GetInternetAttachment()
{
    // ContentUrl must be HTTPS.
    return new Attachment
    {
        Name = @"Resources\architecture-resize.png",
        ContentType = "image/png",
        ContentUrl = "https://docs.microsoft.com/en-us/bot-framework/media/how-it-works/architecture-resize.png",
    };
}

If an attachment is an image, audio, or video, the Connector service will communicate attachment data to the channel in a way that enables the channel to render that attachment within the conversation. If the attachment is a file, the file URL will be rendered as a hyperlink within the conversation.

Send a hero card

Besides simple image or video attachments, you can attach a hero card, which allows you to combine images and buttons in one object, and send them to the user. Markdown is supported for most text fields, but support may vary by channel.

To compose a message with a hero card and button, you can attach a HeroCard to a message.

The source code shown here is based on the Handling Attachments sample.

Bots/AttachmentsBot.cs

private static async Task DisplayOptionsAsync(ITurnContext turnContext, CancellationToken cancellationToken)
{
    // Create a HeroCard with options for the user to interact with the bot.
    var card = new HeroCard
    {
        Text = "You can upload an image or select one of the following choices",
        Buttons = new List<CardAction>
        {
            // Note that some channels require different values to be used in order to get buttons to display text.
            // In this code the emulator is accounted for with the 'title' parameter, but in other channels you may
            // need to provide a value for other parameters like 'text' or 'displayText'.
            new CardAction(ActionTypes.ImBack, title: "1. Inline Attachment", value: "1"),
            new CardAction(ActionTypes.ImBack, title: "2. Internet Attachment", value: "2"),
            new CardAction(ActionTypes.ImBack, title: "3. Uploaded Attachment", value: "3"),
        },
    };

    var reply = MessageFactory.Attachment(card.ToAttachment());
    await turnContext.SendActivityAsync(reply, cancellationToken);
}

Process events within rich cards

To process events within rich cards, use card action objects to specify what should happen when the user clicks a button or taps a section of the card. Each card action has a type and value.

To function correctly, assign an action type to each clickable item on the card. This table lists and describes the available action types and what should be in the associated value property.

Type Description Value
openUrl Opens a URL in the built-in browser. The URL to open.
imBack Sends a message to the bot, and posts a visible response in the chat. Text of the message to send.
postBack Sends a message to the bot, and may not post a visible response in the chat. Text of the message to send.
call Initiates a phone call. Destination for the phone call in this format: tel:123123123123.
playAudio Plays audio. The URL of the audio to play.
playVideo Plays a video. The URL of video to play.
showImage Displays an image. The URL of the image to display.
downloadFile Downloads a file. The URL of the file to download.
signin Initiates an OAuth signin process. The URL of the OAuth flow to initiate.

Hero card using various event types

The following code shows examples using various rich card events.

For examples of all the available cards, see the C# cards sample.

Cards.cs

public static HeroCard GetHeroCard()
{
    var heroCard = new HeroCard
    {
        Title = "BotFramework Hero Card",
        Subtitle = "Microsoft Bot Framework",
        Text = "Build and connect intelligent bots to interact with your users naturally wherever they are," +
               " from text/sms to Skype, Slack, Office 365 mail and other popular services.",
        Images = new List<CardImage> { new CardImage("https://sec.ch9.ms/ch9/7ff5/e07cfef0-aa3b-40bb-9baa-7c9ef8ff7ff5/buildreactionbotframework_960.jpg") },
        Buttons = new List<CardAction> { new CardAction(ActionTypes.OpenUrl, "Get Started", value: "https://docs.microsoft.com/bot-framework") },
    };

    return heroCard;
}

Cards.cs

public static SigninCard GetSigninCard()
{
    var signinCard = new SigninCard
    {
        Text = "BotFramework Sign-in Card",
        Buttons = new List<CardAction> { new CardAction(ActionTypes.Signin, "Sign-in", value: "https://login.microsoftonline.com/") },
    };

    return signinCard;
}

Send an Adaptive Card

Adaptive Card and MessageFactory are used to send rich messages including texts, images, video, audio and files to communicate with users. However, there are some differences between them.

First, only some channels support Adaptive Cards, and channels that do support it might partially support Adaptive Cards. For example, if you send an Adaptive Card in Facebook, the buttons won't work while texts and images work well. MessageFactory is just a helper class within the Bot Framework SDK to automate creation steps for you, and supported by most channels.

Second, Adaptive Card delivers messages in the card format, and the channel determines the layout of the card. The format of messages MessageFactory delivers depends on the channel, and is not necessarily in the card format unless Adaptive Card is part of the attachment.

To find the latest information on Adaptive Card channel support, see the Adaptive Cards Designer.

To use adaptive cards, be sure to add the AdaptiveCards NuGet package.

Note

You should test this feature with the channels your bot will use to determine whether those channels support adaptive cards.

To use Adaptive Cards, be sure to add the AdaptiveCards NuGet package.

The source code shown here is based on the Using cards sample:

Cards.cs

public static Attachment CreateAdaptiveCardAttachment()
{
    // combine path for cross platform support
    string[] paths = { ".", "Resources", "adaptiveCard.json" };
    var adaptiveCardJson = File.ReadAllText(Path.Combine(paths));

    var adaptiveCardAttachment = new Attachment()
    {
        ContentType = "application/vnd.microsoft.card.adaptive",
        Content = JsonConvert.DeserializeObject(adaptiveCardJson),
    };
    return adaptiveCardAttachment;
}

Messages can also include multiple attachments in a carousel layout, which places the attachments side by side and allows the user to scroll across.

The source code shown here is based on the Cards sample.

First, create the reply and define the attachments as a list.

Dialogs/MainDialog.cs

// Cards are sent as Attachments in the Bot Framework.
// So we need to create a list of attachments for the reply activity.
var attachments = new List<Attachment>();

// Reply to the activity we received with an activity.
var reply = MessageFactory.Attachment(attachments);

Then add the attachments. Here we're adding them one at a time, but feel free to manipulate the list to add the cards however you prefer.

Dialogs/MainDialog.cs

// Display a carousel of all the rich card types.
reply.AttachmentLayout = AttachmentLayoutTypes.Carousel;
reply.Attachments.Add(Cards.CreateAdaptiveCardAttachment());
reply.Attachments.Add(Cards.GetAnimationCard().ToAttachment());
reply.Attachments.Add(Cards.GetAudioCard().ToAttachment());
reply.Attachments.Add(Cards.GetHeroCard().ToAttachment());
reply.Attachments.Add(Cards.GetReceiptCard().ToAttachment());
reply.Attachments.Add(Cards.GetSigninCard().ToAttachment());
reply.Attachments.Add(Cards.GetThumbnailCard().ToAttachment());
reply.Attachments.Add(Cards.GetVideoCard().ToAttachment());

Once the attachments are added, you can send the reply just like any other.

Dialogs/MainDialog.cs

// Send the card(s) to the user as an attachment to the activity
await stepContext.Context.SendActivityAsync(reply, cancellationToken);

Additional resources

See design user experience for examples of available cards.

For detailed information on the schema, see the Bot Framework card schema and the message activity section of the Bot Framework Activity schema.

Code sample for processing Adaptive Card input

This sample code shows one way to use Adaptive Card inputs within a bot dialog class. It extends the current sample 06.using-cards by validating the input recieved in the text field from the responding client. We first added text input and button functionality to the existing adaptive card by adding the following code just before the final bracket of adaptiveCard.json, found in the resources folder:

  ,
  "actions": [
    {
      "type": "Action.ShowCard",
      "title": "Text",
      "card": {
      "type": "AdaptiveCard",
      "body": [
        {
          "type": "Input.Text",
          "id": "text",
          "isMultiline": true,
          "placeholder": "Enter your comment"
        }
      ],
      "actions": [
        {
          "type": "Action.Submit",
          "title": "OK"
        }
      ]
    }
  }
]

Note that the input field is labeled "text" so our adaptive card will attach comment text data as Value.[text.]

our validator uses Newtonsoft.json to first convert this to a JObject, and then create a trimmed text string for comparison. So add:

using Newtonsoft.Json.Linq;

to MainDialog.cs and install the latest stable nuget package of Newtonsoft.Json. In the validator code we added the logic flow into the code comments. This ChoiceValidator() code is placed into the 06.using-cards sample just after the closed brace public for declaration of MainDialog:

private async Task ChoiceValidator(
  PromptValidatorContext promptContext,
  CancellationToken cancellationToken)
  {
    // Retrieves Adaptive Card comment text as JObject.
    // looks for JObject field "text" and converts that input into a trimmed text string.
    var jobject = promptContext.Context.Activity.Value as JObject;
    var jtoken = jobject?["text"];
    var text = jtoken?.Value().Trim();
    // Logic: 1. if succeeded = true, just return promptContext
    //        2. if false, see if JObject contained Adaptive Card input.
    //               No = (bad input) return promptContext
    //               Yes = update Value field with JObject text string, return "true".
    if (!promptContext.Recognized.Succeeded && text != null)
    {
       var choice = promptContext.Options.Choices.FirstOrDefault(
       c => c.Value.Equals(text, StringComparison.InvariantCultureIgnoreCase));
       if (choice != null)
       {
           promptContext.Recognized.Value = new FoundChoice
            {
               Value = choice.Value,
             };
            return true;
       }
    }
    return promptContext.Recognized.Succeeded;
  }

Now above in the MainDialog declaration change:

// Define the main dialog and its related components.
AddDialog(new ChoicePrompt(nameof(ChoicePrompt)));

to:

// Define the main dialog and its related components.
AddDialog(new ChoicePrompt(nameof(ChoicePrompt), ChoiceValidator));

This will invoke your validator to look for Adaptive Card input each time a new ChoicePrompt is created.

To test your code, once an Adaptive Card has been displayed, Click the "Text" button, Enter a valid selection such as "Hero Card" and click the "OK" button.

Test Adaptive Card

  1. The first input will be used to start a new dialog.
  2. Click the "OK" button again and this input will be used to select a new card.

Next steps