Share via

Send media attachments with Bot Framework SDK

  • Article

APPLIES TO: SDK v4

Messages exchanged between user and bot can contain media attachments, such as images, video, audio, and files. The Bot Framework SDK supports the task of sending rich messages to the user. To determine the type of rich messages a channel (Facebook, Slack, and so on) supports, consult the channel's documentation for information about limitations.

Note

The Bot Framework JavaScript, C#, and Python SDKs will continue to be supported, however, the Java SDK is being retired with final long-term support ending in November 2023.

Existing bots built with the Java SDK will continue to function.

For new bot building, consider using Microsoft Copilot Studio and read about choosing the right copilot solution.

For more information, see The future of bot building.

Prerequisites

Send attachments

To send the user content like an image or a video, you can add an attachment or list of attachments to a message.

See Design the user experience for examples of available cards.

See also What is the size limit of a file transferred using channels? in the FAQ.

The following source code is from the Handling attachments sample.

To use attachments, include the following libraries in your bot:

bots/attachmentsBot.js

const { ActivityHandler, ActionTypes, ActivityTypes, CardFactory } = require('botbuilder');

To create the reply message, define the text and then set up the attachments. Assigning the attachments to the reply is the same for each attachment type, however the various attachments are set up and defined differently, as seen in the following snippets. The code below is setting up the reply for an inline attachment:

bots/attachmentsBot.js

*/
   const firstChar = turnContext.activity.text[0];
   if (firstChar === '1') {

To send the user a single piece of content like an image or a video, you can send media in a few different ways. First, as an inline attachment:

bots/attachmentsBot.js

 * Returns an inline attachment.
 */
getInlineAttachment() {
    const imageData = fs.readFileSync(path.join(__dirname, '../resources/architecture-resize.png'));
    const base64Image = Buffer.from(imageData).toString('base64');

    return {
        name: 'architecture-resize.png',
        contentType: 'image/png',
        contentUrl: `data:image/png;base64,${ base64Image }`

Then, an uploaded attachment:

bots/attachmentsBot.js

 * @param {Object} turnContext
 */
async getUploadedAttachment(turnContext) {
    const imageData = fs.readFileSync(path.join(__dirname, '../resources/architecture-resize.png'));
    const connectorFactory = turnContext.turnState.get(turnContext.adapter.ConnectorFactoryKey);
    const connector = await connectorFactory.create(turnContext.activity.serviceUrl);
    const conversationId = turnContext.activity.conversation.id;
    const response = await connector.conversations.uploadAttachment(conversationId, {
        name: 'architecture-resize.png',
        originalBase64: imageData,
        type: 'image/png'
    });

    // Retrieve baseUri from ConnectorClient for... something.
    const baseUri = connector.baseUri;
    const attachmentUri = baseUri + (baseUri.endsWith('/') ? '' : '/') + `v3/attachments/${ encodeURI(response.id) }/views/original`;
    return {
        name: 'architecture-resize.png',
        contentType: 'image/png',

Lastly, an internet attachment contained in a URL:

bots/attachmentsBot.js

 * Returns an attachment to be sent to the user from a HTTPS URL.
 */
getInternetAttachment() {
    // NOTE: The contentUrl must be HTTPS.
    return {
        name: 'architecture-resize.png',
        contentType: 'image/png',
        contentUrl: 'https://docs.microsoft.com/en-us/bot-framework/media/how-it-works/architecture-resize.png'

If an attachment is an image, audio, or video, the Connector service will communicate attachment data to the channel in a way that enables the channel to render that attachment within the conversation. If the attachment is a file, the file URL will be rendered as a hyperlink within the conversation.

Send a hero card

Besides simple image or video attachments, you can attach a hero card, which allows you to combine images and buttons in one object, and send them to the user. Markdown is supported for most text fields, but support may vary by channel.

To compose a message with a hero card and button, you can attach a HeroCard object to a message.

The following source code is from the Handling attachments sample.

bots/attachmentsBot.js

 * @param {Object} turnContext
 */
async displayOptions(turnContext) {
    const reply = { type: ActivityTypes.Message };

    // Note that some channels require different values to be used in order to get buttons to display text.
    // In this code the emulator is accounted for with the 'title' parameter, but in other channels you may
    // need to provide a value for other parameters like 'text' or 'displayText'.
    const buttons = [
        { type: ActionTypes.ImBack, title: '1. Inline Attachment', value: '1' },
        { type: ActionTypes.ImBack, title: '2. Internet Attachment', value: '2' },
        { type: ActionTypes.ImBack, title: '3. Uploaded Attachment', value: '3' }
    ];

    const card = CardFactory.heroCard('', undefined,
        buttons, { text: 'You can upload an image or select one of the following choices.' });

    reply.attachments = [card];

Process events within rich cards

To process events within rich cards, use card action objects to specify what should happen when the user selects a button or taps a section of the card. Each card action has a type and value property.

To function correctly, assign an action type to each clickable item on a hero card. This table lists and describes the available action types and what should be in the associated value property.

The messageBack card action has a more generalized meaning than the other card actions. See the Card action section of the Activity schema for more information about the messageBack and other card action types.

Type Description Value
call Initiates a phone call. Destination for the phone call in this format: tel:123123123123.
downloadFile Downloads a file. The URL of the file to download.
imBack Sends a message to the bot, and posts a visible response in the chat. Text of the message to send.
messageBack Represents a text response to be sent via the chat system. An optional programmatic value to include in generated messages.
openUrl Opens a URL in the built-in browser. The URL to open.
playAudio Plays audio. The URL of the audio to play.
playVideo Plays a video. The URL of video to play.
postBack Sends a message to the bot, and may not post a visible response in the chat. Text of the message to send.
showImage Displays an image. The URL of the image to display.
signin Initiates an OAuth sign-in process. The URL of the OAuth flow to initiate.

Hero card using various event types

The following code shows examples using various rich card events.

For examples of all the available cards, see the Using cards sample.

dialogs/mainDialog.js

createHeroCard() {
    return CardFactory.heroCard(
        'BotFramework Hero Card',
        CardFactory.images(['https://sec.ch9.ms/ch9/7ff5/e07cfef0-aa3b-40bb-9baa-7c9ef8ff7ff5/buildreactionbotframework_960.jpg']),
        CardFactory.actions([
            {
                type: 'openUrl',
                title: 'Get started',
                value: 'https://docs.microsoft.com/en-us/azure/bot-service/'
            }
        ])
    );
}

dialogs/mainDialog.js

createOAuthCard() {
    return CardFactory.oauthCard(
        'OAuth connection', // Replace with the name of your Azure AD connection
        'Sign In',
        'BotFramework OAuth Card'
    );
}

Send an Adaptive Card

While you can use the message factory to create a message that contains an attachment (of any sort), an Adaptive Card is one specific type of attachment. Not all channels support Adaptive Cards, and some channels may only partially support Adaptive Cards. For example, if you send an Adaptive Card in Facebook, the buttons won't work while texts and images work well. The message factory is a Bot Framework SDK helper class used to automate creation steps for you.

Adaptive Cards are an open card exchange format enabling developers to exchange UI content in a common and consistent way. However, not all channels support Adaptive Cards.

The Adaptive Cards Designer provides a rich, interactive design-time experience for authoring adaptive cards.

Note

You should test this feature with the channels your bot will use to determine whether those channels support adaptive cards.

To use Adaptive Cards, be sure to add the adaptivecards npm package.

The following source code is from the Using cards sample.

dialogs/mainDialog.js

This example reads the Adaptive Card JSON from a file and creates a message activity with the card attached.

const AdaptiveCard = require('../resources/adaptiveCard.json');

createAdaptiveCard() {
    return CardFactory.adaptiveCard(AdaptiveCard);
}

Messages can also include multiple attachments in a carousel layout, which places the attachments side by side and allows the user to scroll across.

The following source code is from the Using cards sample.

dialogs/mainDialog.js

Add the attachments and set the layout type to carousel. Once the attachments are added, you can send the reply just like any other.

await stepContext.context.sendActivity({
    attachments: [
        this.createAdaptiveCard(),
        this.createAnimationCard(),
        this.createAudioCard(),
        this.createHeroCard(),
        this.createOAuthCard(),
        this.createReceiptCard(),
        this.createSignInCard(),
        this.createThumbnailCard(),
        this.createVideoCard()
    ],
    attachmentLayout: AttachmentLayoutTypes.Carousel
});

Code sample for processing Adaptive Card input

The following sample shows one way to use Adaptive Card inputs within a bot dialog class. It extends the hero cards sample by validating the input received in the text field from the responding client. You first need to add the text input and button functionality to the existing adaptive card by adding the following code just before the final bracket of adaptiveCard.json, located in the resources folder:

"actions": [
  {
    "type": "Action.ShowCard",
    "title": "Text",
    "card": {
      "type": "AdaptiveCard",
      "body": [
        {
          "type": "Input.Text",
          "id": "text",
          "isMultiline": true,
          "placeholder": "Enter your comment"
        }
      ],
      "actions": [
        {
          "type": "Action.Submit",
          "title": "OK"
        }
      ]
    }
  }
]

The ID of the text input field is set to "text". When the user selects OK, the message the Adaptive Card generates will have a value property that has a property named text that contains the information the user entered in the text input field of the card.

Open mainDialog.js and find the run method async run(turnContext, accessor) This method handles incoming activity. Just after the call dialogSet.add(this); add the following:

// The following check looks for a non-existent text input
// plus Adaptive Card input in _activity.value.text
// If both conditions exist, the Activity Card text
// is copied into the text input field.
if(turnContext._activity.text == null
    && turnContext._activity.value.text != null) {
    this.logger.log('replacing null text with Activity Card text input');
    turnContext._activity.text = turnContext._activity.value.text;
  }

If this check finds a non-existent text input from the client, it looks to see if there's input from an Adaptive Card. If an Adaptive Card input exists at _activity.value.text, it copies this into the normal text input field.

Test the Using Cards bot

  1. Run the Using cards sample locally and open the bot in the Bot Framework Emulator.
  2. Follow the prompts in the bot to display a card type, such as an Adaptive Card.

Next steps

Add buttons to guide user action