Integrate media capabilities

This document guides you on how to integrate media capabilities. This integration combines the native device capabilities, such as the camera and microphone with the Teams platform.

You can use Microsoft Teams JavaScript client SDK, that provides the tools necessary for your app to access a user’s device permissions. Use suitable media capability APIs to integrate the native device capabilities, such as the camera and microphone with the Teams platform within your Microsoft Teams mobile app, and build a richer experience.

Advantage of integrating media capabilities

The main advantage of integrating device capabilities in your Teams apps is it leverages native Teams controls to provide a rich and immersive experience to your users. To integrate media capabilities you must update the app manifest file and call the media capability APIs.

For effective integration, you must have a good understanding of code snippets for calling the respective APIs, which allow you to use native media capabilities.

It is important to familiarize yourself with the API response errors to handle the errors in your Teams app.

Note

Currently, Microsoft Teams support for media capabilities is only available for mobile clients.

Update manifest

Update your Teams app manifest.json file by adding the devicePermissions property and specifying media. It allows your app to ask for requisite permissions from users before they start using the camera to capture the image, open the gallery to select an image to submit as an attachment, or use the microphone to record the conversation.

"devicePermissions": [
    "media",
],

Note

The Request Permissions prompt is automatically displayed when a relevant Teams API is initiated. For more information, see Request device permissions.

Media capability APIs

The selectMedia, getMedia, and viewImages APIs enable you to use native media capabilities as follows:

  • Use the native microphone to allow users to record audio (record 10 minutes of conversation) from the device.
  • Use native camera control to allow users to capture and attach images on the go.
  • Use native gallery support to allow users to select device images as attachments.
  • Use native image viewer control to preview multiple images at one time.
  • Support large image transfer (from 1 MB to 50 MB) through the SDK bridge.
  • Support advanced image capabilities allowing users to preview and edit images:
    • Scan document, whiteboard, and business cards through the camera.

Important

  • The selectMedia, getMedia, and viewImages APIs can be invoked from multiple Teams surfaces, such as task modules, tabs, and personal apps. For more details, see Entry points for Teams apps.
  • selectMedia API has been extended to support microphone and audio properties.

You must use the following set of APIs to enable your device's media capabilities:

API Description
selectMedia (Camera) This API allows users to capture or select media from the device camera and return it to the web-app. The users can edit, crop, rotate, annotate, or draw over images before submission. In response to selectMedia, the web-app receives the media IDs of selected images and a thumbnail of the selected media. This API can be further configured through the ImageProps configuration.
selectMedia (Microphone) Set the mediaType to 4 in selectMedia API for accessing microphone capability. This API also allows users to record audio from the device microphone and return recorded clips to the web-app. The users can pause, re-record, and play recording preview before submission. In response to selectMedia, the web-app receives media IDs of the selected audio recording.
Use maxDuration, if you require to configure a duration in minutes for recording the conversation. The current duration for recording is 10 minutes, after which the recording terminates.
getMedia This API retrieves the media captured by selectMedia API in chunks, irrespective of the media size. These chunks are assembled and sent back to the web app as a file or blob. Breaking of media into smaller chunks facilitates large file transfer.
viewImages This API enables the user to view images in full-screen mode as a scrollable list.

Web app experience for selectMedia API for image capability device camera and image experience in Teams

Web app experience for selectMedia API for microphone capability web app experience for microphone capability

Error handling

You must ensure to handle these errors appropriately in your Teams app. The following table lists the error codes and the conditions under which the errors are generated:

Error code Error name Condition
100 NOT_SUPPORTED_ON_PLATFORM API is not supported on the current platform.
404 FILE_NOT_FOUND File specified is not found in the given location.
500 INTERNAL_ERROR Internal error is encountered while performing the required operation.
1000 PERMISSION_DENIED Permission is denied by the user.
2000 NETWORK_ERROR Network issue.
3000 NO_HW_SUPPORT Underlying hardware does not support the capability.
4000 INVALID_ARGUMENTS One or more arguments are invalid.
5000 UNAUTHORIZED_USER_OPERATION User is not authorized to complete this operation.
6000 INSUFFICIENT_RESOURCES Operation could not be completed due to insufficient resources.
7000 THROTTLE Platform throttled the request as the API was invoked frequently.
8000 USER_ABORT User aborts the operation.
9000 OLD_PLATFORM Platform code is outdated and does not implement this API.
10000 SIZE_EXCEEDED Return value is too big and has exceeded the platform size boundaries.

Code snippets

Calling selectMedia API for capturing images using camera:

let imageProp: microsoftTeams.media.ImageProps = {
    sources: [microsoftTeams.media.Source.Camera, microsoftTeams.media.Source.Gallery],
    startMode: microsoftTeams.media.CameraStartMode.Photo,
    ink: false,
    cameraSwitcher: false,
    textSticker: false,
    enableFilter: true,
};
let mediaInput: microsoftTeams.media.MediaInputs = {
    mediaType: microsoftTeams.media.MediaType.Image,
    maxMediaCount: 10,
    imageProps: imageProp
};
microsoftTeams.media.selectMedia(mediaInput, (error: microsoftTeams.SdkError, attachments: microsoftTeams.media.Media[]) => {
    if (error) {
        if (error.message) {
            alert(" ErrorCode: " + error.errorCode + error.message);
        } else {
            alert(" ErrorCode: " + error.errorCode);
        }
    }
    if (attachments) {
        let y = attachments[0];
        img.src = ("data:" + y.mimeType + ";base64," + y.preview);
    }
});

Calling getMedia API to retrieve large media in chunks:

let media: microsoftTeams.media.Media = attachments[0]
media.getMedia((error: microsoftTeams.SdkError, blob: Blob) => {
    if (blob) {
        if (blob.type.includes("image")) {
            img.src = (URL.createObjectURL(blob));
        }
    }
    if (error) {
        if (error.message) {
            alert(" ErrorCode: " + error.errorCode + error.message);
        } else {
            alert(" ErrorCode: " + error.errorCode);
        }
    }
});

Calling viewImages API by ID returned by selectMedia API:

// View images by id:
// Assumption: attachmentArray = select Media API Output
let uriList = [];
if (attachmentArray && attachmentArray.length > 0) {
    for (let i = 0; i < attachmentArray.length; i++) {
        let file = attachmentArray[i];
        if (file.mimeType.includes("image")) {
            let imageUri = {
                value: file.content,
                type: 1,
            }
            uriList.push(imageUri);
        } else {
            alert("File type is not image");
        }
    }
}
if (uriList.length > 0) {
    microsoftTeams.media.viewImages(uriList, (error: microsoftTeams.SdkError) => {
        if (error) {
            if (error.message) {
                output(" ErrorCode: " + error.errorCode + error.message);
            } else {
                output(" ErrorCode: " + error.errorCode);
            }
        }
    });
} else {
    output("Url list is empty");
}

Calling viewImages API by URL:

// View Images by URL:
// Assumption 2 urls, url1 and url2
let uriList = [];
if (URL1 != null && URL1.length > 0) {
    let imageUri = {
        value: URL1,
        type: 2,
    }
    uriList.push(imageUri);
}
if (URL2 != null && URL2.length > 0) {
    let imageUri = {
        value: URL2,
        type: 2,
    }
    uriList.push(imageUri);
}
if (uriList.length > 0) {
    microsoftTeams.media.viewImages(uriList, (error: microsoftTeams.SdkError) => {
        if (error) {
            if (error.message) {
                output(" ErrorCode: " + error.errorCode + error.message);
            } else {
                output(" ErrorCode: " + error.errorCode);
            }
        }
    });
} else {
    output("Url list is empty");
}

Calling selectMedia and getMedia APIs for recording audio through microphone:

let mediaInput: microsoftTeams.media.MediaInputs = {
    mediaType: microsoftTeams.media.MediaType.Audio,
    maxMediaCount: 1,
};
microsoftTeams.media.selectMedia(mediaInput, (error: microsoftTeams.SdkError, attachments: microsoftTeams.media.Media[]) => {
    if (error) {
        if (error.message) {
            alert(" ErrorCode: " + error.errorCode + error.message);
        } else {
            alert(" ErrorCode: " + error.errorCode);
        }
    }
    // If you want to directly use the audio file (for smaller file sizes (~4MB))    if (attachments) {
    let audioResult = attachments[0];
    var videoElement = document.createElement("video");
    videoElement.setAttribute("src", ("data:" + y.mimeType + ";base64," + y.preview));
    //To use the audio file via get Media API for bigger audio file sizes greater than 4MB        audioResult.getMedia((error: microsoftTeams.SdkError, blob: Blob) => {
    if (blob) {
        if (blob.type.includes("video")) {
            videoElement.setAttribute("src", URL.createObjectURL(blob));
        }
    }
    if (error) {
        if (error.message) {
            alert(" ErrorCode: " + error.errorCode + error.message);
        } else {
            alert(" ErrorCode: " + error.errorCode);
        }
    }
});

See also