Embed Azure AI Video Indexer widgets in your apps

Article
02/29/2024

Important

Due to the Azure Media Services retirement announcement, Azure AI Video Indexer announces Azure AI Video Indexer features adjustments. See Changes related to Azure Media Service (AMS) retirement to understand what this means for your Azure AI Video Indexer account. See the Preparing for AMS retirement: VI update and migration guide.

This article shows how you can embed Azure AI Video Indexer widgets in your apps. Azure AI Video Indexer supports embedding three types of widgets into your apps: Insights, Player, and Editor.

A Insights widget includes all visual insights that were extracted from your video indexing process. The Insights widget supports the following optional URL parameters:

Name	Definition	Description
`widgets`	Strings separated by comma	Allows you to control the insights that you want to render. Example: `https://www.videoindexer.ai/embed/insights/<accountId>/<videoId>/?widgets=people,keywords` renders only people and keywords UI insights. Available options: `people`, `keywords`, `audioEffects`, `labels`, `sentiments`, `emotions`, `topics`, `keyframes`, `transcript`, `ocr`, `speakers`, `scenes`, `spokenLanguage`, `observedPeople`, `namedEntities`, `detectedObjects`.
`controls`	Strings separated by comma	Allows you to control the controls that you want to render. Example: `https://www.videoindexer.ai/embed/insights/<accountId>/<videoId>/?controls=search,download` renders only search option and download button. Available options: `search`, `download`, `presets`, `language`.
`language`	A short language code (language name)	Controls insights language. Example: `https://www.videoindexer.ai/embed/insights/<accountId>/<videoId>/?language=es-es` or `https://www.videoindexer.ai/embed/insights/<accountId>/<videoId>/?language=spanish`
`locale`	A short language code	Controls the language of the UI. The default value is `en`. Example: `locale=de`.
`tab`	The default selected tab	Controls the Insights tab that's rendered by default. Example: `tab=timeline` renders the insights with the Timeline tab selected.
`search`	String	Allows you to control the initial search term. Example: `https://www.videoindexer.ai/embed/insights/<accountId>/<videoId>/?search=azure` renders the insights filtered by the word "Azure".
`sort`	Strings separated by comma	Allows you to control the sorting of an insight. Each sort consists of 3 values: widget name, property and order, connected with '_' `sort=name_property_order` Available options: widgets: `keywords`, `audioEffects`, `labels`, `sentiments`, `emotions`, `keyframes`, `scenes`, `namedEntities` and `spokenLanguage`. property: `startTime`, `endTime`, `seenDuration`, `name` and `ID`. order: asc and desc. Example: `https://www.videoindexer.ai/embed/insights/<accountId>/<videoId>/?sort=labels_id_asc,keywords_name_desc` renders the labels sorted by ID in ascending order and keywords sorted by name in descending order.
`location`		The `location` parameter must be included in the embedded links, see how to get the name of your region. If your account is in preview, the `trial` should be used for the location value. `trial` is the default value for the `location` parameter.

You can use the Player widget to stream video by using adaptive bit rate. The Player widget supports the following optional URL parameters.

Name	Definition	Description
`t`	Seconds from the start	Makes the player start playing from the specified time point. Example: `t=60`.
`captions`	A language code / A language code array	Fetches the caption in the specified language during the widget loading to be available on the Captions menu. Example: `captions=en-US`, `captions=en-US,es-ES`
`showCaptions`	A Boolean value	Makes the player load with the captions already enabled. Example: `showCaptions=true`.
`type`		Activates an audio player skin (the video part is removed). Example: `type=audio`.
`autoplay`	A Boolean value	Indicates if the player should start playing the video when loaded. The default value is `true`. Example: `autoplay=false`.
`language`/`locale`	A language code	Controls the player language. The default value is `en-US`. Example: `language=de-DE`.
`location`		The `location` parameter must be included in the embedded links, see how to get the name of your region. If your account is in preview, the `trial` should be used for the location value. `trial` is the default value for the `location` parameter.
`boundingBoxes`	Array of bounding boxes. Options: people (faces), observed people and detected objects. Values should be separated by a comma (",").	Controls the option to set bounding boxes on/off when embedding the player. All mentioned option will be turned on. Example: `boundingBoxes=observedPeople,people,detectedObjects` Default value is `boundingBoxes=observedPeople,detectedObjects` (only observed people and detected objects bounding box are turned on).

You can use the Editor widget to create new projects and manage a video's insights. The Editor widget supports the following optional URL parameters.

Name	Definition	Description
`accessToken`^*	String	Provides access to videos that are only in the account that's used to embed the widget. The Editor widget requires the `accessToken` parameter.
`language`	A language code	Controls the player language. The default value is `en-US`. Example: `language=de-DE`.
`locale`	A short language code	Controls the insights language. The default value is `en`. Example: `language=de`.
`location`		The `location` parameter must be included in the embedded links, see how to get the name of your region. If your account is in preview, the `trial` should be used for the location value. `trial` is the default value for the `location` parameter.

^*The owner should provide accessToken with caution.

Embed videos

This section discusses embedding videos by using the website or by assembling the URL manually into apps.

The location parameter must be included in the embedded links, see how to get the name of your region. If your account is in preview, the trial should be used for the location value. trial is the default value for the location parameter. For example: https://www.videoindexer.ai/accounts/00000000-0000-0000-0000-000000000000/videos/b2b2c74b8e/?location=trial.

The website experience

To embed a video, use the website as described below:

Sign in to the Azure AI Video Indexer website.
Select the video that you want to work with and press Play.
Select the type of widget that you want (Insights, Player, or Editor).
Click </> Embed.
Copy the embed code (appears in Copy the embedded code in the Share & Embed dialog).
Add the code to your app.

Note

Sharing a link for the Player or Insights widget will include the access token and grant the read-only permissions to your account.

Assemble the URL manually

Public videos

You can embed public videos assembling the URL as follows:

https://www.videoindexer.ai/embed/[insights | player]/<accountId>/<videoId>

Private videos

To embed a private video, you must pass an access token (use Get Video Access Token in the src attribute of the iframe:

https://www.videoindexer.ai/embed/[insights | player]/<accountId>/<videoId>/?accessToken=<accessToken>

Provide editing insights capabilities

To provide editing insights capabilities in your embedded widget, you must pass an access token that includes editing permissions. Use Get Video Access Token with &allowEdit=true.

Widgets interaction

The Insights widget can interact with a video on your app. This section shows how to achieve this interaction.

Flow overview

When you edit the transcripts, the following flow occurs:

You edit the transcript in the timeline.
Azure AI Video Indexer gets these updates and saves them in the from transcript edits in the language model.
The captions are updated:
- If you are using Azure AI Video Indexer's player widget - it’s automatically updated.
- If you are using an external player - you get a new captions file user the Get video captions call.

Cross-origin communications

To get Azure AI Video Indexer widgets to communicate with other components:

Uses the cross-origin communication HTML5 method postMessage.
Validates the message across VideoIndexer.ai origin.

If you implement your own player code and integrate with Insights widgets, it's your responsibility to validate the origin of the message that comes from VideoIndexer.ai.

Embed widgets in your app or blog (recommended)

This section shows how to achieve interaction between two Azure AI Video Indexer widgets so that when a user selects the insight control on your app, the player jumps to the relevant moment.

Copy the Player widget embed code.
Copy the Insights embed code.
Add the Mediator file to handle the communication between the two widgets:
<script src="https://breakdown.blob.core.windows.net/public/vb.widgets.mediator.js"></script>

Now when a user selects the insight control on your app, the player jumps to the relevant moment.

For more information, see the Azure AI Video Indexer - Embed both Widgets demo.

If you use a video player other than Video Indexer Player, you must manually manipulate the video player to achieve the communication.

Insert your video player.

For example, a standard HTML5 player:

<video id="vid1" width="640" height="360" controls autoplay preload>
   <source src="//breakdown.blob.core.windows.net/public/Microsoft%20HoloLens-%20RoboRaid.mp4" type="video/mp4" /> 
   Your browser does not support the video tag.
</video>

Embed the Insights widget.

Implement communication for your player by listening to the "message" event. For example:

<script>

    (function(){
    // Reference your player instance.
    var playerInstance = document.getElementById('vid1');

    function jumpTo(evt) {
      var origin = evt.origin || evt.originalEvent.origin;

      // Validate that the event comes from the videoindexer domain.
      if ((origin === "https://www.videoindexer.ai") && evt.data.time !== undefined){

        // Call your player's "jumpTo" implementation.
        playerInstance.currentTime = evt.data.time;

        // Confirm the arrival to us.
        if ('postMessage' in window) {
          evt.source.postMessage({confirm: true, time: evt.data.time}, origin);
        }
      }
    }

    // Listen to the message event.
    window.addEventListener("message", jumpTo, false);

    }())    

</script>

For more information, see the Video Indexer Player + VI Insights demo.

Customizing embeddable widgets

You can choose the types of insights that you want. To do this, specify them as a value to the following URL parameter that's added to the embed code that you get (from the API or from the Azure AI Video Indexer website): &widgets=<list of wanted widgets>.

The possible values are listed here.

For example, if you want to embed a widget that contains only people and keywords insights, the iframe embed URL will look like this:

https://www.videoindexer.ai/embed/insights/<accountId>/<videoId>/?widgets=people,keywords

The title of the iframe window can also be customized by providing &title=<YourTitle> to the iframe URL. (It customizes the HTML <title> value).

For example, if you want to give your iframe window the title "MyInsights", the URL will look like this:

https://www.videoindexer.ai/embed/insights/<accountId>/<videoId>/?title=MyInsights

Notice that this option is relevant only in cases when you need to open the insights in a new window.

If you embed Azure AI Video Indexer player, you can choose the size of the player by specifying the size of the iframe.

For example:

> [!VIDEO https://www.videoindexer.ai/embed/player/<accountId>/<videoId>/]>/<videoId>/" frameborder="0" allowfullscreen />

By default, Azure AI Video Indexer player has autogenerated closed captions that are based on the transcript of the video. The transcript is extracted from the video with the source language that was selected when the video was uploaded.

If you want to embed with a different language, you can add &captions=<Language Code> to the embed player URL. If you want the captions to be displayed by default, you can pass &showCaptions=true.

The embed URL then will look like this:

https://www.videoindexer.ai/embed/player/<accountId>/<videoId>/?captions=en-us

Autoplay

By default, the player will start playing the video. you can choose not to by passing &autoplay=false to the preceding embed URL.

Code samples

See the code samples repo that contains samples for Azure AI Video Indexer API and widgets:

File/folder	Description
`control-vi-embedded-player`	Embed VI Player and control it from outside.
`custom-index-location`	Embed VI Insights from a custom external location (can be customer a blob).
`embed-both-insights`	Basic usage of VI Insights both player and insights.
`customize-the-widgets`	Embed VI widgets with customized options.
`embed-both-widgets`	Embed VI Player and Insights and communicate between them.
`url-generator`	Generates widgets custom embed URL based on user-specified options.
`html5-player`	Embed VI Insights with a default HTML5 video player.

Supported browsers

For more information, see supported browsers.

Embed and customize Azure AI Video Indexer widgets in your app using npmjs package

Using our @azure/video-analyzer-for-media-widgets package, you can add the insights widgets to your app and customize it according to your needs.

Instead of adding an iframe element to embed the insights widget, with this new package you can easily embed & communicate between our widgets. Customizing your widget is only supported in this package - all in one place.

For more information, see our official GitHub.

For information about how to view and edit Azure AI Video Indexer insights, see View and edit Azure AI Video Indexer insights.

Also, check out Azure AI Video Indexer CodePen.

Embed Azure AI Video Indexer widgets in your apps

Widget types

Insights widget

Player widget

Editor widget

Embed videos

The website experience

Assemble the URL manually

Public videos

Private videos

Provide editing insights capabilities

Widgets interaction

Flow overview

Cross-origin communications

Embed widgets in your app or blog (recommended)

Embed the Azure AI Video Indexer Insights widget and use a different video player

Customizing embeddable widgets

Insights widget

Player widget

Autoplay

Code samples

Supported browsers

Embed and customize Azure AI Video Indexer widgets in your app using npmjs package

Feedback

Feedback

Additional resources

Embed Azure AI Video Indexer widgets in your apps

Widget types

Insights widget

Player widget

Editor widget

Embed videos

The website experience

Assemble the URL manually

Public videos

Private videos

Provide editing insights capabilities

Widgets interaction

Flow overview

Cross-origin communications

Embed widgets in your app or blog (recommended)

Embed the Azure AI Video Indexer Insights widget and use a different video player

Customizing embeddable widgets

Insights widget

Player widget

Autoplay

Code samples

Supported browsers

Embed and customize Azure AI Video Indexer widgets in your app using npmjs package

Related content

Feedback

Feedback

Additional resources