Use Azure Media Video Thumbnails to Create a Video Summarization

Overview

The Azure Media Video Thumbnails media processor (MP) enables you to create a summary of a video that is useful to customers who just want to preview a summary of a long video. For example, customers might want to see a short "summary video" when they hover over a thumbnail. By tweaking the parameters of Azure Media Video Thumbnails through a configuration preset, you can use the MP's powerful shot detection and concatenation technology to algorithmically generate a descriptive subclip.

The Azure Media Video Thumbnail MP is currently in Preview.

This topic gives details about Azure Media Video Thumbnail and shows how to use it with Media Services SDK for .NET.

Limitations

In some cases, if your video is not comprised of different scenes, the output will only be a single shot.

Video summary example

Here are some examples of what the Azure Media Video Thumbnails media processor can do:

Original video

Original video

Video thumbnail result

Video thumbnail result

Task configuration (preset)

When creating a video thumbnail task with Azure Media Video Thumbnails, you must specify a configuration preset. The above thumbnail sample was created with the following basic JSON configuration:

{"version":"1.0"}

Currently, you can change the following parameters:

Param Description
outputAudio Specifies whether or not the resultant video contains any audio.
Allowed values are: True or False. Default is True.
fadeInFadeOut Specifies whether or not fade transitions are used between the separate motion thumbnails.
Allowed values are: True or False. Default is True.
maxMotionThumbnailDurationInSecs Integer that specifies how long the entire resultant video shall be. Default depends on original video duration.

The following table describes the default duration, when maxMotionThumbnailInSecs is not used.

Video duration d < 3 min 3 min < d < 15 min
Thumbnail duration 15 sec (2-3 scenes) 30 sec (3-5 scenes)

The following JSON sets available parameters.

{
    "version": "1.0",
    "options": {
        "outputAudio": "true",
        "maxMotionThumbnailDurationInSecs": "10",
        "fadeInFadeOut": "true"
    }
}

Sample code

The following program shows how to:

  1. Create an asset and upload a media file into the asset.
  2. Creates a job with a video thumbnail task based on a configuration file that contains the following json preset.

     {                
         "version": "1.0",
         "options": {
             "outputAudio": "true",
             "maxMotionThumbnailDurationInSecs": "30",
             "fadeInFadeOut": "false"
         }
     }
    
  3. Downloads the output files.

.NET code

using System;
using System.Configuration;
using System.IO;
using System.Linq;
using Microsoft.WindowsAzure.MediaServices.Client;
using System.Threading;
using System.Threading.Tasks;

namespace VideoSummarization
{
    class Program
    {
        // Read values from the App.config file.
        private static readonly string _mediaServicesAccountName =
            ConfigurationManager.AppSettings["MediaServicesAccountName"];
        private static readonly string _mediaServicesAccountKey =
            ConfigurationManager.AppSettings["MediaServicesAccountKey"];

        // Field for service context.
        private static CloudMediaContext _context = null;
        private static MediaServicesCredentials _cachedCredentials = null;

        static void Main(string[] args)
        {

            // Create and cache the Media Services credentials in a static class variable.
            _cachedCredentials = new MediaServicesCredentials(
                            _mediaServicesAccountName,
                            _mediaServicesAccountKey);
            // Used the cached credentials to create CloudMediaContext.
            _context = new CloudMediaContext(_cachedCredentials);


            // Run the thumbnail job.
            var asset = RunVideoThumbnailJob(@"C:\supportFiles\VideoThumbnail\BigBuckBunny.mp4",
                                        @"C:\supportFiles\VideoThumbnail\config.json");

            // Download the job output asset.
            DownloadAsset(asset, @"C:\supportFiles\VideoThumbnail\Output");
        }

        static IAsset RunVideoThumbnailJob(string inputMediaFilePath, string configurationFile)
        {
            // Create an asset and upload the input media file to storage.
            IAsset asset = CreateAssetAndUploadSingleFile(inputMediaFilePath,
                "My Video Thumbnail Input Asset",
                AssetCreationOptions.None);

            // Declare a new job.
            IJob job = _context.Jobs.Create("My Video Thumbnail Job");

            // Get a reference to Azure Media Video Thumbnails.
            string MediaProcessorName = "Azure Media Video Thumbnails";

            var processor = GetLatestMediaProcessorByName(MediaProcessorName);

            // Read configuration from the specified file.
            string configuration = File.ReadAllText(configurationFile);

            // Create a task with the encoding details, using a string preset.
            ITask task = job.Tasks.AddNew("My Video Thumbnail Task",
                processor,
                configuration,
                TaskOptions.None);

            // Specify the input asset.
            task.InputAssets.Add(asset);

            // Add an output asset to contain the results of the job.
            task.OutputAssets.AddNew("My Video Thumbnail Output Asset", AssetCreationOptions.None);

            // Use the following event handler to check job progress.  
            job.StateChanged += new EventHandler<JobStateChangedEventArgs>(StateChanged);

            // Launch the job.
            job.Submit();

            // Check job execution and wait for job to finish.
            Task progressJobTask = job.GetExecutionProgressTask(CancellationToken.None);

            progressJobTask.Wait();

            // If job state is Error, the event handling
            // method for job progress should log errors.  Here we check
            // for error state and exit if needed.
            if (job.State == JobState.Error)
            {
                ErrorDetail error = job.Tasks.First().ErrorDetails.First();
                Console.WriteLine(string.Format("Error: {0}. {1}",
                                                error.Code,
                                                error.Message));
                return null;
            }

            return job.OutputMediaAssets[0];
        }

        static IAsset CreateAssetAndUploadSingleFile(string filePath, string assetName, AssetCreationOptions options)
        {
            IAsset asset = _context.Assets.Create(assetName, options);

            var assetFile = asset.AssetFiles.Create(Path.GetFileName(filePath));
            assetFile.Upload(filePath);

            return asset;
        }

        static void DownloadAsset(IAsset asset, string outputDirectory)
        {
            foreach (IAssetFile file in asset.AssetFiles)
            {
                file.Download(Path.Combine(outputDirectory, file.Name));
            }
        }

        static IMediaProcessor GetLatestMediaProcessorByName(string mediaProcessorName)
        {
            var processor = _context.MediaProcessors
                .Where(p => p.Name == mediaProcessorName)
                .ToList()
                .OrderBy(p => new Version(p.Version))
                .LastOrDefault();

            if (processor == null)
                throw new ArgumentException(string.Format("Unknown media processor",
                                                           mediaProcessorName));

            return processor;
        }

        static private void StateChanged(object sender, JobStateChangedEventArgs e)
        {
            Console.WriteLine("Job state changed event:");
            Console.WriteLine("  Previous state: " + e.PreviousState);
            Console.WriteLine("  Current state: " + e.CurrentState);

            switch (e.CurrentState)
            {
                case JobState.Finished:
                    Console.WriteLine();
                    Console.WriteLine("Job is finished.");
                    Console.WriteLine();
                    break;
                case JobState.Canceling:
                case JobState.Queued:
                case JobState.Scheduled:
                case JobState.Processing:
                    Console.WriteLine("Please wait...\n");
                    break;
                case JobState.Canceled:
                case JobState.Error:
                    // Cast sender as a job.
                    IJob job = (IJob)sender;
                    // Display or log error details as needed.
                    // LogJobStop(job.Id);
                    break;
                default:
                    break;
            }
        }

    }
}

Video thumbnail output

Video thumbnail output

Media Services learning paths

You can view Azure Media Services learning paths here:

Provide feedback

Use the User Voice forum to provide feedback and make suggestions on how to improve Azure Media Services. You can also go directly to one of the following categories:

Azure Media Services Analytics Overview

Azure Media Analytics demos