Translated Captions for Silverlight Videos – It is Dynamic!

This post provides details and code about how to create dynamically translating video captions based on the browser language. By implementing the details in this post you can create the following:

  • A Web based solution that displays machine translated captions for your video
    • A Web page that translates captions on the fly
    • A Silverlight player that displays translated captions
    • A custom Web helper for WebMatrix to make this solution easy to implement and modify.

In this post I’ll show how to create the Silverlight player and how to use the browser language (en = English, ja = Japanese, fr = French, etc.) and the Microsoft Translation API to translate the captions shown in the video player. The code creates a new xml file for each language (caption.ja.xml,, etc.) so that the translation for a specific language is only done once. This way, the Microsoft translator API doesn’t get called more than needed.


While I’ve been on the ASP.NET User Education team at Microsoft I have had the opportunity to work with more than just ASP.NET and Visual Studio tools. I’ve had the chance to work with all sorts of technologies, including Silverlight. Also, I’ve had the chance to create videos and work with Microsoft Translation services. So it occurred to me to create a method to translate the video captions to reach a bigger audience.

After researching a bit, I ended up using the following technologies to create dynamically translated video captions:

Microsoft Silverlight Media Framework 2.0 (SMFv2)

Microsoft Translator API

Visual Studio 2010

WebMatrix (Beta)

Microsoft Expression Encoder 4

I used the Silverlight Media Framework 2.0 because it included support for a nice video player that supports captions and it supports the latest video caption file standard (.dfxp) - W3C’s Timed Text Markup Language (TTML) 1.0. This means I could point the player to a standard xml based caption file and easily display captions. Nice! The Microsoft Translator API gave me a number of ways to perform translations. Good examples too. I used Visual Studio 2010 because I like the debugging experience and it’s a great environment to create Silverlight apps. Web Matrix is a new tool my team is working on. I think the WebMatrix helpers are especially cool! Check out my other blog posts for more Web Matrix helpers. Also, Expression Encoder 4 supports the new captioning file type (.dfxp) that is also supported by the Silverlight Media Framework.

Okay, so how does it all work? Here’s how.

First, I created a Silverlight app in Visual Studio. I modified the MainPage.xaml file with the following XAML as follows:

Code Snippet

  1. <UserControl x:Class="CCtest001.MainPage"
  2.     xmlns=""
  3.     xmlns:x=""
  4.     xmlns:d=""
  5.     xmlns:mc=""
  6.     xmlns:Core="clr-namespace:Microsoft.SilverlightMediaFramework.Core;assembly=Microsoft.SilverlightMediaFramework.Core"
  7.     xmlns:Media="clr-namespace:Microsoft.SilverlightMediaFramework.Core.Media;assembly=Microsoft.SilverlightMediaFramework.Core"
  8.     mc:Ignorable="d"
  9.     d:DesignHeight="300" d:DesignWidth="400">
  11.     <Grid x:Name="LayoutRoot" Background="White">
  12.         <Core:SMFPlayer x:Name="myCore" AutoPlay="True" AutoLoad="True" CaptionsVisibility="Visible">
  13.         </Core:SMFPlayer>
  14.     </Grid>
  15. </UserControl>


Next, I add the following Silverlight Media Framework assembly references to my Silverlight project after installing the Silverlight Media Framework. The following image shows the references I added to the project:


Note: I added all the SMF assemblies to my proto-type. I could reduce this down to the assemblies needed for only captioning.

Then, I modified the code-behind file named App.xaml.cs so that the Application_Startup method could read parameters that are passed from a Web page to the Silverlight app (.xap). Here’s the Application_Startup method:

Code Snippet

  1. private void Application_Startup(object sender, StartupEventArgs e)
  2. {
  3.     this.RootVisual = new MainPage();
  5.     if (e.InitParams != null)
  6.     {
  7.         foreach (var data in e.InitParams)
  8.         {
  9.             this.Resources.Add(data.Key, data.Value);
  10.         }
  11.     }
  12. }

The parameters that I will pass from the Web page to the Siverlight .xap file include the video file that I want to play and the caption file that I want to show.

After that, I modified the code-behind (MainPage.xaml.cs) file so that I could read the passed in parameters and display the video and captions:

Code Snippet

  1. using System;
  2. using System.Linq;
  3. using System.Collections.Generic;
  4. using System.Net;
  5. using System.Windows;
  6. using System.Windows.Controls;
  7. using System.Windows.Documents;
  8. using System.Windows.Input;
  9. using System.Windows.Media;
  10. using System.Windows.Media.Animation;
  11. using System.Windows.Shapes;
  12. using System.Globalization;
  13. using Microsoft.SilverlightMediaFramework.Core;
  14. using Microsoft.SilverlightMediaFramework.Plugins;
  15. using Microsoft.SilverlightMediaFramework.Utilities;
  16. using Microsoft.SilverlightMediaFramework.Core.Media;
  18. namespace CCtest001
  19. {
  20.     public partial class MainPage : UserControl
  21.     {
  22.         public MainPage()
  23.         {
  24.             InitializeComponent();
  25.             this.Loaded += new RoutedEventHandler(Page_Loaded);
  26.         }
  28.         void Page_Loaded(object sender, RoutedEventArgs e)
  29.         {
  30.             string mediaFilePath = "";
  31.             string captionFilePath = "";
  33.             if (App.Current.Resources.Count > 0)
  34.             {
  35.                 foreach (var key in App.Current.Resources.Keys)
  36.                 {
  37.                     switch(key.ToString())
  38.                     {
  39.                         case "mediaFilePath":
  40.                             mediaFilePath = App.Current.Resources["mediaFilePath"].ToString();
  41.                             break;
  42.                         case "captionFilePath":
  43.                             captionFilePath = App.Current.Resources["captionFilePath"].ToString();
  44.                             break;
  45.                     }
  46.                 }
  47.             }
  49.             // Video location.
  50.             System.Uri myMediaFile = new System.Uri(mediaFilePath, UriKind.Relative);
  52.             // Create PlayListItem.
  53.             PlaylistItem myPlayList = new PlaylistItem();
  54.             myPlayList.MediaSource = myMediaFile;
  56.             // Caption file location.
  57.             System.Uri myCaptionFile = new System.Uri(captionFilePath, UriKind.Relative);
  59.             // Set caption format and file.
  60.             MarkerResource myMarkerResource = new MarkerResource();
  61.             myMarkerResource.Format = "TTAF1-DFXP";
  62.             myMarkerResource.Source = myCaptionFile;
  64.             // Set the MarkerResource for the PlayList
  65.             myPlayList.MarkerResource = myMarkerResource;
  67.             // Add the Playlist to the Player
  68.             myCore.Playlist.Add(myPlayList);
  69.         }
  70.     }
  71. }

To test this in Visual Studio, I modified the existing page named CCtest001TestPage.aspx as follows:

Code Snippet

  1. <body>
  2.     <form id="form1" runat="server" style="height:100%">
  3.     <div id="silverlightControlHost">
  4.         <object id="SLObj" data="data:application/x-silverlight-2," type="application/x-silverlight-2" width="100%" height="100%">
  5.           <param name="source" value="ClientBin/CCtest001.xap"/>
  6.           <param name="onError" value="onSilverlightError" />
  7.           <param name="background" value="white" />
  8.           <param name="minRuntimeVersion" value="4.0.50401.0" />
  9.           <param name="autoUpgrade" value="true" />
  10.           <param name="initParams" value="mediaFilePath=Walkthrough-DynamicDataScaffolding.wmv,captionFilePath=ExampleCaptions.xml" />
  11.           <a href="" style="text-decoration:none">
  12.               <img src="" alt="Get Microsoft Silverlight" style="border-style:none"/>
  13.           </a>
  14.         </object><iframe id="_sl_historyFrame" style="visibility:hidden;height:0px;width:0px;border:0px"></iframe></div>
  15.     </form>
  16. </body>

I also added a video file called Walkthrough-DynamicDataScaffolding.wmv and a caption file name ExampleCaptions.xml to the ClientBin folder of the Web project. I used the ClientBin folder because this is where Visual Studio builds the Silverlight .xap file.

The caption file contained the following xml:

Code Snippet

  1. <?xml version="1.0" encoding="utf-8"?>
  2. <tt xml:lang="en-US" xmlns="">
  3. <head>
  4.    <metadata>
  5.      <ttm:title xmlns:ttm="" />
  6.      <ttm:desc xmlns:ttm="" />
  7.      <ttm:copyright xmlns:ttm="" />
  8.    </metadata>
  9.    <styling>
  10.      <style xml:id="backgroundStyle" p5:fontFamily="proportionalSansSerif" p5:fontSize="16px" p5:textAlign="center" p5:origin="10% 85%" p5:extent="80% 10%" p5:backgroundColor="rgba(0,0,0,100)" p5:displayAlign="center" xmlns:p5="" />
  11.      <style xml:id="speakerStyle" p5:style="backgroundStyle" p6:color="white" p6:backgroundColor="transparent" xmlns:p6="" xmlns:p5="" />
  12.    </styling>
  13.    <layout>
  14.      <region p4:begin="00:00:00:00" p4:end="00:01:00:00" xmlns:p4="" xml:id="speaker" p5:style="speakerStyle" p6:zIndex="1" xmlns:p6="" xmlns:p5="" />
  15.      <region p4:begin="00:00:00:00" p4:end="00:01:00:00" xmlns:p4="" xml:id="background" p5:style="backgroundStyle" p6:zIndex="0" xmlns:p6="" xmlns:p5="" />
  16.    </layout>
  17. </head>
  18. <body>
  19.    <div>
  20.      <p p4:region="speaker" p4:begin="00:00:00:20" p4:end="00:00:16:21" xmlns:p4="">This video is based on the topic titled, Walkthrough: Creating a New Dynamic Dat Web Site Using Scaffolding.</p>
  21.      <p p4:region="speaker" p4:begin="00:00:17:21" p4:end="00:00:35:25" xmlns:p4="">Scaffolding is mechanism that takes the power and functionality of the existing ASP.NET page framework and enhances it by dynamically displaying pages based on the data model without a physical page that exists behind the scenes.</p>
  22.      <p p4:region="speaker" p4:begin="00:00:36:25" p4:end="00:00:40:13" xmlns:p4="">In this video, you will learn how to:</p>
  23.      <p p4:region="speaker" p4:begin="00:00:40:13" p4:end="00:00:42:02" xmlns:p4="">Create a Dynamic Data Web site.</p>
  24.      <p p4:region="speaker" p4:begin="00:00:42:02" p4:end="00:00:44:02" xmlns:p4="">Add data to the Web site.</p>
  25.      <p p4:region="speaker" p4:begin="00:00:44:03" p4:end="00:00:47:06" xmlns:p4="">Register the data context.</p>
  26.      <p p4:region="speaker" p4:begin="00:00:47:06" p4:end="00:00:58:21" xmlns:p4="">Text the Dynamic Data Web site.</p>
  27.      <p p4:region="speaker" p4:begin="00:00:58:21" p4:end="00:01:02:02" xmlns:p4="">In the File menu, click New Web Site.</p>
  28.      <p p4:region="speaker" p4:begin="00:01:02:02" p4:end="00:01:03:10" xmlns:p4="">The New Web Site dialog box is displayed. </p>
  29.      <p p4:region="speaker" p4:begin="00:01:03:10" p4:end="00:01:08:13" xmlns:p4="">select ASP.NET Dynamic Data Entities Web Site.</p>
  30.      <p p4:region="speaker" p4:begin="00:01:08:13" p4:end="00:01:18:13" xmlns:p4="">Under Installed Templates, in the left pane, select Visual Basic or Visual C Sharp.</p>
  31.    </div>
  32. </body>
  33. </tt>

When you run the test page (CCtest001TestPage.aspx) in the browser, you'll see the video and captions appear.


Great! Now let's add in dynamic caption translation and make a WebMatrix helper.

In WebMatrix, I created a new project and added the files. The following image shows the added files:


Note: I must have changed the video file name, but you don’t need too.

Next, I added a class file named MediaPlayerCC.cs to the Web app and placed it in the App_Code folder.

In the MediaPlayerCC.cs file I added the following C# code:

Code Snippet

using System;
using System.Collections.Generic;
using System.Text;
using System.Xml;
using System.IO;
using System.Net;
using System.Web;
using System.Web.UI;

/// <summary>WebMatrix Silverlight Caption Helper</summary>
/// <param name="xapFilePath" type="string">The path and file name of the Silverlight .xap file.</param>
/// <param name="mediaFilePath" type="string">The path and file name of the media file. The file must be in the same folder as the .xap file.</param>
/// <param name="captionFilePath" type="string">The path and file name of the source caption file. The file must be in the same folder as the .xap file.</param>
/// <param name="sourceLanguage" type="string">A two letter source language value.</param>
/// <param name="newLanguage" type="string">A two letter new language value.</param>
/// <param name="translationAppID" type="string">ID obtained from Microsoft Translator Web site.</param>
/// <param name="width" type="string">(Optional) The width of the Silverlight player. Default = 720.</param>
/// <param name="height" type="string">(Optional) The height of the Silverlight player. Default = 480.</param>

public static class MediaPlayerCC
static string AppId = "";

static Page myPage = new Page();
static System.Web.HttpServerUtility myServer = myPage.Server;

public static IHtmlString GetHtml(
string xapFilePath,
string mediaFilePath,
string captionFilePath,
string sourceLanguage,
string newLanguage,
string translationAppID,
string width = "640",
string height = "480")

// Call the steps need to convert caption file.

// Set the translation appId which was obtained from Microsoft Translator Web site.
AppId = translationAppID;

// Create the new language-specific caption file name.
char[] delimiterChar = {'.'};
string[] subFileName = captionFilePath.Split(delimiterChar);
string newFileName = subFileName[0] + "." + newLanguage + ".xml";

// Translate the caption file if it does not exist. However, if it does exist,
// use the existing translation file.
//if (!Cache[newFileName]) {
if (!File.Exists(myServer.MapPath(newFileName))) {
// Copy the xml file.
string copySuccess = copyXMLFile(captionFilePath, newFileName, newLanguage);

// Create List<string> of captions.
List<string> captionList = getTextFromFile(captionFilePath);

// Translate captions.
List<string> translatedCaptionList = translateCaptions(captionList, sourceLanguage, newLanguage);

// Replace translated captions in the copied file with translated captions.
replaceTranslatedCaptions(newFileName, translatedCaptionList);

// Generate the HTML
string html = GenerateCode(xapFilePath, mediaFilePath, newFileName, width, height);
return new HtmlString(html);

private static string GenerateCode(
string path,
string mediaFilePath,
string captionFilePath,
string width,
string height)

if (string.IsNullOrEmpty(path))
throw new ArgumentException("The xapFilePath parameter cannot be null or empty.", "xapFilePath");

if (string.IsNullOrEmpty(mediaFilePath))
throw new ArgumentException("The mediaFilePath parameter cannot be null or empty.", "mediaFilePath");

// Media file and captions file must be in same folder as the .xap file.
char[] delimiterChar = { '/' };
string[] mediaFile = mediaFilePath.Split(delimiterChar);
mediaFilePath = mediaFile[mediaFile.Length - 1];
string[] captionFile = captionFilePath.Split(delimiterChar);
captionFilePath = captionFile[mediaFile.Length - 1];

StringBuilder builder = new StringBuilder();
builder.AppendLine("<object ");
builder.Append(string.Format("width = '{0}'", width));
builder.Append(string.Format("height = '{0}'", height));
builder.Append("type='application/x-silverlight-2' data='data:application/x-silverlight-2,' >");
builder.AppendLine(" <param name='source'");
builder.Append(string.Format("value = '{0}' />", path));
builder.Append("<param name='initParams'");
builder.Append(string.Format("value = '{0}' />", "mediaFilePath=" + mediaFilePath + ",captionFilePath=" + captionFilePath));

return builder.ToString();

// Copy the xml file.
static string copyXMLFile(string fileName, string newFileName, string lang)
string newFileNamePath = newFileName.Replace(fileName, newFileName);
File.Copy(myServer.MapPath(fileName), myServer.MapPath(newFileNamePath), true);
return "success";

// Get strings to translate.
static List<string> getTextFromFile(string fileName)
List<string> captionArray = new List<string>();
XmlTextReader reader = new XmlTextReader(myServer.MapPath(fileName));
while (reader.Read())
if (reader.NodeType == XmlNodeType.Text)
//Add the text.

return captionArray;

static List<string> translateCaptions(List<string> captionList, string sourceLanguage, string newLanguage)
List<string> translatedCaptionArray = new List<string>();
string myTranslation;

// Translate and create a string
string myTranslatedText = translateWork(captionList, sourceLanguage, newLanguage);

// Parse XML
XmlDocument docXML = new XmlDocument();
XmlNodeList nodeList = docXML.GetElementsByTagName("TranslateArrayResponse");

// Search through responses
foreach (XmlNode response in nodeList)
XmlNodeList nodeItem = response.ChildNodes;
// Search throught response items
foreach (XmlNode responseItem in nodeItem)
if (responseItem.Name == "TranslatedText")
myTranslation = responseItem.InnerText.ToString();


return translatedCaptionArray;

static void replaceTranslatedCaptions(string newFileName, List<string> translatedCaptionList)
int i = 0;
XmlDocument xmlDoc = new XmlDocument();
XmlNodeList xmlNodeList = xmlDoc.GetElementsByTagName("p");

foreach (XmlNode xmlNode in xmlNodeList)

foreach (XmlNode xmlChild in xmlNode.ChildNodes)
xmlChild.InnerText = translatedCaptionList[i].ToString();
//"Error replacing captions in loc file.";

//Cache[newFileName] = xmlDoc;


static string translateWork(List<string> textToTranslate, string sourceLang, string newLang)

string appId = AppId;
string languageFrom = sourceLang;
string languageTo = newLang;

List<string> textArray = textToTranslate;
string uri = "" + appId;
// Note: The request body is a xml string generated according to the schema specified at

StringWriter swriter = new StringWriter();
XmlTextWriter xwriter = new XmlTextWriter(swriter);





xwriter.WriteAttributeString("xmlns", "");

xwriter.WriteAttributeString("xmlns", "");

xwriter.WriteAttributeString("xmlns", "");

xwriter.WriteAttributeString("xmlns", "");

xwriter.WriteAttributeString("xmlns", "");

xwriter.WriteAttributeString("xmlns", "");


foreach (string text in textArray)
xwriter.WriteAttributeString("xmlns", "");




// create the request
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(uri);
req.ContentType = "text/xml";
req.Method = "POST";
System.IO.Stream stream = req.GetRequestStream();
byte[] arrBytes = System.Text.Encoding.UTF8.GetBytes(swriter.ToString());
stream.Write(arrBytes, 0, arrBytes.Length);

// get the response
WebResponse resp = req.GetResponse();
Stream respStream = resp.GetResponseStream();
StreamReader rdr = new StreamReader(respStream, System.Text.Encoding.UTF8);
string strResponse = rdr.ReadToEnd();

return strResponse;

static string getResponse(HttpWebResponse response)
string myString = "";
// Load the response into an XmlDocument.
XmlDocument document = new XmlDocument();

// Add the default namespace to the namespace manager.
XmlNamespaceManager nsmgr = new XmlNamespaceManager(

XmlNodeList errors = document.DocumentElement.SelectNodes(

if (errors.Count > 0)
// There are errors in the response. Display error details.
return DisplayErrors(errors);
// There were no errors in the response. Display the
// Translation results.
myString = getResults(document.DocumentElement, nsmgr);
return myString.ToString();

static string getResults(XmlNode root, XmlNamespaceManager nsmgr)
string version = root.SelectSingleNode("./@Version", nsmgr).InnerText;
string searchTerms = root.SelectSingleNode(

// Add the Translation SourceType namespace to the namespace manager.

XmlNodeList results = root.SelectNodes(

StringBuilder myString = new StringBuilder();

// Get the Translation results.
foreach (XmlNode result in results)

return myString.ToString();

static string DisplayErrors(XmlNodeList errors)
// Iterate over the list of errors and display error details.
foreach (XmlNode error in errors)
foreach (XmlNode detail in error.ChildNodes)
// Ignore caption translation errors for now.
// Default to source language captions.
string captionError = detail.Name + ": " + detail.InnerText;
return captionError;
return "DisplayError";

static string DisplayTextError(string error)

// Ignore caption translation errors for now.
// Default to source language captions.
string captionError = "Error occurred: " + error;
return captionError;

The above code copies the caption file and translates the captions using the Microsoft Translator API. I added a lot of comments. I hope those are helpful.

Next, I modified the page.cshtml file to call the helper using the following code.

Code Snippet

  1. @{
  2. var lcid = Request.UserLanguages[0];
  3. var browserLanguage = lcid.Substring(0,2);
  4. }
  6. <!DOCTYPE html>
  7. <html>
  8.     <head>
  9.         <title>WebMatrix Silverlight Caption Helper</title>
  10.     </head>
  11.     <body>
  12.     @MediaPlayerCC.GetHtml(
  13.         xapFilePath: "media/MediaPlayerCC.xap",
  14.         mediaFilePath: "media/WalkthroughDynamicDataScaffolding.wmv",
  15.         captionFilePath: "media/ExampleCaptions.xml",
  16.         sourceLanguage: "en",
  17.         newLanguage: browserLanguage,
  18.         translationAppID: "[Add your AppId here]",
  19.         width: "100%",
  20.         height: "800"
  22.     )
  23.     </body>
  24. </html>

Here, I first determine the browser language, then call the helper. In the helper call, I specify the path to a Silverlight .xap file. Then, I specify the media file, the caption file and the new language based on the browser language setting. I set it up so the .xap file, the media file and the caption file all belong in the same folder. You must also include YOUR translator AppId, see Get Your AppID for more information. The width and height are optional.

And there it is, dynamically translated captions. You can use Microsoft Expression Coder to help create the caption file for the video. During my proto-typing process for this solution I saved the caption file as a .dfxp file and renamed it to .xml. I also added in special attributes to each caption line to support the SMF, as you can see in the above caption file.

By including dynamically translated captions you will be able to increase the traffic to your videos.

I hope you find this helpful!


-- Erik Reitan
ASP.NET User Education
This posting is provided "AS IS" with no warranties, and confers no rights.