媒体合成和编辑Media compositions and editing

本文向你介绍如何使用 Windows.Media.Editing 命名空间中的 API 来快速开发应用,从而使用户从音频和视频源文件创建媒体合成。This article shows you how to use the APIs in the Windows.Media.Editing namespace to quickly develop apps that enable the users to create media compositions from audio and video source files. 框架的功能包括以编程方式同时追加多个视频剪辑、添加视频和图像覆盖、添加后台音频,以及同时应用音频和视频效果。Features of the framework include the ability to programmatically append multiple video clips together, add video and image overlays, add background audio, and apply both audio and video effects. 创建媒体合成后,可在平面媒体文件中进行呈现以供播放或共享,但合成还可通过磁盘进行序列化和反序列化,从而允许用户加载并修改之前创建的合成。Once created, media compositions can be rendered into a flat media file for playback or sharing, but compositions can also be serialized to and deserialized from disk, allowing the user to load and modify compositions that they have previously created. 这一完整功能将在易于使用的 Windows 运行时接口中提供,与低级别 Microsoft Media Foundation API 相比,它可大大减少执行这些任务所需的代码数量和复杂性。All of this functionality is provided in an easy-to-use Windows Runtime interface that dramatically reduces the amount and complexity of code required to perform these tasks when compared to the low-level Microsoft Media Foundation API.

创建新的媒体合成Create a new media composition

MediaComposition 类是包含组成合成的所有媒体剪辑的容器,用于负责呈现最终合成、将合成加载并保存到光盘,以及提供合成的预览流,以便用户可以在 UI 中查看它。The MediaComposition class is the container for all of the media clips that make up the composition and is responsible for rendering the final composition, loading and saving compositions to disc, and providing a preview stream of the composition so that the user can view it in the UI. 若要在你的应用中使用 MediaComposition,需包括 Windows.Media.Editing 命名空间以及提供你将需要的相关 API 的 Windows.Media.Core 命名空间。To use MediaComposition in your app, include the Windows.Media.Editing namespace as well as the Windows.Media.Core namespace that provides related APIs that you will need.

using Windows.Media.Editing;
using Windows.Media.Core;
using Windows.Media.Playback;
using System.Threading.Tasks;

在你的代码中可通过多个点访问 MediaComposition 对象,因此,你通常将声明一个要在其中存储该对象的成员变量。The MediaComposition object will be accessed from multiple points in your code, so typically you will declare a member variable in which to store it.

private MediaComposition composition;

MediaComposition 的构造函数不采用任何参数。The constructor for MediaComposition takes no arguments.

composition = new MediaComposition();

将媒体剪辑添加到合成Add media clips to a composition

媒体合成通常包含一个或多个视频剪辑。Media compositions typically contain one or more video clips. 你可以使用 FileOpenPicker 以允许用户选择某个视频文件。You can use a FileOpenPicker to allow the user to select a video file. 选择文件后,通过调用 MediaClip.CreateFromFileAsync 创建新的 MediaClip 对象以包含视频剪辑。Once the file has been selected, create a new MediaClip object to contain the video clip by calling MediaClip.CreateFromFileAsync. 然后,将该剪辑添加到 MediaComposition 对象的 Clips 列表。Then you add the clip to the MediaComposition object's Clips list.

private async Task PickFileAndAddClip()
{
    var picker = new Windows.Storage.Pickers.FileOpenPicker();
    picker.SuggestedStartLocation = Windows.Storage.Pickers.PickerLocationId.VideosLibrary;
    picker.FileTypeFilter.Add(".mp4");
    Windows.Storage.StorageFile pickedFile = await picker.PickSingleFileAsync();
    if (pickedFile == null)
    {
        ShowErrorMessage("File picking cancelled");
        return;
    }

    // These files could be picked from a location that we won't have access to later
    var storageItemAccessList = Windows.Storage.AccessCache.StorageApplicationPermissions.FutureAccessList;
    storageItemAccessList.Add(pickedFile);

    var clip = await MediaClip.CreateFromFileAsync(pickedFile);
    composition.Clips.Add(clip);

}
  • 媒体剪辑按照它们在 Clips 列表中显示的相同顺序显示在 MediaComposition 中。Media clips appear in the MediaComposition in the same order as they appear in Clips list.

  • MediaClip 只能包含在合成中一次。A MediaClip can only be included in a composition once. 尝试添加已由合成使用的 MediaClip 将导致错误。Attempting to add a MediaClip that is already being used by the composition will result in an error. 若要在合成中多次重复使用视频剪辑,可调用 Clone 以创建新的 MediaClip 对象,这些对象随后将添加到合成中。To reuse a video clip multiple times in a composition, call Clone to create new MediaClip objects which can then be added to the composition.

  • 通用 Windows 应用没有访问整个文件系统的权限。Universal Windows apps do not have permission to access the entire file system. StorageApplicationPermissions 类的 FutureAccessList 属性允许你的应用存储已由用户选择的文件的记录,以便你可以保留访问该文件的权限。The FutureAccessList property of the StorageApplicationPermissions class allows your app to store a record of a file that has been selected by the user so that you can retain permissions to access the file. FutureAccessList 最多可以包含 1000 个条目,因此你的应用需要管理列表,以确保列表不会饱和。The FutureAccessList has a maxium of 1000 entries, so your app needs to manage the list to make sure it does not become full. 如果你计划支持加载和修改之前创建的合成,这一点尤其重要。This is especially important if you plan to support loading and modifying previously created compositions.

  • MediaComposition 支持采用 MP4 格式的视频剪辑。A MediaComposition supports video clips in MP4 format.

  • 如果视频文件包含多个嵌入的音轨,通过设置 SelectedEmbeddedAudioTrackIndex 属性可以选择在合成中使用哪个音轨。If a video file contains multiple embedded audio tracks, you can select which audio track is used in the composition by setting the SelectedEmbeddedAudioTrackIndex property.

  • 通过调用 CreateFromColor 并指定该剪辑的颜色和持续时间,创建使用单一颜色填充整个帧的 MediaClipCreate a MediaClip with a single color filling the entire frame by calling CreateFromColor and specifying a color and a duration for the clip.

  • 通过调用 CreateFromImageFileAsync 并指定该剪辑的图像文件和持续时间,从图像文件中创建 MediaClipCreate a MediaClip from an image file by calling CreateFromImageFileAsync and specifying an image file and a duration for the clip.

  • 通过调用 CreateFromSurface 并指定该剪辑的图面和持续时间,从 IDirect3DSurface 创建 MediaClipCreate a MediaClip from a IDirect3DSurface by calling CreateFromSurface and specifying a surface and a duration from the clip.

在 MediaElement 中预览合成Preview the composition in a MediaElement

若要使用户能够查看媒体合成,将 MediaPlayerElement 添加到定义 UI 的 XAML 文件中。To enable the user to view the media composition, add a MediaPlayerElement to the XAML file that defines your UI.

<MediaPlayerElement x:Name="mediaPlayerElement" AutoPlay="False" Margin="5" HorizontalAlignment="Stretch" AreTransportControlsEnabled="True" />

声明类型 MediaStreamSource 的成员变量。Declare a member variable of type MediaStreamSource.

private MediaStreamSource mediaStreamSource;

调用 MediaComposition 对象的 GeneratePreviewMediaStreamSource 方法,为合成创建 MediaStreamSourceCall the MediaComposition object's GeneratePreviewMediaStreamSource method to create a MediaStreamSource for the composition. 通过调用工厂方法 CreateFromMediaStreamSource 创建 MediaSource 对象,并将其分配给 MediaPlayerElementSource 属性。Create a MediaSource object by calling the factory method CreateFromMediaStreamSource and assign it to the Source property of the MediaPlayerElement. 现在,可以在 UI 中查看合成。Now the composition can be viewed in the UI.

public void UpdateMediaElementSource()
{

    mediaStreamSource = composition.GeneratePreviewMediaStreamSource(
        (int)mediaPlayerElement.ActualWidth,
        (int)mediaPlayerElement.ActualHeight);

    mediaPlayerElement.Source = MediaSource.CreateFromMediaStreamSource(mediaStreamSource);

}
  • MediaComposition 必须包含至少一个媒体剪辑才能调用 GeneratePreviewMediaStreamSource,否则返回的对象将为 null。The MediaComposition must contain at least one media clip before calling GeneratePreviewMediaStreamSource, or the returned object will be null.

  • MediaElement 时间线不会自动更新为反映合成中的更改。The MediaElement timeline is not automatically updated to reflect changes in the composition. 建议在每次对合成进行一组更改并希望更新 UI 时,调用 GeneratePreviewMediaStreamSource 并设置 **MediaPlayerElement 的 ** Source 属性。It is recommended that you call both GeneratePreviewMediaStreamSource and set the MediaPlayerElement Source property every time you make a set of changes to the composition and want to update the UI.

建议在用户导航离开页面,以便发布关联的资源时,将 MediaPlayerElementMediaStreamSource 对象和 Source 属性设置为 null。It is recommended that you set the MediaStreamSource object and the Source property of the MediaPlayerElement to null when the user navigates away from the page in order to release associated resources.

protected override void OnNavigatedFrom(NavigationEventArgs e)
{
    mediaPlayerElement.Source = null;
    mediaStreamSource = null;
    base.OnNavigatedFrom(e);

}

将合成呈现到视频文件Render the composition to a video file

若要将媒体合成呈现到平面视频文件,以便在其他设备上进行共享和查看,你将需要使用 Windows.Media.Transcoding 命名空间中的 API。To render a media composition to a flat video file so that it can be shared and viewed on other devices, you will need to use APIs from the Windows.Media.Transcoding namespace. 若要在异步操作过程中更新 UI,你还需要使用 Windows.UI.Core 命名空间中的 API。To update the UI on the progress of the async operation, you will also need APIs from the Windows.UI.Core namespace.

using Windows.Media.Transcoding;
using Windows.UI.Core;

允许用户使用 FileSavePicker 选择输出文件后,通过调用 MediaComposition 对象的 RenderToFileAsync 将合成呈现到选定的文件。After allowing the user to select an output file with a FileSavePicker, render the composition to the selected file by calling the MediaComposition object's RenderToFileAsync. 以下示例中的剩余代码只需遵循处理 AsyncOperationWithProgress 的模式即可。The rest of the code in the following example simply follows the pattern of handling an AsyncOperationWithProgress.

private async Task RenderCompositionToFile()
{
    var picker = new Windows.Storage.Pickers.FileSavePicker();
    picker.SuggestedStartLocation = Windows.Storage.Pickers.PickerLocationId.VideosLibrary;
    picker.FileTypeChoices.Add("MP4 files", new List<string>() { ".mp4" });
    picker.SuggestedFileName = "RenderedComposition.mp4";

    Windows.Storage.StorageFile file = await picker.PickSaveFileAsync();
    if (file != null)
    {
        // Call RenderToFileAsync
        var saveOperation = composition.RenderToFileAsync(file, MediaTrimmingPreference.Precise);

        saveOperation.Progress = new AsyncOperationProgressHandler<TranscodeFailureReason, double>(async (info, progress) =>
        {
            await this.Dispatcher.RunAsync(CoreDispatcherPriority.Normal, new DispatchedHandler(() =>
            {
                ShowErrorMessage(string.Format("Saving file... Progress: {0:F0}%", progress));
            }));
        });
        saveOperation.Completed = new AsyncOperationWithProgressCompletedHandler<TranscodeFailureReason, double>(async (info, status) =>
        {
            await this.Dispatcher.RunAsync(CoreDispatcherPriority.Normal, new DispatchedHandler(() =>
            {
                try
                {
                    var results = info.GetResults();
                    if (results != TranscodeFailureReason.None || status != AsyncStatus.Completed)
                    {
                        ShowErrorMessage("Saving was unsuccessful");
                    }
                    else
                    {
                        ShowErrorMessage("Trimmed clip saved to file");
                    }
                }
                finally
                {
                        // Update UI whether the operation succeeded or not
                    }

            }));
        });
    }
    else
    {
        ShowErrorMessage("User cancelled the file selection");
    }
}
  • MediaTrimmingPreference 允许你设置转换代码操作的速度与剪裁相邻媒体剪辑的精度的优先级。The MediaTrimmingPreference allows you to prioritize speed of the transcoding operation versus the precision of trimming of adjacent media clips. Fast 导致转换代码的速度更快,但剪裁精度较低,Precise 导致转换代码的速度较慢,但剪裁精度较高。Fast causes transcoding to be faster with lower-precision trimming, Precise causes transcoding to be slower but with more precise trimming.

剪裁视频剪辑Trim a video clip

通过设置 MediaClip 对象的 TrimTimeFromStart 属性、TrimTimeFromEnd 属性或同时设置两者,剪裁合成中视频剪辑的持续时间。Trim the duration of a video clip in a composition by setting the MediaClip objects TrimTimeFromStart property, the TrimTimeFromEnd property, or both.

private void TrimClipBeforeCurrentPosition()
{
    var currentClip = composition.Clips.FirstOrDefault(
        mc => mc.StartTimeInComposition <= mediaPlayerElement.MediaPlayer.PlaybackSession.Position &&
        mc.EndTimeInComposition >= mediaPlayerElement.MediaPlayer.PlaybackSession.Position);

    TimeSpan positionFromStart = mediaPlayerElement.MediaPlayer.PlaybackSession.Position - currentClip.StartTimeInComposition;
    currentClip.TrimTimeFromStart = positionFromStart;

}
  • 你可以使用所需的任何 UI,让用户指定开始和结束剪裁值。Your can use any UI that you want to let the user specify the start and end trim values. 上述示例使用与 MediaPlayerElement 关联的 MediaPlaybackSessionPosition 属性,通过检查 StartTimeInCompositionEndTimeInComposition,先确定在合成中的当前位置上播放的是哪个 MediaClipThe example above uses the Position property of the MediaPlaybackSession associated with the MediaPlayerElement to first determine which MediaClip is playing back at the current position in the composition by checking the StartTimeInComposition and EndTimeInComposition. 然后,再次使用 PositionStartTimeInComposition 属性,计算要从剪辑开始处剪裁的时间量。Then the Position and StartTimeInComposition properties are used again to calculate the amount of time to trim from the beginning of the clip. FirstOrDefault 方法是 System.Linq 命名空间中的扩展方法,可简化用于从列表中选择项目的代码。The FirstOrDefault method is an extension method from the System.Linq namespace that simplifies the code for selecting items from a list.
  • 通过 MediaClip 对象的 OriginalDuration 属性,你可以在未应用任何剪辑的情况下知道媒体剪辑的持续时间。The OriginalDuration property of the MediaClip object lets you know the duration of the media clip without any clipping applied.
  • 通过 TrimmedDuration 属性,你可以在应用剪裁后知道媒体剪辑的持续时间。The TrimmedDuration property lets you know the duration of the media clip after trimming is applied.
  • 指定一个大于剪辑原始持续时间的剪裁值不会引发错误。Specifying a trimming value that is larger than the original duration of the clip does not throw an error. 但是,如果合成仅包含单个剪辑,并且通过指定较大的剪裁值将其长度剪裁为零,则对 GeneratePreviewMediaStreamSource 的后续调用将返回 null,就像合成没有剪辑一样。However, if a composition contains only a single clip and that is trimmed to zero length by specifying a large trimming value, a subsequent call to GeneratePreviewMediaStreamSource will return null, as if the composition has no clips.

将后台音轨添加到合成Add a background audio track to a composition

若要将后台音轨添加到合成,请加载音频文件,然后通过调用工厂方法 BackgroundAudioTrack.CreateFromFileAsync 创建 BackgroundAudioTrack 对象。To add a background track to a composition, load an audio file and then create a BackgroundAudioTrack object by calling the factory method BackgroundAudioTrack.CreateFromFileAsync. 然后,将 BackgroundAudioTrack 添加到合成的 BackgroundAudioTracks 属性。Then, add the BackgroundAudioTrack to the composition's BackgroundAudioTracks property.

private async Task AddBackgroundAudioTrack()
{
    // Add background audio
    var picker = new Windows.Storage.Pickers.FileOpenPicker();
    picker.SuggestedStartLocation = Windows.Storage.Pickers.PickerLocationId.MusicLibrary;
    picker.FileTypeFilter.Add(".mp3");
    picker.FileTypeFilter.Add(".wav");
    picker.FileTypeFilter.Add(".flac");
    Windows.Storage.StorageFile audioFile = await picker.PickSingleFileAsync();
    if (audioFile == null)
    {
        ShowErrorMessage("File picking cancelled");
        return;
    }

    // These files could be picked from a location that we won't have access to later
    var storageItemAccessList = Windows.Storage.AccessCache.StorageApplicationPermissions.FutureAccessList;
    storageItemAccessList.Add(audioFile);

    var backgroundTrack = await BackgroundAudioTrack.CreateFromFileAsync(audioFile);

    composition.BackgroundAudioTracks.Add(backgroundTrack);

}
  • MediaComposition 支持采用以下格式的后台音轨:MP3、WAV、FLACA MediaComposition supports background audio tracks in the following formats: MP3, WAV, FLAC

  • 后台音轨A background audio track

  • 与视频文件一样,你应用使用 StorageApplicationPermissions 类保留对合成中文件的访问权限。As with video files, you should use the StorageApplicationPermissions class to preserve access to files in the composition.

  • MediaClip 一样,BackgroundAudioTrack 只能包含在合成中一次。As with MediaClip, a BackgroundAudioTrack can only be included in a composition once. 尝试添加已由合成使用的 BackgroundAudioTrack 将导致错误。Attempting to add a BackgroundAudioTrack that is already being used by the composition will result in an error. 若要在合成中多次重复使用音轨,请调用 Clone 以创建新的 MediaClip 对象,这些对象随后将添加到合成中。To reuse an audio track multiple times in a composition, call Clone to create new MediaClip objects which can then be added to the composition.

  • 默认情况下,后台音轨在合成的开头开始播放。By default, background audio tracks begin playing at the start of the composition. 如果存在多个后台音轨,将在合成的开头开始播放所有音轨。If multiple background tracks are present, all of the tracks will begin playing at the start of the composition. 若要使后台音轨在其他时间开始播放,请将 Delay 属性设置为所需的时间偏移。To cause a background audio track to be begin playback at another time, set the Delay property to the desired time offset.

将覆盖添加到合成Add an overlay to a composition

覆盖允许你在合成中各个覆盖顶部堆栈多层视频。Overlays allow you to stack multiple layers of video on top of each other in a composition. 合成可以包含多个覆盖层,每个覆盖层可以包含多个覆盖。A composition can contain multiple overlay layers, each of which can include multiple overlays. 通过将 MediaClip 传入其构造函数中来创建 MediaOverlay 对象。Create a MediaOverlay object by passing a MediaClip into its constructor. 设置覆盖的位置和不透明度,然后创建新的 MediaOverlayLayer 并将 MediaOverlay 添加到其 Overlays 列表。Set the position and opacity of the overlay, then create a new MediaOverlayLayer and add the MediaOverlay to its Overlays list. 最后,将 MediaOverlayLayer 添加到合成的 OverlayLayers 列表。Finally, add the MediaOverlayLayer to the composition's OverlayLayers list.

private void AddOverlay(MediaClip overlayMediaClip, double scale, double left, double top, double opacity)
{
    Windows.Media.MediaProperties.VideoEncodingProperties encodingProperties =
        overlayMediaClip.GetVideoEncodingProperties();

    Rect overlayPosition;

    overlayPosition.Width = (double)encodingProperties.Width * scale;
    overlayPosition.Height = (double)encodingProperties.Height * scale;
    overlayPosition.X = left;
    overlayPosition.Y = top;

    MediaOverlay mediaOverlay = new MediaOverlay(overlayMediaClip);
    mediaOverlay.Position = overlayPosition;
    mediaOverlay.Opacity = opacity;

    MediaOverlayLayer mediaOverlayLayer = new MediaOverlayLayer();
    mediaOverlayLayer.Overlays.Add(mediaOverlay);

    composition.OverlayLayers.Add(mediaOverlayLayer);
}
  • 某个层中的覆盖根据其在包含层的 Overlays 列表中的顺序,按 z 顺序进行排序。Overlays within a layer are z-ordered based on their order in their containing layer's Overlays list. 在该列表内,较高索引呈现在较低索引的顶部。Higher indices within the list are rendered on top of lower indices. 这对于合成内的覆盖层同样适用。The same is true of overlay layers within a composition. 在合成的 OverlayLayers 列表中,具有较高索引的层将呈现在较低索引的顶部。A layer with higher index in the composition's OverlayLayers list will be rendered on top of lower indices.

  • 因为覆盖将在各个覆盖顶部堆栈,而不是按顺序播放,因此默认情况下,所有覆盖都将在合成的开头开始播放。Because overlays are stacked on top of each other instead of being played sequentially, all overlays start playback at the beginning of the composition by default. 若要使覆盖在其他时间开始播放,请将 Delay 属性设置为所需的时间偏移。To cause an overlay to be begin playback at another time, set the Delay property to the desired time offset.

将效果添加到媒体剪辑Add effects to a media clip

合成中的每个 MediaClip 都具有音频和视频效果列表,可向其中添加多个效果。Each MediaClip in a composition has a list of audio and video effects to which multiple effects can be added. 效果必须分别实现 IAudioEffectDefinitionIVideoEffectDefinitionThe effects must implement IAudioEffectDefinition and IVideoEffectDefinition respectively. 以下示例使用当前 MediaPlayerElement 位置选择当前查看的 MediaClip,然后创建 VideoStabilizationEffectDefinition 的新实例并将其追加到媒体剪辑的 VideoEffectDefinitions 列表。The following example uses the current MediaPlayerElement position to choose the currently viewed MediaClip and then creates a new instance of the VideoStabilizationEffectDefinition and appends it to the media clip's VideoEffectDefinitions list.

private void AddVideoEffect()
{
    var currentClip = composition.Clips.FirstOrDefault(
        mc => mc.StartTimeInComposition <= mediaPlayerElement.MediaPlayer.PlaybackSession.Position &&
        mc.EndTimeInComposition >= mediaPlayerElement.MediaPlayer.PlaybackSession.Position);

    VideoStabilizationEffectDefinition videoEffect = new VideoStabilizationEffectDefinition();
    currentClip.VideoEffectDefinitions.Add(videoEffect);
}

将合成保存到文件Save a composition to a file

媒体合成可序列化为要在以后修改的文件。Media compositions can be serialized to a file to be modified at a later time. 选取输出文件,然后调用 MediaComposition 方法 SaveAsync 以保存合成。Pick an output file and then call the MediaComposition method SaveAsync to save the composition.

private async Task SaveComposition()
{
    var picker = new Windows.Storage.Pickers.FileSavePicker();
    picker.SuggestedStartLocation = Windows.Storage.Pickers.PickerLocationId.VideosLibrary;
    picker.FileTypeChoices.Add("Composition files", new List<string>() { ".cmp" });
    picker.SuggestedFileName = "SavedComposition";

    Windows.Storage.StorageFile compositionFile = await picker.PickSaveFileAsync();
    if (compositionFile == null)
    {
        ShowErrorMessage("User cancelled the file selection");
    }
    else
    {
        var action = composition.SaveAsync(compositionFile);
        action.Completed = (info, status) =>
        {
            if (status != AsyncStatus.Completed)
            {
                ShowErrorMessage("Error saving composition");
            }

        };
    }
}

从文件中加载合成Load a composition from a file

媒体合成可从文件中进行反序列化,以允许用户查看和修改合成。Media compositions can be deserialized from a file to allow the user to view and modify the composition. 选取合成文件,然后调用 MediaComposition 方法 LoadAsync 以加载合成。Pick a composition file and then call the MediaComposition method LoadAsync to load the composition.

private async Task OpenComposition()
{
    var picker = new Windows.Storage.Pickers.FileOpenPicker();
    picker.SuggestedStartLocation = Windows.Storage.Pickers.PickerLocationId.VideosLibrary;
    picker.FileTypeFilter.Add(".cmp");

    Windows.Storage.StorageFile compositionFile = await picker.PickSingleFileAsync();
    if (compositionFile == null)
    {
        ShowErrorMessage("File picking cancelled");
    }
    else
    {
        composition = null;
        composition = await MediaComposition.LoadAsync(compositionFile);

        if (composition != null)
        {
            UpdateMediaElementSource();

        }
        else
        {
            ShowErrorMessage("Unable to open composition");
        }
    }
}
  • 如果合成中的媒体文件不在你的应用可以访问的位置中,并且也不在你的应用的 StorageApplicationPermissions 类的 FutureAccessList 属性中,当加载该合成时,将引发错误。If a media file in the composition is not in a location that can be accessed by your app and is not in the FutureAccessList property of the StorageApplicationPermissions class for your app, an error will be thrown when loading the composition.