使用 MediaFrameSourceGroup 從多個來源擷取Capture from multiple sources using MediaFrameSourceGroup

本文示範如何從多個來源同步擷取視訊到單一檔案,內含多個內嵌視訊播放軌。This article shows you how to capture video from multiple sources simultaneously to a single file with multiple embedded video tracks. 從 RS3 開始,您可以為單一 MediaEncodingProfile 指定多個 VideoStreamDescriptor 物件。Starting with RS3, you can specify multiple VideoStreamDescriptor objects for a single MediaEncodingProfile. 這可讓您同時將多個串流編碼至單一檔案。This enables you to encode multiple streams simultaneously to a single file. 以此操作編碼的視訊串流必須包含在單一 MediaFrameSourceGroup 中,指定可在同一時間使用的目前裝置上的一組相機。The video streams that are encoded in this operation must be included in a single MediaFrameSourceGroup which specifies a set of cameras on the current device that can be used at the same time.

如需使用 MediaFrameSourceGroup 搭配 MediaFrameReader 類別以啟用運用多個相機的即時電腦視覺情景相關資訊,請參閱使用 MediaFrameReader 處理媒體畫面For information on using MediaFrameSourceGroup with the MediaFrameReader class to enable real-time computer vision scenarios that use multiple cameras, see Process media frames with MediaFrameReader.

本文章的其餘部分將逐步引導您從兩部彩色相機錄製影片到包含多個視訊播放軌的單一檔案。The rest of this article will walk you through the steps of recording video from two color cameras to a single file with multiple video tracks.

尋找可用的感應器群組Find available sensor groups

MediaFrameSourceGroup 代表可同時存取的畫面來源集合,通常是相機。A MediaFrameSourceGroup represents a collection of frame sources, typically cameras, that can be accessed simulataneously. 每個裝置的可用畫面來源群組集都不同,因此此範例的第一個步驟就是取得可用畫面來源群組的清單,並找到包含此例所需相機的群組,在此例中這需要兩部彩色相機。The set of available frame source groups is different for each device, so the first step in this example is to get the list of available frame source groups and finding one that contains the necessary cameras for the scenario, which in this case requires two color cameras.

MediaFrameSourceGroup.FindAllAsync 方法會傳回目前裝置上可用的所有來源群組。The MediaFrameSourceGroup.FindAllAsync method returns all source groups available on the current device. 每個傳回的 MediaFrameSourceGroup 都有 MediaFrameSourceInfo 物件的清單,描述群組中的每個畫面來源。Each returned MediaFrameSourceGroup has a list of MediaFrameSourceInfo objects that describes each frame source in the group. Linq 查詢用來尋找包含兩部彩色相機的來源群組,一個在前端面板上,一個在背面。A Linq query is used to find a source group that contains two color cameras, one on the front panel and one on the back. 傳回匿名物件,包含每部彩色相機的選定 MediaFrameSourceGroupMediaFrameSourceInfoAn anonymous object is returned that contains the selected MediaFrameSourceGroup and the MediaFrameSourceInfo for each color camera. 不使用 Linq 語法,您可以改為循環處理每個群組以及每個 MediaFrameSourceInfo 來尋找符合您需求的群組。Instead of using Linq syntax, you could instead loop through each group, and then each MediaFrameSourceInfo to look for a group that meets your requirements.

請注意,並非每個裝置都包含有兩部彩色相機的來源群組,因此您應該要檢查以確認有找到來源群組,然後再擷取視訊。Note that not every device will contain a source group that contains two color cameras, so you should check to make sure that a source group was found before trying to capture video.

var sensorGroups = await MediaFrameSourceGroup.FindAllAsync();

var foundGroup = sensorGroups.Select(g => new
{
    group = g,
    color1 = g.SourceInfos.Where(info => info.SourceKind == MediaFrameSourceKind.Color && info.DeviceInformation.EnclosureLocation.Panel == Windows.Devices.Enumeration.Panel.Front).FirstOrDefault(),
    color2 = g.SourceInfos.Where(info => info.SourceKind == MediaFrameSourceKind.Color && info.DeviceInformation.EnclosureLocation.Panel == Windows.Devices.Enumeration.Panel.Back).FirstOrDefault()
}).Where(g => g.color1 != null && g.color2 != null).FirstOrDefault();

if (foundGroup == null)
{
    Debug.WriteLine("No groups found.");
    return;
}

初始化 MediaCapture 物件Initialize the MediaCapture object

MediaCapture 類別是 UWP app 中用於大部分音訊、視訊和相片擷取作業的主要類別。The MediaCapture class is the primary class that is used for most audio, video, and photo capture operations in UWP apps. 透過呼叫 InitializeAsync、傳遞 MediaCaptureInitializationSettings 包含初始化參數的物件,來初始化物件。Initialize the object by calling InitializeAsync, passing in a MediaCaptureInitializationSettings object that contains initialization parameters. 在此範例中,指定的唯一設定是 SourceGroup 屬性,設為上一個程式碼範例中擷取的 MediaFrameSourceGroupIn this example, the only specified setting is the SourceGroup property, which is set to the MediaFrameSourceGroup that was retrieved in the previous code example.

如需您可搭配 MediaCapture 執行的其他作業以及擷取媒體的其他 UWP app 功能的相關資訊,請參閱相機For information on other operations you can perform with MediaCapture and other UWP app features for capturing media, see Camera.

var settings = new MediaCaptureInitializationSettings()
{
    SourceGroup = foundGroup.group
};

mediaCapture = new MediaCapture();
await mediaCapture.InitializeAsync(settings);

建立 MediaEncodingProfileCreate a MediaEncodingProfile

MediaEncodingProfile 類別會告知媒體擷取管線在將擷取到的音訊和視訊寫入檔案時,應該如何編碼。The MediaEncodingProfile class tells the media capture pipeline how captured audio and video should be encoded as they are written to a file. 對於一般擷取和轉碼案例,此類別提供一組靜態方法來建立常見的設定檔,例如 CreateAviCreateMp3For typical capture and transcoding scenarios, this class provides a set of static methods for creating common profiles, like CreateAvi and CreateMp3. 針對此範例,將使用 Mpeg4 容器和 H264 視訊編碼來手動建立編碼設定檔。For this example, an encoding profile is manually created using an Mpeg4 container and H264 video encoding. 使用 VideoEncodingProperties 物件來指定視訊編碼設定。Video encoding settings are specified using a VideoEncodingProperties object. 針對本案例中使用的每部彩色相機,會設定 VideoStreamDescriptor 物件。For each color camera used in this scenario, a VideoStreamDescriptor object is configured. 使用 VideoEncodingProperties 物件建構描述元,指定編碼。The descriptor is constructed with the VideoEncodingProperties object specifying the encoding. VideoStreamDescriptorLabel 屬性必須設成要擷取至串流的媒體畫面來源的識別碼。The Label property of the VideoStreamDescriptor must be set to the ID of the media frame source that will be captured to the stream. 擷取管線透過此方法得知哪一個串流描述元與編碼屬性應用於每個相機。This is how the capture pipeline knows which stream descriptor and encoding properties should be used for each camera. 畫面來源的識別碼由 MediaFrameSourceInfo 物件公開,選取 MediaFrameSourceGroup 時可在上一區段找到此物件。The ID of the frame source is exposed by the MediaFrameSourceInfo objects that were found in the previous section, when a MediaFrameSourceGroup was selected.

從 Windows 10 版本 1709 開始,您可以透過呼叫 SetVideoTracks,在 MediaEncodingProfile 上設定多個編碼屬性。Starting with Windows 10, version 1709, you can set multiple encoding properties on a MediaEncodingProfile by calling SetVideoTracks. 您可以呼叫 GetVideoTracks 來擷取視訊串流描述元的清單。You can retrieve the list of video stream descriptors by calling GetVideoTracks. 請注意,如果您設定 Video 屬性 (用於儲存單一串流描述元),則您透過呼叫 SetVideoTracks 設定的描述元清單會更換為包含您所指定之單一描述元的清單。Note that if you set the Video property, which stores a single stream descriptor, the descriptor list you set by calling SetVideoTracks will be replaced with a list containing the single descriptor you specified.

var profile = new MediaEncodingProfile();
profile.Container = new ContainerEncodingProperties();
profile.Container.Subtype = MediaEncodingSubtypes.Mpeg4;

List<VideoStreamDescriptor> streams = new List<VideoStreamDescriptor>();

var encodeProps = VideoEncodingProperties.CreateH264();
encodeProps.Subtype = MediaEncodingSubtypes.H264;
var stream1Desc = new VideoStreamDescriptor(encodeProps);
stream1Desc.Label = foundGroup.color1.Id;
streams.Add(stream1Desc);

var encodeProps2 = VideoEncodingProperties.CreateH264();
encodeProps2.Subtype = MediaEncodingSubtypes.H264;
var stream2Desc = new VideoStreamDescriptor(encodeProps2);
stream2Desc.Label = foundGroup.color2.Id;
streams.Add(stream2Desc);

profile.SetVideoTracks(streams);
profile.Audio = null;

編碼媒體檔案中的定時中繼資料Encode timed metadata in media files

從 Windows 10 版本 1803 開始,除了音訊和視訊,您還可以將定時中繼資料編碼至支援該資料格式的媒體檔案。Starting with Windows 10, version 1803, in addition to audio and video you can encode timed metadata into a media file for which the data format is supported. 例如,GoPro 中繼資料 (gpmd) 可以儲存在 MP4 檔案中以傳遞與視訊串流關聯的地理位置。For example, GoPro metadata (gpmd) can be stored in MP4 files to convey the geographic location correlated with a video stream.

編碼中繼資料採用的樣式平行於編碼音訊或視訊的樣式。Encoding metadata uses a pattern that is parallel to encoding audio or video. TimedMetadataEncodingProperties 類別描述性中繼資料的類型、子類型和編碼屬性,例如用於視訊的 VideoEncodingPropertiesThe TimedMetadataEncodingProperties class describes the type, subtype and encoding properties of the metadata, like VideoEncodingProperties does for video. TimedMetadataStreamDescriptor 識別中繼資料串流,如同用於視串流訊的 VideoStreamDescriptorThe TimedMetadataStreamDescriptor identifies a metadata stream, just as the VideoStreamDescriptor does for video streams.

下列範例示範如何初始化 TimedMetadataStreamDescriptor 物件。The following example shows how to intialize a TimedMetadataStreamDescriptor object. 首先,會建立 TimedMetadataEncodingProperties 物件並將 Subtype 設定為用於識別將包含於串流中之中繼資料類型的 GUID。First, a TimedMetadataEncodingProperties object is created and the Subtype is set to a GUID that identifies the type of metadata that will be included in the stream. 此範例使用 GoPro 中繼資料 (gpmd) 的 GUID。This example uses the GUID for GoPro metadata (gpmd). 呼叫 SetFormatUserData 方法來設定格式特定資料。The SetFormatUserData method is called to set format-specific data. 對於 MP4 檔案,格式特定資料儲存在 SampleDescription 方塊 (stsd) 中。For MP4 files, the format-specific data is stored in the SampleDescription box (stsd). 接著,從編碼屬性建立新的 TimedMetadataStreamDescriptorNext, a new TimedMetadataStreamDescriptor is created from the encoding properties. 設定 LabelName 屬性來識別要編碼的串流。The Label and Name properties are set to identify the stream to be encoded.

           TimedMetadataEncodingProperties encodingProperties = new TimedMetadataEncodingProperties
           {
               Subtype = "{67706D64-BF10-48B4-BC18-593DC1DB950F}"
           };

           byte[] streamDescriptionData = GetStreamDescriptionDataForGpmdEncodingSubtype();
           encodingProperties.SetFormatUserData(streamDescriptionData);

           TimedMetadataStreamDescriptor descriptor = new TimedMetadataStreamDescriptor(encodingProperties)
           {
               Name = "GPS Info",
               Label = "GPS Info"
           };

呼叫 MediaEncodingProfile. SetTimedMetadataTracks ,將中繼資料資料流程描述元加入至編碼設定檔。Call MediaEncodingProfile.SetTimedMetadataTracks to add the metadata stream descriptor to the encoding profile. 下列範例顯示的協助程式方法會取得兩個視訊串流描述項、一個音訊串流描述項與一個定時中繼資料串流描述項,並傳回可用於編碼串流的 MediaEncodingProfileThe following example shows a helper method that takes two video stream descriptors, one audio stream descriptor, and one timed metadata stream descriptor and returns a MediaEncodingProfile that can be used to encode the streams.

public MediaEncodingProfile CreateProfileForTranscoder(VideoStreamDescriptor videoStream1, VideoStreamDescriptor videoStream2, AudioStreamDescriptor audioStream, TimedMetadataStreamDescriptor timedMetadataStream)
{
    ContainerEncodingProperties container = new ContainerEncodingProperties()
    {
        Subtype = MediaEncodingSubtypes.Mpeg4
    };

    MediaEncodingProfile profile = new MediaEncodingProfile()
    {
        Container = container
    };


    VideoStreamDescriptor encodingVideoStream1 = videoStream1.Copy();
    encodingVideoStream1.EncodingProperties.Subtype = MediaEncodingSubtypes.H264;
    encodingVideoStream1.Label = videoStream1.Name;

    VideoStreamDescriptor encodingVideoStream2 = videoStream2.Copy();
    encodingVideoStream2.EncodingProperties.Subtype = MediaEncodingSubtypes.H264;
    encodingVideoStream2.Label = videoStream2.Name;

    AudioStreamDescriptor encodingAudioStream = audioStream.Copy();
    encodingAudioStream.EncodingProperties.Subtype = MediaEncodingSubtypes.Ac3;
    encodingAudioStream.Label = audioStream.Name;

    TimedMetadataStreamDescriptor encodingTimedMetadataStream = timedMetadataStream.Copy();

    profile.SetTimedMetadataTracks(new TimedMetadataStreamDescriptor[] { encodingTimedMetadataStream });
    profile.SetVideoTracks(new VideoStreamDescriptor[] { encodingVideoStream1, encodingVideoStream2 });
    profile.SetAudioTracks(new AudioStreamDescriptor[] { encodingAudioStream });
    return profile;
}

使用多串流 MediaEncodingProfile 錄製Record using the multi-stream MediaEncodingProfile

此範例的最後一個步驟是透過呼叫 StartRecordToStorageFileAsync、傳遞 StorageFile 至所擷取媒體的寫入目的,以及在前一個程式碼範例中建立的 MediaEncodingProfile,來起始視訊擷取功能。The final step in this example is to initiate video capture by calling StartRecordToStorageFileAsync, passing in the StorageFile to which the captured media is written, and the MediaEncodingProfile created in the previous code example. 等候幾秒後,呼叫 StopRecordAsync 來停止錄製。After waiting a few seconds, the recording is stopped with a call to StopRecordAsync.

var recordFile = await Windows.Storage.KnownFolders.CameraRoll.CreateFileAsync("record.mp4", Windows.Storage.CreationCollisionOption.GenerateUniqueName);
await mediaCapture.StartRecordToStorageFileAsync(profile, recordFile);
await Task.Delay(8000);
await mediaCapture.StopRecordAsync();

作業完成時,會建立視訊檔案,其中包含從每部相機擷取的視訊,並編碼為檔案中的個別串流。When the operation is complete, a video file will have been created that contains the video captured from each camera encoded as a separate stream within the file. 如需播放包含多個視訊播放軌之媒體檔案的相關資訊,請參閱媒體項目、播放清單與曲目For information on playing media files containing multiple video tracks, see Media items, playlists, and tracks.