UWP Strange MediaComposition issue with png transparent image overlay

Question

question

MahStudio1-5125 asked May 29, '20 MahStudio1-5125 commented Jun 2, '20

UWP Strange MediaComposition issue with png transparent image overlay

Hi everyone.
I'm here with a new question again 😅
I want t use media composition with some PNG image overlays.
I tried the Microsoft Media Editing sample and everything was ok with the same exact files!
I tried same base video and same overlay image and get 2 different results!!

As you can see in Media Editing sample of Microsoft transparent PNG image will be added without problem, But in my code it has a white background. I don't why, however, my code is 95% copy paste of the media editing sample but in another project.
Here's my sample code

         private MediaComposition composition;
         private StorageFile baseVideoFile;
         private StorageFile overlayVideoFile;
         private MediaStreamSource mediaStreamSource;
         public static MainPage Main { get; set; }
         public MainPage()
         {
             this.InitializeComponent();
             Main = this;
         }
    
         protected override void OnNavigatedTo(NavigationEventArgs e)
         {
             base.OnNavigatedTo(e);
             ChooseBaseVideo();
             ChooseOverlayVideo();
         }
    
         protected override void OnNavigatedFrom(NavigationEventArgs e)
         {
             mediaElement.Source = null;
             mediaStreamSource = null;
             base.OnNavigatedFrom(e);
         }
    
         private async void ChooseBaseVideo()
         {
             var picker = new FileOpenPicker();
             picker.SuggestedStartLocation = Windows.Storage.Pickers.PickerLocationId.VideosLibrary;
             picker.FileTypeFilter.Add(".mp4");
             baseVideoFile = await picker.PickSingleFileAsync();
             mediaElement.SetSource(await baseVideoFile.OpenReadAsync(), baseVideoFile.ContentType);
         }
    
         private async void ChooseOverlayVideo()
         {
             var picker = new FileOpenPicker();
             picker.SuggestedStartLocation = Windows.Storage.Pickers.PickerLocationId.VideosLibrary;
             picker.FileTypeFilter.Add(".png");
             overlayVideoFile = await picker.PickSingleFileAsync();
    
             CreateOverlays();
         }
    
         private async void CreateOverlays()
         {
             var baseVideoClip = await MediaClip.CreateFromFileAsync(baseVideoFile);
             composition = new MediaComposition();
             composition.Clips.Add(baseVideoClip);
    
             var overlayVideoClip = await MediaClip.CreateFromImageFileAsync(overlayVideoFile, TimeSpan.FromSeconds(10));
    
             // Overlay video in upper left corner, retain its native aspect ratio
             Rect videoOverlayPosition;
             //var encodingProperties = overlayVideoClip.GetVideoEncodingProperties();
             videoOverlayPosition.Height = mediaElement.ActualHeight / 3;
             videoOverlayPosition.Width = (double)mediaElement.ActualWidth / 3;
             videoOverlayPosition.X = 0;
             videoOverlayPosition.Y = 0;
    
             var videoOverlay = new MediaOverlay(overlayVideoClip);
             videoOverlay.Position = videoOverlayPosition;
             videoOverlay.Opacity = 0.75;
    
             var overlayLayer = new MediaOverlayLayer();
             overlayLayer.Overlays.Add(videoOverlay);
             composition.OverlayLayers.Add(overlayLayer);
    
             // Render to MediaElement
             mediaElement.Position = TimeSpan.Zero;
             mediaStreamSource = composition.GeneratePreviewMediaStreamSource((int)mediaElement.ActualWidth, (int)mediaElement.ActualHeight);
             mediaElement.SetMediaStreamSource(mediaStreamSource);
         }

and here is the files I have problem with them

8857-250646952029196.png - attachment

8806-1750116211982526.png - attachment

But some files are also ok even with my code like this

8875-297436944045492-2.png - attachment

You can find my full repos here:
https://www.mediafire.com/file/tyi2wyaicdjk3p8/Repos.rar/file
and my edited code of MediaEding sample here :
8914-scenario4-addoverlaysxamlcs.txt

windows-uwp

output.jpg (95.3 KiB)

250646952029196.png (7.8 KiB)

1750116211982526.png (5.0 KiB)

297436944045492-2.png (10.8 KiB)

scenario4-addoverlaysxamlcs.txt (4.4 KiB)

· 3

5 |1600 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

FayWang-MSFT · Jun 01, 2020 at 02:43 AM

You mentioned "in Media Editing sample of Microsoft, the transparent PNG image will be added without problem", I checked the official sample, it chooses the video as the overlays and it still has the background. How did you let the sample can choose the PNG image as overlays? Changing the FileTypeFilter of picker as png like your code? What's the difference between your code and official sample? Can you show more details about these?

1 ·

MahStudio1-5125 FayWang-MSFT · Jun 01, 2020 at 10:24 AM

yes, I changed the file format in Scenario4_AddOverlays
In ChooseOverlayVideo_Click function I just changed the third line to picker.FileTypeFilter.Add(".png");
then in CreateOverlays() function I changed the 4th line to add the image to the baseclip
var overlayVideoClip = await MediaClip.CreateFromImageFileAsync(overlayVideoFile, TimeSpan.FromSeconds(10));
and everything is fine by these simple changes in MediaEditing sample.
My code also completely shared in my question, I created a new project just with a mediaelement and the code I shared for the back code.@FayWang-MSFT

0 ·

MahStudio1-5125 MahStudio1-5125 · Jun 01, 2020 at 11:02 AM

I also added my own repos and the modified code of MediaEditing sample

0 ·

Answer 1 · 2020-06-02T07:06:26.543Z

FayWang-MSFT answered Jun 2, '20 MahStudio1-5125 commented Jun 2, '20

Hello,

Welcome to Microsoft Q&A!

When you used overlay image, the alpha value has not been premultiplied. Before using the image, you could set the BitmapAlphaMode as Premultiplied to premultiply the alpha value. For example:

.cs:

 private async void CreateOverlays()
 {
     var baseVideoClip = await MediaClip.CreateFromFileAsync(baseVideoFile);
     composition = new MediaComposition();
     composition.Clips.Add(baseVideoClip);
    
     using (IRandomAccessStream stream = await overlayVideoFile.OpenAsync(FileAccessMode.ReadWrite))
     {
    
         var bitmap = new BitmapImage();
         await bitmap.SetSourceAsync(stream);
         var writeableBitmap = new WriteableBitmap(bitmap.PixelWidth, bitmap.PixelHeight);
         stream.Seek(0);
         await writeableBitmap.SetSourceAsync(stream);
    
         var logicalDpi = DisplayInformation.GetForCurrentView().LogicalDpi;
         var encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.PngEncoderId, stream);
         encoder.SetPixelData(
             BitmapPixelFormat.Bgra8,
             BitmapAlphaMode.Premultiplied,
             (uint)writeableBitmap.PixelWidth,
             (uint)writeableBitmap.PixelHeight,
             logicalDpi,
             logicalDpi,
             writeableBitmap.PixelBuffer.ToArray());
         await encoder.FlushAsync();
         stream.Dispose();
    
         MediaClip overlayVideoClip = await MediaClip.CreateFromImageFileAsync(overlayVideoFile, TimeSpan.FromSeconds(10));
    
         Rect videoOverlayPosition;
         videoOverlayPosition.Height = mediaElement.ActualHeight / 3;
         videoOverlayPosition.Width = (double)mediaElement.ActualWidth / 3;
         videoOverlayPosition.X = 0;
         videoOverlayPosition.Y = 0;
    
         var videoOverlay = new MediaOverlay(overlayVideoClip);
         videoOverlay.Position = videoOverlayPosition;
         videoOverlay.Opacity = 0.75;
    
         var overlayLayer = new MediaOverlayLayer();
         overlayLayer.Overlays.Add(videoOverlay);
         composition.OverlayLayers.Add(overlayLayer);
    
        // Render to MediaElement
        mediaElement.Position = TimeSpan.Zero;
        mediaStreamSource = composition.GeneratePreviewMediaStreamSource((int)mediaElement.ActualWidth, (int)mediaElement.ActualHeight);
        mediaElement.SetMediaStreamSource(mediaStreamSource);
     }
 }

· 1

5 |1600 characters needed characters left characters exceeded

Attachments: Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

MahStudio1-5125 · Jun 02, 2020 at 12:03 PM

Yeeees it works. Thanks

0 ·

question