Playback H.264 video from non-IDR I-VOP with recovery point SEI via Media Source Extensions

Question

I am trying to playback H.264 video stream using hls.js or mpegts.js which uses Media Source Extensions on the Microsoft Edge or Google Chrome.

It works fine if the video stream has IDR I-VOPs (or keyframe) but it not work with non-IDR I-VOPs with recovery point SEI.

I found following issues reported on chromium forum

Mpeg-dash clip with no IDR frames in AVC ISOBMFF bytestream is no longer loaded/played back in Shaka player

HLS.js player cannot play mpeg-ts file

Consider relaxing AVC keyframeness logic to consider a SEI recovery point as a keyframe

These reports show that Google Chrome (chromium) had been degraded since 73.0.3683.103 and chromium would not fix the issue any more (won't fix).

A contributor commented:

Not all of our H.264 decoders support recovery, and in particular the Mac hardware decoder will drop all frames until it sees an IDR. Hardware decoders are often buggy when recovery is required, to the point of crashing in some known cases. We may be able to qualify some subset of decoders for SEI recovery (basically turn off hardware decode on most platforms), but this is unlikely to be worth the effort.

In other words, a derivative browser like Microsoft Edge should know that the decoder can playback from the recovery point, nevertheless, Microsoft Edge does not seem to support playback from the recovery point.

I'm wondering why does Microsoft Edge not support playback from H.264 non-IDR I-VOPs with recovery point SEI?

Answer 1

Hi @wakabayashik

Different browsers have different requiremens on playing videos. The situation you described works in Firefox but doesn't work in Edge and Chrome. We can only say it's by design in chromium. But for why, it's not documented. I think only the product team can explain, so I suggest you can provide feedback to Edge team by pressing Alt+Shift+I in Edge. But as the issue is marked as "won't fix" in chromium, I doubt if there will be any change in Edge.

---

If the answer is the right solution, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".

Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

Regards,

Yu Zhou