Media Foundation generates a bad MP4 video when it is fed frame too quickly

Question

Hello,
I am using Media Foundation to generate video from a sequence of images (rendered with openGL on the fly) but I am faced with a race condition/weird bug that leads to bad videos being outputed.

The video being outputed visually appears to be 5 fps when it should be 30 fps. Looking at information on the file correctly reports the fps of the video to be 30 so the vdeo is corrupted in antoher way (missing frames? duplicated frames?).
The video looks like this: actual.mp4
The expected result is: expected.mp4

The strange thing about this bug is that is only occurs when samples are fed to the IMFSinkWriter quickly, it doesn't occur when I introduce a delay between each sample. In fact the expected.mp4 video was generated with the same code with only an additionnal std::this_thread::sleep_for(std::chrono::milliseconds(33)) in the rendering loop.

The video file is an MP4 file with a single H264 stream. The MF_SINK_WRITER_DISABLE_THROTTLING attribute of the IMFSinkWriteris set to false (meaning the throttling should be enabled) so I would expect the sink writer to block when adding sample if I'm really giving samples too fast but it is not the case (I tried setting this attribute to both true and false as it seemed relevant but it makes no difference).
I am fairly confident my usage of Media Foundation is correct as it isn't widly different from what is done in the Microsoft samples and works when slowed down.

Unfortunately I was unable to reproduce the issue in a minimal program, I am still trying to do so and will post an update if I manage to do so but so far my minimal programs don't show the issue and work as expected.

I get that finding the origin of this bug without reproducing code may not be possible so I guess I am looking out for suggestion on how to investigate this deeper, or hypothesis on what could cause this behavior. (FOr example in the real program the code is using openGL and is located in a separate DLL from the main program, I've tried reproducing both in my minimal program but neither seemed to be a factor).