I have an UWP (C++/WinRT) application running on Hololens 2, that reads frames from an external camera (USB-attached). I am reading the frames via winrt::Windows::Media::Capture::MediaCapture:
(1) I obtain a winrt::Windows::Media::Capture::Frames::MediaFrameSource for the camera (stream type is MediaStreamType::VideoRecord, source kind is MediaFrameSourceKind::Color).
(2) select a format for the MediaFrameSource, out of its SupportedFormats() for a particular resolution, framerate etc.
(3) then get a MediaFrameReader from the said source
(4) hook on the MediaFrameReader's event FrameArrived and call StartAsync
This works alright - the FrameArrived handler fires at the expected rate, giving me frames. My issue is the following:
- when I query SupportedFormats in MediaFrameSource, there are formats with MJPG subtype (camera supports this)
- when I select one of these MJPG formats, the FrameArrived event still fires, but the frame is (internally) converted to NV12
I need to disable this internal conversion to NV12 and retrieve the MJPG frames as generated by the camera.
I have also tried to use Media Foundation, from which I can get the MJPG frames successfully, but only on desktop. On Hololens, the API is limited for some reason, leaving me unable to obtain IMFMediaSource for the webcam. So MF is probably not an option on Hololens 2. Encoding the frames myself from NV12 to JPEG is also not an option, as HL2 does not have enough computational resources to do the required conversions.
Any ideas what else I can do to receive MJPG from the webcam?