Camera Capture Performance (Part1)

Still Image Capture


Windows Mobile 5.0 contains an Image Sink Filter (a DShow filter). This filter encodes the image data and writes the encoded image to a file. For encoding, the Image Sink Filter uses Imaging API to access the installed Imaging encoders. The still image encoders which Microsoft ships with Windows Mobile 5.0 are not optimized and should, in general be replaced by OEMs with optimized encoders for the particular platform. One limitation of Imaging framework is that it only accepts RGB input, i.e., the raw buffer sent to Imaging encoder must be in RGB format. For this very reason, the Image Sink filter does not accept YCbCr.

Besides accepting RGB data, the Image Sink Filter also accepts pre-encoded JPG data. This is useful when there is an optimized still image encoder available on a given platform, which accepts YCbCr. For this to work the actual encoding is done by the driver and the camera driver exposes a format with SubFormat field of CSDATARANGE equal to MEDIASUBTYPE_IJPG GUID. The encoded buffer will then be passed down to the Image Sink Filter simply for file I/O. This optimized encoding can also be done in a DShow Transform filter with the transform filter exposing MEDIASUBTYPE_IJPG on its output pin.

The OEMs can still use the encoders that are shipped with Windows Mobile 5.0. These encoders provide a stable, platform independent solution. However, as the resolution of still image increases, the performance of these encoders deteriorates.


Following is how you would change the sample null camera driver to provide JPG data on STILL pin.


Step 1: Define the format

In adapterprops.h define the following

#define FOURCC_IJPG mmioFOURCC('I', 'J', 'P', 'G')

#define MEDIASUBTYPE_IJPG {0x47504A49, 0x0000, 0x0010, 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}

#define DX 176

#define DY 144

#define DBITCOUNT 16

#define FRAMERATE 15





sizeof (CS_DATARANGE_VIDEO), // Flags


SAMPLESIZE, // Replace with maximum Sample Size that the JPG data would take for the current resolution and image quality

0, // Reserved





TRUE, // BOOL, bFixedSizeSamples (all samples same size?)

TRUE, // BOOL, bTemporalCompression (all I frames?)


0, // MemoryAllocationFlags (CS_VIDEO_ALLOC_*)



// Omitted for this sample. Please refer to adapterprops.h in the sample null camera driver for ways to fill in this structure






0,0,DX,DY, // RECT rcSource;

0,0,DX,DY, // RECT rcTarget;

BITRATE, // DWORD dwBitRate;

0L, // DWORD dwBitErrorRate;



DX, // LONG biWidth;

DY, // LONG biHeight;

3, // WORD biPlanes;

DBITCOUNT, // WORD biBitCount;



0, // LONG biXPelsPerMeter;

0, // LONG biYPelsPerMeter;

0, // DWORD biClrUsed;

0, // DWORD biClrImportant;

0, 0, 0 // DWORD dwBitMasks[3]



Step 2: Add this format to the list of formats of Still pin

In sample null camera driver, you would modify CCameraDevice::Initialize() in cameradevice.cpp to have

    m_PinVideoFormat[STILL].categoryGUID = PINNAME_VIDEO_STILL;

    m_PinVideoFormat[STILL].ulAvailFormats = 1;

    m_PinVideoFormat[STILL].pCsDataRangeVideo[0] = &DCAM_StreamMode;

Step 3: Modify the BufferFill() function

Use biCompression field of the current format to determine whether MEDIASUBTYPE_IJPG format is selected. In BufferFill() you can have following check


BufferFill( PUCHAR pImage, PCS_VIDEOINFOHEADER pCsVideoInfoHdr, IMAGECOMMAND Command, bool FlipHorizontal, LPVOID lpParam )


    if( NULL == pCsVideoInfoHdr )


        return -1;



    DWORD biCompression = pCsVideoInfoHdr->bmiHeader.biCompression;


    if ( (FOURCC_IJPG == (biCompression & ~BI_SRCPREROTATE)) )


      // Real drivers would add code here to handle JPEG encoding.

// The biSizeImage that is returned here should be the actual

// size of JPEG data that is written to the buffer.

        return biSizeImage;






The default implementation(AdapterHandleCompressionRequests()) for PROPSETID_VIDCAP_VIDEOCOMPRESSION returns ERROR_INVALID_PARAMETER. Camera application on Windows Mobile 5.0 and later, uses this interface to set encoder quality. You would need to change this to return ERROR_SUCCESS after correctly handling various properties in this property set. Failure to do this would result in camera application raising an exception at initialization.


Camera diagram.JPG