Camera Capture Performance (Part1)
Still Image Capture
Windows Mobile 5.0 contains an Image Sink Filter (a DShow filter). This filter encodes the image data and writes the encoded image to a file. For encoding, the Image Sink Filter uses Imaging API to access the installed Imaging encoders. The still image encoders which Microsoft ships with Windows Mobile 5.0 are not optimized and should, in general be replaced by OEMs with optimized encoders for the particular platform. One limitation of Imaging framework is that it only accepts RGB input, i.e., the raw buffer sent to Imaging encoder must be in RGB format. For this very reason, the Image Sink filter does not accept YCbCr.
Besides accepting RGB data, the Image Sink Filter also accepts pre-encoded JPG data. This is useful when there is an optimized still image encoder available on a given platform, which accepts YCbCr. For this to work the actual encoding is done by the driver and the camera driver exposes a format with SubFormat field of CSDATARANGE equal to MEDIASUBTYPE_IJPG GUID. The encoded buffer will then be passed down to the Image Sink Filter simply for file I/O. This optimized encoding can also be done in a DShow Transform filter with the transform filter exposing MEDIASUBTYPE_IJPG on its output pin.
The OEMs can still use the encoders that are shipped with Windows Mobile 5.0. These encoders provide a stable, platform independent solution. However, as the resolution of still image increases, the performance of these encoders deteriorates.
Following is how you would change the sample null camera driver to provide JPG data on STILL pin.
Step 1: Define the format
In adapterprops.h define the following
#define FOURCC_IJPG mmioFOURCC('I', 'J', 'P', 'G')
#define MEDIASUBTYPE_IJPG {0x47504A49, 0x0000, 0x0010, 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71}
#define DX 176
#define DY 144
#define DBITCOUNT 16
#define FRAMERATE 15
CS_DATARANGE_VIDEO DCAM_StreamMode =
{
// CSDATARANGE
{
sizeof (CS_DATARANGE_VIDEO), // Flags
0,
SAMPLESIZE, // Replace with maximum Sample Size that the JPG data would take for the current resolution and image quality
0, // Reserved
STATIC_CSDATAFORMAT_TYPE_VIDEO,
MEDIASUBTYPE_IJPG,
STATIC_CSDATAFORMAT_SPECIFIER_VIDEOINFO
},
TRUE, // BOOL, bFixedSizeSamples (all samples same size?)
TRUE, // BOOL, bTemporalCompression (all I frames?)
CS_VIDEOSTREAM_CAPTURE, // StreamDescriptionFlags (CS_VIDEO_DESC_*)
0, // MemoryAllocationFlags (CS_VIDEO_ALLOC_*)
// _CS_VIDEO_STREAM_CONFIG_CAPS
{
// Omitted for this sample. Please refer to adapterprops.h in the sample null camera driver for ways to fill in this structure
………
………
},
// CS_VIDEOINFOHEADER
{
0,0,DX,DY, // RECT rcSource;
0,0,DX,DY, // RECT rcTarget;
BITRATE, // DWORD dwBitRate;
0L, // DWORD dwBitErrorRate;
REFTIME_15FPS, // REFERENCE_TIME AvgTimePerFrame;
sizeof (CS_BITMAPINFOHEADER), // DWORD biSize;
DX, // LONG biWidth;
DY, // LONG biHeight;
3, // WORD biPlanes;
DBITCOUNT, // WORD biBitCount;
FOURCC_IJPG | BI_SRCPREROTATE, // DWORD biCompression;
SAMPLESIZE, // DWORD biSizeImage;
0, // LONG biXPelsPerMeter;
0, // LONG biYPelsPerMeter;
0, // DWORD biClrUsed;
0, // DWORD biClrImportant;
0, 0, 0 // DWORD dwBitMasks[3]
}
};
Step 2: Add this format to the list of formats of Still pin
In sample null camera driver, you would modify CCameraDevice::Initialize() in cameradevice.cpp to have
m_PinVideoFormat[STILL].categoryGUID = PINNAME_VIDEO_STILL;
m_PinVideoFormat[STILL].ulAvailFormats = 1;
m_PinVideoFormat[STILL].pCsDataRangeVideo[0] = &DCAM_StreamMode;
Step 3: Modify the BufferFill() function
Use biCompression field of the current format to determine whether MEDIASUBTYPE_IJPG format is selected. In BufferFill() you can have following check
UINT
BufferFill( PUCHAR pImage, PCS_VIDEOINFOHEADER pCsVideoInfoHdr, IMAGECOMMAND Command, bool FlipHorizontal, LPVOID lpParam )
{
if( NULL == pCsVideoInfoHdr )
{
return -1;
}
DWORD biCompression = pCsVideoInfoHdr->bmiHeader.biCompression;
if ( (FOURCC_IJPG == (biCompression & ~BI_SRCPREROTATE)) )
{
// Real drivers would add code here to handle JPEG encoding.
// The biSizeImage that is returned here should be the actual
// size of JPEG data that is written to the buffer.
return biSizeImage;
}
……….
……….
}
Step 4: Handle PROPSETID_VIDCAP_VIDEOCOMPRESSION
The default implementation(AdapterHandleCompressionRequests()) for PROPSETID_VIDCAP_VIDEOCOMPRESSION returns ERROR_INVALID_PARAMETER. Camera application on Windows Mobile 5.0 and later, uses this interface to set encoder quality. You would need to change this to return ERROR_SUCCESS after correctly handling various properties in this property set. Failure to do this would result in camera application raising an exception at initialization.
