Using the Content Iterator

SharePoint Server provides a new API, ContentIterator, to help with accessing more than 5,000 items in a large list without hitting a list throttling limit and receiving an SPQueryThrottleException. ContentIterator implements a callback pattern for segmenting the query for processing a single item at a time. Consider using this capability if you need to process a large number of items that may exceed a throttling limit. The following trivial example demonstrates the approach used with ContentIterator tested with a list returning 20,001 items from the query.

static int exceptions = 0;
static int items = 0;

protected void OnTestContentIterator(object sender, EventArgs args)
{
    items = 0;
    exceptions = 0;
    string query1 = @"<View>
        <Query>
            <Where>
                <And>
                    <BeginsWith>
                        <FieldRef Name='SKU' />
                        <Value Type='Text'>S</Value>
                    </BeginsWith>
                </And>
            </Where>
        </Query>
    </View>";

    ContentIterator iterator = new ContentIterator();
    SPQuery listQuery = new SPQuery();
    listQuery.Query = query1;
    SPList list = SPContext.Current.Web.Lists["Parts"];
    iterator.ProcessListItems(list,
        listQuery,
        ProcessItem,
        ProcessError
    );
}

public    bool ProcessError(SPListItem item, Exception e) 
{ 
    // process the error
    exceptions++; 
    return true; 
}
public void ProcessItem(SPListItem item)
{
    items++;
    //process the item.
}

ContentIterator will run through each item in the list, invoking the callback provided for list item processing—in this case, ProcessItem. If an error occurs while iterating the list, then the error function is invoked—in this case, ProcessError. Using this approach the ContentIterator processes the list in pieces and avoids any excessively large queries. This functionality is provided as part of Enterprise Content Management (ECM) in SharePoint Server 2010. ContentIterator has additional functionality not described in this section, such as the ability to order result sets. For more information see the ContentIterator Class.