SharePoint 2010 Search - Experiences and Best Practices from the Microsoft ‘Dogfood’ Implementation

  • Article

Hello, Dan Blood here with another series of blog postings about running SharePoint Server 2010 Search.  Some of you may be familiar with a similar set of postings that I wrote for MSS or SharePoint Search 2007.  The following set of postings will be targeted explicitly at SharePoint 2010 Search and are not applicable to the previous release.  Similarly these posts do not contain any data specific to FAST Search Server for SharePoint 2010.

I am very excited about the changes and new features in SharePoint 2010 Search. There are a plethora of things that make building and running the system at scale much easier.  

  • The scale limits have dramatically increased from 2007.
  • The system has been componentized to allow scaling crawls and queries separately.
  • The index now has the ability to be partitioned into smaller more manageable pieces and spread out across multiple machines.
  • Crawlers have become stateless and no longer house the entire index. Resulting in reduced hardware requirements for the crawlers.
  • You can allocate more than one machine to crawl the content for improved crawl freshness.
  • The system is much more transparent, providing admin's with information about what is going on under the hood.

 

Over the last 3 years I have been managing our internal "Dogfood" deployment (SearchBeta) that mirrors the enterprise search deployment within Microsoft and is used for product testing and validation.  This deployment provides search over ~72 million items and services a peak of 120 queries per minute.  I've also had help from Hernando Silva in maintaining a separate DogFood system for the Office organization.  This is a divisional portal providing a search experience over ~7 million items.  Hernando and I will detail these two systems in following posts to provide you with insight into the hardware choices. These efforts have helped us validate the product utilizing real world data for both large & small scale deployments.  I believe that our experiences will provide you, our customer, with a valuable set of best practices and lessons about how to run SharePoint 2010 Search in production.   

 

While I am working on the next post I would encourage you to read the following documents as starter for what the new system now looks like:

 

Dan Blood

Senior Test Engineer

Microsoft Corp