Manage crawler impact rules (FAST Search Server 2010 for SharePoint)

 

Applies to: FAST Search Server 2010

You can increase or decrease the request frequency of the crawl by adding Crawler impact rules.

1. Verify that the user account that is performing this procedure is a service application administrator for your FAST Search Content Search Service Application.

2. In Central Administration, in the Application Management section, click Manage service applications.

3. On the Service Applications page, in the list of service applications, click your FAST Search Content SSA.

4. On the Search Administration page, under Crawling, click Crawler Impact Rules.

5. Click Add Rule.

   1. In the Site box, type the name of the site the crawl rule should apply to.

   2. In the Request Frequency box, set the desired number of simultaneous request and/or the desired request interval.

      Warning

      Do not overload sites with requests. Request few documents at the same time and set a delay between requests.

   3. Click OK.

The request rate depends on the hardware of the server from which the FAST Search Content SSA is running. On high-end servers, you might (by default) request up to 16 documents in parallel from a single server. Set crawler impact rules to throttle the request rate for servers that should not be crawled with a high request rate, such as external Web sites for example.

See Also

Concepts

Plan for crawling and federation (FAST Search Server 2010 for SharePoint)