"Item not crawled" error when you crawl a site
Symptoms
When you try to crawl a SharePoint 2016 or 2013 website, you receive the following error message:
Item not crawled due to one of the following reasons: Preventive crawl rule; Specified content source hops/depth exceeded; URL has query string parameter; Required protocol handler not found; Preventive robots directive.
Resolution
To fix this issue, try the following methods in the given order. If one method doesn't fix the issue, try the next method.
- Examine the crawl rules in the Search service application to make sure that there isn't an existing rule that prevents the site from being crawled. For more information about crawl rules, see Manage crawl rules in SharePoint Server.
- If you use a Robots.txt file for the site, examine the User-agent and Disallow settings to make sure that they don't exclude the crawler from the site.
- If the crawler uses TLS 1.2, make sure that the site supports TLS 1.2. For more information about TLS support, see Enable TLS and SSL support in SharePoint.
- If the site is in a content source, examine the crawl settings in the content source. For example, examine the page depth and number of server hops.
References
Manage crawling in SharePoint Server
More information
Still need help? Go to SharePoint Community.
Feedback
Coming soon: Throughout 2024 we will be phasing out GitHub Issues as the feedback mechanism for content and replacing it with a new feedback system. For more information see: https://aka.ms/ContentUserFeedback.
Submit and view feedback for