question

UmrF avatar image
0 Votes"
UmrF asked EchoDu-MSFT commented

Crawl logs showing Site URL crawled twice successfully. Search result showing duplicate results

Hello ,

when user search for the site name , it shows up twice in the search results. Both the results are for the same SharePoint site home page.
When checked in the crawl logs , i see 2 successful crawled results for the same URL with 2 different ID . If i remove one of them using "Remove item from the index" search results shows one item in the search results but at the next crawl it shows 2 items in the crawl logs and 2 in search results again.
enabling "trim duplicate search results" also does not help. My question is why is it crawling same URL twice in crawl logs with different ID , same URL and same content source as shown in the image below? Its happening on the home page of the sub sites.

Any advice where to further look for this ?
Thanks in advance.

82182-image.png


office-sharepoint-server-administrationoffice-sharepoint-server-itprooffice-sharepoint-server-search-itpro
image.png (16.5 KiB)
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

EchoDu-MSFT avatar image
0 Votes"
EchoDu-MSFT answered EchoDu-MSFT edited

Hello @UmrF ,

If Item ID is different, it means that these are two different items. Although the two items are the same and point to the same page, they will be treated as two items in the crawling.

82311-crawl1.png

If you want to exclude duplicate search results, please do the following steps:

1.Go to the results.aspx and click "Edit page"

2.Select Search Results web part and click on "Edit Web Part"

82273-search1.png

3.On the Search Results web part, click "Change query" button

82200-search2.png

4.On the Build Your Query page, go to the "SETTINGS" page and select "Remove duplicates" checkbox.

82289-search3.png

5.Apply it and Save page

82269-search4.png

Thanks,
Echo Du
===================
If an Answer is helpful, please click "Accept Answer" and upvote it.
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.





search4.png (28.8 KiB)
crawl1.png (11.0 KiB)
search1.png (35.2 KiB)
search2.png (13.9 KiB)
search3.png (33.3 KiB)
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

UmrF avatar image
0 Votes"
UmrF answered EchoDu-MSFT commented

Hi Echo,

Thanks for testing and sharing he screenshots. In my case like mentioned earlier "Remove Duplicate" is checked. it is checked by default btw, but I am still seeing duplicate results. the only time I don't see duplicate result when hard remove from index from crawl logs.

I also tried using api with "trimduplicates=true" and also used search query tool that allow you to see the additional properties of the results. In each case I am seeing link for the same page twice.

Any other tips, please share.
Thanks.

· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @UmrF ,

I suggest you reset the search index in SharePoint Server and then perform a Full crawl.

You could refer to this article to reset index.

Thanks,
Echo Du
======================
You can directly click “Comment” option under “My Answered” to put forward your opinions and thoughts about solution that I propose.




0 Votes 0 ·
UmrF avatar image
0 Votes"
UmrF answered EchoDu-MSFT commented

Yeah, resetting index might fix lots of issues but in this case thousands of sites home page is dependent on index as they are build on using search web parts. So cant take all of them down for hours. cuz recrawl in this environment can easily take 48 hours to build again.
Thanks for your suggestions thou. Any other advice will be helpful.

· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @UmrF ,

In my testing, I couldn't reproduce this issue. Under normal circumstances, there will be no duplicate search results.

In this case, I would propose a new support ticket to be raised to have a dedicated Technical Professional to support you from there. The contact number for your region could be easily found from below website, you can simply refer to the Customer Service Representative and he/she will be glad to help you with creating a new ticket.

Global Customer Service phone numbers

Thanks,
Echo Du


0 Votes 0 ·