: Should I exclude file with duplicate content from sitemap? I have a site where the same content can be found in different ways, for example from a search. The canonical URL is always a URL
I have a site where the same content can be found in different ways, for example from a search.
The canonical URL is always a URL with 'product-item', but other ways [to get to same content] might end up as a URL with 'search-item' (even though it has the product-item canonical URL).
Should I exclude search-item (say, in robots.txt) so bots don't crawl that potential dupe content, and also exclude from sitemap files? Or does having the canonical URL take care of this?
More posts by @Shelton105
1 Comments
Sorted by latest first Latest Oldest Best
Yes, you should exclude the duplicate content from sitemap, you need to add only your preferred url for duplicate webpages, as Google says:
Pick a canonical (preferred) URL for each of your pages, and tell us
about your preference by submitting these canonical URLs in a sitemap.
We don't guarantee that we'll use the URLs you submit in a sitemap,
but submitting one is a useful way to tell Google about the pages on
your site you consider most important.
Terms of Use Create Support ticket Your support tickets Stock Market News! © vmapp.org2024 All Rights reserved.