: Should I prevent search engines indexing tag/category pages? On my site, I currently have no special rules for search engines. It is a blog, statically generated using a Python program. When
On my site, I currently have no special rules for search engines. It is a blog, statically generated using a Python program. When I search for some of my articles on Google, there is usually a tag or category page included in the results. Sometimes it even ranks ahead of the article itself. Obviously, as these links aren't always going to have the article on them, this aren't the results I want people to click on.
So, I'm thinking of setting noindex on these pages. Is there any possible downside to doing so? Is this possible to do via robots.txt, or do I have to add it to all the relevant templates? All I can find for robots.txt are ways to stop the search engine crawling those pages, which isn't what I want - while I don't want them indexed, it's still the only surefire way to find all my blog posts.
More posts by @Ogunnowo487
2 Comments
Sorted by latest first Latest Oldest Best
No. Read this(all of it; there's a lot of useful stuff), though of particular relevance here:
Google no longer recommends blocking
crawler access to duplicate content on
your website, whether with a
robots.txt file or other methods.
[...]
Duplicate content on a site is not
grounds for action on that site unless
it appears that the intent of the
duplicate content is to be deceptive
and manipulate search engine results.
Search engines are aware that sites(especially blogs) have pages which are basically just listings of excerpts. They're looking for more than that a far as duplicate content.
My first answer is: why would you want less exposure on Google?
Surely people will still get to the article, even if they have to go via the listing page. And that is a good thing.
Maybe rather than stopping Google from indexing certain pages, why not do some work on the articles to get them higher. Look at the text on the pages as well as the keywords, title and description (all basic SEO stuff that I'm sure you know).
Terms of Use Create Support ticket Your support tickets Stock Market News! © vmapp.org2024 All Rights reserved.