: Hiding search pages via robots.txt I have full text search engine installed on my site, accessible from URL like /search/<search-query>, and search landing in /search, containing large "Search"

@LarsenBagley505

Indexing search result pages isn't always the best idea - check this article from Matt Cutts
quote: "Typically, web search results don’t add value to users, and since our
core goal is to provide the best search results possible, we generally
exclude search results from our web search index."

It's quite possible that your search result pages do offer added value for visitors - so difficult to judge whether or not you should block them in the robots.txt. If it's just a list with articles on the same topic (similar to /tag/ pages) - I would consider to put them on noindex / follow.

Vote Up Vote Down