: Is there any advantage/disadvantage to using robots.txt to disallow access to legal pages such as terms, privacy policy, etc.? As I understand, having repetitive content is a detriment to search

@Chiappetta492

No, it won't matter that much.

Matt Cutts recommends not focusing too much on this, but on your content:

Vote Up Vote Down

@BetL925

On the one hand, Google probably doesn't want to index these pages on your site. Most of the words in these pages aren't going to be relevant to the rest of your site. If somebody searches for these words, your site will not make a great landing page.

On the other hand, so many sites have these types of pages, that it isn't going to cause a problem for Google to let them crawl them. Other sites allow them to be crawled, your site can too. Here are some good reasons that you might want to have them crawled:


There are some terms for which you would like have them show up in search: "contact <brand>" or "<brand> TOS".
They often make good "site links" (the links that Google shows under your listing when somebody searches for your brand).
Just having these pages is the mark of a reputable site. You certainly want Google to know that you have these pages so that they consider your site reputable.


Your site wouldn't be penalized for putting them in robots.txt, but overall having these pages crawlable will help your site.

Vote Up Vote Down