Mobile app version of vmapp.org
Login or Join
Goswami781

: Is there any advantage/disadvantage to using robots.txt to disallow access to legal pages such as terms, privacy policy, etc.? As I understand, having repetitive content is a detriment to search

@Goswami781

Posted in: #Legal #PrivacyPolicy #RobotsTxt #Seo #TermsOfUse

As I understand, having repetitive content is a detriment to search engine placement.

Given that many websites that use similar or even identical "Terms and Conditions" and "Privacy Policy" pages due to similar legal wording or due to copy & pasting from the same source, would it be a good idea to disallow access to these pages via robots.txt, in order to avoid being penalized for "non-original content"?

Or, on the contrary, could the search engines identify this as circumvention and penalize the site for trying to hide content?

Or does it not matter?

10.02% popularity Vote Up Vote Down


Login to follow query

More posts by @Goswami781

2 Comments

Sorted by latest first Latest Oldest Best

 

@Chiappetta492

No, it won't matter that much.

Matt Cutts recommends not focusing too much on this, but on your content:

10% popularity Vote Up Vote Down


 

@BetL925

On the one hand, Google probably doesn't want to index these pages on your site. Most of the words in these pages aren't going to be relevant to the rest of your site. If somebody searches for these words, your site will not make a great landing page.

On the other hand, so many sites have these types of pages, that it isn't going to cause a problem for Google to let them crawl them. Other sites allow them to be crawled, your site can too. Here are some good reasons that you might want to have them crawled:


There are some terms for which you would like have them show up in search: "contact <brand>" or "<brand> TOS".
They often make good "site links" (the links that Google shows under your listing when somebody searches for your brand).
Just having these pages is the mark of a reputable site. You certainly want Google to know that you have these pages so that they consider your site reputable.


Your site wouldn't be penalized for putting them in robots.txt, but overall having these pages crawlable will help your site.

10% popularity Vote Up Vote Down


Back to top | Use Dark Theme