: Should I use nofollow on internal links that don't need to be crawled? We have a product navigation engine that allow our customers to filter products based on attributes (size, color, etc.).

@Lengel546

Since you mentioned the pages can be "exponentially different" you might want to consider using the "follow, noindex" meta tag, the different classifiers can provide Google with additional information about the product so it is potentially beneficial to let them know about the additional content even if you don't want it indexed (this can help for long tail queries).

Vote Up Vote Down

@Kevin317

If you're sure Google is reaching the all of your products, and it seems like you are since you're using canonical URLs, then it's a good idea to prevent Google from crawling those pages since all it will do is eat up your bandwidth and server resources.

Using nofollow on any internal links is a good start. I'd also consider using the robots meta tag or x-robots-tag HTTP header to tell them not to index it, too. That way you're sure even if they find those URLs through external linking or otherwise that they will definitely not index those page. Blocking them via robots.txt is also a good thing to since you can use wildcards to catch the pages with query strings, etc.

Vote Up Vote Down

@Bryan171

Nofollow doesn't stop Google from crawling pages, it stops any 'link juice' being passed to those pages.

Robots.txt is what you want to prevent google from crawling pages.
www.robotstxt.org/robotstxt.html

Vote Up Vote Down