: To disallow indexing the category and tag listings in a blog Mark Wilson says that category and tag listings in a blog should be disallowed in order to prevent duplicate content. I understand

@Jessie594

Google Webmaster Central says


Google no longer recommends blocking
crawler access to duplicate content on
your website, whether with a
robots.txt file or other methods.


So then I don't need to disallow crawling of category/tag listings in my blog. Duplicate content is penalized only when used for malicious purposes:


In the rare cases in which Google
perceives that duplicate content may
be shown with intent to manipulate our
rankings and deceive our users, we'll
also make appropriate adjustments in
the indexing and ranking of the sites
involved. As a result, the ranking of
the site may suffer, or the site might
be removed entirely from the Google
index, in which case it will no longer
appear in search results.

Vote Up Vote Down