: Should I use "Disallow", "NoFollow", or "NoIndex" for paginated listing pages? I own a web site which has mainly two types of pages: Content pages: they are pretty much static, and link to
I own a web site which has mainly two types of pages:
Content pages: they are pretty much static, and link to the previous and next content page (chronologically). The structure is that of a double linked list.
Listing pages: they are dynamic, and are paginated. The first one shows the most recent content. Older content is paginated via a GET parameter (e.g ?page=4). Every listing page links to several content pages.
Google is indexing some of the listing pages, which will most likely be outdated when the users find them (new content is added at a pretty fast rate). This is undesirable, of course. Also, listing pages have extracts of the content, which could be seen as repeated content from Google's point of view (which I think is also undesirable, I'm unsure on this).
The problem is, there are hundreds of thousands of content pages. Listing pages are very convenient both for visitors and for search engines to crawl the site. I am unsure if the linked list of content pages will be sufficient to allow search engines to crawl the site in a short time.
I have thought about the following options:
Add Disallow: /?page* to robots.txt: listing pages (other than the first one) won't be accessed.
Add meta nofollow to listing pages: similar to the first one, listing pages won't be accessed.
Add meta noindex to listing pages: they will be accesses, but not added to the index
Leave things as they are: eventually search engines will realize the 'page' parameter is for pagination, and won't index it.
What are your suggestions? Adding the NoIndex meta to listing pages sounds like the best solution to me, but I don't dare approach such a large task without asking first :)
Any other possible solutions I didn't think of?
More posts by @Lengel546
1 Comments
Sorted by latest first Latest Oldest Best
I'd say one of the most common practices hear would be adding noindex to the listing pages, apart from the first one, so as you said they will still get crawled but will not get added to the index. I think this is preferable to blocking them in robots.txt as Google is still free to crawl through the site and can still follow links on the pages.
Adding nofollow wouldn't be much use especially if the pages are already indexed - they will remain indexed and if they aren't indexed they will still most likely get indexed in Google if they are linked to else where. Matt Cutts also mentions that you should try and avoid nofollowing internal links Should I use rel="nofollow" on internal links
There is another option, which is close to your 'let it be' solution, which is using rel=”next” and rel=”prev” html tags to indicate the relationship between component URLs in a paginated series.This helps Google accurately index your content and serve to users the most relevant page (commonly the first page). A brief explanation below:
www.example.com/article?story=abc&page=1 http://www.example.com/article?story=abc&page=2 www.example.com/article?story=abc&page=3 http://www.example.com/article?story=abc&page=4
On the first page, www.example.com/article?story=abc&page=1, you’d include in the <head> section:
<link rel="next" href="http://www.example.com/article?story=abc&page=2" />
On the second page, www.example.com/article?story=abc&page=2:
<link rel="prev" href="http://www.example.com/article?story=abc&page=1" />
<link rel="next" href="http://www.example.com/article?story=abc&page=3" />
On the third page, www.example.com/article?story=abc&page=3:
<link rel="prev" href="http://www.example.com/article?story=abc&page=2" />
<link rel="next" href="http://www.example.com/article?story=abc&page=4" />
And on the last page, www.example.com/article?story=abc&page=4:
<link rel="prev" href="http://www.example.com/article?story=abc&page=3" />
More info can be found in Google guidelines Pagination with rel=“next” and rel=“prev” and there is a good guide here Conquering Pagination – A Guide to Consolidating your Content
Terms of Use Create Support ticket Your support tickets Stock Market News! © vmapp.org2024 All Rights reserved.