: How can I make Google crawl content behind an AJAX load more button I have a feed that has 10 questions and answers loaded when you first open the page. And then I have a "Load More" button

@Welton855

Why not have a normal "fallback"? E.g.

<a onClick="ajaxPage(2); return false;" href="app/page/2">Page 2</a> |
<a onClick="ajaxPage(3); return false;" href="app/page/3">Page 3</a>


You can make this hidden (I wouldn't)...

Vote Up Vote Down

@Hamaas447

I think this is what you're looking for:
Google's AJAX Crawling: Guide for Webmasters and Developers

Here's the full specification

I think a jQuery plugin called BBQ is worth being familiar with too (it sounds similar to history.js)

That said, AJAX and JS in general can be problematic for accessibility and who's to say how other search engines might handle it as this is Google's method.

I'd definitely consider making an easy to find link to a feed or a place where the content is on static pages (you could then use robots.txt to block Google from it so you wouldn't risk a duplicate content penalty...more applicable to pages than feeds).

Vote Up Vote Down

@Heady270

You should provide a mechanism other than pagination to reach all the questions and answers on your site. Pagination is a poor navigation technique because:


It is rarely used by visitors. Usually only 2% of visitors actually use pagination.
Using AJAX for it makes it uncrawlable
Even without AJAX, it causes PageRank to dissipate so quickly that by page 4, none of the questions get any juice.


Instead you should:


Create a sitemap XML file that lists all your content and submit it in Google Webmaster Tools.
Use tags to group you questions by theme. Have tag pages that list questions in each tag.
Link questions directly to other questions using "related" links or something similar.
Consider expanding the number of items on the first page of pagination. If you have 100 items per page, that means that search engines will be able to find 10 times the amount of content from the first page alone.

Vote Up Vote Down

@Angela700

The best option is to have a noscript fallback to regular pagination. The search engine will pick up on the links in the noscript section and index those pages. This has the added benefit that anyone browsing your site without JavaScript enabled will get a better user experience.

Vote Up Vote Down

@Heady270

If you are using the pagination with basic reloading then it will automatically crawl by search engines(then it should have different URLs for each page).

When you are using AJAX to load more content to the same page then above case will not work, but you can use an alternatives like dynamically change the URL with AJAX (without # append to the URL because when you append # to the URL, after the # will not crawl the search engines).

You can achieve it with history.js jQuery library it will work only with HTML5 supported browsers i.e. IE # symbol is appended.

By using this method you will get different URLs for each page then search engine will index your page content.

Also make sure you have added all those urls in sitemap.xml.

Vote Up Vote Down