: How to get Google to crawl AJAX pages without using #! URLs? In Google's advice on "Making AJAX Applications Crawlable", they advise making AJAX URLs that aren't using hash-bang fragments (#!)

@Reiling115

check this thread in Google forums, AJAX Crawling working with hash but not meta tag. There is an issue with google Fetch tool.

Vote Up Vote Down

@Berumen354

In the preceeding sections on allowing AJAX content to be crawled, Google states:



3.Handle pages without hash fragments

Some of your pages may not have hash fragments. For example, you might want your home page to be example.com, rather than example.com#!home. For this reason, we have a special provision for pages without hash fragments.


Note: Make sure you use this option only for pages that contain dynamic, Ajax-created content. For pages that have only static content, it would not give extra information to the crawler, but it would put extra load on your and Google's servers.


In order to make pages without hash fragments crawlable, you include a special meta tag in the head of the HTML of your page.

The meta tag takes the following form:

<meta name="fragment" content="!">


This indicates to the crawler that it should crawl the ugly version of this URL. As per the above agreement, the crawler will temporarily map the pretty URL to the corresponding ugly URL. In other words, if you place into the page example.com, the crawler will temporarily map this URL to example.com?_escaped_fragment_= and will request this from your server. Your server should then return the HTML snapshot corresponding to example.com. Please note that one important restriction applies to this meta tag: the only valid content is "!". In other words, the
meta tag will always take the exact form: <meta name="fragment" content="!">, which indicates an empty hash fragment, but a page with AJAX content.



Therefore, you still need to enable the mapping of the URL to the 'ugly' version containing ?_escaped_fragment_= using the headless browser.

They follow-up with the warning:


Should the content for example.com?_escaped_fragment_= return a 404 code, no content will be indexed for example.com!

So you should test for that as well.

Vote Up Vote Down