: Google INDEED crawls ajax? I don't understand how this didn't make a bigger BOOM online. It is well known in the industry that google does not index AJAX (just google-search "google crawls ajax")

Posted in: #Ajax #Google #Javascript #Seo #WebCrawlers

I don't understand how this didn't make a bigger BOOM online.

It is well known in the industry that google does not index AJAX (just google-search "google crawls ajax") and see dozens of articles stating that google does not crawl ajax by default.

There's a whole Google AJAX Crawling Guidelines thing going on, if you want to help google crawl your AJAX based content (by creating mirror pages for the search engine).

But 4 days ago, this technique had been deprecated.

Now, in their blog post about it Google explains that

Back then, our systems were not able to render and understand pages that use JavaScript to present content to users. [...] Times have changed. Today, as long as you're not blocking Googlebot from crawling your JavaScript or CSS files, we are generally able to render and understand your web pages like modern browsers.

(googlewebmastercentral.blogspot.com/2015/10/deprecating-our-ajax-crawling-scheme.html)

This is BIG and I don't see it getting attention online.

This is also a big f**k-up for me personally.

A couple of years ago I started using AJAX for loading user comments on my website, knowing that google does not crawl ajax.

Now I looked at a couple of my website's pages that are seemingly banned, or over-looked by Google.

And to my surprise, lots of these exact pages contain bad quality content (keyword stuffing etc.) within the comments.

How can one know if his AJAX based content is being crawled?

Using the "Fetch as Google" tool will NOT show the AJAX based content. Also, google-searching for exact phrases that appear in the AJAX based content does not return results.

The whole subject is very confusing.