: How do I stop Google from indexing AJAX calls within Javascript? We are getting a number of odd results come up on Google EG: http://www.somedomain.com/somepage/AjaxFunction.aspx?stuff=XXX&other=XXX

@BetL925

I would put the Ajax function into robots.txt:

Disallow: /somepage/AjaxFunction.aspx


That will prevent Google from crawling it. Google doesn't typically index URLs it can't crawl. It will only index them if they are linked prominently, especially numerous external links. Even if Google does index the URL, it won't index the content of the URL. Google will show "this page was blocked by robots.txt" in the search results.

In many cases it is actually desirable to allow Google to crawl Ajax URLs. Those URLs may provide content that you want Google to index after JavaScript writes it into another page. In that case, robots.txt is not appropriate. You just don't want Google including the Ajax URL itself in the search results. You can use a header directive for that:

X-Robots-Tag: noindex


On an Apache server you can add that header with .htaccess code like:

<Files "AjaxFunction.aspx">
Header append X-Robots-Tag "noindex"
</Files>


In aspx code you can set it like this:

<% Response.AddHeader "X-Robots-Tag", "noindex" %>

Vote Up Vote Down