: How to prevent Alexa and Google from indexing our stuff? My traffic is not from search engine and I want to prevent bots or others from displaying or accessing it. What would be a good .htaccess

@Turnbaugh106

All of those htaccess parameters are just requests, Google etc can still crawl your site since it is on the internet and publicly available. You should try to define what you wnat to allow them to crawl in a robots.txt file

This might help yoast.com/x-robots-tag-play/

Vote Up Vote Down

@Eichhorn148

Use a robots.txt file to prevent spiders from crawling and indexing your content.

From Alexa's "for webmasters" page:

User-agent: ia_archiver
Disallow: /


From Googe's robots.txt help page:

User-Agent: Googlebot
Disallow: /


If you want to block all robots and prevent all spiders from crawling and indexing your content:

User-Agent: *
Disallow: /


Alexa does not need to crawl or index your content to display an "Alexa Ranking" for your website. It bases its rankings on the number of people that visit your website. It gets this data from its toolbar or by purchasing data from ISPs. As such, blocking the crawler will not prevent Alexa from displaying the ranking. There does not appear to be a way to have your site opt-out of the Alexa rankings.

Vote Up Vote Down

@Lee4591628

To prevent all robots from indexing a page on your site, place the following meta tag into the section of your page:


For more info read : support.google.com/webmasters/answer/93710?hl=en
I don't think its possible to prevent alexa from indexing a page. Since its indexing a page from it's toolbar, i don't know is it possible to prevent that..

Vote Up Vote Down