: How to prevent Google from indexing a domain? I have a few domains that I used for developing and testing websites. As it usually has the actual content of the production site, I don't want
I have a few domains that I used for developing and testing websites. As it usually has the actual content of the production site, I don't want Google to index those domains ever.
I don't want to use robots.txt as a solution. I'm constantly deleting and reinstalling everything on these domains, robots.txt included. I don't want to set some sort of HTTP status code, because 200 OK is what I need during testing. Password protecting the site also is a pain because clients get confused, it prevents me from certain testing, etc.
Is there something in Webmaster Tools that I can use to tell Google not to index the site? Also, how do I remove any and all of the pages Google has indexed so far?
More posts by @Reiling115
1 Comments
Sorted by latest first Latest Oldest Best
Honestly if you recreate everything each time, I do not see why the robots.txt is not just reconstructed too. The part to prevent Googlebot is static and just needs to be there when you restore the site.
While Google Search console will not help getting the site in index, it will confirm if you have done so by listing the pages as Blocked Resources.
There several ways to block resources:
Use tags like: <meta name="robots" content="noindex"> See info by Google.
robots.txt which you want to avoid but it designed exactly for that and, unlike using Meta goes not have Googlebot fetch to discard, saving you some bandwidth and requests.
Sent a 403 Forbidden just to Googlebot (or bots in general). You are generally not testing as a bot or at least not Googlebot, so that status will be hidden from your users and you are free to sent a 200 OK status to humans.
Terms of Use Create Support ticket Your support tickets Stock Market News! © vmapp.org2024 All Rights reserved.