: Remove subdomains from Google index and stop indexing them I am running static content through a CDN, cdn1-cdn5 I am using such subdomains for that. I am loading just images, CSS and JS files

@LarsenBagley505

You can remove the sub-domains in webmaster tools, but first you need to add the sub domains as seperate sites and then submit a site removal. They should be gone within a day or so.

See these instructions for removing a site from google : support.google.com/webmasters/answer/1663427?hl=en

Vote Up Vote Down

@Jamie184

Short answer.

Put <meta name="robots" content="noindex"> in the header of your HTML for all pages. Once the search engines have spidered these pages and you are sure of it, put

User-agent: *
Disallow: /


...in a robots.txt file in the root directory of each sub-domain.

This will take time of course. It can take 30-60 days typically for say Google to notice the changes and reflect it in the SERPs. It can take less or more time depending upon how Googles gauges freshness for your sub-domains.

Vote Up Vote Down

@Yeniel560

There are different way, here you have some, you can use only one or combine them


Use rel="canonical". More examples and details
If you can use a .htaccess file, set a 301 redirect on the servers that you don't want to get indexed.


About the robots.txt, you can use it, but it's much better option to use a solution that is more robust and that all crawlers will have to follow, like the redirect.

Here you can see a short video from Matt Cutts talking about 301 redirects vs rel="canonical". A short extract from that page and video would be:


Okay, I sometimes get a question about whether Google will always use the url from rel=canonical as the preferred url. The answer is that we take rel=canonical urls as a strong hint, but in some cases we won’t use them:


For example, if we think you’re shooting yourself in the foot by accident (pointing a rel=canonical toward a non-existent/404 page), we’d reserve the right not to use the destination url you specify with rel=canonical.
Another example where we might not go with your rel=canonical preference: if we think your website has been hacked and the hacker added a malicious rel=canonical. I recently tweeted about that case. On the “bright” side, if a hacker can control your website enough to insert a rel=canonical tag, they usually do far more malicious things like insert malware, hidden or malicious links/text, etc.



On the video he mentions some more situations and reasons, like the fact that 301 has to be followed by everybody.

Vote Up Vote Down