: Deindexing a whole domain with robots.txt created problems in google webmaster tools In my robots.txt for my domain that just hosts images, I used the following lines: user-agent: * noindex: /

@LarsenBagley505

Posted in: #Google #Googlebot #GoogleIndex #Images #Indexing

In my robots.txt for my domain that just hosts images, I used the following lines:

user-agent: *
noindex: /


I thought I was doing search engines a wonderful favor by not having them crawl through their database of URLs that are part of my site but instead, only crawl at one file to get the hint. The problem is that google has a problem.

In google webmaster tools, in the domain menu the image hosting domain has a red exclamation mark to the left of it and to the right of it, instead of seeing "No new messages or recent critical issues." I instead see "Check property health". I click on it and see "Some important page is blocked by robots.txt.". I then click on "Some important page" which directs me to my image hosting domain name itself. The thing is that link is supposed to direct visitors to the other page content domain.

I understand I could do a 301 redirect from the image domain root to the right place but that doesn't concern me. Personally I'd rather have everything on that image domain de-indexed including the root.

I double-checked the sitemap section and google states that no sitemaps have been submitted. When I check the index status, google states I have one URL indexed on that image domain.

The only idea I have in mind is to somehow modify noindex: / so that it references everything but the root folder in the domain and make the default page include a no-index directive in the HTML.

Does anyone have a better solution to this so google doesn't complain?

The idea is to have the image domain serve strictly images to the pages accessible at the content domain.

Vote Up Vote Down

Login to post a comment!