: Prevent Google Indexing SubDomains I have cPanel Hosting with only 1 IP. I'm in the process of pointing all my TLDS over to this server. I have created 'Addon Domains' and all the folders

@Jamie184

Google, like all decent web crawlers, honors Robots Exclusion Standard (robots.txt). If there really is a limitation not to use separate folders for every domain and you don't want to treat all domains as cPanel "Add on domains", you could prevent the indexing of these folders (but not access to the folders) by adding /home/user/public_html/client/robots.txt:

User-agent: *
Disallow: /site1-directory/
Disallow: /site2-directory/


As the additional domains have different roots, this robots.txt will only affect the main site.

Vote Up Vote Down

@Megan663

If you don't use cPanel, the correct way to do this is to put each of domain into is very own directory. Apache is not really designed to handle overlapping domains the way that cPanel sets things up. This seems to be a limitation of cPanel.

One technique that should work is to treat all of your domains as "add on domains." Set your main domain name in cPanel to a subdomain that doesn't actually have a DNS entry. Something like allmysites.example.com. Then Google can't actually crawl the main directory at all. Your directory structure would then look like:


allmysites (serves allmysites.example.com which isn't accessible through DNS)

site1 (serves site1.example.com)
site2 (serves site2.example.com)
site3 (serves site3.example.com)

Vote Up Vote Down