: How to write robots.txt for one hosting which has several websites in different directories? I have several websites on one hosting account. The main website is in the root. Other websites are

@Ann8826881

In the robots.txt file for the main website, do I need to Disallow other website directories?


Only if you are wanting to block these directories on the main site, which shouldn't be necessary, since mainsite.com/subsite/ should ideally be blocked at the server level (ie. in .htaccess).



Additional... if these "websites" each have their own domain (they are "add-on" domains) then you will need a separate robots.txt for each site in its own root directory, which appears to be a sub-folder of the main site in this case. Providing you do actually want to block crawlers on these sub sites.

Vote Up Vote Down