Mobile app version of vmapp.org
Login or Join
Reiling115

: Robots.txt for website hosted in a subdirectory I have 2 website which is hosted in a shared hosting. 1st website example.com hosted in a root directory as /public_html/ 2nd website example2.com

@Reiling115

Posted in: #Googlebot #RobotsTxt #WebCrawlers

I have 2 website which is hosted in a shared hosting.
1st website
example.com hosted in a root directory as /public_html/

2nd website
example2.com hosted in a sub-directory directory as /public_html/example_2/

I have this in example.com/robots.txt

User-agent: *
Disallow: /example_2/
sitemap: www.example.com/sitemap.xml

I have this in example2.com/robots.txt

User-agent: *
sitemap: www.example2.com/sitemap.xml

I want google to index both the website separately but do not mixed each another, what I know is if a website is submitted then google scrap whole directory, except the Disallow one; So if I Disallow: /example_2/ will it effect example2.com. If so then what is the best way to write robots.txt for 2 website.

I'll be very great-full if some one can show me the right path.

10.01% popularity Vote Up Vote Down


Login to follow query

More posts by @Reiling115

1 Comments

Sorted by latest first Latest Oldest Best

 

@Jamie184

It appears that for cPanel, and perhaps others, certainly to be expected with web hosts these days, while creating a second or third web site or even a sub-domain, cPanel in order to save on setting file permissions, creates any additional site within the web space of the first site created. This may be a result of the old Apache schema using httpd.conf and compatibility. Who knows? The result is that there is confusion and potential problems that can arise.

/public_html/ is the original sites web space for example.com

/public_html/example_2/ is the second site for example2.com

For example.com, the directory directive within the configuration file points to /public_html/ and for example2.com, the directory directive within the configuration file points to /public_html/example_2/.

These are local directories on your file system and not how things are seen from the web.

What is put into /public_html/ such as index.html will be seen as example.com/index.html and what is put into /public_html/example_2/ such as index.html will be seen as example2.com/index.html.

The same will be true for robots.txt and any .htaccess file you choose to create. Example.com robots.txt would go into /public_html/ and example2.com robots.txt would go into /public_html/example_2/.

The drawback to this configuration is that example.com/example_2/index.html will show the sites home page for example2.com/index.html. This is generally not a problem. Just something to remember.

10% popularity Vote Up Vote Down


Back to top | Use Dark Theme