: Do I need disallow other websites directory in robots.txt? I have a hosting which has several websites in www folder. I set up a WordPress website in root directory for main domain. Do I need

I have a hosting which has several websites in www folder. I set up a WordPress website in root directory for main domain.

Do I need disallow other websites directory in robots.txt? I think these directories are not relate to the main domain. But is this will totally disallow other website?

10.01% popularity Vote Up Vote Down

: You will need to add an MX record to your domains DNS. If you're using your Registrar to do the email forwarding, you'll add their e-mail servers for the MX records. Without an MX record,

@Megan663

0 Comments

: Possible to add code tag button in MediaWiki editor? In the MediaWiki editor are there buttons for bold, italic and other styling, but not one for adding the <code> ... </code> tags

@Megan663

Posted in: #Mediawiki

1 Comments

: How much time does Googlebot take to crawl the page after submitting it to index I've a daily changing job site and I've also setup a dynamically generated XML sitemap for the search engines.

@Megan663

Posted in: #Googlebot #GoogleSearchConsole

3 Comments

: Plus sign in front of URLs in user agents I run a small web crawler and had to decide on what user agent to use for it. Lists of crawler agents as well as Wikipedia suggest the following

@Megan663

Posted in: #Http #UserAgent #WebCrawlers

2 Comments

Login to post a comment!

1 Comments

Sorted by latest first Latest Oldest Best

@Radia820

You could use robots.txt to block the directories but there is an easier way by using canonical pages on your websites.

This will ensure sites that are hosted within containers within 1 hosting account for example:

/public_html/ = domain1.com
/public_html/domain2.com = domain2.com would only be indexed on the domain that the canonical is setup on.

10% popularity Vote Up Vote Down

Feed

: Do I need disallow other websites directory in robots.txt? I have a hosting which has several websites in www folder. I set up a WordPress website in root directory for main domain. Do I need

More posts by @Megan663

: You will need to add an MX record to your domains DNS. If you're using your Registrar to do the email forwarding, you'll add their e-mail servers for the MX records. Without an MX record,

: Possible to add code tag button in MediaWiki editor? In the MediaWiki editor are there buttons for bold, italic and other styling, but not one for adding the <code> ... </code> tags

: How much time does Googlebot take to crawl the page after submitting it to index I've a daily changing job site and I've also setup a dynamically generated XML sitemap for the search engines.

: Plus sign in front of URLs in user agents I run a small web crawler and had to decide on what user agent to use for it. Lists of crawler agents as well as Wikipedia suggest the following

Login to post a comment!

1 Comments

Back to top | Use Dark Theme