: Does Alexa's crawler need a robots.txt on each sub domain? I have a robots.txt on my www site. Do I also need one on other sites like my app and forums sites? Alexa audit keeps finding bad

I have a robots.txt on my www site. Do I also need one on other sites like my app and forums sites?

Alexa audit keeps finding bad links, missing tags, and other "errors" on my vanilla forums site even though the www site's robots.txt says to ignore specific directories that are on my app and forum sites.

10.01% popularity Vote Up Vote Down

: SEO executive .htaccess file knowledge Should SEO executive in a company have knowledge of .htaccess file to make changes in that file or it is the task of developer in the company to make

@Samaraweera270

Posted in: #Htaccess #Seo

2 Comments

: SEO: Merge two websites My freind has a website in the same activity (Real estate) as my website. He wants to close his business + his website but before to do that, we want to see if is

@Samaraweera270

Posted in: #Seo

0 Comments

: New content is showing up in a site: search for my old domain name, despite 301 redirects to new domain So, I changed my website domain one week ago, made 301 redirect from the old domain

@Samaraweera270

Posted in: #GoogleSearch

0 Comments

: Are there settings that must go into the IIS ApplicationHost.config file only? Are there setting that must go into ApplicationHost.config opposed to into web.config file under the IIS web server?

@Samaraweera270

Posted in: #Iis #Iis7

0 Comments

Login to post a comment!

1 Comments

Sorted by latest first Latest Oldest Best

@Jamie184

The first thing you have to remember is that example.com, example.com, sub-domain.example.com, even HTTP versus HTTPS are all different sites.

Robots.txt cannot reference other sites, it is only for the site it resides upon. So for any sub-domain you will need a robots.txt just for that site.

10% popularity Vote Up Vote Down

Feed

: Does Alexa's crawler need a robots.txt on each sub domain? I have a robots.txt on my www site. Do I also need one on other sites like my app and forums sites? Alexa audit keeps finding bad

More posts by @Samaraweera270

: SEO executive .htaccess file knowledge Should SEO executive in a company have knowledge of .htaccess file to make changes in that file or it is the task of developer in the company to make

: SEO: Merge two websites My freind has a website in the same activity (Real estate) as my website. He wants to close his business + his website but before to do that, we want to see if is

: New content is showing up in a site: search for my old domain name, despite 301 redirects to new domain So, I changed my website domain one week ago, made 301 redirect from the old domain

: Are there settings that must go into the IIS ApplicationHost.config file only? Are there setting that must go into ApplicationHost.config opposed to into web.config file under the IIS web server?

Login to post a comment!

1 Comments

Back to top | Use Dark Theme