: When you move a site via a 301 redirect should you setup a robots.txt disallowing robots to crawl the old address? A site I am working on moved a subdomain to another subdomain via a 301

A site I am working on moved a subdomain to another subdomain via a 301 redirect. However when checking robots.txt of the old subdomain, it has a robots.txt which disallows search engine web crawlers to crawl it.

Is this the right move? I believe no because the crawlers won't be able to crawl the old site to see the 301 status codes.

10.03% popularity Vote Up Vote Down

: Adding Schema.org’s "WPHeader" with Microdata I'd like to add Schema.org data, and am trying to follow instructions given to me about how to do so. The instructions indicate that I should

@Annie201

Posted in: #Headers #Microdata #SchemaOrg #Seo

1 Comments

: Using HTML5 EME (encrypted media extensions) for JS I am thinking about developing a paid JS script and i wanted to prevent its piracy by using HTML5's EME to safely decrypt and use my javascript

@Annie201

Posted in: #Html5 #Javascript

0 Comments

: One of our DNS servers is hosted by our old host and one by the new. The old doesn't have the correct values and can't be changed Because of misbehavior of our old hoster justhost.com, our

@Annie201

Posted in: #Dns #DnsServers #WebHosting

1 Comments

: Can't add preferred version to Google Webmasters / Search Console When I select example.com as my preferred site version like below. It gives me an error as seen in the image. The site was

@Annie201

Posted in: #GoogleSearchConsole

1 Comments

Login to post a comment!

3 Comments

Sorted by latest first Latest Oldest Best

@Megan663

I would keep the robots.txt as is, but make sure that the old and new pages have the appropriate version of Canonical tags. This will help the search engine make sense of what has gone on.

10% popularity Vote Up Vote Down

@Welton855

Your suspicions are right, with the exact reasoning that you mentioned. If you disallow robots from accessing the old site, they won't be able to see the 301 redirect.

10% popularity Vote Up Vote Down

@Holmes151

I believe no because the crawlers won't be able to crawl the old site to see the 301 status codes.

Yes, exactly - this is not the "right move". If you implement a 301 redirect, presumably to preserve SEO, then blocking the URLs that are being redirected is going prevent the search engines from seeing the redirect and indexing the new content.

10% popularity Vote Up Vote Down

Feed

: When you move a site via a 301 redirect should you setup a robots.txt disallowing robots to crawl the old address? A site I am working on moved a subdomain to another subdomain via a 301

More posts by @Annie201

: Adding Schema.org’s "WPHeader" with Microdata I'd like to add Schema.org data, and am trying to follow instructions given to me about how to do so. The instructions indicate that I should

: Using HTML5 EME (encrypted media extensions) for JS I am thinking about developing a paid JS script and i wanted to prevent its piracy by using HTML5's EME to safely decrypt and use my javascript

: One of our DNS servers is hosted by our old host and one by the new. The old doesn't have the correct values and can't be changed Because of misbehavior of our old hoster justhost.com, our

: Can't add preferred version to Google Webmasters / Search Console When I select example.com as my preferred site version like below. It gives me an error as seen in the image. The site was

Login to post a comment!

3 Comments

Back to top | Use Dark Theme