: Redirecting robots.txt when moving to a new domain I'm moving a site to a new domain, to a local tld. I already redirected everything successfully to the new domain, except the robots.txt. My

@Odierno851

This is really interesting question, It took me few minutes to search official statement from gary tweet..

When Google see 404/30X status, then Google will ignore your robots.txt completely. It is like not having a robots.txt at all.

But if Google see 50X status or server side errors, then Google will stop crawling your site, because may be you're busy to configure your server.

So do not redirect your robots.txt or if you do then make sure you've block same thing in new site robots.txt

Vote Up Vote Down

@Margaret670

The robots.txt on the old domain is now useless cause it helps google crawling the current domain website.

So if you correctly redirect olddomain.com/* to newdomain.com/*
The robots.txt will be forwared to, and you should put approriate rules in your new robots.txt (change the content if you also change the cms or url format ...)

hope it helps ...

Vote Up Vote Down