: Google lists redirected URL instead of target URL We've recently moved much of our site to a Web CMS which published to www1 rather than the old www. To clean up the search results,
We've recently moved much of our site to a Web CMS which published to www1 rather than the old
To clean up the search results, we've added an exclusion in robots.txt for the old hostname and a server redirect on the old homepage from www to www1:
RedirectMatch 301 /$ www1.example.com/ RedirectMatch 301 /index.htm www1.example.com/
This has worked well for search results, removing many outdated pages from the index, but when searching for the main site we see the old www URL listed, followed by the message:
A description for this result is not available because of this site's robots.txt.
What we really want is www1 URLs in the search result (and no message about robots.txt). Any ideas?
More posts by @Shakeerah822
1 Comments
Sorted by latest first Latest Oldest Best
If you have a robots.txt on the www server that prevents crawling there, then Googlebot will never see the 301 redirects that you put in place. It won't be able to crawl the URLs that are redirecting.
You want to remove the www site from robots.txt and let Google crawl it. Then they will know what the correct canonical URL for your website is now.
Terms of Use Create Support ticket Your support tickets Stock Market News! © vmapp.org2024 All Rights reserved.