Mobile app version of vmapp.org
Login or Join
Annie201

: Moving a site from one subdomain to another subdomain, what to do with pages which aren't "mappable"? In the process of moving a site from one subdomain to another subdomain. For a lot of

@Annie201

Posted in: #Noindex #RobotsTxt #Seo #WebCrawlers

In the process of moving a site from one subdomain to another subdomain. For a lot of the subfolders a 301 redirect to their respective pages on the new site. However what should I do with subdomains and pages on the old site which have no similar comparable page on the new site. I want Google to stop wasting crawl budget on these pages. So a robots.txt would be in order, however, I also want these pages to be dropped out of the index and robots.txt may not do that. Would noindex, nofollow meta robots achieve both crawl budget waste and dropping out of the index?

10.02% popularity Vote Up Vote Down


Login to follow query

More posts by @Annie201

2 Comments

Sorted by latest first Latest Oldest Best

 

@Samaraweera270

what should I do with subdomains and pages on the old site which have no similar comparable page on the new site.


If the pages will stay live, Add a robots.txt file or add noindex, nofollow meta tag, this will stop this pages from being crawled.

Deleting the pages from the server will drop the pages from the index automatically.

Also, a well drafted 404 page with relevant links will help the users who still have this links saved.

10% popularity Vote Up Vote Down


 

@Goswami781

There are at least two things you can do with pages that are outdated: redirect them (usually via a 301 permanent redirect) or let them go to a 404, as Stephen said.

In most cases, I would redirect, for a couple of reasons:


Your old pages and subdirectories/categories may carry link juice. You can use Moz's Open Site Explorer to see their inbound link profiles and authority (as perceived by Moz). You want to pass that link profile and authority to another page on your site.
If those pages have been indexed, they may hang around in the index for a while before search engines remove them. You want users who may come across these pages to go to another page on your site.
After a while, when search engines realize that your pages are redirecting, they will take the old pages out of the index.


When redirecting, your old and new pages don't have to be a 1:1 match. If the section is sort of related, you can still redirect - that is, if you think the new section will benefit the user who was looking for the old one.

If nothing fits, you can redirect to the home page, or to a contact page. You won't be penalized; at worst, the redirect will pass no link juice. (If you're really worried about it, you can use a 302 temporary redirect, but 301's do a better job of removing things from the index.)

It's also fine to let it go to a 404, if there is absolutely nothing that's semantically similar to the page you're taking down, or if that page has a poor backlink profile or got very little traffic. In that scenario, it'll get dropped from the index eventually too. I'd make sure that your 404 page is helpful to the user, displaying suggested links and the nav, for instance.

You don't want to use noindex, nofollow in this case, because there's no point in keeping this page on the server. The robots.txt solution is not a solution, because disallowing directories you've deleted anyway will create redundant rules, and as you said won't necessarily take it out of the index.

Whichever way you choose to go, run a site:example.com query now and again, and go deep into the SERPs. If you see pages you don't want still in the index, you can take a different course, and also request manual removal via Google Search Console and Bing Webmaster Tools.

10% popularity Vote Up Vote Down


Back to top | Use Dark Theme