Mobile app version of vmapp.org
Login or Join
Goswami781

: Search engines crawling non-existent subdomains A few years ago, a previous developer added my company's website to a link farm he ran. My company has a wildcard subdomain, so any subdomains

@Goswami781

Posted in: #Domains #GoogleSearch #SearchEngines #Subdomain #WebCrawlers

A few years ago, a previous developer added my company's website to a link farm he ran. My company has a wildcard subdomain, so any subdomains will return the user to the homepage. As such, the dev added multiple subdomains to the link farm (e.g., foo.example.com, bar.example.com, foobar.example.com) Before this developer left, he claims to have removed all our links from the link farm. However, to this day, we still get search engines crawling sub-domains that do not exists. I should add that the developer added many, many, subdomain links.

What can I do to prevent these subdomains from getting crawled?

10.01% popularity Vote Up Vote Down


Login to follow query

More posts by @Goswami781

1 Comments

Sorted by latest first Latest Oldest Best

 

@Radia820

If the links still exist on other sites then Google will continue to look for them on your site, there isn't much you can do to stop it if you can't remove the links.

The only thing that you can do is add the code below to your htaccess file, this 410 gone message will let Google know the content is gone for good. Google may not stop looking but at least it will stop generating 404's each time it does look.
#Stuff to 410

Redirect gone /path/to/page.html
Redirect gone /directory-path/
Redirect gone foo.domain.com

10% popularity Vote Up Vote Down


Back to top | Use Dark Theme