: How to tell Google to stop crawling my website via IP? After a database server migration, I've noticed that GoogleBot started getting errors when trying to crawl my site. The reason seems to

After a database server migration, I've noticed that GoogleBot started getting errors when trying to crawl my site.

The reason seems to be that it is hitting my site via IP address (and due to my config setup, my PHP scripts try to hit the old database server, which has been shut down). When the site is accessed by correct hostname, it uses the new DB server.

I can fix this by pointing the 'IP address' config to use the new DB server, but ideally I want to tell GoogleBot (and also Bing) to stop accessing via IP, and use hostname instead.

Google Webmaster Tools also shows a lot of entries for 'sites linking in' to my website, which are actually listed as the IP address of my site, as if it were a different site.

I do not want to lose SEO value by making the wrong move.

Should I change my website to stop responding on IP address? Or set up HTTP redirects from IP address/url -> hostname/url?

I already have "rel canonical" URLs which include the FQDN on a lot of my pages, but Google does not seem to respect them, or at least it continues to hit the IP address for those pages.

10.01% popularity Vote Up Vote Down

: Google Analytics is not tracking all of our pages Our website is insynchq.com. In the All Pages report under Content -> Site Content we can only see data for some our pages, like /, /getstarted,

@Cofer257

Posted in: #GoogleAnalytics

3 Comments

: Creating an SSL Certificate I have a barracuda SSL VPN. I also have a website hosted on winhost.com. On my website, I have a link to login to my VPN. I always get the "untrusted certificate"

@Cofer257

Posted in: #CertificateGeneration #SecurityCertificate #Vpn

1 Comments

: Best solution for online video classes? If I want to make a website where visitors can sign up to online video classes, which of the following options is the best solution? 1-) All the students

@Cofer257

Posted in: #Video

1 Comments

: If the files "shouldn't be public" then they should be on the public internet. You can remove the files from Google listings (via robots.txt and other methods), but if the files are still

@Cofer257

0 Comments

Login to post a comment!

1 Comments

Sorted by latest first Latest Oldest Best

@Shanna517

Looks like you asked and answered your own question.

The popular suggestions seem to be setting up a 301 redirect, stopping your http server from responding to IP requests, setting up "rel canonical" urls, or trying to put no follow on links to the old database. (I think I'd try removing the old db links from the scripts first, but anyway...)

10% popularity Vote Up Vote Down

Feed

: How to tell Google to stop crawling my website via IP? After a database server migration, I've noticed that GoogleBot started getting errors when trying to crawl my site. The reason seems to

More posts by @Cofer257

: Google Analytics is not tracking all of our pages Our website is insynchq.com. In the All Pages report under Content -> Site Content we can only see data for some our pages, like /, /getstarted,

: Creating an SSL Certificate I have a barracuda SSL VPN. I also have a website hosted on winhost.com. On my website, I have a link to login to my VPN. I always get the "untrusted certificate"

: Best solution for online video classes? If I want to make a website where visitors can sign up to online video classes, which of the following options is the best solution? 1-) All the students

: If the files "shouldn't be public" then they should be on the public internet. You can remove the files from Google listings (via robots.txt and other methods), but if the files are still

Login to post a comment!

1 Comments

Back to top | Use Dark Theme