: Baidu spider does not obey DNS TTL I moved my website from one VPS to another on 16th April. The A record of the domain name has a TTL of 86400. I made the IP address change at 19:00

I moved my website from one VPS to another on 16th April.

The A record of the domain name has a TTL of 86400. I made the IP address change at 19:00 GMT 16th April.

I do not want the site to face downtime so I am keeping the old VPS alive.

Today is the third day and the access log on the old server shows hits from Baidu spider.

It has been more than 60 hours but why is this crawler still hitting the old IP address?

I am using AWS Router53 for DNS.

10.01% popularity Vote Up Vote Down

: Facebook open graph doesn't work I have used open graph meta tags in my website <meta name="og:description" content="My page descriptions ..." /> <meta name="og:title" content="سخت‌افزار20"

@BetL925

Posted in: #Facebook #MetaTags #OpenGraphProtocol

1 Comments

: Webpage loading with wrong content-type after setting up CloudFlare I recently migrated my blog to the Ghost service, I've also setup an alias DNS record with CloudFlare. While showing the blog

@BetL925

Posted in: #Cloudflare #ContentType #GhostBlog

1 Comments

: A request for a directory listing will have a URL that ends in /. So http://example.com/images/ will be a 403 because of the directory listing. http://example.com/private/document.html will be

@BetL925

0 Comments

: Why do servers need to know their domain? Besides routing traffic, why do servers need to be configured with the server name? It seems like a waste in all cases except for routing or hosting

@BetL925

Posted in: #Apache #Httpd #Nginx #Server #WebHosting

2 Comments

Login to post a comment!

1 Comments

Sorted by latest first Latest Oldest Best

@BetL925

Baidu spider also does not properly obey robots.txt in some cases. They seem to have a history of doing things their own (wrong) way.

Unless you have a Chinese website that gets a significant amount of traffic from the Baidu search engine, why would you keep old server up just to make the Baidu spider happy?

10% popularity Vote Up Vote Down

Feed

: Baidu spider does not obey DNS TTL I moved my website from one VPS to another on 16th April. The A record of the domain name has a TTL of 86400. I made the IP address change at 19:00

More posts by @BetL925

: Facebook open graph doesn't work I have used open graph meta tags in my website <meta name="og:description" content="My page descriptions ..." /> <meta name="og:title" content="سخت‌افزار20"

: Webpage loading with wrong content-type after setting up CloudFlare I recently migrated my blog to the Ghost service, I've also setup an alias DNS record with CloudFlare. While showing the blog

: A request for a directory listing will have a URL that ends in /. So http://example.com/images/ will be a 403 because of the directory listing. http://example.com/private/document.html will be

: Why do servers need to know their domain? Besides routing traffic, why do servers need to be configured with the server name? It seems like a waste in all cases except for routing or hosting

Login to post a comment!

1 Comments

Back to top | Use Dark Theme