: Googlebots request types Does Google's bot send HEAD requests as well as GET requests? I wondering about this because I'm running a Tornado application that does not support HEAD request and

Posted in: #Googlebot #HttpHeaders #Soft404 #WebCrawlers

Does Google's bot send HEAD requests as well as GET requests?

I wondering about this because I'm running a Tornado application that does not support HEAD request and I have a lot of 404 errors in Google Search Console.

To verify that the Tornado application is not accepting HEAD requests I do:

curl -I example.com/this-url-exists

HTTP/1.1 404 Not Found
Date: Thu, 10 Mar 2016 08:49:07 GMT
Content-Type: text/html; charset=UTF-8
Content-Length: 7520
Connection: close
X-Ua-Compatible: IE=edge,chrome=1
Server: TornadoServer/4.2.1

I suspect that the Googlebots are doing the same for some of the crawling.

Can this be right?

10.01% popularity Vote Up Vote Down

: The part of the URL after the first question mark and before the fragment identifier, often used to pass variable URL parameters to the underlying script.

@Eichhorn148

0 Comments

: Why google not index my website's pages? We have a website with around 293.000 pages. We optimized its SEO but Google doesn't come it in search. We also have sitemap.xml that update in each

@Eichhorn148

Posted in: #Google #Seo

0 Comments

: You should add both. You can add both and the two may show you different information. It is most important to add the one that you intend having your visitors use. You should pick

@Eichhorn148

0 Comments

: AdSense on pages for logged in users with different content for each user Is it possible to show AdSense to logged in users when each user sees different content? I'm developing an application

@Eichhorn148

Posted in: #Authentication #GoogleAdsense #GoogleAdsensePolicies #UserGeneratedContent

2 Comments

Login to post a comment!

1 Comments

Sorted by latest first Latest Oldest Best

@Jamie184

I'm pretty sure Googlebot does not send HEAD requests - at least not with respect to crawling and indexing a website.

My access logs from January and February 2016 show no HEAD requests from the Googlebot.

Even if Googlebot did use HEAD requests I would be very surprised if these resulted in a 404 error in Google Search Console. Google should only report a 404 error on a failed GET request.

10% popularity Vote Up Vote Down

Feed

: Googlebots request types Does Google's bot send HEAD requests as well as GET requests? I wondering about this because I'm running a Tornado application that does not support HEAD request and

More posts by @Eichhorn148

: The part of the URL after the first question mark and before the fragment identifier, often used to pass variable URL parameters to the underlying script.

: Why google not index my website's pages? We have a website with around 293.000 pages. We optimized its SEO but Google doesn't come it in search. We also have sitemap.xml that update in each

: You should add both. You can add both and the two may show you different information. It is most important to add the one that you intend having your visitors use. You should pick

: AdSense on pages for logged in users with different content for each user Is it possible to show AdSense to logged in users when each user sees different content? I'm developing an application

Login to post a comment!

1 Comments

Back to top | Use Dark Theme