: Does the below-mentioned server header impact google indexing? Date: Sat, 04 Mar 2017 06:26:50 GMT Server: Apache/2.4.7 (Ubuntu) X-Powered-By: PHP/5.5.9-1ubuntu4.20 Expires: Sat, 01 Jan 2000 00:00:01

@Jessie594

Nothing you do on your end can really affect the crawl rate Google uses. You have the ability to slow down crawls through webmaster tools but there is no way to speed it up through webmaster tools or through your site. Your headers look fine and appear to be much the same as many sites out there. If Google has slowed down after having a long time of frequent re-indexing it may be due to not detecting much updated content and so not needing to recrawl regularly. If it is a relatively new site then Google won't have gotten up to speed with it as it can take some time before Google starts doing regular crawls. The only thing you can really do is regularly update your site and be patient.

During the crawling process there are two headers which mainly come into play and they are the Last-Modified header and the If-Since-Modified header. What happens is that the first time a page is requested by the crawler it saves the Last-Modified header if it is available, and the next time the page is crawled it will send the If-Since-Modified header. Now if the server (or proxy if there is one in the process) is configured to handle this request it will check to see if the requested page has been updated since the last time it was crawled. If it has then it will return the full page but if it hasn't then the server will return a 304 not modified header without a document (saves time and bandwidth).

Now I should note this is the way it works if the server has been configured to handle it otherwise the request will always result in a document being returned to the crawler but the headers you have specified do not really play a part in SEO (extremely low, possibly non existent signal). They can play a part with crawl frequency if the caching was configured or the expire date was in the future but as your's effectively says never cache and always get the latest copy from the server as it always expires then it won't be having an effect on crawl rates.

Vote Up Vote Down