: What should be the standard time for a crawler to revisit my site? I've found mostly on the site meta name "revisit-after" content="3 days" and also I'm using this. But the fact is, what should

@Si4351233

We can't tell you the correct use for your specific website. It depends on the content of your webpage.

Googlebot crawls all sites and makes updates to their database if it notices any changes on your content. If you change the content, the crawler can easily analyze that.

Vote Up Vote Down

@Ann8826881

How often you would like the search engine spiders to visit your site is entirely dependent on your content and how often it changes. There is no "standard". If it changes daily then once a day might be nice. However, if the site predominantly contains archived/static documents and new documents are only rarely added then search engine spiders only need to visit your site... rarely.


"revisit-after" content="3 days"


No search engines actually use this meta tag, so this serves no SEO benefit whatsoever. Unless you have a specific requirement of your own then this meta tag should be removed. In fact, Google themselves describe the revisit-after tag as "completely useless":


To our knowledge only one search engine has ever supported it, and that search engine was never widely used — at this point, it is nothing more than a good luck charm. A remarkably widely used one. More pages use the completely worthless <meta name="revisit-after"> than use the <em> element!


Overtime Google tries to intelligently determine how often it should crawl your pages. You have limited control in Google Search Console (Site Settings > Crawl Rate) and even then you can only slow it down, not speed it up. (As stated on that page, this feature should only be used if Googlebot is causing you server problems by crawling too much!)

Vote Up Vote Down

@Shelley277

Unless needed I wouldn't recommend using the meta name="revisit-after" tag unless there is a specific reason.

That tag limits crawlers from basically crawling whenever they want to the time you tell them, don't crawl again until X.

Vote Up Vote Down