: Why do Google FreshBot crawled and indexed pages disappear from SERPs? Will they come back? Sometimes a page on our site will show up in the Google SERPs that was previously not indexed (yet)
Sometimes a page on our site will show up in the Google SERPs that was previously not indexed (yet) due to a news story or blog posting on another site that points to it.
From what I understand, the Google FreshBot has crawled it. The FreshBot's job is to find information that is current or newsworthy to the time and get it indexed and into the SERPs quickly.
However, the pages that benefit from that drop out of the SERPs after two or three days. As a matter of fact, if I do an advanced query and request that page specifically from our site (using "site:"), it appears to not even be indexed any more.
What's happening to our Google FreshBot crawled pages?
Do pages that are indexed by the FreshBot still need to be re-indexed by the DeepBot?
For what it's worth, we're ordinarily getting indexed daily at a healthy rate, but we don't yet have all of our pages indexed (large site). It has been encouraging to see the effect of the FreshBot indexing, but watching it fade away is hard to understand. On our pages indexed as part of our normal crawls, our SERPs are strong -- without the added relevancy of news or other off-page links.
More posts by @Jamie184
1 Comments
Sorted by latest first Latest Oldest Best
The conversation about "Freshbot" and "Deepbot" are pretty deprecated. I haven't heard those words used since the early 2000s on intermittent Geocities websites. And I can't find any good sources on them that aren't websites riddled with Adsense. The woes of Google indexing is such a nebulous mechanism that you're better off not trying to put such a fine point on the question by trying to peg distinct Google crawler specificities that probably don't even exist anymore.
More contemporary buzzword guesses for how Google indexing works for new websites include the Sandbox effect, but it's a better perspective to ask "why am I not getting indexed?", and then understanding that unless you have a reason to be penalized by Google or if Google simply doesn't understand your navigation heirarchy, you're likely just in the waiting game.
Sounds like your website is just new. Sometimes, new websites get fully indexed in less than a week. Sometimes, they finally get indexed after four months of wait no matter what you do.
Terms of Use Create Support ticket Your support tickets Stock Market News! © vmapp.org2024 All Rights reserved.