: Indexed webpages stopped rising, while Submited keep rising normally Last months I have noticed a strange behaviour in Google's Webmaster Tools. Although I provide a valid Sitemap for my website,

@Odierno851

I think it's a problem on Google's end.

I keep a spreadsheet monitoring my GWT sitemap stats on a site and I have seen the exact same behavior. The number of pages indexed on Jan 20 was different from the number when I checked Jan 10, as it nearly always was every few days I checked for the last three and a half years, but since Jan 20, the number has not changed even though the number of pages submitted has increased and I have even added another sitemap.

Like you, all other indicators seem normal, and the site is a fairly dynamic, large, well-indexed site (well over 100K pages) that, until the last three months, had previously never had the same number of Indexed pages for more than a week.

I've been wondering if it was a bug but I couldn't find anyone else on the Internet talking about it and as I've often seen different aspects of GWT be temporarily slow or outdated, I kept expecting it to fix itself and haven't looked into it further. I'm guessing it doesn't effect everyone or we would have heard about it elsewhere by now (though anyone could confirm this if your Indexed numbers have changed since mid-January), but I think you've provided evidence that it's more than just me.

Vote Up Vote Down

@Cooney921

Submitting content with a Sitemap doesn't guarantee it will be indexed, though it is an aid to getting it crawled, among other things.

A common reason for a mismatch between what's submitted and indexed is duplication - either multiple versions of pages are being submitted, or else the pages are unique but Google judges them to be insufficiently unique to warrant indexing them.

You might try:


Verifying that all URLs submitted by the XML Sitemap are unique (e.g., you don't have the same URL multiple times due to parameters that don't affect page content), and valid (not redirects, 404s, etc.)
Checking that your page content is unique, or pretty much so. Look for things like large amounts of "boilerplate" copy used across numerous pages, or repeats of the same content in different categories.
See if your site's taxonomy creates duplication, like archive and tag pages in WordPress for example.
Run crawler software on your site to verify that content is actually accessible. If it finds less content than you expect, look for reasons, e.g., JavaScript-based navigation, nofollow links, etc. (Google Webmaster Tools will have warning notifications if, say, robots.txt is blocking access).

Vote Up Vote Down

@Becky754

Welcome to the world of Google. If you submit a sitemap, the number of pages indexed never seems to match the number submitted in Google Webmaster Tools. It has been this way for years. I am not sure why, but I can tell you this much. Google seems to apply an algorithm that manages the speed of which the pages are fetched from a sitemap. The more pages in a sitemap, the faster it goes. As it begins to near the end, it slows down more and more. I suspect that it stops almost entirely short of the total number of pages at least giving the appearance that the entire sitemap is not indexed. And that may be the case. It just may be that the number of pages fetched from the sitemap each day reaches such a low number that the final few pages (often measured in a few thousand) takes a long time. For my site, this can range from 2000 to 4000 which seems to be exactly what you are experiencing.

Another consideration along the way is that the sitemap is read and all of the entries are put into the queue to be fetched. As Google slows down fetching pages from the sitemap, other pages that are deemed important, discovered via other means, perform well in searches, and so on are moved to the top of the queue and are a part of the mix. So if Google slows down to about 1000-2000 pages fetched each day, fewer and fewer are the remaining pages from the sitemap as each day goes along making way for other items in the queue.

Vote Up Vote Down