Mobile app version of vmapp.org
Login or Join
Vandalay111

: Why is google not indexing (or ignoring) those 200,000+ pages ? I was wonrdering why on google search console, 200,000+ of my pages are not indexed (it's not because of robots.txt or volontary

@Vandalay111

Posted in: #CanonicalUrl #Seo

I was wonrdering why on google search console, 200,000+ of my pages are not indexed (it's not because of robots.txt or volontary excluding them):

+------------------------------------+---------+
| Exclusion reason | Number |
+------------------------------------+---------+
| Discovered - currently not indexed | 119,000 |
+------------------------------------+---------+
| Crawled - currently not indexed | 101,000 |
+------------------------------------+---------+




This google article says :


Crawled - currently not indexed: The page was crawled by Google, but
not indexed. It may or may not be indexed in the future; no need to
resubmit this URL for crawling.

Discovered - currently not indexed: The page was found by Google, but
not crawled yet. Typically, Google tried to crawl the URL but the site
was overloaded; therefore Google had to reschedule the crawl. This is
why the last crawl date is empty on the report.


Some examples of URL's:

Discovered - currently not indexed:

www.seek-team.com/en/player/league-of-legends/killmushroom475/22844 https://www.seek-team.com/fr/player/league-of-legends/brutal-krogalolz/8627 www.seek-team.com/en/player/counter-strike-global-offensive/bobby/32181

Crawled - currently not indexed:

www.seek-team.com/en/player/starcraft-ii/baptistozor/38174 https://www.seek-team.com/en/player/league-of-legends/evikaz/28652 www.seek-team.com/en/player/counter-strike-global-offensive/sixkaa/65156

--

Those page I gave are all user generated and exist in multiple language (the user content is not translated, but the website interface is)


Is it because of my canonical markup ? All website exist in 2 languages, and each page has a canonical tag to itself (ex: /fr/example points to /fr/example, and /en/example points to /en/example.
Could it be because of some technical reason ? (bad use of canonical ? bas use of hreflang ?

10.01% popularity Vote Up Vote Down


Login to follow query

More posts by @Vandalay111

1 Comments

Sorted by latest first Latest Oldest Best

 

@Steve110

I was running a site last year that had about 30 million pages indexed with Google. I found that Google would crawl many pages but choose not to index a large percentage of them. I felt that many of my best pages that were crawled weren't indexed while pages with far weaker content were. Ultimately I concluded that Google chooses which pages to index and which pages not to and I had very little control over the selection process. Even proper internal linking with many links pointing to a specific page didn't mean it would get indexed. Pages with very few, in some cases even zero internal links were often indexed.

Google may discover links on your site but choose not to crawl those pages. And it may crawl pages and choose not to index them even if they are high quality pages.

I wish there was a simple fix for this issue to help you. But in the end I think you'll just have to cross your fingers and hope the pages you want indexed end up there.

If you can increase the links pointing to that page from external domains this will send a huge signal to Google that it needs to index them because those pages will have larger page authority than other pages on your site.

If you think there could be errors in the crawling process of these pages you should always try fetch and render with Googlebot. Enter some of the URLs into the following link and click fetch and render, then click on the result after Google has crawled it. Google will show you a screenshot of the page it has crawled which will let you know if it sees all of your content or if there are any issues. If it sees all of your content and still chooses not to index it then it's simply an algorithmic thing on Google's end and there is only so much you can do to improve your index status.
www.google.com/webmasters/tools/googlebot-fetch?hl=en&siteUrl=
This is a screenshot of search console for one of my sites. Google has chosen to exclude many of my URLs as well.

10% popularity Vote Up Vote Down


Back to top | Use Dark Theme