: Page appears indexed in Google but not findable for any search terms? (Note that I am going to use screenshots here because I suspect writing about this will change the behavior over time.)

@Reiling115

I wonder if there's any location-specific issue here?

From my desk in central London (the UK version) I just Googled uiviewcontroller best practices and the first three results were SO, with a single efreedom link at #4 . That's a google.co.uk search.

In general, I think I see SO links higher in Google's search results - I actually wasn't aware of eFreedom at all prior to today.

I note that Alexa's eFreedom entry has it at #568 for India - perhaps there's some particular bias with Indian-based searches?

Vote Up Vote Down

@Bethany197

Very short and super easy answer. Ping the question.

I work in SEO, and this is how I make my tens of dollars a year ;)... things showing up in the big G.

I have spent thousands upon thousands of dollars working on tool that speed all of this up for me.

2 things make you show up before the rest (for exact phrases)


Spider Crawls
Inbound links


Ping the page, ping the sitemap, and let the mirrors point back to you... There is a phrase we have in the SEO world, it is the Google Dance, you will bounce up and down the results latter while Google finds where to place you "permanently."

EDIT

For better clarity, here is a PDF about the different google bots www.telezent.com/telezent/Resources/FAMILY-OF-GOOGLE-CRAWLERS.pdf

Vote Up Vote Down

@Murray432

I don't know if it makes a difference, but looking at efreedom's page source code I noticed they're using google_ad_section_start and google_ad_section_end markers (http://www.google.com/support/adsense/bin/answer.py?hl=en&answer=23168). Could this allow Google to better weigh the actual content of the site?

Also, the actual content seems to be closer to the top of the page, whereas with SO there's a lot of ceremony going on before the actual content (this may not matter either, just a guess).

Vote Up Vote Down

@Sims2060225

There's not much you can do about this. There's a teensy bit you can do to help, but the problem is endemic to Google's index.


There isn't just one Google "index;" it's sharded. Pages with low PageRank appear in very few shards. Using site:stackoverflow.com forces your query onto a shard that has a larger fraction of your URLs indexed. That explains the contradictory results you see: the search results are coming from different shards.
PageRank applies to pages, not domains. Yes, really. Since the StackOverflow home page has high PageRank, the home page can flow PageRank to other question pages, but if your home page doesn't link to this particular Question page, then the Question's effective PageRank is very low.
You can also flow PageRank via your XML sitemap. According to this paper:


The root of the domain is assumed to contain an implicit link to the Sitemaps file. In addition, the Sitemaps file is assumed to contain links to all of its URLs, either uniformly weighted or weighted by priority if this field is provided. Over this augmented
graph, PageRank can be calculated for every URL in the Sitemaps page.

StackOverflow doesn't link to every question on the site; it only links to a tiny minority of questions for a very brief period of time. Furthermore, it's my understanding that you guys even delete links from your sitemap, only linking to the most recent questions. So many pages on the site receive no benefit from the home page's very high PageRank.


So, you can help this problem a little bit by making sure your XML sitemap is 100% complete (not just the newest questions, but ALL of them) and by making sure that every question page can be reached by a short chain of links starting from the home page.

But even that won't always work; you can't reasonably expect every page on your site to have high PageRank. In those cases, those pages will be crawled, but probably won't appear on a lot of shards, so Google won't always return them.

Good luck!

EDIT: Jeff hates sitemaps, so I updated this answer to make it easier for him to read.

Vote Up Vote Down