: Trick to index infinite scroll content on Google? I have a Question & Answer website, sort of like stackoverflow. When you open it, it displays 20 latest questions. When scroll to bottom

Posted in: #Ajax #Google #GoogleIndex #Indexing #WebCrawlers

I have a Question & Answer website, sort of like stackoverflow.
When you open it, it displays 20 latest questions.
When scroll to bottom of page, and it fetches 20 more, and so on, infinitely.
Each question has its own page, just like here.

So my trick is create a empty page that contains only <a href='question_link'> links to 100 latest questions, followed by a link to the next 100 latest questions. No text, only links. Example:

Fake page 1

<a href='/question/1'></a>
<a href='/question/2'></a>
<a href='/question/3'></a>
<a href='/question/4'></a>
...
<a href='/nextpage?after=100'></a>

Fake page 2

<a href='/question/101'></a>
<a href='/question/102'></a>
<a href='/question/103'></a>
<a href='/question/104'></a>
...
<a href='/nextpage?after=200'></a>

and so on, until all questions are exhausted. Notice it has no text, no page title or page description, this is to avoid Search Engine from indexing it.

My question is, would Google allow something like this? I mean it's fully legitimate, and all we're doing is telling Google about the URLs of our pages, so it's okay right?

10.01% popularity Vote Up Vote Down

: Time for an official answer. What I learned is that hackers like to try to exploit the shell shock bug with variants. They like to try to use lines containing (){} and ( ){:;} and similar,

@Angela700

0 Comments

: Here's other code that is similar to yours that makes it easy to understand what is blocking: RewriteCond %{HTTP_REFERER} !^http://mySite1.com$ [NC] RewriteCond %{HTTP_REFERER} !^http://mySite2.com$

@Angela700

0 Comments

: Must the image url in a sitemap file begin with the same domain the file is stored on? I remember when making a sitemap, all URLs defined in between <loc> and </loc> must be part

@Angela700

Posted in: #Html #Images #Sitemap #Url #XmlSitemap

1 Comments

: DHIS 2 is a web-based open-source information system with visualization features including GIS, charts and pivot tables.

@Angela700

0 Comments

Login to post a comment!

1 Comments

Sorted by latest first Latest Oldest Best

@Bryan171

Don't do this. This will create a bad ranking page, with duplicates (the next/prev pages) and will cost your more than you will gain.
Rule of thumb: Don't make pages just for bots.
To prefent indexing you could just add the robots metatag with the value NOINDEX.

What you need is a sitemap xml :) Those are perfect fort situations like this; To indicate pages which cant easily be found by crawling.
Also, if you have Google Analytics, they'll find the pages because you trigger the code on those pages.
Also, if you have visitors using Chrome, they'll find the pages.

Long story short / TL;DR:
Do the sitemap, and don't worry, you'll be found.

10% popularity Vote Up Vote Down

Feed

: Trick to index infinite scroll content on Google? I have a Question & Answer website, sort of like stackoverflow. When you open it, it displays 20 latest questions. When scroll to bottom

More posts by @Angela700

: Time for an official answer. What I learned is that hackers like to try to exploit the shell shock bug with variants. They like to try to use lines containing (){} and ( ){:;} and similar,

: Here's other code that is similar to yours that makes it easy to understand what is blocking: RewriteCond %{HTTP_REFERER} !^http://mySite1.com$ [NC] RewriteCond %{HTTP_REFERER} !^http://mySite2.com$

: Must the image url in a sitemap file begin with the same domain the file is stored on? I remember when making a sitemap, all URLs defined in between <loc> and </loc> must be part

: DHIS 2 is a web-based open-source information system with visualization features including GIS, charts and pivot tables.

Login to post a comment!

1 Comments

Back to top | Use Dark Theme