: Reverse pagination Quick question to speed my life up considerable Can you reverse engineer pagination, in other words instead of having to click through 20 pages of 10 results on each Is

@Speyer207

If its just client side your after then Super User may be the site you need. To save the re-post AutoPager for Firefox and Chrome are good or you can use Fiddler to have proxy support for different browsers. However this will require some JavaScript programming.

Assuming (as this is Webmasters) that you are probably after the setting in the system giving you the pagination. This is generally under Listings or Results and should be turned off. Sometimes this is by setting the number of results per page to 0, -1 or UNLIMITED (please check the documentation before changing settings randomly, as the developer(s) would have probably left a note or admin control). More complex systems may have multitude of options for granular control.

Although if you are writing a screen scraper be careful of DCMA Takedowns. Not only do normal Copyright restrictions exist if you republish content but depagination can be counted as


circumventing an access control


Sometimes the number of results can be increased or removed (watch out for defaults here) from a querystring if the search form puts it there. Other forms use POST and will require some debugging (Fiddler is brilliant for this, as is HTTP Headers for Firefox). Remember that the occasional page (normally ASP.Net) will generate session cookies that you have to send back for requests to be valid. CGI based systems can be difficult, for instance the London Borough of Croydon's Planning Permission site uses RIPSESSION fields that must be interpreted to work. Beautiful Soup is a useful tool if you like Python, otherwise Yahoo!'s YQL is a close second but it is client side and there is a jQuery plugin.

Vote Up Vote Down