: Should I block a dynamically generated data results page in robots.txt? I have a tiny two page site: the parameters page and results page. The results page can either work with URL parameters

@Si4351233

There is no need to block the page in its entirety, what you can do is apply a change to your site so that if no parameters are passed then the site returns a 404 not found error (important this has to be a http code and not just printed on the screen) and allow Google to spider your results page.

Why do this?
Well blocking any spider access to the results page will mean that if someone links to the results page with specific parameters from another site and Google tries to follow it will miss a large number of keywords and page content which could improve your Google ranking. By returning a HTTP 404 error when no parameters are passed Google won't index the URL however as I recall Google treats pages with different query strings as different pages and so will still index the results pages when query string filters are applied and linked to. As for the POST parameters, well ideally all the parameters would be passed as a GET allowing someone to link to it easily (if you wish this) however you wouldn't be receiving links from external sites trying to pass POST parameters so makes no real difference for the Google side of things.

Vote Up Vote Down