: Restrict number of articles user allowed to read, but allow Google to index all I have seen several newspaper sites that have started to restrict the number of articles the user is allowed

@Shelley277

I'd suggest using the google crawler user agent and add a condition to your logic... Here's a list of user agents used by the G Crawlers.. support.google. com/webmasters/answer/1061943?hl=en
In fact there's another way to give access to the spider with "special user account" i'd rather suggest accessing your google webmaster tools, in the "authorization" section where you have to precise where it should send the post request and the data etc...

EDIT 1 : Reverse lookup to safely check if its actually GoogleCrawler

EDIT 2 : Google WebMaster Tools URL parameters

The use of the URL parameters : GoogleCrawler will try to access your website with another _GET parameters for example if you set the parameters as follows (user=google,pwd=crawler009) then if you had yoursite.com/login.php , The crawler will access your website with yoursite.com/login.php?user=google&pwd=crawler009

What you have to do is to add a sort of "logic" to accept logins even from the GET params or simply make an exception for the crawler, its almost the same we do with _POST params :)

Vote Up Vote Down