: How to handle pages that requires the use of session cookie? I have a search page that uses session to track search criteria made by the user. A session cookie is created automatically when
I have a search page that uses session to track search criteria made by the user. A session cookie is created automatically when the page is launched. As bots do not use cookie, 1000s of sessions could be created when they crawl, thereby wasting server resources. There are a few solutions which I can think of, but with corresponding side effects:
List this page in robot.txt file, but this would affect SEO for the
page.
Check the user agent against a blacklist of known robots before
deciding whether to start the session. The list could be very large.
Set a test cookie and do a page redirection to check for the
presence of the cookie before starting a session? This means loading the same page twice.
Do away with anonymous sessions and put all criteria in the URL
using GET to retrieve them? But this means that I have to verify ALL criteria each time a minor change occur.
What would be a good way to handle this problem?
More posts by @Murray155
1 Comments
Sorted by latest first Latest Oldest Best
The simple answer here is: Search engine bots do not perform searches, so "1000's of sessions" would not occur.
They won't use sessions either, so you need to provide links in a sitemap that they can crawl containing the search parameters already in them, if you want to index dynamic content that results from searches.
Here's a Google Webmasters Tool answer that should help with how to structure a search URL: URL Structure
Note that you should not add links to a sitemap that might produce duplicate content, even if the search terms in them differ.
Update:
Since as further explained, the search page returns default results, you could add rel="nofollow" to the search page so that robots will not follow links on it, but consequently only the default search page results would be indexed, and no further pages.
Terms of Use Create Support ticket Your support tickets Stock Market News! © vmapp.org2024 All Rights reserved.