: Google Bot creating invalid GET Request I have checked multiple times in page, and there are no recursive sort of links anywhere nor are there any links without base href but still I can see
I have checked multiple times in page, and there are no recursive sort of links anywhere nor are there any links without base href but still I can see lot of links with 10+ level deep. like GET www.example.com/a/a/a/a/a/a/a/a/a/foo_bar_page which is invalid for me, but example.com/a/foo_bar_page is a valid page. I can temporarily block it in robots.txt but what if it starts requesting for b/b/b/b/ sort of links ? How can I stop this ?
More posts by @Moriarity557
2 Comments
Sorted by latest first Latest Oldest Best
Check the Google Web Master Tools. If you are not registered with them, do so.
Once in Google Web Master Tools, expand the Heath section and click on Crawl Errors. You will then see a list of URL Errors on the right. Find the offending URL in the list and click on it. A popup will appear with information on the error. The most important one usually is Linked From. This will list who links to it and that is the page to check.
The page can be one of your own or an external one. If it is one of yours, obviously things should be easy to fix. If not, you have to figure out how to contact the site who is linking or block it if you are unsuccessful.
Given the URL in your example, I suspect you have a relative link where you should have an absolute one. Maybe a menu bug were the page is supposed to point to itself. Just a guess.
Sometimes Google does try to GET pages that simply don't exist, finding the cause can be troublesome. If these pages are returning a valid 404 response there is nothing to worry about. As I say in numerous answers on this site a site that returns a 404 response is not broken and Google has said many times that 404's are completely normally (within reason of course).
Obvious things to check is:
On page internal links
Less Obvious things to check:
Plugins
CSS
JavaScript
Less Obvious and Normally Ignored Situations
Off Page Backlinks (Sometimes people, or automated sites can back link to pages that do not exist - at some point Google may discover that link and crawl back to your site and do a GET on that page which doesn't exist, and again if a 404 is returned! nothing to worry about.
Terms of Use Create Support ticket Your support tickets Stock Market News! © vmapp.org2024 All Rights reserved.