: Multiple encoded requests from crawlers Some crawlers repeatedly try to access URLs on my page that are multiple encoded in a strange way. An example: Bing bot tries to access (which gives a

@Smith883

Posted in: #UrlEncoding #WebCrawlers

Some crawlers repeatedly try to access URLs on my page that are multiple encoded in a strange way. An example: Bing bot tries to access (which gives a 404)
/Guitars/Children%C3%C6%92%E2%E2%82%AC%C5%A1%C3%E2%80%9A%C2%B4s-Guitars/
which once was
/Guitars/Children%C2%B4s-Guitars/
and perfectly works.

Others are even longer. How does this happen? And what kind of encoding is it? Apparently not UTF8...

If I could restore the original URL I could 301 them to the right address, which hopefully would stop them crawling this.

Vote Up Vote Down

Login to post a comment!