: What is `/&wd=test` URL that is being requested from my site, probably by bots I'm seeing error logs on a website because something tried to access: example.com/&wd=test the HTTP_REFERER

@Steve110

It's most likely Baidu just being weird, and their bots sending traffic to invalid URLs.

If Baidu is sending you traffic, even to weird parameters such as that I wouldn't necessarily start blocking their bot. But you block them if you want in your robots.txt and because Baidu is a legitimate crawler, they should most likely respect your robots.txt file. Baidu is a major Chinese search engine.

The invalid URLs that they are sending traffic to shouldn't cause any issues for your site unless there is something on your server that causes errors in this regard. Most likely, the visitor will just be sent an invalid page.

You can remove these ? parameters in your htaccess file and 301 redirect back to the correct page. If you want to do that, simply run a Google search for "remove everything after htaccess stackoverflow". Remove every character after the question mark, and then have htaccess simply remove the question mark itself. This will keep the user on the page that was intended.

Vote Up Vote Down

@Heady270

This is a request from a kind of Baidu searchbot. Baidu is a search engine from China. Like Google has its own searchbot, Googlebot, so Baidu has its own. There is nothing suspicious and dangerous on this request.

If you don't like it, from statistical point of view, you can block it with your robots.txt, like
#Baiduspider
User-agent: Baiduspider
Disallow: /


Or block it with server configuration, like on Apache with:

<IfModule mod_rewrite.c>
RewriteCond %{HTTP_USER_AGENT} baidu [NC]
RewriteRule .* - [F,L]
</IfModule>

Vote Up Vote Down