: Why does the Bing bot request this URL? While checking server logs, I have seen that often the Bing bot requests this URL. URL: /da-net/products/cadpac/cadpac_jis.shtml Browser: Mozilla/5.0
While checking server logs, I have seen that often the Bing bot requests this URL.
URL: /da-net/products/cadpac/cadpac_jis.shtml
Browser: Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
Can anyone explain this?
More posts by @Correia994
1 Comments
Sorted by latest first Latest Oldest Best
Its probably because you have a link to that page.
More than likely you have a ../../cadpac/cadpac_jis.shtml link is a url that is on your site.
Bing also reads AJAX urls, so if you have javascript that is creating this url, bing will crawl it.
Easiest way to handle this is just put it in your robots.txt file as a page not to index / carwl.
Terms of Use Create Support ticket Your support tickets Stock Market News! © vmapp.org2024 All Rights reserved.