: Yandex frequently replaces page names with ampersands The Yandex spider is a frequent visitor to one of the sites I manage. On occasion it replaces the page name with two ampersands and a space.

Posted in: #CrawlErrors #WebCrawlers #Yandex

The Yandex spider is a frequent visitor to one of the sites I manage. On occasion it replaces the page name with two ampersands and a space. So if the page is:

/mypage.aspx?param=value

then it will try and crawl it as:

/&& ?param=value

Any idea why it is doing this?

EDIT:
If I remember correctly the IP that this mistake is coming from is based in California and not Russia. I believe that they crawl US sites from a US based IP address. Not sure if that helps.

More information about the request:

IP: 199.21.99.82
City: Palo Alto
State: California
Country: United States
ISP: Yandex Inc.
User-Agent: Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)

10.02% popularity Vote Up Vote Down

: How to remove Google plus author profile picture from blog search results I created a company blog and linked it to my Google plus account for the SEO weight, my problem is now the blog has

@Si4351233

Posted in: #Blog #GooglePlus #GoogleSearch #Wordpress

2 Comments

: Facebook sharing won't pull through ANY images anymore, I have created a company blog and have begun implementing various social sharing plugins, had a look at integrating with OG and now Facebook

@Si4351233

Posted in: #Addthis #Facebook #Sharing #Wordpress

1 Comments

: It's usually meant to keep incoming emails to a minimum. Not just spam but emails in general if an email address is displayed on a website it's not only spam they have to worry about but

@Si4351233

0 Comments

: Custom or Open Source CMS? I am in the process of having our corporate site redeveloped. My problem is do I choose a bespoke CMS or open source solution? We have used a bespoke CMS in

@Si4351233

Posted in: #Cms #Drupal #OpenSource #WebDevelopment #Wordpress

2 Comments

Login to post a comment!

2 Comments

Sorted by latest first Latest Oldest Best

@Megan663

There could only be two explanations for this behavior:

The crawler found a link to that malformed URL, either on your site or on some other site.
The Yandex crawler has a bug.

If you aren't seeing that URL crawled by other bots or visited by real users, then I suspect it is a bug with the Yandex crawler. As for why Yandex would have that particular bug, I can't say. There is no valid reason for any user agent to make that type of substitution in a URL.

10% popularity Vote Up Vote Down

@Yeniel560

Its probably mis-translating your URL, may be if your URL was more simplified such as mypage.com/page-two

10% popularity Vote Up Vote Down

Feed

: Yandex frequently replaces page names with ampersands The Yandex spider is a frequent visitor to one of the sites I manage. On occasion it replaces the page name with two ampersands and a space.

More posts by @Si4351233

: How to remove Google plus author profile picture from blog search results I created a company blog and linked it to my Google plus account for the SEO weight, my problem is now the blog has

: Facebook sharing won't pull through ANY images anymore, I have created a company blog and have begun implementing various social sharing plugins, had a look at integrating with OG and now Facebook

: It's usually meant to keep incoming emails to a minimum. Not just spam but emails in general if an email address is displayed on a website it's not only spam they have to worry about but

: Custom or Open Source CMS? I am in the process of having our corporate site redeveloped. My problem is do I choose a bespoke CMS or open source solution? We have used a bespoke CMS in

Login to post a comment!

2 Comments

Back to top | Use Dark Theme