Mobile app version of vmapp.org
Login or Join
Si4351233

: Unexplainable non-existing page requested for months While I am quite familiar with bots requesting well-know pages as used for WordPress and phpMyAdmin on any site the can reach, I'm now dealing

@Si4351233

Posted in: #Botattack #Page #RobotsTxt #WebCrawlers

While I am quite familiar with bots requesting well-know pages as used for WordPress and phpMyAdmin on any site the can reach, I'm now dealing with a bot requested a page I really can't explain.

For months now, a page /featured/like?id=XXX (with XXX being a 3-digit integer) is being requested 10+ times a day for the last six months, from multiple IP addresses from countries like France, Germany and Ukraine (site is hosted and targeted at The Netherlands).

This page doesn't exist and it's not indexed by any search engine.

I'm almost sure this is just another bot trying to find a page used by some framework, but it's been going on for months now, so I just want to make sure.

Does anyone know why this page keeps being requested?

10.01% popularity Vote Up Vote Down


Login to follow query

More posts by @Si4351233

1 Comments

Sorted by latest first Latest Oldest Best

 

@Berumen354

Hm...

The quick way to prevent robots visiting your site is put these two lines into the /robots.txt file on your server:

User-agent: *
Disallow: /


Try to put this for one day, and see what happens. This only helps with well-behaved robots.

But, if you like to disallow that page specifically,

Disallow: /featured/


Do that, and see what happens.

10% popularity Vote Up Vote Down


Back to top | Use Dark Theme