: Mysterious visits to privileged URLs from anonymous user/bot I maintain a Rails app. Every page except login and "forgot password" requires user authentication. Yesterday I noticed entries like

@LarsenBagley505

Are you displaying ads on your site from Amobee (formerly known as Kontera)?

The PTR record for this IP is nat.aws.kontera.com, suggesting that it is a crawler looking for your page's content in order to determine what ads may be relevant to that URL.

If you've loaded the Kontera JavaScript on login-protected pages, then you will find that they are crawled any time a logged-in user visits those pages. Try removing the JavaScript call from protected pages, and see if the crawling of those pages stops.

It's also possible that code has been added to the web page by a man in the middle attack before it reached the user who actually visited your site. Such an attack might have been launched by a network operator or malicious party in order to gain revenue from your content, or for other reasons.

This is one of many reasons that every web site should run on HTTPS.

Vote Up Vote Down

@LarsenBagley505

There is nothing to be concerned about with this as it commonly shows up with secured applications on the internet. A user accesses a link through their bookmarks bar or through their browser history to go back to a page that they want to go back to and because their session has expired they are redirected to the login page to login first. No worries here.

Update #1
Further investigations into the IP in question have shown the following points...


The IP adress has been detected through project honeypot but was classed as possibly a harmless web crawler as none of the IP's visits have caused any problems. www.projecthoneypot.org/ip_54.209.60.63 Some users on the AbuseIP database report that removing the Web of Trust has resulted in the connections stopping however this can not be verified as the connections are being made against web servers not web browsers and WoT uses human ratings to rate websites. www.abuseipdb.com/check/54.209.60.63 https://www.mywot.com/en/support/how-wot-works
The IP has been listed on the BadIP's website as a malicious website as March 2015. www.badips.com/info/54.209.60.63

Based on the fact that none of these abuse reports about the IP address 54.209.60.63 actually state that anything malicious was done to the web server and even Project Honeypot confirms it is a possible web crawler and not a malicious machine there should be no concern.

The IP address has also since been de-activated from the offending account with Amazon and is no longer assigned to the original EC2 instance in question so there should be no further issues from this IP address regardless.

Vote Up Vote Down