: Is there anything unusual or nefarious about private IP addresses in server logs? A site I help take care of has just started getting crawled by somebody from 172.31.13.241 and 172.31.42.227
A site I help take care of has just started getting crawled by somebody from 172.31.13.241 and 172.31.42.227 . According to WHOIS info, these are part of a block reserved by IANA for special purposes, specifically private networks.
They're making 400 rpm - while not too bad, and not a problem for the infrastructure, it doesn't entirely feel like benign crawling (e.g. Googlebot will make far fewer requests it seems, and I've previously had other sites crawled at 1 to 5 rpm).
I've banned them for the moment, but would like to know if there is a plausible explanation which points to private IPs in Apache access logs being a normal, "usual" thing for some reason. So far I've been unable to find any such explanations, in Superuser.com, Webmasters SE, or the wider net.
EDIT: Thanks for the comprehensive overview John McNamara. The 2 IPs turned out to belong to the Elastic Load Balancer in front of our AWS instances. The thing is, it usually passes on the IP of the request directly to the instances (e.g. I can see GoogleBot's IP, I can see users' and my own IP while browsing). I used the load balancer's logs to find out the public IP, it turned out to be a product we are trialling out which requires crawling. I believe they usually crawl much bigger websites than ours, hence the unusually high number.
More posts by @Carla537
1 Comments
Sorted by latest first Latest Oldest Best
Is there anything unusual or nefarious about private IP addresses in server logs?
would like to know if there is a plausible explanation which points to private IPs in Apache access logs being a normal, "usual" thing for some reason
Unusual, no. Nefarious, possibly, it really depends on what the traffic is attempting to do.
from www.iana.org/help/abuse-answers
If you see an apparent attack, or spam, coming from one of these address ranges, then either it is coming from your local environment, your ISP, or the address has been "spoofed".
The root cause could be any of the following
misconfigured device on your LAN causing many requests
misconfigured device on your ISP's network causing many requests
malware/active attacker on your LAN
malware/active attacker on your ISP's network
active attacker in internet spoofing IP addresses
Terms of Use Create Support ticket Your support tickets Stock Market News! © vmapp.org2024 All Rights reserved.