: Why am I getting bot hits from compute-1.amazonaws.com? I have a WordPress website with AWS on use, namely the Cloudfront service, to serve CSS, images and JS from the cloud. Lately, I noticed

Posted in: #AmazonAws #WebCrawlers #Wordpress

I have a WordPress website with AWS on use, namely the Cloudfront service, to serve CSS, images and JS from the cloud.

Lately, I noticed a lot of hits from bots coming from :

IP: 54.236.71.87 Hostname: ec2-54-236-71-87.compute-1.amazonaws.com
IP: 54.147.229.75 Hostname: ec2-54-147-229-75.compute-1.amazonaws.com
IP: 34.207.96.105 Hostname: ec2-34-207-96-105.compute-1.amazonaws.com
IP: 52.202.239.36 Hostname: ec2-52-202-239-36.compute-1.amazonaws.com
IP: 34.203.222.34 Hostname: ec2-34-203-222-34.compute-1.amazonaws.com
...

Goeip trace them to Ashburn, USA. They crawl all the RSS feeds of my website (posts, categories), almost every minute, coming from www.google.com/.
Their user-agent are a bit random :

Browser: undefined
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9a3pre) Gecko/20070330

or :

Browser: Chrome version 0.0 running on MacOSX
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1309.0 Safari/537.17

or :

Browser: undefined
Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.13; ) Gecko/20101203

How can I find out what they want ? Are they related to the cache generation for Cloudfront ?

I wonder because in the last 9 days, more than half of the bandwidth of my hosting has been consumed by unidentified bots.

10.01% popularity Vote Up Vote Down

: How would I list my online freelance services as an "online business" and not "LocalBusiness" in rich snippets? I currently use LocalBusiness as Schema.org type and I feel it's decreasing my

@Cofer257

Posted in: #Business #RichSnippets #SchemaOrg #Seo

1 Comments

: Does itemprop have a different meaning on links vs. span tags? I'm using Schema.org markup for a Person, but I get inconsistent results in Google's Rich Snippet Testing Tool when using an <a>

@Cofer257

Posted in: #Html #Microdata #Url

1 Comments

: How do I redirect direct image paths to my website? I recently noticed that some of the images on my Wordpress website are the pages with the most visits according to Google Analytics. The

@Cofer257

Posted in: #GoogleAnalytics #Images #Seo #Traffic #WebTraffic

1 Comments

: How do I export 7, 14 and 28 day Active users in Google Analytics? For years, we've been able to pull data out of the Active Users report on Google Analytics. Now, when we try to export

@Cofer257

Posted in: #GoogleAnalytics

1 Comments

Login to post a comment!

1 Comments

Sorted by latest first Latest Oldest Best

@Cugini213

These look to me like bots/crawlers running on EC2 instances, rather than anything to do with Cloudfront. AWS publish a list of IP ranges used by Cloudfront here : docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/LocationsOfEdgeServers.html
They're probably harvesting content or email addresses, looking for vulnerable apps, open comment forms where they can post comment spam, or something else not to your benefit. Sadly it's not uncommon for a lot of your traffic to come from bots like this.

You might want to block those IP addresses in the security group for your instance or ELB. If those IP's are hitting your origin servers rather than coming through cloudfront, and you only use cloudfront to access those servers, you might consider using something like this which blocks everything except the cloudfront IPs github.com/awslabs/aws-cloudfront-samples/tree/master/update_security_groups_lambda

10% popularity Vote Up Vote Down

Feed

: Why am I getting bot hits from compute-1.amazonaws.com? I have a WordPress website with AWS on use, namely the Cloudfront service, to serve CSS, images and JS from the cloud. Lately, I noticed

More posts by @Cofer257

: How would I list my online freelance services as an "online business" and not "LocalBusiness" in rich snippets? I currently use LocalBusiness as Schema.org type and I feel it's decreasing my

: Does itemprop have a different meaning on links vs. span tags? I'm using Schema.org markup for a Person, but I get inconsistent results in Google's Rich Snippet Testing Tool when using an <a>

: How do I redirect direct image paths to my website? I recently noticed that some of the images on my Wordpress website are the pages with the most visits according to Google Analytics. The

: How do I export 7, 14 and 28 day Active users in Google Analytics? For years, we've been able to pull data out of the Active Users report on Google Analytics. Now, when we try to export

Login to post a comment!

1 Comments

Back to top | Use Dark Theme