: App Engine (Google) Agent - safe to block? From an SEO perspective I am seeing my website 'cloned' in a proxy site, which is on a subdomain of Google's appspot.com. And these 'clones' are

Posted in: #Google

I am seeing my website 'cloned' in a proxy site, which is on a subdomain of Google's appspot.com. And these 'clones' are being indexed, so I want to stop them.

I can do it, for appspot particulalury, by adding this to myhtaccess file:

RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} ^AppEngine [NC]
RewriteRule ^(.*)$ - [F,L]

Before I put it live, I just wanted to find out if this agent is actually used by Google for it's usual work/indexing/crawling. Would I be damaging my site's SEO if I block the AppEngine agent?

Thanks

10.01% popularity Vote Up Vote Down

: Does a cache busting fingerprint in image file names hurt SEO rankings? The Ruby on Rails asset pipeline automatically adds a fingerprint to assets (JS, CSS, and other files in the assets folder

@Moriarity557

Posted in: #Filenames #Images #RubyOnRails #Seo

1 Comments

: How to get indexed for particular keyword in google? i have tried so many ways that to rank mobile app development keyword for my website, it is not showing even in last page of google search

@Moriarity557

Posted in: #Google #Googlebot #GoogleSearch #GoogleSearchConsole #Seo

0 Comments

: Does Google DFP honor the Do Not Track (DNT) browser setting? In the sites I manage I've tended to default to honour the Do Not Track request (if it exists), I'm working on Drupal sites at

@Moriarity557

Posted in: #GoogleDfp

0 Comments

: Can I use CloudFlare DNS without using their service and point DNS to Incapsula? I have been using CloudFlare for a few years as well as Incapsula and while CloudFlare excels in CDN, it is

@Moriarity557

Posted in: #Cdn #Cloudflare #Dns

1 Comments

Login to post a comment!

1 Comments

Sorted by latest first Latest Oldest Best

@Heady270

The AppEngine user agent is crawling done by apps created through Google's app engine. These are third party applications that are not related to Googlebot or the Google search engine.

From an SEO standpoint it is safe to block AppEngine user agent requests. Here is a thread on WebmasterWorld from folks who have done so: www.webmasterworld.com/search_engine_spiders/4475764.htm
AppEngine apps have caused problems for others as well. Somebody complains about one on ServerFault. If you find one that is crawling excessively, you can report it to Google here: code.google.com/support/bin/request.py?contact_type=AppEngineContact The AppEngine user agent has the name of the app that is causing problems near the end: appid: myexampleapp. You can use that when reporting to Google or learn more about that specific app by visiting its page: myexampleapp.appspot.com

10% popularity Vote Up Vote Down

Feed

: App Engine (Google) Agent - safe to block? From an SEO perspective I am seeing my website 'cloned' in a proxy site, which is on a subdomain of Google's appspot.com. And these 'clones' are

More posts by @Moriarity557

: Does a cache busting fingerprint in image file names hurt SEO rankings? The Ruby on Rails asset pipeline automatically adds a fingerprint to assets (JS, CSS, and other files in the assets folder

: How to get indexed for particular keyword in google? i have tried so many ways that to rank mobile app development keyword for my website, it is not showing even in last page of google search

: Does Google DFP honor the Do Not Track (DNT) browser setting? In the sites I manage I've tended to default to honour the Do Not Track request (if it exists), I'm working on Drupal sites at

: Can I use CloudFlare DNS without using their service and point DNS to Incapsula? I have been using CloudFlare for a few years as well as Incapsula and while CloudFlare excels in CDN, it is

Login to post a comment!

1 Comments

Back to top | Use Dark Theme