: Is it ok to block search engine bots from China and Russia on USA based eCommerce site? After blocking all traffic from China and Russia ( using Incapsula ) on an eCommerce application which

Posted in: #Performance #Seo #WebTraffic

After blocking all traffic from China and Russia ( using Incapsula ) on an eCommerce application which only ship into USA, I noticed a significant performance improvement. Is there any drawback of country wide crawlers blocking consider SEO? Am I losing anything?

Fig 1: Look at AWS cloud watch. CPU usages, web traffic significantly reduced.

Fig 2: Server response time incredibly improved - Newrelic.

Fig 3: App index jumped up - Newrelic.

10.01% popularity Vote Up Vote Down

: SEO: Moving domain without having access to the old domain The mother of my girlfriend has a contract with a company that provides her website and domain. The contract costs 360€ per year

@Jennifer507

Posted in: #Contract #Googlebot #Seo

1 Comments

: Gh-pages not allowing multiple subdomains I am following the Github Pages tutorial. I thought I could have two domains: api.website.com and website.com, and the first is created via a folder called

@Jennifer507

Posted in: #Cname #Dns #Godaddy #MultiSubdomains #Subdomain

1 Comments

: Make 404 error not redirect when on different domain/subdomain I have a PHP file on my error subdomain (https://errors.example.com). And this is meant to serve as the error page for all subdomains.

@Jennifer507

Posted in: #Htaccess #Php

1 Comments

: How should I protect "secret" links I send in Emails from being indexed by search engines? I have noticed that bing/msnbot tries to index pages that were only ever linked to in one single

@Jennifer507

Posted in: #WebCrawlers

1 Comments

Login to post a comment!

1 Comments

Sorted by latest first Latest Oldest Best

@Cofer257

Unless you get meaningful business from Yandex or Baidu then there's no disadvantage to blocking them.

However, they're known to ignore robots.txt. I'm unsure if this has changed in the last 4 years but you'll be looking at a server level block. Their user agents are easy to spot, a .htaccess example would be:

SetEnvIfNoCase User-agent “Baidu” spammer=yes
SetEnvIfNoCase User-agent “Yandex” spammer=yes

<Limit GET PUT POST>
order deny,allow
deny from env=spammer
</Limit>

10% popularity Vote Up Vote Down

Feed

: Is it ok to block search engine bots from China and Russia on USA based eCommerce site? After blocking all traffic from China and Russia ( using Incapsula ) on an eCommerce application which

More posts by @Jennifer507

: SEO: Moving domain without having access to the old domain The mother of my girlfriend has a contract with a company that provides her website and domain. The contract costs 360€ per year

: Gh-pages not allowing multiple subdomains I am following the Github Pages tutorial. I thought I could have two domains: api.website.com and website.com, and the first is created via a folder called

: Make 404 error not redirect when on different domain/subdomain I have a PHP file on my error subdomain (https://errors.example.com). And this is meant to serve as the error page for all subdomains.

: How should I protect "secret" links I send in Emails from being indexed by search engines? I have noticed that bing/msnbot tries to index pages that were only ever linked to in one single

Login to post a comment!

1 Comments

Back to top | Use Dark Theme