Mobile app version of vmapp.org
Login or Join
Samaraweera270

: What dangers await if I block non-standard, non-major-usa search engine bots from my USA only website? I noticed tons of bandwidth being used by non-USA search engine bots, so I began blocking

@Samaraweera270

Posted in: #RobotsTxt #SearchEngines #Seo #WebCrawlers

I noticed tons of bandwidth being used by non-USA search engine bots, so I began blocking them in an effort to save bandwidth and cpu cycles for actual users and the search engines they come from (Google, Bing, Yahoo, Ask, etc.).

Other than potentially losing some international traffic (which isn't really important to us since all of our content is very USA-centric), what additional dangers should I be concerned about?

I'm using a modified version of Jeff Starr's User Agent Blocklist

10.01% popularity Vote Up Vote Down


Login to follow query

More posts by @Samaraweera270

1 Comments

Sorted by latest first Latest Oldest Best

 

@Si4351233

Love it, Dangers...

Anyway, if you depend on overseas sales from international customers, you eliminate their ability to find your site through their preferred search. Eliminating Yandex, for example will mean that Russian Federation and Eastern European countries will be finding results elsewhere as you are now invisible to any searches through that SE.

If you aren't doing overseas sales to the countries that search engine represents, you're saving (as you've noted) your server a lot of wear and tear for no good reason at all. The only harm I noticed when bringing a new site on line and blocking certain foreign search engines from the reindex flurry due to their ability to overload the website was reduced access from the language pools/countries that they represented. Once Google/Bing/Yahoo got through battering the site and we'd improved its performance to handle the load, we added Yandex, et. al. as we do sales in these countries. Yandex in particular was very quick on getting the site back in and fully indexed.

So in short, no SEO danger was noted, except what was expected, loss of visibility on that search engine.

10% popularity Vote Up Vote Down


Back to top | Use Dark Theme