Mobile app version of vmapp.org

Moriarity557

: Set bot crawl delay 10 seconds EXCEPT Googlebot? Is it possible set the crawl delay for all bots to 10 seconds except Googlebot and Bing/Yahoo which can proceed at any pace? I like being indexed

Posted in: #RobotsTxt #SearchEngines #WebCrawlers

Is it possible set the crawl delay for all bots to 10 seconds except Googlebot and Bing/Yahoo which can proceed at any pace?

I like being indexed by all these bots but with Yandex and Baidu it is just a bit overwhelming for my app which is very very database heavy.

Eventually I will cache stuff but right now i'm in beta. I do enjoy the traffic. Just need to slow these bots down a bit.

10% popularity Vote Up Vote Down

Login to follow query

More posts by @Moriarity557

Moriarity557

: Google Cloud Platform - Application suddenly start charging billing hours? I have several Java applications running on Google Cloud Platform. There is a free quota of 28 frontend instance hours

Posted in: #CloudHosting #Google #GoogleAppEngine

Moriarity557

: Sending bulk email with different names in reply address and spam filtering I have a web service where I send out bulk email on behalf of customers. The sending email address is the same

Posted in: #BulkEmail #Email #EmailAddress #Spam

Moriarity557

: Change Nameservers on a domain that I don't control I'm working for a client that's in a bit of a pickle. Their domain is hosted through Nearly Free Speech by a webmaster who has fallen

Posted in: #DomainRegistrar #Nameserver

Moriarity557

: SEO put no-indexed pages in sitemap or not? On my site I have products. Each product has many releases. Each release has many changelogs. A changelog is one of many categories (eg, "Bug

Posted in: #Noindex #Seo #Sitemap

Login to post a comment!

0 Comments

Sorted by latest first Latest Oldest Best

Terms of Use Create Support ticket Your support tickets Stock Market News! © vmapp.org2024 All Rights reserved.

Back to top | Use Dark Theme