: Searches for "more information about text formats" I have a site using Drupal as CMS, and got "infected" badly with spammer bots. Moreover, recently I signed up for Google's Webmaster Tools,

Posted in: #Drupal #RobotsTxt #Seo #SpamPrevention

I have a site using Drupal as CMS, and got "infected" badly with spammer bots.

Moreover, recently I signed up for Google's Webmaster Tools, and I see that one of the top search queries that result in my site is "more information about text formats"
And I know that this block of text appears in drupal sites.

So I'm guessing that this is the bots' way to find spammable drupal sites. So is there a precaution for this? I already made robots.txt to Disallow: /drupal-subfolder/ but apparently this does not affect malicious bots.

10.01% popularity Vote Up Vote Down

: Getting a lot of '/_' errors from webmaster tools I'm using a WordPress site and I thought I got all the kinks out of it. For some reason Webmaster Tools is crawling my website and showing

@Cugini213

Posted in: #GoogleSearchConsole #Seo

1 Comments

: Should I use rel=index or rel=contents in this instance? I’m creating an MMA website. There’s the home page, there’s a fighters section whose index page lists fighters in the organisation,

@Cugini213

Posted in: #Html #Seo

1 Comments

: Google Webmaster Tools average position? I see that my site's average position is 7.0 for a certain search query. However, it is kind of impossible for now. And when I search that query, I

@Cugini213

Posted in: #GoogleIndex #GoogleSearchConsole #Ranking

1 Comments

: One million redirects What's the best way to redirect 1,000,000 unique URLs to another 1,000,000 unique URLs? These pages are all combinations of multiple search filters and queries. We're migrating

@Cugini213

Posted in: #301Redirect #Htaccess #Redirects

2 Comments

Login to post a comment!

1 Comments

Sorted by latest first Latest Oldest Best

@Lee4591628

You can prepare a list of bots which you don't want to allow on your website & mention them in robots.txt like

User-agent: <bot_identifier>
Disallow: /

Here is a exhaustive bad bots list.

But still many bad bots will not respect it & crawl your site. For that you need to install application like Fail2ban to restrict the crawlers which are crawling at a higher rate.

10% popularity Vote Up Vote Down

Feed

: Searches for "more information about text formats" I have a site using Drupal as CMS, and got "infected" badly with spammer bots. Moreover, recently I signed up for Google's Webmaster Tools,

More posts by @Cugini213

: Getting a lot of '/_' errors from webmaster tools I'm using a WordPress site and I thought I got all the kinks out of it. For some reason Webmaster Tools is crawling my website and showing

: Should I use rel=index or rel=contents in this instance? I’m creating an MMA website. There’s the home page, there’s a fighters section whose index page lists fighters in the organisation,

: Google Webmaster Tools average position? I see that my site's average position is 7.0 for a certain search query. However, it is kind of impossible for now. And when I search that query, I

: One million redirects What's the best way to redirect 1,000,000 unique URLs to another 1,000,000 unique URLs? These pages are all combinations of multiple search filters and queries. We're migrating

Login to post a comment!

1 Comments

Back to top | Use Dark Theme