: Securing webserver from spam/phishing/malware - beyond RBLs? We offer a service that requires URLs from our domain to be linked to external URLs. About ten years ago, we discovered that this
We offer a service that requires URLs from our domain to be linked to external URLs.
About ten years ago, we discovered that this functionality can be abused by miscreants -- to link to malicious URLs. So we have multiple protection mechanisms, using Google Malware database (hourly updated) to check the domain against, and similarly checking the domain and IP on spam Relay Black Lists such as SpamHaus and SpamCop.
Yet, there are always new and upcoming domains to which people can link from our site, domains that are NOT YET in any RBLs.
Some such domains have been (ab)used by people recently, and as a result, Google has included our domain on their misguided malware list too.
Questions:
Apart from the automation of RBL etc, what am I missing, what else can we do to work against spammers of this kind? Our RBL check is basically "live", but how can we capture domains that are malicious before they're in RBLs?
Any good advice on getting Google to un-list our website?
(We actually blocked Google's News website on our site too, because news.google.com has a vulnerability where there "goto" URL can take any site behind the URL and forward the user to it...so we spotted many spammers using this functionality. Google has not fixed this weakness on their site, so we have blocked news.google.com from being used on our site. Could Google have added us to the malware DB as a result of this, in a vindictive move?)
Many thanks for any pointers or advice regarding best practices for this kind of server protection.
More posts by @Phylliss660
1 Comments
Sorted by latest first Latest Oldest Best
No, there isn't really much else that you can do. The bad guys can create new domains way faster than anyone can detect that these new domains are malicious - that's just the nature of the Internet. The simplest way around it is to have someone approve all created links, though that is going to be a very time-consuming process.
Since you aren't the first person to run such a site (and likely won't be the last), you should take a hint from what other services are doing. Bit.ly, which some say is the largest URL-shortener out there, uses lists from Google Safe Browsing and SURBL (among others) for its malicious link detection, and shows a page directing users to StopBadware.org and AntiPhishing.org. Twitter's t.co shortener also uses the Google Safe Browsing list.
Another thing you might be able to do is to look for URLs that seem like they are obfuscated and prevent them from being used. See here for some information about what to look for.
Finally, if you're up for a real challenge, you could design some kind of machine learning, reputation-based, and/or heuristic "Malicious Link Detection System" that works better than those that are currently available, solve your problem, and get really rich selling your solution to other sites :).
To get Google to unlist your website, go the the Safe Browsing Diagnostic page at www.google.com/safebrowsing/diagnostic?site=YOURSITE.COM and follow the instructions in the "Next Steps" section at the bottom.
I very highly doubt that Google would blacklist you as "revenge" for blacklisting them. Google's blacklists are mostly automated, and are largely reputation-based. See the Safe Browsing section (more specifically, the "FAQ" subsection) of Google's Transparency Report for more information.
If you sign up for Google Webmaster Tools, you will get notification emails from Google if they detect malware, and you will be able to request a re-review. (There are also lots of other useful things you can do with Webmaster Tools.)
Terms of Use Create Support ticket Your support tickets Stock Market News! © vmapp.org2024 All Rights reserved.