Cloudmark SpamNet

December 19, 2002 7:37 PM

Distributed spam filtering. Sure, your spam filter may be hot stuff, but Spamnet takes filtering to the communal level. With its easy install, point and click simplicity, and Outlook support could Spamnet be the SpamCop for the masses?
posted by skallas (36 comments total)

Maybe when they start supporting Outlook Express.
posted by katieinshoes at 7:43 PM on December 19, 2002

i've been using it since reading about it here. it does a great job of getting rid of spam and i've noticed it has gotten more accurate over time. and when it doesn't automagically put the spam in spam folder, it is somewhat fun to hit the 'block' button and zap the spam.
posted by birdherder at 7:56 PM on December 19, 2002

I'm with katieinshoes, while this is definitely geeky and drool-worthy and all that good stuff, 99.999999999% of my customers use OE (which also means they can't use SpamAssassin, dang it all) and thus I can't, as much as I'd love to, point any of them to this resource.

Sigh.
posted by WolfDaddy at 7:57 PM on December 19, 2002

... which also means they can't use SpamAssassin ...

Quoted from SpamAssassin's website: However, Deersoft sell Exchange and Outlook versions of SpamAssassin, and there is also Spamnix, a commercial Eudora plug-in. Other interfaces for plugging SpamAssassin into your mail systems are listed on this page.
posted by chrish at 8:13 PM on December 19, 2002

chrish, commercial solutions would be fine if our customers thought it was incumbent upon them to filter their spam.

But they don't ... most of our customers will loudly declaim that it's the ISP's job to filter spam (and this isn't helped by adverts from most major online providers claiming they have tools that "get rid of all spam!")--though we'd better not do it too aggressively, else we get nailed with censorship complaints.

The server-side SpamAssassin solution we chose was a compromise. Neither we on the ISP side of things nor our customers are 100% happy with it (especially since in the last two months the number of e-mails designed to *specifically* bypass many of SpamAssassins checks has increased by a huge amount).

How to change the end-user's perceptions of how and why they receive spam is a task that's almost impossible to complete. The end-user believes they do nothing to encourage their receipt of ever-increasing amounts of spam (even though they might have blithely agreed to it when subscribing to a porn site, for example), and often blame their ISP--accusing us of selling our user list is the prime complaint, for example.
posted by WolfDaddy at 8:28 PM on December 19, 2002

I've been using it for four months; initially it got about 95% of my Spam (on the order of 30 to 40 pieces a day), now for whatever reason it's dropped off to about 15% of my Spam, which makes it much less useful. Eventually I'll have to switch to a permission based system, where no email gets through without being verified by a human sender.
posted by jonson at 8:31 PM on December 19, 2002

I developed this theory about people and fighting spam after switching over to Apple's mail.app. I had played around with spamassassin and a bunch of other time wasting tarballs, and realized how much time I was wasting tweaking on the command line in order to save mere seconds in my GUI.

My theory is this:
Spam tools only need to be about 75% effective in order for people to be happy with them. Above that threshold, fighting spam is a game that only requires active participation in order to feel some semblance of winning the battle. Below that threshold, people start looking for other solutions.

In a world where people get their mail on different computers at work and home, different platforms and through different interfaces (webmail), there really isn't a silver bullet that won't be eclipsed by tenacious spammers eventually. That's a hard business model for any software developer to thrive within. Oh, and those of you on windows with a pop3 mail client that can't use spamnet or spamassassin should check out the freeware called spampal. It acts as proxy in between your client and your server through a 127.0.0.1 loopback address.
posted by machaus at 8:55 PM on December 19, 2002

Let me put in a quick endorsement for POPFile (second link above). It's free, dirt-simple to install, deadly effective, and works with Outlook Express (and any other POP3 email reader, for that matter). I've tried all the other freeware filtering programs, and none of them have made me evangelical like POPFile has.
posted by ewagoner at 8:59 PM on December 19, 2002

I've been using SpamNet for a few weeks now, with excellent results. I'd heartily recommend it.
posted by muckster at 9:15 PM on December 19, 2002

I can put in another recommendation for popfile. One day of training and it was about 70% accurate. A week and it is running about 95%+. It's not just for filtering spam, either. It can sort into any categories you can define and train it to recognize (and training is dead simple). Sorting work email from personal email, for example.

A hybrid of popfile and spamnet would be great. It would be as nearly perfect as I can imagine a spam filter being. It could learn to recognize new spammer tricks in literally a matter of minutes after they were first used, would not care what email program you use, and wouldn't cause the problems that false positives can present when they happen on the ISP side.
posted by Nothing at 9:22 PM on December 19, 2002

Popfile seems to just sort incoming email into different boxes based on rules you give it...Outlook Express does that. What am I missing?
posted by rushmc at 9:45 PM on December 19, 2002

If you understand any statistical analysis, check out Paul Graham's A Plan for Spam. I'm using a Bayesian plugin for Outlook called Spammunition, and it was 99% accurate from day one, given corpora of "good" and "bad" email.

As Paul Graham says:
"Feature-recognizing spam filters are right in many details; what they lack is an overall discipline for combining evidence."
posted by bruceo at 10:10 PM on December 19, 2002

bruceo, thanks for the tip - I've just uninstalled Cloudmark, and I'll give Spammunition a try.
posted by jonson at 11:07 PM on December 19, 2002

Is there any decent server-side tools besides spamassassin?

I can't use cloudmark or any plugins because I read my email 50% of the time at home on a mac running entourage, and 50% of the time I read email in eudora on a PC at work. I don't know who these tools are catered too, but I've been reading my email in more than one place for at least the past 5 years, and I can't use a client-side solution.

Heck, I'd pay $5, $10, or even $20 a month to someone offering a constantly updated, secure IMAP/POP email server that tried to be spam free using any/all of these tools. I want to read useful email, not configure server scripts all day, why doesn't someone get a colocated box and become a spam free email host? I'd forward all my mail to them and download it to my multiple machines for reading. I'm a designer and not much of a linux server admin. I'd happily pay someone else to keep my mail server up for me, if someone would just offer the service. I'm sure other people would be open to the service as well.

I've tried spamcop, and it sucks. It blacklists whole swaths of the internet, has been overly aggressive from day one while still allowing spam to get through, and the owner is a bit of a nut about criticism. I sincerely wish someone else would enter the fray, because I'd happily forward all my mail there.
posted by mathowie at 11:20 PM on December 19, 2002

Actually, it does even less than that, Rushmc. It only adds a header to each message which you then have to filter for in your email program. The part you are missing is that the rules it uses are based on Bayesian probabilities, and they work much better than the simple text matching ones in outlook. You train it by showing it what spam looks like and what good mail looks like (or what mail from any category you have defined looks like) and it sorts based on the probability of a message belonging to one of your categories.
posted by Nothing at 11:37 PM on December 19, 2002

mathowie: a friend of mine has his email hosted by EvilMail. In addition to decent uptime standard and lots of cool domains to choose from, evidentally the business owner has a special, hearty vendetta against spammers. They always run the latest and greatest in server-sized junk killers.

Their latest status report
posted by blindcarboncopy at 11:41 PM on December 19, 2002

i need ot look at these programs closer but they mostly just filter the e-mails into folders, correct?

my problem is that if they filter the e-mails into something like a "suspected SPAM" folder then i still have ot go through that folder to make sure it was really SPAM.

on the other hand, my level of paranoia is high so i'm always concerned that if the e-mails are being filtered before they get to me then maybe something is being filtered that i didn't want filtered.

i used to get more SPAM but now i may get 1 or 2 SPAM e-mails a day so i don't mind deleting them and ignoring it.
posted by suprfli at 12:10 AM on December 20, 2002

WolfDaddy, have you tried Open Relay Filter for Exchange? I've installed it on one client site and it does a decent job of filtering out spam using SpamCop's blacklist or any of the other dozen or so blacklists it supports. Its unbelievably cheap compared to the commercial Spamassassin solution. I think its still $25 dollars per server, unlimited mailboxes. Its not a "smart" filter, but I've had no problems with pass-through rules.
posted by skallas at 12:11 AM on December 20, 2002

suprfli - I think you've understood the core of the problem here. There is no 100% accurate solution against spam. It just can't and won't exist. Whatever probability-driven or blacklist-using processes, you're always going to get (i) bad stuff creeping through or (ii) good stuff getting blocked. I'd prefer to err on (i) rather than (ii), as it only takes a second or two to delete the crap. And if you can use Spamnet/Cloudmark, then their 'block' button deletes the email and sends a report back to them that the thing is spam.

There are methods out there that will work on trusted email addresses - can't think of the name of them at the moment. The first time someone emails you, they are sent an automatic response. If they reply correctly to that, they're allowed to email you. So spammers using fake email addresses don't get through.
posted by humuhumu at 1:32 AM on December 20, 2002

Network administrators, the word you are looking for is Postini.

85% of mail to our users yesterday filtered as spam. All filtered mail available to users via Postini's web interface. No support required from my staff. Cheap. Spam doesn't use my bandwidth.

My customers mock their friends that don't have access to it.

They. Are. The. Bomb.
posted by dglynn at 2:12 AM on December 20, 2002

can't you run spamassassin on the mail server? that way outlook users receive the benefits.

(i'm very happy with my mail setup: exim pulls mail from my isp's pop account and delivers it locally; procmail calls spamassassin and filters to maildir folders; courier imap provides local imap access; squirrelmail gives web-based access to the imap (https with password protection to my local machine (running iptables firewall, of course) and lets me read email from anywhere - i have a "dynamic" cable modem address, but it has been constant since i configured this, a couple of months ago; if it starts to change i intend to use a script to update a link on my isp's site that gives the current value...))
posted by andrew cooke at 4:47 AM on December 20, 2002

Another thumbs up from a satisfied Cloudmark user. Blocking occasional spams is kinda fun... as long as it's only an occasional one. I wish it kept some kinda stats, though -- "I HAVE BLOCKED 20,003 PIECES OF SPAM!"

... and, of course, it's hard to beat the price, even if I don't really understand their business model.
posted by ph00dz at 5:23 AM on December 20, 2002

If you are somewhat familiar with Windows, and like to work with new applications, then POPFile is for you.

If on the other hand your knowledge of Windows is limited and all you do is email, and browse, then stick with Spamnet.
posted by CrazyJub at 5:33 AM on December 20, 2002

ph00dz - the business model is that they do this for the 'little people' and then sell bigger server based stuff to companies. In a positive way, the free users are improving what they can offer businesses.

Server based? Sorry, I mean 'enterprise level'. Must get the lingo right...
posted by humuhumu at 5:46 AM on December 20, 2002

Mailwasher is nice to use. Preview the messages, bounce back and delete the spam, leave the rest, and download only real messages to your client.

Sure, it's not the fanciest one around, but I have 0 spam in my Inbox, Deleted folder, and my whole .pst file for that matter.
posted by adampsyche at 5:53 AM on December 20, 2002

Re what mathowie was asking about (secure POP/IMAP host with very good spam filtering), I've been using Fastmail.Fm for the past 10 months and its been doing just exactly that for me.
posted by ssheth at 6:50 AM on December 20, 2002

fastmail.fm is good. It sometimes has trouble with valid emails that look like spam (Dilbert for example), but otherwise, it's fast, looks good, has pop3, imap and web access. Plus - it filters for viruses as well as spam. Oh - and while I'm at it... If anyone asks, user "seanbamforth" reccomended you.
posted by seanyboy at 8:09 AM on December 20, 2002

Dunno. I run everything via four blackhole lists, then string and address filter, then through AV on my server. Of course, I also have a whitelist. I just wish my mail hosts ran as tight a ship as I do. The sad thing is: It's a hobby for me, not any kind of business.
posted by Samizdata at 8:17 AM on December 20, 2002

Thanks, Nothing.
posted by rushmc at 8:51 AM on December 20, 2002

can't you run spamassassin on the mail server? that way outlook users receive the benefits.

OE doesn't have the ability to create rules based on non-standard e-mail headers, which is what SpamAssassin creates to help people filter their mail. The best one can do with OE is to create filters for mail that's not specifically addressed to themselves, which creates all sorts of false positives if they're on any sort of valid mailing list.

Can someone explain why no one with interest in this field cares to develop for Outlook Express? It can't be an anti-Microsoft thing ... so why?
posted by WolfDaddy at 10:12 AM on December 20, 2002

I have procmail run a simple whitelist that lets non-spammers add themselves. I don't get any spam.
posted by nicwolff at 10:15 AM on December 20, 2002

OE doesn't have the ability to create rules based on non-standard e-mail headers, which is what SpamAssassin creates to help people filter their mail.

WolfDaddy, have you installed spamassassin server-side before? Every single message has its subject appended to read:

*****SPAM***** Original Subject Name

If Outlook Express can't make a rule based on subjects containing a key phrase, OE is the problem, not spamassassin.
posted by mathowie at 10:23 AM on December 20, 2002

Hmm, mathowie, I'm no sysadmin, but our implementation of SpamAssassin adds headers, it doesn't modify the subject line.

The headers added are X-SPAM-STATUS (which is either YES or NO) and X-SPAM-LEVEL (which is a number of asterisks, the higher the number, the more likely it's spam...so people who can do it can filter either on the Yes or No flag, or customize a filter further based on tolerance via the number of *'s).

My sysadmin is also a MeFite, I'll ask him to review this thread for good ideas :-)
posted by WolfDaddy at 11:30 AM on December 20, 2002

My sysadmin reports we didn't modify subject lines because as a fairly large ISP, we judged that the number of customers who'd scream at us for modifying (or, worse, "reading") their e-mail would far outweigh the benefits of giving easy SpamAssassin functionality for OE users. Not too sure I agree with this line of thought, but I don't make these types of decisions.

However, he *also* informed me that Bayesian rules support is being built into SpamAssassin 2.5 :-)
posted by WolfDaddy at 11:37 AM on December 20, 2002

Here I am. We decided that adding the ****SPAM**** subject tag to every mail tagged as spam would create more complaints from users whose legitimate mails were tagged incorrectly, since the subject tag couldn't be turned off. We're very close to getting a per-user preferences and server-side filtering system up that will allow everyone to do their own tagging and filtering, which should help a lot. Sort of like postini, but done on our own terms instead of theirs.

Another good idea for those server admins out there is to make the blacklists on your backup MX servers much more aggressive than the blacklists on your primary server. Lots of spammers seem to send directly to the backup MX's, bypassing the primary, in the hopes of getting around the filters on the primary server. Making the MX servers tougher than the primary makes that strategy backfire. :)
posted by fengshui at 11:46 AM on December 20, 2002

OK, I know I'm way late to this discussion, but I have an issue which I haven't seen addressed here--how easy or difficult would it be to subvert Spamnet, via a quasi-Googlebomb type effect?

For example, suppose I am anti-X. I and some of my anti-X friends join some pro-X mailing lists and start submitting all of the pro-X messages as spam. How many friends would I need before SpamNet would start blocking at least some pro-X messages as spam?
posted by DevilsAdvocate at 12:35 PM on December 23, 2002