Cloudmark SpamNet

June 19, 2002 8:31 AM

Cloudmark SpamNet "Just like Napster allowed us to share our favorite music, Cloudmark SpamNet allows us to share the spam we receive. Individually this reported spam isn't very powerful, but the collective reports of millions of email users networked together blocks virtually all spam on the Internet."
will this put an end to spam? at least it looks like a pretty interesting solution, brought to us by the co-founder of napster, jordan ritter.
posted by HeikoH (25 comments total)

It's just Vipul's Razor for Windows. Literally the same thing. Razor hasn't put an end to spam, although as a component of a larger system such as SpamAssassin it certainly helps. Not one single piece of spam has made it past my SpamAssassin filter in the last couple of months.
posted by majick at 8:37 AM on June 19, 2002

I'm going to give it a shot and see how it works. For those of us (unwillingly) trapped in the hell of MS Exchange their don't appear to be many good server side spam filters available.

If I can do something good on the client side I'll go for it.
posted by dhacker at 8:45 AM on June 19, 2002

well, yes, it is razor. but now that they released the outlook plug-in, the tool should become even more powerful, imho.
posted by HeikoH at 8:53 AM on June 19, 2002

Still, where is the mechanism for stopping spam from ever being sent? Filtering at the mailbox is too late in the process.

PS: I catch more than half of all the spam sent to me with a single rule: Any message with more than five consecutive spaces in the subject header is tagged as spam.
posted by Mo Nickels at 8:53 AM on June 19, 2002

It's always ironic to watch a new commercial product (CloudMark) gain more press than a heavily-used existing one (SpamAssassin). BTW, SA is for Unix right now (some use it in MacOS X) but there is a Windows port in development from what I heard today.

Spamassassin is amazing effective at catching bulk commercial mail, but lately it's been totally failing to catch tons of virus-generated mail (I had like 40 in my inbox this morning, all from one guy, but it's been various senders.)

The auto filters seem to have a tough time with these because 1. they come from valid senders in non rbl-domains, 2. they have innocuous subjects (no !!! or caps) and 3. they don't actually seem to have a MIME payload. It's harmless, but damn annoying.
posted by tomkarlo at 9:18 AM on June 19, 2002

Wierdly, no sign of it in K5... nor at the OSDN.

Hmm, they say its been working for a year, and has 3,000+ members, so is this the best kept secret in Geekdom?
posted by dash_slot- at 9:21 AM on June 19, 2002

Seems like a good idea with the potential for a lot of for mischief...

If I understand correctly, any message tagged as spam is automatically added to the database, right? I took the tutorial, but didn’t see what a legitimate e-mailer can do to see if he is on the list and, more importantly, what can be done to be removed from the list once you’re on it.

If someone had a vendetta against, say Amazon, Amnesty International or whoever, they could tag legitimate messages as spam and create a lot of no-so-easily-resolved trouble. Or am I missing something?
posted by nobody_knose at 9:28 AM on June 19, 2002

I'm a bit confused by the security aspects of it. They include several descriptions of how it works, but none are really very satisfactory. What I've gathered so far is this:

they hash every single message I recieve, and then send the hash to a central server. If the hash matches hashes reported by other users, they call it Spam and move it to the Spam folder.

"Remember, your mail never leaves your computer"

Well, right. A hash does. But I'm still a bit uncomfortable with that. What kind of hash? They say the Hash is unique to my mail, but if they're checking for the same mails across many machines, it can't be unique to my mail right? It's unique to that message.

It still strikes me as a bit suspect. I'd be happier to suck down the whole database on a twice daily basis than to send my hashes off to a giant data store every time I "send/recieve".

I'm not so worried about Cloudmark hacking my mail, but it strike me that somebody along the way might be able to do it...
posted by daver at 9:29 AM on June 19, 2002

Still, where is the mechanism for stopping spam from ever being sent?

well, unfortunately, that would be a very different task, similar to convincing an idiot (= the spammer) that his actions are plain stupid and annoying.

recently, i picked up reporting spam again after not having done so in a very long time. so you wind up checking IP addresses and find out that the email was sent from a server in india and the web site carrying the ad copy is hosted somewhere in russia. that is very, very discouraging. i still wonder if there is really, truely anyone out there that actually buys something that was advertised in a spam message. those guys don't even reveal a company name or address (or they fake one), don't provide you with a customer service phone number, or anything of that nature. who will do business with them? who will go as far as handing over a credit card number? anyone? i sure hope not.
posted by HeikoH at 9:31 AM on June 19, 2002

"Occasionally, a trusted SpamNet user may incorrectly identify a legitimate message as spam. To resolve this rare problem, trusted spam reporters may also revoke incorrect submissions. This process removes the mistaken report from the system and reduces the mistaken-report member's TES rating"

This may answer, nobody knose.
posted by thekorruptor at 10:00 AM on June 19, 2002

Yes, thanks TK.

However, it still doesn't answer:

1) How an e-mailer can find out if they're on the list
2) What a legitimate e-mailer needs to do to get off the list
posted by nobody_knose at 10:10 AM on June 19, 2002

Most of these systems work on a "guilty until proven innocent" edict. In my former job we got nailed by SpamCop numerous times for legitimate business mailings. The only plus side was that, at least with SpamCop, your blacklist block expired over time. In addition, users could report your e-mail as "legitimate" thereby calculating a percentage of Spam vs. Non Spam that led the reporting.
posted by dhacker at 11:05 AM on June 19, 2002

"However, it still doesn't answer: 1) How an e-mailer can find out if they're on the list"

People aren't placed on a list, so there's no way for a legitimate e-mailer to be on it. The database is a collection of hashes assembled from the message body and some header elements. A legitimate message would be blocked only if sent as exactly the same text to several people, and one of those people decided the message was spam and added it to the Razor database. Subsequent messages from the same sender would not be blocked as there is no blacklist involved, and there is no record of where the spam-tagged message came from.

A good description of the mechanism by which this all works is to check out the Vipul's Razor link -- that's the system this new client is using.
posted by majick at 11:38 AM on June 19, 2002

If this is a completely free service, than I don't understand the model. How will they make $?
posted by thekorruptor at 12:13 PM on June 19, 2002

Filtering at the mailbox is too late in the process.

I don't care where in the process spam is filtered out, as long as I don't see it.
posted by kindall at 12:23 PM on June 19, 2002

The new version of Razor, on which this is based, uses a system for deciding how reliable each registered user reporting spam to the database is. That way if someone repeatedly adds legitimate e-mails to the central database, which then has to be re-reported by other users as not spam, then their contributions will get a lower score in the future.

In time, users who have been regularly and reliably reporting spam over a long period will have their submissions rated higher than someone who has just registered and adds in a load of legitimate e-mail for a laugh.
posted by kerplunk at 12:37 PM on June 19, 2002

While spam is annoying, I'd be a much happier boy if I could filter the 30-40 Klez worm e-mails I get a day. Outlook eats the attachment so I can't create a rule off of that. And there's no "if Outlook eats a potentially dangerous attachment" rule.
posted by frenetic at 2:20 PM on June 19, 2002

So I assume that Outlook Express is different from Outlook 2000/XP, right?
posted by crunchland at 5:11 PM on June 19, 2002

Anyone know of a similar product for those of us who use Eudora?
posted by mattee at 6:53 PM on June 19, 2002

crunchland, Outlook Express is the "free" IE-bundled mailnews client; it shares no code with Outlook 2000/XP which is a component of the very much not free Microsoft Office.
posted by majick at 6:57 PM on June 19, 2002

Figured as much, thanks. So much for ridding my world of spam.
posted by crunchland at 9:46 PM on June 19, 2002

I highly, highly reccommend Mailshell (and to show you how much I won't even put in one of those damned affiliate links) for whatever program you use. I went and got my own domain through the service. Mailshell can "clean" existing accounts and forward them to you.

Since I started using it 3 months ago I would say that I agree with their claims of 99% Spam removal.
posted by dhacker at 6:02 AM on June 20, 2002

Mattee, Cloudmark is working on other mail client plug-ins for release soon.
posted by jasonshellen at 11:24 AM on June 20, 2002

I'm skeptical about if Razor can seriously improve the selectivity of SpamAssassin. I would much rather deal have a good content analysis script than deal with a P2P system. For a cheap laugh, its worth looking at the SpamAssassin test page for a list of how they spot spam.
posted by KirkJobSluder at 2:54 PM on June 20, 2002

SpamAssassin is a functional superset of Razor. One of the many tests it makes is for a listing in the Razor database. While it may not cripple SA substantially to remove the automatic Razor test, I don't see any point in going out of my way to disable it. Leaving the test in place might add a second or two to my mail latency, but to me that's noncritical.

It was more worth my time to craft 8 or 10 additional rules (such as my "Nobody *really* writes email in FrontPage" test) and skew the test scoring upward for tests where the GA scoring seemed irrational.
posted by majick at 3:38 PM on June 20, 2002