On Karma and building web reputation systems
March 23, 2010 11:47 AM Subscribe
On Karma: Top-line Lessons on User Reputation Design is an excellent overview of reputation system design concepts from the excellent-in-general blog of Randy Farmer and Bryce Glass, authors of the recently-released O'Reilly book Building Web Reputation Systems.
Of all the blogs I read, I like slashdot's system the best. From what I can tell, this guys advice sounds similar (or maybe its just the use of 'karma')
posted by rosswald at 12:08 PM on March 23, 2010
posted by rosswald at 12:08 PM on March 23, 2010
as someone who's best friend almost lost his job because of a negative (and quite probably spurious) review on yelp, i gotta say: This. This is what scares me most about the future. "have a bad day, be fucked forever."
also, doesn't publishing this sort of info=libel ?
posted by sexyrobot at 12:10 PM on March 23, 2010
also, doesn't publishing this sort of info=libel ?
posted by sexyrobot at 12:10 PM on March 23, 2010
That said, I do take account of karma in one place: eBay, even though I already know it's quickly becoming useless.
Yeah, I don't think it's any coincidence that eBay is a recurring topic on the blog, as one of the grand-daddies of applied reputation systems, both in terms of how it functions and how people break it.
posted by cortex at 12:11 PM on March 23, 2010
Yeah, I don't think it's any coincidence that eBay is a recurring topic on the blog, as one of the grand-daddies of applied reputation systems, both in terms of how it functions and how people break it.
posted by cortex at 12:11 PM on March 23, 2010
Someone needs to do a serious academic paper entitled "XFN Links And Why They Feel All Stalkery"
posted by Jofus at 1:02 PM on March 23, 2010
posted by Jofus at 1:02 PM on March 23, 2010
Ah, so that's what favorites are for.
The mechanics of online reputation/karma systems has always kind of fascinated me. It pains me to see some websites try to half-assedly implement a system that ends up encouraging a completely different type of behavior than what was intended. The guys behind Stack Overflow used to talk about their thought processes a lot back when the site was still in its early stages. Strange how it took so many years for a book on this topic to get published.
posted by The Lurkers Support Me in Email at 1:19 PM on March 23, 2010
The mechanics of online reputation/karma systems has always kind of fascinated me. It pains me to see some websites try to half-assedly implement a system that ends up encouraging a completely different type of behavior than what was intended. The guys behind Stack Overflow used to talk about their thought processes a lot back when the site was still in its early stages. Strange how it took so many years for a book on this topic to get published.
posted by The Lurkers Support Me in Email at 1:19 PM on March 23, 2010
Of all the blogs I read, I like slashdot's system the best.
I don't know, I always thought it mainly just encouraged groupthink (and trolling). Getting off the ground was not always easy, but once your karma hit +2, you were much more likely to be modded up and it was quite easy to craft a comment that would gain at least one or two (+1, Insightful)'s. So instead of interesting comments, the system encouraged posting comments most users would agree with. The worst bit, of course, was effectively giving one group of users power over other users' posting rights by making a certain number of downmods lead to an automatic ban - so that if you regularly expressed opinions different from the groupthink, someone with an axe to grind could spend all their mod points modding your comments down, leading to a posting ban. This drove many people away from the site.
posted by daniel_charms at 2:51 PM on March 23, 2010
I don't know, I always thought it mainly just encouraged groupthink (and trolling). Getting off the ground was not always easy, but once your karma hit +2, you were much more likely to be modded up and it was quite easy to craft a comment that would gain at least one or two (+1, Insightful)'s. So instead of interesting comments, the system encouraged posting comments most users would agree with. The worst bit, of course, was effectively giving one group of users power over other users' posting rights by making a certain number of downmods lead to an automatic ban - so that if you regularly expressed opinions different from the groupthink, someone with an axe to grind could spend all their mod points modding your comments down, leading to a posting ban. This drove many people away from the site.
posted by daniel_charms at 2:51 PM on March 23, 2010
Oh sure, you implement a system like this and the next thing you know Peter Wiggin is Hegemon.
posted by George_Spiggott at 4:16 PM on March 23, 2010 [2 favorites]
posted by George_Spiggott at 4:16 PM on March 23, 2010 [2 favorites]
....eBay is a recurring topic on the blog, as one of the grand-daddies of applied reputation systems, both in terms of how it functions and how people break it.
That link about how people 'broke' the ebay reputation system was pretty interesting. When the smoke cleared though it looks like the system did okay. Sellers who were intermediating Drop-shippers had to pay an extra cost for resolving problems.
posted by storybored at 6:40 PM on March 23, 2010
That link about how people 'broke' the ebay reputation system was pretty interesting. When the smoke cleared though it looks like the system did okay. Sellers who were intermediating Drop-shippers had to pay an extra cost for resolving problems.
posted by storybored at 6:40 PM on March 23, 2010
I believe the quality of commentary on a site is much more dependent on its community than its rating system. I think the reason that Slashdot, for example, was excellent for so long was because it was established very early and quickly became the place to discuss “news for nerds”.
Unfortunately I think any open online community that has established itself for insightful commentary and analysis devolves in quality over time as more mediocre thinkers increasingly become the majority. I think that a comment rating system only helps to delay the slide in quality and can't every really completely forestall it. I've certainly noticed the comment quality on Slashdot sliding over the last two or so years.
Metafilter and some other sites are interesting in that rather than using a full karma system that filters comment visibility they rely on a small barrier to entry (joining fee) to keep the more frivolous users at bay.
posted by Pranksome Quaine at 8:08 PM on March 23, 2010
Unfortunately I think any open online community that has established itself for insightful commentary and analysis devolves in quality over time as more mediocre thinkers increasingly become the majority. I think that a comment rating system only helps to delay the slide in quality and can't every really completely forestall it. I've certainly noticed the comment quality on Slashdot sliding over the last two or so years.
Metafilter and some other sites are interesting in that rather than using a full karma system that filters comment visibility they rely on a small barrier to entry (joining fee) to keep the more frivolous users at bay.
posted by Pranksome Quaine at 8:08 PM on March 23, 2010
This is excellent.
I didn't know about the authors nor the book.
I soon as I read "Karma is contextual and has limited utility globally", I was interested. Some (brilliant) people are calling for a kind or "universal reputation system" that makes me shudder. We have all seen people behave completely differently in different communities. Stating from the start the limits of such systems is great.
I also like "Public karma should be used sparingly" and "Negative public karma should be avoided all together." These guys know their stuff.
Strangely, I didn't find any "About the authors" on their own blog and did a little googling.
I am impressed.
Thanks cortex.
posted by bru at 8:17 PM on March 23, 2010
I didn't know about the authors nor the book.
I soon as I read "Karma is contextual and has limited utility globally", I was interested. Some (brilliant) people are calling for a kind or "universal reputation system" that makes me shudder. We have all seen people behave completely differently in different communities. Stating from the start the limits of such systems is great.
I also like "Public karma should be used sparingly" and "Negative public karma should be avoided all together." These guys know their stuff.
Strangely, I didn't find any "About the authors" on their own blog and did a little googling.
I am impressed.
Thanks cortex.
posted by bru at 8:17 PM on March 23, 2010
So I have some questions now:
1. Where did karma (both the concept and execution) originate? In my hazy memory I don't recall if plastic.com or slashdot got there first... or was it someone else that first implemented it.
2. I would say that metafilter favorites are somewhat opaque, although they could be more so; i.e. by separating a different user's favorite count by a few clicks through a profile or by setting public/private views of favorite counts. Admittedly I have not read through all of the infinite metatalk threads on favorites here, but had obscuring total favorites (not just on individual comments) to other users been considered in the great favorite rejiggering of 2009?
3.This could provoke yet another in the infinite series of favorite threads, have you considered that cortex?
–everything else I could say would go straight into that thread.
posted by Cold Lurkey at 8:17 PM on March 23, 2010
1. Where did karma (both the concept and execution) originate? In my hazy memory I don't recall if plastic.com or slashdot got there first... or was it someone else that first implemented it.
2. I would say that metafilter favorites are somewhat opaque, although they could be more so; i.e. by separating a different user's favorite count by a few clicks through a profile or by setting public/private views of favorite counts. Admittedly I have not read through all of the infinite metatalk threads on favorites here, but had obscuring total favorites (not just on individual comments) to other users been considered in the great favorite rejiggering of 2009?
3.This could provoke yet another in the infinite series of favorite threads, have you considered that cortex?
–everything else I could say would go straight into that thread.
posted by Cold Lurkey at 8:17 PM on March 23, 2010
Having made my quite unashamedly elitist comment above I want to somewhat play devil's advocate is giving an example of a site where intellectual quality of commentary is not so important and where comment rating more helps to shape the flow of a conversation. My example is reddit, which certainly began as quite high quality in both the level of submissions and commentary but quite quickly devolved. However I think the site is still useful just for people to talk through issues anonymously and chat in a playful way. In other words, I think reddit is mainly valuable to teenagers who are opening up to new ideas and to let them express themselves without being judged too harshly.
posted by Pranksome Quaine at 8:29 PM on March 23, 2010
posted by Pranksome Quaine at 8:29 PM on March 23, 2010
3.This could provoke yet another in the infinite series of favorite threads, have you considered that cortex?
It occurred to me, but I decided to cross my fingers and hope for the best. Heh.
posted by cortex at 8:56 PM on March 23, 2010
It occurred to me, but I decided to cross my fingers and hope for the best. Heh.
posted by cortex at 8:56 PM on March 23, 2010
Of all the blogs I read, I like slashdot's system the best. From what I can tell, this guys advice sounds similar (or maybe its just the use of 'karma')
Slashdot sucks. At least it did back in the late 90s. The people who ran it seemed to despise their own users. It was a weird dynamic. Maybe they've changed since then. I see they have "achievements" now. And I got the "member of the 5 digit UID club". Um, great.
posted by delmoi at 10:10 PM on March 23, 2010
Slashdot sucks. At least it did back in the late 90s. The people who ran it seemed to despise their own users. It was a weird dynamic. Maybe they've changed since then. I see they have "achievements" now. And I got the "member of the 5 digit UID club". Um, great.
posted by delmoi at 10:10 PM on March 23, 2010
I knew Randy Farmer a little bit years ago. He's a really good person. Where's the real-world favorite button? +1.
posted by prefpara at 6:47 PM on March 24, 2010
posted by prefpara at 6:47 PM on March 24, 2010
I knew Randy Farmer a little bit years ago. He's a really good person. Where's the real-world favorite button? +1.
You could just bake the guy some brownies.
posted by P.o.B. at 1:48 AM on April 1, 2010
You could just bake the guy some brownies.
posted by P.o.B. at 1:48 AM on April 1, 2010
« Older "50 jobs (and 500,000 watts) in 50 years" | Good-night, Crown Vic; And flights of angels sing... Newer »
This thread has been archived and is closed to new comments
Trying to transform this subconscious process into nice and neat sums comes from the same nerdy place that wants to find the formula for Getting Hot Chicks To Like You. With similar results. (In fact, StackOverflow For Dating Tips would be the perfect storm of rankings, karma, points, ratings and nobody scoring).
That said, I do take account of karma in one place: eBay, even though I already know it's quickly becoming useless.
posted by bonaldi at 12:02 PM on March 23, 2010 [2 favorites]