Comments on: Psychologist FTW

Psychologist FTW

AceRock — Sat, 01 Mar 2008 09:48:03 -0800

This Psychologist Might Outsmart the Math Brains Competing for the Netflix Prize

By: lonemantis

lonemantis — Sat, 01 Mar 2008 10:10:37 -0800

Great reading! Thanks so much for posting this. I was reading this out loud just now to some friends, and while I was thoroughly fascinated, the friends were looking at me like I was insane! I always cherish those moments.

By: flatluigi

flatluigi — Sat, 01 Mar 2008 10:24:41 -0800

Thanks For The Link, AceRock, It's Great Reading.

By: weapons-grade pandemonium

weapons-grade pandemonium — Sat, 01 Mar 2008 10:28:29 -0800

Yes, interesting.

By: escabeche

escabeche — Sat, 01 Mar 2008 11:11:29 -0800

I wrote this article! There's lots of great stuff that didn't make it into Wired for space reasons. Like Todd Holloway's beautiful visualizations of the Netflix dataset. Or Brandyn Webb, aka Simon Funk, explaining completely from scratch how to set up an SVD attack on the Netflix problem.

By: escabeche

escabeche — Sat, 01 Mar 2008 11:35:15 -0800

Oh, and here's "Just a Guy in a Garage"'s no-longer-anonymous blog. By the way, despite the title of the piece (in the print magazine it's the more sober "The Netflix Challenge") I hope the article doesn't give the impression that psychology beats math. Much closer is to say that psychology can serve as a complement to math when analyzing giant datasets arising from human behavior. Also, in an interesting development that took place after we went to press, the team from AT&T has been overtaken by "When Gravity and Dinosaurs Unite," a collaboration between a group of three recent Princeton grads and a team from the Budapest University of Technology and Economics.

By: vronsky

vronsky — Sat, 01 Mar 2008 11:38:18 -0800

Well done escabeche and ace rock. Interesting stuff!

By: TheOnlyCoolTim

TheOnlyCoolTim — Sat, 01 Mar 2008 11:47:12 -0800

This prize is very interesting, and despite the titling and hype so far Math is still beating Psychology. Netflix should use some of either, though, to figure out that thanks to their pop up ads I will never ever give them any money.

By: Justinian

Justinian — Sat, 01 Mar 2008 11:49:28 -0800

Looks like "When Gravity and Dinosaurs Unite" is a collaboration between team Gravity and team Dinosaur Planet. I wonder if we'll see any more of the top teams group up to eke out another percent.

By: billyfleetwood

billyfleetwood — Sat, 01 Mar 2008 12:06:42 -0800

Is there going to be some way to keep this out of the hands of the movie studios? I don't need complicated algorithms to predict what the studios will do with this technology. Norbit 3: The Vampire Slayer My theory is that the moviegoing public rates all movies on a simple scale with High School Musical on one end, and the Saw movies on the other. The perfect movie being a seamless combination of the two. Thank you Netflix, as soon as i convert this theory to code, I would like my million dollars in small non-consecutive bills.

By: growabrain

growabrain — Sat, 01 Mar 2008 12:24:19 -0800

I understand about Quebec, but why are residents of "Cuba, Iran, Syria, North Korea, Myanmar (formerly Burma) and Sudan" ineligible to participate in this contest?

By: PareidoliaticBoy

PareidoliaticBoy — Sat, 01 Mar 2008 12:26:56 -0800

Fantastic post. Very illuminating article, escabeche! It's fascinating stuff for a non-mathematician. Posts like this is what makes this a great place to visit. Two thumbs up .

By: leibniz

leibniz — Sat, 01 Mar 2008 12:32:10 -0800

Nice article. But this should be easy. All I need to do is give completely accurate assessments of the objective goodness of each movie. Then see how stupid people are compared to me. If for instance, you liked Mr and Mrs Smith, then I'll know that you won't like 2001AD (it's too long and boring yeah?).

By: davey_darling

davey_darling — Sat, 01 Mar 2008 12:32:13 -0800

I understand about Quebec, but why are residents of "Cuba, Iran, Syria, North Korea, Myanmar (formerly Burma) and Sudan" ineligible to participate in this contest? There are probably various embargoes in place that prevent American companies from sending money to those places.

By: Izner Myletze

Izner Myletze — Sat, 01 Mar 2008 12:38:50 -0800

I understand about Quebec, but why are residents of "Cuba, Iran, Syria, North Korea, Myanmar (formerly Burma) and Sudan" ineligible to participate in this contest? According to Netflix's Prize FAQ: Why are some countries excluded from participation? Most of those countries appear are on the U.S. Treasury Office of Foreign Assets Control's list of embargoed counties for which we cannot provide economic assistance. If this list changes, we'll post a change to the rules and let you know. Quebec has other reasons.

By: ooga_booga

ooga_booga — Sat, 01 Mar 2008 13:15:26 -0800

Good show, escabeche! It's not often that I see science/tech writing which doesn't shy away from giving the reader some credit while at the same time giving enough technical background so that interested parties have good starting points to do further research. Bravo! And, quite frankly, I was rather surprised to see this published by Wired, of all places.

By: jcruelty

jcruelty — Sat, 01 Mar 2008 13:34:49 -0800

Yeah that was good pop science, I enjoyed the article.

By: Mick

Mick — Sat, 01 Mar 2008 13:43:00 -0800

for instance, you liked Mr and Mrs Smith, then I'll know that you won't like 2001AD I like both; the first being 3.5 stars, the latter, a solid 5.

By: Justinian

Justinian — Sat, 01 Mar 2008 14:21:12 -0800

If for instance, you liked Mr and Mrs Smith, then I'll know that you won't like 2001AD When being snobby it's good to get the titles correct.

By: delmoi

delmoi — Sat, 01 Mar 2008 15:22:32 -0800

This Psychologist Might Outsmart the Math Brains Competing for the Netflix Prize If by "Psychologist" you mean "Amateur Mathematician who happens to have a psych degree."

By: tkolar

tkolar — Sat, 01 Mar 2008 15:22:34 -0800

This was very interesting. Oddly enough I had never even heard of the contest. Thanks for posting this AceBase, and thanks for writing the article escabeche.

By: leibniz

leibniz — Sat, 01 Mar 2008 15:45:01 -0800

Yawn. When being a pedant, it's good to first ascertain whether your target was presenting a serious argument or not. If pushed Mick, I suppose all I have to do is slightly adjust your insanity rating (they're supposed to get more accurate the more information you give right?) Not wishing to troll of course, but do you really want to start a discussion in which you have to defend the merits of Mr and Mrs Smith?

By: Mick

Mick — Sat, 01 Mar 2008 16:14:11 -0800

It met my needs (I was entertained).

By: painquale

painquale — Sat, 01 Mar 2008 17:30:34 -0800

escabeche, that was an excellent article. I was hoping that we'd get to see some more of the psychological insights that he's bringing to bear on the problem. The anchoring effect was mentioned, but nothing else. Just going down the list of cognitive biases at Wikipedia, I can see more than a few that might be relevant.

By: Dillonlikescookies

Dillonlikescookies — Sat, 01 Mar 2008 20:34:33 -0800

studying psychology was very interesting in that it taught me to examine the ways i go about solving various problems, in a sort of round about way. i'm not smarter for it, but here and there i've improved my own algorithms and found solutions laterally. the list of cognitive biases, for example, is gold for anyone who enjoys a good argument. great article, well written. good on ya :)

By: LobsterMitten

LobsterMitten — Sat, 01 Mar 2008 22:18:33 -0800

fun

By: Mental Wimp

Mental Wimp — Sat, 01 Mar 2008 22:41:31 -0800

As a statistician, I have two observations. First, is that the anchoring effect is pretty well known to statisticians working in the field of behavior. And the second is that I'm pretty sure by now that the algorithms are all overfit to the validation dataset, and therefore won't do as well in predicting new observations. That is all.

By: tkolar

tkolar — Sat, 01 Mar 2008 23:10:57 -0800

Taking a look through the data, I note that despite having 100,000,000 ratings from 500,000 users, the median film only has 546 ratings. My hat is off to anyone who can make predictions with data that sparse.

By: leibniz

leibniz — Sun, 02 Mar 2008 02:31:58 -0800

Ok Mick, you've forced me to criticise Mr and Mrs Smith. But first of all, I'm not even sure that 'it met my needs' would qualify for 3.5 stars. Eating a bag of crisps* meets your needs, but would you then describe it as 70% as good as the finest meal you've ever had?? Or a quick wank 70% as good as the best sex you've ever experienced? Anyway lets suppose we accept the pretensions of Mr and Mrs Smith as an uncomplicated mainstream hollywood film. And forget this was a action-romantic-comedy, with boring action, no romantic warmth, and no laughs, wooden dialogue, generic soundtrack, unrealistic sets etc. Also forget about the moral bankruptcy of the characters (why am I supposed to be sympathetic to these smiling murderers again?). And the utterly predictable plot starting with the ridiculous contrivances needed just to get these brain dead characters not to realise the other is an assassin - finishing with an utterly dull last scene of endless repetitive shooting with a laughable timing of 'them finally helping each other' for no particular reason, but signalled by a cue in the sound track and a new camera angle as they wipe out some more anonymous bad guys - a million loose ends left hanging at the tacked on happy ending. Suppose we ignore all that. Let's just think about so-called 'chemistry' between the leads. It had all the chemistry of an athletic porno fuck without the guts to make it sexually explicit. There was not a single moment of emotional warmth or tenderness between them. The only chemistry between these two actors was possibly that of the audience member ogling at these beautiful dolls. It had all the romantic charm of a bicep pumping body builder leering at you with his dead steriod glaze, forever. The success of this film was the success of a marketing bubble and the fame/beauty of the leads, blinding you to its manifest rubbishness. I guarantee that history will consign this film to oblivion. *chips in US right?

By: tkolar

tkolar — Sun, 02 Mar 2008 08:10:20 -0800

I wrote... Taking a look through the data, I note that despite having 100,000,000 ratings from 500,000 users, the median film only has 546 ratings. ... and the median user has 96 ratings to their name. Which, frankly, is a lot more than I expected.

By: philomathoholic

philomathoholic — Mon, 03 Mar 2008 00:21:08 -0800

How do you give half ratings on Netflix? I can't seem to do that.

By: tkolar

tkolar — Mon, 03 Mar 2008 07:24:01 -0800

How do you give half ratings on Netflix? I can't seem to do that. You can't. Your only choices are 1,2,3,4, and 5.

By: Mental Wimp

Mental Wimp — Mon, 03 Mar 2008 09:34:39 -0800

Besides inventing calculus, leibniz has beautifully deconstructed Mr. and Mrs. Smith. Bravo.