Netflix gives up $1M software prize

September 22, 2009 2:32 AM

The winners of the prize - for software 10% better at recommending movies than Netflix own Cinemax - were a team described here back in June. They beat another team by getting their results in 20 minutes earlier. Netflix was happy: “You look at the cumulative hours and you’re getting Ph.D.’s for a dollar an hour.” - so happy they're offering two new $half-million prizes.

No mention yet whether there's been any progress on the "Napolean Dynamite problem" ... the movies it's hard to predict your reaction to.

When Bertoni showed me a list of his 25 most-difficult-to-predict movies, I noticed they were all similar in some way to “Napoleon Dynamite” — culturally or politically polarizing and hard to classify, including “I Heart Huckabees,” “Lost in Translation,” “Fahrenheit 9/11,” “The Life Aquatic With Steve Zissou,” “Kill Bill: Volume 1” and “Sideways.”

posted by Twang (97 comments total)

Oddly enough, the movie about this struggle only gets a rating of 2.1 from Netflix users and the DVD itself is a "Very Long Wait".
posted by twoleftfeet at 2:56 AM on September 22, 2009

Pretty crazy that in the end the software was tied. Seems a little unfair that one of these teams worked for such a long time and doesn't get any money. But The Ensemble was so large anyway, how much money would have gone to each member?
posted by delmoi at 3:01 AM on September 22, 2009

When Bertoni showed me a list of his 25 most-difficult-to-predict movies, I noticed they were all similar in some way to “Napoleon Dynamite” — culturally or politically polarizing and hard to classify, including “I Heart Huckabees,” “Lost in Translation,” “Fahrenheit 9/11,” “The Life Aquatic With Steve Zissou,” “Kill Bill: Volume 1” and “Sideways.”

It seems to me you could pretty much group these "25 hardest to predict movies" together into a category and recommend them based on each other.
posted by Jimbob at 3:05 AM on September 22, 2009 [3 favorites]

Okay, I count 26 people and one company as members of the The Ensemble, so they would have gotten 37,000 each. But apparently they were setup in a corporate structure with shareholders, and one person apparently had 16%, so he lost $160k.
posted by delmoi at 3:06 AM on September 22, 2009

It seems to me you could pretty much group these "25 hardest to predict movies" together into a category and recommend them based on each other.

Huh why would that work? People who love Kill Bill: Volume 1 are not necessarily going to like Fahrenheight 9/11. The whole problem is that they're not correlated to any others, including eachother.
posted by delmoi at 3:08 AM on September 22, 2009 [1 favorite]

Jimbob: "It seems to me you could pretty much group these "25 hardest to predict movies" together into a category and recommend them based on each other."

And to think, that could have been your million dollars!
posted by Roman Graves at 3:27 AM on September 22, 2009 [8 favorites]

It seems to me you could pretty much group these "25 hardest to predict movies" together into a category and recommend them based on each other.

Well I saw Kill Bill I twice in the cinema and own in on DVD... I hated Lost In Translation. Loved Sideways but could not finish watching I Heart Huckabees etc. So no, not really.
posted by fearfulsymmetry at 3:32 AM on September 22, 2009

there are some interesting privacy questions raised by this second netflix prize.

Netflix should cancel this new, irresponsible contest, which it has dubbed Netflix Prize 2. Researchers have known for more than a decade that gender plus ZIP code plus birthdate uniquely identifies a significant percentage of Americans (87% according to Latanya Sweeney's famous study.) True, Netflix plans to release age not birthdate, but simple arithmetic shows that for many people in the country, gender plus ZIP code plus age will narrow their private movie preferences down to at most a few hundred people. Netflix needs to understand the concept of "information entropy": even if it is not revealing information tied to a single person, it is revealing information tied to so few that we should consider this a privacy breach.

I have no doubt that researchers will be able to use the techniques of Narayanan and Shmatikov, together with databases revealing sex, zip code, and age, to tie many people directly to these supposedly anonymized new records.

posted by russm at 3:39 AM on September 22, 2009 [2 favorites]

Twang: Netflix own Cinemax

Okay, that gave me a bit of a laugh (after I parsed the grammar and figured out what was going on.)

from 'Netflix was happy' link: Netflix, Mr. Hastings said, did not do a crisp cost-benefit analysis of its investment in the contest. But several crucial techniques garnered from the contest have been folded into the company’s in-house movie recommendation software, Cinematch, and customer retention rates have improved slightly. Better recommendations, Netflix says, enhance customer satisfaction.

Ah yes. This is a difficult science, but I have no doubt that CineMatch and other taste-matching systems like LastFM will improve dramatically over the next few years, as our context-scanning technology advances and as human beings become simpler and stupider.
posted by koeselitz at 4:00 AM on September 22, 2009

I like each of the difficult to predict movies. Hey Netflix, I'll tell you what other movies I like for only 250k. What, I already did tell you? Damn!
posted by diogenes at 4:19 AM on September 22, 2009 [1 favorite]

Surprised there's no mention of MovieLens, which has worked extremely well, for me, for many years.
posted by RichardS at 4:59 AM on September 22, 2009 [2 favorites]

The Kill Bill movies are among my all-time favorites. I didn't like Sideways (the acting by all four principals was top-notch, I'll grant you, but I don't enjoy watching an alleged adult act like a whiny six-year-old for two hours, even if it's excellently acted) or Napoleon Dynamite at all.

What I'd really like, though, is something like this for news articles. Possibly correlating my past results with new articles based both on the votes of other users as well as key words and phrases in the article. A week or two of voting thumbs-up or thumbs-down on articles and then, for the rest of my life, I could remain blissfully unaware of the manufactured antics of any rappers at manufactured awards shows. And I'd see several articles about the substance of a major presidential speech and none about childish outbursts by congresspersons during the speech. And as long as I'm dreaming, I'd like a pony.
posted by DevilsAdvocate at 5:35 AM on September 22, 2009

gender plus ZIP code plus age will narrow their private movie preferences down to at most a few hundred people

Personally I think there is a huge difference privacy-wise between "uniquely identitifiable" and "within a group of 500".
posted by smackfu at 5:37 AM on September 22, 2009 [3 favorites]

“You look at the cumulative hours and you’re getting Ph.D.’s for a dollar an hour.”

A precedent which, I'm sure, has not gone unnoticed in certain quarters.
posted by Thorzdad at 5:59 AM on September 22, 2009 [3 favorites]

Such machine-based solutions to the question "what movie should I watch next" are only useful if you see watching movies as a simple practical necessity like eating or buying gas for your car. They're the kinds of things that have always and will always recommend movies like Koyanisqatsi (which I can't stand) just because I say I like Sans Soleil. So what if a whole bunch of people who've liked the movies I like enjoy that movie?

I can't see this encouraging anything but intellectual decay and insularity. People should be watching movies that people like them hated. It would open their minds more. That's worked quite nicely for me so far. Movies are an opportunity to enliven the mind, to lay a groundwork for the understanding that is higher than what it might otherwise have reached. Even on the most basic level of simple entertainment, I think that means we should be actively seeking out movies that are nothing like what we thought we'd like. Machines can help us do a lot of things, but that isn't one of them.

In fact, if anything this is evidence of the way that television has really affected film in the last two decades. I just watched a beautiful little movie that Wim Wenders made during the mid-eighties; he's probably my favorite director as of this moment, if only because of his quiet, unassuming film essays like Tokyo-Ga, a gorgeous little meditation which deserves far more than to be the disc-two bonus on my copy of Late Spring. Anyhow, the thing I saw was Room 666: it's nothing but an introduction and an epilogue which bookend a set of solo "interviews" with a whole host of directors, from Jean-Luc Godard to Steven Spielberg (!) wherein the interviewees are simply given a piece of paper with the question "is film in danger of being destroyed by television?" and left alone with the camera running. Great stuff, and it's clear what a controversy it was even then; there were still people who said then that film was the only artistic medium, of course, and that no true artist would work on magnetic tape, but most have more nuanced thoughts about the whole thing.

Thinking about it almost thirty years later, I think television has changed film immensely - but not for the reasons that seemed obvious then. Werner Herzog catches it best, I think, in his interviews in the movie: he says that movies are different because you can't pause them, because you can't get up and turn them off - and because they're so big, so scary. In fact, I think part of what's scary about them is the fact that they're partially a social experience; we have to parse or at least live with the fact that other people are experiencing this big loud thing too, that other people are watching it and reacting to it at the same time right next to us. We have to watch as a society, and that takes some effort, some connection, some real relation.

It's changed in ways they could never have forseen. But at the very least I know that almost every single one of them, even Steven, might have felt at least a little insulted to think that people would reduce the experience of watching movies to an algorithm. But this is only because now we consume movies as we would TV dinners; we have to keep consuming, so we need a machine to tell us what to consume.
posted by koeselitz at 5:59 AM on September 22, 2009 [4 favorites]

"Difficult to predict a viewer's reaction" should be considered a feature, not a bug. Sorry, marketers. (I never trust web sites' recommendations anyhow.)
posted by aught at 5:59 AM on September 22, 2009 [1 favorite]

aught: "Difficult to predict a viewer's reaction" should be considered a feature, not a bug. Sorry, marketers. (I never trust web sites' recommendations anyhow.)

More to the point: the most interesting people that I've known are people whose tastes are impossible to predict.
posted by koeselitz at 6:02 AM on September 22, 2009 [5 favorites]

But at the very least I know that almost every single one of them, even Steven, might have felt at least a little insulted to think that people would reduce the experience of watching movies to an algorithm. But this is only because now we consume movies as we would TV dinners; we have to keep consuming, so we need a machine to tell us what to consume.

But people don't limit themselves purely to what a computer suggests to them on a computer screen when it comes to selecting movies to watch. Is a mathematical equation that much different than the whims of a movie critic?
posted by Atreides at 6:07 AM on September 22, 2009

I'll admit I didn't have time to read the whole article, so I don't know if this came up, but I bet they could do a lot better with a little more information than they get from stars. Stuff like "would you watch this movie again? Would you recommend it to a friend?"
posted by carmen at 6:10 AM on September 22, 2009

Atreides: Is a mathematical equation that much different than the whims of a movie critic?

Only slightly.

Notice I didn't mention movie critics.
posted by koeselitz at 6:15 AM on September 22, 2009

To clarify: yes, there are similar limitations whenever you cut out all interaction beyond a one-dimentional media interface. You don't talk to a movie critic's column; you don't talk to an algorithm, either.

You're right that people certainly use this in larger contexts; but I don't see how that can be useful, really. I've tried that, too, but inevitably I find that these things recommend movies that are so obvious on reflection that if I heard them from a guy at a party, I'd laugh nervously and move on to the next group as I'd know I'd gotten a really simple, stupid person. In other words, the very best way to learn about what movies are good is to talk to other people about what movies they love. There is really no other way.

And honestly: at this point this stuff really isn't anything but a marketing tool designed to get more people to sign up for Netflix.
posted by koeselitz at 6:20 AM on September 22, 2009

I can't see this encouraging anything but intellectual decay and insularity. People should be watching movies that people like them hated. It would open their minds more.

The traditional way of finding movies is to watch trailers and then see what looks good. How is this any worse than that?

It sounds more like you have a problem with which movies are popular now (and always have been) and are just blaming whatever you can for that. Using phrases like "people should be watching" is bad road to walk.
posted by smackfu at 6:20 AM on September 22, 2009 [3 favorites]

In other words, the very best way to learn about what movies are good is to talk to other people about what movies they love.

But the point of this whole thing is that it's not just "other people". It's a limited group of people who are "like you". Even in your post, you give an example of someone you wouldn't trust the opinions of. So if Netflix can automatically exclude that guy's opinions in their recommendations, wouldn't that give a better answer for you?
posted by smackfu at 6:25 AM on September 22, 2009

It seems to me you could pretty much group these "25 hardest to predict movies" together into a category and recommend them based on each other.

How does that make sense?

I loved Sideways; I hated I Heart Huckabees, etc.
posted by Jaltcoh at 6:28 AM on September 22, 2009

There is no answer.
posted by koeselitz at 6:29 AM on September 22, 2009

smackfu: So if Netflix can automatically exclude that guy's opinions in their recommendations, wouldn't that give a better answer for you?

For there to be an answer, there has to be a question.
posted by koeselitz at 6:29 AM on September 22, 2009

I find I'm often so cynical that even if I "Really Liked" a movie, and I see everyone else has given it 5 stars, I'll mark it 3 stars in some sort of "Oh come on people, it wasn't THAT good" kind of thing.

Also, if I "Love" a movie, but doubt I'll like similar movies, I'll rate it lower, fearing it'll make suggestions I won't like.

Take THAT prediction engine!
posted by toekneebullard at 6:32 AM on September 22, 2009 [2 favorites]

I don't do spec work
posted by Mick at 6:37 AM on September 22, 2009 [2 favorites]

For there to be an answer, there has to be a question.

If a tree claps in a movie theater, does it make a sound?
posted by smackfu at 6:42 AM on September 22, 2009 [1 favorite]

Even on the most basic level of simple entertainment, I think that means we should be actively seeking out movies that are nothing like what we thought we'd like.

You do know there are three Mighty Ducks movies, koeselitz? :)
posted by Jody Tresidder at 6:44 AM on September 22, 2009 [6 favorites]

And four sequels to Bring it On. I bet you wouldn't like those either so you better watch them. Cheerleaders, yay!
posted by smackfu at 6:46 AM on September 22, 2009 [5 favorites]

How are they seriously going to group a political documentary with a modern slasher action flick? Wouldn't one go with documentaries and the other go with action or would that be too simple? I could see them saying Hey if you like Kill Bill then you might like Last Samurai because they both have swords but that should be at the very end of the extreme.

Maybe they should just group the movies up with the director, cast members, category, things that are related to the movie you just watched, etc.
posted by Mastercheddaar at 6:46 AM on September 22, 2009

koeselitz: I think that means we should be actively seeking out movies that are nothing like what we thought we'd like

If you liked: El Topo
You might also like: Air Bud: Golden Receiver
posted by shakespeherian at 6:54 AM on September 22, 2009 [15 favorites]

Actually, why couldn't Jimbob's idea of group these "25 hardest to predict movies" together into a category and recommend them based on each other work? (I know it doesn't because no one uses this approach, I just don't understand why.)

When a user has watched at least a certain number of movies in this exotic group, find others users in the population that have watched a similar amount and set from this group and made recommendations based on those who have largely agreeing ratings?

E.g. Alice has watched A, B, C, D, E, F and only rated A, B, C highly. Bob has watched B, C, D, E, Y, Z and only rated B, C, Z highly. Based on Bob, Alice is recommended Z but not Y. And Bob is recommended A but not F. Meanwhile, Carl who has watched A to J is not made recommendations based on Alice or Bob because he's watched that much more movies from the exotic group.
posted by tksh at 6:57 AM on September 22, 2009

Kill Bill is not hard to classify. The reason it is hard to predict is because it has all the elements of a movie that someone who liked Pulp Fiction would like, except it sucks ass. Tarantino, Uma Thurman, non-linear plot, check, check, check, etc. Does anyone care what happens? No.

The problem is there is no movie. It's like watching a video game. Fight fight fight cutscene miniboss cutscene fight fight fight cutscene miniboss cutscene fight fight fight cutscene BOSS LEVEL. The film is modeled after Karateka for the Apple ][c.
posted by Pastabagel at 6:57 AM on September 22, 2009 [15 favorites]

Even on the most basic level of simple entertainment, I think that means we should be actively seeking out movies that are nothing like what we thought we'd like.

Interestingly enough, it would be very easy to have a recommendation engine do just that.

You don't talk to a movie critic's column; you don't talk to an algorithm, either.

That's true of the movie critic but not of the algorithms that Netflix uses. If Netflix recommends something, I can tell it what I thought of the movie. It's an iterative process. Of course, I think what you really meant is that you can't have a conversation with Netflix in the same way two people might have a conversation.

So ultimately you're suggesting that conversations about movies are a better way to find new things to watch than a recommendation engine. This is a testable proposition; perhaps Netflix should commission a study ("Research shows Netflix better than word of mouth"). But you're also suggesting that computers will never be capable of the hard-AI task of having a meaningful conversation about movies. I disagree, though it may be many decades out.
posted by jedicus at 6:57 AM on September 22, 2009 [1 favorite]

Damn koeselitz: you clearly enjoy movies on a level that none of the rest of us will ever experience, and you are an unto a superman when compared to us.
posted by TypographicalError at 6:59 AM on September 22, 2009

I think I read somewhere that Magnolia and The Sixth Sense were the two most divisive movies on Netflix or IMDB. (personally, I hated both)

Kinda hated Napoleon Dynamite too, but only because it bothers me when movies and TV shows feature a "nerd" character who isn't actually smart. Like it does nerds everywhere an injustice or something.
posted by Afroblanco at 6:59 AM on September 22, 2009

Well, if people want examples: I have to say that Pootie Tang was sincerely one of the funniest movies I've ever seen. And I don't think anyone will think me a pretentious person for going into it thinking it was going to be the worst thing I'd ever seen.
posted by koeselitz at 7:01 AM on September 22, 2009 [1 favorite]

TypographicalError: Damn koeselitz: you clearly enjoy movies on a level that none of the rest of us will ever experience, and you are an unto a superman when compared to us.

You probably experience them better than I do. So don't cheat yourself.
posted by koeselitz at 7:03 AM on September 22, 2009

Afroblanco, do you just watch Warhol's Empire over and over again, or what?
posted by shakespeherian at 7:03 AM on September 22, 2009

I swear, the fear and hatred of pretentiousness is probably going to destroy us one of these days. Or at least make us really unhappy.
posted by koeselitz at 7:04 AM on September 22, 2009 [1 favorite]

shakespeherian: Afroblanco, do you just watch Warhol's Empire over and over again, or what?

'OOhh! Look at me! I'm not a snob at all! Look at how easily I make fun of the silly snobs - that way everybody likes me!'
posted by koeselitz at 7:05 AM on September 22, 2009

The problem is there is no movie. It's like watching a video game. Fight fight fight cutscene miniboss cutscene fight fight fight cutscene miniboss cutscene fight fight fight cutscene BOSS LEVEL. The film is modeled after Karateka for the Apple ][c.

And all the cutscenes are stolen from other movies/commercials/pop songs/tv shows/anime
posted by cyphill at 7:09 AM on September 22, 2009

It's like watching a video game. Fight fight fight cutscene miniboss cutscene fight fight fight cutscene miniboss cutscene fight fight fight cutscene BOSS LEVEL.


That's exactly why it's an awesome movie.
posted by oinopaponton at 7:10 AM on September 22, 2009 [4 favorites]

I liked Sideways; Kill Bill not so much. I prefer watching an alleged adult act like a whiny six-year-old for two hours to watching four hours of chop-socky pastiche even if it does star Uma Thurman.

now we consume movies as we would TV dinners; we have to keep consuming, so we need a machine to tell us what to consume.

I've seen this one. In the last ten minutes, Kirk tricks the machine into destroying itself and before he leaves the entire planet is like "OH NOES! Now we must decide what to watch for ourselves!!"
posted by octobersurprise at 7:12 AM on September 22, 2009 [4 favorites]

Maybe it's because I love math with all my little black heart, but I really think this whole thing is pretty revolutionary and great.

I mean, the more movies you rate and rate accurately to your own tastes, the more likely the system is to throw back at you something that you will love but you never would've imagined you had loved. I mean, let's say I rate hundreds of movies and watch hundreds based on Netflix's recommendations. Then one day, the engine comes along and says, "Maybe you'll love White Chicks." I say to myself, no one with any sense would love White Chicks! But then if I can just trust the mathematics and statistics behind everything, and watch the movie, I may find it turns into a classic for me.

And at the end of the day, yes, I would like a challenging movie. But I want to watch challenging movies that I find interesting and enjoyable. And who is to say that Netflix isn't capable of leading me to such things, if it is what I express a preference for in my ratings?
posted by TypographicalError at 7:16 AM on September 22, 2009 [1 favorite]

koeselitz: 'OOhh! Look at me! I'm not a snob at all! Look at how easily I make fun of the silly snobs - that way everybody likes me!'

Um, okay. I was really just ribbing the guy, not trying to create some sort of us-vs-them thing. Personally I much more enjoy threads wherein people talk about the things they love rather than the things they hate. Invariably, though, the former becomes the latter.
posted by shakespeherian at 7:16 AM on September 22, 2009

In other words, the very best way to learn about what movies are good is to talk to other people about what movies they love. There is really no other way.

That's why I think Criticker works better than most other recommendation engines. You rank a lot of films you've seen, it normalizes those rankings, and then compares your normalized rankings with other users to find the users whose rankings tend to be the closest to your's. It then calculates estimated scores for films you haven't seen based on the rankings of users that are similar to you.

But the real key is that it's more nuanced than that. For one thing, along with your rankings you can also write a short review of the film explaining your rating. And when you view the page for a given film, it shows the rankings and reviews that other users have given it, sorted by how similar those users are to you. You can also easily view any given user's full list of rankings, with the ones you've already ranked highlighted, so that you can quickly check what films they highly recommend that you haven't seen.

A lot of the films that are highly recommended are ones that are pretty obvious, mostly films that are widely considered to be classics but that I haven't seen yet. But it has also been very useful in finding more obscure films and giving me an idea of what makes them worth watching.
posted by burnmp3s at 7:24 AM on September 22, 2009 [1 favorite]

I've seen several businesses, large multi-nationals all, recently musing about prizes as mechanism to encourage innovation. This is motivated by corporate managers noticing that R&D costs a lot of money and doesn't always pay out. Perhaps they can encourage someone else to take that risk? If you don't think about it too hard, perhaps that sounds ok. There are very few researchers who can actually work this way, however.

My main problem, as a researcher, is start up money. Who is going to provide me with the money to buy equipment and supplies, to hire the techs and pay for my own time to do the work? I can't get this on easy credit---returns are uncertain, remember, even if they do cover the costs. I need a sugardaddy to provide start-up capital, either as government grants or as a direct investment.

Government grants often come with strings on IP, unless they're explicitly for business development. Direct investment, going public or finding angle funding, for example, means giving up control. Straight loans are hard to arrange, because you have no capital or assets.

Even forgetting the start-up costs for the moment, however, most researchers are academic or governmental, they can't carry profits on account either. Public researchers can't win one prize then use this to fund the next. If money isn't spent by a certain date, even if it was obtained from non-governmental sources, it gets returned annually to the general government coffers. A prize-based researcher would have to be private, which means giving up access to a lot of existing infrastructure.

So, to base research on prize winnings, I need to be private and I need to be largely self-funded. That means a small company or an investor-funded start-up; remember these prizes are given by large companies so that they don't have to have an in-house research arm any more. But really, then the question is, if I can produce something based on a start-up model, why do I need the prize anyway? If I retain IP on my product, instead of assigning it to the prize-giver, couldn't I make more money that way anyway? There's got to be more than one company who wants a good product recommendation engine. Might Amazon be interested, perhaps?

So, prizes aren't much use for academic or governmental researchers, corporate doesn't want to do it anyway, and start-ups cover high-risk, high-payout innovation. What problem do prizes fix? Why take the risk of a challenge for the sub-optimal payout of a prize?
posted by bonehead at 7:27 AM on September 22, 2009

How are they seriously going to group a political documentary with a modern slasher action flick?

This is addressed in the last link of the FPP. The "single value decomposition" approach used by the top teams doesn't predefine any categories, whether it's "documentaries" or "Tom Hanks" movies. They just feed the software the data and the computer comes up with categories on its own based on the ratings. Some of these categories are obvious; some are more subtle; and some are completely inexplicable, but they work anyway.

One thing also addressed in that article is the issue of people's changing tastes over time. Netflix's own recommendations don't take this into account, but some of the competitors' algorithms do by weighting recent recommendations more heavily than older ones. I've enjoyed Battlestar Galactica and 24 and Lost and House, but lately I find myself getting burned out on dark, gritty dramas and find myself, much to my own surprise, seeking out shows and movies that are decidedly more towards the idealistic end of the idealism vs. cynicism scale. The late lamented Pushing Daisies was my favorite show of the past two years. And I find that I've been getting into King of the Hill since it started airing in reruns on [adult swim]. Had you told me five years ago some of my favorite shows would be those where nearly every episode had a heartwarming ending, and what's more, that I would like them largely because of (not in spite of) those heartwarming endings, I would have laughed in your face. But there it is.
posted by DevilsAdvocate at 7:31 AM on September 22, 2009

I think I read somewhere that Magnolia and The Sixth Sense were the two most divisive movies on Netflix or IMDB.

"Most divisive" doesn't necessarily mean "most unpredictable," though.
posted by DevilsAdvocate at 7:34 AM on September 22, 2009 [1 favorite]

Even on the most basic level of simple entertainment, I think that means we should be AT LEAST OPEN TO seeking out movies that are nothing like what we thought we'd like.

ftfy

... and on the Tarantino tip, I couldn't agree with Pastabagel more on the Kill Bill awfulness ... Fight fight fight cutscene miniboss cutscene fight fight fight cutscene miniboss cutscene fight fight fight cutscene BOSS LEVEL ... but but am darned glad I didn't let this put me off laying down a few bucks and 2.5 hours of my life for Inglorious Basterds.
posted by philip-random at 7:38 AM on September 22, 2009 [1 favorite]

When Bertoni showed me a list of his 25 most-difficult-to-predict movies, I noticed they were all similar in some way to “Napoleon Dynamite” — culturally or politically polarizing and hard to classify, including “I Heart Huckabees,” “Lost in Translation,” “Fahrenheit 9/11,” “The Life Aquatic With Steve Zissou,” “Kill Bill: Volume 1” and “Sideways.”

I'm with Jimbob: I liked all of these, to varying degrees. So you might as well call it "Uncategorizable" and sign me up to that category.
posted by rokusan at 7:39 AM on September 22, 2009 [1 favorite]

“Napoleon Dynamite” — “I Heart Huckabees,” “Lost in Translation,” “Fahrenheit 9/11,” “The Life Aquatic With Steve Zissou,” “Kill Bill: Volume 1” and “Sideways.”

I wish I could say I did liked all of these films, even slightly. It would make commentary easier. But, as already mentioned, Kill Bill really did leave me cold (and bored, and annoyed). And, based on this thread and other discussions I've had, I've heard much the same sentiment put forth for all of these titles.

Man, I love culture. Just so darned ... complex, confusing, convoluted (other things that start with "c").
posted by philip-random at 7:52 AM on September 22, 2009

So, prizes aren't much use for academic or governmental researchers, corporate doesn't want to do it anyway, and start-ups cover high-risk, high-payout innovation. What problem do prizes fix? Why take the risk of a challenge for the sub-optimal payout of a prize?

I would guess that it's a trade between higher profits and more predictable profits. If you know that Company A is going to pay $X for some achievement, it's easier to put a business plan together to justify the funding than to estimate that some set of companies might pay some amount of money to license or buy it. So the two main factors would be the amount of the prize versus the expected amount of profit from selling it, and the amount of uncertainty in the market for the finished product. It seems like prizes would work especially well in situations where there is no current market, because estimating how much a technological innovation would be worth in those situations would be very difficult, so it makes sense that prizes like the X Prize for space flight or the DARPA Grand Challenge for autonomous vehicles would draw a lot of competitors.
posted by burnmp3s at 7:53 AM on September 22, 2009

I love this software that helps pick similar movies that I might like. I would be lost without it.

One of my favorite movies is Friday the 13th Part VI: Jason Lives. When I checked to see what similar movies it recommended, it chose Friday the 13th, and then also a little less predictably chose Friday the 13th Part III. But imagine my surprise when it also chose Friday the 13th Part VIII: Jason Takes Manhattan. I mean, how could it even know that? It was the perfect pick. I would never have thought of that on my own. I really enjoyed that movie a great deal, and I have relied almost exclusively on those recommendations since then.
posted by flarbuse at 8:15 AM on September 22, 2009 [4 favorites]

Yeah, I think that recommendation systems kick ass, and anyone putting forward a "robots in the future picking our entertainment for us" plot needs to lay off the Jetsons or is really just trying to be provocative.

I mean, the recommendation engines on Amazon and Netflix really just serve to remind me of things that I've heard about or had recommended to me, but kept forgetting about. "Oh yeah, so-and-so was talking about this book, I really should check that out." or "Yeah, I've been meaning to see this movie FOREVER. Totally putting that in my queue." More of a memory device than anything else.

Services like Pandora that actually serve up content are a little different. Those really are robots in the future picking our entertainment for us. But they're so good at it! Wonderful little robots.......
posted by Afroblanco at 8:42 AM on September 22, 2009

I sort of like this gentleman's solution to the age-old dilemma of what to put in your Netflix queue. (The answer: Everyone else's queue)
posted by porn in the woods at 8:42 AM on September 22, 2009

The only film out of the list above I've seen is Napoleon Dynamite, which on paper, I should have loved, but in practise was just a bit nasty - nerds seen through the lens of the nerd-baiters. Also, despite the clever use of colour, it was bloody dull.
posted by mippy at 8:44 AM on September 22, 2009

You might also like: Air Bud: Golden Receiver

Three films that have made me cry: The Ice Storm, Annie Hall, and Air Bud. This is not a lie.
posted by mippy at 8:46 AM on September 22, 2009

People should be watching movies that people like them hated.

So Rollerball, then?
posted by Durn Bronzefist at 8:47 AM on September 22, 2009

All I know is the software told me I would like Dziga Vertov's Man With a Movie Camera, which is some kind of pioneering experimental Soviet-realist thing. The software thinks I'm cool. I will hear nothing against my buddy the software.
posted by ormondsacker at 8:48 AM on September 22, 2009 [6 favorites]

The other day I added "Mister Lonely" to my queue. They suggested that one of the films most like it was "Bring It On: All or Nothing".

So there's a little ways to go before it turns into Skynet.
posted by 235w103 at 8:58 AM on September 22, 2009

Man With A Movie Camera is even cooler with the Cinematic Orchestra score.
posted by box at 9:01 AM on September 22, 2009

235w103: The other day I added "Mister Lonely" to my queue. They suggested that one of the films most like it was "Bring It On: All or Nothing".

Now I would like to see a Harmony Korine high school sports rivalry film.
posted by shakespeherian at 9:06 AM on September 22, 2009 [2 favorites]

Race for the prize?
posted by Antidisestablishmentarianist at 9:07 AM on September 22, 2009

Did not like Napoleon Dynamite, but love the fact that there are movies that drive automatic raters bonkers.
posted by edgeways at 9:29 AM on September 22, 2009

The problem is there is no movie. It's like watching a video game. Fight fight fight cutscene miniboss cutscene fight fight fight cutscene miniboss cutscene fight fight fight cutscene BOSS LEVEL. The film is modeled after Karateka for the Apple ][c.

Kill Bill is an homage to the kung fu genre. That's what it's supposed to be.
posted by krinklyfig at 9:37 AM on September 22, 2009

Well, if people want examples: I have to say that Pootie Tang was sincerely one of the funniest movies I've ever seen. And I don't think anyone will think me a pretentious person for going into it thinking it was going to be the worst thing I'd ever seen.

I think that's part of its brilliance, and I'd recommend anyone see it for the first time without knowing anything about it going in.
posted by krinklyfig at 9:39 AM on September 22, 2009

Incidentally, I was thinking about it and came up with an analogy for why the "Napoleon Dynamite problem" exists.

Imagine the following process: an algorithm asks people a series of yes/no questions, and based on those questions, attempts to determine that person's place of residence inside the US. This isn't so crazy: if you are asked, "Do you speak Spanish on a daily basis?" and answer yes, you are more likely than not to live in the southwest.

In any case, after a series of 50-100 questions, the algorithm will (likely) be able to determine a person's location fairly well. Now comes the fun part: using your predicted location, the algorithm attempts to predict your answers to future questions. So if it had predicted that you lived on a coast, it would also predict that you eat seafood on a regular basis, for example.

The problem comes when you ask: do you own a dog? Dog ownership is in no way connected with geographic location, and so the predictive power of the algorithm is completely useless. The only thing that you can do to save it is create a new dimension to the location information called "dog ownership" and now mark every place on the map with a number representing the percentage of dog owners at that location. However (and here's the really unfortunate bit) the only way to predict if someone is a dog owner is to ask them if they are a dog owner, as dog ownership is not really connected with most any other information about people - not wealth, or profession, or location, or any of that stuff.

In Netflix's algorithm, people are located not on a map of the US, but in a 7- or 8- (or 240-)dimensional space. (The specific dimensions are not named, but are generally believed to correspond to personality traits or collections of personality traits.) Ratings are used to locate you inside that space, and movies can be viewed as regions in that space where everyone in the region is likely to enjoy the movie. The problem is that liking Napoleon Dynamite is orthogonal (in a mathematical sense as well as the normal sense!) to all the other information you have at your disposal, much like owning a dog is not predicted by your geographical location. Further, there's no reason each of these movies can't be orthogonal to each other. That is, the only way to know if someone likes one of them is to simply ask them about that one specific movie, which defeats the purpose of the rating system in that case.
posted by TypographicalError at 9:47 AM on September 22, 2009 [4 favorites]

I'm curious...what would be the effect of taking those top 25 movies and excluding them from the ranking algorithm? Sort of of throwing out the most troublesome data? I know it wouldn't solve the issue, but...I wonder if the end result, in this case, might not be a MORE accurate suggestion engine?

Of course...the top 25 most troublesome movie might change, but...just curious.
posted by Richat at 9:56 AM on September 22, 2009

Kill Bill is an homage to the kung fu genre. That's what it's supposed to be.

3.5 hour homage that, along the long, long way, pretends to be much, much more (ie: art) ... and ends up being far less than what it's "honoring"; hence the hate, at least from me.
posted by philip-random at 10:15 AM on September 22, 2009

Maybe it'd really work the best if the algorithm would recognize if a film was polarizing, and then it would let the user know it's kind of a love-it-or-hate it thing. Perhaps even linking reviews from relevant published critics would help, so that the viewer gets a brief synopsis and critique to know what they're getting into.

Maybe it could have two suggestion windows, one being "I bet you'll really like this" and the other being "Why not try something a little different?" That way, it has a good out if it thinks a polarizing movie would be a good idea even when its track record is bad for the film.
posted by mccarty.tim at 10:22 AM on September 22, 2009

Burnmp3s, the way you describe Cinetracker makes it sound like exactly the same thing as the Netflix recommendation engine. I mean, you can write a review of a movie after ranking it on Netflix, and look at other people's reviews on the film page, sorted by how similar their preferences are to your own.

Kill Bill was an homage to kung fu movies, and it was art. Loved it. Napolean Dynamite was all about laughing at the strange kid. Pure meanness.
posted by Thoughtcrime at 10:25 AM on September 22, 2009

The predictor I always found most useful was CinemaScore. It's exit polling for movies, so you can see whether the people who thought they would like the movie enough to watch it actually liked it. Not that it would predict movies to see, but you could watch all the trailers, and then moderate those feelings with reality. They used to send out the data for free via email every weekend, but I think now they are back to just selling it.
posted by smackfu at 10:33 AM on September 22, 2009

I loved "Napoleon Dynamite" in part because I grew up in southern Idaho and northern Utah. Those are my people! How the hell could Netflix figure that out based only on my simplistic ratings of other movies?

I think the limits of a persons taste in films are beyond the bounds of the information set.

I can't help thinking that Netflix ought to reframe the problem. Couldn't they embrace the unknowable and identify the truly challenging films in their catalog as such? Polarizing films can be the most interesting.

In the past few years, a certain flavor of critical hostility has been a tag for me to see the film. Both Tideland and Southland Tales fit that category and I enjoyed them both immensely. Would I recommend them to others? Nope. Netflix needs a new category: "Films you'll either love or despise".
posted by Carmody'sPrize at 10:35 AM on September 22, 2009

How the hell could Netflix figure that out based only on my simplistic ratings of other movies?

That is kind of the point of this new challenge. You extra stuff like ZIP code, age, and gender.
posted by smackfu at 10:38 AM on September 22, 2009

I love the list of 25 movies most likely to turn a small happy party into a group of people who sorta hate each other.

Steve Zissou almost ruined a budding relationship because I rolled my eyes the ENTIRE TIME!

Now I like it though, so you don't have to write my name down in a little book of people who SUCK and probably hate kittens.
posted by kathrineg at 10:56 AM on September 22, 2009

I rather like it--these sort of algorithms are the closest thing to an omniscient god that we will ever know.

And all these intelligent people spending all of this time just to recommend you a movie! Wow, what a cool time to be alive.
posted by kathrineg at 10:59 AM on September 22, 2009 [3 favorites]

Carmody'sPrize: Both Tideland and Southland Tales fit that category and I enjoyed them both immensely. Would I recommend them to others? Nope.

Would you recommend Southland Tales to me? I love Tideland, I think Donnie Darko was kinda neat before I figured out what was going on but the more I learned about what Richard Kelly intended the more I think it's stupid and overreaching and ridiculous; I haven't seen Southland Tales but I too tend to gravitate towards films the merits of which are hotly debated.
posted by shakespeherian at 11:01 AM on September 22, 2009

This is pretty easy, actually. I don't like moves that suck. Please recommend movies that don't suck.

Always happy to help.
There were four sequels to Bring it On? Seriously?
posted by Crash at 11:22 AM on September 22, 2009

Bring It On Again
Bring It On: All or Nothing
Bring It On: In It to Win It
Bring It On: Fight to the Finish

There were also 2 sequels to The Cutting Edge (14 years after the original):

The Cutting Edge: Going for the Gold
The Cutting Edge 3: Chasing the Dream

The most interesting thing about these is that they share zero cast members with any prior versions.
posted by smackfu at 11:36 AM on September 22, 2009

They just need to add a third category: We are pretty sure you'll either love it or hate it.
posted by callmejay at 11:48 AM on September 22, 2009

I'm with you, Crash. Finding movies that don't suck, however, is pretty hard to do. Heck, in a society that thinks King of Queens is quality TV, one has to wonder.

FWIW, we tried Netflix as a trial and had trouble watching two movies in the one-month trial period. Guess we're too busy with life.
posted by Man with Lantern at 11:52 AM on September 22, 2009

Also, it helps if you own a tv.
posted by box at 11:54 AM on September 22, 2009

Not much of a fan of any of those movies. Meh.
posted by Maztec at 11:58 AM on September 22, 2009

Alice has watched A, B, C, D, E, F and only rated A, B, C highly. Bob has watched B, C, D, E, Y, Z and only rated B, C, Z highly. Based on Bob, Alice is recommended Z but not Y. And Bob is recommended A but not F.

Mallory has replaced every DVD with "The Grudge II".
posted by Kickstart70 at 12:19 PM on September 22, 2009

Steve Zissou almost ruined a budding relationship because I rolled my eyes the ENTIRE TIME!

Lost in Translation almost ruined a budding relationship because she couldn't shut up about the "power differential" between the B.M. character and the S.J. character. If only it did.
posted by Durn Bronzefist at 2:05 PM on September 22, 2009

Napoleon Dynamite...I Heart Huckabees...Lost In Translation...The Life Aquatic With Steve Zissou...Kill Bill...Sideways...

Throw The Royal Tennenbaums and You And Me And Everyone Else in there and this is the list of films that, if I see any of them on somebody's shelf, I no longer associate with this person because pretty soon they'll be random-playing Deerhunter and Animal Collective and Fleet Foxes and complaining that Trader Joe's has no more fucking...no more fucking instant sable-banana latkes. Jesus Christ. At least Commando had a plot.
posted by turgid dahlia at 3:45 PM on September 22, 2009 [4 favorites]

I don't care how good their recommendation engine is, I'm not subscribing to netflix until they stop putting pop-under ads on half the sites I visit. Yes, I know about adblock. Not the point.
posted by desjardins at 5:22 PM on September 22, 2009

My favorite thing about Metafilter is the lack of lazy knee-jerk stereotyping that goes on based solely on superficial criteria like taste in movies or music.

Oh. Wait.
posted by tkchrist at 6:40 PM on September 22, 2009

I'm curious...what would be the effect of taking those top 25 movies and excluding them from the ranking algorithm? Sort of of throwing out the most troublesome data? I know it wouldn't solve the issue, but...I wonder if the end result, in this case, might not be a MORE accurate suggestion engine?

If the ratings for these movies were pure noise, the equivalent of a die role, then excluding them could result in more accurate ratings. However, if they are honest ratings that really do give insight to a person's opinions on movies, then it might be helpful to give them more importance during learning rather than less, which is called boosting. I always liked boosting because it makes intuitive sense in the way of practicing the things that you're bad at, but it's additionally a very important theoretical result in a field where it's extremely difficult to build theory.
posted by Llama-Lime at 6:42 PM on September 22, 2009

My main problem, as a researcher, is start up money. Who is going to provide me with the money to buy equipment and supplies, to hire the techs and pay for my own time to do the work? I can't get this on easy credit---returns are uncertain, remember, even if they do cover the costs. I need a sugardaddy to provide start-up capital, either as government grants or as a direct investment.

Don't worry, I'm sure the people who were abusing Lilly Allen will be here shortly to explain why if you were a real scientist with any itegrity you'd be happy to work for free, just like musicians should.

I'm sure they'll move on to explaining to farmers, power companies, and doctors why you should get their services for free in time for you not to starve.
posted by rodgerd at 8:30 PM on September 22, 2009

A friend of mine from undergrad was on that second, tied-but-losing team. Man, did I ever feel for him...
posted by ilana at 9:30 PM on September 22, 2009

It took 358 years for a (fat) proof of Fermat's Theorum. It may take longer to decide whether (Napolean Dynamite)^n + (Kill Bill)^n = (I Heart Huckabees)^n.

Especially since none of them are prime.
posted by Twang at 12:29 AM on September 23, 2009

I'm with Jimbob: I liked all of these, to varying degrees. So you might as well call it "Uncategorizable" and sign me up to that category.

Yeah, sorry for my bias, but when I saw the list of movies that are impossible to categorize, my first thought was "Hey, I like all of those, and if a new movie comes out that they have a similar degree of difficulty in generating recommendations for, I guess I'd probably like it too!". I guess I'm a weird data-point myself, then.
posted by Jimbob at 7:40 PM on September 23, 2009