Why Netflix Never Implemented The Algorithm That Won The Netflix $1 Million Challenge
April 18, 2012 3:46 PM   Subscribe

Why Netflix never implemented the algorithm that won the Netflix $1 Million Challenge.
posted by reenum (45 comments total) 19 users marked this as a favorite
 
Huh, that's really interesting, although I think another way of translating "For DVDs our goal is to help people fill their queue with titles to receive in the mail over the coming days and weeks...For streaming members are looking for something great to watch right now" is "Our streaming selection is just not as good as our DVD selection". There's ton of stuff I would watch on streaming that is not available, so I end up watching hours of TLC shows & old sitcoms instead. I still have a good time, but I wouldn't mind more new releases on streaming.
posted by ThePinkSuperhero at 3:52 PM on April 18, 2012 [15 favorites]


We looked at the two underlying algorithms with the best performance in the ensemble: Matrix Factorization

The second was the Juno Razor.
posted by nathancaswell at 3:52 PM on April 18, 2012 [5 favorites]


ThePinkSuperhero, that's a really good point. The space of possible recommendations for streaming must be so much smaller (and duller) that a recommendation algorithm that worked well on a larger space may simply not work with the new set.

If most streaming videos are TV shows and C-level direct-to-video content, having a super-awesome algorithm that picks out just the right snooty French film for a Criterion Collection type may not be so useful.
posted by zippy at 3:57 PM on April 18, 2012 [1 favorite]


I really feel like there's a lot more pushing on Netflix streaming in order to cover up the fact that they don't have a lot of great movies. Instead of suggesting movies I might actually like, they see I like one gritty crime film and suggest every gritty crime film they have to me, including a couple of movies that it tells me I will likely rate one star. Any time I give one of these movies a chance, I do indeed rate them zero stars. They're pushing movies at me they have every reason to think that I will hate, because they don't have enough movies they think I will love.

Sigh, Netflix. I had big hopes for you. Now, like PinkSuperhero above, I mostly use you to watch old sitcoms. And I'm running out of those.
posted by Bookhouse at 4:02 PM on April 18, 2012 [6 favorites]


I have yet to be truly sorry I chose DVD delivery over streaming. I half-enjoyed streaming movies - occasionally, when I was bored and didn't have a DVD handy - while it was bundled for free with the DVD service; but by the time they wanted to start charging me for it I'd already determined that I had pretty much seen everything available that I really cared to. Nice (not really) to know they haven't changed a damn thing about it in the time since. Get back to me when the streaming content rivals the DVD content, Netflix...I'll hold my breath in the meantime....
posted by Greg_Ace at 4:14 PM on April 18, 2012 [1 favorite]


I agree, zippy, but isn't there a lot of Criterion collection films on Instant Play?

But just to be clear, Netflix instant is anemic unless you are primarily into documentaries and TV (which aren't too hard to find from other free/cheap streaming sources like Hulu and Amazon Prime). The movies aren't all bad, but they're few and generally middling.

Still, I don't see quitting Netflix any time soon, or going to a DVD only account. I do like TV and documentaries enough I'll forgive the movie selection weakness.
posted by mccarty.tim at 4:15 PM on April 18, 2012


It has never been more apparent than now that recommending you the movie you'll actually like best is no longer Netflix's business — pushing the mediocre crap they happen to have available for streaming is. The weird thing is that they seem to have consciously decided to ignore the fact that they still have the world's best selection of movies on DVD and could make a lot of money on cinephiles and people with obscure tastes; streaming five-year-old Law & Order reruns is somehow a way more exciting business model.
posted by RogerB at 4:16 PM on April 18, 2012 [8 favorites]


I agree, zippy, but isn't there a lot of Criterion collection films on Instant Play?

Not as many as you'd think. The real Criterion action is over at Hulu Plus, which has a dedicated subcategory filled with movies that Criterion has the rights to, i.e. more than they have out on disc. As I recall, Netflix wouldn't play ball with them as far as maintaining a separate "Criterion section" and so lost out.
posted by Bromius at 4:20 PM on April 18, 2012 [2 favorites]


It wasn't just that the improvement was marginal, but that Netflix's business had shifted ... Suddenly, the prize winning solution just wasn't that useful...

Did you hear that? Sounded like a group of engineers just climbed out onto a high ledge...?
posted by Cool Papa Bell at 4:21 PM on April 18, 2012


Tangentiality, I wish Netflix kept the search so that it would show titles on DVD or even hot yet-released so I can "save" them for when they do show up. The current streaming-only search bar will just show titles I can watch at this moment. So if a movie is going to become available next week, it isn't' listed in the search. It is as if it doesn't exist.

I use the queue as a big list to get to later. The problem is streaming titles rotate in and out all the time so if on Monday I put it in the queue, it might not be there on Tuesday.

The suggestion engine when I had both DVD and streaming seemed to almost read my mind. Now with a smaller selection, not so much. But there's gems in there sometimes but then, like my TiVo I get the feeling that it doesn't know me at all.

I'm sure Netflix would prefer to have all the great movies on streaming but that's not what the studios want. Why get a few cents when you can sell a DVD or iTunes/Amazon download for much more?

Maybe in 5 years it will be better. God I hope so.
posted by birdherder at 4:31 PM on April 18, 2012 [1 favorite]


I'm still upset that Netflix got rid of their "Friends" pages. Some of the best suggestions I ever got were from friends on MetaFilter. Also: I got to discover that I was 78% movie compatible with AmbrosiaVoyeur, which is why I'll always love her.
posted by ColdChef at 4:38 PM on April 18, 2012 [10 favorites]


Let me guess: It would have cost too much money and we don't need to care about our subscribers. Or it was Reed Hastings call. Or both.
posted by Big_B at 4:39 PM on April 18, 2012


My job is machine learning research. I am not at all surprised that Netflix would implement the algorithms that they viewed mostly responsible for an 8.43% increase in accuracy and stop there, rather than devoting a lot of engineering effort to an additional 1.57% increase in accuracy. This amount is negligible, and possibly not reproducible on their actual dataset.

I think some people are seriously underestimating the amount of work that goes into taking an ensemble classifier which uses not just one algorithm, but hundreds of algorithms written and run by different people on different machines1 and getting all of that working in a production environment. It's hard to imagine that any customers would actually prefer Netflix to spend the hundreds of thousands of dollars necessary to get a <2% increase in accuracy than to spend it on additional content rights.

1 I imagine the form of the final solution was not what they had originally envisioned either.
posted by grouse at 4:59 PM on April 18, 2012 [9 favorites]


What information is available on whether how much Netflix's recommendations are based on payment/advertising agreements with movie/TV studios? On Amazon for instance, the stuff you see in "Recommended for You" is partly based on your shopping history + partly based on which books/movies Amazon is getting paid to target-suggest today, a la "pimp this teen book to everyone who has The Hunger Games in their checkout cart." Is the targeted advertising part of Netflix's algorithm(s) or do they say they don't paid placement at all?
posted by nicebookrack at 4:59 PM on April 18, 2012


Cool Papa Bell: Did you hear that? Sounded like a group of engineers just climbed out onto a high ledge...?

I could be wrong, but for the mathematicians, scientists and engineers in the optimization contest, their goal wasn't about getting their team's work implemented, but about getting recognition for their work. To that point, even the money wasn't worth too much. With more than 2,000 hours of work from the winning team, that's $500 per hour. But was it 2,000 "man-hours" from all the members combined, or hours the total team put in? And how much did their computing time cost? And then there are the teams that didn't win.
posted by filthy light thief at 4:59 PM on April 18, 2012


From the Netflix blog: We have adapted our personalization algorithms to this new scenario in such a way that now 75% of what people watch is from some sort of recommendation.

That's smart of them to credit their algorithms for the reason more people watch things based on recommendations. As others have pointed out in this thread, their streaming selection is more limited than their collection of physical discs, so people are less likely to find what they want, and have to settle for something else.
posted by filthy light thief at 5:03 PM on April 18, 2012


I guess I'm weird for being fine with paying a $23 flat fee every month and having streaming and two blu-rays out at any given time.
posted by beaucoupkevin at 5:13 PM on April 18, 2012 [3 favorites]


And then there are the teams that didn't win.

I am sitting about five meters away from a member of the #2 team. He has previously told me that his motivations were fun, a great way to learn more about machine learning, and a shot at the money. I could be wrong, but I doubt many were doing this because they wanted to see their algorithm implemented exactly as they wrote it.
posted by grouse at 5:14 PM on April 18, 2012 [1 favorite]


If most streaming videos are TV shows and C-level direct-to-video content,

My current "top ten" recommendations on Netflix streaming:
Union Station--1950 Rudolph Mate film
Dreamworks Spooky Stories--huh?
Impact--1949 Arthur Lubin film noir
The Scar--1948 Paul Henreid film
The Other Love--1947 Barbara Stanwyck, David Niven romance
Mesrine: Part One: Killer Instinct--2008 French movie starring Vincent Cassel
London Blackout Murders--1943 thriller.
Darling, How Could You--1951 British film based on a J. M. Barrie comedy.
The Girl Who Dared--1944 thriller.
De-Lovely--2004 biopic about Cole Porter

Not a single TV show and only one direct-to-video production in the mix. I don't deny that they have lost content, but there's still a lot of terrific stuff available on Netflix streaming. The problem is that no one can figure out just what's a fair price to charge Netflix for content. I'm guessing that eventually a fair price that allows Netflix to make a profit and gets the material available to viewers will be arrived at, but I can understand why the media companies feel leery about making a deal that ends up making them look like chumps.

Tangentiality, I wish Netflix kept the search so that it would show titles on DVD or even hot yet-released so I can "save" them for when they do show up

I still do this all the time. I just now tested it by searching on Cabin In The Woods and, sure enough, there it is with a little green "save" button under it. Mystery.
posted by yoink at 5:15 PM on April 18, 2012 [2 favorites]


My guess is that all the crap on their streaming offerings created a plethora of very low ratings that threw off the algorithm completely. When getting the DVDs I tended to watch movies that I knew I might already rate highly. When streaming I went fishing for *something*, *anything* decent and usually gave low ratings.
posted by thorny at 5:21 PM on April 18, 2012


I just got annoyed by Netflix magically disappearing items from my queue. I had The Apartment (French movie) in my queue, and I knew I had last year watched The Apartment (New York movie). So my boyfriend said he had to watch The Apartment (NY) for class, and I told him to come over to my place to watch it on Netflix - and it's gone. Doesn't even turn up in search, and the french one is no longer in my queue. I would much rather it was there and 'n/a' so I didn't spent ten minutes searching and thinking I was crazy.
posted by jacalata at 5:23 PM on April 18, 2012


Jacalata: if you mean the film with Monica Bellucci and Vincent Cassel--they still have that, it's currently in my queue. Oh, sorry--you mean because it used to be available on Instant Streaming and now it's not. But what is Netflix supposed to do about that? If the company that owns the film only sells them the instant viewing licence for a limited amount of time, then they have to pull it when their time's up, no?
posted by yoink at 5:32 PM on April 18, 2012


I'm still upset that Netflix got rid of their "Friends" pages. Some of the best suggestions I ever got were from friends on MetaFilter. Also: I got to discover that I was 78% movie compatible with AmbrosiaVoyeur, which is why I'll always love her.

It takes a little bit of work, but if you fill out a decent number of rankings at Criticker, it does a really good job of predicting your ratings for films you haven't seen. The recommendations are hit and miss for me, but one of the best things about it is that you can have the site generate a list of other users that rate films similar to you, and then you can go through their list and pick out films to watch.
posted by burnmp3s at 5:37 PM on April 18, 2012


I don't mind so much that it is no longer available to stream - I just found the experience really frustrating. I search, I get a stream of irrelevant results, scan through them, it doesn't turn up, I think 'oh do I need 'The' in the title? Do I need it not in the title? Did I misremember the name altogether? I guess I should check IMDB'.

I'd prefer to find the movie, and see 'Not available for streaming'. Hell, if that happened enough, I'd think about turning my DVD subscription back on.
posted by jacalata at 5:55 PM on April 18, 2012 [1 favorite]


I’m a total sucker and still pay for both Netflix services, and use neither very often anymore. I gave up on trying to understand what they’re thinking long ago. They got rid of all their social aspects just as everyone else was falling all over themselves to implement them and that was one of the main things I liked. Then they put all their effort and push into the streaming, yet they don’t have much of anything to stream.

I’ve watched 3 streaming movies in the entire time I’ve had Netflix, 2 of them just to see if it worked. We are not browsers and that’s who the system is set up for. The last time we went through our list of "movies to see" we had 30 movies; 29 of them were available on DVD, 1 was available streaming.
posted by bongo_x at 5:57 PM on April 18, 2012


I'd prefer to find the movie, and see 'Not available for streaming'. Hell, if that happened enough, I'd think about turning my DVD subscription back on.

Yes, odd that they don't, really. I guess they just aren't interested in trying to scare up customers for their DVD service. Because, of course, it would happen almost every time you fail to find the movie on streaming. They really do have a fantastic amount of stuff on DVD (including both versions of The Apartment of course).
posted by yoink at 6:07 PM on April 18, 2012


I don't think they want to mail DVDs out to people. Yes, getting DVDs in the mail is nice for the customer because of the selection, but they don't want to foot the bill (they are a MAJOR customer of the USPS). They keep doing it so they can keep customers and retain whatever is left of their brand.

I have sympathy for them as a company re their streaming woes. My job has me in a Netflix-like situation where my team has a better way to present information, but that information is produced by other players and we're completely dependent on them. So I don't think badly of Netflix just because content producers aren't playing along. I just happen to no longer give them my money. (I also think that they really dropped the ball with their completely separated DVD and streaming queues, but that's a different matter).
posted by a snickering nuthatch at 6:08 PM on April 18, 2012


I have a Netflix Streaming and Hulu Plus account. Both have strengths (Netflix is really good for kids programming - which is very useful and Hulu has a pretty good selection of this season of TV shows although why some shows are available on computer and not TV playback I have no idea). Both pretty much suck in terms of movie selection.

It seems like the big media companies are desperate to strangle Netflix streaming in the cradle as they'd much rather have people pay some crazy amount for itunes based digital downloads instead of an all you can eat for a low price model. Companies that have big collections of crap that doesn't command a price premium like first run blockbusters probably don't mind offering their digital offerings as one big bundle because some money is better than no money but the big boys seem to be desperate to avoid cannibalizing their current market (cheap rentals through kiosks and more expensive single disk purchases/boxed sets).

The problem with this model is that some people are never willing to go back to premium movie price and that money is just staying in people's pockets.
posted by vuron at 6:19 PM on April 18, 2012


Movie recommendations is a tough nut to crack. Previously - "The Napoleon Dynamite problem"
posted by ymgve at 6:23 PM on April 18, 2012


true fact I tested last week: Netflix has 18 movies with Billy Zane in them compared to 7 with Al Pacino.
posted by Bookhouse at 6:30 PM on April 18, 2012 [2 favorites]


Netflix Streaming, obviously
posted by Bookhouse at 6:30 PM on April 18, 2012


7 with Al Pacino.

And one of those is Gigli!
posted by yoink at 6:42 PM on April 18, 2012


It's interesting that they used the first result and all the pragmatic reasons for a pass on the best algorithm just follows the internet rule that best is the enemy of the good, er the Voltare rule. As jacalata suggests, it's there's value in tracking what I'm interested in, even if I'm not watching it right now.

I'd think that it would be both valuable to me and to Netflix to maintain a repository of my various interests both past, current and many potential futures. There's a lot of stuff I'd like to make note of that I know I'll be interested in sometime. I'd think that knowledge base would be useful to them strategically.

I'd also really like to be able to download a simple big list of what's available rather than thumbing through icons. Carpel tunnel film selection, garrr.
posted by sammyo at 6:51 PM on April 18, 2012


If most streaming videos are TV shows and C-level direct-to-video content
I'm guessing that new releases are expensive for Netflix to stream. TV networks don't want to stream a season of Game of Thrones that just came out on DVD, as it will damage sales of the DVD.

A while back, Netflix was trying to run their own original programming on the streaming service, but that doesn't seem to have worked out yet. You can imagine how hard it would be for an innovator entering the TV business to break through the huge barriers that the networks have in place.

Remember when cable first came out? (Well, I don't because I'm too young, but anyway..) Originally, cable channels had their own niche markets. The history channel had actual history programming, and the cooking channel had its own cooking shows – not the weird reality show fare that is on every cable channel now. The big networks didn't start buying up cable channels until cable TV had a sizable viewing audience. I imagine the same thing will happen with internet streaming services. As bandwidth gets cheaper and more people have internet-ready TVs and mobile devices, the demand for streaming content will increase, and there will finally be enough money in it for the networks to think it's worth their time to develop shows for Hulu and Netflix.

In the meantime, the whole business of streaming is starting to push into the TV networks' territory, and I wouldn't be surprised to learn there has been a lot of resistance or hesitation from the networks.
posted by deathpanels at 7:01 PM on April 18, 2012 [1 favorite]


Maybe it's just me, but I've actually been pretty happy with Netflix. My only major complaint is the lack of movies available on streaming, but the selection they have is still not too bad - plenty of good documentaries, most TV shows (at least the ones I watch), etc. It's not perfect, but for the price it's still way better than cable.
posted by photo guy at 7:26 PM on April 18, 2012


I'm a dual-track Netflix user, and at this point I'm pretty happy with the viewing rhythm I've worked out. I get movies I really really want to see via the DVD service, preferably Blu-Ray. On streaming, I primarily watch TV series I missed the last decade since I haven't had any real access, so things like 24, 30 Rock, The Office, OR things I want to finally rewatch or catch up on, mainly SF, so Star Trek, Farscape, Blackadder. I also try to make use of the streaming movies as a sort of jackpot thing, which balances well with the other options. Finally, I am trying to watch a lot of classic movies -- some that I have in my years-long Netflix queue and for which there aren't Blu-Ray releases -- via my local library and county interlibrary service. Right now, I'm pretty well satisfied with how this is working out.

I don't think they want to mail DVDs out to people.

Yeah, they made that clear enough last year. Somebody like Paul Graham pointed out how ass-backwards this whole New Coke maneuver was. They have a nearly-unique technological and logistical advantage (comparable to Amazon and Wal-Mart in many ways) that is their DVD warehouse network. Startup and learning costs are huge for any potential competitor. They own this space and as long as they don't screw up the brand could get rich off it forever. It's the ideal rent-seeking business model. But no. They need to stream. So they move into a space where the startup costs are negligible, the infrastructure costs are borne by others, the learning feedback can be instantaneous, and their biggest suppliers also want to be their biggest competitors. This is a brutal business environment and in some ways they may not really understand it as well, and in fact their business model might not work as well as the a-la-carte approach that exists with iTunes/Amazon/etc. Still, it's the windmill at which they are determined to tilt. Strange way to think.

Anyhoo. I have recently gotten to a point, especially with the way I get "Top Picks" delivered on my Sony box, where I'm frequently very happy with the recommendations. (Indeed, it's a bit spooky that they seem able to recommend based on my box apparently telling them what I watched on DVD -- e.g. I saw a Paul Newman movie on DVD from the library and Netflix suddenly started recommending Paul Newman movies to me.) I have an instant queue, current TV series included, that's about 75 titles right now, and I've had to develop a sort of first refusal rule for my queue based on the day of the month -- last night I made myself watch #17. The similar movies recommendations on the DVD side, up on the website, are also pretty darn good. (I was never too successful at the Instant side's theme recommendations, e.g. "mind-bending foreign action thrillers", for what that's worth.)

I will say that one reason I'm not so disappointed by the recommendation system is that I have never relied on it. I came too late for most of the social features, but when I started I seeded my queue with a lot of top-movie lists, such as the top british film list that was here on MeFi last year, and where possible with Library of Congress National Film Registry entries (though I'm getting these more quickly via my local library). I also periodically trawl my queue and promote like crazy movies that have high predictive ratings for me and/or that are on Blu-Ray. Seriously -- I'm on the one-at-a-time plan and my actual viewing rate is about one title every six days, so I have seven years of viewing I'll probably like queued up.

So, I dunno. Maybe if you're really disappointed ur doin it rong? I don't want to be unfair, but it seems to me it's possible to work with the limitations to some degree and mitigate them with other means.

I'm guessing that new releases are expensive for Netflix to stream.

I imagine this is in fact true, but more realistically, the studios will not license them for this type of viewing. They think it dilutes other revenue streams and cheapens their premium content by being peered with other crap like C-list movies and TV shows. So even if Netflix were richer I don't think it would necessarily be available. Whenever you're looking at the streaming service's limitations I think you need to keep this in mind -- it probably isn't Netflix that's to blame.

I just got annoyed by Netflix magically disappearing items from my queue.

Oh, then you need to keep an eye on Instantwatcher.com. I check this occasionally but it doesn't drive my viewing that often. If I had only streaming I would check it much more so I know what's going to magically disappear when the rights run out. Also, I don't know how consistent it is, but I've seen things magically reappear when they get the rights back.'

I don't deny that they have lost content, but there's still a lot of terrific stuff available on Netflix streaming.

Well, the loss of Starz Play was probably bad for them. I liked and watched a handful of those titles, but they were broadly "popular movies", stuff like Lethal Weapon and so forth. Still, they get new content on a regular basis, according to my box's new arrivals queue, so it isn't just loss, it's more like churn.

I wish Netflix kept the search so that it would show titles on DVD or even hot yet-released so I can "save" them for when they do show up

It's there, but it may depend on how esoteric or immediate your tastes are. I find it works about 3/4 of the time, but the other 1/4 means I have no way to easily find out when it will suddenly become available. It's pretty good at new releases but spottier, and there are many more obscure titles that just aren't in the system at all even though they are critical favorites or historically important or whatnot and you'd just expect them to be there.

Anyway, I fear a future where this continues to be the norm and availability of any given title is inconsistent or spotty or, yes, untrackable, or where the market fragments and you're never sure whether you can get, oh, Drive on Netflix Instant, Hulu, Amazon, HBO Go, iTunes, etc., or Netflix DVD, or online rental or .... And there's probably going to be differences in whether you can see it on your PC, your TV, or your mobile device to boot, and then fine-grained DRM/platform issues along the lines of PC codecs ...

I want an interface, similar to iTunes or Amazon or Netflix or any of them really, that lets me locate any accessible movie and then determine if I have the means to watch it or want to spend the fee to get the means. That open-access ideal seems remote, though, even if what we have is a cornucopia of plenty compared to the old pre-VHS days of hoping ABC might run your movie some Sunday night.
posted by dhartung at 10:31 PM on April 18, 2012 [3 favorites]


Netflix was trying to run their own original programming on the streaming service, but that doesn't seem to have worked out yet.

Oh, Lilyhammer has been out a while, although it turns out not to have been produced for them exactly, but is in fact a Norwegian TV production for a Norwegian audience. Although it's fun in its own way, and I'll recommend it, at first it can seem like a really mixed bag of slightly comic stuff mixed with really tired mobster-movie tropes, until I realized that what it is, fundamentally, is a social comedy about Norwegian culture by and for Norwegians. At that level it's remarkably comparable to the Wallander approach (Mankell speaks of writing about "the Swedish problem") and can be enjoyably wry. Also, Steve Van Zandt is no actor and seems to have few expressions other than this rubbery scowl, but manages to charm nonetheless and at the end pulls off a nice parody of things his real Little Steven persona has contempt for, like American jingoism, racism, and so forth.

Later this year, supposedly, House of Cards will drop, the Kevin Spacey production recapitulating a famed British series of novel adaptations, now in an American context. I'm keen because those Ian Richardson vehicles were spectacularly good and you should watch them anyway. Then there's the new season of Arrested Development but I don't know when that's coming.

But this can never be seen as producing enough content to take up a lot of viewing time or replace their need to license other content. It's just premium stuff to entice people to use the service who might otherwise not.
posted by dhartung at 10:41 PM on April 18, 2012


This was all mostly about publicity anyway. It's really hard to imagine there could have been a business model (like, an actual computational or formal model) for Netflix that showed they would gain any additional profit by improving the error rate on their recommended stars from 1 star to .9 stars. Even if they were in the business of selling DVDs directly, such a close connection between rating accuracy and profit would be hard to imagine, let alone when their business was actually selling recurring subscriptions.

But I'm okay with that. It was actually an amazingly illuminating challenge to have attempted and watched. The lesson was a deep one: it's really, really hard to beat the mean. The whole challenge was beating the default 1-star-on-average error you get just making the mean movie rating as your guess for everyone. But the challenge was to reduce that to about .9 stars! Essentially, indistinguishable to any user. More importantly, it shows that even with vast amounts of data, making personalized recommendations that go beyond just recommending what people in general like is really hard. Hard, as in unsolved by anyone in any domain. I think if anyone managed to reduce the average error to, say, .2 stars, that would require not just a revolution in machine learning, but a deep theoretical revolution, Fields-medal work.

On the other hand, when I was trying my hand at the Netflix challenge, I found that you could indeed do fairly well for a lot of people -- but there were a whole lot more who had rated relatively few movies, and rated those pretty randomly. So maybe the challenge is hard just due to pervasive idiocy... I hope not.
posted by chortly at 12:16 AM on April 19, 2012


It wasn't just that the improvement was marginal, but that Netflix's business had shifted and the way customers used its product, and the kinds of recommendations the company had done, had shifted too.

This is a really, really, really important point, and reveals the big flaw of rewarding invention with prizes: even the most impartial prize-setting organization will have a natural bias towards rewarding an invention that would have been useful in the past, rather than an invention that will be useful in the future.

As Steve Jobs once famously said about market research: "You can't just ask customers what they want and then try to give that to them. By the time you get it built, they'll want something new."

For all its flaws, this is the big comparative strength of the patent system compared with prize-drive innovation: since the eventual payoff is proportional to the actual subsequent commercial success of the invention, it rewards those forward-looking or lucky enough to actually anticipate future needs rather than respond to past ones.
posted by Skeptic at 1:28 AM on April 19, 2012


I just now tested it by searching on Cabin In The Woods and, sure enough, there it is with a little green "save" button under it.

don't get your hopes up. netflix is a popular outlet for knockoff movies, knockoff titles, and knockoff covers that take advantage of the current buzz. i think they have a whole department for it.
posted by fallacy of the beard at 7:30 AM on April 19, 2012


It takes a little bit of work, but if you fill out a decent number of rankings at Criticker, it does a really good job of predicting your ratings for films you haven't seen.

Yeah, and remember the Female Character Flowchart that was posted here a while back? They put in on their front page and talked about how it demonstrated how rich and varied women's roles are nowadays.
posted by heatvision at 8:28 AM on April 19, 2012


don't get your hopes up. netflix is a popular outlet for knockoff movies, knockoff titles, and knockoff covers that take advantage of the current buzz. i think they have a whole department for it.

I've seen those--I don't think they're in any way affiliated with Netflix--I just think they offer their rights really cheaply so Netflix buys them. But they don't get to use the official movie poster and list the director under his/her real name and rename all the stars with the names of the actors from the real film etc. etc. etc.

So, no, this is the real CITW.
posted by yoink at 8:45 AM on April 19, 2012


> We looked at the two underlying algorithms with the best performance in the ensemble: Matrix Factorization

The second was the Juno Razor.

nathancaswell, help me out. Google is useless here.
posted by IAmBroom at 9:25 AM on April 19, 2012


The joke is if you want to find out what kind of movie a person likes you ask if they liked The Matrix or Juno.

I wanted to do Napoleon Dynamite but it didn't fit as well.
posted by nathancaswell at 2:25 PM on April 19, 2012 [2 favorites]


> So, no, this is the real CITW.

yoink, do you have a disc-by-mail account? Netflix removed this functionality for streaming only accounts. If I go to the Netflix CITW page (which I can only find via google), there is no "save" button - it just tells me the title is unavailable. If I search for it using the Netflix search function, it doesn't show up at all.
posted by JerryCornelius at 8:24 AM on April 20, 2012


« Older It's rude to blurb in public   |   That blowed up real good Newer »


This thread has been archived and is closed to new comments