The Decomposition of Rotten Tomatoes
September 6, 2023 5:51 AM Subscribe
I learned a valuable lesson about the way Rotten Tomatoes ratings worked when Mrs. Fedora’s friend wanted to see Oz the Great and Powerful. I thought it looked pretty bad but I glanced at Rotten Tomatoes, and saw that it had a score of 60, maybe 65%, and I figured, hey, I guess maybe it’s all right.
Then we watched it and it was very bad, and when I came home I took a closer look at the reviews themselves that Rotten Tomatoes had aggregated. Turned out the breakdown was basically “40% of critics said it was irredeemable garbage” balanced by “60% of critics said it was barely passable.”
posted by DoctorFedora at 6:17 AM on September 6, 2023 [4 favorites]
Then we watched it and it was very bad, and when I came home I took a closer look at the reviews themselves that Rotten Tomatoes had aggregated. Turned out the breakdown was basically “40% of critics said it was irredeemable garbage” balanced by “60% of critics said it was barely passable.”
posted by DoctorFedora at 6:17 AM on September 6, 2023 [4 favorites]
First, this makes me want to start reviewing movies. I have lots of opinions! I will absolutely say "this is a positive review for a movie I like" when I mean "this is an interesting movie with flaws".
Second, and as usual, a case of the internet externalizing and deferring drawbacks in the name of convenience. You no longer need to seek out a movie critic, decide if their taste and judgement suit you and then read the whole review - you just get a convenient number. The fact that it's bad number that means very little doesn't matter, the fact that difficult or interesting but flawed movies can't get a look-in doesn't matter... because that all happens off-screen, as it were.
posted by Frowner at 6:25 AM on September 6, 2023 [5 favorites]
Second, and as usual, a case of the internet externalizing and deferring drawbacks in the name of convenience. You no longer need to seek out a movie critic, decide if their taste and judgement suit you and then read the whole review - you just get a convenient number. The fact that it's bad number that means very little doesn't matter, the fact that difficult or interesting but flawed movies can't get a look-in doesn't matter... because that all happens off-screen, as it were.
posted by Frowner at 6:25 AM on September 6, 2023 [5 favorites]
I watch about 4-5 movies a week (sometimes more) so I find RT a helpful tool to get a general idea of some movies I've never heard of (I also used JustWatch & Letterboxd). It's absolutely imperfect, but it's a starting point.
posted by edencosmic at 6:37 AM on September 6, 2023 [1 favorite]
posted by edencosmic at 6:37 AM on September 6, 2023 [1 favorite]
I use RT, but with a healthy dose of salt. If the film features women in any significant way -- directors, writers, main characters -- you can automatically bump the RT meter up by about 20 points, for example. (Or just go to CherryPicks instead, but there you might want to bump it down about 10 points.) If a movie is from Marvel (and isn't about a woman) you can automatically bump it down by about 20 points.
It's most useful feature for me is as a place where I can get links to a bunch of reviews in one spot. I don't regularly read particular film critics in order to decide what I want to watch based on what they have written about -- I already know what I'm thinking about watching, and I want to get an overview of whether it is good or bad in case that changes my mind. Having all the reviews in one spot is helpful -- I usually click through to Top Critics and then glance over reviews written by women.
posted by jacquilynne at 6:39 AM on September 6, 2023 [16 favorites]
It's most useful feature for me is as a place where I can get links to a bunch of reviews in one spot. I don't regularly read particular film critics in order to decide what I want to watch based on what they have written about -- I already know what I'm thinking about watching, and I want to get an overview of whether it is good or bad in case that changes my mind. Having all the reviews in one spot is helpful -- I usually click through to Top Critics and then glance over reviews written by women.
posted by jacquilynne at 6:39 AM on September 6, 2023 [16 favorites]
Metacritic assigns scores to reviews and averages those, so ten raves show up much differently than ten OKs. They also have tighter standards for the reviews they count.
posted by Horace Rumpole at 6:41 AM on September 6, 2023 [25 favorites]
posted by Horace Rumpole at 6:41 AM on September 6, 2023 [25 favorites]
I don't regularly read particular film critics in order to decide what I want to watch based on what they have written about -- I already know what I'm thinking about watching, and I want to get an overview of whether it is good or bad in case that changes my mind.
Seconding this. I actually pick things to watch based on a gut-instinct feeling after watching the trailer, and sometimes I get second thoughts and pass (or word of mouth convinces me to pass). I don't actually read reviews until after I've seen something.
posted by EmpressCallipygos at 6:43 AM on September 6, 2023 [5 favorites]
Seconding this. I actually pick things to watch based on a gut-instinct feeling after watching the trailer, and sometimes I get second thoughts and pass (or word of mouth convinces me to pass). I don't actually read reviews until after I've seen something.
posted by EmpressCallipygos at 6:43 AM on September 6, 2023 [5 favorites]
“If there was a new film by, I don’t know, Klaus Von Boringstein,” says Schrader, “and he had a three-hour drama about a housewife in the Middle Ages, do you think people would go see it because it had a 90 percent on Rotten Tomatoes? No. But if it were a movie about a serial killer in the wilds of Alaska and it had a 50 percent? They might check that out.”
This dude is talking about misogyny and I can't tell if he's even aware of it.
I want to see your average Housewife of the Middle Ages. That movie would be full of interesting details. How many people even live in "the wilds of Alaska"* anyway I feel like the serial killer would spend the whole movie trudging through snow just trying to find people to kill.
*Sorry this sounds rude but isn't all of Alaska pretty wild?
posted by RobinofFrocksley at 6:49 AM on September 6, 2023 [42 favorites]
This dude is talking about misogyny and I can't tell if he's even aware of it.
I want to see your average Housewife of the Middle Ages. That movie would be full of interesting details. How many people even live in "the wilds of Alaska"* anyway I feel like the serial killer would spend the whole movie trudging through snow just trying to find people to kill.
*Sorry this sounds rude but isn't all of Alaska pretty wild?
posted by RobinofFrocksley at 6:49 AM on September 6, 2023 [42 favorites]
> "“If there was a new film by, I don’t know, Klaus Von Boringstein,” says Schrader..."
That... sounds like a comment from Oppositeland.
posted by kyrademon at 6:52 AM on September 6, 2023 [2 favorites]
That... sounds like a comment from Oppositeland.
posted by kyrademon at 6:52 AM on September 6, 2023 [2 favorites]
Metacritic assigns scores to reviews and averages those, so ten raves show up much differently than ten OKs. They also have tighter standards for the reviews they count.
Metacritic has been poison in the video game industry. Metascores by their very nature trend towards the mean, and punish interesting outliers, resulting in everyone focusing on What Makes Line Go Up to the detriment of the game. In addition, publishers will tie money to hitting certain Metacritic scores, and missing that peg can make the difference in survival for small studios.
Metascores like Metacritic and Rotten Tomatoes are harmful "tools" that undermine creative industries, and people need to stop defending them because if you sort of squint, they seem somewhat useful.
posted by NoxAeternum at 6:58 AM on September 6, 2023 [23 favorites]
Metacritic has been poison in the video game industry. Metascores by their very nature trend towards the mean, and punish interesting outliers, resulting in everyone focusing on What Makes Line Go Up to the detriment of the game. In addition, publishers will tie money to hitting certain Metacritic scores, and missing that peg can make the difference in survival for small studios.
Metascores like Metacritic and Rotten Tomatoes are harmful "tools" that undermine creative industries, and people need to stop defending them because if you sort of squint, they seem somewhat useful.
posted by NoxAeternum at 6:58 AM on September 6, 2023 [23 favorites]
When I started seeing those 'Rotten Tomatoes Approved' logos on DVD boxes, I figured things were headed in a bad direction.
posted by box at 7:10 AM on September 6, 2023 [12 favorites]
posted by box at 7:10 AM on September 6, 2023 [12 favorites]
The value proposition of a movie review is a stark binary: see the movie, or do not, and aggregating those go/no gos is useful.
RT has a structural vulnerability for bad movies that critics still feel compelled to say "go anyway" -
the "Certified Fresh" The Last Jedi being a classic example. Metacritic captures that ambivalence well.
posted by MattD at 7:18 AM on September 6, 2023
RT has a structural vulnerability for bad movies that critics still feel compelled to say "go anyway" -
the "Certified Fresh" The Last Jedi being a classic example. Metacritic captures that ambivalence well.
posted by MattD at 7:18 AM on September 6, 2023
MattD noooooo I'm screaming like Chewbacca over here hahaha
Anyway, Schrader may be misogynist but he's right--certain movie genres consistently do well with mediocre reviews and audience scores, and certain genres consistently score high and don't attract a paying audience.
I distrust RT because I'm not sure how they apply the fresh/rotten standard--if the critic is mostly miserable through Broken Lizard's Beach Blanket Orgy Massacre, that's going to get a rotten, but through a big studio's movie? Well, there's probably one edible spot in that rotten apple... try taking another bite!
That and it's useless for franchise content because there's always going to be a bunch of fan blogs that suddenly show up as registered critics. These used to be more white male fanboys, as they aged out it's become more diverse in terms of the writers but the reviews will always be the exact same--ten thumbs up!
posted by kingdead at 7:40 AM on September 6, 2023 [3 favorites]
Anyway, Schrader may be misogynist but he's right--certain movie genres consistently do well with mediocre reviews and audience scores, and certain genres consistently score high and don't attract a paying audience.
I distrust RT because I'm not sure how they apply the fresh/rotten standard--if the critic is mostly miserable through Broken Lizard's Beach Blanket Orgy Massacre, that's going to get a rotten, but through a big studio's movie? Well, there's probably one edible spot in that rotten apple... try taking another bite!
That and it's useless for franchise content because there's always going to be a bunch of fan blogs that suddenly show up as registered critics. These used to be more white male fanboys, as they aged out it's become more diverse in terms of the writers but the reviews will always be the exact same--ten thumbs up!
posted by kingdead at 7:40 AM on September 6, 2023 [3 favorites]
Boiling down reviews to a single score is poison. Even worse is when that score has high precision "this movie is 4.3 stars". Really? What objective measure means it's 4.3 and not 4.2? I like MattD's binary but in my perfect world reviews would be three possibilities: "bad / ok / good". (Read the review for any more subtle take.
The meta-review sites like Rotten Tomatoes have a different problem, averaging numbers. Don't show me an average. Show me a histogram of all review scores. Also normalize each score by the reviewer's average score. Admittedly that part is hard.
OpenCritic is my favorite review aggregator but it's for games only. See Starfield for instance. They are also guilty of publishing a single average number but they have a simple chart view which shows you all the scores. Not quite a histogram but at least you can get a feel for the distribution. Mostly though I look at the review details, the pull quotes from each review are very well selected.
Still OpenCritic causes problems. Like with MetaCritic publishers do dumb things like tie developer bonusses to whether the game got an average of 84 or 85. Also folks manipulate the review window, just like this article talks about with movies and Rotten Tomatoes. Microsoft chose not to give advance review copies of Starfield to some well known UK reviewers. There are various theories on why, but one of them is that Eurogamer only gives 1-5 stars and while a "4 of 5" review is very very good from them, the resulting 80 will bring the average down. It's all dumb marketing games.
posted by Nelson at 7:46 AM on September 6, 2023 [5 favorites]
The meta-review sites like Rotten Tomatoes have a different problem, averaging numbers. Don't show me an average. Show me a histogram of all review scores. Also normalize each score by the reviewer's average score. Admittedly that part is hard.
OpenCritic is my favorite review aggregator but it's for games only. See Starfield for instance. They are also guilty of publishing a single average number but they have a simple chart view which shows you all the scores. Not quite a histogram but at least you can get a feel for the distribution. Mostly though I look at the review details, the pull quotes from each review are very well selected.
Still OpenCritic causes problems. Like with MetaCritic publishers do dumb things like tie developer bonusses to whether the game got an average of 84 or 85. Also folks manipulate the review window, just like this article talks about with movies and Rotten Tomatoes. Microsoft chose not to give advance review copies of Starfield to some well known UK reviewers. There are various theories on why, but one of them is that Eurogamer only gives 1-5 stars and while a "4 of 5" review is very very good from them, the resulting 80 will bring the average down. It's all dumb marketing games.
posted by Nelson at 7:46 AM on September 6, 2023 [5 favorites]
RT is a useful tool, but it can't be used as an instant metric. I wouldn't say it has soured or gone downhill so much as people are becoming aware of the limitations of lazily using it as a one data point source.
That said, gaming the review system was always a thing in the movie business. People needing to pay attention to sources and read between the lines was always a thing. Yes, if a movie is at a sort of unclear level (like the low to mid 60's) you have to pay attention to sources. But having to check the web sites on a film with an RT rating of 62% to see how many of them are the likes of TerryLikesMovies.fart as opposed to the NYT or what have you is a pretty low hurdle. And didn't we used to get lobby cards and TV ads for movies that said "A TOUR DE FORCE!" and we had to read the small print to see if the critic saying that was Pauline Kael or some dude at the Idaho Register? At least these days, we can limit reviews to "Top Critics" with a single click, and that's helpful.
The way that movies that are broadly agreed-upon to be kinda good end up with great scores is also known to most serious moviegoers. M3GAN, for instance, got a 93% RT rating, because something like 9 out of 10 reviewers said yeah, kinda good, not because very many critics gave it the equivalent of an A. The recent German horror film Dawn Breaks Behind the Eyes got a 100%, but not one of the reviews considered it a flawless film. It's just that everyone agreed there was at least something to recommend.
Another thing I watch for with RT: theatrically released horror films are often artificially low. This is for the simple reason that many critics forced to review large batches of theatrical releases do not actually like horror films. This is getting to be less of an issue over the last ten years, thanks to generational shift, but it's a legacy problem when you look back at older reviews, with large swaths of boomer reviewers (sorry, that's who they are) giving what amount to automatic pans to anything less than Rosemary's Baby. Many horror fans I have spoken with use an automatic "add 20%" rule on RT for horror movies 10+ years old to get an actual sense of reviews.
As a hilarious example of this, consider the Final Destination film series. The first, often considered a modern horror classic now, has an RT rating of 36%. The second film has a higher RT rating of 50%, even as the consensus is that it was a clear step down from the not so bad original. By the time, they get to the third, it dips to a 43% fresh, with the consensus being that by that point, the series was no longer as classic and genius as it once was. The only film in the series to be rated fresh is the fifth, sitting at 63% fresh, even as the consensus is that by that point, it's a lesser film "only for the gore-thirsty faithful."
You have to click into RT and see who actually said what and try to get a sense of how convincing you find their arguments. But that was always true. I remember having to keep in mind that while I loved Roger Ebert and agreed with him on most things, he had: 1) a total lack of appreciation for unpretentious horror films that delivered the goods; and 2) a weird art boner for immersive CGI worlds... he loved Spawn for chrissakes. It's not a matter of finding critics who always agree with you. It's about finding good writers who are gifted at articulating why they like/dislike a film. I didn't have to agree with Ebert, I just had to know he would explain his take well enough that I'd be able to guess from what he wrote how I might expect to feel, knowing where we tended to agree and where we tended to differ.
It's the same now. As a horror fan, I have to recognize which critics seem to actually take horror for what it is, and which just pinch their nose, hoping for the next it's okay to like this "elevated horror" hit.
I do sort of kind of vaguely share the lament that film critics do not get to be recognized voices and tastemakers in the way that they used to be. But this isn't RT's doing, so much as a sad after effect of the decimation of the newspaper/print media industry.
posted by DirtyOldTown at 7:57 AM on September 6, 2023 [21 favorites]
That said, gaming the review system was always a thing in the movie business. People needing to pay attention to sources and read between the lines was always a thing. Yes, if a movie is at a sort of unclear level (like the low to mid 60's) you have to pay attention to sources. But having to check the web sites on a film with an RT rating of 62% to see how many of them are the likes of TerryLikesMovies.fart as opposed to the NYT or what have you is a pretty low hurdle. And didn't we used to get lobby cards and TV ads for movies that said "A TOUR DE FORCE!" and we had to read the small print to see if the critic saying that was Pauline Kael or some dude at the Idaho Register? At least these days, we can limit reviews to "Top Critics" with a single click, and that's helpful.
The way that movies that are broadly agreed-upon to be kinda good end up with great scores is also known to most serious moviegoers. M3GAN, for instance, got a 93% RT rating, because something like 9 out of 10 reviewers said yeah, kinda good, not because very many critics gave it the equivalent of an A. The recent German horror film Dawn Breaks Behind the Eyes got a 100%, but not one of the reviews considered it a flawless film. It's just that everyone agreed there was at least something to recommend.
Another thing I watch for with RT: theatrically released horror films are often artificially low. This is for the simple reason that many critics forced to review large batches of theatrical releases do not actually like horror films. This is getting to be less of an issue over the last ten years, thanks to generational shift, but it's a legacy problem when you look back at older reviews, with large swaths of boomer reviewers (sorry, that's who they are) giving what amount to automatic pans to anything less than Rosemary's Baby. Many horror fans I have spoken with use an automatic "add 20%" rule on RT for horror movies 10+ years old to get an actual sense of reviews.
As a hilarious example of this, consider the Final Destination film series. The first, often considered a modern horror classic now, has an RT rating of 36%. The second film has a higher RT rating of 50%, even as the consensus is that it was a clear step down from the not so bad original. By the time, they get to the third, it dips to a 43% fresh, with the consensus being that by that point, the series was no longer as classic and genius as it once was. The only film in the series to be rated fresh is the fifth, sitting at 63% fresh, even as the consensus is that by that point, it's a lesser film "only for the gore-thirsty faithful."
You have to click into RT and see who actually said what and try to get a sense of how convincing you find their arguments. But that was always true. I remember having to keep in mind that while I loved Roger Ebert and agreed with him on most things, he had: 1) a total lack of appreciation for unpretentious horror films that delivered the goods; and 2) a weird art boner for immersive CGI worlds... he loved Spawn for chrissakes. It's not a matter of finding critics who always agree with you. It's about finding good writers who are gifted at articulating why they like/dislike a film. I didn't have to agree with Ebert, I just had to know he would explain his take well enough that I'd be able to guess from what he wrote how I might expect to feel, knowing where we tended to agree and where we tended to differ.
It's the same now. As a horror fan, I have to recognize which critics seem to actually take horror for what it is, and which just pinch their nose, hoping for the next it's okay to like this "elevated horror" hit.
I do sort of kind of vaguely share the lament that film critics do not get to be recognized voices and tastemakers in the way that they used to be. But this isn't RT's doing, so much as a sad after effect of the decimation of the newspaper/print media industry.
posted by DirtyOldTown at 7:57 AM on September 6, 2023 [21 favorites]
I want to see your average Housewife of the Middle Ages. That movie would be full of interesting details.
Heh. I'm in the midst of Kristin Lavransdatter right now and it's basically the book form of just that.
(I'ts got a film adaptation, too. Three hours, and only covering the first book of the trilogy. There aren't enough professional reviews for Rotten Tomatoes to give it a statistically significant score, but if it did it'd be a 50%.)
posted by jackbishop at 8:33 AM on September 6, 2023 [18 favorites]
Heh. I'm in the midst of Kristin Lavransdatter right now and it's basically the book form of just that.
(I'ts got a film adaptation, too. Three hours, and only covering the first book of the trilogy. There aren't enough professional reviews for Rotten Tomatoes to give it a statistically significant score, but if it did it'd be a 50%.)
posted by jackbishop at 8:33 AM on September 6, 2023 [18 favorites]
I think one of the biggest problems with the Tomatometer isn't due to anything inherent with its own structure, but what people think it means.
People seem to think that a 100% on the Tomatometer means something is a classic -- maybe it is, but that's not what it really means. What it really means is that 100% of the available reviews were positive.
It's just as likely that everyone agreed it's OK.
People seem to stake a LOT on the Tomatometer mistaking it as a metric of quality. It's a metric of consensus, not quality. A very low score means very few people thought it was at least pretty good.
I've found that the most interesting movies either have a very high Tomatometer score (consensus that it's good), or a middling score (often "rotten" below 60%) because it's a polarizing movie and it evokes a strong response either positive or negative.
posted by tclark at 8:34 AM on September 6, 2023 [7 favorites]
People seem to think that a 100% on the Tomatometer means something is a classic -- maybe it is, but that's not what it really means. What it really means is that 100% of the available reviews were positive.
It's just as likely that everyone agreed it's OK.
People seem to stake a LOT on the Tomatometer mistaking it as a metric of quality. It's a metric of consensus, not quality. A very low score means very few people thought it was at least pretty good.
I've found that the most interesting movies either have a very high Tomatometer score (consensus that it's good), or a middling score (often "rotten" below 60%) because it's a polarizing movie and it evokes a strong response either positive or negative.
posted by tclark at 8:34 AM on September 6, 2023 [7 favorites]
Rotten Tomatoes ratings have always been garbage IMO (maybe not as bad IMDB ratings which are probably the most gamed). I get that it can work for you in a consumer security kind of way but its ubiquity is mostly just due to marketing not for any real authority to the movie valuing experience it conceivable adds. The other main aggregator, Metacritic, is a bit better at capturing review ambivalence and has a better clearer website but it falls into similar pitfalls because assigning numbers to works of art or extruded product is, as others above have pointed out, poison to creative work. Ultimately, neither stands up to word of mouth or the opinions of others whose taste you gel with (that includes movie reviewers and random social media acquaintances) - those are much better weighted in your favour.
Frowner: this makes me want to start reviewing movies
Join Letterboxd.com. You'll be in good company.
posted by Ashwagandha at 8:39 AM on September 6, 2023 [1 favorite]
Frowner: this makes me want to start reviewing movies
Join Letterboxd.com. You'll be in good company.
posted by Ashwagandha at 8:39 AM on September 6, 2023 [1 favorite]
My trouble with RT (even when it's not being gamed as the article describes) is I can't think of the score as a "should I watch this?" metric, unless it's very high and sometimes not even then.
I like good movies: ones where the director and actors are sincere in their work, revealing beauty and fresh perspectives and doing them justice.
I also like bad movies: teen horror flicks, old movies with cheesy acting, crude physical effects and ridiculous plots.
What I don't like is "lazy trying" movies: crude, obvious, superficial attempts to guide the viewer to emotion or laughs or irony in a certain way by pushing certain buttons, and claiming social relevance while still directed by White men upholding establishment narratives. Many of these movies are Oscar-winning (though not usually Cannes-winning), like Driving Miss Daisy, so it appears that Hollywood filmmakers are big fans of this kind of movie that I despise.
posted by splitpeasoup at 8:45 AM on September 6, 2023 [3 favorites]
I like good movies: ones where the director and actors are sincere in their work, revealing beauty and fresh perspectives and doing them justice.
I also like bad movies: teen horror flicks, old movies with cheesy acting, crude physical effects and ridiculous plots.
What I don't like is "lazy trying" movies: crude, obvious, superficial attempts to guide the viewer to emotion or laughs or irony in a certain way by pushing certain buttons, and claiming social relevance while still directed by White men upholding establishment narratives. Many of these movies are Oscar-winning (though not usually Cannes-winning), like Driving Miss Daisy, so it appears that Hollywood filmmakers are big fans of this kind of movie that I despise.
posted by splitpeasoup at 8:45 AM on September 6, 2023 [3 favorites]
I have found that if a film has a score between 65 and 85%, it's probably an interesting movie. It's interesting enough that critics are split. I don't trust anything below 60, and honestly I don't trust anything with a 90 or above. I have found that there seem to be a whole host of movies where it feels like critics are willing to overlook flaws to stay...hip? in good graces? I dunno, I just know I have seen a bunch of films with 90+ scores that weren't very good.
This only works with films where there is a tipping point of critics. Rotten tomatoes (in theory) only works when you have at least 20-30 reviews.
posted by nushustu at 8:45 AM on September 6, 2023 [2 favorites]
This only works with films where there is a tipping point of critics. Rotten tomatoes (in theory) only works when you have at least 20-30 reviews.
posted by nushustu at 8:45 AM on September 6, 2023 [2 favorites]
In February 2016, Rotten Tomatoes and its parent site Flixster were acquired by Fandango (which is owned by Comcast). Warner Bros. retains a minority stake in the merged entities, including Fandango.
And since New York magazine is owned by Penske, hmmm.
posted by Ideefixe at 8:58 AM on September 6, 2023 [2 favorites]
And since New York magazine is owned by Penske, hmmm.
posted by Ideefixe at 8:58 AM on September 6, 2023 [2 favorites]
I've been trying to train myself to ignore Rotten Tomatoes. A movie has a horrible rotten "splat" symbol if only 40% of critics like it, so I'd be likely to skip it--but that still means a lot of people who saw it liked it, right? Especially funny with the something like the Ophelia example: If I'm tempted to see a movie with Daisy Ridley in a feminist Hamlet, should I really care whether 4 out of 10 critics like it, or 6 out of 10 do? I should just see it.
The article was interesting, but as it basically admits the studio attempts to game the system are pretty standard. Movies would stagger openings and screenings all the time. The bigger problem IMO is people like me paying attention to the score and letting it dampen my enthusiasm.
posted by mark k at 9:01 AM on September 6, 2023 [2 favorites]
The article was interesting, but as it basically admits the studio attempts to game the system are pretty standard. Movies would stagger openings and screenings all the time. The bigger problem IMO is people like me paying attention to the score and letting it dampen my enthusiasm.
posted by mark k at 9:01 AM on September 6, 2023 [2 favorites]
The meta-review sites like Rotten Tomatoes have a different problem, averaging numbers. Don't show me an average. Show me a histogram of all review scores. Also normalize each score by the reviewer's average score. Admittedly that part is hard.
I want each person's reviews turned into a percentile of THEIR reviews. If half of their reviews are 9/10 or better, this a 9/10 is a 50% percentile review. Then subtract 50% and double it.
-100% is the worst movie you'll review, +100% is the best movie you'll review, 0% is the median movie you'll review.
We can also normalize by your review standard deviation. Replace 0% by a z-score of 0, and +/- by your review z score compared to your other reviews.
A histogram of that is interesting and a good place to start. It is also pretty.
A downside to that is every bad movie you review in effect inflates the positive reviews you give. So we can do a second pass -- map your z-scores of each movie to the other reviewer's z-scores of the same movie.
That comparison tells you if you are generally more or less critical compared to other reviewers, instead of compared to yourself. We can use that as a bias on your reviews, to better cross-compare you with others.
Finally, we can cluster your opinions with other like minded reviewers, and colour the parts of the z-score histogram based on the dominant "taste" of the reviewer in that region. Is it getting a high z-score from people who like rom-coms? Or from people who like action movies? Depending on which, the z-score histogram has a different colour in that region.
posted by NotAYakk at 9:17 AM on September 6, 2023 [8 favorites]
I want each person's reviews turned into a percentile of THEIR reviews. If half of their reviews are 9/10 or better, this a 9/10 is a 50% percentile review. Then subtract 50% and double it.
-100% is the worst movie you'll review, +100% is the best movie you'll review, 0% is the median movie you'll review.
We can also normalize by your review standard deviation. Replace 0% by a z-score of 0, and +/- by your review z score compared to your other reviews.
A histogram of that is interesting and a good place to start. It is also pretty.
A downside to that is every bad movie you review in effect inflates the positive reviews you give. So we can do a second pass -- map your z-scores of each movie to the other reviewer's z-scores of the same movie.
That comparison tells you if you are generally more or less critical compared to other reviewers, instead of compared to yourself. We can use that as a bias on your reviews, to better cross-compare you with others.
Finally, we can cluster your opinions with other like minded reviewers, and colour the parts of the z-score histogram based on the dominant "taste" of the reviewer in that region. Is it getting a high z-score from people who like rom-coms? Or from people who like action movies? Depending on which, the z-score histogram has a different colour in that region.
posted by NotAYakk at 9:17 AM on September 6, 2023 [8 favorites]
And didn't we used to get lobby cards and TV ads for movies that said "A TOUR DE FORCE!" and we had to read the small print to see if the critic saying that was Pauline Kael or some dude at the Idaho Register?
And even if it was Pauline Kael, it's possible that what she actually said was that it was "a tour de force in mediocrity" or something.
posted by EmpressCallipygos at 9:21 AM on September 6, 2023 [3 favorites]
And even if it was Pauline Kael, it's possible that what she actually said was that it was "a tour de force in mediocrity" or something.
posted by EmpressCallipygos at 9:21 AM on September 6, 2023 [3 favorites]
I've long since given up on review aggregators. No meta score will ever outweigh a review from one or two critics whose tastes are compatible with mine.
Aggregators are just marketing fodder and places to grind culture war axes.
The market: bad actually.
posted by Reyturner at 9:30 AM on September 6, 2023 [2 favorites]
Aggregators are just marketing fodder and places to grind culture war axes.
The market: bad actually.
posted by Reyturner at 9:30 AM on September 6, 2023 [2 favorites]
People seem to think that a 100% on the Tomatometer means something is a classic -- maybe it is, but that's not what it really means. What it really means is that 100% of the available reviews were positive.
It's just as likely that everyone agreed it's OK.
this is the main reason I tuned out of Rotten Tomatoes a long time ago. It champions a kind of nice mediocrity. Not that some of its top rated movies aren't great, but most? Nah. How great can a movie be that doesn't piss at least a few people off, make them uncomfortable at least. This is particularly true of newer movies, those that consciously push some envelope or other.
Older movies tend to have time to settle in our consciousness, time and zeitgeist having moved on. For instance, I notice Blue Velvet has a 95% rating (88% audience score). I can assure you that, at the time, these numbers weren't near this high. In fact, I seem to remember a certain Roger Ebert giving it a thumb's down. Indeed a lot of what drove Blue Velvet to at least break even at the box office was how it became one of the movies people just had to see for themselves.
A healthy culture needs such movies and an outfit like Rotten Tomatoes seems little help in this regard.
posted by philip-random at 9:34 AM on September 6, 2023 [4 favorites]
It's just as likely that everyone agreed it's OK.
this is the main reason I tuned out of Rotten Tomatoes a long time ago. It champions a kind of nice mediocrity. Not that some of its top rated movies aren't great, but most? Nah. How great can a movie be that doesn't piss at least a few people off, make them uncomfortable at least. This is particularly true of newer movies, those that consciously push some envelope or other.
Older movies tend to have time to settle in our consciousness, time and zeitgeist having moved on. For instance, I notice Blue Velvet has a 95% rating (88% audience score). I can assure you that, at the time, these numbers weren't near this high. In fact, I seem to remember a certain Roger Ebert giving it a thumb's down. Indeed a lot of what drove Blue Velvet to at least break even at the box office was how it became one of the movies people just had to see for themselves.
A healthy culture needs such movies and an outfit like Rotten Tomatoes seems little help in this regard.
posted by philip-random at 9:34 AM on September 6, 2023 [4 favorites]
Another thing I watch for with RT: theatrically released horror films are often artificially low. This is for the simple reason that many critics forced to review large batches of theatrical releases do not actually like horror films ...
This is so simple and obvious that I can't believe I didn't figure it out. The most recent horror movie I've missed because it had a bad score was The Last Voyage of the Demeter, and I want to get back to that if I can now. It left the local theater pretty quickly.
I have found that IRL, RT has become an all-purpose way to encourage or discourage choosing a movie on the spot. Whether or not people are very online or have strong film opinions, it means something. But the RT score won't tell you, as a single critic could: "If you like this sort of thing, this is the sort of thing you'll like." That's important sometimes, especially for horror movies. And a bad RT score won't tell you anything about the wildness and weirdness that can result in cult films -- it actively discourages cult films from forming.
jackbishop: I saw that movie -- I had to order the DVD from Netflix -- and it was definitely of the school of Klaus von Boringstein. It wasn't even the whole book. I think there should be a streaming series, but I'm not holding my breath.
posted by Countess Elena at 9:47 AM on September 6, 2023 [2 favorites]
This is so simple and obvious that I can't believe I didn't figure it out. The most recent horror movie I've missed because it had a bad score was The Last Voyage of the Demeter, and I want to get back to that if I can now. It left the local theater pretty quickly.
I have found that IRL, RT has become an all-purpose way to encourage or discourage choosing a movie on the spot. Whether or not people are very online or have strong film opinions, it means something. But the RT score won't tell you, as a single critic could: "If you like this sort of thing, this is the sort of thing you'll like." That's important sometimes, especially for horror movies. And a bad RT score won't tell you anything about the wildness and weirdness that can result in cult films -- it actively discourages cult films from forming.
jackbishop: I saw that movie -- I had to order the DVD from Netflix -- and it was definitely of the school of Klaus von Boringstein. It wasn't even the whole book. I think there should be a streaming series, but I'm not holding my breath.
posted by Countess Elena at 9:47 AM on September 6, 2023 [2 favorites]
“If there was a new film by, I don’t know, Klaus Von Boringstein,” says Schrader, “and he had a three-hour drama about a housewife in the Middle Ages, do you think people would go see it because it had a 90 percent on Rotten Tomatoes? No. But if it were a movie about a serial killer in the wilds of Alaska and it had a 50 percent? They might check that out.”
Years ago I watched a double bill of Jean de Florette and its sequel Manon des Sources which constitute basically a four-hour French-language film of circa-1920 French farmers feuding over water rights and it was fantastic. Is that Von Boringstein enough for Schrader? What is their RT score? The amount I care about that approaches zero from below.
I've seen probably a hundred movies about serial killers and there are perhaps three or four I would rewatch or have rewatched.
Not sure I agree with his premise here.
posted by ricochet biscuit at 10:07 AM on September 6, 2023 [2 favorites]
Years ago I watched a double bill of Jean de Florette and its sequel Manon des Sources which constitute basically a four-hour French-language film of circa-1920 French farmers feuding over water rights and it was fantastic. Is that Von Boringstein enough for Schrader? What is their RT score? The amount I care about that approaches zero from below.
I've seen probably a hundred movies about serial killers and there are perhaps three or four I would rewatch or have rewatched.
Not sure I agree with his premise here.
posted by ricochet biscuit at 10:07 AM on September 6, 2023 [2 favorites]
The most recent horror movie I've missed because it had a bad score was The Last Voyage of the Demeter
See? Forget the aggregators, you just need to rely on DirtyOldTown's Fanfare posts. They won't lead you astray (maybe a little...)
posted by Ashwagandha at 10:38 AM on September 6, 2023 [5 favorites]
See? Forget the aggregators, you just need to rely on DirtyOldTown's Fanfare posts. They won't lead you astray (maybe a little...)
posted by Ashwagandha at 10:38 AM on September 6, 2023 [5 favorites]
Is that Von Boringstein enough for Schrader?
So it might be worth noting, if one is not familiar with Schrader, that Schrader has been Mr. "Boring" Movie for about 50 years, which is why kyrademon made that Oppositeland comment above. He uses the concept of the "scalpel of boredom" (his phrase) in his critical writing to describe how some directors have learned to wield what he calls "dead time" to intentionally slow films down and alter the way the audience perceives the film.
That's why I think RobinofFrocksley is incorrect above in saying he's talking about misogyny. Certainly, there is an element of that affecting viewership, but that "medieval housewives" thing was just an example of the larger point that at some level there is simply a disconnect between commercial and artistic film-making. There are types of films that will just never be widely popular no matter what meta-score they receive. For those "boring" films it's more about being the type of person who would seek them out rather than their score.
posted by star gentle uterus at 10:39 AM on September 6, 2023 [8 favorites]
So it might be worth noting, if one is not familiar with Schrader, that Schrader has been Mr. "Boring" Movie for about 50 years, which is why kyrademon made that Oppositeland comment above. He uses the concept of the "scalpel of boredom" (his phrase) in his critical writing to describe how some directors have learned to wield what he calls "dead time" to intentionally slow films down and alter the way the audience perceives the film.
That's why I think RobinofFrocksley is incorrect above in saying he's talking about misogyny. Certainly, there is an element of that affecting viewership, but that "medieval housewives" thing was just an example of the larger point that at some level there is simply a disconnect between commercial and artistic film-making. There are types of films that will just never be widely popular no matter what meta-score they receive. For those "boring" films it's more about being the type of person who would seek them out rather than their score.
posted by star gentle uterus at 10:39 AM on September 6, 2023 [8 favorites]
The completely opaque process by which Rotten Tomatoes flattens detailed written reviews into a binary "positive/negative".
(That's it. That's the comment.)
posted by mediareport at 10:42 AM on September 6, 2023 [3 favorites]
(That's it. That's the comment.)
posted by mediareport at 10:42 AM on September 6, 2023 [3 favorites]
Also, quoting Brett fucking Ratner about "the destruction of our business" is a hoot and a half. Search his name with the words "shrimp cocktail."
posted by mediareport at 10:46 AM on September 6, 2023
posted by mediareport at 10:46 AM on September 6, 2023
Brett fucking Ratner ...Search his name with the words "shrimp cocktail."
Or better yet don't do that, and just take our word for it that he's a shitbird instead.
posted by DirtyOldTown at 10:57 AM on September 6, 2023 [2 favorites]
Or better yet don't do that, and just take our word for it that he's a shitbird instead.
posted by DirtyOldTown at 10:57 AM on September 6, 2023 [2 favorites]
Many years ago, I was reading a collection of notebook jottings by W.H. Auden. In one, he said that in the "nursery stage" of critical development, a person only has two possible reactions to a literary work: "I liked it" and "I didn't like it."
As people mature, they develop more possible reactions. The four he listed were:
1. This is good, and I like it.
2. This is not good and I don't like it.
3. I don't like this, but I recognize that it is good. I might come to like it in time.
4. I recognize that this is not good, but I like it anyway.
If I set up a review aggregation website, I'd make people choose from a matrix kind of like this. Still lacks nuance, but rejects the idea that there is a single linear measure of how "good" something is.
posted by Well I never at 10:59 AM on September 6, 2023 [25 favorites]
As people mature, they develop more possible reactions. The four he listed were:
1. This is good, and I like it.
2. This is not good and I don't like it.
3. I don't like this, but I recognize that it is good. I might come to like it in time.
4. I recognize that this is not good, but I like it anyway.
If I set up a review aggregation website, I'd make people choose from a matrix kind of like this. Still lacks nuance, but rejects the idea that there is a single linear measure of how "good" something is.
posted by Well I never at 10:59 AM on September 6, 2023 [25 favorites]
MetaFilter: It's absolutely imperfect
posted by chavenet at 11:04 AM on September 6, 2023 [1 favorite]
posted by chavenet at 11:04 AM on September 6, 2023 [1 favorite]
Here's an ungated version of the article
posted by chavenet at 11:05 AM on September 6, 2023 [2 favorites]
posted by chavenet at 11:05 AM on September 6, 2023 [2 favorites]
Still lacks nuance, but rejects the idea that there is a single linear measure of how "good" something is.
Yeah, I hate numeric / linear rating systems. I don't care tons about movie ratings, but music ratings are my jam and I loathe number scores.
I can't stand the 5-star rating thing you're stuck with with music player tools... Obviously my favorite songs and albums are all 5s, but some of them are absolute masterpieces, some are just really really good, and some are meaningful and enjoyable to me but I know it's not in the same universe as a masterpiece. Doesn't mean I won't play it on heavy repeat for a month...
As for the topic of the article: All systems will be gamed for profit if they can be gamed for profit. Of course RT has been gamed. I just wait for word of mouth / things to turn up on streaming these days, or I go in blind. Very few movies suck so badly that I don't enjoy a couple of hours with snacks and air conditioning. If they do I can vote with my feet.
posted by jzb at 11:10 AM on September 6, 2023 [3 favorites]
Yeah, I hate numeric / linear rating systems. I don't care tons about movie ratings, but music ratings are my jam and I loathe number scores.
I can't stand the 5-star rating thing you're stuck with with music player tools... Obviously my favorite songs and albums are all 5s, but some of them are absolute masterpieces, some are just really really good, and some are meaningful and enjoyable to me but I know it's not in the same universe as a masterpiece. Doesn't mean I won't play it on heavy repeat for a month...
As for the topic of the article: All systems will be gamed for profit if they can be gamed for profit. Of course RT has been gamed. I just wait for word of mouth / things to turn up on streaming these days, or I go in blind. Very few movies suck so badly that I don't enjoy a couple of hours with snacks and air conditioning. If they do I can vote with my feet.
posted by jzb at 11:10 AM on September 6, 2023 [3 favorites]
I’m perplexed by the way people are reading the Paul Schrader quote. Regardless of whether you like Schrader and his work, he’s an art house skewing writer and director. The closest he got to making blockbusters was writing Scorsese movies. He’s offering a cynical assessment of the tastes of the public, not criticizing other filmmakers for being boring, and he’s probably more right than not even if you personally love slow arty movies.
posted by atoxyl at 11:59 AM on September 6, 2023 [9 favorites]
posted by atoxyl at 11:59 AM on September 6, 2023 [9 favorites]
So it might be worth noting, if one is not familiar with Schrader, that Schrader has been Mr. "Boring" Movie for about 50 years, which is why kyrademon made that Oppositeland comment above.
I am moderately familiar with him, I suppose; I've seen six or eight of his films, but seemingly nothing since 1999's Bringing Out the Dead. However, the level of irony he is employing here has so overwhelmed my finely-calibrated Gen-X receptors that I could not help but fail to find myself unable to feel unsurprised at his views. Or maybe not; I have lost track of the reversals here.
posted by ricochet biscuit at 12:10 PM on September 6, 2023 [1 favorite]
I am moderately familiar with him, I suppose; I've seen six or eight of his films, but seemingly nothing since 1999's Bringing Out the Dead. However, the level of irony he is employing here has so overwhelmed my finely-calibrated Gen-X receptors that I could not help but fail to find myself unable to feel unsurprised at his views. Or maybe not; I have lost track of the reversals here.
posted by ricochet biscuit at 12:10 PM on September 6, 2023 [1 favorite]
When I cared about such things, I could read a single review in the NYT or Village Voice and divine my estimated wannaseeitness from the review and my knowledge of the reviewer's known biases.
Now I mostly watch free old stuff or DVDs from our local library. If I get ratings, it's from IMDB (1.0-9.0?) where the stink concentrates around 4.0-6.0 -- higher means it's ok at the price ($0 and you can stop anytime) and lower means it's a howl.
The last 2 movies I saw in a theater were Barbie and Avatar. The first Avatar.
I spent most of Avatar wondering how they would get the protagonist and antagonist into a bar for the big film-ending bar fight. But it did have lots of pretty stuff.
Barbie is a wonder. I hope the next film is an even greater storification challenge. Like an amazing tale about a spunky pickle-jar lid.
posted by hexatron at 12:15 PM on September 6, 2023 [2 favorites]
Now I mostly watch free old stuff or DVDs from our local library. If I get ratings, it's from IMDB (1.0-9.0?) where the stink concentrates around 4.0-6.0 -- higher means it's ok at the price ($0 and you can stop anytime) and lower means it's a howl.
The last 2 movies I saw in a theater were Barbie and Avatar. The first Avatar.
I spent most of Avatar wondering how they would get the protagonist and antagonist into a bar for the big film-ending bar fight. But it did have lots of pretty stuff.
Barbie is a wonder. I hope the next film is an even greater storification challenge. Like an amazing tale about a spunky pickle-jar lid.
posted by hexatron at 12:15 PM on September 6, 2023 [2 favorites]
Seconding this. I actually pick things to watch based on a gut-instinct feeling after watching the trailer, and sometimes I get second thoughts and pass (or word of mouth convinces me to pass). I don't actually read reviews until after I've seen something.
Samesies. I have an excellent bad-movie filter, so I rarely watch bad movies unless I choose to. I definitely don't read reviews until after I've seen the movie, especially since they're often recaps more than reviews.
posted by kirkaracha at 12:22 PM on September 6, 2023
Samesies. I have an excellent bad-movie filter, so I rarely watch bad movies unless I choose to. I definitely don't read reviews until after I've seen the movie, especially since they're often recaps more than reviews.
posted by kirkaracha at 12:22 PM on September 6, 2023
> Many years ago, I was reading a collection of notebook jottings by W.H. Auden. In one, he said that in the "nursery stage" of critical development, a person only has two possible reactions to a literary work: "I liked it" and "I didn't like it."
As people mature, they develop more possible reactions.
Been there, done that; I am now enjoying my second childhood.
posted by aws17576 at 12:41 PM on September 6, 2023 [1 favorite]
As people mature, they develop more possible reactions.
Been there, done that; I am now enjoying my second childhood.
posted by aws17576 at 12:41 PM on September 6, 2023 [1 favorite]
I use imperfect heuristics for selecting what movies to watch because I don't want to turn choosing two+ hours of entertainment into a dissertation.
posted by srboisvert at 1:08 PM on September 6, 2023
posted by srboisvert at 1:08 PM on September 6, 2023
There's a review score "controversy" in the offing right now re the game Starfield, as I think someone noted above. Scores are fundamentally reductive, of course, but an average of many scores is remarkably reductive. That said, it's still signal, not noise. It's just a question of figuring out what that signal is.
For me, a high RT score says "this show or film is successful at what it attempts to do," because ultimately that's what a professional critic, ostensibly, aims to evaluate and express in their review.
So when I see a score of 50% on RT, it tells me that many people think that the work does not do itself justice. That's usually not something I enjoy, even if it's otherwise in my wheelhouse.
For example (off the top of my head) that Special Ops: Lioness show has a bad score not because it is a flag-waving military thriller, but because it does not do "flag-waving military thriller" well. Whereas, countless critics watched Top Gun: Maverick and made a point of saying, "flag waving military thriller is not my thing, but this movie is incredibly good at it and you should watch it even if it isn't your thing."
Whether you will like a show is a matter of taste, but whether something is a high quality production that achieves what it sets out to do is more (but not completely) measurable. I treat RT as a signal of the latter, and look at most shows above 70% even if they don't sound like my thing - sometimes it gets me to move outside my comfort zone because people are saying "even if you don't like coming-of-age dramas/romantic comedies/martial arts flicks, get you in front of a TV and watch this."
Anyway, the full opinions of all the reviewers are only a click away, and I avail myself of that resource very frequently as well to see if their particular issues or enjoyments align themselves with my own, though I get that for many that is a click too far.
And let it not go unsaid here that any studio execs or once-was directors blaming RT or whatever for their troubles can take their complaints out "where the fox and the rabbit say good night," as I recall the saying going.
posted by BlackLeotardFront at 1:12 PM on September 6, 2023 [2 favorites]
For me, a high RT score says "this show or film is successful at what it attempts to do," because ultimately that's what a professional critic, ostensibly, aims to evaluate and express in their review.
So when I see a score of 50% on RT, it tells me that many people think that the work does not do itself justice. That's usually not something I enjoy, even if it's otherwise in my wheelhouse.
For example (off the top of my head) that Special Ops: Lioness show has a bad score not because it is a flag-waving military thriller, but because it does not do "flag-waving military thriller" well. Whereas, countless critics watched Top Gun: Maverick and made a point of saying, "flag waving military thriller is not my thing, but this movie is incredibly good at it and you should watch it even if it isn't your thing."
Whether you will like a show is a matter of taste, but whether something is a high quality production that achieves what it sets out to do is more (but not completely) measurable. I treat RT as a signal of the latter, and look at most shows above 70% even if they don't sound like my thing - sometimes it gets me to move outside my comfort zone because people are saying "even if you don't like coming-of-age dramas/romantic comedies/martial arts flicks, get you in front of a TV and watch this."
Anyway, the full opinions of all the reviewers are only a click away, and I avail myself of that resource very frequently as well to see if their particular issues or enjoyments align themselves with my own, though I get that for many that is a click too far.
And let it not go unsaid here that any studio execs or once-was directors blaming RT or whatever for their troubles can take their complaints out "where the fox and the rabbit say good night," as I recall the saying going.
posted by BlackLeotardFront at 1:12 PM on September 6, 2023 [2 favorites]
“No meta score will ever outweigh a review from one or two critics whose tastes are compatible with mine.”
I use the Metacritic movie scores as a first-pass filter, then read two or so reviews from reviewers I trust, as well as scanning some others for possibly relevant information. That works pretty reliably for me.
RT doesn't wok for me because they include so many obscure and unreliable reviewers. Audience scores are almost but not quite inversely reliable for me and the RT score seems closer to the audience scores than the MC scores.
I know this makes me sound like a dick, but I've developed an almost visceral negative response to audience scores because people really do like a lot of shitty stuff. RT's critic scores are just too similar to audience scores to be useful to me and that's why I rely upon Metacritic (for film/tv) as a portion — not all — of my process.
I've used Goodreads for many years, but only to bookmark and track what I read. The ratings and recs are similarly almost unusable for me, unfortunately. I'd love a Metacritic for books which uses a carefully selected roster of high quality reviewers — there is a new site like that, but neither its reviewers nor books are numerous enough to be very helpful.
Of course, if your tastes are much closer to the popular median, then the audience scores which are unreliable for me should work well for you. If that's the case, though, then why not just use RT and IMDBs audience scores, and skip the professional reviewers entirely?
And, yeah, the discussion above about more mature aesthetic judgements that allow for "it's good but I didn't like it" and "it's bad but I liked it" is really the key dividing line between how the two biggest audience groups think of all this. Far too many people merely equate their enjoyment with quality for their judgements to be of any use to me.
posted by Ivan Fyodorovich at 1:49 PM on September 6, 2023 [4 favorites]
I use the Metacritic movie scores as a first-pass filter, then read two or so reviews from reviewers I trust, as well as scanning some others for possibly relevant information. That works pretty reliably for me.
RT doesn't wok for me because they include so many obscure and unreliable reviewers. Audience scores are almost but not quite inversely reliable for me and the RT score seems closer to the audience scores than the MC scores.
I know this makes me sound like a dick, but I've developed an almost visceral negative response to audience scores because people really do like a lot of shitty stuff. RT's critic scores are just too similar to audience scores to be useful to me and that's why I rely upon Metacritic (for film/tv) as a portion — not all — of my process.
I've used Goodreads for many years, but only to bookmark and track what I read. The ratings and recs are similarly almost unusable for me, unfortunately. I'd love a Metacritic for books which uses a carefully selected roster of high quality reviewers — there is a new site like that, but neither its reviewers nor books are numerous enough to be very helpful.
Of course, if your tastes are much closer to the popular median, then the audience scores which are unreliable for me should work well for you. If that's the case, though, then why not just use RT and IMDBs audience scores, and skip the professional reviewers entirely?
And, yeah, the discussion above about more mature aesthetic judgements that allow for "it's good but I didn't like it" and "it's bad but I liked it" is really the key dividing line between how the two biggest audience groups think of all this. Far too many people merely equate their enjoyment with quality for their judgements to be of any use to me.
posted by Ivan Fyodorovich at 1:49 PM on September 6, 2023 [4 favorites]
RT has a structural vulnerability for bad movies that critics still feel compelled to say "go anyway" - the "Certified Fresh" The Last Jedi being a classic example. Metacritic captures that ambivalence well.
I'm not sure what you're trying to say here. The Last Jedi is currently ranked second second among the Star Wars movies on Metacritic, and fourth by the Tomatometer. TLJ was the target of an inorganic review-bombing campaign (by the Usual Suspects), which is why it has a 42% audience score, versus 85% for The Force Awakens and 86% (!!!) for The Rise of Skywalker. But you seem to be saying that TLJ is ranked artificially high on RT?
posted by The Tensor at 2:52 PM on September 6, 2023 [6 favorites]
I'm not sure what you're trying to say here. The Last Jedi is currently ranked second second among the Star Wars movies on Metacritic, and fourth by the Tomatometer. TLJ was the target of an inorganic review-bombing campaign (by the Usual Suspects), which is why it has a 42% audience score, versus 85% for The Force Awakens and 86% (!!!) for The Rise of Skywalker. But you seem to be saying that TLJ is ranked artificially high on RT?
posted by The Tensor at 2:52 PM on September 6, 2023 [6 favorites]
Every reviewer should switch to the Flophouse system of “Good Bad Movie,” “Bad Bad Movie,” or “Actually Good Movie”
posted by DoctorFedora at 3:01 PM on September 6, 2023 [5 favorites]
posted by DoctorFedora at 3:01 PM on September 6, 2023 [5 favorites]
Goodreads has the usual problem that everything is rated about 3.8 stars. The useful signal on Goodreads is how many reviews a book has. If I'm approaching a new author or genre it's a fantastic filter. "Which book by Iain Banks is the one I should read first?" Often it's the one with the most reviews, not the one with the best score.
That won't work for aggregators of movie reviews from professionals. It might work for movies with Letterboxd, though.
posted by Nelson at 3:02 PM on September 6, 2023 [3 favorites]
That won't work for aggregators of movie reviews from professionals. It might work for movies with Letterboxd, though.
posted by Nelson at 3:02 PM on September 6, 2023 [3 favorites]
90% of everything on Letterboxd sometimes seems to be between 2.4-3.5/5.if you see anything with a few dozen reviews that averages out below that, it's almost certainly garbage. A substantial number of reviews and it's above that, take note. Could be very good.
(One caveat to the above is that films from India operate at about 0.5 stars higher, say, 2.9-3.9 as a default range.)
posted by DirtyOldTown at 3:40 PM on September 6, 2023
(One caveat to the above is that films from India operate at about 0.5 stars higher, say, 2.9-3.9 as a default range.)
posted by DirtyOldTown at 3:40 PM on September 6, 2023
I don’t really pay attention to reviews as I like to form my own opinion, but a lot of people I know and/or work with often ask me for recommendations. Seeing as I don’t know too much about them I find asking their opinions on Cast Away and Crazy Rich Asians. That usually clues me into their tastes so I can make a recommendation or two. I can’t possibly see how a RT score would give me any indication if someone would like that movie.
posted by JakeEXTREME at 3:44 PM on September 6, 2023 [1 favorite]
posted by JakeEXTREME at 3:44 PM on September 6, 2023 [1 favorite]
I really hope Schrader is right because the next True Detective series is Jodie Foster wandering around Alaska looking for serial killers, and I might be the only one looking forward to it.
posted by betweenthebars at 4:13 PM on September 6, 2023 [3 favorites]
posted by betweenthebars at 4:13 PM on September 6, 2023 [3 favorites]
The upside to RT’s dumb system is that sometimes you’re expecting a movie to be awful and you get the pleasant experience of it actually being *interesting.*
Earlier in the year I watched two movies in short succession. Both had ~45% on RT. One was The 355, which — at the very least — was *not nothing.* I love a pleasant surprise! The other I legitimately do not even remember.
posted by TangoCharlie at 4:44 PM on September 6, 2023
Earlier in the year I watched two movies in short succession. Both had ~45% on RT. One was The 355, which — at the very least — was *not nothing.* I love a pleasant surprise! The other I legitimately do not even remember.
posted by TangoCharlie at 4:44 PM on September 6, 2023
an amazing tale about a spunky pickle-jar lid
a beguiling AU Sausage Party (2016) [82%, certified fresh, 239 reviews; audience score of 50%, from 50K+ ratings]
posted by Iris Gambol at 5:12 PM on September 6, 2023
a beguiling AU Sausage Party (2016) [82%, certified fresh, 239 reviews; audience score of 50%, from 50K+ ratings]
posted by Iris Gambol at 5:12 PM on September 6, 2023
I chuckled when I saw Schrader quoted in the article because his filmography perfectly embodies the concept of 'interesting failure'* which is a type of movie that's pretty much impossible to represent well in a numerical score.
And "...three-hour drama about a housewife..." is probably a veiled reference to the amazing Jeanne Dielman, 23 quai du Commerce, 1080 Bruxelles, which topped the 2022 Sight and Sound poll as the greatest film of all time and which Shrader publicly complained about in a very assholish way.
*He's also made a handful of very good movies and several flat-out shitty ones.
posted by theory at 5:38 PM on September 6, 2023 [8 favorites]
And "...three-hour drama about a housewife..." is probably a veiled reference to the amazing Jeanne Dielman, 23 quai du Commerce, 1080 Bruxelles, which topped the 2022 Sight and Sound poll as the greatest film of all time and which Shrader publicly complained about in a very assholish way.
*He's also made a handful of very good movies and several flat-out shitty ones.
posted by theory at 5:38 PM on September 6, 2023 [8 favorites]
I will say that I did have another crisis of faith regarding Rotten Tomatoes after watching the Japanese animated feature film Belle, which was pretty to look at but was narratively a mess and had an ending that was offensively bad, but it got a standing ovation at Cannes and also it had like a 95% score on Rotten Tomatoes, but admittedly I think that this was also an example of Japanese animated feature films being graded on a curve
posted by DoctorFedora at 6:51 PM on September 6, 2023
posted by DoctorFedora at 6:51 PM on September 6, 2023
Rotten Tomatoes isn't a reliable metric for movie quality.
But name the thing that is more reliable.
posted by AlSweigart at 7:17 PM on September 6, 2023
But name the thing that is more reliable.
posted by AlSweigart at 7:17 PM on September 6, 2023
there is no thing. For me, it's a complex mix of a pile of things, including my network (the people whose taste I trust), my own sense of what looks promising/interesting, the literally thousands of reviews I have read over the years (since I was ten or eleven years old), and, yeah, learning how to read a trailer, and for that matter, how to read between the lines of a review (good or bad). I''m not always right. I still get surprised ... but it doesn't happen that often.
Most recent positive surprise would've been Cocaine Bear. As dumb as it sounds and even better. Most recent negative surprise ... ? Nothing leaps quickly mind. I suppose because one of my current tactics is to give pretty much anything that pops up on streaming a chance, but not a very long one. If it hasn't grabbed me by the ten or twelve minute point, it's unlikely I'm going to see it through. This is proving far more useful than any review site I'm aware of.
posted by philip-random at 8:37 PM on September 6, 2023
Most recent positive surprise would've been Cocaine Bear. As dumb as it sounds and even better. Most recent negative surprise ... ? Nothing leaps quickly mind. I suppose because one of my current tactics is to give pretty much anything that pops up on streaming a chance, but not a very long one. If it hasn't grabbed me by the ten or twelve minute point, it's unlikely I'm going to see it through. This is proving far more useful than any review site I'm aware of.
posted by philip-random at 8:37 PM on September 6, 2023
But name the thing that is more reliable.
Back when I was more engaged I would absolutely put any one critic I liked--or even something like the old AV Club--above a Rotten Tomatoes score.
Rotten Tomatoes is like polling people at work on where to go to lunch and trying to make sure no one's miserable. Every lunch will be things like a pizza place that's not too expensive and doesn't take too long. Which is fine, but not the same as getting a friend to go into detail on the best things to order at their favorite places.
posted by mark k at 9:19 PM on September 6, 2023 [6 favorites]
Back when I was more engaged I would absolutely put any one critic I liked--or even something like the old AV Club--above a Rotten Tomatoes score.
Rotten Tomatoes is like polling people at work on where to go to lunch and trying to make sure no one's miserable. Every lunch will be things like a pizza place that's not too expensive and doesn't take too long. Which is fine, but not the same as getting a friend to go into detail on the best things to order at their favorite places.
posted by mark k at 9:19 PM on September 6, 2023 [6 favorites]
I've used Goodreads for many years, but only to bookmark and track what I read. The ratings and recs are similarly almost unusable for me, unfortunately.
On Goodreads in particular, I find I get a lot more out of reading the one- and two-star reviews than the positive ones. For example, with sci-fi, if the low-star reviews are full of people calling it “pretentious” and “confusing” there’s a good chance I’ll love it. On the flip side (especially with recently published stuff), sometimes I’ll see a book with tons of rave reviews that aren’t particularly specific, and then the low-star reviews have very thorough takedowns complaining about exactly the kinds of things I always hate in books.
(I need to stop doing this for books I already intend to read, though (like book club books). No sense deciding what I’ll probably hate about the thing before I even read it!)
posted by showbiz_liz at 12:19 AM on September 7, 2023 [4 favorites]
On Goodreads in particular, I find I get a lot more out of reading the one- and two-star reviews than the positive ones. For example, with sci-fi, if the low-star reviews are full of people calling it “pretentious” and “confusing” there’s a good chance I’ll love it. On the flip side (especially with recently published stuff), sometimes I’ll see a book with tons of rave reviews that aren’t particularly specific, and then the low-star reviews have very thorough takedowns complaining about exactly the kinds of things I always hate in books.
(I need to stop doing this for books I already intend to read, though (like book club books). No sense deciding what I’ll probably hate about the thing before I even read it!)
posted by showbiz_liz at 12:19 AM on September 7, 2023 [4 favorites]
Metacritic has been poison in the video game industry.
Surprised no one has mentioned the most famous example of this, which is Fallout New Vegas. Quite literally everyone working on it was screwed out of money because of its metacritic score. FNV is of course today known as one of the greatest videogames of all time. But worker bonuses were associated to the score, and of course the capitalists rushed to not pay anyone anything because a website had a number.
posted by Pyrogenesis at 5:57 AM on September 7, 2023 [4 favorites]
Surprised no one has mentioned the most famous example of this, which is Fallout New Vegas. Quite literally everyone working on it was screwed out of money because of its metacritic score. FNV is of course today known as one of the greatest videogames of all time. But worker bonuses were associated to the score, and of course the capitalists rushed to not pay anyone anything because a website had a number.
posted by Pyrogenesis at 5:57 AM on September 7, 2023 [4 favorites]
Even if you fall into the camp where a person should do the work and read the reviews to decide what to see, not just use an aggregator score... Is there a better tool for finding reviews than RT? I don't think that there is.
I'm actually prone to looking up RT after seeing a movie, because it's useful for me to find the reviewers whose opinions were quite a bit like mine, especially when my take is at odds with the consensus.
posted by DirtyOldTown at 6:56 AM on September 7, 2023 [2 favorites]
I'm actually prone to looking up RT after seeing a movie, because it's useful for me to find the reviewers whose opinions were quite a bit like mine, especially when my take is at odds with the consensus.
posted by DirtyOldTown at 6:56 AM on September 7, 2023 [2 favorites]
90% of everything on Letterboxd sometimes seems to be between 2.4-3.5/5.if you see anything with a few dozen reviews that averages out below that, it's almost certainly garbage. A substantial number of reviews and it's above that, take note. Could be very good.
I think is generally true for most mainstream films with reviews by regular people. However, anything international or with poor distribution? They will have less reviews, ratings, and views (See this article in the Ringer to give an idea of what I'm talking about). That has no bearing on actual quality. Numerical ratings have too many variables and caveats to be entirely reliable regardless of where or how they are obtained. I counter that in a couple ways. So in the example with Letterboxd, if you read an individual review and look at that user's previous watched films, their favourite films, their other reviews you can easily determine whether an individual review has any value to you - someone who rated something as a 5 but has watched only 3 films this year and their favourites are 4 films you hate, their value as a reviewer to me is negligible. So I tend to use the ratings of people I know - for instance a friend of mine uses the Letterboxd star rating system in an idiosyncratic way:
5 stars - Excellent. Ne Plus Ultra. Challenging and fulfilling piece of art.
4 stars - Competently made, distinctive and interesting.
3 and half stars - I don't like this, but I recognize its inherent competency. I might come to like it in time with more context.
3 stars - [This is their lowest rating] Mediocre at best. This is not good and I don't like it.
2 stars - Junk and maybe boring but in the right context could be fun.
1 star - A disasterpiece! Not well made but transcends its crappiness to be a singular work of art.
So for them the only really bad score is 3 to 3 and half stars. However, I know that they don't like French films so scoring a French film at 3 doesn't necessarily indicate I would dislike it. However, I know our tastes coincide with things they list as 4 and 1 so those are often good bets. So in this example a more idiosyncratic review is more useful to me over a Rotten Tomato or Metacritic consensus number.
Another more reliable way to gauge how I might take a film can be determined, on Letterboxd at least but in the real world as well, is to follow people who I know consume a lot of movies and may be film creatives themselves. It is especially helpful if their tastes coincide with yours. Varied tastes though can give you a perspective you wouldn't consider.
Context is always important in any valuation as is the source of the valuation.
posted by Ashwagandha at 8:05 AM on September 7, 2023
I think is generally true for most mainstream films with reviews by regular people. However, anything international or with poor distribution? They will have less reviews, ratings, and views (See this article in the Ringer to give an idea of what I'm talking about). That has no bearing on actual quality. Numerical ratings have too many variables and caveats to be entirely reliable regardless of where or how they are obtained. I counter that in a couple ways. So in the example with Letterboxd, if you read an individual review and look at that user's previous watched films, their favourite films, their other reviews you can easily determine whether an individual review has any value to you - someone who rated something as a 5 but has watched only 3 films this year and their favourites are 4 films you hate, their value as a reviewer to me is negligible. So I tend to use the ratings of people I know - for instance a friend of mine uses the Letterboxd star rating system in an idiosyncratic way:
5 stars - Excellent. Ne Plus Ultra. Challenging and fulfilling piece of art.
4 stars - Competently made, distinctive and interesting.
3 and half stars - I don't like this, but I recognize its inherent competency. I might come to like it in time with more context.
3 stars - [This is their lowest rating] Mediocre at best. This is not good and I don't like it.
2 stars - Junk and maybe boring but in the right context could be fun.
1 star - A disasterpiece! Not well made but transcends its crappiness to be a singular work of art.
So for them the only really bad score is 3 to 3 and half stars. However, I know that they don't like French films so scoring a French film at 3 doesn't necessarily indicate I would dislike it. However, I know our tastes coincide with things they list as 4 and 1 so those are often good bets. So in this example a more idiosyncratic review is more useful to me over a Rotten Tomato or Metacritic consensus number.
Another more reliable way to gauge how I might take a film can be determined, on Letterboxd at least but in the real world as well, is to follow people who I know consume a lot of movies and may be film creatives themselves. It is especially helpful if their tastes coincide with yours. Varied tastes though can give you a perspective you wouldn't consider.
Context is always important in any valuation as is the source of the valuation.
posted by Ashwagandha at 8:05 AM on September 7, 2023
Huh, I didn't know the story about Fallout: New Vegas getting an 84 Metacritic score and thus Obsidian (the developer) losing out--by one point!--on a bonus from Bethesda. So that's why I didn't bring it up Pyrogenesis! To add insult to injury, a lot of the low reviews were due to bugs, which were at least in part Bethesda's bailiwick (since the did QC).
As the link says, Obsidian (despite making great games) would continue to struggle and lay off a big chunk of people soon thereafter. Kickstarter eventually saved them by letting them self-fund and publish their own game.
posted by mark k at 8:36 AM on September 7, 2023
As the link says, Obsidian (despite making great games) would continue to struggle and lay off a big chunk of people soon thereafter. Kickstarter eventually saved them by letting them self-fund and publish their own game.
posted by mark k at 8:36 AM on September 7, 2023
I guess we are back to finding a few critics that one trusts and ignoring everything else.
We need a new Siskel and Ebert. I told tell if I was or wasn't going to like a movie by the way they described.
posted by KaizenSoze at 9:43 AM on September 7, 2023 [1 favorite]
We need a new Siskel and Ebert. I told tell if I was or wasn't going to like a movie by the way they described.
posted by KaizenSoze at 9:43 AM on September 7, 2023 [1 favorite]
1. This is good, and I like it.
2. This is not good and I don't like it.
3. I don't like this, but I recognize that it is good. I might come to like it in time.
4. I recognize that this is not good, but I like it anyway
Metacognition critic
5) I recognize that this is not good, but I like it anyway, but I don't like that I like it
posted by justsomebodythatyouusedtoknow at 1:05 PM on September 7, 2023 [7 favorites]
2. This is not good and I don't like it.
3. I don't like this, but I recognize that it is good. I might come to like it in time.
4. I recognize that this is not good, but I like it anyway
Metacognition critic
5) I recognize that this is not good, but I like it anyway, but I don't like that I like it
posted by justsomebodythatyouusedtoknow at 1:05 PM on September 7, 2023 [7 favorites]
After the Vulture article, RT has disappeared the movie Ophelia from its listings.
posted by DirtyOldTown at 1:20 PM on September 7, 2023 [2 favorites]
posted by DirtyOldTown at 1:20 PM on September 7, 2023 [2 favorites]
I find Metacritic much more helpful for movies than for games; movie reviewers more or less agree on what they think their scale means - 3 stars being "this is worth seeing", 2 stars being "I don't recommend this", 4 stars being "I recommend this" - which means that Metacritic can capture some of the nuance of interesting but divisive films vs mediocre pablum. Metacritic for games is garbage.
posted by Merus at 7:09 AM on September 9, 2023 [1 favorite]
posted by Merus at 7:09 AM on September 9, 2023 [1 favorite]
We need a new Siskel and Ebert.
You can make your own! I hate to advertise since it'll be enshittified some day, but join Letterboxd and start following people whose reviews you like. It works well!
posted by rhizome at 3:31 PM on September 9, 2023
You can make your own! I hate to advertise since it'll be enshittified some day, but join Letterboxd and start following people whose reviews you like. It works well!
posted by rhizome at 3:31 PM on September 9, 2023
Dan Murrell has a response to this that deals with fundamental flaws with the Vulture piece.
- The Ophelia stuff is an example corruption by a PR firm to change outcomes.
- RT isn't really at fault for that, and probably shouldn't have removed the movie from their website.
- RT is not a particularly great metric, and it's scores are massively misused by the industry.
- Vulture cherrypicked data to blame it on RT widening its pool of critics some years ago (outside of the legacy print media and into more diverse voices) to show how that's the real problem with RT, and that legacy media like Vulture is more trustworthy than online critics.
End result: Vulture did great on the Ophelia scoop, but anything else about RT more broadly is quite a reach and compromised by their own ax to grind.
posted by Lord Chancellor at 8:49 AM on September 13, 2023
- The Ophelia stuff is an example corruption by a PR firm to change outcomes.
- RT isn't really at fault for that, and probably shouldn't have removed the movie from their website.
- RT is not a particularly great metric, and it's scores are massively misused by the industry.
- Vulture cherrypicked data to blame it on RT widening its pool of critics some years ago (outside of the legacy print media and into more diverse voices) to show how that's the real problem with RT, and that legacy media like Vulture is more trustworthy than online critics.
End result: Vulture did great on the Ophelia scoop, but anything else about RT more broadly is quite a reach and compromised by their own ax to grind.
posted by Lord Chancellor at 8:49 AM on September 13, 2023
« Older “That monster will never forgive us.” | Their Helicopter Newer »
This thread has been archived and is closed to new comments
posted by kyrademon at 6:16 AM on September 6, 2023 [4 favorites]