"We didn’t just get unlucky: We made a big mistake..."
May 18, 2016 6:34 AM Subscribe
How I Acted Like A Pundit And Screwed Up On Donald Trump, by Nate Silver "...along with a couple of marginal ones." Data journalist Nate Silver soul-searches and course-corrects while defending data journalism. "Basically, my view is that putting Trump’s chances at 2 percent or 5 percent was too low, but having him at (for instance) 10 percent or 15 percent, where we might have wound up if we’d developed a model or thought about the problem more rigorously, would have been entirely appropriate. If you care about that sort of distinction, you’ve come to the right website!"
This long piece comes in the wake of, as Silver puts it, "a lot of critical self-assessments from data-minded journalists" at 538, The New York Times, RealClearPolitics, Bloomberg, and others, along with plenty of crowing from the other crowd ("...politics is an essentially human endeavor and therefore can defy prediction and reason").
All of the above are referenced in Silver's piece, which compiles a wealth of recent articles revolving around Trump and data journalism. Other views include Sam Wang of the Princeton Election Consortium, who nearly called the race for Hillary and Trump back in January, and like Silver draws a distinction between data journalism and data punditry:
In contrast, websites like FiveThirtyEight and the New York Times are under pressure to create interest and suspense, even when the outcome is not in doubt. The result is wrong statements like how much Indiana mattered and how Cruz had a chance there. Both statements were false. But we expect our data pundits to be better than cable and television news media, which are polluted with such statements. It is a disappointment when they play the clickbait game.
This long piece comes in the wake of, as Silver puts it, "a lot of critical self-assessments from data-minded journalists" at 538, The New York Times, RealClearPolitics, Bloomberg, and others, along with plenty of crowing from the other crowd ("...politics is an essentially human endeavor and therefore can defy prediction and reason").
All of the above are referenced in Silver's piece, which compiles a wealth of recent articles revolving around Trump and data journalism. Other views include Sam Wang of the Princeton Election Consortium, who nearly called the race for Hillary and Trump back in January, and like Silver draws a distinction between data journalism and data punditry:
In contrast, websites like FiveThirtyEight and the New York Times are under pressure to create interest and suspense, even when the outcome is not in doubt. The result is wrong statements like how much Indiana mattered and how Cruz had a chance there. Both statements were false. But we expect our data pundits to be better than cable and television news media, which are polluted with such statements. It is a disappointment when they play the clickbait game.
Silver's going to get kicked out of the political pundit club. The first rule is that you never ever admit that you got anything wrong.
posted by octothorpe at 6:42 AM on May 18, 2016 [27 favorites]
posted by octothorpe at 6:42 AM on May 18, 2016 [27 favorites]
In the UK, after opinion polls failed to predict a right-wing victory: General election opinion poll failure down to not reaching Tory voters.
posted by alasdair at 6:49 AM on May 18, 2016
posted by alasdair at 6:49 AM on May 18, 2016
Our fictional pundit predicted more correct primary results than Nate Silver did.
posted by logicpunk at 6:53 AM on May 18, 2016 [10 favorites]
posted by logicpunk at 6:53 AM on May 18, 2016 [10 favorites]
What it's like casting a ballot in a state that takes voting seriously, Rolling Stone reports on Oregon's voter friendly vote-by-mail system.
posted by chrchr at 6:56 AM on May 18, 2016 [5 favorites]
posted by chrchr at 6:56 AM on May 18, 2016 [5 favorites]
I forget where I saw this quote (maybe in Jacobin somewhere?), but it went something like:
"Statistics is the idea that you can predict future election results from previous ones. Politics is the idea that you can't."
posted by Noisy Pink Bubbles at 6:57 AM on May 18, 2016 [16 favorites]
"Statistics is the idea that you can predict future election results from previous ones. Politics is the idea that you can't."
posted by Noisy Pink Bubbles at 6:57 AM on May 18, 2016 [16 favorites]
Silver's going to get kicked out of the political pundit club. The first rule is that you never ever admit that you got anything wrong.
Here's hoping, since he was much better outside of it. I need clear and accurate predictions of how terrified I should be.
posted by Artw at 6:57 AM on May 18, 2016 [32 favorites]
Here's hoping, since he was much better outside of it. I need clear and accurate predictions of how terrified I should be.
posted by Artw at 6:57 AM on May 18, 2016 [32 favorites]
Our fictional pundit predicted more correct primary results than Nate Silver did.
Half of me is like "Virgil give me a break you got lucky and your methods are just guess-work, don't be anti-intellectual." and the other half is like "TELL ME WHO WILL WIN THE GENERAL CARL YOU MANIAC WIZARD!"
posted by Potomac Avenue at 7:02 AM on May 18, 2016 [5 favorites]
Half of me is like "Virgil give me a break you got lucky and your methods are just guess-work, don't be anti-intellectual." and the other half is like "TELL ME WHO WILL WIN THE GENERAL CARL YOU MANIAC WIZARD!"
posted by Potomac Avenue at 7:02 AM on May 18, 2016 [5 favorites]
I need clear and accurate predictions of how terrified I should be.
Extremely.
Everybody Sucks 2016!
On a personal note: I just watched my birth country (Philippines) elect a strongman so I'm already quite terrified for family back there (even though half of them voted for the guy).
posted by linux at 7:04 AM on May 18, 2016 [3 favorites]
Extremely.
Everybody Sucks 2016!
On a personal note: I just watched my birth country (Philippines) elect a strongman so I'm already quite terrified for family back there (even though half of them voted for the guy).
posted by linux at 7:04 AM on May 18, 2016 [3 favorites]
His approach was frankly nutty.
His work in 2012 was awesome -- he rigorously focused on swing state polls and cross-tabs therein and wouldn't be spun by horse-race rooting journalists or Romney team spin/self-delusion.
But in 2015 and early 2016 he looked at Trump, with a hugely bigger and more consistent lead than most Republican nominees, to say the least of any Republican who failed to be nominated, in modern history, and assumed the WSJ editorial board and Jeb Bush's Jedi powers were going to overcome that.
posted by MattD at 7:07 AM on May 18, 2016 [10 favorites]
His work in 2012 was awesome -- he rigorously focused on swing state polls and cross-tabs therein and wouldn't be spun by horse-race rooting journalists or Romney team spin/self-delusion.
But in 2015 and early 2016 he looked at Trump, with a hugely bigger and more consistent lead than most Republican nominees, to say the least of any Republican who failed to be nominated, in modern history, and assumed the WSJ editorial board and Jeb Bush's Jedi powers were going to overcome that.
posted by MattD at 7:07 AM on May 18, 2016 [10 favorites]
I think the difference between that tweet Silver posted and the article he wrote is telling. His article is all about modeling the past performance of various candidates when he actually should have been modeling the voters.
posted by spudsilo at 7:08 AM on May 18, 2016 [7 favorites]
posted by spudsilo at 7:08 AM on May 18, 2016 [7 favorites]
Sam Wang on
posted by zamboni at 7:10 AM on May 18, 2016 [24 favorites]
The Party Decides:
Now that we know better what is happening this year, I propose to replace Noel’s catchphrase with the following syllogism framework. Call it “Successful Parties Decide”:Now that's something I'd be interested in reading about.There’s your story – the Republican Party is broken. It probably broke slowly, from 1994 to 2014. Data geeks, write about how that happened!
- Functioning political parties are in the business of deciding: evaluating candidates, helping pick who will rise to the top.
- This year, the national Republican Party didn’t decide.
- Therefore, the national Republican Party is not a functioning political party.
posted by zamboni at 7:10 AM on May 18, 2016 [24 favorites]
When you start making guesses based on stuff you believe in, you've gone pretty far from any sort of statistical analysis. The biggest mistake was sacrificing the rigor they've been trying to cultivate to get a few more pageviews. It's pretty easy to go back and say "oh sure we should have sliced the pie like THIS instead" but that doesn't really mean much now or much for future races. It's still going to be guessing and the results will be just as bad.
posted by lubujackson at 7:12 AM on May 18, 2016 [1 favorite]
posted by lubujackson at 7:12 AM on May 18, 2016 [1 favorite]
His article is all about modeling the past performance of various candidates when he actually should have been modeling the voters.
I don't see how you can possibly do the latter without also doing the former.
posted by howfar at 7:15 AM on May 18, 2016 [1 favorite]
I don't see how you can possibly do the latter without also doing the former.
posted by howfar at 7:15 AM on May 18, 2016 [1 favorite]
I stood up for Nate Silver in Slate this week. If he'd said 10-15% instead of 5%, Silver's detractors would still be crowing. I think they would be wrong to crow, and I think they're wrong to crow now.
One thing I didn't get to in that interview, though, is that at a much later stage of the primary, Wang saw Trump as having a clear path to a majority of delgates, while Silver still had Trump coming in at around 1150. That's the place where I think Wang's methodology might actually have been better than Silver's, and I'd love to see a breakdown of how they came to different conclusions from the same data.
posted by escabeche at 7:15 AM on May 18, 2016 [14 favorites]
One thing I didn't get to in that interview, though, is that at a much later stage of the primary, Wang saw Trump as having a clear path to a majority of delgates, while Silver still had Trump coming in at around 1150. That's the place where I think Wang's methodology might actually have been better than Silver's, and I'd love to see a breakdown of how they came to different conclusions from the same data.
posted by escabeche at 7:15 AM on May 18, 2016 [14 favorites]
sitting down to read it right now--a preliminary thought: if you assign some event a 5% probability of happening and then it happens, it doesn't mean you were wrong. in fact, you were right, because you predicted the outcome. the thing happened. again, this is without RTFA.
silvers has a good track record.
posted by Zerowensboring at 7:19 AM on May 18, 2016 [3 favorites]
silvers has a good track record.
posted by Zerowensboring at 7:19 AM on May 18, 2016 [3 favorites]
What I find aggravating about this is that my social media feed is now going to be full of doomsaying progressives who will take any poll showing Trump behind as evidence that we simply have no idea how much he's secretly winning.
We're probably a few months out from reliable data on the state of the general election. You probably want both national conventions over, and enough polls in In-Play states to create a rolling average, so real numbers are not likely until August. But at a certain point, you either believe that it's possible to make political predictions that are more accurate than a coin flip, or you don't. If you don't, then polls cannot be meaningful, and if you do, then you need to consider bad predictions in light of how their model produced bad results, and what might be done to correct that.
posted by GameDesignerBen at 7:20 AM on May 18, 2016 [6 favorites]
We're probably a few months out from reliable data on the state of the general election. You probably want both national conventions over, and enough polls in In-Play states to create a rolling average, so real numbers are not likely until August. But at a certain point, you either believe that it's possible to make political predictions that are more accurate than a coin flip, or you don't. If you don't, then polls cannot be meaningful, and if you do, then you need to consider bad predictions in light of how their model produced bad results, and what might be done to correct that.
posted by GameDesignerBen at 7:20 AM on May 18, 2016 [6 favorites]
This election cycle has completely destroyed my faith in 538 as a predictor - not just the Trump predictions, but the Sanders ones too.
Either Silver lost his touch, or he never had one to begin with (given so many people trying to do statistical forecasting, even people with wrong models will be perfectly right some of the time...)
posted by lupus_yonderboy at 7:21 AM on May 18, 2016 [1 favorite]
Either Silver lost his touch, or he never had one to begin with (given so many people trying to do statistical forecasting, even people with wrong models will be perfectly right some of the time...)
posted by lupus_yonderboy at 7:21 AM on May 18, 2016 [1 favorite]
I don't see how you can possibly do the latter without also doing the former.
Watch this. Silver uses forecasting versus predictive modeling, so he missed the change in voters, the uptick in voter registration for a certain kind of voter, etc.
posted by linux at 7:22 AM on May 18, 2016 [1 favorite]
Watch this. Silver uses forecasting versus predictive modeling, so he missed the change in voters, the uptick in voter registration for a certain kind of voter, etc.
posted by linux at 7:22 AM on May 18, 2016 [1 favorite]
> if you assign some event a 5% probability of happening and then it happens, it doesn't mean you were wrong.
Sure, no individual failure disproves you. But if you keep predicting a 5% chance of rain, and it keeps raining, at some point we're going to switch channels to a different weatherman.(*)
---
(* - yeah, I know, the weatherman doesn't actually make the forecasts, it's a metaphor, OK?)
posted by lupus_yonderboy at 7:22 AM on May 18, 2016 [3 favorites]
Sure, no individual failure disproves you. But if you keep predicting a 5% chance of rain, and it keeps raining, at some point we're going to switch channels to a different weatherman.(*)
---
(* - yeah, I know, the weatherman doesn't actually make the forecasts, it's a metaphor, OK?)
posted by lupus_yonderboy at 7:22 AM on May 18, 2016 [3 favorites]
What I find aggravating about this is that my social media feed is now going to be full of doomsaying progressives who will take any poll showing Trump behind as evidence that we simply have no idea how much he's secretly winning.
I don't think this is aggravating at all if it scares people enough to GOTV.
posted by roomthreeseventeen at 7:22 AM on May 18, 2016 [5 favorites]
I don't think this is aggravating at all if it scares people enough to GOTV.
posted by roomthreeseventeen at 7:22 AM on May 18, 2016 [5 favorites]
“Successful Parties Decide”
The Democrats seem to do a pretty good job of deciding and then losing (ie. John Kerry, Al Gore, and every state/local election the DNC takes an interest in).
I don't think the party chose in 2008. Clinton v McCain in 2008 would have been a very different election, and I'm not sure who would have won.
IMO, both parties are doing a rather poor job of producing candidates that appeal to their voters and are also electable in the general election.
posted by schmod at 7:23 AM on May 18, 2016 [2 favorites]
The Democrats seem to do a pretty good job of deciding and then losing (ie. John Kerry, Al Gore, and every state/local election the DNC takes an interest in).
I don't think the party chose in 2008. Clinton v McCain in 2008 would have been a very different election, and I'm not sure who would have won.
IMO, both parties are doing a rather poor job of producing candidates that appeal to their voters and are also electable in the general election.
posted by schmod at 7:23 AM on May 18, 2016 [2 favorites]
Our fictional pundit predicted more correct primary results than Nate Silver did.
TBH, I've never really understood what the point of creating a fake pundit was. I can't tell if it's a symptom of the anti-intellectual movement, schadenfreude that someone who was praised is being taken down a peg, or simply an attempt to convince people that polling is suspect. Or perhaps it's something else?
Silver's success in '08 and '12 wasn't wizardry. Most states choose red or blue pretty reliably and it's not hard to guess whether or not they're going to go for one candidate or another. Only a small number were swing states, so he only had to take an aggregate of polls and other factors and make an informed guess in a few races. Guess right by paying close attention to the details, and you'd do well. Guess wrong or underestimate a candidate's appeal (as he's done this cycle) and he won't. In the past, he's guessed well. But it's important to realize that a successful prediction doesn't guarantee he'll be able to do so in the future. Informed or not, there are too many variables to model to 100% accuracy.
He's had (I believe) an over 90% success rate this year despite that but most of those have been in races where the outcome was never going to be in question. Guessing that Clinton was going to win New York or Sanders would win Vermont, for examples, were pretty safe bets.
This time around, Silver spent too much time dismissing dark horse candidates and ignoring evidence they had popular appeal -- and that clearly biased his results long term. He wasn't the only one who did so. But in an election cycle where so many people are taking pride in upending the status quo and proving the "elites" wrong, some hay will be made, I guess.
posted by zarq at 7:24 AM on May 18, 2016 [3 favorites]
TBH, I've never really understood what the point of creating a fake pundit was. I can't tell if it's a symptom of the anti-intellectual movement, schadenfreude that someone who was praised is being taken down a peg, or simply an attempt to convince people that polling is suspect. Or perhaps it's something else?
Silver's success in '08 and '12 wasn't wizardry. Most states choose red or blue pretty reliably and it's not hard to guess whether or not they're going to go for one candidate or another. Only a small number were swing states, so he only had to take an aggregate of polls and other factors and make an informed guess in a few races. Guess right by paying close attention to the details, and you'd do well. Guess wrong or underestimate a candidate's appeal (as he's done this cycle) and he won't. In the past, he's guessed well. But it's important to realize that a successful prediction doesn't guarantee he'll be able to do so in the future. Informed or not, there are too many variables to model to 100% accuracy.
He's had (I believe) an over 90% success rate this year despite that but most of those have been in races where the outcome was never going to be in question. Guessing that Clinton was going to win New York or Sanders would win Vermont, for examples, were pretty safe bets.
This time around, Silver spent too much time dismissing dark horse candidates and ignoring evidence they had popular appeal -- and that clearly biased his results long term. He wasn't the only one who did so. But in an election cycle where so many people are taking pride in upending the status quo and proving the "elites" wrong, some hay will be made, I guess.
posted by zarq at 7:24 AM on May 18, 2016 [3 favorites]
TBH, I've never really understood what the point of creating a fake pundit was.
Carl Diggler has been the only part of this election cycle that hasn't made me want to burn the nation down and salt the earth.
posted by griphus at 7:26 AM on May 18, 2016 [8 favorites]
Carl Diggler has been the only part of this election cycle that hasn't made me want to burn the nation down and salt the earth.
posted by griphus at 7:26 AM on May 18, 2016 [8 favorites]
In other words, this is Shakespearean Tragedy and you can't predict that shit.
posted by Slarty Bartfast at 7:29 AM on May 18, 2016 [6 favorites]
posted by Slarty Bartfast at 7:29 AM on May 18, 2016 [6 favorites]
But if you keep predicting a 5% chance of rain, and it keeps raining, at some point we're going to switch channels to a different weatherman.
And well you should--which is why i think it's important to bear in mind that Silvers (and his team) are "right" about 90% of the time. if trump's win is giving silvers impetus to improve his models, great. but i hope he doesn't overcorrect out of some institutional need for public contrition. no one can predict a black swan. i'm still pro-Nate Silvers.
okay, reading it now...
posted by Zerowensboring at 7:30 AM on May 18, 2016
And well you should--which is why i think it's important to bear in mind that Silvers (and his team) are "right" about 90% of the time. if trump's win is giving silvers impetus to improve his models, great. but i hope he doesn't overcorrect out of some institutional need for public contrition. no one can predict a black swan. i'm still pro-Nate Silvers.
okay, reading it now...
posted by Zerowensboring at 7:30 AM on May 18, 2016
Now that's something I'd be interested in reading about.
Every time I read something by Sam Wang and something by Nate Silver in close succession, I get the very clear feeling that Sam Wang is a much sharper "data journalist" than Nate Silver.
posted by mediareport at 7:32 AM on May 18, 2016 [9 favorites]
Every time I read something by Sam Wang and something by Nate Silver in close succession, I get the very clear feeling that Sam Wang is a much sharper "data journalist" than Nate Silver.
posted by mediareport at 7:32 AM on May 18, 2016 [9 favorites]
Carl Diggler has been the only part of this election cycle that hasn't made me want to burn the nation down and salt the earth.
Okay. Why?
posted by zarq at 7:34 AM on May 18, 2016
Okay. Why?
posted by zarq at 7:34 AM on May 18, 2016
Trump led in the polls all along, from shortly after the moment he descended the elevator at Trump Tower in June...
Escalator.
Tsk, tsk, Nate.
posted by rokusan at 7:38 AM on May 18, 2016 [3 favorites]
Escalator.
Tsk, tsk, Nate.
posted by rokusan at 7:38 AM on May 18, 2016 [3 favorites]
This election cycle has completely destroyed my faith in 538 as a predictor - not just the Trump predictions, but the Sanders ones too.
Yeah, they didn't do a great job, but they're admitting that. I don't frequent their site, but I listen to their political podcasts, and they've been really humble over the last month or so in assessing what they got wrong. More important than their candor is them going into detail as to how this is going to change how they do things in the future.
That shows an assload more integrity than most political coverage.
posted by furnace.heart at 7:39 AM on May 18, 2016 [30 favorites]
Yeah, they didn't do a great job, but they're admitting that. I don't frequent their site, but I listen to their political podcasts, and they've been really humble over the last month or so in assessing what they got wrong. More important than their candor is them going into detail as to how this is going to change how they do things in the future.
That shows an assload more integrity than most political coverage.
posted by furnace.heart at 7:39 AM on May 18, 2016 [30 favorites]
Okay. Why?
For some reason, every single election cycle (which is currently like 2+ years??) the media puts anyone with half a thought about it up on the media stage as a pundit, and claims they have some sort of insight into the process. They don't, they never did, and they're doing little more than just Having Opinions amplified by the same dumbass media process that got Trump to where he currently is.
Carl Diggler is no different than any of them but is at least written by people who understand humor.
posted by griphus at 7:39 AM on May 18, 2016 [10 favorites]
For some reason, every single election cycle (which is currently like 2+ years??) the media puts anyone with half a thought about it up on the media stage as a pundit, and claims they have some sort of insight into the process. They don't, they never did, and they're doing little more than just Having Opinions amplified by the same dumbass media process that got Trump to where he currently is.
Carl Diggler is no different than any of them but is at least written by people who understand humor.
posted by griphus at 7:39 AM on May 18, 2016 [10 favorites]
Be sure to click through zamboni's link to Sam Wang's piece above, Did data journalism lose – or just data pundits?, which includes the following highly accurate paragraph:
FiveThirtyEight committed to a wrong path with theTrump’s Six Stages of Doom theory last July. In the following months, it has been uncomfortable to watch this statement get walked back so gradually and grudgingly. It appeared to be a case of motivated reasoning, a cognitive process in which evidence, however persuasive, is more likely to be rejected if it is disagreeable. And there is no doubt that they find Trump to be disagreeable. But that is not why readers turn to them.
posted by mediareport at 7:40 AM on May 18, 2016 [2 favorites]
FiveThirtyEight committed to a wrong path with theTrump’s Six Stages of Doom theory last July. In the following months, it has been uncomfortable to watch this statement get walked back so gradually and grudgingly. It appeared to be a case of motivated reasoning, a cognitive process in which evidence, however persuasive, is more likely to be rejected if it is disagreeable. And there is no doubt that they find Trump to be disagreeable. But that is not why readers turn to them.
posted by mediareport at 7:40 AM on May 18, 2016 [2 favorites]
The biggest mistake was sacrificing the rigor they've been trying to cultivate to get a few more pageviews.
I don't see how this earned them any extra pageviews, though, because Silver was saying what most pundits were saying--that at some point Trump would implode and his chances of actually getting the nomination were slim. It seems like he would have had just as many (or more) pageviews by going against the tide and saying "According to the data, Trump has a much better chance than anyone thinks."
posted by Pater Aletheias at 7:40 AM on May 18, 2016 [2 favorites]
I don't see how this earned them any extra pageviews, though, because Silver was saying what most pundits were saying--that at some point Trump would implode and his chances of actually getting the nomination were slim. It seems like he would have had just as many (or more) pageviews by going against the tide and saying "According to the data, Trump has a much better chance than anyone thinks."
posted by Pater Aletheias at 7:40 AM on May 18, 2016 [2 favorites]
Pack journalism works, Pater.
posted by mediareport at 7:41 AM on May 18, 2016
posted by mediareport at 7:41 AM on May 18, 2016
Virgil Texas says:
If it seems I’m being too hard on Silver now, that’s because I am. But we should all feel bamboozled. If the quants had not ignored Trump’s soaring popularity all last year, perhaps the GOP establishment would not have sat on their hands as he waltzed to the nomination. And if the same pundits had not been writing Sanders’s obituary before any votes were cast, perhaps that race would be even closer. Maybe a more subjective form of analysis, such as going out and listening to voters, would have understood their passions better than the data journalists’ models.
No, no, no. Polls are a way of listening to voters. How much more willfully can you ignore the stated purpose of election forecasting? Is the critique that 538 should stop issuing forecast-based predictions because "perhaps" "maybe" some people are uncritical consumers of statistical models?
The idea that you can "maybe" more accurately predict an election by talking to a few voters instead of thousands is as respectable a claim as intelligent design, and derived from the same logical principles.
posted by radicalawyer at 7:47 AM on May 18, 2016 [3 favorites]
If it seems I’m being too hard on Silver now, that’s because I am. But we should all feel bamboozled. If the quants had not ignored Trump’s soaring popularity all last year, perhaps the GOP establishment would not have sat on their hands as he waltzed to the nomination. And if the same pundits had not been writing Sanders’s obituary before any votes were cast, perhaps that race would be even closer. Maybe a more subjective form of analysis, such as going out and listening to voters, would have understood their passions better than the data journalists’ models.
No, no, no. Polls are a way of listening to voters. How much more willfully can you ignore the stated purpose of election forecasting? Is the critique that 538 should stop issuing forecast-based predictions because "perhaps" "maybe" some people are uncritical consumers of statistical models?
The idea that you can "maybe" more accurately predict an election by talking to a few voters instead of thousands is as respectable a claim as intelligent design, and derived from the same logical principles.
posted by radicalawyer at 7:47 AM on May 18, 2016 [3 favorites]
Gee guys, someone should ask Stewart and Colbert what the point of creating fake cable news shows is for
posted by Apocryphon at 7:48 AM on May 18, 2016 [6 favorites]
posted by Apocryphon at 7:48 AM on May 18, 2016 [6 favorites]
Be sure to click through zamboni's link to Sam Wang's piece above, Did data journalism lose – or just data pundits?
I just relinked it from the original post- it's the
posted by zamboni at 7:48 AM on May 18, 2016 [1 favorite]
I just relinked it from the original post- it's the
draws a distinction between data journalism and data punditrylink.
posted by zamboni at 7:48 AM on May 18, 2016 [1 favorite]
If he'd said 10-15% instead of 5%, Silver's detractors would still be crowing. I think they would be wrong to crow, and I think they're wrong to crow now.
"I was proved fucking right."
posted by ennui.bz at 8:05 AM on May 18, 2016
"I was proved fucking right."
posted by ennui.bz at 8:05 AM on May 18, 2016
The bit about Rubio is more telling imo. They kept coming up with multitude of reasons and cherry picked stats to keep pushing him. Every single time they went out of their way to discount polls and instead tried convoluted reasoning to affirm their predictions.
posted by asra at 8:06 AM on May 18, 2016 [6 favorites]
posted by asra at 8:06 AM on May 18, 2016 [6 favorites]
I don't see how any statistical or other analysis can assign a value to Trump as anything other than a legitimate "outlier". There just is no data for a non-politician dark horse candidate catching the zeitgeist of a dissatisfied electorate.
posted by sammyo at 8:06 AM on May 18, 2016 [2 favorites]
posted by sammyo at 8:06 AM on May 18, 2016 [2 favorites]
For some reason, every single election cycle (which is currently like 2+ years??) the media puts anyone with half a thought about it up on the media stage as a pundit, and claim they have some sort of insight into the process. They don't, they never did, and they're doing little more than just Having Opinions amplified by the same dumbass media process that got Trump to where he currently is.
My impression is that this is true for television media -- those outlets that breathlessly live and die by the idea that everything that happens in the news is a ratings-pulling "game changer." But I do not think it is true for print journalism as a general rule -- and perhaps for certain digital media outlets -- typically those who are not overly dependent on attracting eyeballs and clicks.
For one thing, print political journalists -- especially those who cover state politics -- often bring decades of experience and knowledge to their work. They tend to be far more aware that change in politics is not something that happens overnight, and that a focus on smaller regional stories and (how those stories relate to a bigger picture) is what matters to their readers. Ultimately, they're invested (or should be) in the idea that the deeper context matters -- which is something we don't get from our television news unless we're watching shows like Frontline.
Anyway, there are a lot of journalists who have quite literally been focused on politics for decades who are covering this election cycle at local levels. Follow CJR. They tend to highlight great journalism. Follow people like Molly Ball, Jon Ralston, Craig Gilbert, etc. Ron Fournier is biased, but he's still insightful. Or, browse this great list. I think it's worth the effort.
posted by zarq at 8:10 AM on May 18, 2016 [1 favorite]
My impression is that this is true for television media -- those outlets that breathlessly live and die by the idea that everything that happens in the news is a ratings-pulling "game changer." But I do not think it is true for print journalism as a general rule -- and perhaps for certain digital media outlets -- typically those who are not overly dependent on attracting eyeballs and clicks.
For one thing, print political journalists -- especially those who cover state politics -- often bring decades of experience and knowledge to their work. They tend to be far more aware that change in politics is not something that happens overnight, and that a focus on smaller regional stories and (how those stories relate to a bigger picture) is what matters to their readers. Ultimately, they're invested (or should be) in the idea that the deeper context matters -- which is something we don't get from our television news unless we're watching shows like Frontline.
Anyway, there are a lot of journalists who have quite literally been focused on politics for decades who are covering this election cycle at local levels. Follow CJR. They tend to highlight great journalism. Follow people like Molly Ball, Jon Ralston, Craig Gilbert, etc. Ron Fournier is biased, but he's still insightful. Or, browse this great list. I think it's worth the effort.
posted by zarq at 8:10 AM on May 18, 2016 [1 favorite]
Ralston is truly terrible though.
posted by roomthreeseventeen at 8:12 AM on May 18, 2016 [2 favorites]
posted by roomthreeseventeen at 8:12 AM on May 18, 2016 [2 favorites]
I don't think this is aggravating at all if it scares people enough to GOTV.
While I understand this line of thinking, I don't think fear is a great motivator in this situation because, again, the argument is "If it looks like Trump is losing, that means the polls are wrong and he's secretly winning." Successful GOTV efforts could never improve that situation. Instead it feels more like a way to create an artificial bandwagon effect for the Trump campaign.
To be more specific, if general election polls show Trump behind, I strongly expect Trump will take every opportunity point out how the primary season polls didn't know how to find his supporters, and that any poll that shows he's losing is actually a sign that he's winning.
The doomsayers in my social media feed who make the argument that you can't trust polls don't seem very interested in explaining why you can't trust polls, or why future polls will show the same inaccuracies (despite polling and data analysis agencies having every incentive to correct their models when they give inaccurate results). Mostly, they seem interested in spreading a fatalist sense that the Democrats are going to drop the ball (because the Democrats are the party of ball-droppers) and Trump is going to serve 4-6 terms as a sort of orangey Sun King Hitler.
posted by GameDesignerBen at 8:15 AM on May 18, 2016
While I understand this line of thinking, I don't think fear is a great motivator in this situation because, again, the argument is "If it looks like Trump is losing, that means the polls are wrong and he's secretly winning." Successful GOTV efforts could never improve that situation. Instead it feels more like a way to create an artificial bandwagon effect for the Trump campaign.
To be more specific, if general election polls show Trump behind, I strongly expect Trump will take every opportunity point out how the primary season polls didn't know how to find his supporters, and that any poll that shows he's losing is actually a sign that he's winning.
The doomsayers in my social media feed who make the argument that you can't trust polls don't seem very interested in explaining why you can't trust polls, or why future polls will show the same inaccuracies (despite polling and data analysis agencies having every incentive to correct their models when they give inaccurate results). Mostly, they seem interested in spreading a fatalist sense that the Democrats are going to drop the ball (because the Democrats are the party of ball-droppers) and Trump is going to serve 4-6 terms as a sort of orangey Sun King Hitler.
posted by GameDesignerBen at 8:15 AM on May 18, 2016
Ralston is truly terrible though.
I disagree. Ralston's been covering local politics in Las Vegas and Nevada for decades. He's relatively liberal in his outlook, yet has still gone toe-to-toe with both Republican and Democratic politicians for years. He is particularly knowledgeable with regard to Senate Democratic Leader Harry Reid, so his columns can help us understand the man, and in a wider sense: DNC priorities, funding and political choices.
posted by zarq at 8:19 AM on May 18, 2016 [2 favorites]
I disagree. Ralston's been covering local politics in Las Vegas and Nevada for decades. He's relatively liberal in his outlook, yet has still gone toe-to-toe with both Republican and Democratic politicians for years. He is particularly knowledgeable with regard to Senate Democratic Leader Harry Reid, so his columns can help us understand the man, and in a wider sense: DNC priorities, funding and political choices.
posted by zarq at 8:19 AM on May 18, 2016 [2 favorites]
Every time I read something by Sam Wang and something by Nate Silver in close succession, I get the very clear feeling that Sam Wang is a much sharper "data journalist" than Nate Silver.
The problem is that the whole "data journalist" thing is the ultimate end-point of the US media culture of treating politics as a "horse race," by hiring someone like Silver, who is literally a sports bettor, and pretending his betting strategy is some objective approach to politics. It's American political "science" culture meets reality TV and totally noxious.
The statistical reality is that "polls" are predictively meaningless without some "model." Where do models come from?Hunches... They come from pretending that the whole political process, which depend on continual "interventions" from the bottom to top, are some objective event. An election is the kind of experiment where the scientific lab is constantly interfering to get their preferred outcomes, where the most important member of the "lab" is the news media and the mass media in general.
posted by ennui.bz at 8:24 AM on May 18, 2016 [8 favorites]
The problem is that the whole "data journalist" thing is the ultimate end-point of the US media culture of treating politics as a "horse race," by hiring someone like Silver, who is literally a sports bettor, and pretending his betting strategy is some objective approach to politics. It's American political "science" culture meets reality TV and totally noxious.
The statistical reality is that "polls" are predictively meaningless without some "model." Where do models come from?
posted by ennui.bz at 8:24 AM on May 18, 2016 [8 favorites]
Escalator.
Tsk, tsk, Nate.
In the abstract, an escalator is a sort of elevator. In fact, in it's original patent, it was called an "Endless Conveyor or Elevator."
Thus proving again my theory of internet pedantry.
posted by maxsparber at 8:25 AM on May 18, 2016 [3 favorites]
Tsk, tsk, Nate.
In the abstract, an escalator is a sort of elevator. In fact, in it's original patent, it was called an "Endless Conveyor or Elevator."
Thus proving again my theory of internet pedantry.
posted by maxsparber at 8:25 AM on May 18, 2016 [3 favorites]
The biggest mistake was sacrificing the rigor they've been trying to cultivate to get a few more pageviews.
See also: Vox.
posted by schmod at 8:28 AM on May 18, 2016 [1 favorite]
See also: Vox.
posted by schmod at 8:28 AM on May 18, 2016 [1 favorite]
In fact, in it's original patent, it was called
Actually, that description was in the patent application; referring to the patent as the text of the application rather than the license given by the government in response to that application is a misunderstanding of the sorry I just, I, I, something welled up inside me and
posted by cortex at 8:28 AM on May 18, 2016 [24 favorites]
Actually, that description was in the patent application; referring to the patent as the text of the application rather than the license given by the government in response to that application is a misunderstanding of the sorry I just, I, I, something welled up inside me and
posted by cortex at 8:28 AM on May 18, 2016 [24 favorites]
Silver may have screwed the pooch, but I am batting a thousand.
By the way, I loved hearing Silver and company go off on the Times in their podcast this week. It all seemed very petty and demonstrates that there is an unspoken competition between traditional punditry and so-called data-driven journalism, and that the Times was so badly burned by Silver and co in the last election that they could not wait to rub their noses in it when they failed, and it reminds me that journalists and human too. And, based on my own experiences as a journalist, barely that.
posted by maxsparber at 8:29 AM on May 18, 2016 [3 favorites]
By the way, I loved hearing Silver and company go off on the Times in their podcast this week. It all seemed very petty and demonstrates that there is an unspoken competition between traditional punditry and so-called data-driven journalism, and that the Times was so badly burned by Silver and co in the last election that they could not wait to rub their noses in it when they failed, and it reminds me that journalists and human too. And, based on my own experiences as a journalist, barely that.
posted by maxsparber at 8:29 AM on May 18, 2016 [3 favorites]
Actually, that description was in the patent application;
Well played, you magnificent bastard.
posted by maxsparber at 8:29 AM on May 18, 2016 [4 favorites]
Well played, you magnificent bastard.
posted by maxsparber at 8:29 AM on May 18, 2016 [4 favorites]
Anyway, there are a lot of journalists who have quite literally been focused on politics for decades who are covering this election cycle at local levels. Follow CJR. They tend to highlight great journalism. Follow people like Molly Ball, Jon Ralston, Craig Gilbert, etc. Ron Fournier is biased, but he's still insightful. Or, browse this great list. I think it's worth the effort.
Counterpoint: Ron Fournier is exactly the kind of press-addled idiot Carl Diggler was invented to mock, a man whose wisdom is so conventional it barely counts as knowledge and whose predictions often turn out wrong. Recommending him in defense of print punditry, even with a caveat, is like defending a castle by raising the portcullis and unbarring the doors.
posted by Rustic Etruscan at 8:33 AM on May 18, 2016 [7 favorites]
Counterpoint: Ron Fournier is exactly the kind of press-addled idiot Carl Diggler was invented to mock, a man whose wisdom is so conventional it barely counts as knowledge and whose predictions often turn out wrong. Recommending him in defense of print punditry, even with a caveat, is like defending a castle by raising the portcullis and unbarring the doors.
posted by Rustic Etruscan at 8:33 AM on May 18, 2016 [7 favorites]
My biggest disappointment with 538 hasn't been that they got Trump wrong, it's that they've gotten caught up on reporting the usual day-to-day minutia. Live blogging the debates, morning after recaps, reacting to whatever the latest thing in the news cycle is. What I want from data journalism is less of a focus on these superficial events and a broader, deeper look at what is actually happening in the world. If the electorate has "lost it's mind", there ought to be a way to quantify that, and by doing so, things like the rise of Trump should not be such a surprise.
posted by spudsilo at 8:33 AM on May 18, 2016 [5 favorites]
posted by spudsilo at 8:33 AM on May 18, 2016 [5 favorites]
"Trump led in the polls all along, from shortly after the moment he descended the elevator at Trump Tower in June..."
>>Escalator.
Tsk, tsk, Nate.
Nice catch, rokusan.
I am hugely looking forward to reading this -because as a resident alien (on Long Island) who was anxious to sound informed about Trump to friends abroad, I started picking up early on just how grotesquely & consistently wrong most of the polls were. By quoting them, I ended up sounding like a total idiot.
As for figuring out the zeitgeist - seriously, try riding the 'patent' escalators at Trump Tower!
I did it last week (taking a nephew around the city on his first trip to Manhattan) and the whole, famously kitsch, brass and glossy pink marble atrium - with its uber 1980s water feature - a 60ft indoor waterfall -absolutely reeks of entrenched mold.
I've been in the Trump Tower atrium several times over the years, & I've clocked an unpleasant mildew smell around the waterfall before. But this is the worst the Trump stench has been.
posted by Jody Tresidder at 8:36 AM on May 18, 2016 [4 favorites]
>>Escalator.
Tsk, tsk, Nate.
Nice catch, rokusan.
I am hugely looking forward to reading this -because as a resident alien (on Long Island) who was anxious to sound informed about Trump to friends abroad, I started picking up early on just how grotesquely & consistently wrong most of the polls were. By quoting them, I ended up sounding like a total idiot.
As for figuring out the zeitgeist - seriously, try riding the 'patent' escalators at Trump Tower!
I did it last week (taking a nephew around the city on his first trip to Manhattan) and the whole, famously kitsch, brass and glossy pink marble atrium - with its uber 1980s water feature - a 60ft indoor waterfall -absolutely reeks of entrenched mold.
I've been in the Trump Tower atrium several times over the years, & I've clocked an unpleasant mildew smell around the waterfall before. But this is the worst the Trump stench has been.
posted by Jody Tresidder at 8:36 AM on May 18, 2016 [4 favorites]
there is an unspoken competition between traditional punditry and so-called data-driven journalism, and that the Times was so badly burned by Silver and co in the last election that they could not wait to rub their noses in it when they failed...
what? 538 was being published by the nytimes in the last election. afterwards, there was a competition between ESPN and the nytimes to bid for free-agent Silver, which ESPN won. If 538 were still under the nytimes banner, I seriously doubt Virgil Texas's Diggler-cry-of-victory-piece would have been published by the Post ie. no one at the Washpost would want to screw up the chance to move up to the Times by being mean...
posted by ennui.bz at 8:36 AM on May 18, 2016
what? 538 was being published by the nytimes in the last election. afterwards, there was a competition between ESPN and the nytimes to bid for free-agent Silver, which ESPN won. If 538 were still under the nytimes banner, I seriously doubt Virgil Texas's Diggler-cry-of-victory-piece would have been published by the Post ie. no one at the Washpost would want to screw up the chance to move up to the Times by being mean...
posted by ennui.bz at 8:36 AM on May 18, 2016
Obviously, Nate didn't call me. If he had, I would have set him straight about Trump. Lost opportunity there, Silver.
posted by Thorzdad at 8:37 AM on May 18, 2016
posted by Thorzdad at 8:37 AM on May 18, 2016
what? 538 was being published by the nytimes in the last election.
Yep. And they go into detail what a fraught releationship that was on the podcast, how much of a cold shoulder they got from the other writers, and how angry other pundits were when their predictions didn't line up with their own, including complaints they received that the predictions were angering the mainstream pundits sources, who were apparently campaign managers for losing campaigns.
posted by maxsparber at 8:42 AM on May 18, 2016 [7 favorites]
Yep. And they go into detail what a fraught releationship that was on the podcast, how much of a cold shoulder they got from the other writers, and how angry other pundits were when their predictions didn't line up with their own, including complaints they received that the predictions were angering the mainstream pundits sources, who were apparently campaign managers for losing campaigns.
posted by maxsparber at 8:42 AM on May 18, 2016 [7 favorites]
max, that was probably my favorite moment on any 538 podcast I've heard. And then to hear this Virgil Texas guy make the same print-journo argument as a critique, after 538 devotes a huge article to acting as its own public editor? Ugh.
Don't get me wrong; I like hearing what rando voters are thinking as much as the next schadenfreude addict. But I don't assume that a given rando is indicative of anything other than what exactly one rando is thinking.
posted by radicalawyer at 8:49 AM on May 18, 2016 [2 favorites]
Don't get me wrong; I like hearing what rando voters are thinking as much as the next schadenfreude addict. But I don't assume that a given rando is indicative of anything other than what exactly one rando is thinking.
posted by radicalawyer at 8:49 AM on May 18, 2016 [2 favorites]
Counterpoint: Ron Fournier is exactly the kind of press-addled idiot Carl Diggler was invented to mock, a man whose wisdom is so conventional it barely counts as knowledge and whose predictions often turn out wrong. Recommending him in defense of print punditry, even with a caveat, is like defending a castle by raising the portcullis and unbarring the doors.
Well, that's certainly... vivid... imagery.
I've been happy with his focus on certain topics lately. Flint. Autism and Asperger's. Differential pay for the disabled. Trump and GOP fearmongering, and the outsized media attention Trump's been attracting by playing the system. We routinely attack the media for not concentrating on what matters. It seems counterproductive not to at least pay attention when they do.
posted by zarq at 8:50 AM on May 18, 2016 [1 favorite]
Well, that's certainly... vivid... imagery.
I've been happy with his focus on certain topics lately. Flint. Autism and Asperger's. Differential pay for the disabled. Trump and GOP fearmongering, and the outsized media attention Trump's been attracting by playing the system. We routinely attack the media for not concentrating on what matters. It seems counterproductive not to at least pay attention when they do.
posted by zarq at 8:50 AM on May 18, 2016 [1 favorite]
And they go into detail what a fraught releationship that was on the podcast, how much of a cold shoulder they got from the other writers, and how angry other pundits were when their predictions didn't line up with their own, including complaints they received that the predictions were angering the mainstream pundits sources, who were apparently campaign managers for losing campaigns.
God forbid I should defend the nytimes, but if they had come up with the right dollar amount ahead of ESPN, I guarantee Silver would not be trash-talking the company in the media. There's nothing different about what the "data journalist" does vs. the pundit except for the "science" branding. Models or "priors" are "hunches" promoted to seem objective. Which is the whole point of the Diggler project.
Just like product marketing, political marketing is 9/10s about creating the market you want to sell your products in not objectively analysing some a priori "true" market.
posted by ennui.bz at 8:59 AM on May 18, 2016 [2 favorites]
God forbid I should defend the nytimes, but if they had come up with the right dollar amount ahead of ESPN, I guarantee Silver would not be trash-talking the company in the media. There's nothing different about what the "data journalist" does vs. the pundit except for the "science" branding. Models or "priors" are "hunches" promoted to seem objective. Which is the whole point of the Diggler project.
Just like product marketing, political marketing is 9/10s about creating the market you want to sell your products in not objectively analysing some a priori "true" market.
posted by ennui.bz at 8:59 AM on May 18, 2016 [2 favorites]
The #MBET's polls-plus-memes-minus-polls model is still the only one I trust.
posted by ethansr at 9:01 AM on May 18, 2016 [1 favorite]
posted by ethansr at 9:01 AM on May 18, 2016 [1 favorite]
I guarantee Silver would not be trash-talking the company in the media
Possibly. I take no side in this. I just like to hear them dish.
posted by maxsparber at 9:03 AM on May 18, 2016
Possibly. I take no side in this. I just like to hear them dish.
posted by maxsparber at 9:03 AM on May 18, 2016
I just sort of assume that the problem with Nate Silver is that the savvy politicos are targeting his methodology now. Once they know how the benchmarks are generated, they game them.
posted by srboisvert at 9:06 AM on May 18, 2016
posted by srboisvert at 9:06 AM on May 18, 2016
zarq: TBH, I've never really understood what the point of creating a fake pundit was. I can't tell if it's a symptom of the anti-intellectual movement, schadenfreude that someone who was praised is being taken down a peg, or simply an attempt to convince people that polling is suspect. Or perhaps it's something else?
Satire. It was for the same reason the Onion creates fake news articles. Or I guess more on the point, the same reason Stephen Colbert created his in-show character. To offer humorous commentary on current events but simultaneous meta-commentary on the people who publish about them in real outlets.
Carl Diggler was one of several satirical characters created by the same authors (I believe) for Cafe.com well before the primaries started. There's Professor Jeff Wilhelm, a self-aggrandizing blowhard professor at a military college who never seems to get anything right and can't keep his dick in his pants, clearly modeled after John Schindler (or at least his Twitter presence). There's Marshall Harford III, a kind of clueless trust fund anarchist Millennial who I believe was inspired by Malcolm Harris. And finally there's The Dig himself, who I'm not sure is supposed to represent any one pundit in particular but kind of a general parody of a certain Politico-friendly, op-ed publishing type - the bio at the start of his posts describes him as Cafe's "Chief Beltway Insider Hack," which is as good a description as any.
These guys, and a lot of the people in their social circle on Twitter, spend a lot of their time on harsh criticism of the political media. (Biederman is also one of the co-hosts of the Chapo Trap House podcast, which is equal parts smart commentary on current events and relentless shit talking of people in the media. It's fantastic, but probably not up the average MeFite's alley) Throughout the debates, Diggler published stupid debate scorecards, because that's what real pundits were doing. When the primaries started, he started predicting winners, because that's what real pundits were doing. I don't think the authors ever expected they would be this surprisingly accurate, but because they were, they kept doing it and more and more people started paying attention. That's it.
I don't think it's anti-intellectualism exactly, because their characters are all about mocking the stupid things that actual professional pundits with huge national platforms say and do. Even in the article linked in the FPP, Nate Silver says that he screwed up because he was acting like a pundit, which he allowed to affect his statistical analysis. Their goal has typically been more, "let's say the stupidest possible things in a serious voice and marvel at how close it comes to what actual, non-satire people are saying." (And they've been vindicated when they've had, for instance, actual Twitter conversations with real pundits while in character as Diggler, or had their parody articles retweeted by actual presidential candidates)
posted by cobra_high_tigers at 9:30 AM on May 18, 2016 [14 favorites]
Satire. It was for the same reason the Onion creates fake news articles. Or I guess more on the point, the same reason Stephen Colbert created his in-show character. To offer humorous commentary on current events but simultaneous meta-commentary on the people who publish about them in real outlets.
Carl Diggler was one of several satirical characters created by the same authors (I believe) for Cafe.com well before the primaries started. There's Professor Jeff Wilhelm, a self-aggrandizing blowhard professor at a military college who never seems to get anything right and can't keep his dick in his pants, clearly modeled after John Schindler (or at least his Twitter presence). There's Marshall Harford III, a kind of clueless trust fund anarchist Millennial who I believe was inspired by Malcolm Harris. And finally there's The Dig himself, who I'm not sure is supposed to represent any one pundit in particular but kind of a general parody of a certain Politico-friendly, op-ed publishing type - the bio at the start of his posts describes him as Cafe's "Chief Beltway Insider Hack," which is as good a description as any.
These guys, and a lot of the people in their social circle on Twitter, spend a lot of their time on harsh criticism of the political media. (Biederman is also one of the co-hosts of the Chapo Trap House podcast, which is equal parts smart commentary on current events and relentless shit talking of people in the media. It's fantastic, but probably not up the average MeFite's alley) Throughout the debates, Diggler published stupid debate scorecards, because that's what real pundits were doing. When the primaries started, he started predicting winners, because that's what real pundits were doing. I don't think the authors ever expected they would be this surprisingly accurate, but because they were, they kept doing it and more and more people started paying attention. That's it.
I don't think it's anti-intellectualism exactly, because their characters are all about mocking the stupid things that actual professional pundits with huge national platforms say and do. Even in the article linked in the FPP, Nate Silver says that he screwed up because he was acting like a pundit, which he allowed to affect his statistical analysis. Their goal has typically been more, "let's say the stupidest possible things in a serious voice and marvel at how close it comes to what actual, non-satire people are saying." (And they've been vindicated when they've had, for instance, actual Twitter conversations with real pundits while in character as Diggler, or had their parody articles retweeted by actual presidential candidates)
posted by cobra_high_tigers at 9:30 AM on May 18, 2016 [14 favorites]
As someone above said - I don't fault the man so much for failing to accept the eventuality of the repusive scenario we're in now (although I did correctly call it in my Metafilter election bet hmm hmmmm), but for the life of me I can't understand how or why he bought into the bullshit punditry surrounding Rubio. Talk about an emperor having no clothes..or no brain...
posted by sallybrown at 9:34 AM on May 18, 2016
posted by sallybrown at 9:34 AM on May 18, 2016
There's Marshall Harford III, a kind of clueless trust fund anarchist Millennial who I believe was inspired by Malcolm Harris.
ahaha some of his Poe's Law victims include members of MeFi
posted by Apocryphon at 9:36 AM on May 18, 2016 [2 favorites]
ahaha some of his Poe's Law victims include members of MeFi
posted by Apocryphon at 9:36 AM on May 18, 2016 [2 favorites]
I like the fundamental paradox of data journalism: You're writing about statistics, so the only way to tell if your models are right is to measure their long-run predictive power, meaning that the accuracy of any individual prediction, taken on it's own, is unknowable.
Now, in practice I ain't got time for that! Which means that it reduces, for me, to the same reputational test i.e. "hey this gal sounds like she knows what she's talking about" as classical punditry, but substituting charts and graphs (which, to be fair, are probably better at uncovering long-term trends and fundamental changes) for persuasive speaking and confidence.
n-thing the note on the rise of horse-racism.
posted by The Ted at 9:49 AM on May 18, 2016 [2 favorites]
Now, in practice I ain't got time for that! Which means that it reduces, for me, to the same reputational test i.e. "hey this gal sounds like she knows what she's talking about" as classical punditry, but substituting charts and graphs (which, to be fair, are probably better at uncovering long-term trends and fundamental changes) for persuasive speaking and confidence.
n-thing the note on the rise of horse-racism.
posted by The Ted at 9:49 AM on May 18, 2016 [2 favorites]
ahahaha I know what you mean by "horse-racism" but also it looks really funny
imagining headline "Donald Trump calls for ban on importation of Arabian horses"
posted by cobra_high_tigers at 9:52 AM on May 18, 2016 [5 favorites]
imagining headline "Donald Trump calls for ban on importation of Arabian horses"
posted by cobra_high_tigers at 9:52 AM on May 18, 2016 [5 favorites]
cobra_high_tigers, thanks. Your explanation was very helpful!
posted by zarq at 10:09 AM on May 18, 2016
posted by zarq at 10:09 AM on May 18, 2016
We all made a big mistake by mistaking horse race political journalism with journalism. It's transcription with some flowery illumination filled in to allow us to mistake it as wisdom.
It's political coverage that got rid of the boring stuff, like policy and actual outcomes. It turned ostensibly smart news consumers into sports fans who argue about box scores. Good work, journalism.
posted by turntraitor at 10:11 AM on May 18, 2016 [2 favorites]
It's political coverage that got rid of the boring stuff, like policy and actual outcomes. It turned ostensibly smart news consumers into sports fans who argue about box scores. Good work, journalism.
posted by turntraitor at 10:11 AM on May 18, 2016 [2 favorites]
Oh my. Horse race journalism is the infographics of political journalism.
posted by Apocryphon at 10:21 AM on May 18, 2016 [2 favorites]
posted by Apocryphon at 10:21 AM on May 18, 2016 [2 favorites]
I can't tell if it's a symptom of the anti-intellectual movement
LOL, it is quite precisely the opposite. This is the lesson of the entire debacle he's attempting to rationalize here. Nate Silver himself is part of the vanguard of acceptable middlebrow anti-intellectualism in American culture. By building false certainties on methodological quicksand, and selling them into the soi-disant "skeptical" college-educated liberal audience, he militates against abstraction, theory, and complexity in the name of "facts" that aren't.
posted by RogerB at 10:55 AM on May 18, 2016 [3 favorites]
LOL, it is quite precisely the opposite. This is the lesson of the entire debacle he's attempting to rationalize here. Nate Silver himself is part of the vanguard of acceptable middlebrow anti-intellectualism in American culture. By building false certainties on methodological quicksand, and selling them into the soi-disant "skeptical" college-educated liberal audience, he militates against abstraction, theory, and complexity in the name of "facts" that aren't.
posted by RogerB at 10:55 AM on May 18, 2016 [3 favorites]
Well, they were re-fighting the last war, where the GOP field had a revolving door of chuckleheads leading the polls in the months before the actual primaries but it ended up not being any of them but instead Blandy McBlanderson who was chosen by the party elite. Of course, at some point there should have been an adjustment to the fact that this time around it was only one non-revolving chucklehead and a lack of elite chosen ones.
But, my main disappointment with 538 was that the prediction models never had any depth beyond win/lose per state. The name 538 is itself a reminder that you can't simply look at the national popular vote polling when it's really the electoral vote that decides things, but ignoring the delegate selection rules in favor of just calling a state one way or another amounts to failing your own lesson.
posted by ckape at 11:00 AM on May 18, 2016 [1 favorite]
But, my main disappointment with 538 was that the prediction models never had any depth beyond win/lose per state. The name 538 is itself a reminder that you can't simply look at the national popular vote polling when it's really the electoral vote that decides things, but ignoring the delegate selection rules in favor of just calling a state one way or another amounts to failing your own lesson.
posted by ckape at 11:00 AM on May 18, 2016 [1 favorite]
I love this explanation:
lupus_yonderboy: "(* - yeah, I know, the weatherman doesn't actually make the forecasts, it's a metaphor, OK?)"
Actually a lot of the people you see on TV are doing their own forecasts.
posted by Mitheral at 11:01 AM on May 18, 2016 [1 favorite]
Without a model as a fortification, we found ourselves rambling around the countryside like all the other pundit-barbarians, randomly setting fire to things.
lupus_yonderboy: "(* - yeah, I know, the weatherman doesn't actually make the forecasts, it's a metaphor, OK?)"
Actually a lot of the people you see on TV are doing their own forecasts.
posted by Mitheral at 11:01 AM on May 18, 2016 [1 favorite]
538's most embarrassing failure this year is their "polls-plus" model, which was supposed to include a bunch of party-decides special sauce but has proved much less accurate than their simple average of polls. In other words, a simple average() function in excel would do better than Silver's special creation this year. It's a pretty spectacular failure; not only did his proprietary "polls-plus" model not perform well, he included a bunch of irrelevant explanatory variables that actually hurt his predictions.
People tend to overstate the complexity of Silver's methods, but the only thing he really does well is create a weighted average of polls based on previous poll performance. His "polls-only" predictions are just weighted averages of the same polls you can find anywhere else. It really isn't special, even from a statistical point of view, and his success predicting the 2012 election (one of the most predictable elections in US history) isn't particularly noteworthy, either.
God bless Carl Diggler for taking the piss out of our pathetically, shamefully awful pundit/media class that does almost no real reporting or even talking to voters, that takes their twitter mentions as a representative sample of voter sentiment, that covers idiotic horse race bullshit to the complete exclusion of any interest in policy or voters' lives, that hasn't cared about poor people since before the last Clinton administration, that has an extreme bias in favor of power, that clings to their stupid hidebound rules of artificial civility while treating life-or-death issues like academic debates. Someone captured this nicely on twitter the other day: "The hilarious self-regard of middlebrow B-list journalism is really something. Cavalier about death, deathly serious about campaign junk." That's the kind of idiotic destructive journalism that Diggler rightly lampoons.
posted by dialetheia at 12:26 PM on May 18, 2016 [22 favorites]
People tend to overstate the complexity of Silver's methods, but the only thing he really does well is create a weighted average of polls based on previous poll performance. His "polls-only" predictions are just weighted averages of the same polls you can find anywhere else. It really isn't special, even from a statistical point of view, and his success predicting the 2012 election (one of the most predictable elections in US history) isn't particularly noteworthy, either.
God bless Carl Diggler for taking the piss out of our pathetically, shamefully awful pundit/media class that does almost no real reporting or even talking to voters, that takes their twitter mentions as a representative sample of voter sentiment, that covers idiotic horse race bullshit to the complete exclusion of any interest in policy or voters' lives, that hasn't cared about poor people since before the last Clinton administration, that has an extreme bias in favor of power, that clings to their stupid hidebound rules of artificial civility while treating life-or-death issues like academic debates. Someone captured this nicely on twitter the other day: "The hilarious self-regard of middlebrow B-list journalism is really something. Cavalier about death, deathly serious about campaign junk." That's the kind of idiotic destructive journalism that Diggler rightly lampoons.
posted by dialetheia at 12:26 PM on May 18, 2016 [22 favorites]
People tend to overstate the complexity of Silver's methods
Yes! I don't recall the exact numbers but I compared Silver's results in... 2008 (? I think) with the prediction you would make if all you did was look at who was ahead in the last half dozen polls of each state and simply doing that with no weighting or special sauce of any sort would have gotten 48 states correct compared to Silver's 49. And clearly sometimes you'd do even better than Silver et al as in this year's primary. Just look at the polls!
posted by Justinian at 12:38 PM on May 18, 2016 [2 favorites]
Yes! I don't recall the exact numbers but I compared Silver's results in... 2008 (? I think) with the prediction you would make if all you did was look at who was ahead in the last half dozen polls of each state and simply doing that with no weighting or special sauce of any sort would have gotten 48 states correct compared to Silver's 49. And clearly sometimes you'd do even better than Silver et al as in this year's primary. Just look at the polls!
posted by Justinian at 12:38 PM on May 18, 2016 [2 favorites]
His main function was not adding a dumb commentary that pretended things were neck and neck when they weren't. When he started doing that he lost his value.
posted by Artw at 12:41 PM on May 18, 2016 [5 favorites]
posted by Artw at 12:41 PM on May 18, 2016 [5 favorites]
Silver's going to get kicked out of the political pundit club.
Which would boost his credibility more than the accuracy of his statistical models ever could.
posted by echocollate at 3:28 PM on May 18, 2016 [2 favorites]
Which would boost his credibility more than the accuracy of his statistical models ever could.
posted by echocollate at 3:28 PM on May 18, 2016 [2 favorites]
Sam Wang's post today is worth reading. An excerpt:
I think journalists have missed the point about Nate Silver’s error. Since Silver personifies data analysis, it is easy to get mixed up about what failed. As I wrote last week, the data didn’t fail – clear signs pointed toward Trump for a long time. However, Silver went beyond the data – in his words, he “acted like a pundit.” Here are his comments. The essay is long, but the title is on point. Basically I agree with points #1 (he didn’t make a real statistical model) and #4 (“fundamentals”-based models might not add that much value).
This isn't really a story about the failure of data journalism -- and this isn't really a 'no true Scotsman' argument. As he admits, Silver failed because his take on Trump came completely out of his ass and wasn't based in any actual data.
That's been the big disappointment of the rebirth of 538 in a post-NYT world. Whatever Silver is saying is his podcast (and I haven't listened to it), it's clear that at ESPN he's been asked to respond much more closely to the 24 hours cycle and media narratives, and it's not to his advantage.
posted by crazy with stars at 3:48 PM on May 18, 2016
I think journalists have missed the point about Nate Silver’s error. Since Silver personifies data analysis, it is easy to get mixed up about what failed. As I wrote last week, the data didn’t fail – clear signs pointed toward Trump for a long time. However, Silver went beyond the data – in his words, he “acted like a pundit.” Here are his comments. The essay is long, but the title is on point. Basically I agree with points #1 (he didn’t make a real statistical model) and #4 (“fundamentals”-based models might not add that much value).
This isn't really a story about the failure of data journalism -- and this isn't really a 'no true Scotsman' argument. As he admits, Silver failed because his take on Trump came completely out of his ass and wasn't based in any actual data.
That's been the big disappointment of the rebirth of 538 in a post-NYT world. Whatever Silver is saying is his podcast (and I haven't listened to it), it's clear that at ESPN he's been asked to respond much more closely to the 24 hours cycle and media narratives, and it's not to his advantage.
posted by crazy with stars at 3:48 PM on May 18, 2016
People tend to overstate the complexity of Silver's methods
One person who doesn't do this is Silver himself, for what it's worth.
posted by escabeche at 4:39 PM on May 18, 2016 [2 favorites]
One person who doesn't do this is Silver himself, for what it's worth.
posted by escabeche at 4:39 PM on May 18, 2016 [2 favorites]
The special sauce had no historical justification.
Early leaders, be they long-time favorites or insurgents, don't blow up -- they are surpassed by stronger candidates. Silver never had a polling case for Jeb, Rubio, Kasich or Cruz doing this. Jeb never led anything except his check book. Rubio and Kasich each won one state and polling never showed them nor competitive. Cruz for a guy who won a bunch of states was spectacularly unpopular in a bunch of others ...
Athletes and entertainers when they get into politics aren't disfavored such that you deprecate their polls. They are proven winners -- Franken, Schwarzenegger, Ventura, Kemp, Bradley, Reagan ...
posted by MattD at 5:43 PM on May 18, 2016 [1 favorite]
Early leaders, be they long-time favorites or insurgents, don't blow up -- they are surpassed by stronger candidates. Silver never had a polling case for Jeb, Rubio, Kasich or Cruz doing this. Jeb never led anything except his check book. Rubio and Kasich each won one state and polling never showed them nor competitive. Cruz for a guy who won a bunch of states was spectacularly unpopular in a bunch of others ...
Athletes and entertainers when they get into politics aren't disfavored such that you deprecate their polls. They are proven winners -- Franken, Schwarzenegger, Ventura, Kemp, Bradley, Reagan ...
posted by MattD at 5:43 PM on May 18, 2016 [1 favorite]
Wang is on target as usual, particularly at the end of his post:
Since estimating a prior probability relies on an element of taste, it seems to be an error to cling to apparent quantification (“10-12%”). It creates the appearance of rigor, but not the substance. Revising such a number post hoc doesn’t seem constructive. This move up from 2% goes high enough to reduce embarrassment, but stays low to retain credibility. Hypothetically, if Trump had lost, would we be reading about this revision? Probably not.
“Too frequentist” seems to be leaning hard on terminology. If Bayesian means “exercising judgment in interpreting data,” I would say he was too Bayesian. But let’s forget those terms. Basically, I was fortunate enough to notice that multi-election trends like “The Party Decides” were giving strange and contradictory answers. So I went back to polls, which were being quite clear. I say: let’s stick with polls when we can, and since modeling can be intrusive, keep it separate.
The thing about priors is, everyone has them, and anyone can apply them -- just multiply. There's no need for data pundits to do priors themselves. Just tell us your model, and let us apply the priors. So when Silver writes:
Another way to put it is that a model gives you discipline, and discipline is a valuable resource when everyone is losing their mind in the midst of a campaign.
he's still got it backwards. We don't care about you, or your discipline. Just give us the model(s), and let us do the disciplining. All these bullet points about fundamentals, polls, uniform priors, hindsight bias, etc -- forget it; this isn't silicon valley, we don't want a secret proprietary sauce. Just give us what the models suggest, even if they are contradictory, and let us do our own Bayesian weighting with our own opinions and biases.
Or heck, sure, they can even go ahead and add their own punditry weights if they want -- just keep them in posts separate from the models. That's win-win for them -- more posts, more eyeballs, more ads. And whichever simple model does best, they can then write a few more posts after the fact speculating about why.
posted by chortly at 6:45 PM on May 18, 2016 [2 favorites]
Since estimating a prior probability relies on an element of taste, it seems to be an error to cling to apparent quantification (“10-12%”). It creates the appearance of rigor, but not the substance. Revising such a number post hoc doesn’t seem constructive. This move up from 2% goes high enough to reduce embarrassment, but stays low to retain credibility. Hypothetically, if Trump had lost, would we be reading about this revision? Probably not.
“Too frequentist” seems to be leaning hard on terminology. If Bayesian means “exercising judgment in interpreting data,” I would say he was too Bayesian. But let’s forget those terms. Basically, I was fortunate enough to notice that multi-election trends like “The Party Decides” were giving strange and contradictory answers. So I went back to polls, which were being quite clear. I say: let’s stick with polls when we can, and since modeling can be intrusive, keep it separate.
The thing about priors is, everyone has them, and anyone can apply them -- just multiply. There's no need for data pundits to do priors themselves. Just tell us your model, and let us apply the priors. So when Silver writes:
Another way to put it is that a model gives you discipline, and discipline is a valuable resource when everyone is losing their mind in the midst of a campaign.
he's still got it backwards. We don't care about you, or your discipline. Just give us the model(s), and let us do the disciplining. All these bullet points about fundamentals, polls, uniform priors, hindsight bias, etc -- forget it; this isn't silicon valley, we don't want a secret proprietary sauce. Just give us what the models suggest, even if they are contradictory, and let us do our own Bayesian weighting with our own opinions and biases.
Or heck, sure, they can even go ahead and add their own punditry weights if they want -- just keep them in posts separate from the models. That's win-win for them -- more posts, more eyeballs, more ads. And whichever simple model does best, they can then write a few more posts after the fact speculating about why.
posted by chortly at 6:45 PM on May 18, 2016 [2 favorites]
The title gets it right but I'm less impressed by the text*. From my somewhat lackadaisical observations it really looked like Silver had picked a narrative ("Trump will implode") and then started selectively picking stats to support it. Trump had broader support (usually leading nationally, in Iowa and in New Hampshire) and for a longer period than any of the modern comparators, but Silver was just grabbing the most obvious similarities ("remember Cain led at this point") exactly like any other pundit supporting his pet theory.
Has anyone lost in Trump's situation in the modern era? Doing a polls-only guesstimate would have given a higher number, and higher than the "10-15%" he now cops to. How would people have read his prediction if he said "different assumptions give me a range of Trump's probability at between 2% and 40% and there's no objective way to weight them and produce a meaningful single number." It's a fancy way of saying "I dunno" but he didn't know. I completely agree with the quote chortly pulls above chiding Silver for still sticking to fake precision.
There's nothing different about what the "data journalist" does vs. the pundit except for the "science" branding. Models or "priors" are "hunches" promoted to seem objective.
The value of Silver in 2008 and 2012 was precisely that the "model" was 99%"the polls are accurate" and it completely showed up the people going on hunches. And in the process those trying to construct a narrative out of the day-to-day gaffes and horse race crap were shown to be full of it.
The brand "data journalism" in general includes a lot of what you're saying but it wasn't why Silver got the initial respect.
It was of the more dramatic examples that convinced me that the general approach--"believe the simple measurements and forget about models"--should be applied by everyone who does quantitative work, all the time, as a reality check to more sophisticated predictions and help call yourself out when you might be fooling yourself.
*Although graded on the pundit curve, admitting any mistake is good.
posted by mark k at 10:09 PM on May 18, 2016 [2 favorites]
Has anyone lost in Trump's situation in the modern era? Doing a polls-only guesstimate would have given a higher number, and higher than the "10-15%" he now cops to. How would people have read his prediction if he said "different assumptions give me a range of Trump's probability at between 2% and 40% and there's no objective way to weight them and produce a meaningful single number." It's a fancy way of saying "I dunno" but he didn't know. I completely agree with the quote chortly pulls above chiding Silver for still sticking to fake precision.
There's nothing different about what the "data journalist" does vs. the pundit except for the "science" branding. Models or "priors" are "hunches" promoted to seem objective.
The value of Silver in 2008 and 2012 was precisely that the "model" was 99%"the polls are accurate" and it completely showed up the people going on hunches. And in the process those trying to construct a narrative out of the day-to-day gaffes and horse race crap were shown to be full of it.
The brand "data journalism" in general includes a lot of what you're saying but it wasn't why Silver got the initial respect.
It was of the more dramatic examples that convinced me that the general approach--"believe the simple measurements and forget about models"--should be applied by everyone who does quantitative work, all the time, as a reality check to more sophisticated predictions and help call yourself out when you might be fooling yourself.
*Although graded on the pundit curve, admitting any mistake is good.
posted by mark k at 10:09 PM on May 18, 2016 [2 favorites]
I'm confused. It was too long and I stopped at the end of the intro.
But is Nate Silver a wizard or not?
posted by Drinky Die at 1:04 AM on May 19, 2016
But is Nate Silver a wizard or not?
posted by Drinky Die at 1:04 AM on May 19, 2016
I loved that Fark thread. A discussion of Nate Silver turns into a deep nerd-fest over Lord of the Rings minutia.
posted by Slap*Happy at 5:41 AM on May 19, 2016 [1 favorite]
posted by Slap*Happy at 5:41 AM on May 19, 2016 [1 favorite]
I don't live in the US so I'm not really qualified, but what the hell's going on over there? "The Republicans hate Trump, he'll never get elected!", and he's disliked by 56% of voters (that's what I remember from a month ago, anyway). "Hillary's the middle ground, but she's the tool of Wall St!" Too bad about Bernie.... Trump gets the GOP nomination over the objections of the party leaders, but people in general don't like him. Ok, fine.
And then, all of a sudden, Trump and Clinton are neck and neck. How does that work? I keep thinking about how NBC has already spent $1.23 billion for the broadcast right for the 2016 Olympics. How much is the presidential race worth to the big networks?
posted by sneebler at 9:11 AM on May 19, 2016
And then, all of a sudden, Trump and Clinton are neck and neck. How does that work? I keep thinking about how NBC has already spent $1.23 billion for the broadcast right for the 2016 Olympics. How much is the presidential race worth to the big networks?
posted by sneebler at 9:11 AM on May 19, 2016
Lesser of two evils voting in a two party system where each party sees the other as always a greater evil.
posted by Drinky Die at 2:45 PM on May 19, 2016
posted by Drinky Die at 2:45 PM on May 19, 2016
Fox? IBD? Rasmussen? Those polls are hot garbage. Let's see what 538's polls only model says, Nate's team generally does a good job of vetting pollsters.
posted by Slap*Happy at 2:56 PM on May 19, 2016
posted by Slap*Happy at 2:56 PM on May 19, 2016
I thought this said, "Stop the Polling Industry", but whatever.
posted by sneebler at 6:37 AM on May 20, 2016
posted by sneebler at 6:37 AM on May 20, 2016
« Older The best basket in China | Letters make nice buildings Newer »
This thread has been archived and is closed to new comments
posted by Kybard at 6:36 AM on May 18, 2016 [9 favorites]