"Lately, the Sea of Polls is deeper than ever before, and darker."
November 22, 2015 11:11 AM   Subscribe

 
Polls are like advertising. First of all, they work best when only one does it. Second, as soon as s/he achieves success, someone else will get into the game, and more and more. Third, the more people doing it the less effective it becomes.

Advertising eventually reaches a saturation point where the desired audience tunes it out and ignores it completely, and then it doesn't accomplish anything useful to the advertiser. It's a classic example of "spoiling the commons".

As to polling, when more and more polls are run, more and more often, more and more people in the polled population get called more than once, and get sick of it and hang up. Which means that the poll results is based only on those people willing to put up with lots of calls, and they aren't necessarily statistically representative of the population as a whole. So the polls become less and less accurate.

How much polling has been done in Iowa and New Hampshire by this point in this election cycle? Surely we've reached the point where a large percentage of people hang up immediately when they here the beginning of a robocall spiel.

Another problem is cell phones. More and more people (especially hip young people) are eschewing landlines completely, and it's against FCC regulations to call cell phones for this kind of thing, so cell-phone-only people are left completely out of the poll result.

(By the way, the Haaretz article is behind a paywall.)
posted by Chocolate Pickle at 11:24 AM on November 22, 2015 [1 favorite]


I would like to congratulate both Jill Lepore, for making a pun involving Donald Trump, English etymology, and the word "flaxen," and the editors of the New Yorker, for letting it stand.
posted by ostro at 11:55 AM on November 22, 2015 [9 favorites]


The UK Polling Report blog had a piece on why the UK election prediction was wrong.
If the problems had been caused by people incorrectly reporting their voting intentions ("shy Tories") or people saying they would when they did not then it is likely that exactly the same problems would have shown up in the British Election Study... The difference... suggests that the error is associated with the thing that makes the BES f2f so different from the pre-election polls – the way it is sampled...

I think most people in market research would agree a proper random sample like the BES is the ideal, but the cost is exponentially higher... a completely difference scale of expense, the difference between a couple of thousand and a couple of hundred thousand. No media outlet could ever justify the cost of a full scale random poll...
posted by TheophileEscargot at 12:33 PM on November 22, 2015


My take on polls:
1. it is much too early to take the recent polls seriously
2. Despite the polls, Republican biggies behind the scene the WSJ reported, are planning to sink Trump.
3. Polls in Iowa and New Hampshire are hardly indicative of the entire nation.
4. There is at least one poll that is always right leaning and seldom reflects what the other polls suggest..Zogby
5. Polls are like election signs placed on lawns: some believe that if candidate A is way ahead of B, then we need to go with a presumed winner and thus will support A rather than B.
6. The only thing polls are perhaps useful for is that they may be more accurate than a cable station asking its biased readers to send in their opinion on a given topic.
7. If the polls are wrong when election time comes around, you can be sure they will be able to account for having been wrong.
posted by Postroad at 1:08 PM on November 22, 2015 [1 favorite]


4. There is at least one poll that is always right leaning and seldom reflects what the other polls suggest..Zogby

Zogby's house effects have varied from election to election. In 2008, the Zogby house effect was -1 (i.e., it predicted the vote was -1 less Democratic than it really was). In 2012, the Zogby house effect was +2 in Obama's favor. Zogby himself is a Lebanese-American Democrat, but it's still possible that his current polls have a "house effect" that overestimates the Republican vote share. A Democratic or Republican skew by a specific pollster can easily be the result of a methodological artifact and not the reflection of the pollster's personal ideology at all.
posted by jonp72 at 6:19 PM on November 22, 2015 [2 favorites]


Polls are like advertising. First of all, they work best when only one does it. Second, as soon as s/he achieves success, someone else will get into the game, and more and more. Third, the more people doing it the less effective it becomes.


This is pithy, although the phenomenal accuracy of Sam Wang's and other's poll aggregators proves that it's also untrue. This article is a nice history of polls written by a historian, but fundamentally disconnected from the measurable reality that polls have, in aggregate, been quite good at predicting election outcomes.
posted by one_bean at 6:46 PM on November 22, 2015 [1 favorite]


At least at the presidential level in the US analysis of the polls (eg Wang's Princeton analysis) has yielded good predictions.
posted by persona au gratin at 12:54 AM on November 23, 2015


I went to an event run by the Royal Statistical Society earlier this year that looked at big data and elections - lots of the segments are on YouTube, and this one by John Curtice (approx 10 minutes long) is worth a watch. He's president of the British Polling Council and is chairing the investigation into how the election polls were so wonky* in 2015.

*technical statistical term
posted by theseldomseenkid at 3:59 AM on November 23, 2015


Polls are like advertising. First of all, they work best when only one does it. Second, as soon as s/he achieves success, someone else will get into the game, and more and more. Third, the more people doing it the less effective it becomes.

"A man with a watch knows what time it is. A man with two watches is never sure."
posted by Foosnark at 6:22 AM on November 23, 2015 [2 favorites]


I would like to congratulate both Jill Lepore, for making a pun involving Donald Trump, English etymology, and the word "flaxen," and the editors of the New Yorker, for letting it stand.

ostro, - I think that may even be a triple entendre - the first word of the paragraph is "straw."
posted by univac at 9:41 AM on November 23, 2015


« Older It spent all its money on whiskey and beer   |   Kaizo Trap - full of dead ends Newer »


This thread has been archived and is closed to new comments