Prizes Trump Biases
June 4, 2013 10:03 AM Subscribe
I'd settle for one that just fined them for obvious lies.
posted by MCMikeNamara at 10:09 AM on June 4, 2013 [5 favorites]
posted by MCMikeNamara at 10:09 AM on June 4, 2013 [5 favorites]
Yeah, as they point out, it doesn't really work for voting, which is where it counts...
posted by RedOrGreen at 10:14 AM on June 4, 2013
posted by RedOrGreen at 10:14 AM on June 4, 2013
The best part of sharing this on my social media networks is partisans on both sides trying to smear the others with it, then pointing out it specifies no, both of you do it, and watching their brains lock up.
posted by Ghostride The Whip at 10:19 AM on June 4, 2013 [3 favorites]
posted by Ghostride The Whip at 10:19 AM on June 4, 2013 [3 favorites]
What if I don't care if there were more or less casualties in Iraq in a given year, but rather care that there were any at all?
posted by The 10th Regiment of Foot at 10:25 AM on June 4, 2013 [1 favorite]
posted by The 10th Regiment of Foot at 10:25 AM on June 4, 2013 [1 favorite]
What if I don't care if there were more or less casualties in Iraq in a given year, but rather care that there were any at all?
What does that have to do with whether or not you know what was, in fact, the case?
posted by yoink at 10:27 AM on June 4, 2013 [3 favorites]
What does that have to do with whether or not you know what was, in fact, the case?
posted by yoink at 10:27 AM on June 4, 2013 [3 favorites]
Brennan and Lomasky used to be dubiously summarized as "politics with romance". Seems odd that the idea there is politics in politics is new again.
posted by hawthorne at 10:29 AM on June 4, 2013
posted by hawthorne at 10:29 AM on June 4, 2013
Pffff. This is just another scheme by neo-conservatives to get more money out of minorities.
posted by ishrinkmajeans at 10:33 AM on June 4, 2013 [3 favorites]
posted by ishrinkmajeans at 10:33 AM on June 4, 2013 [3 favorites]
We need a cable news network that only pays commentators for factual statements.
"Factual statements" will come to be interpreted as "what the boss thinks is factual." I think all this has shown is that people are able to answer factually when asked to do so by someone neutral and unknown for a reward. If the reward-giver was known to have a partisan bias, then the participants might answer according to that bias just to get the reward.
posted by cosmic.osmo at 10:36 AM on June 4, 2013 [3 favorites]
"Factual statements" will come to be interpreted as "what the boss thinks is factual." I think all this has shown is that people are able to answer factually when asked to do so by someone neutral and unknown for a reward. If the reward-giver was known to have a partisan bias, then the participants might answer according to that bias just to get the reward.
posted by cosmic.osmo at 10:36 AM on June 4, 2013 [3 favorites]
Yeah, as they point out, it doesn't really work for voting, which is where it counts...
Making each vote an entry into a drawing for a million-dollar Amazon gift card would be a pretty big incentive for poorer people to get out and vote...
posted by GuyZero at 10:37 AM on June 4, 2013 [4 favorites]
Making each vote an entry into a drawing for a million-dollar Amazon gift card would be a pretty big incentive for poorer people to get out and vote...
posted by GuyZero at 10:37 AM on June 4, 2013 [4 favorites]
This does not work on Red Sox fans.
posted by Mister_A at 10:38 AM on June 4, 2013 [1 favorite]
posted by Mister_A at 10:38 AM on June 4, 2013 [1 favorite]
What does that have to do with whether or not you know what was, in fact, the case?
It matters because I don't care what the answer is and I don't care to find out what the answer is, if it was more than zero it was too many.
posted by The 10th Regiment of Foot at 10:41 AM on June 4, 2013 [1 favorite]
It matters because I don't care what the answer is and I don't care to find out what the answer is, if it was more than zero it was too many.
posted by The 10th Regiment of Foot at 10:41 AM on June 4, 2013 [1 favorite]
Another way to interpret this is in the context of Kahneman's ideas in Thinking, Fast and Slow. Maybe if it's just an answer on a pointless questionnaire people will be answering quickly and without much care, so if the answer doesn't happen to be at the top of their minds they'll tend towards answers that "feel" right, flattering their partisan beliefs. But when there's money on the line they start to pay attention, and actually engage their brains sufficiently to think it through and work out the correct answer.
(I have to confess that I haven't read the actual paper, as it's 22 pages plus appendix and references. Maybe the authors address this?)
What if I don't care if there were more or less casualties in Iraq in a given year, but rather care that there were any at all?
It's possible to both disapprove of an event and know suff about it.
posted by metaBugs at 10:42 AM on June 4, 2013 [11 favorites]
(I have to confess that I haven't read the actual paper, as it's 22 pages plus appendix and references. Maybe the authors address this?)
What if I don't care if there were more or less casualties in Iraq in a given year, but rather care that there were any at all?
It's possible to both disapprove of an event and know suff about it.
posted by metaBugs at 10:42 AM on June 4, 2013 [11 favorites]
It matters because I don't care what the answer is and I don't care to find out what the answer is, if it was more than zero it was too many.
But what would you have answered? Would you have answered that the number went up between 2007 and 2008, that it went down, or would you have answered (honestly) that you don't know?
The whole point of this study was that people answer these kinds of surveys dishonestly, that they answer according to their own partisan bias. I.e. people who absolutely hated the Iraq war would answer that the number went up in those years, even though the truthful answer was that they didn't know.
This study disproves the (quite alarming) finding that people who have different political views live in different realities, with different facts about the world. It turns out they mostly don't, they just like to make their opinions known to annoying people who call them on the phone during dinner.
posted by gkhan at 10:55 AM on June 4, 2013 [3 favorites]
But what would you have answered? Would you have answered that the number went up between 2007 and 2008, that it went down, or would you have answered (honestly) that you don't know?
The whole point of this study was that people answer these kinds of surveys dishonestly, that they answer according to their own partisan bias. I.e. people who absolutely hated the Iraq war would answer that the number went up in those years, even though the truthful answer was that they didn't know.
This study disproves the (quite alarming) finding that people who have different political views live in different realities, with different facts about the world. It turns out they mostly don't, they just like to make their opinions known to annoying people who call them on the phone during dinner.
posted by gkhan at 10:55 AM on June 4, 2013 [3 favorites]
Alternate explanation: We really believe the "lies", but when cash is on the line we go with the answer we think is most likely to be scored as correct, all the while believing that the "correct" answer is a distortion by media or data-gathering sources friendly to the opposite party. Under that explanation, people will break with their beliefs and lie when cash is on the line, rather than tell the truth for cash. I dunno about you, but hearing that people will be immoral for cash tends to better fit my preconceived notions of the world. And the explanation that fits my preconceived notions is, of course, the correct one.
posted by agentofselection at 10:58 AM on June 4, 2013 [10 favorites]
posted by agentofselection at 10:58 AM on June 4, 2013 [10 favorites]
But what would you have answered?
The truth: "I don't know" Why would I answer differently?
The whole point of this study was that people answer these kinds of surveys dishonestly
But the point is skewed by the fact that the answers are not black or white particularly when taken in context of people's observation biases.
posted by The 10th Regiment of Foot at 11:10 AM on June 4, 2013 [1 favorite]
The truth: "I don't know" Why would I answer differently?
The whole point of this study was that people answer these kinds of surveys dishonestly
But the point is skewed by the fact that the answers are not black or white particularly when taken in context of people's observation biases.
posted by The 10th Regiment of Foot at 11:10 AM on June 4, 2013 [1 favorite]
"Belief" is a pretty slippery concept. If you loudly proclaim belief in one idea while actually making decisions based on a contradictory one, which do you believe? What if, while proclaiming your belief, you're also making cunning rational arguments for it--but still not using it to make decisions outside of conversation?
These sorts of questions would normally be thrown out as sophistic derails, but I think they're quite important for interpreting the results of surveys about belief.
posted by LogicalDash at 11:15 AM on June 4, 2013 [1 favorite]
These sorts of questions would normally be thrown out as sophistic derails, but I think they're quite important for interpreting the results of surveys about belief.
posted by LogicalDash at 11:15 AM on June 4, 2013 [1 favorite]
You have to pay five bucks to read the actual paper. I wonder what that says, as opposed to what the authors tell us it says when we don't pay.
posted by Flunkie at 11:22 AM on June 4, 2013 [4 favorites]
posted by Flunkie at 11:22 AM on June 4, 2013 [4 favorites]
Good point, LogicalDash, but I would argue that these survey participants aren't making decisions based on their beliefs about, e.g., the unemployment rate, but rather their beliefs about the unemployment rate as reported by the surveys used as a basis for the test they are taking. I can simultaneously "know" that the unemployment rate rose, and "know" that surveys say it actually fell. Then I can believe, and act (i.e. vote) on the knowledge that it rose, while answering questions by saying that it fell. In that case I guess I wouldn't be lying at all, though I would be cutting the truth pretty fine.
posted by agentofselection at 11:25 AM on June 4, 2013 [1 favorite]
posted by agentofselection at 11:25 AM on June 4, 2013 [1 favorite]
What's interesting is the idea that this almost certainly extends beyond just partisan attempts to "skew" telephone polls in a particular direction. If the study is to be believed, ordinary people with strong political views will manage their advocacy in a calculated and cynical manner, advancing knowingly false or misleading narratives in service of a larger agenda.
Before this study, I'd suggest that it was conventionally believed that calculated political bullshit was manufactured and promoted by consultants and hacks to an unsuspecting and credulous public. This study suggests that at least some people know they're consuming and spreading bullshit, and that they do so with open eyes.
So take the Obama "birther" claims, for instance. One could use this study to hypothesize that a significant portion of "birthers" don't really believe that Obama was born in Kenya. Rather, they've just latched onto something minimally plausible in order to undermine the President's legitimacy.
So the next question is whether we should find it assuring or troubling that a good portion of people we thought were fools are actually knaves.
posted by BobbyVan at 11:37 AM on June 4, 2013 [4 favorites]
Before this study, I'd suggest that it was conventionally believed that calculated political bullshit was manufactured and promoted by consultants and hacks to an unsuspecting and credulous public. This study suggests that at least some people know they're consuming and spreading bullshit, and that they do so with open eyes.
So take the Obama "birther" claims, for instance. One could use this study to hypothesize that a significant portion of "birthers" don't really believe that Obama was born in Kenya. Rather, they've just latched onto something minimally plausible in order to undermine the President's legitimacy.
So the next question is whether we should find it assuring or troubling that a good portion of people we thought were fools are actually knaves.
posted by BobbyVan at 11:37 AM on June 4, 2013 [4 favorites]
You have to pay five bucks to read the actual paper.
I thought NBER working papers were free, but I'm in my office now...
Anyhow, dump the title into google and you'll find preprint/conference versions of the paper, as you would for most current papers/articles.
posted by ROU_Xenophobe at 11:46 AM on June 4, 2013
I thought NBER working papers were free, but I'm in my office now...
Anyhow, dump the title into google and you'll find preprint/conference versions of the paper, as you would for most current papers/articles.
posted by ROU_Xenophobe at 11:46 AM on June 4, 2013
From the NBER website:
"You may purchase this paper on-line in .pdf format from SSRN.com ($5) for electronic delivery.
Information about Free Papers
You should expect a free download if you are a subscriber, a corporate associate of the NBER, a journalist, an employee of the U.S. federal government with a ".GOV" domain name, or a resident of nearly any developing country or transition economy."
posted by MisantropicPainforest at 11:55 AM on June 4, 2013
"You may purchase this paper on-line in .pdf format from SSRN.com ($5) for electronic delivery.
Information about Free Papers
You should expect a free download if you are a subscriber, a corporate associate of the NBER, a journalist, an employee of the U.S. federal government with a ".GOV" domain name, or a resident of nearly any developing country or transition economy."
posted by MisantropicPainforest at 11:55 AM on June 4, 2013
You have to pay five bucks to read the actual paper. I wonder what that says, as opposed to what the authors tell us it says when we don't pay.
These questions aside, the core of our paper is the exposition of a model of expressive survey response and the implementation of a pair of experiments designed to distinguish that cheerleading behavior from sincere partisan divergence. We find that small financial inducements for correct responses can substantially reduce partisan divergence, and that these reductions are even larger when inducements are also provided for “don’t know” answers. In light of these results, survey responses that indicate partisan polarization with respect to factual matters should not be taken at face value. Researchers and general analysts of public opinion should consider the possibility that the appearance of polarization is to a great extent an artifact of survey measurement rather than evidence of real differences in beliefs.posted by the man of twists and turns at 12:45 PM on June 4, 2013
Yeah, count me among the skeptics. You win at tests by understanding what the test wants you to answer. I'd think it more likely that people are aware of the "conventional wisdom" on a given question and are more likely to give that as the answer when there's money at stake than simply go with their own opinion. It's Keynes and the beauty contest all over again.
posted by Diablevert at 1:12 PM on June 4, 2013 [1 favorite]
posted by Diablevert at 1:12 PM on June 4, 2013 [1 favorite]
Yeah, count me among the skeptics. You win at tests by understanding what the test wants you to answer. I'd think it more likely that people are aware of the "conventional wisdom" on a given question and are more likely to give that as the answer when there's money at stake than simply go with their own opinion. It's Keynes and the beauty contest all over again.
But for objective things like unemployment rates and inflation rates, there is only one valid truth: the numbers published by the government. Presumably the study asked the questions in a manner that made this clear.
So if partisans were more likely to answer the same questions correctly when it was in their direct best interest to do so, what was their motivation in answering wrong when it wasn't? The only conclusion I can draw is that the people who didn't have a financial interest in answering correctly were answering based on some other motivation that benefited them: perpetuating their chosen lies about politics.
These weren't opinion questions.
posted by gjc at 4:01 PM on June 4, 2013
But for objective things like unemployment rates and inflation rates, there is only one valid truth: the numbers published by the government. Presumably the study asked the questions in a manner that made this clear.
So if partisans were more likely to answer the same questions correctly when it was in their direct best interest to do so, what was their motivation in answering wrong when it wasn't? The only conclusion I can draw is that the people who didn't have a financial interest in answering correctly were answering based on some other motivation that benefited them: perpetuating their chosen lies about politics.
These weren't opinion questions.
posted by gjc at 4:01 PM on June 4, 2013
But for objective things like unemployment rates and inflation rates, there is only one valid truth: the numbers published by the government. Presumably the study asked the questions in a manner that made this clear.
Eh, you be the judge. Here's the wording from the paper narain provided above:
One interesting thing is that the respondents were required to answer on a sliding scale --- that is, instead of selection option 1, 2, or 3, etc., they answered by moving a slider between markers 1, 2, 3, etc. So if option one was +10% and option 2 was 0% they could effectively answer +2% +7% by leaving the slider inbetween the options.
This suggests to me that the respondents could effectively hedge their guesses, which is interesting. Perhaps my liberal soul instinctively answers that employment was like, basically flat during the Bush administration, and so if called upon to merely name an option I might say (3) 0%. But with a slider I could go hmmmm....well, basically flat of course, but maybe there was a slight increase? and nudge it between options (3) and (4) for an effective answer of 1 percent or so. It seems clear to me that the incentive to hedge will be greater when there's money involved.
These weren't opinion questions.
Yes an no. Take the unemployment rate. You're absolutely right that the "unemployment rate" is a figure published monthly by a the Bureau of Labor, and inasmuch as that is the case there is a simple, factual answer as to whether it went up or down during a given period.
But whether the BoL's monthly unemployment rate is an accurate reflection of the "true" unemployment level in the U.S. has been a matter of controversy in the recent recession, with many conservatives, among them some prominent economists, arguing that either a) the BoL was outright cooking the books or b) that BoL's headline number was an inaccurate measure of the true unemployment figure at this juncture and that something like the U-16 should be used instead.
So now, to return to that simple question, "did unemployment rate go up or down during the Obama administration?" Well, do you believe the BoL or not? What sources were you reading? How much do you really remember from those sources? It seems entirely possible to me that a conservative might believe both a) that it is true that the unemployment rate has increased sharply under Obama and that b) that the mainstream media and official government statistics do not accurately reflect that fact. If you were to ask such a person, what is true? They might give you one answer. And if you were to tell them "Answer the question correctly and you win a prize" they might give you a different answer, knowing that what's on the answer sheet might not be "the truth," but rather an "official truth."
The only conclusion I can draw is that the people who didn't have a financial interest in answering correctly were answering based on some other motivation that benefited them: perpetuating their chosen lies about politics.
Nah, I think that would require far, far too much consciousness, I think. It's like grocery shopping when you're hungry --- you spend 20% more when you do that not because of any conscious decision to do but simply because more stuff looks good to you. People's moods will change a lot about how they answer a survey, particularly one like this where you're free to supply a range. Merely the fact of being more cautious and wary because now there's more at stake may spur people to adhere closer to the mean --- I definitely don't think any of the participants was thinking to themselves, "well, not that there's an Amazon gift card in it i shall rip off the mask and reveal my true understanding of the world."
posted by Diablevert at 5:02 PM on June 4, 2013 [1 favorite]
Eh, you be the judge. Here's the wording from the paper narain provided above:
From January 2009, when President Obama first took office, to February 2012, how had the unemployment rate in the country changed?Running my eye over the questions asked, they all appear to be about like this. There's no appeal to/citation of authority in any of the questions. All of them are questions which have factual answers.
From January 2001, when President Bush first took office, to January 2009, when President Bush left office, how had the unemployment rate in the country changed?
One interesting thing is that the respondents were required to answer on a sliding scale --- that is, instead of selection option 1, 2, or 3, etc., they answered by moving a slider between markers 1, 2, 3, etc. So if option one was +10% and option 2 was 0% they could effectively answer +2% +7% by leaving the slider inbetween the options.
This suggests to me that the respondents could effectively hedge their guesses, which is interesting. Perhaps my liberal soul instinctively answers that employment was like, basically flat during the Bush administration, and so if called upon to merely name an option I might say (3) 0%. But with a slider I could go hmmmm....well, basically flat of course, but maybe there was a slight increase? and nudge it between options (3) and (4) for an effective answer of 1 percent or so. It seems clear to me that the incentive to hedge will be greater when there's money involved.
These weren't opinion questions.
Yes an no. Take the unemployment rate. You're absolutely right that the "unemployment rate" is a figure published monthly by a the Bureau of Labor, and inasmuch as that is the case there is a simple, factual answer as to whether it went up or down during a given period.
But whether the BoL's monthly unemployment rate is an accurate reflection of the "true" unemployment level in the U.S. has been a matter of controversy in the recent recession, with many conservatives, among them some prominent economists, arguing that either a) the BoL was outright cooking the books or b) that BoL's headline number was an inaccurate measure of the true unemployment figure at this juncture and that something like the U-16 should be used instead.
So now, to return to that simple question, "did unemployment rate go up or down during the Obama administration?" Well, do you believe the BoL or not? What sources were you reading? How much do you really remember from those sources? It seems entirely possible to me that a conservative might believe both a) that it is true that the unemployment rate has increased sharply under Obama and that b) that the mainstream media and official government statistics do not accurately reflect that fact. If you were to ask such a person, what is true? They might give you one answer. And if you were to tell them "Answer the question correctly and you win a prize" they might give you a different answer, knowing that what's on the answer sheet might not be "the truth," but rather an "official truth."
The only conclusion I can draw is that the people who didn't have a financial interest in answering correctly were answering based on some other motivation that benefited them: perpetuating their chosen lies about politics.
Nah, I think that would require far, far too much consciousness, I think. It's like grocery shopping when you're hungry --- you spend 20% more when you do that not because of any conscious decision to do but simply because more stuff looks good to you. People's moods will change a lot about how they answer a survey, particularly one like this where you're free to supply a range. Merely the fact of being more cautious and wary because now there's more at stake may spur people to adhere closer to the mean --- I definitely don't think any of the participants was thinking to themselves, "well, not that there's an Amazon gift card in it i shall rip off the mask and reveal my true understanding of the world."
posted by Diablevert at 5:02 PM on June 4, 2013 [1 favorite]
My interpretation is simply that the participants hold contradictory beliefs, and will act on one or the other depending on which is more beneficial at the moment. Answering "with their hearts" when the satisfaction thereof is the only benefit to gain, but "with their heads" when there's money.
I don't think this is really equivalent to simply lying for cash. In both cases, if pressed, a person will stand by their stated opinion, and continue to do so after the benefit is no longer available. That's my prediction.
If you somehow manage to get them to act on both beliefs at once, all bets are off.
posted by LogicalDash at 8:28 PM on June 4, 2013
I don't think this is really equivalent to simply lying for cash. In both cases, if pressed, a person will stand by their stated opinion, and continue to do so after the benefit is no longer available. That's my prediction.
If you somehow manage to get them to act on both beliefs at once, all bets are off.
posted by LogicalDash at 8:28 PM on June 4, 2013
agentofselection: Alternate explanation: We really believe the "lies", but when cash is on the line we go with the answer we think is most likely to be scored as correct, all the while believing that the "correct" answer is a distortion by media or data-gathering sources friendly to the opposite party. Under that explanation, people will break with their beliefs and lie when cash is on the line, rather than tell the truth for cash. I dunno about you, but hearing that people will be immoral for cash tends to better fit my preconceived notions of the world. And the explanation that fits my preconceived notions is, of course, the correct one.Reposted, because it's an important alternate explanation.
Even really good scientific experiments can result in complete shit explanations. The result is not the explanation. The explanation is just the sound-bite wrapper for the world at large.
posted by IAmBroom at 9:13 PM on June 4, 2013
They could probably correct for that by presenting themselves to the subjects as a partisan group.
posted by gjc at 11:09 PM on June 4, 2013
posted by gjc at 11:09 PM on June 4, 2013
Here's what I don't get and why I'm strongly inclined toward agentofselection's alternative explanation: why do the authors assume people are trying to curry favor with their non-rewarded survey responses but not with their rewarded responses when the rewards by definition come from someone else who's judging their responses?
posted by inara at 3:22 PM on June 5, 2013 [1 favorite]
posted by inara at 3:22 PM on June 5, 2013 [1 favorite]
« Older "This moment couldn't get any more perfect"... | This Guy is on FIRE Newer »
This thread has been archived and is closed to new comments
posted by Drinky Die at 10:08 AM on June 4, 2013 [8 favorites]