I'm not crazy, you are!
June 11, 2016 4:27 AM Subscribe
Oops. In 2012 a study was published that linked liberalism with social desirability, and conservatism with psychosis. A series of papers were published, some in high profile outlets. Now, they have been retracted. Why? The codings in the data were reversed--liberals were coded as conservatives, and vice-versa.
From the erratum in the American Journal of Political Science,
"The interpretation of the coding of the political attitude items in the descriptive and preliminary analyses portion of the manuscript was exactly reversed. Thus, where we indicated that higher scores in Table 1 (page 40) reflect a more conservative response, they actually reflect a more liberal response. Specifically, in the original manuscript, the descriptive analyses report that those higher in Eysenck’s psychoticism are more conservative, but they are actually more liberal; and where the original manuscript reports those higher in neuroticism and social desirability are more liberal, they are, in fact, more conservative."
From the erratum in the American Journal of Political Science,
"The interpretation of the coding of the political attitude items in the descriptive and preliminary analyses portion of the manuscript was exactly reversed. Thus, where we indicated that higher scores in Table 1 (page 40) reflect a more conservative response, they actually reflect a more liberal response. Specifically, in the original manuscript, the descriptive analyses report that those higher in Eysenck’s psychoticism are more conservative, but they are actually more liberal; and where the original manuscript reports those higher in neuroticism and social desirability are more liberal, they are, in fact, more conservative."
Now I get to be smug about not being smug about the original paper! Metasmug is the best smug.
posted by escabeche at 5:28 AM on June 11, 2016 [118 favorites]
posted by escabeche at 5:28 AM on June 11, 2016 [118 favorites]
smugness is linked to psychosis.
posted by Obscure Reference at 5:33 AM on June 11, 2016 [15 favorites]
posted by Obscure Reference at 5:33 AM on June 11, 2016 [15 favorites]
and meta-smugness is linked to metempsychosis
posted by nebulawindphone at 5:38 AM on June 11, 2016 [16 favorites]
posted by nebulawindphone at 5:38 AM on June 11, 2016 [16 favorites]
well, if they're that incompetent, why even bother with what the studies say?
posted by pyramid termite at 5:48 AM on June 11, 2016 [29 favorites]
posted by pyramid termite at 5:48 AM on June 11, 2016 [29 favorites]
As this comment points out, the word choice here is really odd. This paper did not research psychosis at all. From the comment: P is positively correlated with tough-mindedness, risk-taking, sensation-seeking, impulsivity, and authoritarianism”– these traits don’t go together; [...] Why lump these traits together and then confusingly call them “psychoticism”?? None of those traits are delusional, hallucinatory, paranoid, disconnected, or anything else that we today think of as psychotic.
posted by blub at 5:59 AM on June 11, 2016 [26 favorites]
posted by blub at 5:59 AM on June 11, 2016 [26 favorites]
Oh, wow, you mean a paper that reads as being deliberately constructed to grab headlines turns out to have methodological flaws? (The straight-up coding errors are an unexpected twist, though, I'll give you that.)
posted by tobascodagama at 6:10 AM on June 11, 2016 [29 favorites]
posted by tobascodagama at 6:10 AM on June 11, 2016 [29 favorites]
is this part of the study?
posted by kokaku at 6:12 AM on June 11, 2016 [26 favorites]
posted by kokaku at 6:12 AM on June 11, 2016 [26 favorites]
*waits to be taken away ha ha*
posted by jonmc at 6:15 AM on June 11, 2016 [2 favorites]
posted by jonmc at 6:15 AM on June 11, 2016 [2 favorites]
Sick burn, brah.
posted by briank at 6:39 AM on June 11, 2016 [2 favorites]
posted by briank at 6:39 AM on June 11, 2016 [2 favorites]
I realize it sounds smug to say so after the fact, but I didn't take this study seriously the first time around, and I don't take it seriously now.
There's an ongoing replication crisis in the social sciences: a substantial fraction of papers published in top journals are turning out to be unreproducible statistical flukes. And my guess is that the less scrupulous the researchers, the more likely they are to work with their university's PR department to puff up their findings. So my default assumption is that any widely-publicized psychology study is probably bunk.
posted by teraflop at 7:17 AM on June 11, 2016 [23 favorites]
There's an ongoing replication crisis in the social sciences: a substantial fraction of papers published in top journals are turning out to be unreproducible statistical flukes. And my guess is that the less scrupulous the researchers, the more likely they are to work with their university's PR department to puff up their findings. So my default assumption is that any widely-publicized psychology study is probably bunk.
posted by teraflop at 7:17 AM on June 11, 2016 [23 favorites]
Isn't this just another Sokal-affair - meaning: the social sciences being totally ignorant when evaluating research from other fields of knowledge.
In my view, this does not really degrade the social sciences (just a little bit ;-)). But it does hint that we should all have a better general education. Which goes against most politicians' view of education.
posted by mumimor at 7:23 AM on June 11, 2016
In my view, this does not really degrade the social sciences (just a little bit ;-)). But it does hint that we should all have a better general education. Which goes against most politicians' view of education.
posted by mumimor at 7:23 AM on June 11, 2016
Maybe the actually study is how we all respond to the study and the revision of the study.
posted by Joey Michaels at 7:51 AM on June 11, 2016 [11 favorites]
posted by Joey Michaels at 7:51 AM on June 11, 2016 [11 favorites]
I was curious who could make such a colossal fuckup in an otherwise dubious semi-science. The flawed study's authors are listed on PubMed as
Lord knows I've screwed up plenty of things in my academic and professional career, so I have some sympathies. And admire their retraction. OTOH, ouch.
posted by Nelson at 8:20 AM on June 11, 2016 [2 favorites]
Brad Verhulst, Virginia Commonwealth University, Virginia Institute of Psychiatric and Behavioral GeneticsMore info on the Virginia Institute for Psychiatric and Behavioral Genetics.
Lindon J. Eaves, Human Genetics, Virginia Commonwealth University, Virginia Institute of Psychiatric and Behavioral Genetics
Peter K. Hatemi, The Pennsylvania State University and Research Fellow at the United States Studies Centre, University of Sydney.
Lord knows I've screwed up plenty of things in my academic and professional career, so I have some sympathies. And admire their retraction. OTOH, ouch.
posted by Nelson at 8:20 AM on June 11, 2016 [2 favorites]
I'm surprised I can't find a (previously) for this. Did it not show up on Metafilter? Did it show up but get deleted by the mods?
posted by clawsoon at 8:21 AM on June 11, 2016
posted by clawsoon at 8:21 AM on June 11, 2016
linked liberalism with social desirability, and conservatism with psychosis
That's what I call broad strokes !
posted by nicolin at 8:26 AM on June 11, 2016 [2 favorites]
That's what I call broad strokes !
posted by nicolin at 8:26 AM on June 11, 2016 [2 favorites]
A study with no serious impact surprises everyone that its authors don't even know what "conservative" means in a political context, or what a political context consists of. Fascinating.
posted by gorgor_balabala at 8:28 AM on June 11, 2016 [2 favorites]
posted by gorgor_balabala at 8:28 AM on June 11, 2016 [2 favorites]
This is the best ever.
posted by chapps at 9:44 AM on June 11, 2016 [2 favorites]
posted by chapps at 9:44 AM on June 11, 2016 [2 favorites]
I'm no more surprised at these retractions than I was at the original findings. People are complicated. If anything, this is a reminder to not use broad strokes when thinking about anyone.
posted by rebent at 10:07 AM on June 11, 2016 [1 favorite]
posted by rebent at 10:07 AM on June 11, 2016 [1 favorite]
Can someone just tell me who I'm supposed to sneer at.
Is it me?
It's probably me.
posted by emjaybee at 10:29 AM on June 11, 2016 [9 favorites]
Is it me?
It's probably me.
posted by emjaybee at 10:29 AM on June 11, 2016 [9 favorites]
I eagerly await the next study on this topic by someone else.
posted by sfts2 at 10:31 AM on June 11, 2016
posted by sfts2 at 10:31 AM on June 11, 2016
You might wait a while; studies with negative results tend not to get published.
posted by biogeo at 11:03 AM on June 11, 2016 [2 favorites]
posted by biogeo at 11:03 AM on June 11, 2016 [2 favorites]
In my teenage-atheist phase, I had a lot of fun being really smug about horoscopes and fortune telling. How could anyone believe, total nonsense, etc. But more and more I've gotten more sympathetic to these epistemically carefree ways of understanding. Maybe we should adopt them once again.
Because what the replication crisis and retractions like this one show is that a conclusion being totally spurious scientifically does not interfere at all with our ability to use it as a way of talking about how some people are like this and other people are like that. Which in turn implies that science was never the point. The point was this kind of... impersonal gossip about what people are like. The science was a spray-painted veneer, a nanometer thick, that no one actually cared about, and which was in fact usually totally invalid anyway. So now we have three options:
1) Deny there is a problem, and just try to do better science. But human nature is so complex the temptations to bias are so huge, and the underlying theories that define inquiry are themselves so doubtful and controversial that anyone who tries is guilty of hubris and the gods will surely strike them down.
2) Cultivate a reverent agnosticism for human nature, and solemnly try to eradicate bias and stereotypes from our thinking, no matter how appealing they seem. Approach each new person as a totally unique individual whose soul has depths beyond all mortal understanding. Come to understand that any patterns we seem to perceive in society are but faces in clouds, vain pretensions to understanding by fragmentary, limited wanderers who cannot even understand themselves. Five or six people on Earth may succeed at this. They will never get invited to any parties.
And finally, the right answer, which is...
3) Folk mysticism. Throw away any pretense of validity, and just adopt an arbitrary system for thinking about how people are based on admittedly non-scientific principles. The rules should be very precise to give a pleasing feeling of structure, but the actual predictions should be so vague that you can interpret them to fit any purpose. Palm-reading. Horoscopes. Magick-with-a-k. I use to say some very mean things about all of you. I'm sorry. I was wrong. You were right.
posted by officer_fred at 11:19 AM on June 11, 2016 [12 favorites]
Because what the replication crisis and retractions like this one show is that a conclusion being totally spurious scientifically does not interfere at all with our ability to use it as a way of talking about how some people are like this and other people are like that. Which in turn implies that science was never the point. The point was this kind of... impersonal gossip about what people are like. The science was a spray-painted veneer, a nanometer thick, that no one actually cared about, and which was in fact usually totally invalid anyway. So now we have three options:
1) Deny there is a problem, and just try to do better science. But human nature is so complex the temptations to bias are so huge, and the underlying theories that define inquiry are themselves so doubtful and controversial that anyone who tries is guilty of hubris and the gods will surely strike them down.
2) Cultivate a reverent agnosticism for human nature, and solemnly try to eradicate bias and stereotypes from our thinking, no matter how appealing they seem. Approach each new person as a totally unique individual whose soul has depths beyond all mortal understanding. Come to understand that any patterns we seem to perceive in society are but faces in clouds, vain pretensions to understanding by fragmentary, limited wanderers who cannot even understand themselves. Five or six people on Earth may succeed at this. They will never get invited to any parties.
And finally, the right answer, which is...
3) Folk mysticism. Throw away any pretense of validity, and just adopt an arbitrary system for thinking about how people are based on admittedly non-scientific principles. The rules should be very precise to give a pleasing feeling of structure, but the actual predictions should be so vague that you can interpret them to fit any purpose. Palm-reading. Horoscopes. Magick-with-a-k. I use to say some very mean things about all of you. I'm sorry. I was wrong. You were right.
posted by officer_fred at 11:19 AM on June 11, 2016 [12 favorites]
There has to be a good German word for the warm smuggly feeling this provides.
codenfraude?
posted by chavenet at 1:25 PM on June 11, 2016 [3 favorites]
codenfraude?
posted by chavenet at 1:25 PM on June 11, 2016 [3 favorites]
There's an ongoing replication crisis in the social sciences...
Isn't this just another Sokal-affair - meaning: the social sciences being totally ignorant when evaluating research from other fields of knowledge.
I'm tired of people thinking these are problems in the social sciences alone. It points to a bias many already had, one that was usually quite elitist and/or sexist. If you're confirming a long-held bias, maybe pause a moment and do further research. Retraction Watch, the website linked to here, is great for this, as is Replicability-Index. Depending on how you decide to interpret their information, your faith in humanity will either be restored or destroyed.
posted by iamfantastikate at 3:53 PM on June 11, 2016 [5 favorites]
Isn't this just another Sokal-affair - meaning: the social sciences being totally ignorant when evaluating research from other fields of knowledge.
I'm tired of people thinking these are problems in the social sciences alone. It points to a bias many already had, one that was usually quite elitist and/or sexist. If you're confirming a long-held bias, maybe pause a moment and do further research. Retraction Watch, the website linked to here, is great for this, as is Replicability-Index. Depending on how you decide to interpret their information, your faith in humanity will either be restored or destroyed.
posted by iamfantastikate at 3:53 PM on June 11, 2016 [5 favorites]
Ahh, broad correlations. The best way to tar your chosen group with all sorts of fun things ... Assuming you don't suck at math.
posted by -1 at 5:38 PM on June 11, 2016
posted by -1 at 5:38 PM on June 11, 2016
But really though, this seems like a classic case of someone finding something that was "too good a story to check out".
posted by -1 at 5:39 PM on June 11, 2016
posted by -1 at 5:39 PM on June 11, 2016
Fehlerforschungfreude ?
posted by Napoleonic Terrier at 6:19 PM on June 11, 2016 [1 favorite]
posted by Napoleonic Terrier at 6:19 PM on June 11, 2016 [1 favorite]
There was this, Cognitive Differences Between Liberals and Conservatives?, previously.
posted by WinstonJulia at 7:39 PM on June 11, 2016
posted by WinstonJulia at 7:39 PM on June 11, 2016
My husband says they probably got the SAS backwards.
I like it because it sound like "ass backwards" but he says there's a quirk in SAS code that means your positive (1) value is reversed from what most studies use.
Or he could just be pranking me and now all the sas coders will come and mock me.
posted by chapps at 8:15 PM on June 11, 2016
I like it because it sound like "ass backwards" but he says there's a quirk in SAS code that means your positive (1) value is reversed from what most studies use.
Or he could just be pranking me and now all the sas coders will come and mock me.
posted by chapps at 8:15 PM on June 11, 2016
When you learn about a study which seems to say nice things about you and less nice things about people unlike you, it's probably a good idea to stop and imagine them eventually discovering they'd reversed the coding, before sharing it with people.
posted by edheil at 10:34 PM on June 11, 2016 [4 favorites]
posted by edheil at 10:34 PM on June 11, 2016 [4 favorites]
In related news, it turns out Virginia is for Haters.
posted by Sys Rq at 5:19 PM on June 12, 2016 [8 favorites]
posted by Sys Rq at 5:19 PM on June 12, 2016 [8 favorites]
Studies like this one seemed to pop up pretty frequently back when Bush was president. Maybe there weren't many studies, but sites like The Guardian and Salon and Daily Kos and the like certainly delighted in spreading them around. It was an odd way to cope with the times. "The Republicans may control all three branches of government, but we're still smarter/kinder/better people, so there!"
It's not a healthy way to live, to carry the attitude that half of your fellow citizens are idiots or physcos. One good thing about this news is it's going to throw doubt on any similar study.
posted by riruro at 9:09 PM on June 12, 2016 [3 favorites]
It's not a healthy way to live, to carry the attitude that half of your fellow citizens are idiots or physcos. One good thing about this news is it's going to throw doubt on any similar study.
posted by riruro at 9:09 PM on June 12, 2016 [3 favorites]
Personality systems like this are actually something I mostly took an interest in because of trying to come up with deeper characters for roleplaying games, it's not like I'm a trained psychologist or something, so maybe I've got some huge part of this wrong? But the one thing this seems to suggest to me as far as useful knowledge is that Eysenck, who had right-wing affiliations, was not particularly unbiased about what he considered to be positive personality traits. The personality traits for his idea of psychoticism are framed in terms of "non-acceptance of cultural norms" or "disregard for common sense"... at which point it seems like it should have been worth asking where those cultural norms came from and who decided what was "common sense". In other words, a subjective scale for antisocial personality traits is naturally going to have liberals score high if the author was conservative, or conservatives score high if the author was liberal. We all have our own ideas about what constitute appropriate social engagement.
This goes a step beyond "some old idea of what psychosis meant"--Eysenck really thought these personality traits were going to predict schizophrenia and such, and I don't think there's actually any such link. There are so many better personality scales out there, I don't know why you'd pick this one unless you were really trying to publish something inflammatory about how group A is better than group B.
posted by Sequence at 5:23 AM on June 13, 2016 [2 favorites]
This goes a step beyond "some old idea of what psychosis meant"--Eysenck really thought these personality traits were going to predict schizophrenia and such, and I don't think there's actually any such link. There are so many better personality scales out there, I don't know why you'd pick this one unless you were really trying to publish something inflammatory about how group A is better than group B.
posted by Sequence at 5:23 AM on June 13, 2016 [2 favorites]
« Older 3 heads of the Black Dog, decades of post-techno... | Would humans remain human on Mars? Newer »
This thread has been archived and is closed to new comments
posted by backwards compatible at 4:48 AM on June 11, 2016 [17 favorites]