Another scandal in academic psychology
August 22, 2013 5:49 PM Subscribe
Most work in the psychological and social sciences suffers from a lack of conceptual rigor. It’s a bit sloppy around the edges, and in the middle, too. For example, “happiness research” is a booming field, but the titans of the subdiscipline disagree sharply about what happiness actually is. No experiment or regression will settle it. It’s a philosophical question. Nevertheless, they work like the dickens to measure it, whatever it is—life satisfaction, “flourishing,” pleasure minus pain—and to correlate it to other, more easily quantified things with as much statistical rigor as deemed necessary to appear authoritative. It’s as if the precision of the statistical analysis is supposed somehow to compensate for, or help us forget, the imprecision of thought at the foundation of the enterprise.
I was wondering what Alan Sokal had been up to recently...
posted by googly at 6:12 PM on August 22, 2013 [1 favorite]
posted by googly at 6:12 PM on August 22, 2013 [1 favorite]
(Post)positivism is not the only way to understand positivity, as it happens. The assumption that there is one objective truth about any psychological phenomenon is a pretty big philosophical assumption. There are multiple ways to understand phenomena, and the fact that different people claim that their way is "best" is a feature, not a bug, IMO. It spurs discussion, debate, further refinement/clarification of problem definitions, etc.
It is the case, on the other hand, that social scientists (and laypeople who write/think about social science) tend to default to postpositivist standards of rigor, like reliability/validity, regardless of whether a given inquiry actually makes sense through a postpositivist lens. Honestly, the lack of training on the philosophy of science, and this default postpositivist stance associated with the lack of training, are way bigger "scandals" in social science than the fact that not every social scientist will be going for capital-T-Truth in their work.
posted by quiet coyote at 6:12 PM on August 22, 2013 [5 favorites]
It is the case, on the other hand, that social scientists (and laypeople who write/think about social science) tend to default to postpositivist standards of rigor, like reliability/validity, regardless of whether a given inquiry actually makes sense through a postpositivist lens. Honestly, the lack of training on the philosophy of science, and this default postpositivist stance associated with the lack of training, are way bigger "scandals" in social science than the fact that not every social scientist will be going for capital-T-Truth in their work.
posted by quiet coyote at 6:12 PM on August 22, 2013 [5 favorites]
Here's the arXiv entry for the debunking paper. It's written for a general audience; it presumes no knowledge of the mathematics underlying the work that it is debunking. And then it proceeds to fucking demolish that work. A thing of beauty.
That this stuff was published originally is frankly embarrassing. That it has been cited hundreds of times is scandalous.
posted by mr_roboto at 6:15 PM on August 22, 2013 [14 favorites]
That this stuff was published originally is frankly embarrassing. That it has been cited hundreds of times is scandalous.
posted by mr_roboto at 6:15 PM on August 22, 2013 [14 favorites]
I'm a PhD student (not in psychology) and we get lots of information about how to deal with Impostor Syndrome. "Do you feel like you are about to be unmasked as a fraud?," the brochures say. "Do you believe that you're simply scamming everyone by pretending you have mastered skills which you do not, in fact, possess? Don't worry! Everyone feels that way! You are almost certainly very smart! You must simply buck up, increase your confidence, and carry on!"
I experience occasional bouts of Imposter Syndrome, but I also suffer from its more disturbing and lesser-studied twin, Everybody Else Is An Imposter Syndrome, where I read some to-me-impenetrable paper and I think, this person doesn't know what they're talking about. They just made up a bunch of gobbeldygook and trusted that nobody would call them on it. And for a minute, I feel dizzyingly certain that not only is this true, but everyone else above me knows that it's true, and that there's a covert agreement among all the leading lights of my discipline to simply pretend that everything everyone else is saying makes sense when it doesn't, and that it's not that people are confused, or bad writers, or illogical thinkers and desperately trying to hide it, it's that that they actually know exactly what they're doing and they're enjoying themselves immensely. "You'll never believe the nosebleed-inducing nonsense I'm planning to present at the annual conference next year," tenured faculty whisper to each other, cackling. "The graduate students are going to to pee themselves trying to figure it out." It's like a secret club, and the handshake will be revealed to me at my dissertation defense, and afterwards we'll all chuckle to ourselves in the back room over whiskey and cigars.
It's not surprising to me that this 'scandal' was broken by a grad student, in other words, because obviously nobody's clued him in to the way things work yet. He wasn't scheduled to be inducted via the secret ceremony until next week. And Alan Sokol isn't the brilliant, emperor-has-no-clothes truth-teller everyone outside the academy takes him for; he's actually a dull, literal-minded wet blanket whom none of the other professors want to invite to their parties.
posted by pretentious illiterate at 6:34 PM on August 22, 2013 [46 favorites]
I experience occasional bouts of Imposter Syndrome, but I also suffer from its more disturbing and lesser-studied twin, Everybody Else Is An Imposter Syndrome, where I read some to-me-impenetrable paper and I think, this person doesn't know what they're talking about. They just made up a bunch of gobbeldygook and trusted that nobody would call them on it. And for a minute, I feel dizzyingly certain that not only is this true, but everyone else above me knows that it's true, and that there's a covert agreement among all the leading lights of my discipline to simply pretend that everything everyone else is saying makes sense when it doesn't, and that it's not that people are confused, or bad writers, or illogical thinkers and desperately trying to hide it, it's that that they actually know exactly what they're doing and they're enjoying themselves immensely. "You'll never believe the nosebleed-inducing nonsense I'm planning to present at the annual conference next year," tenured faculty whisper to each other, cackling. "The graduate students are going to to pee themselves trying to figure it out." It's like a secret club, and the handshake will be revealed to me at my dissertation defense, and afterwards we'll all chuckle to ourselves in the back room over whiskey and cigars.
It's not surprising to me that this 'scandal' was broken by a grad student, in other words, because obviously nobody's clued him in to the way things work yet. He wasn't scheduled to be inducted via the secret ceremony until next week. And Alan Sokol isn't the brilliant, emperor-has-no-clothes truth-teller everyone outside the academy takes him for; he's actually a dull, literal-minded wet blanket whom none of the other professors want to invite to their parties.
posted by pretentious illiterate at 6:34 PM on August 22, 2013 [46 favorites]
For lack of a less expensive and convenient alternative, academic psychologists mainly study the inner workings of American college students, observing them in labs and paying them to fill out surveys. While this method is sure to reveal interesting truths about the distinctive cultural milieu of the American college student, it is unlikely to uncover solid, universal truths about the psychology of Homo sapiens. A number of recent studies suggest that the rich, well-educated youths of industrialized Western democracies who make up the subject pool of most psychological research are not actually the generic, typical humans that the rich, well-educated elders of industrialized Western democracies take them to be.
This is a fascinating idea. Anyone know which studies he is referring to here?
posted by 2bucksplus at 6:36 PM on August 22, 2013 [1 favorite]
This is a fascinating idea. Anyone know which studies he is referring to here?
posted by 2bucksplus at 6:36 PM on August 22, 2013 [1 favorite]
Can you imagine anything sketchier than saying in the subtitle that your book is based on "top-notch research"? It's like a guy who calls himself "classy".
posted by Horace Rumpole at 6:37 PM on August 22, 2013 [10 favorites]
posted by Horace Rumpole at 6:37 PM on August 22, 2013 [10 favorites]
Psychology is not my field, but I have never read a published paper as brutal as the one linked by mr_roboto above.
I am somewhat in disbelief, to be honest. There is no "great insight" in that paper, the mistakes, misunderstandings, and general sloppiness of the work being criticized are almost unbelievable.
I have heard from time to time of the lack of rigor in psychology, but I can't help but think that this is an egregious example, and not the norm. But the mention of "366 citations" gives me pause.
posted by LoopyG at 6:39 PM on August 22, 2013 [5 favorites]
I am somewhat in disbelief, to be honest. There is no "great insight" in that paper, the mistakes, misunderstandings, and general sloppiness of the work being criticized are almost unbelievable.
I have heard from time to time of the lack of rigor in psychology, but I can't help but think that this is an egregious example, and not the norm. But the mention of "366 citations" gives me pause.
posted by LoopyG at 6:39 PM on August 22, 2013 [5 favorites]
One article that discusses the lack of generalizability of research on US college students is available at this link [PDF; many pages long]. Because this was published in Behavioral and Brain Sciences, it comes with lots of replies to the original article, as well as replies to the replies. Lots of thoughtful stuff in here.
posted by anaphoric at 6:45 PM on August 22, 2013 [3 favorites]
posted by anaphoric at 6:45 PM on August 22, 2013 [3 favorites]
Honestly, the lack of training on the philosophy of science, and this default postpositivist stance associated with the lack of training, are way bigger "scandals" in social science than the fact that not every social scientist will be going for capital-T-Truth in their work.
You haven't read any of the links, have you?
posted by mr_roboto at 6:45 PM on August 22, 2013 [1 favorite]
You haven't read any of the links, have you?
posted by mr_roboto at 6:45 PM on August 22, 2013 [1 favorite]
@ quiet coyote There! You're doing it! Aren't you? Aren't you?? Everybody stop messing with me!
posted by pretentious illiterate at 6:47 PM on August 22, 2013
posted by pretentious illiterate at 6:47 PM on August 22, 2013
I appreciate the direct link to the paper. Fun read so far.
Anyone else getting an extremely "written in the library during lunch" and "oh no there's a minimum word count" feel to this blog post? Not really sure if it adds... anything.
posted by a birds at 6:49 PM on August 22, 2013
Anyone else getting an extremely "written in the library during lunch" and "oh no there's a minimum word count" feel to this blog post? Not really sure if it adds... anything.
posted by a birds at 6:49 PM on August 22, 2013
From the paper: when the authors' restraint finally gives out.
Instead, we invite the reader to contemplate the implications of the third and fourth sentences. They appear to assert that the predictive use of differential equations abstracted from a domain of the natural sciences to describe human interactions can be justified on the basis of the linguistic similarity between elements of the technical vocabulary of that scientific domain and the adjectives used metaphorically by a particular observer to describe those human interactions. If true, this would have remarkable implications for the social sciences. One could describe a team’s interactions as “sparky” and confidently predict that their emotions would be subject to the same laws that govern the dielectric breakdown of air under the influence of an electric field. Alternatively, the interactions of a team of researchers whose journal articles are characterized by “smoke and mirrors” could be modeled using the physics of airborne particulate combustion residues, combined in some way with classical optics.posted by tss at 6:55 PM on August 22, 2013 [27 favorites]
I busted up out load at that point tss. Drew some of my coworkers into my reading when I did =)
posted by LoopyG at 7:03 PM on August 22, 2013
posted by LoopyG at 7:03 PM on August 22, 2013
But the mention of "366 citations" gives me pause.
I actually looked at a bunch of the papers that cited the Frederickson-Losada paper, and most of the ones I saw seemed to be citing it for the general finding that people did better when they had a higher ratio of positive to negative interactions, not the mystical exact-to-five-decimal-places tipping point that Brown ably disposes of in his paper. So I didn't get the sense that there was a big industry in psychology built around the existence of the thing that, well, doesn't exist.
posted by escabeche at 7:31 PM on August 22, 2013 [4 favorites]
I actually looked at a bunch of the papers that cited the Frederickson-Losada paper, and most of the ones I saw seemed to be citing it for the general finding that people did better when they had a higher ratio of positive to negative interactions, not the mystical exact-to-five-decimal-places tipping point that Brown ably disposes of in his paper. So I didn't get the sense that there was a big industry in psychology built around the existence of the thing that, well, doesn't exist.
posted by escabeche at 7:31 PM on August 22, 2013 [4 favorites]
Good to hear escabeche. Is such a claim not made in other papers, perhaps with more justification? Or is the Losada paper the one cited due to a popularity effect: some folks cited it, then it became the "go-to" citation for that claim?
I agree with Brown et al that the claim of higher P/N (almost however you define it) correlating with higher performance in life (again, almost however you define it) is not controversial. Not that it's not worth testing!
posted by LoopyG at 7:40 PM on August 22, 2013
I agree with Brown et al that the claim of higher P/N (almost however you define it) correlating with higher performance in life (again, almost however you define it) is not controversial. Not that it's not worth testing!
posted by LoopyG at 7:40 PM on August 22, 2013
That being said, though, Frederickson certainly did put it in her pop-psych book, so she has something to be embarrassed about there.
posted by escabeche at 7:43 PM on August 22, 2013
posted by escabeche at 7:43 PM on August 22, 2013
In my limited experience I feel that this has a lot to do with tightening funding, since it's harder to get money for stuff that isn't based in (said in God's booming voice) FANCY MATH.
posted by codacorolla at 7:57 PM on August 22, 2013
posted by codacorolla at 7:57 PM on August 22, 2013
I think that there are a lot of psych researchers out there that aren't terribly good at math but would like to use the work of colleagues that seems to be mathematically sophisticated. It's a little embarrassing but I have hope that the future of research psychology will be better.
posted by a snickering nuthatch at 8:43 PM on August 22, 2013
posted by a snickering nuthatch at 8:43 PM on August 22, 2013
Seymour Zamboni: "I never thought such a high level of unmitigated bullshit could ever make it through a peer review process."
Sokal probably wasn't too surprised.
posted by Proofs and Refutations at 8:45 PM on August 22, 2013
Sokal probably wasn't too surprised.
posted by Proofs and Refutations at 8:45 PM on August 22, 2013
Losada is not a real sociologist, he is a corporate scientist based out of Ann Arbor. Sokal's paper can't and won't make this attributive leap, because it would be unscientific, and also undiplomatic, etc (and technically, an ad-hominem). But with a little additional context, the low academic standard of his papers should not be surprising at all.
posted by polymodus at 8:56 PM on August 22, 2013 [3 favorites]
posted by polymodus at 8:56 PM on August 22, 2013 [3 favorites]
That's interesting to learn. I wonder how many of the citations would fall under the "obviously junk" category, too?
posted by a birds at 9:45 PM on August 22, 2013
posted by a birds at 9:45 PM on August 22, 2013
nope can't figure out this Web of Knowledge thing fast enough but it's probably what i wanted. good job academia
posted by a birds at 9:54 PM on August 22, 2013
posted by a birds at 9:54 PM on August 22, 2013
Losada is not a real sociologist, he is a corporate scientist based out of Ann Arbor. Sokal's paper can't and won't make this attributive leap, because it would be unscientific, and also undiplomatic, etc (and technically, an ad-hominem). But with a little additional context, the low academic standard of his papers should not be surprising at all.
I'm not following you. Is it common knowledge that industry scientists working in social psychology do subpar work? This certainly isn't the case with the fields I'm familiar with, in which publications from industry might be a little mundane, but are typically at least competent.
And the papers were subject to peer review, which should have caught their utter absurdity. And then all the people who cited them...
posted by mr_roboto at 10:47 PM on August 22, 2013
I'm not following you. Is it common knowledge that industry scientists working in social psychology do subpar work? This certainly isn't the case with the fields I'm familiar with, in which publications from industry might be a little mundane, but are typically at least competent.
And the papers were subject to peer review, which should have caught their utter absurdity. And then all the people who cited them...
posted by mr_roboto at 10:47 PM on August 22, 2013
Yeah, that "expansive emotional space" thing really came across like a lot of (pardon) hot air. It's a little tricky in that it sounds like it's just figurative, simply meant to make the concepts a bit more intuitive to the reader. However, it's actually a reasonable literal description of the hypothesized phenomenon, which makes it really silly. I can't read it without this picture coming to mind.
posted by Riki tiki at 11:24 PM on August 22, 2013
posted by Riki tiki at 11:24 PM on August 22, 2013
I'm not following you. Is it common knowledge that industry scientists working in social psychology do subpar work? This certainly isn't the case with the fields I'm familiar with, in which publications from industry might be a little mundane, but are typically at least competent.
Surely any profession with the ability to fudge figures and get away with it, combined with incentives, will tend to exhibit more corruption than others. The fact that the social sciences makes fudging easy is pretty lamentable, and it's certainly common knowledge that industry aligned social scientists are to be viewed with extreme prejudice.
I would like to know what rock you live under that you don't associate, for example, industry aligned chemists, geologists and doctors with corruption. I would like to join you, occasionally. I will bring curry and ginger beer when I do.
And the papers were subject to peer review, which should have caught their utter absurdity. And then all the people who cited them...
Let me be clear that I'm not defending the fraud in the social sciences here, but considering what you said immediately prior I do think it's worth pointing out that the strong impression I've gotten from articles linked from here, and if I recall correctly commenters here, is that fraud happens where fraud can happen. And the peer review system in all fields is far from perfect.
Which is why you get things like authority trumping verification, backwards incentives in universities producing fraudulent papers, backwards incentives in nations producing fraudulent papers, peers highlighting peers papers while ignoring others, increasing numbers of withdrawn papers, and so on.
Basically, backwards incentives cause problems. And it's much easier to capitalize on that if you're in the social sciences.
posted by tychotesla at 11:34 PM on August 22, 2013 [2 favorites]
Surely any profession with the ability to fudge figures and get away with it, combined with incentives, will tend to exhibit more corruption than others. The fact that the social sciences makes fudging easy is pretty lamentable, and it's certainly common knowledge that industry aligned social scientists are to be viewed with extreme prejudice.
I would like to know what rock you live under that you don't associate, for example, industry aligned chemists, geologists and doctors with corruption. I would like to join you, occasionally. I will bring curry and ginger beer when I do.
And the papers were subject to peer review, which should have caught their utter absurdity. And then all the people who cited them...
Let me be clear that I'm not defending the fraud in the social sciences here, but considering what you said immediately prior I do think it's worth pointing out that the strong impression I've gotten from articles linked from here, and if I recall correctly commenters here, is that fraud happens where fraud can happen. And the peer review system in all fields is far from perfect.
Which is why you get things like authority trumping verification, backwards incentives in universities producing fraudulent papers, backwards incentives in nations producing fraudulent papers, peers highlighting peers papers while ignoring others, increasing numbers of withdrawn papers, and so on.
Basically, backwards incentives cause problems. And it's much easier to capitalize on that if you're in the social sciences.
posted by tychotesla at 11:34 PM on August 22, 2013 [2 favorites]
it seems a priori unlikely that the emotions of business teams composed of normal individuals would oscillate wildly and continuously throughout a one-hour meeting in the manner of the Lorenz attractor.
The authors clearly do not work for my employer.
posted by problemspace at 12:52 AM on August 23, 2013 [2 favorites]
The authors clearly do not work for my employer.
posted by problemspace at 12:52 AM on August 23, 2013 [2 favorites]
Here's another paper by Joe Heinrich on WEIRD people.
Western, Educated, Industrialized country, Rich (compared to Bangledesh), Democratic government.
posted by justsomebodythatyouusedtoknow at 1:04 AM on August 23, 2013
Western, Educated, Industrialized country, Rich (compared to Bangledesh), Democratic government.
posted by justsomebodythatyouusedtoknow at 1:04 AM on August 23, 2013
Tangent: backwards incentives? I get the idea, but it's a curious phrase.
posted by quiet earth at 1:09 AM on August 23, 2013
posted by quiet earth at 1:09 AM on August 23, 2013
I think I wrote that because four thoughts and one flaw came together suddenly:
First it seems analogous to the Positive Incentive vs. Negative Incentive premodifier form that already exists. Second, because I had a strong suspicion that there is a phrase meaning "incentive that results in the opposite of the goal desired" (not sure anymore). Third, the word "Backward" is associated with a lack of science, as well as meaning opposite from the expected or desired. Fourth, despite being intelligent, decently read and impulsively reflective, I do not words and good sentence. Or punctuation, whom. Fifth, I am tired and have rats in my apartment wall and their squeaking was making me write faster because I was flexing to slam a cupboard door to shut them up god damn.
posted by tychotesla at 2:10 AM on August 23, 2013 [4 favorites]
First it seems analogous to the Positive Incentive vs. Negative Incentive premodifier form that already exists. Second, because I had a strong suspicion that there is a phrase meaning "incentive that results in the opposite of the goal desired" (not sure anymore). Third, the word "Backward" is associated with a lack of science, as well as meaning opposite from the expected or desired. Fourth, despite being intelligent, decently read and impulsively reflective, I do not words and good sentence. Or punctuation, whom. Fifth, I am tired and have rats in my apartment wall and their squeaking was making me write faster because I was flexing to slam a cupboard door to shut them up god damn.
posted by tychotesla at 2:10 AM on August 23, 2013 [4 favorites]
Ah, the Lovecraft defense.
posted by benzenedream at 3:09 AM on August 23, 2013 [6 favorites]
posted by benzenedream at 3:09 AM on August 23, 2013 [6 favorites]
On the question of peer review: It is far from perfect. For instance, I had a abstract for a conference paper rejected this week, mostly on the grounds of being unoriginal - one of the reviewers referred me to a PhD thesis that had already covered the issues I was addressing. Which was helpful given that I'd spent 4 years writing that thesis, and while I'm not totally across it yet I have a reasonable sense of what the author is arguing.
I'm not a psych or social science researcher, so don't really have a dog in the quantitative research fight, but I do think it's going to generally be problematic when you go from using theories and concepts from the physical sciences as a metaphor (which in itself is fraught with danger) to actually using the underlying mathematical models to explain the phenomena you're studying.
posted by Hello, I'm David McGahan at 3:28 AM on August 23, 2013 [4 favorites]
I'm not a psych or social science researcher, so don't really have a dog in the quantitative research fight, but I do think it's going to generally be problematic when you go from using theories and concepts from the physical sciences as a metaphor (which in itself is fraught with danger) to actually using the underlying mathematical models to explain the phenomena you're studying.
posted by Hello, I'm David McGahan at 3:28 AM on August 23, 2013 [4 favorites]
Seymour Zamboni: "I never thought such a high level of unmitigated bullshit could ever make it through a peer review process."
Sokal probably wasn't too surprised.
Sokal submitted his hoax paper to a non-peer reviewed journal. From your link: In 1996, Social Text did not conduct peer review because its editors believed that an open editorial policy would stimulate more original, less conventional research.
posted by Elementary Penguin at 3:59 AM on August 23, 2013 [1 favorite]
Sokal probably wasn't too surprised.
Sokal submitted his hoax paper to a non-peer reviewed journal. From your link: In 1996, Social Text did not conduct peer review because its editors believed that an open editorial policy would stimulate more original, less conventional research.
posted by Elementary Penguin at 3:59 AM on August 23, 2013 [1 favorite]
I'd like to take a moment to ask people not to come away with the impression that the sight of a differential equation in a psychology paper implies dreamy analogizing with a physical science. There's a lot of rigorous, solid work going on by people who understand their models and their data.
posted by spbmp at 7:06 AM on August 23, 2013
posted by spbmp at 7:06 AM on August 23, 2013
Attaching scientific rigor to a subjective concept is just dumb. Or the great fakeroo, college students picking out photos of faces tells us who people are mating with.
posted by Ironmouth at 8:29 AM on August 23, 2013
posted by Ironmouth at 8:29 AM on August 23, 2013
Attaching scientific rigor to a subjective concept is just dumb
Some bad science does not equal science being impossible any more than some inane criticism invalidates all criticism.
posted by srboisvert at 9:28 AM on August 23, 2013 [2 favorites]
Some bad science does not equal science being impossible any more than some inane criticism invalidates all criticism.
posted by srboisvert at 9:28 AM on August 23, 2013 [2 favorites]
Surely any profession with the ability to fudge figures and get away with it, combined with incentives, will tend to exhibit more corruption than others...
...that fraud happens where fraud can happen...
Except this wasn't a case of fudged figures or fraudulent work. It was plainly and transparently incompetent. For all we know, Losada is simply a fool who believed his analysis was valid.
...it's certainly common knowledge that industry aligned social scientists are to be viewed with extreme prejudice.
I would love to see some data on this. I tried to do some quick research on scientific misconduct from academic vs. industrial scientists. I don't believe anything had been published on this, yet, but the misconduct literature is just beginning to get of the ground. I'm looking forward to the upcoming special issue of Publications edited by Grant Steen.
I did a quick informal review of the first twenty entries on Retraction Watch. They were all from researchers who at the time they published the retracted article were at academic institutions. That said, industrial researchers publish far less, so we really need to compare rates of retractions. This is definitely doable, but not a morning's job, and I don't have time for it right now.
Anecdotally, however, the vast majority of prominent cases of scientific fraud in recent years have been related to academic labs or researchers. Potti, Stapel, Hauser, and Hwang were all university-affiliated researchers. The only major scandal I can think of in recent years not originating in a university is the Schön affair, but the incentives for misconduct at Bell labs were much more similar to those in a university environment (reputation, prominence) than the profit motives you cite.
In fact, it seems to me in general that the motives to publish questionable work are much stronger in Academia than in Industry. After all, industrial profits come from selling goods and services, not publishing. In the academy, publishing is everything; a career hinges upon a strong publication record.
I would like to know what rock you live under...
It's called empiricism, and you're more than welcome to join us.
posted by mr_roboto at 10:17 AM on August 23, 2013 [2 favorites]
...that fraud happens where fraud can happen...
Except this wasn't a case of fudged figures or fraudulent work. It was plainly and transparently incompetent. For all we know, Losada is simply a fool who believed his analysis was valid.
...it's certainly common knowledge that industry aligned social scientists are to be viewed with extreme prejudice.
I would love to see some data on this. I tried to do some quick research on scientific misconduct from academic vs. industrial scientists. I don't believe anything had been published on this, yet, but the misconduct literature is just beginning to get of the ground. I'm looking forward to the upcoming special issue of Publications edited by Grant Steen.
I did a quick informal review of the first twenty entries on Retraction Watch. They were all from researchers who at the time they published the retracted article were at academic institutions. That said, industrial researchers publish far less, so we really need to compare rates of retractions. This is definitely doable, but not a morning's job, and I don't have time for it right now.
Anecdotally, however, the vast majority of prominent cases of scientific fraud in recent years have been related to academic labs or researchers. Potti, Stapel, Hauser, and Hwang were all university-affiliated researchers. The only major scandal I can think of in recent years not originating in a university is the Schön affair, but the incentives for misconduct at Bell labs were much more similar to those in a university environment (reputation, prominence) than the profit motives you cite.
In fact, it seems to me in general that the motives to publish questionable work are much stronger in Academia than in Industry. After all, industrial profits come from selling goods and services, not publishing. In the academy, publishing is everything; a career hinges upon a strong publication record.
I would like to know what rock you live under...
It's called empiricism, and you're more than welcome to join us.
posted by mr_roboto at 10:17 AM on August 23, 2013 [2 favorites]
There is no doubt the paper is junk, but not for the full set of reasons Wilkinson says. He is using too broad a brush.
Often mathmatical techniques developed in one area are successfully applied to others. E.g., inferential statistics started in agriculture; ways to look at networks have been used in all sorts of areas.
As LoopyG said, the simple finding of Fredrickson and Losada that more "positive" statements correlate with some particular measure of social health isn't controversial (nor does it contribute much).
It is fine that they coded statements as to how "positive" or "negative" they were. This is a useful and widely applied method (though I do not claim that it was used here properly or not).
It *may* be ok that the dependent variable is light on operational or theoretical grounding. Successful areas of research often start with fuzzy concepts that get refined over time.
The problem is that they tried to make elaborate claims about the relationship between these two variables using a mathmatical technique which requires a lot more clarity, and better operationalization and measurement, about the hypothesized forces and flows.
It might, in theory, make sense to at some point try to use mathmatical techniques from fluid dynamics to derive insights about social behavior and emotional states. But as this paper shows, the area needs to develop a lot more before something like that could work.
posted by neutralmojo at 10:37 AM on August 23, 2013 [1 favorite]
Often mathmatical techniques developed in one area are successfully applied to others. E.g., inferential statistics started in agriculture; ways to look at networks have been used in all sorts of areas.
As LoopyG said, the simple finding of Fredrickson and Losada that more "positive" statements correlate with some particular measure of social health isn't controversial (nor does it contribute much).
It is fine that they coded statements as to how "positive" or "negative" they were. This is a useful and widely applied method (though I do not claim that it was used here properly or not).
It *may* be ok that the dependent variable is light on operational or theoretical grounding. Successful areas of research often start with fuzzy concepts that get refined over time.
The problem is that they tried to make elaborate claims about the relationship between these two variables using a mathmatical technique which requires a lot more clarity, and better operationalization and measurement, about the hypothesized forces and flows.
It might, in theory, make sense to at some point try to use mathmatical techniques from fluid dynamics to derive insights about social behavior and emotional states. But as this paper shows, the area needs to develop a lot more before something like that could work.
posted by neutralmojo at 10:37 AM on August 23, 2013 [1 favorite]
There is next to zero basic social science research being done privately.
posted by srboisvert at 10:53 AM on August 23, 2013
posted by srboisvert at 10:53 AM on August 23, 2013
The closest thing I can think of are think tanks which are quasi-academic and often absurdly partisan and ridiculous. But then they are mostly economists which .....
posted by srboisvert at 10:54 AM on August 23, 2013
posted by srboisvert at 10:54 AM on August 23, 2013
After all, industrial profits come from selling goods and services, not publishing. In the academy, publishing is everything; a career hinges upon a strong publication record.
In academy, published papers (and the career advancement and passport to the bourgeois that goes along with them) are the product. If you can publish bullshit papers, you can ship more product.
posted by Golden Eternity at 11:04 AM on August 23, 2013
In academy, published papers (and the career advancement and passport to the bourgeois that goes along with them) are the product. If you can publish bullshit papers, you can ship more product.
posted by Golden Eternity at 11:04 AM on August 23, 2013
Independent confirmation of the happiness ratio courtesy of Achewood's Raymond Smuckles
posted by codacorolla at 11:24 AM on August 23, 2013 [1 favorite]
posted by codacorolla at 11:24 AM on August 23, 2013 [1 favorite]
Bahahaha, I just left the UNC Psychology Department for the UNC Statistics Department! Major factor was the lack of mathematical rigor in psychology! :D
posted by zscore at 12:12 PM on August 23, 2013
posted by zscore at 12:12 PM on August 23, 2013
zscore: "Bahahaha, I just left the UNC Psychology Department for the UNC Statistics Department! Major factor was the lack of mathematical rigor in psychology! :D"
One of the big points in the refuting piece was that - even with outstanding statistical rigor - the failure here was primarily one of conceptual rigor. You can decide, on that basis, that the study of social phenomena is inherently without rigor - I suppose. But the consequences of that decision (never studying social phenomena) are pretty dire, wouldn't you say?
posted by Apropos of Something at 3:58 PM on August 23, 2013 [1 favorite]
One of the big points in the refuting piece was that - even with outstanding statistical rigor - the failure here was primarily one of conceptual rigor. You can decide, on that basis, that the study of social phenomena is inherently without rigor - I suppose. But the consequences of that decision (never studying social phenomena) are pretty dire, wouldn't you say?
posted by Apropos of Something at 3:58 PM on August 23, 2013 [1 favorite]
« Older Meet the Town That's Being Swallowed by a Sinkhole | Funny, he seemed like such a normal fellow... a... Newer »
This thread has been archived and is closed to new comments
posted by Apropos of Something at 6:11 PM on August 22, 2013 [6 favorites]