Your Happy Ending
March 12, 2013 8:25 AM Subscribe
Allegations of flawed research techniques at an NIH-funded medical lab at Johns Hopkins get notice in a Washington Post article. Interesting piece on a scientific dispute, the accuser's loss of his job at Hopkins, and the suicide of one researcher from the lab whose analysis, published in Nature, came into question.
Or, more accurately, under review for publication in Nature.
No, it was published.
Weird, messy story (especially with Yuan having been fired--the fact that he has such an obvious motive for disgruntlement doesn't help). Whatever the rights and wrongs of this I think Nature have to publish some kind of follow-up to clarify the issue.
posted by yoink at 8:58 AM on March 12, 2013 [1 favorite]
No, it was published.
Weird, messy story (especially with Yuan having been fired--the fact that he has such an obvious motive for disgruntlement doesn't help). Whatever the rights and wrongs of this I think Nature have to publish some kind of follow-up to clarify the issue.
posted by yoink at 8:58 AM on March 12, 2013 [1 favorite]
I think it is a correction that is still under review, at least according the hopkins pr people. (From what other parties were quoted as saying, it is a little unclear to me if one has actually been submitted.)
posted by advil at 9:04 AM on March 12, 2013
posted by advil at 9:04 AM on March 12, 2013
single page version. (the multipage article was loading slowly and awkwardly for me, but YMMV).
posted by sciencegeek at 9:05 AM on March 12, 2013
posted by sciencegeek at 9:05 AM on March 12, 2013
Also the note that Yuan was not allowed to take his cell lines is interesting - the journalist is making this seem like it is something the reader should be alarmed about.
Lines are usually property of the lab/institution in which they are made. If they were published, Yuan would have the right to request copies from Boeke's lab and the lab should give them to him. If not, he doesn't have the right to take them.
posted by sciencegeek at 9:10 AM on March 12, 2013
Lines are usually property of the lab/institution in which they are made. If they were published, Yuan would have the right to request copies from Boeke's lab and the lab should give them to him. If not, he doesn't have the right to take them.
posted by sciencegeek at 9:10 AM on March 12, 2013
This was a story that could have been much more engaging, but it was written in a confusing manner. I'm interested to see the correction, if it ever gets published in Nature. Hard for me to tell what really went on here, unfortunately.
I wish the article had made a better case about what was supposedly wrong. "The methodology was flawed" is not specific, and leaves me guessing about what really happened here.
Still, interesting post, and I'll be on the lookout for updates.
posted by k8lin at 9:18 AM on March 12, 2013
I wish the article had made a better case about what was supposedly wrong. "The methodology was flawed" is not specific, and leaves me guessing about what really happened here.
Still, interesting post, and I'll be on the lookout for updates.
posted by k8lin at 9:18 AM on March 12, 2013
The article is suitably detailed about the paper's alleged problems, in my opinion: "the numerical threshold the investigators used for determining when a hit had arisen was too low". In other words, the paper allegedly contains an excess number of type I errors, also known as false positives or errors of excess credulity, a common potential pitfall when considering the statistical significance of up- or down-regulation of thousands of genes. Remember that the Post is not a scientific journal.
posted by Mapes at 9:28 AM on March 12, 2013 [1 favorite]
posted by Mapes at 9:28 AM on March 12, 2013 [1 favorite]
Not much to guess, really - stats geek was sidelined and then fired for uncovering significant fraud in a lab under pressure to produce ahead of a major publication that could seal the deal on further funding. Everyone involved has clammed up and circled the wagons when a major and credible news organization, the WaPo, came asking for their side of it - they may be wholly and entirely innocent of any and all wrongdoing, but it smells to high heaven.
Remember, kids, it's never the crime that gets you into real trouble - it's the coverup.
posted by Slap*Happy at 9:35 AM on March 12, 2013
Remember, kids, it's never the crime that gets you into real trouble - it's the coverup.
posted by Slap*Happy at 9:35 AM on March 12, 2013
I'm a little astonished at people's diffidence here.
A scientist who has just published a first authorship paper in Nature, been named head of a major new lab, and who has three daughters does not commit suicide on the very day he was due to respond to allegations of research misconduct unless he sees his world crashing down around him.
There is clearly major misconduct being hidden at Johns Hopkins, and no one really wants to know about it-- not Nature, not the lab head Boeke, not Johns Hopkins' administration, and not the NIH.
Yuan is a very brave man, and may very well get the deserts we reserve for just this kind of bravery: destruction of his career and reputation.
posted by jamjam at 9:42 AM on March 12, 2013 [9 favorites]
A scientist who has just published a first authorship paper in Nature, been named head of a major new lab, and who has three daughters does not commit suicide on the very day he was due to respond to allegations of research misconduct unless he sees his world crashing down around him.
There is clearly major misconduct being hidden at Johns Hopkins, and no one really wants to know about it-- not Nature, not the lab head Boeke, not Johns Hopkins' administration, and not the NIH.
Yuan is a very brave man, and may very well get the deserts we reserve for just this kind of bravery: destruction of his career and reputation.
posted by jamjam at 9:42 AM on March 12, 2013 [9 favorites]
I'm a little astonished at people's diffidence here.
Really? Pointing out a case of scientific malfeasance around here is like bringing up paedophilia at a papal conclave.
posted by No Robots at 10:37 AM on March 12, 2013
Really? Pointing out a case of scientific malfeasance around here is like bringing up paedophilia at a papal conclave.
posted by No Robots at 10:37 AM on March 12, 2013
A scientist who has just published a first authorship paper in Nature, been named head of a major new lab, and who has three daughters does not commit suicide on the very day he was due to respond to allegations of research misconduct unless he sees his world crashing down around him.
That's a silly claim. Just because we know two facts about this person's life (that he was being asked to respond to an accusation against a paper he'd published and that he committed suicide) in no way whatsoever allows us to draw the firm conclusion that the one event caused the other.
People whose lives seem, from the outside, to be going well commit suicide all the time for all kinds of reasons.
posted by yoink at 10:45 AM on March 12, 2013 [1 favorite]
That's a silly claim. Just because we know two facts about this person's life (that he was being asked to respond to an accusation against a paper he'd published and that he committed suicide) in no way whatsoever allows us to draw the firm conclusion that the one event caused the other.
People whose lives seem, from the outside, to be going well commit suicide all the time for all kinds of reasons.
posted by yoink at 10:45 AM on March 12, 2013 [1 favorite]
"The e-mail, which Yuan saved, essentially blamed him for driving Lin to suicide."
posted by No Robots at 11:04 AM on March 12, 2013
posted by No Robots at 11:04 AM on March 12, 2013
...in no way whatsoever allows us to draw the firm conclusion that the one event caused the other.
Looks like a duck. Walks like a duck. I'm thinking duck.
posted by Slap*Happy at 11:13 AM on March 12, 2013
Looks like a duck. Walks like a duck. I'm thinking duck.
posted by Slap*Happy at 11:13 AM on March 12, 2013
Really? Pointing out a case of scientific malfeasance around here is like bringing up paedophilia at a papal conclave.
I've never gotten this impression before. Mefites in general seem pretty anti-scientific-malfeasance in my impression.
Particular mefites might be particularly defensive of certain scientists in their field of study, but that doesn't translate to being pro-coverup.
posted by muddgirl at 11:18 AM on March 12, 2013 [1 favorite]
I've never gotten this impression before. Mefites in general seem pretty anti-scientific-malfeasance in my impression.
Particular mefites might be particularly defensive of certain scientists in their field of study, but that doesn't translate to being pro-coverup.
posted by muddgirl at 11:18 AM on March 12, 2013 [1 favorite]
I can't say that I can really assign any opinion on this. I have been on both sides of this type of mess, no not the suicide part. I do not feel that either side is reliable enough to believe fully.
Yuan as an MIT-Hopkins doctor getting demoted to entry level researcher after 10 years? I don't know how this happens unless there are some other deep problems. If you are in a bad spot you move, and move up, he must have had connections to do so and that skill set is very much in demand. It is hard for me to see him accepting those demotions without some other problems.
On the other hand, over the top hard driven researcher fudges data. I have seen that too, someone whose data seemed too good and rapid to be true, and as it turns out it wasn't all real.
I have often thought that the personalities in the labs I have been in were perfect fodder for novels and dramas. If only the experiments didn't take so damn long. A court case is finite and good time for a TV show, lab work is never so convenient.
posted by oshburghor at 12:09 PM on March 12, 2013 [1 favorite]
Yuan as an MIT-Hopkins doctor getting demoted to entry level researcher after 10 years? I don't know how this happens unless there are some other deep problems. If you are in a bad spot you move, and move up, he must have had connections to do so and that skill set is very much in demand. It is hard for me to see him accepting those demotions without some other problems.
On the other hand, over the top hard driven researcher fudges data. I have seen that too, someone whose data seemed too good and rapid to be true, and as it turns out it wasn't all real.
I have often thought that the personalities in the labs I have been in were perfect fodder for novels and dramas. If only the experiments didn't take so damn long. A court case is finite and good time for a TV show, lab work is never so convenient.
posted by oshburghor at 12:09 PM on March 12, 2013 [1 favorite]
I really don't think all of these organizations, including Nature and the NIH, intend to sweep this under the rug. That just never makes sense in the sciences; as the field progresses, people will revisit your results. If they are wrong, people will see, and dig deeper - if you were unjustified or fraudulent, it will come to the surface eventually. Nature in particular has a hell of a lot of reputation to lose if they openly ignore fraudulent results in their journal.
My guess is that he overstates the extent of the problem. In the sciences, people vary somewhat in how 'conservative' (resistant to type 1 error) they require a method to be. Some people will use methods that other people regard as being unreliable, particularly in fields in which the statistics are difficult to interpret as traditional p values. So it's entirely possible that neither he nor the lab managers are being intentionally deceptive; this might ultimately boil down to a difference of opinion on a difficult and highly technical issue, which may not yet be settled in the literature.
posted by Mitrovarr at 12:53 PM on March 12, 2013 [1 favorite]
My guess is that he overstates the extent of the problem. In the sciences, people vary somewhat in how 'conservative' (resistant to type 1 error) they require a method to be. Some people will use methods that other people regard as being unreliable, particularly in fields in which the statistics are difficult to interpret as traditional p values. So it's entirely possible that neither he nor the lab managers are being intentionally deceptive; this might ultimately boil down to a difference of opinion on a difficult and highly technical issue, which may not yet be settled in the literature.
posted by Mitrovarr at 12:53 PM on March 12, 2013 [1 favorite]
There is clearly major misconduct being hidden at Johns Hopkins...
I'm not entirely sure, I don't think the article gives enough information to state there has been 'clear', deliberate misconduct. It could just be a chain of leadership incompetence and honest mistakes that went on for too long with tragic consequences. And the fact that the man who was responsible is now dead makes sorting this all out that much more difficult.
Here's one way it could have gone down - Lin has data he needs analyzed, he goes to Yuan for help and is refused (which can definitely read as sour grapes to me - it's not the new guy's fault things in the lab have been going on this way for years, and Yuan's *job* in the lab is to help lab members with this stuff, if you have serious ethical issues with doing your job why haven't you quit yet?), so Lin has to do it himself - Lin then makes serious mistakes in the analysis (likely, given that Yuan might have been the only guy in the lab who really understood the intricacies of how the code/software worked). The lab head is oblivious to the mistakes too, the paper gets published - but Yuan is the guy who can take you down since he knows this stuff inside and out, and he picks it apart bit by bit.
Now the way it should work in labs is that the 'point guy' for the stats should be training *someone* to do his job once he is gone, and then that person should have caught any errors that Lin was introducing into the analysis before the paper gets sent out - but since Yuan seems to have a troubled history in the lab (the demotion, then the firing) it wouldn't surprise me if they lost all that knowledge when he left.
What came across most clearly in the article to me is that Yuan has a serious axe to grind with Hopkins and his previous lab.
posted by aiglet at 3:45 PM on March 12, 2013 [3 favorites]
I'm not entirely sure, I don't think the article gives enough information to state there has been 'clear', deliberate misconduct. It could just be a chain of leadership incompetence and honest mistakes that went on for too long with tragic consequences. And the fact that the man who was responsible is now dead makes sorting this all out that much more difficult.
Here's one way it could have gone down - Lin has data he needs analyzed, he goes to Yuan for help and is refused (which can definitely read as sour grapes to me - it's not the new guy's fault things in the lab have been going on this way for years, and Yuan's *job* in the lab is to help lab members with this stuff, if you have serious ethical issues with doing your job why haven't you quit yet?), so Lin has to do it himself - Lin then makes serious mistakes in the analysis (likely, given that Yuan might have been the only guy in the lab who really understood the intricacies of how the code/software worked). The lab head is oblivious to the mistakes too, the paper gets published - but Yuan is the guy who can take you down since he knows this stuff inside and out, and he picks it apart bit by bit.
Now the way it should work in labs is that the 'point guy' for the stats should be training *someone* to do his job once he is gone, and then that person should have caught any errors that Lin was introducing into the analysis before the paper gets sent out - but since Yuan seems to have a troubled history in the lab (the demotion, then the firing) it wouldn't surprise me if they lost all that knowledge when he left.
What came across most clearly in the article to me is that Yuan has a serious axe to grind with Hopkins and his previous lab.
posted by aiglet at 3:45 PM on March 12, 2013 [3 favorites]
Can anyone actually read the paper to assess the quality of the method? I tried, but it's extremely technical and far enough off my specialty (phylogenetics) that I can't tell if their method was any good or not.
posted by Mitrovarr at 6:33 PM on March 12, 2013
posted by Mitrovarr at 6:33 PM on March 12, 2013
The work isn't close enough to my subfield to comment on the analysis, but from the article tone and leadup, I was expecting something along the lines of the Sames-Sezen or van Parijs scandals. There may well have been some serious flaws in the data analysis or other misconduct and the timing of Lin's suicide is suspicious, but Yuan is also clearly nursing a grudge and his demotion is odd, given 10 years with the group and in light of his PhD MD (and presumable connections). Many things the article brings up, like Yuan not being able to take his cell lines with him and Boeke needing more than two weeks to coordinate 12 co-authors across 3 academic institutions seem normal and quite reasonable.
Is it possible there's some sort of nefarious coverup? Of course. But incorrect statistical analysis is relatively minor on the bad data spectrum, and it's easy to make these mistakes when you don't have a solid grasp of stats. I've worked in a lab where people normalized the major dataset for a C/N/S paper incorrectly -- not out of ill-intent, but because they didn't realize it was wrong, and the PI glanced at it and things looked ok and he was stressed and it was a big lab and he didn't have time to run through the numbers himself, so off it all went to the journal, and we didn't realize things were funny until years later. aiglet's suggested explanation rings very true to me.
posted by angst at 8:01 PM on March 12, 2013
Is it possible there's some sort of nefarious coverup? Of course. But incorrect statistical analysis is relatively minor on the bad data spectrum, and it's easy to make these mistakes when you don't have a solid grasp of stats. I've worked in a lab where people normalized the major dataset for a C/N/S paper incorrectly -- not out of ill-intent, but because they didn't realize it was wrong, and the PI glanced at it and things looked ok and he was stressed and it was a big lab and he didn't have time to run through the numbers himself, so off it all went to the journal, and we didn't realize things were funny until years later. aiglet's suggested explanation rings very true to me.
posted by angst at 8:01 PM on March 12, 2013
I don't have the time right now to fully evaluate it, but a quick skim leads me to believe that Mitrovarr, aiglet, and angst are right. In addition, I'll add that the senior statistician on that paper, Rafa Irizarry, is an extremely well-respected researcher who works on analytical methodologies for high-throughput microarray data. He has made seminal, rigorous contributions to the field. While it's true that sometimes senior authors aren't as aware of the nitty-gritty details as they should be (esp if they, too, are being decieved by an underling), I find it very hard to believe that a paper with serious and obvious statistical flaws would have gone out with RI's name on it.
I also feel that the WaPo is sensationalizing things which are quite reasonable (requesting more that 2 weeks to respond, not letting Yuan copy the cell lines). I really wish that the author had given more attention to the "retraction epidemic" instead. Paylines at NIH aren't even 1 in 5 these days; R01s need to be percentiled around 7% to get funded. At the same time, research faculty in medical schools (even those with tenure) operate on a salary-recapture model in which they are expected to support 75-80% of their own salaries on grants (this is in contrast to regular non-medschool faculty, who have 9mos hard money salaries). This is completely antithetical to the idea of academic research, which is supposed to be free of economic pressures. If we are to have reliable results, researchers need the academic freedom to obtain null findings, but under these conditions they quite literally can't afford them -- there's a selection pressure is for positive findings rather than true ones. Either the paylines need to go back up, or the NIH needs to start putting conditions on the grants (eg, requiring that univerisities support faculty lines with hard money as a cost-sharing measure); as things are, we're creating a system that encourages the fudging of data.
posted by Westringia F. at 9:08 PM on March 12, 2013 [1 favorite]
I also feel that the WaPo is sensationalizing things which are quite reasonable (requesting more that 2 weeks to respond, not letting Yuan copy the cell lines). I really wish that the author had given more attention to the "retraction epidemic" instead. Paylines at NIH aren't even 1 in 5 these days; R01s need to be percentiled around 7% to get funded. At the same time, research faculty in medical schools (even those with tenure) operate on a salary-recapture model in which they are expected to support 75-80% of their own salaries on grants (this is in contrast to regular non-medschool faculty, who have 9mos hard money salaries). This is completely antithetical to the idea of academic research, which is supposed to be free of economic pressures. If we are to have reliable results, researchers need the academic freedom to obtain null findings, but under these conditions they quite literally can't afford them -- there's a selection pressure is for positive findings rather than true ones. Either the paylines need to go back up, or the NIH needs to start putting conditions on the grants (eg, requiring that univerisities support faculty lines with hard money as a cost-sharing measure); as things are, we're creating a system that encourages the fudging of data.
posted by Westringia F. at 9:08 PM on March 12, 2013 [1 favorite]
A scientist who has just published a first authorship paper in Nature, been named head of a major new lab, and who has three daughters does not commit suicide on the very day he was due to respond to allegations of research misconduct unless he sees his world crashing down around him.
Not necessarily - Howard Eisen killed himself over the course of an NIH investigation, which was looking into fraud that he himself had found and reported.
posted by en forme de poire at 4:19 PM on March 14, 2013
Not necessarily - Howard Eisen killed himself over the course of an NIH investigation, which was looking into fraud that he himself had found and reported.
posted by en forme de poire at 4:19 PM on March 14, 2013
Mitrovarr, this is actually sort of my area, but I'm in the middle of prepping for some interviews - if I get some time to read it thoroughly I'll try and respond.
posted by en forme de poire at 4:21 PM on March 14, 2013 [2 favorites]
posted by en forme de poire at 4:21 PM on March 14, 2013 [2 favorites]
« Older Annihilated Angel | "Please don't use any less than a 16px base... Newer »
This thread has been archived and is closed to new comments
posted by smrtsch at 8:45 AM on March 12, 2013