Journal of Irreproducible Results
October 3, 2013 1:31 PM Subscribe
This is brilliant! The author clearly put a huge amount of time and effort into the sting, and the results are staggering.
posted by medusa at 1:47 PM on October 3, 2013
posted by medusa at 1:47 PM on October 3, 2013
A key quotation comes at the end:
If I had targeted traditional, subscription-based journals, Roos told me, "I strongly suspect you would get the same result."
Why didn't the author also send his fake paper to traditional journals as a control?
posted by crazy with stars at 1:48 PM on October 3, 2013 [21 favorites]
If I had targeted traditional, subscription-based journals, Roos told me, "I strongly suspect you would get the same result."
Why didn't the author also send his fake paper to traditional journals as a control?
posted by crazy with stars at 1:48 PM on October 3, 2013 [21 favorites]
"Why didn't the author also send his fake paper to traditional journals as a control?"
Maybe he still wants to be published by traditional journals in the future. They might not take kindly to this experiment.
posted by Kevin Street at 1:56 PM on October 3, 2013 [2 favorites]
Maybe he still wants to be published by traditional journals in the future. They might not take kindly to this experiment.
posted by Kevin Street at 1:56 PM on October 3, 2013 [2 favorites]
Let me get this straight... The number one closed source of retracted articles pulled a Sokal questioning the peer review of open journals?
posted by 3.2.3 at 1:56 PM on October 3, 2013 [6 favorites]
posted by 3.2.3 at 1:56 PM on October 3, 2013 [6 favorites]
When we were in grad school a decade or so ago, a friend and I blew off steam by coming up with TenureJournal: The Cross-Disciplinary Report. It would be dedicated to the idea of helping students and young faculty pad their CVs by publishing any paper that was submitted, after a cursory peer review by a committee made of other people looking to pad their CVs by appearing on peer review committees.
Apparently this was not an original idea.
posted by telophase at 2:02 PM on October 3, 2013 [22 favorites]
Apparently this was not an original idea.
posted by telophase at 2:02 PM on October 3, 2013 [22 favorites]
Also worth noting:
posted by en forme de poire at 2:10 PM on October 3, 2013 [9 favorites]
Some open-access journals that have been criticized for poor quality control provided the most rigorous peer review of all. For example, the flagship journal of the Public Library of Science, PLOS ONE, was the only journal that called attention to the paper's potential ethical problems, such as its lack of documentation about the treatment of animals used to generate cells for the experiment. The journal meticulously checked with the fictional authors that this and other prerequisites of a proper scientific study were met before sending it out for review. PLOS ONE rejected the paper 2 weeks later on the basis of its scientific quality.But as others have mentioned, framing this as a problem with open-access journals seems to reveal an axe to grind. As the author even acknowledges within this piece, there are plenty of sketchy, predatory closed-access journals, too, and it would have been interesting to conduct the experiment on them as well. The difference is that closed-access journals fleece academic libraries by being packaged as filler with subscriptions to more respectable journals, instead of fleecing the authors of the paper. It's also worth noting that many of the journals in this sting were published by Elsevier, which is notorious for predatory practices and for talking out of both sides of its mouth about OA. By making this about open-access specifically, it tars mainstream OA publishers like BMC and PLoS with the same brush used to paint scammy plagiarism-fests whose only similarity to the above is their author-pays business model, ignoring the more important differences like quality of constituent articles, quality of staffing, citations within other journals, scientific reputations of journal heads, etc.
posted by en forme de poire at 2:10 PM on October 3, 2013 [9 favorites]
Uh, this person committed fraud. Like the kind that gets you arrested.
posted by Ironmouth at 2:13 PM on October 3, 2013
posted by Ironmouth at 2:13 PM on October 3, 2013
This is both fascinating and damning.
Did anybody else get a really bad vibe off of his choice to make all of his authors African, and to make sure his English was bad in the papers by translating it into and out of French?
I know it was supposed to be a clever move to make sure it was hard to judge papers based on the lack of academic track record of the authors, because scientists in the developing world aren't as likely to have publicly searchable academic records, but still, it smells kinda racist.
Especially because their names were randomly-generated out of a list of "African" first and last names (you know, names of people from the country of Africa who speak the African language).
posted by edheil at 2:17 PM on October 3, 2013 [4 favorites]
Did anybody else get a really bad vibe off of his choice to make all of his authors African, and to make sure his English was bad in the papers by translating it into and out of French?
I know it was supposed to be a clever move to make sure it was hard to judge papers based on the lack of academic track record of the authors, because scientists in the developing world aren't as likely to have publicly searchable academic records, but still, it smells kinda racist.
Especially because their names were randomly-generated out of a list of "African" first and last names (you know, names of people from the country of Africa who speak the African language).
posted by edheil at 2:17 PM on October 3, 2013 [4 favorites]
Rebuttal: http://www.michaeleisen.org/blog/?p=1439
posted by mithrandir at 2:20 PM on October 3, 2013 [13 favorites]
posted by mithrandir at 2:20 PM on October 3, 2013 [13 favorites]
BTW, next time somebody gets all smug about the humanities because of the Sokal hoax paper being published, I'm gonna be thinking about this.....
posted by edheil at 2:21 PM on October 3, 2013 [3 favorites]
posted by edheil at 2:21 PM on October 3, 2013 [3 favorites]
Ironmouth, I really doubt that, since most cases of scientific misconduct are not actually criminal activity. Can you be more specific?
posted by en forme de poire at 2:22 PM on October 3, 2013 [5 favorites]
posted by en forme de poire at 2:22 PM on October 3, 2013 [5 favorites]
Especially because their names were randomly-generated out of a list of "African" first and last names (you know, names of people from the country of Africa who speak the African language).
Isn't the hoaxer African? Asmara is in Eritrea. I'm guessing the author realizes that Africa isn't a country.
posted by IvoShandor at 2:23 PM on October 3, 2013
Isn't the hoaxer African? Asmara is in Eritrea. I'm guessing the author realizes that Africa isn't a country.
posted by IvoShandor at 2:23 PM on October 3, 2013
Also, LMFHO at Mike Eisen's headline: "I confess, I wrote the Arsenic DNA paper to expose flaws in peer-review at subscription based journals." thx, mithrandir.
posted by en forme de poire at 2:24 PM on October 3, 2013 [1 favorite]
posted by en forme de poire at 2:24 PM on October 3, 2013 [1 favorite]
Are you sure it wasn't all just a.........*shades* benzenedream?
I'll see myself out.
posted by nevercalm at 2:32 PM on October 3, 2013 [1 favorite]
I'll see myself out.
posted by nevercalm at 2:32 PM on October 3, 2013 [1 favorite]
I assume the John Bohannon is this John Bohannon, who's from Harvard, not Eritrea.
I would hope somebody from Harvard also knows Africa isn't a country, but he did say he was randomly choosing "African" first and last names and places, which is kind of what the whole Africa-is-a-country is all about: treating Africa as a single entity.
It's almost as if *this article itself* is supposed to be faking us out by presenting conclusions which are not justified by the results because the methodology was fatally flawed.
HE JUST BLEW MY MIND.
posted by edheil at 2:32 PM on October 3, 2013 [7 favorites]
I would hope somebody from Harvard also knows Africa isn't a country, but he did say he was randomly choosing "African" first and last names and places, which is kind of what the whole Africa-is-a-country is all about: treating Africa as a single entity.
The fictitious authors are affiliated with fictitious African institutions. I generated the authors, such as Ocorrafoo M. L. Cobange, by randomly permuting African first and last names harvested from online databases, and then randomly adding middle initials. For the affiliations, such as the Wassee Institute of Medicine, I randomly combined Swahili words and African names with generic institutional words and African capital cities. My hope was that using developing world authors and institutions would arouse less suspicion if a curious editor were to find nothing about them on the Internet.As en forme de poire mentioned, it does seem funny that in an experiment which is supposed to prove that open access journals are particularly problematic, he has no control group of subscription-based journals.
It's almost as if *this article itself* is supposed to be faking us out by presenting conclusions which are not justified by the results because the methodology was fatally flawed.
HE JUST BLEW MY MIND.
posted by edheil at 2:32 PM on October 3, 2013 [7 favorites]
Wow - two months to get a paper accepted, who cares about the quality? I currently have two papers that have been in journal review systems for 19 and 5 months respectively and it is a significant problem for me and no doubt others in terms of delivering on university expectations of academics.
I recently edited a special edition of a journal in my field and it is extremely difficult to find reviewers and at the same time the number of papers being submitted has been growing substantially, the same journal Is publishing at least 5 times as much as it did 10 years ago. It would not surprise me at all if some traditional journals were struggling to keep up with both demand and with quality.
posted by biffa at 2:34 PM on October 3, 2013
I recently edited a special edition of a journal in my field and it is extremely difficult to find reviewers and at the same time the number of papers being submitted has been growing substantially, the same journal Is publishing at least 5 times as much as it did 10 years ago. It would not surprise me at all if some traditional journals were struggling to keep up with both demand and with quality.
posted by biffa at 2:34 PM on October 3, 2013
And apparantly nobody uses google to look up the Wassee Institute of Medicine?
posted by MartinWisse at 2:35 PM on October 3, 2013
posted by MartinWisse at 2:35 PM on October 3, 2013
From Eisen's rebuttal:
If we had, instead, a [review] system where the review process was transparent and persisted for the useful life of a work
We do have that, in the form of reproducability: empirical results that should be repeatable, if the science is valid. If it isn't repeatable, write a letter or another paper.
posted by Blazecock Pileon at 2:35 PM on October 3, 2013
If we had, instead, a [review] system where the review process was transparent and persisted for the useful life of a work
We do have that, in the form of reproducability: empirical results that should be repeatable, if the science is valid. If it isn't repeatable, write a letter or another paper.
posted by Blazecock Pileon at 2:35 PM on October 3, 2013
Wow - two months to get a paper accepted, who cares about the quality? I currently have two papers that have been in journal review systems for 19 and 5 months respectively and it is a significant problem for me and no doubt others in terms of delivering on university expectations of academics.
Yeah, I've noticed this too. It's getting worse. I just had something accepted after a 10-month review process. Yay?
posted by mr_roboto at 2:36 PM on October 3, 2013
Yeah, I've noticed this too. It's getting worse. I just had something accepted after a 10-month review process. Yay?
posted by mr_roboto at 2:36 PM on October 3, 2013
Uh, this person committed fraud. Like the kind that gets you arrested.
Yea, well fraud ain't what it used to be.
I believe pages here on The Blue can be dedicated to deception for someone's gain that have been uttered by elected and appointed officials.
But as someone else said:
Can you be more specific?
Feel free to quote the chapter and verse of the laws that a properly functioning Grand Jury would true bill.
posted by rough ashlar at 2:36 PM on October 3, 2013 [1 favorite]
Yea, well fraud ain't what it used to be.
I believe pages here on The Blue can be dedicated to deception for someone's gain that have been uttered by elected and appointed officials.
But as someone else said:
Can you be more specific?
Feel free to quote the chapter and verse of the laws that a properly functioning Grand Jury would true bill.
posted by rough ashlar at 2:36 PM on October 3, 2013 [1 favorite]
A year or more in review was par for the course for me when I did this schtick. It's a perverse sort of arms race, since the number of tenure track positions out there is surely not growing very fast. It feels like a big make-work program for hiring committees and their admins to filter out all the non-contenders. The actual contenders and where they publish has probably not changed much in ten years or longer, with the possible addition of PLOS.
posted by simra at 2:41 PM on October 3, 2013
posted by simra at 2:41 PM on October 3, 2013
Another succinct explanation of the problems with this study.
posted by en forme de poire at 2:42 PM on October 3, 2013 [10 favorites]
posted by en forme de poire at 2:42 PM on October 3, 2013 [10 favorites]
Another succinct explanation of the problems with this study.
That piece is incredibly well written. Good find.
posted by mr_roboto at 2:48 PM on October 3, 2013 [1 favorite]
That piece is incredibly well written. Good find.
posted by mr_roboto at 2:48 PM on October 3, 2013 [1 favorite]
Remember when Britannica and that ilk would publish studies about all of the flaws and inaccuracies in Wikipedia articles? This reminds me of that.
posted by bbuda at 3:09 PM on October 3, 2013 [1 favorite]
posted by bbuda at 3:09 PM on October 3, 2013 [1 favorite]
Well, the so-called "gonzo scientist" has certainly managed to live up to his name and make himself part of the "story". What a joke. It's not surprising that he left science to become a journalist if this is the best experimental design that he can come up with.
posted by gene_machine at 3:44 PM on October 3, 2013 [1 favorite]
posted by gene_machine at 3:44 PM on October 3, 2013 [1 favorite]
Dodgy conferences, mostly in China, also seems to be a newish industry. I seem to get an awful lot of invitations to submit my esteemed research or chair a session for conferences with absurdly broad mix and match interests. I guess the aim is to tempt you with being able to put "invited to international conference" on your CV ... handy for your annual review or promotions.
[FWIW, pretty much all journals suffer from poor reviewing, in my experience. Mostly in the form of poor reviewing where the reviewer clearly doesn't have the expertise needed.]
posted by drnick at 4:08 PM on October 3, 2013
[FWIW, pretty much all journals suffer from poor reviewing, in my experience. Mostly in the form of poor reviewing where the reviewer clearly doesn't have the expertise needed.]
posted by drnick at 4:08 PM on October 3, 2013
Medical pseudoscience that's borderline fraud is practically an American tradition, guys.
I mean, we're talking about a country that requires nearly double the length medical education of every other first world nation and still has trouble getting doctors to believe in evolution.
The amount of felony-level outright fraud I saw during a single rotation of my residency would probably take years of investigation and trials to properly prosecute. Don't even get me started on what they do to get a passing rate for the hospital whenever ..whatever the joint commission is calling themselves these days... comes to town.
Published papers in the field of pharmacology is such a wild west of crazy I consistently wake up in astonished bewilderment that the world hasn't collapsed into Ragnarok. Do not even get me started on 'evidence based medicine'.
This is the end result of a system that requires you publish extensively, something, anything, in order to advance in your career.
posted by hobo gitano de queretaro at 4:21 PM on October 3, 2013 [5 favorites]
I mean, we're talking about a country that requires nearly double the length medical education of every other first world nation and still has trouble getting doctors to believe in evolution.
The amount of felony-level outright fraud I saw during a single rotation of my residency would probably take years of investigation and trials to properly prosecute. Don't even get me started on what they do to get a passing rate for the hospital whenever ..whatever the joint commission is calling themselves these days... comes to town.
Published papers in the field of pharmacology is such a wild west of crazy I consistently wake up in astonished bewilderment that the world hasn't collapsed into Ragnarok. Do not even get me started on 'evidence based medicine'.
This is the end result of a system that requires you publish extensively, something, anything, in order to advance in your career.
posted by hobo gitano de queretaro at 4:21 PM on October 3, 2013 [5 favorites]
Now if you guys will excuse me I'm going to go drink heavily and overindulge in costco frozen pizzas, I got a big day tomorrow giving lectures about lifestyle choices at a middle school health fair.
posted by hobo gitano de queretaro at 4:23 PM on October 3, 2013
posted by hobo gitano de queretaro at 4:23 PM on October 3, 2013
Seems like there's a few people with an axe to grind in the open access advocacy world. I'm a pretty big supporter of open access journals, but that doesn't invalidate the results of this little experiment (Science's spin aside): namely, that there's a boatload of essentially meaningless "journals" out there. Which, to me, is worth noting and doing something about. The claim that there are so many meaningless open access ones, including ones owned by major publishers and from the DOAJ, should be cause for concern and should be addressed by open access advocates (and advocates of better science in general) rather than dismissed.
posted by parudox at 5:47 PM on October 3, 2013 [2 favorites]
posted by parudox at 5:47 PM on October 3, 2013 [2 favorites]
He states one reason he didn't send this out to closed access peer-reviewed journals: a desire not to waste swathes of reviewers' time, which is an actual concern. And he points out that many of these journals are published by the massive profiteers that also run closed access journals, Elsevier raising its ugly head once more. His axe did not seem to me to be as much with open access as with journals that don't do peer review and then charge people for the privilege of publishing with them. These are essentially ripping people off: some people who have legitimate research may be conned into publishing with them only to find out their work is then ignored because it's tossed in with junk.
(I am not a fan of the current system of closed access journals, but pretending the open access system is working at the moment is ignoring reality.)
posted by lesbiassparrow at 5:55 PM on October 3, 2013
(I am not a fan of the current system of closed access journals, but pretending the open access system is working at the moment is ignoring reality.)
posted by lesbiassparrow at 5:55 PM on October 3, 2013
A Bantu sounding name for a fictional resident of a Semitic speaking country?
Poor form.
posted by ocschwar at 5:59 PM on October 3, 2013
Poor form.
posted by ocschwar at 5:59 PM on October 3, 2013
Lesbiassparrow, I think you are confusing open access with peer review. All of the journals he submitted to here are allegedly peer-reviewed. It also seems (from browsing the terribly-formatted supplement) that many of the journals rejected the paper during consideration by the editor before it was even sent to reviewers. This type of triage is very common, among both closed- and open-access journals. Between these two facts I don't buy the argument that closed-access journals would be intrinsically more likely to waste the time of reviewers.
Parudox, is it your contention that open access journals are more likely to be weak or predatory? As per the rebuttal I linked above, this study does not actually show that there is a problem specific to OA.
I do agree that dishonest publishers should be taken to task, regardless of whether they are open- or closed-access, and that the DOAJ should establish tighter guidelines for listing OA journals. Although I should add that we don't know how random the sample of DOAJ journals was. I would bet that the more respectable OA publishers, like PLoS and BMC, were underrepresented, though again this is based on my trying to interact with their 3D spinning supplement, which does not seem to be available as a table.
posted by en forme de poire at 6:03 PM on October 3, 2013
Parudox, is it your contention that open access journals are more likely to be weak or predatory? As per the rebuttal I linked above, this study does not actually show that there is a problem specific to OA.
I do agree that dishonest publishers should be taken to task, regardless of whether they are open- or closed-access, and that the DOAJ should establish tighter guidelines for listing OA journals. Although I should add that we don't know how random the sample of DOAJ journals was. I would bet that the more respectable OA publishers, like PLoS and BMC, were underrepresented, though again this is based on my trying to interact with their 3D spinning supplement, which does not seem to be available as a table.
posted by en forme de poire at 6:03 PM on October 3, 2013
Parudox, is it your contention that open access journals are more likely to be weak or predatory? As per the rebuttal I linked above, this study does not actually show that there is a problem specific to OA.
I do not contend that. What I do contend is that it's a good sign that there is a problem in open access journals, regardless of whether there is the same or different problems in conventional journals.
posted by parudox at 6:12 PM on October 3, 2013
I do not contend that. What I do contend is that it's a good sign that there is a problem in open access journals, regardless of whether there is the same or different problems in conventional journals.
posted by parudox at 6:12 PM on October 3, 2013
Lesbiassparrow, I think you are confusing open access with peer review.
But I think he proved that a lot of open access journals aren't doing even basic peer review, nor, even worse, hiring editors who know the subjects covered in the journal or basic information on scientific research, because as you say this would have been bounced by any competent editor right away.
(this is not to say that closed access journals don't have their own issues, though.)
posted by lesbiassparrow at 6:14 PM on October 3, 2013
But I think he proved that a lot of open access journals aren't doing even basic peer review, nor, even worse, hiring editors who know the subjects covered in the journal or basic information on scientific research, because as you say this would have been bounced by any competent editor right away.
(this is not to say that closed access journals don't have their own issues, though.)
posted by lesbiassparrow at 6:14 PM on October 3, 2013
And what's worse to me is the charging: some of these journals are ripping people off for the promise of a publication that no serious researcher will respect and probably won't even know about. It's no better than those people who email you and say you've been selected to go in a version of Who's Who and they'll send you your copy for $500.
posted by lesbiassparrow at 6:16 PM on October 3, 2013
posted by lesbiassparrow at 6:16 PM on October 3, 2013
But I think he proved that a lot of open access journals aren't doing even basic peer review, nor, even worse, hiring editors who know the subjects covered in the journal or basic information on scientific research, because as you say this would have been bounced by any competent editor right away.
However, he did not prove that this problem is worse in OA than in CA journals, since he did not compare to any CA journals.
posted by en forme de poire at 6:20 PM on October 3, 2013 [1 favorite]
However, he did not prove that this problem is worse in OA than in CA journals, since he did not compare to any CA journals.
posted by en forme de poire at 6:20 PM on October 3, 2013 [1 favorite]
Also, not all OA journals charge author fees, while some CA journals do.
posted by en forme de poire at 6:21 PM on October 3, 2013
posted by en forme de poire at 6:21 PM on October 3, 2013
We do have that, in the form of reproducability: empirical results that should be repeatable, if the science is valid. If it isn't repeatable, write a letter or another paper.
I agree that that would happen in a perfect world but:
a) people aren't attempting to reproduce any but the most startling results, because in today's fraught academic environment you get promoted on the basis of the originality of your work and, reproducing other people's results is really expensive and time consuming and,
b) journals won't publish 'checking' papers that just reproduce results already obtained because such papers are unexciting and unoriginal.
Furthermore, unless the original paper is found to be actively fraudulent, it stays in the journal with science's imprimatur upon it for any non-expert to come along and read, with no indication that other articles might have superseded it.
We need a system that allows us to publish quickly, and receive transparent, ongoing and constantly updating, discipline-wide review, accessible from the article itself. The current peer review system is hopeless out of date and wrong for today's academy.
posted by Dreadnought at 7:10 PM on October 3, 2013 [2 favorites]
I agree that that would happen in a perfect world but:
a) people aren't attempting to reproduce any but the most startling results, because in today's fraught academic environment you get promoted on the basis of the originality of your work and, reproducing other people's results is really expensive and time consuming and,
b) journals won't publish 'checking' papers that just reproduce results already obtained because such papers are unexciting and unoriginal.
Furthermore, unless the original paper is found to be actively fraudulent, it stays in the journal with science's imprimatur upon it for any non-expert to come along and read, with no indication that other articles might have superseded it.
We need a system that allows us to publish quickly, and receive transparent, ongoing and constantly updating, discipline-wide review, accessible from the article itself. The current peer review system is hopeless out of date and wrong for today's academy.
posted by Dreadnought at 7:10 PM on October 3, 2013 [2 favorites]
I was so excited the other day when I received an email with the subject "Regarding Constructing 5-configurations with chiral symmetry". I thought "hooray! Somebody's read my paper! Awesome!"
Then I read the email.
You click through to the website, and their description of their journal is
posted by leahwrenn at 7:20 PM on October 3, 2013 [1 favorite]
Then I read the email.
Dear [leahwrenn],Crap. Another spam journal. The fact that it claims that it's an *engineering* journal is an especially nice touch---this paper is a pretty pure mathematical paper in discrete geometry. No actual applications were harmed in the preparation of this paper.
I came across to your research paper titled "Constructing 5-configurations with chiral symmetry" and feel that your research is having a very good impact.
With a view to begin a long-term fruitful association with you, I invite you to submit your upcoming research articles / papers for publication in Global Journal of Research in Engineering (GJRE), an international double blind peer reviewed research journal.
This may be noted that due date of forthcoming issue GJRE is 15th October, 2013.
The process of publication is expected to be completed within three to five weeks. Our processes follow rigid quality control measures and comply with international standards in journal publishing.
You click through to the website, and their description of their journal is
The Global Journal of Researches in Engineering (GJRE) is an international journal for publishing research paper of computer science. Yielding in a true sense is the objective equally aims to encourage and provide international publication to researchers, scientists and engineers. We welcome original researches, articles, surveys and review papers from all over the world.That's not even coherent. I'm especially fond of the "original researches".
posted by leahwrenn at 7:20 PM on October 3, 2013 [1 favorite]
"Researches" is super common; it's a Chinese thing, I think. I get a lot of Chinese people applying for postdocs bragging about their high-impact researches.
posted by mr_roboto at 8:50 PM on October 3, 2013
posted by mr_roboto at 8:50 PM on October 3, 2013
Oh man, sorry to threadsit but Jonathan Eisen has an amazing link about an ill-fated spammy science journal that hired a completely made up person for their board.
posted by en forme de poire at 9:37 PM on October 3, 2013
posted by en forme de poire at 9:37 PM on October 3, 2013
From Eisen's "rebuttal" link above:
First, and foremost, we need to get past the antiquated idea that the singular act of publication – or publication in a particular journal – should signal for all eternity that a paper is valid, let alone important. Even when people take peer review seriously, it is still just represents the views of 2 or 3 people at a fixed point in time.
This is why one of my favorite models for journal publishing is Current Anthropology, which includes commentary from colleagues and replies from the authors with every major article. I can see why not every journal would adopt this (on the basis of cost, effort, etc.), but it shouldn't be so hard to have peer reviewers identify shortcomings and limitations of a paper that wouldn't keep it from being published, but would situate it within the context of the current knowledge of the field. Accepting a paper is a tacit sign of approval, having an "insta-rebuttal" may not be great for the ego of the authors, but it certainly would allow the journal to be more nuanced in their presentation of data.
posted by Panjandrum at 6:58 AM on October 4, 2013 [1 favorite]
First, and foremost, we need to get past the antiquated idea that the singular act of publication – or publication in a particular journal – should signal for all eternity that a paper is valid, let alone important. Even when people take peer review seriously, it is still just represents the views of 2 or 3 people at a fixed point in time.
This is why one of my favorite models for journal publishing is Current Anthropology, which includes commentary from colleagues and replies from the authors with every major article. I can see why not every journal would adopt this (on the basis of cost, effort, etc.), but it shouldn't be so hard to have peer reviewers identify shortcomings and limitations of a paper that wouldn't keep it from being published, but would situate it within the context of the current knowledge of the field. Accepting a paper is a tacit sign of approval, having an "insta-rebuttal" may not be great for the ego of the authors, but it certainly would allow the journal to be more nuanced in their presentation of data.
posted by Panjandrum at 6:58 AM on October 4, 2013 [1 favorite]
Panjandrum, BMC Biology used to do something similar and I thought it was an interesting experiment. I like that it forces peer reviewers to take a little extra care to be professional about their responses, and it makes conflicts of interest a little more readily apparent (it would be obvious if you were just selectively trashing your competitors). But it also exposes the reviewer to potential retribution, which is less great. Not sure which wins on balance.
posted by en forme de poire at 1:05 PM on October 4, 2013
posted by en forme de poire at 1:05 PM on October 4, 2013
« Older No, no, no---the other custom of the sea | The One and Only Damita Jo Freeman Newer »
This thread has been archived and is closed to new comments
posted by Potomac Avenue at 1:43 PM on October 3, 2013 [1 favorite]