"[T]hey will be removed, not retracted, since they are all nonsense."
June 15, 2014 12:39 PM   Subscribe

Earlier this year venerable academic publishers Springer and IEEE were impelled to remove more than 120 physics papers from their published proceedings because the papers were computer-generated nonsense. The SCIgen program (and its math-oriented fork) is available for the non-discriminating would-be author to generate such word salad. It's previously been used to perform hoaxes (previously, previouslier) of the kind that Alan Sokal wrought on a post-modern journal. (After the papers are published, the hoaxers claim incompetence by editors.) But in this case the papers don't appear to be hoaxes -- they're instead perhaps generated to pad academic CVs, with the publishers all too willing to take the publication fees.

Springer has responded by deploying a SCIgen detector for its papers, and on reflection decided that even nonsense was worthy of a retraction rather than mere deletion. No word of more qualitative review of the flow of papers going into its journals.

Jeffrey Beall maintains a list of predatory publishers who write to academics (previously) in the hopes of collecting publishing fees from authors desperate to be published in prestigious-sounding journals. The list has grown from 18 in 2011 to 477 in 2014.

As early as 1994 there was worry about an increasing volume and price of scholarly literature, with the Internet and electronic publishing perhaps both the problem and, ultimately, a solution. Outright gibberish in sophisticated journals may inspire continued progress towards alternative means of cultivating scholarly work and dialogue.

[My first attempt at an FPP -- feedback welcome.]
posted by zittrain (39 comments total) 45 users marked this as a favorite
 
How did I miss the post about MathGen? The output is startlingly good.
posted by leahwrenn at 12:48 PM on June 15, 2014


Interesting content, Clear and concise, relevant links. Good post!
posted by the man of twists and turns at 12:58 PM on June 15, 2014 [9 favorites]


Where is the list of authors & their institutions? This type of thing deserves a long perp walk.
posted by pjenks at 1:00 PM on June 15, 2014 [3 favorites]


From what I recall, this was mostly isolated to Asian conferences; I expect the list to contain pretty much zero institutions I recognize.
posted by pwnguin at 1:03 PM on June 15, 2014


Where is the list of authors & their institutions? This type of thing deserves a long perp walk.

And I wonder how many students plagiarized those papers...
posted by Alexandra Kitty at 1:03 PM on June 15, 2014 [3 favorites]


pjenks: Where is the list of authors & their institutions? This type of thing deserves a long perp walk.

That's risky, because there's nothing keeping the 'authors' of these papers from sticking other people in the authorship list without their knowledge or approval. In fact, you can easily imagine someone doing just that to improve the fake paper's verisimilitude or to make it look like they have important collaborators they don't.
posted by Mitrovarr at 1:24 PM on June 15, 2014 [1 favorite]


another author claimed his paper was submitted on purpose to test out a conference

Yes, I was testing the concept, that's it!
posted by arcticseal at 1:27 PM on June 15, 2014 [1 favorite]


This is a real problem in science -- not so much SciGen per se, but the fact that there's so much garbage out there (in no small part because there's a financial incentive for people without standards to publish garbage.) It's not just the Beall's types, not just the IEEEs and Springers, but also the Elseviers and the Taylor&Francises and the Mary Lieberts and all the rest out there who are making money by performing "peer review" that's unable to catch even the most transparently phony papers.

From what I recall, this was mostly isolated to Asian conferences...

Not only. WMSCI was the first hit, and the editor of the journal Complexity -- himself a character of dubious moral standing -- had to step down after his journal published another SciGen paper.


(For those who are interested in this issue, I devote half a chapter to the dilution of peer review, phony publications, and the like in my book coming out the week after next.)
posted by cgs06 at 1:48 PM on June 15, 2014 [9 favorites]


Related
posted by divabat at 1:50 PM on June 15, 2014


Welcome to MeFi and nice job on a post (btw, was a student of yours back in the day, now an academic myself)!

Second, the always-great Retraction Watch has more - including some names. All institutions from China or Taiwan, at least on this list.
posted by blahblahblah at 2:03 PM on June 15, 2014 [2 favorites]


The predatory journals are fascinating, in the "hypnotized by snake eyes" sense. I'm an English professor, and yet I frequently get "requests" for papers from supposed science publications (or, sometimes, interdisciplinary publications), at least one of which had an editor who did not seem to exist outside of cyberspace. Sometimes, I'll get multiple emails in one week from the same journal, signed by different "assistants" who, for some mysterious (ahem) reason, only use their first name + last initial (why, it's as though they're trying to prevent me from verifying their existence, or non-existence...now, why could that be...).
posted by thomas j wise at 2:12 PM on June 15, 2014


Retraction Watch has more

I'm bit disappointed that I don't get to see what scigen had to say on the topics of "802.11B Considered Harmful" and "Deconstructing the Ethernet".
posted by effbot at 3:26 PM on June 15, 2014 [1 favorite]


My eye was caught by the first paper, " Application of Amphibious Technology in the ReutoMail". I thought the term "amphibious technology" sounded funny, and I had never heard of "ReutoMail", so I Googled it. According to Google, the word "ReutoMail" appears only in pages referring to this paper, and in a few spam salad sites that presumably captured the word from it. In other words, the title itself is nonsense. We're not talking about people passing a paper that turns out to be nonsense; we're talking about people not even reading the title.
posted by Joe in Australia at 3:28 PM on June 15, 2014 [3 favorites]


Workers will adjust to optimize for whatever it is that you're measuring to evaluate them. Blindly measure the average call duration in a tech support center and you'll get people doing almost anything to get customers off the phone including telling lies, just saying "reboot and call back" again and again, or even just hanging up in the middle of speaking so that the customer thinks they got accidentally disconnected.

In an environment when winning tenure is like winning a minor lottery, and when that depends on having massive numbers of publications, of course people will publish crap. They'll publish as much crap as they think they can get away with, in fact. Why wouldn't they? This is perfectly rational behavior when the stakes are so high.
posted by LastOfHisKind at 3:29 PM on June 15, 2014 [6 favorites]


So Science is now effectively discrediting Science...

Next we're going to find out Neil Degrasse Tyson is a combination of AI and CGI programmed by a team of CompSci undergrads.
posted by oneswellfoop at 3:35 PM on June 15, 2014 [1 favorite]


In an environment when winning tenure is like winning a minor lottery, and when that depends on having massive numbers of publications, of course people will publish crap. They'll publish as much crap as they think they can get away with, in fact. Why wouldn't they? This is perfectly rational behavior when the stakes are so high.

So Science is now effectively discrediting Science...

I'm in the humanities so maybe it's different, but at least to me, much of the discussion regarding these predatory journals is a non-starter even though we have the same "problem". There are reports of authors in my field where they'll send in a paper to prestigious Journal X owned by Springer, have it get declined but asked whether they'd like to have it published in another Springer-owned journal, Journal Y, for a publishing fee. In the humanities though at least, it's not so much about the quantity of one's publications but the quality. No reasonably educated person then follows through with publishing in Journal Y because it's a journal without any standing in the field.

I think one thing not made clear in the FPP, is that there's a clear separation between the legitimate journals and these predatory journals, such that the author is wrong to believe that he is going to be, "published in prestigious-sounding journals," if these journals are asking these authors for a fee. The rest of the community recognizes that these journals are not "prestigious-sounding," because they're not prestigious.

That's of course not to say that we shouldn't look to get rid of these predatory practices. But just that the only danger behind their existence is their moral turpitude, then any deeper indictment against the legitimacy of academic progress. That is, these predatory journals, fake journal articles, and "legitimate" journal articles that nevertheless haven't gone through a legitimate peer-review process, aren't then influencing the legitimate work being done in the field.
posted by SollosQ at 3:55 PM on June 15, 2014


Chicken
posted by briank at 4:02 PM on June 15, 2014 [5 favorites]


I'm sure that I have mentioned this in previous discussions of journal shenanigans, but I have had to pay page charges for every journal article I've been involved with, all in reputable journals, from Ecology to Journal of Geophysical Research to Science. Fees suck, but are not how you identify a disreputable journal.

Now, if you want to talk about the preposterousness where we write the grants to pay for the science, do the science, and peer-review articles by other people for journals all at no cost to the journal and then are asked to pay page charges, I agree that is completely ludicrous. But page charges alone do not identify a disreputable journal.
posted by hydropsyche at 4:23 PM on June 15, 2014 [10 favorites]


I think one thing not made clear in the FPP, is that there's a clear separation between the legitimate journals and these predatory journals, such that the author is wrong to believe that he is going to be, "published in prestigious-sounding journals," if these journals are asking these authors for a fee.

That used to be true, but there are some very good open access journals that also charge page fees.

Even Springer wants to charge me a hefty fee (I forget, but maybe $2000USD) if I want to make my article open access---and this was for an absolutely good journal (Discrete & Computational Geometry). So the old standard of "charges money"="scam" doesn't work any more.
posted by leahwrenn at 4:46 PM on June 15, 2014 [2 favorites]


Not just open access. In my field, everybody from societies to Elsevier charges page charges (and remember that is on top of the subscription fees that charge to libraries). "The economics of ecology journals" Page charges are listed in Table 3.
posted by hydropsyche at 5:00 PM on June 15, 2014


I'm reminded of a plot element in Neal Stephenson's Ananthem, where the book's equivalent of the Internet had suffered an invasion of autonomous programs spamming all the search engines and databases with false and misleading information as part of some sort of cyber warfare attack. Something like the above randomly-generated scientific papers could well be used in such an attack -- if thousands (or more) of semi-valid sounding papers on every conceivable subject were created and inserted into the standard databases, faster than mere humans could remove them, the entire concept could be rendered useless.
posted by Blackanvil at 5:19 PM on June 15, 2014 [3 favorites]


Wow, hydropsyche, those numbers are crazy. Seriously, $150/page in a non-profit journal? How do researchers cope? I know at big universities like the UC system, they have budget lines for this, but surely there are researchers in ecology at the small liberal arts colleges and directional state universities whose departments aren't so flush. And not everyone has a grant...right? (After all, you have to be able to publish to get grants, no?)
posted by leahwrenn at 5:37 PM on June 15, 2014


Lol at the fact that apparently the solution is a "SCIgen detector" and not, you know, having someone read the research before it's published.
posted by threeants at 5:53 PM on June 15, 2014 [5 favorites]


Wow, hydropsyche, those numbers are crazy. Seriously, $150/page in a non-profit journal? How do researchers cope?

Pay to publish was standard in Astrophysics journals (US based) (at least 15 years ago). It dates back to when Astronomy was the domain of rich dilettantes. So, you couldn't actually even publish in Astrophysics without a publication line on a grant. The upshot was that even purely theoretical astrophysics *had* to be grant supported and grants were largely attached to big sensors/telescopes/etc. So, theoretical astrophysics (seemed to me as a grad student) to be largely dependent on big "telescopes" getting built.
posted by ennui.bz at 6:34 PM on June 15, 2014 [1 favorite]


Out of idle curiosity, I started poking around the List of Predatory Publishers. After only a few minutes, I came across an article that misspells the word "studies." ...in its title. Yep.
posted by threeants at 6:47 PM on June 15, 2014


threeants: Out of idle curiosity, I started poking around the List of Predatory Publishers. After only a few minutes, I came across an article that misspells the word "studies." ...in its title. Yep.

One of the papers I was involved in during grad school went online in a completely mangled fashion, with typographical errors and such, despite being fine when we submitted it and in the print copy. I'm not sure what exactly happened, but the entire article really looked like it had been subjected to poor-quality OCR scan. My guess is someone in the online publication chain had done exactly that (maybe they lost the electronic copy?) This was in a journal that was definitely legitimate (long established history and all that) with a major publisher.

Of course they fixed it when I complained, but for a while, it was full of lots of errors of this type despite being a real paper in a real journal.
posted by Mitrovarr at 8:28 PM on June 15, 2014


Blackanvil: "I'm reminded of a plot element in Neal Stephenson's Ananthem, where the book's equivalent of the Internet had suffered an invasion of autonomous programs spamming all the search engines and databases with false and misleading information as part of some sort of cyber warfare attack. Something like the above randomly-generated scientific papers could well be used in such an attack"

I have had an old friend contact me discussing (in part) a plan to DDoS journals by loading a lot of existing datasets into Tableau and seeing what falls out the other side. He seemed to believe that journal formats are sufficiently standardized that you could find some datasets, run correlations, and feed 'significant' findings into a template. I got the impression he was primarily motivated by a tenure formula -- even low impact factor publications total to a mighty sum if stacked high enough. I never got a satisfactory answer out of him on whether there would be any mention of the number of regressions run in the production of the articles, so I hope this remained largely theoretical.
posted by pwnguin at 9:22 PM on June 15, 2014


ennui.bz: Though the main US journals in astrophysics have page charges, over the last decade more and more people have noticed that there are prestigious journals (notably, the Monthly Notices of the Royal Astronomical Society) which do not. So you can publish in a journal with a classier name, and do it for free.

The Astrophysical Journal switched publishers a few years ago; page charges soon went up significantly, accelerating the trend away from them. They recently announced that they will soon switch to being an online-only journal.

I'm not sure how Monthly Notices is handling the associated increase in their costs -- I suspect the tragedy of the commons will take its toll.
posted by janewman at 9:36 PM on June 15, 2014 [1 favorite]


I'm guessing the Royal Astronomical Society is not a profit-driven corporation. Is there a clue in there somewhere?
posted by Segundus at 11:20 PM on June 15, 2014


I'm not quite sure how British non-profit-equivalents would be structured, but yeah, the RAS is definitely not profit-oriented. The money to run it has to come from somewhere (no idea where, though -- the Queen?).

I guess I might have been unclear: Monthly Notices' costs must go up due to more people wanting to publish there given that it's free to authors, increasing the amount of editing, layout, etc. they will have to do. When the price of publishing in Ap.J. goes up, people will inevitably seek cheaper alternatives, so long as they are viewed as being of equal quality.
posted by janewman at 11:50 PM on June 15, 2014


I have to say, based upon my own preliminary findings on Excel usability, this (from the fake IEEE abstract in the Nature article) sounds like a promising line of inquiry:
we concentrate our efforts on disproving that spreadsheets can be made knowledge-based, empathic, and compact.
posted by Sonny Jim at 12:22 AM on June 16, 2014


As leahwrenn points out the shift to open access and authors or their institutions having to pay for publication does undermine the easy assumption that a charge equals a scam (leaving aside the physics and other journals were it has long been legit). In the UK the next assessment of the quality of research will require open access if institutions want a paper to count to their assessment; this is effective from April 2016. So there will be a requirement to pay a journal in some form. Educating junior researchers will need to play a role in dealing with this, so they are clear about impact factors and such, its not really in the interests of institutions to be letting people waste money on badly thought of journals.
posted by biffa at 12:37 AM on June 16, 2014


Wow, hydropsyche, those numbers are crazy. Seriously, $150/page in a non-profit journal? How do researchers cope? I know at big universities like the UC system, they have budget lines for this, but surely there are researchers in ecology at the small liberal arts colleges and directional state universities whose departments aren't so flush. And not everyone has a grant...right? (After all, you have to be able to publish to get grants, no?)

I am at a directional state university. How it has worked is I beg the dean for money, I beg my coauthors for money, and when those don't work, I then throw myself at the mercy of the publisher, who sometimes is able to reduce (but not waive) the fees for hardship situations. Then I pay the reduced fees out of pocket from my ample state salary.

There are still days when I am not at all sure what distinguishes good, high impact factor journals published by societies from the bullshit journals in the FPP.
posted by hydropsyche at 4:27 AM on June 16, 2014 [1 favorite]


Seems to me like someone ought to be redesigning SCIgen & Mathgen to produce filable patents. The PTO pretty much doesn't check anything these days, and many companies give bonuses for filed patents, so a suitably ambitious programmer might garner themselves quite the tidy side-line in autofiled patents....

I mean, I'm vaguely aware of some attempts to autogenerate patents (but cannot now find them due to difficulty searching those terms), but they're not as well-done as SCIgen or Mathgen.
posted by aramaic at 7:41 AM on June 16, 2014 [1 favorite]


I'm guessing the Royal Astronomical Society is not a profit-driven corporation. Is there a clue in there somewhere?

The Astrophysical Journal is run by the American Astronomical Society, which is a 501(c)(3) non-profit .
posted by BrashTech at 8:01 AM on June 16, 2014


huh, I'd love to see this turned to other uses: astroturfing, tumblers, new aesthetic blogs, newspaper editorials and comics, etc
posted by rebent at 8:13 AM on June 16, 2014


Even the newer publisher for the Astrophysical Journal (IOP Publishing), which is who actually sets the page charges, is a non-profit (AAS oversees the journal, IOP does the infrastructure work -- it used to be the University of Chicago Press in the latter role). Some non-profits are more heavily subsidized than others, though; Monthly Notices has to pay their costs somehow.
posted by janewman at 8:22 AM on June 16, 2014


I'm very amused by MathGen and SciGen. I agree with aramaic that patent offices should be flooded with generated nonsense as well. LOL
posted by jeffburdges at 8:47 AM on June 18, 2014


See also theproofistrivial.com
posted by jeffburdges at 8:51 AM on June 18, 2014


« Older Only marginally less gross than that tongue eating...   |   posting such things on an Internet forum could... Newer »


This thread has been archived and is closed to new comments