This post is not a hoax
October 4, 2018 6:47 AM   Subscribe

To "expose the reality of 'grievance studies,'" three scholars submitted 20 "hoax" papers to a variety of journals. Seven were accepted. The authors say their goal was to expose an "undeniable problem in academic research on important issues relevant to social justice." Critics say it reveals only that it's easy to lie to people who assume honesty. Dubbed "Sokal squared," after the 1996 hoax article "Transgressing the Boundaries: Towards a Transformative Hermeneutics of Quantum Gravity," the incident has generated robust debate on Twitter.
posted by Mr.Know-it-some (143 comments total) 26 users marked this as a favorite
 
System built on assumptions of good faith ill equipped to deal with deliberate bad faith conduct. That's all this shows.
posted by NoxAeternum at 6:56 AM on October 4, 2018 [56 favorites]


Excellent, a meta post about a meta study on a meta site.

Must go deeper, each level a day is worth a year, deep enough we can live forever (proof in footnotes)
posted by sammyo at 7:09 AM on October 4, 2018 [7 favorites]


System built on assumptions of good faith ill equipped to deal with deliberate bad faith conduct. That's all this shows.

Really? I mean, it does show that, no argument there. But if respected*, peer-reviewed, ostensibly scientific journals accept (and in some cases, praise) bad-faith drivel just because it's written in a superficially believable manner and in line with the relevant political stances, it seems a bit suspect to say that it wouldn't be indicative of a bigger problem.

I'm not qualified to say what this experiment proves, but I'd say it's not only that bad faith actors are hard to weed out.

*I have no personal knowledge of the respectability status of any of these journals, apart from taking the authors' word for it from the "authors say" link.
posted by jklaiho at 7:09 AM on October 4, 2018 [30 favorites]


>Part III: Why Did We Do This?
>Because we’re racist, sexist, bigoted, misogynistic, homophobic, transphobic, transhysterical...
>No. None of those apply. Nevertheless, we’ll be accused of it, and we have some insights into why.

When so many people think you're a racist, a misogynist, or just a plain old asshole that you have to issue a blanket disclaimer along with your byline... it's possible all those people are onto something.
posted by Sing Or Swim at 7:12 AM on October 4, 2018 [63 favorites]


But NoxAeternum, even if that's the only thing here - and not everyone will agree that it is - it still seems like a valuable point. We are seeing lots of systems being hikacked right now, because they assume good faith. One of the things I'm picking up in the zeitgeist is that we need to be more careful to design our systems to be resistant to bad-faith atracks. That conclusion itself is a worthy result of sociologic study.
posted by dbx at 7:12 AM on October 4, 2018 [33 favorites]


"Mother Night" seems instructive on the point the hoaxers are attempting here...
posted by hilberseimer at 7:14 AM on October 4, 2018 [4 favorites]


The project's official statement and reflection.

Article on Quillette.

Inside Higher Ed's brief article.

Wall Street Journal article (may require sub).

From Twitter:

Yascha Mounk praises the project.

Tressie McMillan Cottom: "I've never heard of these journals. But this should generate a few thousand Op-Eds and think pieces over the coming months or years so you may as well get acquainted now." (her thread)

Thread from Reviewer 1 for the Masturbation = Rape hoax paper.

Critiquing the hoax's method as bad social science.

Rolin Moe: "If your methodology in production seeks to deflect or nullify any criticism, you are not contributing to the scholastic field. As Pluckrose et al have stated they expect academia to ‘attack’ the findings, I wonder what they believe they produced, because it is not scholarship" (his thread)
posted by doctornemo at 7:27 AM on October 4, 2018 [16 favorites]


David Schieber:
I was Reviewer 1 for the Masturbation = Rape hoax paper that tried to get published in Sociological Theory. As a grad student, it was my first time being asked to review a paper for a journal. I'm glad I recommended a reject, and the paper was rejected. I remember thinking at the time that it was probably a master’s thesis that a student immediately turned around to try to get published. Lots of long block quotes with no explanation. Long sections with no organization. I mentioned this all in the review. So I structured my review off of a constructive rejection I received from ASQ where the reviewer clearly read the paper, pointed out problems, and offered suggestions for how to proceed. It was the type of rejection where I immediately wanted to work on the paper again. I don't like reviews that reject the premise of the paper outright. I've received reviews like that since my papers are on the porn industry. So I tried to buy into the paper and offer paths forward. These are the comments that the hoax authors quoted in their write up. Anyways, I guess I could be more critical in the future, but I assumed a grad student had written a confusing paper and I tried to be constructive. I'm embarrassed that I took it as seriously as I did, I'm annoyed I wasted time writing a review, and I'm glad I rejected it.
...
I just asked my wife if she remembered me talking about a paper I reviewed awhile ago, and she replied "The one that you said was stupid and didn't make any sense?" so now I feel better.

posted by Greg Nog at 7:29 AM on October 4, 2018 [88 favorites]


"If you're not asking yourself 'how could this be used to hurt someone' in your design/engineering process, you've failed." - Zoe Quinn

An assumption of good faith is often a symptom of insularity or a filter bubble that hasn't been popped yet.
posted by turntraitor at 7:29 AM on October 4, 2018 [39 favorites]


System built on assumptions of good faith ill equipped to deal with deliberate bad faith conduct. That's all this shows.

It may be what it shows, but unfortunately it may not be limited to the authors. The problem with the system is that it depends not just on good faith from the authors but also from the reviewers (and the editorial staff). I have no way of knowing the extent of the work that the reviewers did in this specific case but it doesn't look like they did the job they agreed to when they signed up as reviewers. That aside there are clear problems with incredibly thin reviews (either positive or negative), reviews where the reviewer is only in it to try and drive self-citation by recommending their own papers or to block theories with which they disagree. I would argue (as an academic author and reviewer) that these failings are growing due to the ongoing growth in papers and the increasing difficulty this creates in getting academics to agree to review.
posted by biffa at 7:29 AM on October 4, 2018 [8 favorites]


Is it even possible, let alone desirable, for systems to be built on good faith assumptions anymore?
posted by kevinbelt at 7:30 AM on October 4, 2018 [12 favorites]


System built on assumptions of good faith ill equipped to deal with deliberate bad faith conduct. That's all this shows.

I would contend that peer review in any rigorous academic field should be prepared to contend with all forms of malfeasance, from detectably falsified or skewed results, poor organization, fallacious thinking, and just plain half-assed research.

A system robust enough to weed out crappy good-faith research will catch bad-faith submissions. If you can't catch the trolls, you aren't catching the well-meaning incompetents.

And if a troll is so good, so meticulously researched and makes extremely well-supported and powerful points strong enough to pass a serious, strong, and rigorous peer review, maybe it's real science, even if only by accident.

I expect -- further, I demand -- better from the gatekeepers of academia. They can step up or they can continue to be made fools of.
posted by tclark at 7:39 AM on October 4, 2018 [38 favorites]


Is it even possible, let alone desirable, for systems to be built on good faith assumptions anymore?

Yes and yes, because the alternative is Bitcoin. And no, that's not a joke - the whole point of cryptocurrencies is that they are designed to deal with the assumption that some of the participants are acting in bad faith, and those design elements are what introduce many of the problems with cryptocurrencies in the first place.

Now, that's not to say that we should blindly assume good faith - bad actors do pop up, and need to be dealt with. But systems built on assumptions of bad faith tend to be wildly inefficient because of that assumption.
posted by NoxAeternum at 7:42 AM on October 4, 2018 [20 favorites]


Is it even possible, let alone desirable, for systems to be built on good faith assumptions anymore?

You may be right, in that an underlying assumption of good faith leads to irremediable system vulnerabilities.

But the alternative - assuming bad faith unless proven otherwise - yields systems that simply don't work at all. As a particularly pungent present-day example, consider the corrosive effects of the post-Reagan Republican commitment to the notion that Government Is The Problem and the iron determination of its supporters to force that to be the case.
posted by flabdablet at 7:45 AM on October 4, 2018 [9 favorites]


The Dog Park study included falsified data to support its claims. Fake data could get work published in a physics journal. Unlike Sokal, this is straightforward academic misconduct.
posted by justsomebodythatyouusedtoknow at 7:46 AM on October 4, 2018 [29 favorites]


I expect -- further, I demand -- better from the gatekeepers of academia. They can step up or they can continue to be made fools of.

Aren't...most reviewers volunteers? I thought this was something that people did unpaid as part of their academic service or because they wanted to support the field.

Sure, let's improve the review process, but "why aren't you doing better free labor while you also adjunct at three different colleges" doesn't seem like where I'd start.
posted by Frowner at 7:47 AM on October 4, 2018 [84 favorites]


This feels like proving I am not a vegetarian by secretly hiding meat in my food.
posted by robself at 7:48 AM on October 4, 2018 [33 favorites]


Oh, here is a post from Crooked Timber on this matter. The comments at CT tend to skew far righter than they used, but there's still several with interesting details.

This comment in particular has some useful additional information about the journals, the fact that most of these articles were submitted but rejected and some other details I didn't know.
posted by Frowner at 7:54 AM on October 4, 2018 [8 favorites]


Whelp, why am I not surprised to see self-proclaimed New Atheists involved in a trolling attempt to put Gender Studies in its place?
Stop making atheists look like jerks, thx.
posted by Kitty Stardust at 7:55 AM on October 4, 2018 [29 favorites]


This is not my field but I'm not really sold on the "we assumed good faith" argument as a get out of jail free card. Peer review is supposed to catch incompetence, laziness, spin and hype. If there are lab results it assumes those aren't faked but it is supposed to be able to catch nonsense by people who believe their own nonsense, which means it should also catch a lot of types of nonsense offered cynically. This is a breakdown and it needs more to explain it than just dishonesty on one side. (It might be still all on one side, but it's not just the lying.)

OTOH Henry Farrell has been making the point that a lot of nonsense that seemed like it violated common sense and turned out to be empirically wrong has been considered among mainstream economist types, so . . .
posted by mark k at 7:57 AM on October 4, 2018 [4 favorites]


This feels like proving I am not a vegetarian by secretly hiding meat in my food.

I'd say a better analogy is someone handing you a big cut of meat and you putting it in anyway. I'm entirely sympathetic to Frowner's point that these people are volunteers and are poorly supported by their institutions and the journals that make bank off of their free labor, but this is the peer review process. It is the very core of research, and the only thing that has delineated true science and investigation from convenient bull.

And we do a bad job of it. Look at all the elite journals that let Brian Wansink's crappy science just slide right through, a man who hoodwinked hundreds of millions with bad research. I know I tried to make a point of not shopping when I was hungry. I changed my habits, and not only is Wansink to blame, but everyone who let his feel-good crap get published.
posted by tclark at 7:57 AM on October 4, 2018 [10 favorites]


just because it's written in a superficially believable manner and in line with the relevant political stances

It's that "just because", I think, that is the biggest issue here. The people who did this certainly want people to believe that's the only reason this worked. But Frowner's second link there brings up something that occurred to me, where the authors were clearly trying to be as ridiculous as possible, so they came up with a bunch of things that they considered per se unreasonable notions, and then... wound up in several cases with topics that people who actually spend a lot of time thinking about these things do not consider per se unreasonable.

The whole fact that one of their topics was "men maybe should actually try being on the receiving end of penetrative sex" and they thought it was silly probably says something about our culture and how that impacts how men feel about receiving penetrative sex, seems like.
posted by Sequence at 7:59 AM on October 4, 2018 [61 favorites]


But systems built on assumptions of bad faith tend to be wildly inefficient because of that assumption.

Yes, inefficient or so bloated with necessary redundancies to prevent failure that they become impractical. That level of safeguard is what is hoped will prevent accidental or malicious nuclear weapon launch for example.

Almost everything else has some degree of good faith assumptions baked in. You go to a restaurant and assume they are handling the food properly, or close enough to be safe. You move along sidewalks next to busy streets assuming drivers will stay on the road. Human behavior is always the weakest element. Expecting people to be on highest alert at all times for possible failures in a system is almost doomed to fail since that level of attention is draining and we become used to routines and expect the norms will hold or "something" will clue us in when they don't. Even things like password protection for our own security is not taken seriously, so expecting greater care over potential issues that "might" occur where it doesn't even feel like we have anything directly at stake is not going to work well for long.

Unfortunately, the 21st century is proving to be the era of asymmetry, where individual bad actors can wreak greater havoc than ever before and are making use of that ability. Previous eras had bad actors of course, but they were either constrained by limitations of unamplified individual action or had power so the violations were built into the system as an expected function. That was terrible in many cases, but much more predictable. Now we have both the bad systems and less predictability for having individuals with such disproportionate ability to inflict harm on a massive scale. The commensurate ability to do good has also changed, but it can't easy get past the asymmetrical advantage bad actors have.
posted by gusottertrout at 8:03 AM on October 4, 2018 [17 favorites]




I would contend that peer review in any rigorous academic field should be prepared to contend with all forms of malfeasance, from detectably falsified or skewed results, poor organization, fallacious thinking, and just plain half-assed research.

This would require training reviewers, and possibly (*gasp*) compensating them for their time.
posted by BrashTech at 8:07 AM on October 4, 2018 [18 favorites]


This would require training reviewers, and possibly (*gasp*) compensating them for their time.

Indeed. There are easy wins, here. Fix the process, stop pretending at rigor and really do it. Compensate experts for their time. Maybe Elsevier would make a little less profit. I'll weep for their shareholders, but they'll survive.
posted by tclark at 8:11 AM on October 4, 2018 [19 favorites]


I have no difficulty whatsoever in assuming essential bad faith on the part of Elsevier.
posted by flabdablet at 8:17 AM on October 4, 2018 [19 favorites]


People are talking about "science", but these aren't scientific journals, are they? They're all in the humanities, even if some (one?) of the papers contain some kind of fabricated experimental results. The Journal of Poetry Therapy, for example. I thought that the common understanding about these kinds of journals is that nobody actually reads them... which is probably a problem, but not exactly the problem the hoaxers want to point at.
posted by A Thousand Baited Hooks at 8:20 AM on October 4, 2018 [10 favorites]


One time I cheated on a math test and didn't get caught. Ever since, I have known the terrible secret that numbers are a lie.
posted by mobunited at 8:21 AM on October 4, 2018 [40 favorites]


In practice, the assumption of good faith is often actually the assumption that people will play by the cultural rules, and those rules have been shown, sometimes only after generational changes, to often fail along racial lines, or gender lines, etc., where no one in the dominant culture was capable of noticing at the time.
posted by Obscure Reference at 8:29 AM on October 4, 2018 [3 favorites]


I expect -- further, I demand -- better from the gatekeepers of academia. They can step up or they can continue to be made fools of

THEN PAY US. We already have so many other demands on our time and our attention that it is absurd that we are expected to do this for free, and it's no wonder that so many reviews are half-assed. I don't for the life of me understand why a reviewer strike hasn't happened yet. We should at the very least be paid minimum wage, for god's sake.
posted by IjonTichy at 8:29 AM on October 4, 2018 [30 favorites]


“The topic of this essay is certainly novel and addresses an issue relevant to a disenfranchised demographic. The essay addresses bodybuilding as a stigmatizing activity toward the fat body and presents fat bodybuilding as a “way to disrupt the cultural space” of traditional bodybuilding” -Reviewer 1, Fat Studies


As someone who has written a number of peer reviews, this looks very much like a first paragraph "OK, I'm going to try to sum up what's going on here, give a very faint complement, before jumping in to talk about what's wrong here".

Interestingly, one of the journals they got accepted into is Hypatia, which published that "Transracial = transgender" article, and thus had a whole lot of restructuring/other issues this past year.
posted by damayanti at 8:30 AM on October 4, 2018 [4 favorites]


I agree that this hoax does show that a lot of research published within these fields is of dubious quality. But...with that said...

What actually really irks me about this is that these kinds of hoaxes are always, always targeted at humanities, social sciences, and gender studies, with the very specific intent of "proving" that it's all po-mo nonsense that needs to be defunded. As if hard science journals aren't also publishing research of dubious quality *all the time* as well. Most rigorous academics across all fields don't pay attention to the word salad that gets published on random, low- to middle-tier journals like the ones in this hoax. But outside of academia, there's this bizarre perception that if something can get published in a journal, any journal, then it's regarded within that field as the height of Truth.

Quality research is being published all the time in the the humanities and the social sciences, but the types of folks who keep rehashing the Sokal affair over and over again don't care about that. Yes, inadequate peer review structures are a problem, as it is in all fields, and a major problem is the economic structure of academic journals more generally, as people have pointed out upthread.
posted by adso at 8:32 AM on October 4, 2018 [48 favorites]


peer-reviewed, ostensibly scientific journals

Not sure most of these are "ostensibly scientific" publications.
posted by atoxyl at 8:45 AM on October 4, 2018 [6 favorites]


Quality research is being published all the time in the the humanities and the social sciences, but the types of folks who keep rehashing the Sokal affair over and over again don't care about that.

Yeah, that's why these kinds of stunts are so harmful. They don't just show there are weaknesses in the system, but serve to discredit it entirely, the good with the bad. Some bad articles getting through is indeed a bad result, but as important a measure is the influence of the articles that are published. The benefit of the high quality research is in its further use, which, at some point, requires some measurable effect or "proof". A bad article that points to no further use has little negative consequence. There are of course going to be low quality studies that will be cited or put to further use, but there is something of a built in limit of how long that can work if they have no value to begin with.

Damaging the reputation of journals as a whole has a whole other level of harm to it though, where the idea that communication within the field as a whole can't be trusted creates a systemic wound that is hard to heel for reasons of the inherent difficulties of foolproof research standards. On even a greater scale we may be starting to experience the results of this sort of experiment writ large by the constant attacks on newspapers and news media. When that system loses the trust of the people for at least following minimal standards of ethics, then there is no fallback to replace it and the possibility of shared believed communication becomes impossible. Repairing the damage to these systems can take considerable amounts of time and effort far out of proportion to the effort put into the stunts and what possible weaknesses they may prove.
posted by gusottertrout at 8:50 AM on October 4, 2018 [7 favorites]


Listen, how else are three also-rans gonna become New-New Atheists? I mean they know there are a steady supply of reactionary assholes out there. They just have to get them away from the Chaos Dragon and hooked on memes.
posted by mobunited at 8:56 AM on October 4, 2018 [3 favorites]


THEN PAY US. We already have so many other demands on our time and our attention that it is absurd that we are expected to do this for free, and it's no wonder that so many reviews are half-assed. I don't for the life of me understand why a reviewer strike hasn't happened yet. We should at the very least be paid minimum wage, for god's sake.

The last time we had this discussion, the point was made that peer review was expected as "service to the community".

And people wondered why I have a dim view of academia.
posted by NoxAeternum at 8:57 AM on October 4, 2018 [7 favorites]


This is also kind of interesting in that the hoaxers have announced publicly that they have deliberately defrauded both the reviewers and the institutions at which the reviewers work. I wonder if any of them are state schools where it would then be defrauding government?
posted by srboisvert at 8:58 AM on October 4, 2018 [5 favorites]


This does happen in the sciences. Just a couple of weeks ago Neuroskeptic had a post “Rick and Morty” Sting Predatory Journals:
Now, another sci-fi sting has taken place, based this time on Rick and Morty. The stinger, Farooq Ali Khan, created a hilarious paper called Newer Tools to Fight Inter-Galactic Parasites and their Transmissibility in Zygirion Simulation.

The paper, reporting experiments on the treatment of space parasites, was published in three supposedly peer-reviewed journals. You can find it at ARC Journal of Pharmaceutical Sciences, IOSR Journal of Pharmacy and Biological Sciences, and Clinical Biotechnology and Microbiology
The difference is that the stings on poor scientific journals got essentially zero attention in the mainstream media, but this story is everywhere because it's what a large group of people want to hear.
posted by TheophileEscargot at 8:58 AM on October 4, 2018 [46 favorites]


What actually really irks me about this is that these kinds of hoaxes are always, always targeted at humanities, social sciences, and gender studies, with the very specific intent of "proving" that it's all po-mo nonsense that needs to be defunded.

It's academia's version of Project Veritas, and like PJ, it is performed mainly by disgruntled males who are pissed off that girls are being allowed into their clubhouse.
posted by Lunaloon at 9:01 AM on October 4, 2018 [27 favorites]


I don't for the life of me understand why a reviewer strike hasn't happened yet. We should at the very least be paid minimum wage, for god's sake.

This is your regularly scheduled reminder that the IWW will organize anyone, anywhere, regardless of whether they are paid for the work they should be paid for or not.

But also: there are real problems with assuming good faith in papers submitted that other social scientists are then going to use to move forward. If there is no ability to verify the data, then there should be an enormous disclaimer posted that: we don't verify the data, we're just checking the basic methodology or what have you, do not use this as something to build on until it is verified. But I think that magazines wouldn't like that because half of their value is that they /are/ checking to make sure these aren't faked science.
posted by corb at 9:04 AM on October 4, 2018 [5 favorites]


[Deleted a few comments. Remember, folks, bringing people's profile information into the conversation is potentially bannable. Do not do this.]
posted by restless_nomad (staff) at 9:23 AM on October 4, 2018 [6 favorites]


When so many people think you're a racist, a misogynist, or just a plain old asshole that you have to issue a blanket disclaimer along with your byline... it's possible all those people are onto something.

Yep. "I'm not a bigot, I just play one on the internet!" is no longer an excuse, and does far more to identify one as firmly in the domain of bigots and neo-Nazis than not. In this case, they apparently chose targets based on how much the perpetrators believe this will discredit their positions. On top of that, they're being deliberately and pre-emptively shitty to their targets. When you use a word like "transhysterical," which seems to be used entirely by misogynist and transphobic bigots who are Extremely Online (remind you of any recent popular movements?), you don't get to claim you're not bigots no matter what.

The difference is that the stings on poor scientific journals got essentially zero attention in the mainstream media, but this story is everywhere because it's what a large group of people want to hear.

Oh, come on. For one, Khan's targets were specifically journals with predatory business practices, whereas this bunch of assholes was, in their own words, specfically trying to undermine those seeking social justice. Second, he didn't go around using bigoted terms as disclaimer. And finally, no: Khan's one-off getting little coverage compared to this does not somehow mean the reporting is all some plot to gin up outrage by the dastardly SJW Cabal or whoever.
posted by zombieflanders at 9:33 AM on October 4, 2018 [9 favorites]


the authors were clearly trying to be as ridiculous as possible, so they came up with a bunch of things that they considered per se unreasonable notions, and then... wound up in several cases with topics that people who actually spend a lot of time thinking about these things do not consider per se unreasonable.

Thank you Sequence for articulating another thing that bugged me about this hoax, and often the assumptions of the Conservative Campus Crusaders that any area of inquiry which doesn't apply to straight cis (often male) whites is on its face ridiculous and proof that someone has "gone too far."
posted by Kitty Stardust at 9:37 AM on October 4, 2018 [10 favorites]


I can't seem to find a full list of the journals they submitted to. They have a full list of their papers, some of the "retired" papers say "Last rejection:" but give no indication how many times they were rejected before that last one and by which journals. Given that omission, I'm unclear about whether the "successful" papers were successful on their first try or whether they had to go to multiple journals to find one that would accept them.

I can't really get a clear idea of their real success rate without that info. And shouldn't we be interested in any journals that rejected everything?
posted by RobotHero at 9:39 AM on October 4, 2018 [7 favorites]


And finally, no: Khan's one-off getting little coverage compared to this does not somehow mean the reporting is all some plot to gin up outrage by the dastardly SJW Cabal or whoever.

I'm confused. The comment you're responding to seems to be pointing out that the media is responding to this because there's a big market for undermining and attacking the concept of social justice and research associated with it. It's not suggesting that there's an "SJW cabal", as far as I can see. I think it's suggesting that there is an anti "SJW" cabal, if anything.
posted by howfar at 9:44 AM on October 4, 2018 [13 favorites]


Yeeeah. I mean, hell, technically that area of inquiry does in fact completely apply to cishet white men, but the cishet white men in question feel a bit threatened by the notion. And I mean, the paper itself may still not be very good, but "paper on legit topic that isn't very good still got published" is not anywhere near the level of attention-grabbing as "paper on something our conservative audience thinks just plain should never be seriously discussed still got published", which is the real point of this stunt.

Or rather, the more I read, the more I suspect the real point is to let the three individuals involved spend the rest of their careers explaining that their underachievement is due to discrimination because of their political beliefs and not the standard problem with there being more people interested in studying the humanities than can actually achieve real notability studying the humanities.
posted by Sequence at 9:45 AM on October 4, 2018 [11 favorites]


The difference is that the stings on poor scientific journals got essentially zero attention in the mainstream media, but this story is everywhere because it's what a large group of people want to hear.

The claim this time is that they got stuff published in comparatively respectable publications - I don't happen to know to what extent this is true. Boghossian actually tried this before with a minor, essentially pay-to-play journal closer to what the Rick and Morty guys were going after, and that didn't move the needle much outside of the crowd he was playing to.

At some point I'd think the response would be - "yes, we publish some out-there shit. That's actually within our mission, we're not a physics journal."

I haven't seen the content of any of these papers but the Hooters one sounds like a pretty regular topic to get published in gender studies. The dog park one is the most blantantly dubious premise of the ones I've heard that actually made it to print.
posted by atoxyl at 9:47 AM on October 4, 2018 [3 favorites]


One promising development for reviewing academic publications is the reproducible data science movement.

It's hard to know how widely adopted R will get, but something that is (1) capable of robust statistics, (2) extensible into most other digital applications and (3) allows reviewers et al to just plug and play with someone else's research should clear a lot of brush in these woods.
posted by Reasonably Everything Happens at 9:49 AM on October 4, 2018 [3 favorites]


I don't for the life of me understand why a reviewer strike hasn't happened yet. We should at the very least be paid minimum wage, for god's sake.

I mean, the party line of peer reviewers themselves is that peer review isn’t done for the sake of the journal, but for the sake of fellow scholars. So if you want to be paid, fine, but you will have to pay other scholars in turn when they review your work. I don’t know that I am convinced by this argument, but this is what I hear from professors and scholars themselves— peer review is the closest thing most of them have to collectively workshopping their research. I spoke to a researcher recently who said this was actually a warning sign about Brian Wansink— his preference for industry money instead of applying for government grants was, in essence, a way to avoid the peer review process that is mandated by said grants, and it prevented him from getting caught sooner.

submitted them to "the best journals in the relevant fields."

This is demonstrably false, and they are counting on non-academics not having any sense of journal credibility to further this claim. So far it seems to be working in the expected circles, which, sure. The people who think studying certain topics at all is dumb are going to think this experiment is awesome. Cool. We already knew that.

When I was in grad school, our professors frequently gave us journal breakdowns in our own corner of the humanities/social sciences, along the lines of

-Top journals (a very short list)
-Second tier but still good
-Fine but don’t waste your time until you have tenure
-Journals known more for trying to be “controversial” than conduct rigorous inquiry, still interesting stuff sometimes but everyone knows their reputation for publishing eye-roll inspiring stuff to fill the pages
-Very narrow topic journals, best for padding a CV if your school has tenure committees that don’t know the nuances of your field, not a place to get real feedback
-Vanity journals, maybe place a conference paper here if your friend is one of the editors, or if you just really really like the topic, but otherwise don’t bother
-Predatory journals, avoid

I wouldn’t say that any of the journals they targeted fit into the top three, and definitely not the top two, which are the ones that matter to people working in those fields. Hypatia is a big name journal, but it definitely has a reputation of “lol Hypatia, of course” when they publish something bonkers.

To me, this is like going to a very niche industry conference and then flipping tables when it turns out that some of the presentations are being given by people with bad PowerPoints. It demonstrates a lack of understanding about what the field values, what function lower level journals are seen as serving, how much anyone actually reads some of these journals, how many tenured academics absolutely write clickbait trash articles, etc.

Are parts of academia a scam? Uh, yeah, absolutely. Going after niche journals to own the libs seems like a pretty weak target, though. “This person who runs a vanity journal published a paper written to flatter them and their schtick” is not really impressive, as far as pwnage goes.

And conflating any of this “it’s a social experiment bro!!!!” stunt with “scientific journals don’t actually review their papers” is WILDLY irresponsible, so that is definitely what most write-ups are doing, because of course they are.
posted by a fiendish thingy at 9:58 AM on October 4, 2018 [43 favorites]


I mean, the party line of peer reviewers themselves is that peer review isn’t done for the sake of the journal, but for the sake of fellow scholars. So if you want to be paid, fine, but you will have to pay other scholars in turn when they review your work.

But the journals themselves already charge authors thousands of dollars per paper.
posted by IjonTichy at 10:12 AM on October 4, 2018 [2 favorites]


Also, what they did would not in a million years pass IRB ethics review. If even minor plagiarism is a fireable offense at most universities, I don't see why this wouldn't be as well.
posted by adso at 10:20 AM on October 4, 2018 [5 favorites]


From the project page:

"(A judgment of “Revise and Resubmit” usually results in publication following the satisfactory completion of requested revisions."

This may be true in some fields and for some journals, I suppose. To me that sounds more like a conditional accept. A revise and resubmit might sometimes result in publication after revision. Often it does not. In some highly-ranked journals it usually does not.
posted by If only I had a penguin... at 10:20 AM on October 4, 2018 [7 favorites]


I wouldn’t say that any of the journals they targeted fit into the top three, and definitely not the top two, which are the ones that matter to people working in those fields.

I did see at least one journal that for sure is in the top two categories and some others that I'm pretty sure are top 3 categories, though I would look up to confirm. All of those journals rejected the articles.
posted by If only I had a penguin... at 10:23 AM on October 4, 2018 [1 favorite]


Well, strictly, all R&R's with satisfactory revisions result in acceptance, because being accepted is what makes the revisions satisfactory.
posted by GCU Sweet and Full of Grace at 10:25 AM on October 4, 2018 [2 favorites]


But the journals themselves already charge authors thousands of dollars per paper.

In some journals? Maybe. In all journals? Not even close. And definitely not in humanities journals.
posted by a fiendish thingy at 10:26 AM on October 4, 2018 [1 favorite]


I did see at least one journal that for sure is in the top two categories and some others that I'm pretty sure are top 3 categories, though I would look up to confirm. All of those journals rejected the articles.

You're right, sorry-- I was just looking at the list of places that had accepted the articles. But "bottom of the barrel journals accept trash articles" doesn't really fit their narrative, so...
posted by a fiendish thingy at 10:28 AM on October 4, 2018 [6 favorites]


Next time I get a request to review, I’ll ask whether the publisher would prefer to pay me at my hourly rate, or flat cost for the job.

I’ll try very hard to make sure someone at the publisher sees it and responds (as opposed to the journal, which is usually very different and not so much to blame for poor review).

We should all strike: fuck the big publishers, getting almost all their work for free while enjoying minimal distribution costs and a nearly nearly captive market. It’s a staggeringly resilient multinational consolidation of world wealth and powers.
posted by SaltySalticid at 10:29 AM on October 4, 2018 [3 favorites]


The work done in these fields claims to continue the vital work of the civil rights movements, liberal feminism, and Gay Pride.
This is just a minor point but did anyone else find the author's mention of the (oddly-capitalized) "Gay Pride" (as opposed to say, "LGBT rights" or something like that) odd? Also calling out both "liberal feminism" and "Gay Pride" as distinct from "civil rights movements"?

To me, this tiny gesture seems like it could speak volumes about where these authors are coming from.
posted by treepour at 10:40 AM on October 4, 2018 [14 favorites]


As I mentioned, they even have bigotry dogwhistles in their disclaimer. They're not accidental, either, since the one I noticed does not appear to be used anywhere outside the places where bigots hang out. They're not even a whole step removed from "we totally cucked a bunch of beta soyboys" rhetoric.
posted by zombieflanders at 10:47 AM on October 4, 2018 [13 favorites]


If you believe that scientific thought is objectively true, then the content of publications must be objectively true (except for errors). The role of peer review then is to establish the objective truth of the content of a work. This is a flawed way of thinking about the world which leads you into the fascist fantasia we are now inextricably audience to.

A better way to think about things is that publication renders content immutable. The 7 papers can't wiggle and twist into another form to deceive researchers; they already are known to be frauds. The 13 other papers are arguably more dangerous to scientific thought because their false content is free from the constraints of publication.
posted by ethansr at 10:51 AM on October 4, 2018 [4 favorites]


Yeeeah. I mean, hell, technically that area of inquiry does in fact completely apply to cishet white men, but the cishet white men in question feel a bit threatened by the notion.

I should have said benefit rather than apply.
posted by Kitty Stardust at 10:53 AM on October 4, 2018


This may be true in some fields and for some journals, I suppose. To me that sounds more like a conditional accept. A revise and resubmit might sometimes result in publication after revision. Often it does not. In some highly-ranked journals it usually does not.

I'm married to a psych prof who is currently an assistant editor. 'Revise and Resubmit' in psychological science journals pretty much means you are accepted if you make the requested revisions. In my wife's career she has been strung along on repeated revise and resubmits once and when she spoke to other colleagues about it she learned that it was something this particular editor did repeatedly to many people and they were despised for lacking the guts to reject papers instead of stringing people along and wasting their time.
posted by srboisvert at 11:00 AM on October 4, 2018 [2 favorites]


I reviewed for a while for one journal in a field that rapidly become very popular, and good God we got some absolute bollocks submitted once that happened. Just really, really bad. Like incoherently written and the methodology didn't make sense and then the final section was weirdly racist, all in the same paper.

I don't think any of that was because of an attempt to publicly undermine the discipline, but it generally wasn't 'good faith' either. Much of it I suspect came from people who had noticed this field was popular now and thought it would look good on their publications list, but didn't really see the point or the value of it, so oh whatever just cobble together anything and throw it in. Little journal in a trendy field that's probably half gobbledegook anyway, easy win!

There were a lot of rejections.

So while I have some serious issues with the approach these authors took,* I also can't buy that there isn't a problem here. Journals should not be unable to distinguish between nonsense and scholarship, and nor should the ability to distinguish between nonsense and scholarship be reliant on a presumption of good faith, and nor should distinguishing between nonsense and scholarship be the sort of thing that's reserved for the top-tier journals. Which these weren't, but they are still journals that are publishing (I presume!) otherwise decent scholars, these are not pay-to-play scams or Anytown University Undergraduate Journal of Philosophy.

There are enough problems with academic publishing as it is. It really doesn't need any journals or fields to take a "will publish total bollocks, and then defend that because said bollocks was maybe introduced by bigots in the first place" stance added on.

(*not least: however they're defining 'top journals'; whatever's going on with this idea of 'grievance studies'; selective quoting from reviewers who rejected the papers, clearly presented to imply "even the ones who wouldn't publish us liked us, see!" when it reads way more to me like "here's the placatory 'good effort!' before the list of serious problems with the work", and also I don't feel like 'reviewers are overly nice and try too hard to be constructive' is a massive problem in academic publishing anyway; and what even is the Journal of Poetry Therapy, I mean come on.)
posted by Catseye at 11:11 AM on October 4, 2018 [14 favorites]


In my previous discipline, most acceptances started off as revise-and-resubmits but an R&R was no guarantee of an acceptance. It was received wisdom that if you ever had a paper accepted straight off without a revise and resubmit, you'd probably pitched too low in your journal selection to begin with.
posted by Catseye at 11:14 AM on October 4, 2018 [3 favorites]


A revise and resubmit might sometimes result in publication after revision. Often it does not. In some highly-ranked journals it usually does not.

R&R sometimes means "Your data seems fine, but can you follow instructions, since you ignored all of them at submission? Here are the instructions. You need to follow them."

If the answer to the question "can you follow instructions?" ends up being "NOPE" then the R&R often becomes a rejection. (And it isn't just tyranny-- if you can't follow basic publication requirements, how can we be sure you followed all necessary ethical steps up to this point? Especially when a bunch of those requirements are asking you to provide documentation of ethics compliance?)
posted by a fiendish thingy at 11:21 AM on October 4, 2018 [3 favorites]


I briefly confused Peter Boghossian, the philosopher involved in this hoax, with Paul Boghossian, a much better known philosopher who also happens to write about relativism and constructivism. I can't speak of Peter's work, but Paul Boghossian's "Fear of Knowledge: Against Relativism and Constructivism" is a critical but fair take on the specific flavor of relativism that seems to exercise folks like Peter Boghossian and his fans. It's about as far from the thinly veiled right-wing agitprop about the dangers of moral relativism as a book on the topic can be. (If anything, I think he's too sympathetic to his targets.) It's remarkably accessible, and I recommend it to anyone who isn't an academic philosopher but is interested in the topic.
posted by This time is different. at 11:30 AM on October 4, 2018 [3 favorites]


So while I have some serious issues with the approach these authors took,* I also can't buy that there isn't a problem here. Journals should not be unable to distinguish between nonsense and scholarship, and nor should the ability to distinguish between nonsense and scholarship be reliant on a presumption of good faith,

I don't know...I mean from a pure content point of view*, I kind of feel like I could write a paper in a week or so** that could be accepted in a medical journal. Look, if I can just make up models (I mean why bother even making up the data when you skip straight to the regression tables?) surely I can make up some finding that a medical journal would think worth publishing. And how would they know from the content (and reviewers would have only the content, though editors would have more context) that I hadn't actually done a double-blind randomized trial of this approach to treating that thing?

Does that mean medical journals are bogus or does that mean liars gonna lie?

* I saw from a "content" point of view because it's my understanding that in some fields, including medicine, people pre-register their hypotheses and studies. I suppose given more than a week, I could pre-register a study and some hypotheses and sit on it for a couple of years before publishing my made-up confirmation of my hypotheses, but it would be hard to convince a journal I had done such a study if I have no institutional affiliation to support that. Hence the "from a content point of view."

** Most of that week would be spent researching a plausible topic and what an plausible effect-size would be for my completely-made up statistical analyses.
posted by If only I had a penguin... at 11:31 AM on October 4, 2018 [6 favorites]


Helen Pluckrose, upon further inspection:

Evolutionary psychology has been undervalued in the humanities due to a reliance on what Jonathan Gottschall entitles the “liberationist paradigm;” a project of denaturalization which focuses solely on cultural constructionism. I address common criticisms of evolutionary psychology, and argue that its inclusion in literary criticism allows for a fuller exploration of the human subject and facilitates greater clarity in our identification and examination of social and historical forces.

There's more: “Not All Men” is Not a Fallacy. It is Humanism.
All the other conservative stalking horses are here, too: IdPol is bad, campus free-speech pearl-clutching, "meninism", anti-intersectionality (she calls out Kimberle Crenshaw as the architect of modern leftist IdPol). She even does covers of everyone's favorite Philosopher/Lobster King:

"We are criticizing the radical skepticism which negates the possibility of objective truth and pointing out the problems with the cultural relativism which makes consistent liberal ethics impossible to maintain. We are criticizing the intense sensitivity to language (discourses) with an onus on interpretation being more significant than intended meaning which underlies so much of the minefield of what is often referred to as “political correctness.” Above all, perhaps, we are criticizing the understanding of humans as positioned within structures of power by their identity and the associated loss of regard for or even belief in shared humanity and individuality."

It's all here folks! Marvel at the radical leftist strawman who somehow lives in denial of empirical facts! Feast your eyes on the horror that comes when you can't brush off others with "you misunderstood me"! Watch as ravenous hordes of leftists give zero fucks that you had good intentions behind that racist joke! Come one, come all to witness the awesome power of the individual, who scoffs at your structural barriers! After all, they pulled themselves up by their bootstraps, WHY CAN'T YOOOOOOU?
posted by Kitty Stardust at 11:33 AM on October 4, 2018 [22 favorites]


Does that mean medical journals are bogus or does that mean liars gonna lie?

It's not 'lying' in that sense, though. I mean, the journal I mentioned above published some quantitative stuff, I guess people could have got a fraud past the reviewers if they worked at it? Maybe some did. But the stuff that was just incoherent, poorly-argued nonsense got weeded out, regardless of whether the authors meant it sincerely or not.
posted by Catseye at 11:37 AM on October 4, 2018


And, for what it's worth, the latest PhilPapers survey of academic philosophers found that only about 3% lean toward or accept epistemological relativism.
posted by This time is different. at 11:38 AM on October 4, 2018 [4 favorites]


Paul Boghossian's "Fear of Knowledge: Against Relativism and Constructivism" is a critical but fair take on the specific flavor of relativism that seems to exercise folks like Peter Boghossian and his fans. It's about as far from the thinly veiled right-wing agitprop about the dangers of moral relativism as a book on the topic can be.

Right, but the danger zone seems to be where people
A. over-apply the belief that questioning/deconstructing our epistemological models means there are vast numbers of researchers out there who literally believe there is nothing objectively knowable and facts don't exist
B. then come to believe this fringe point-of-view is commonly accepted in academic circles, which leads to
C. college students are effectively being taught that nothing is real. And then there's the people who
D. hear that these leftist academics "deny objective reality" and begin to apply the term "objective reality" to their preferred ideological positions , i.e. hierarchy is natural and inevitable, and white men have the highest IQs, therefore white men should be at the top of the hierarchy, etc. So if you agree that there is a hierarchy, but refute that it's natural and inevitable, you're "denying objective reality."

Like, I understand why advocating for the position that facts are literally unknowable and everyone gets to inhabit their own personalized bubble of reality would make a lot of academics lose their shit, but that's kind of what postmodern analysis was charting. (Those "alternative facts" people didn't come by that position because of any academic. Most people in general don't read and definitely never engage with any academic theories.) Not creating, just recording from existing social forces. Outside the academic world, yeah, plenty of people do live in hermetically sealed bubbles of self-defined reality that are impervious to reason. Some people see that and think "postmodernism described this, so postmodernism must have created this!" Other people just can't accept that The Enlightenment fucked up so fully in having huge privileged blind spots that actually reinforced hierarchy instead of spreading the equality that was its own stated aim. Rather than admit that, they think it must be the people critiquing that broken promise who ruined the dream of progress. If only they didn't insist on Making a Thing out of being so female, black, disabled, etc, we'd all be equal by now. It must be those pesky IdPol-ers who made people start to notice that a hierarchy still existed even though the people in charge of it kept telling them they were equal now.
posted by Kitty Stardust at 11:57 AM on October 4, 2018 [20 favorites]


"Shared humanity" is a hell of a value for anyone on the right to be advocating at this particular historical moment. In fact, it's funny that they should be doing this project at all at a time when the right-wing president has documentedly lied thousands of times over the past couple of years, to total indifference from his supposedly Objective-Truth-Worshipping supporters.

Remember: bad-faith claims don't require extensive good-faith engagement.
posted by praemunire at 11:57 AM on October 4, 2018 [10 favorites]


Here's a really nice post showing the ranking of the journals that these were placed in:

http://www.favstats.eu/post/hoax_papers/

Bottom line: these's weren't predatory or pay-to-publish low ranking crap journals. But they were low ranking journals, and the higher ranked journals that they submitted them to were more likely to desk reject the papers.
posted by MisantropicPainforest at 12:37 PM on October 4, 2018 [9 favorites]


Oh, I also love how these conservative intellectuals talk about the primacy of individuality when there have literally been whole disciplines--legal, scientific, intellectual concepts with mind-bending twists of logic (just look at who is included/excluded in whiteness, for example) that have been devoted to elevating them on the basis of in-group membership.
posted by Kitty Stardust at 12:37 PM on October 4, 2018 [3 favorites]


It's not 'lying' in that sense, though. I mean, the journal I mentioned above published some quantitative stuff, I guess people could have got a fraud past the reviewers if they worked at it? Maybe some did. But the stuff that was just incoherent, poorly-argued nonsense got weeded out, regardless of whether the authors meant it sincerely or not.

Well, let's look at the data on data. This is admitedly poorly collected data, but I looked at all the papers and it seems like the papers that were published were disproportionately those that DID claim to collect and analyze data. Note: I did this quickly, without any previously-determined coding guidelines. I did not read the papers, I looked at the abstracts and scanned the papers for a methods section. I have in several places used the word "Review essay" which you should take to mean something along the lines of "paper format I am unfamiliar with in my field. Seems to use existing literature to make a new argument but not collect new data."

Here is where they seem to be on data use:

Dog Park: Claimed to collect and analyze data (Accepted )
Fat Body Building: No data. Looks like a quasi-editorial type thing (Accepted)
Dildos: Claimed to collect and analyze data (Accepted)
Hooters: Claimed to collect and analyze data (Accepted)
Hoax on Hoaxes: Does not appear to collect data. Looks kind of like a review essay? (Accepted)
Moon meetings: not anything I would call data. More like a reflection. This is a poetry journal, I’m not sure data is a thing in poetry journals? (Accepted)
Feminist Mein Kempf: No data. Also has a review essay sort of look (Accepted).
Porn: Claims to collect and analyze data. (Revise and resubmit)
Progressive Stack: No data, looks like a review essay (reject and resubmit)
Feminist AI: No data. Review essay (abstract actually uses the word essay). (Minor revision)
Feminist Astronomy: No data. Review essay. (Revise and resubmit)
CisNorm Claimed to collect and analyze data (Under review)
Masturbation: No data. Review essay (Rejected)
White Mein Kempf: Claimed to collect data (not the kind of data I’d normally see, but who knows what other fields do). (Rejected)
Queering Plato: No data. Review essay. (rejected)
Feminist Body Building 2: No data. (Retired...whatever that means)
BJ-Gay: No data (retired)
Hoax on Hoaxes 2: No data (Retired)
Autoethnography: Nothing I would call data. Reflection. (Retired)
Schizophrenia: Nothing I would call data. Reflection (Retired).
Rebranding masculinity: can’t find it in paper archive.

Breakdown
So 3/7 accepted papers claimed to collect and analyze data.
1/4 of R&R papers claim to collect and analyze data.
Either 0 or 1 of 3 rejected papers claimed to collect or analyze data (depending on what one calls data).
1 of 1 still under review papers claimed to collect or analyze data.
and 0 of 5 retired papers claimed to collect and analyze data.

I think there's a solid element of liars gonna lie here.
posted by If only I had a penguin... at 12:45 PM on October 4, 2018 [5 favorites]


Like, I understand why advocating for the position that facts are literally unknowable and everyone gets to inhabit their own personalized bubble of reality would make a lot of academics lose their shit, but that's kind of what postmodern analysis was charting.

I'm not sure I'm following your comment KS. Are you saying that postmodernists weren't really endorsing constructivism or relativism, they were just recording the way that most people experience the world? I don't want toPostmodernists are a diverse bunch with a variety of takes on what postmodernism is really about, but there's no fair reading of the movement that says substantive commitments to controversial kinds of relativism or constructivism weren't characteristic of the movement, even if they weren't universal.

I think that those commitments were pretty thoroughly mistaken and ultimately detrimental to the academy, but that none of that has anything to do with what is animating Peter Boghossian and his fans. As you say, they're mad that people are calling attention to oppression at all, and 'postmodernism' is nothing but the label they use for that.

So I'm saying, "This Boghossian's attack on 'relativism' is right-wing agitprop. But if you're actually interested in this sort of relativism and constructivism, here's an accessible, academically rigorous book by a different Boghossian that offers a sympathetic but ultimately critical take on the relevant kinds of relativism and constructivism." For what it's worth, the first chapter includes a discussion of the gap between the significance and popularity people *believe* these kinds of relativism and constructionism enjoy on college campuses and their actual status as fringe views.
posted by This time is different. at 12:49 PM on October 4, 2018 [2 favorites]


Here's a really nice post showing the ranking of the journals that these were placed in:

Ah, that graph is useful if you want to actually look for differences between the journals that did and did not publish them, rather than if you want to discredit the entire field of study. Though it might be missing additional journals that rejected them. The graph pulls data from the same list I had read before which used the phrase "Last rejection:" which is what got me wondering how many previous rejections were unlisted.
posted by RobotHero at 12:51 PM on October 4, 2018 [3 favorites]


Oh, I see that "retired" means the paper is fatally flawed and that in fact the retired papers have been rejected. So the updated numbers are:

So 3/7 accepted papers claimed to collect and analyze data.
1/4 of R&R papers claim to collect and analyze data.
1 of 1 still under review papers claimed to collect or analyze data.
Either 0 or 1 of 8 rejected papers claimed to collect or analyze data (depending on what one calls data).
posted by If only I had a penguin... at 1:05 PM on October 4, 2018 [1 favorite]


Yeah, "retired" vs. "rejected" seems to reflect whether the authors believe they could have got it published somewhere else if they kept trying. Which raises the question of how many repeated attempts did their published papers take.
posted by RobotHero at 1:32 PM on October 4, 2018 [3 favorites]


"There are too many angry people out there trying to push an agenda, instead of seeking out The Truth," said the people who spent 10 months publishing bad papers to troll SJWs.
posted by shapes that haunt the dusk at 2:09 PM on October 4, 2018 [32 favorites]


Re: Bad Faith...

The discourse of these hoaxsters is political speech and not descriptive speech, i.e. meant to evoke feelings and provoke reactions. For example, this is a project that claimed to go after "cultural relativism" by targeting feminist journals. Consider that for a moment.

Even their own description of "grievance studies" implies an ethical framework (in which a grievance should be validated), a cultural context for the grievance (in which a grievance can be said to exist), and a normative claim to rectify the grievance (i.e., that's the whole point). That contradicts relativism.

What they are evoking, however, can be formulated in a proposition: social justice work does not belong in academia. This means:

1) You can't trust their descriptions of their own hoax-papers, e.g. as "outlandish," because any paper for the journals they submit to would be "outlandish" in the political speech they use.

2) You cannot trust that their conclusions follow from their premises. What is the relation between cultural relativism, for instance, and feminist journals? On what basis are they claiming that, for example, falsifiability is the heart of academic inquiry in the humanities and theoretical parts of the social sciences, when their own work is not falsifiable? Can that very argument about falsifiability be verified?

3) You should be suspicious of the means by which they make *any* argument, because the argument itself is in bad faith. The components of the argument cannot be taken with good faith any more than the argument as a whole. We can see this with, for example, the reviewer comments, which they excerpt rather than show in full, and do not display at all for the papers that were rejected.

4) There is a very real target here, about academic pluralism. These papers largely got through peer review not because the reviewers necessarily agreed with them, but because reviewers are generally pluralists about theoretical frameworks. THIS IS AS IT SHOULD BE. Academics are no better and no worse than anybody else, and allowing critical frameworks past review that we do not agree with is how we establish conviviality in academia. Basically no academics are actually relativists; in fact, we couldn't **grade** our students' work if we were. But we agree to get along in a space that could be described as relativistic so that we can all move forward. Claiming that some types of work do not belong in academia attacks the social foundation by which the humanities and qualitative social sciences are able to function.

I hope the people who reviewed these papers file IRB grievances with Portland State. A lot of them.
posted by migrantology at 2:57 PM on October 4, 2018 [26 favorites]


I'm not sure I'm following your comment KS. Are you saying that postmodernists weren't really endorsing constructivism or relativism, they were just recording the way that most people experience the world?

I just typed out a novel here, but then my browser crashed & I lost it.

No, I agree that postmodernists definitely characterize knowledge as socially/culturally constructed, and they do talk about how there are often no objective external scales of value for most cultural practices, since all those scales originate in at least one culture's values (the dreaded cultural relativism). I guess I'm saying I never read postmodern theory as prescriptive, though. Like, it seemed to be explaining existing phenomena rather than laying out a plan. I'm not sure I'm describing it well. Maybe that their work was intended to demonstrate that knowledge is constructed rather than compel producers of knowledge to engage in construction while doing so? Also that a stance of skepticism toward claims of epistemic authority wasn't invented by postmodern philosophers (people in general were already questioning the authority of religion, state, social order, family in the Modernist period) but it was widely applied by them.
posted by Kitty Stardust at 3:04 PM on October 4, 2018 [4 favorites]


For example, this is a project that claimed to go after "cultural relativism" by targeting feminist journals.

Because these sorts of head-in-the-sand bozos think the only way you could question the validity of traditional gender roles is some sort of "cultural relativism" that ignores "common sense."
posted by straight at 3:17 PM on October 4, 2018 [5 favorites]


skepticism toward claims of epistemic authority wasn't invented by postmodern philosophers

Yeah. Setting up "postmodern relativism" against "realist philosophy" isn't really descriptive of what actually happened. There are, and always have been, varying degrees of scepticism among philosophers of all schools about the possibility of knowledge, the reality of facts and the relativity of truth; that's it, that's the whole story.

I know people thoroughly embedded in the analytic tradition who are strong epistemological sceptics, and at least one committed Deconstructionist who is a devout physical realist. People differ.
posted by howfar at 3:20 PM on October 4, 2018 [4 favorites]


The thing that frustrates me the most is that they keep acting as thought having an agenda is this terrible thing, and that Real Scholars are neutral and objective, and just observe truths. Bullshit! Everyone has an agenda. Claiming that you don't have one is just a way to avoid having to answer for it.
posted by shapes that haunt the dusk at 3:21 PM on October 4, 2018 [11 favorites]


Bottom line: these's weren't predatory or pay-to-publish low ranking crap journals. But they were low ranking journals, and the higher ranked journals that they submitted them to were more likely to desk reject the papers.

This post is good - looking at all the papers together you can also see that by title and purported topic many of these are not exactly outlandish for the field. It doesn't speak well of the amount of effort that went into reading some of them, and for someone to be able to say "hah! I can fake your discipline and I don't even believe in it!" has some sting - but how much would one really bet that one couldn't pull this off with e.g. whoever publishes Satoshi Kanazawa and Gad Saad?
posted by atoxyl at 3:23 PM on October 4, 2018 [3 favorites]


As a non-academic and someone who after reading this thread sees some merit both in the original project and the thoughtful critiques here, one additional element of information I would find illuminating is the citation records/quantities of the journals that did accept the spurious articles, i.e. how often are their contents cited by others, compared to the journals classified as Tier One above and/or the journals which rejected the offered articles.
posted by PhineasGage at 3:28 PM on October 4, 2018 [1 favorite]


I guess I'm saying I never read postmodern theory as prescriptive, though. Like, it seemed to be explaining existing phenomena rather than laying out a plan. I'm not sure I'm describing it well.

Endorsing some form of relativism or constructivism doesn't commit them to staking out any sort of position about what to do about it---though postmodernists definitely did stake out such positions. But regardless, that doesn't make the sorts of relativism or constructivism they did endorse any less controversial.

Maybe that their work was intended to demonstrate that knowledge is constructed rather than compel producers of knowledge to engage in construction while doing so?

Maybe so, but then they're on the hook for some kind of constructivism, and that's a substantive and controversial claim unless "constructed" is reduced to something trivial enough that even someone like Boghossian (either one!) can agree.

None of this is to say that the postmodernists were wrong about anything, or that nobody should consider their claims about truth and knowledge. But I don't think it's right to present their work as largely descriptive, or to limit their claims to broadly acceptable forms of questioning oppressive structures. Lots of different movements did that without adopting some of the weirder and controversial claims about the nature of truth and knowledge that were characteristic of postmodernism.

Though, to be fair, I think a lot of folks who have said stuff like "knowledge is constructed" or "truth is relative" don't really mean it. They really are trying to say we should be more epistemically humble and more suspicious of people who claim they have found the truth. That's a good attitude, but it is also one that has been deployed *against* relativism since Plato's Protagoras.
posted by This time is different. at 3:31 PM on October 4, 2018 [1 favorite]


Oh , for one journalist to ask: "what was the most prestigious journal you managed to defraud?".

Although, with the exception of papers where they claimed to have measured (presumably interesting and genuine) data, some of the blame has to go to the journals. But then, some of these things look like the academic version of outsider art, which is what these guys produced, with confessedly fraudulent intent.
posted by Wrinkled Stumpskin at 3:33 PM on October 4, 2018 [2 favorites]


And some of the wackier shit on the list got accepted by, like, The Journal of Poetry Therapy. Yeah man, you really took the Journal of Poetry Therapy down a rung.
posted by atoxyl at 3:34 PM on October 4, 2018 [19 favorites]


An image I saw on my FB feed seems appropriate.

It was a Venn Diagram with 2 spheres that were almost entirely overlapping.

One said: "People who think Sokal hoax proves that cultural studies is bullshit."

The other said: "People who think that because lobsters have social hierarchies, men should dominate women."
posted by Saxon Kane at 4:15 PM on October 4, 2018 [29 favorites]


Excellent, a meta post about a meta study on a meta site.

They needed a meta filter.
posted by Quackles at 4:23 PM on October 4, 2018 [1 favorite]


You go to a restaurant and assume they are handling the food properly, or close enough to be safe.

I go to a restaurant and assume that they have been reviewed and certified by my local health department as following required standards for food safety and sanitation. It's not that I trust the restaurant owner to do the right thing because they're a good person, I trust that the systems of examination and certification are doing their job. (Incidentally, a health inspector was at one of my favorite restaurants when I got lunch the other day. The owner was chatting with the inspector, apparently completely relaxed and unworried about what might be found. I smiled and dug into my meal.)

There's a middle ground between "assumption of good will" meaning I don't bother to ask how you're doing things and "assumption of ill will/incompetence" meaning I assume you'll spit in my food unless I watch you all the time.
posted by Lexica at 4:34 PM on October 4, 2018 [8 favorites]


Because the link was in a long list above, much earlier in this conversation, it's worth highlighting a Twitter thread by Yasha Mounk, who addresses many of the criticisms that have been raised here. Short version: "...even if you think the authors are idiots, their hoax was unethical, and other disciplines are crap, it wouldn’t change a simple fact: Serious journals in which academics who teach at top universities publish cannot distinguish between bullshit and true scholarship."
posted by PhineasGage at 5:40 PM on October 4, 2018 [2 favorites]


I go to a restaurant and assume that they have been reviewed and certified by my local health department as following required standards for food safety and sanitation

I think in the restaurant analogy, when the health inspectors came, the restaurant owners claimed to be sterilizing their probe thermometers with alcohol swabs between uses*, but actually the swabs were just soaked in water and lemon juice.

*ok, I had to look up what a restaurant would be required to sanitize and how, the point is, even a food safety inspection has some assumption of good faith. They don't send the cleaning solution or dish soap off to a lab to make sure it's up to code.
posted by If only I had a penguin... at 5:42 PM on October 4, 2018 [2 favorites]


One more thing I thought about... Desk rejects are more common in some fields than others. I think in some fields a desk reject means "this is not our area at all" (i.e. why did you send a paper analyzing the content of medical textbooks to a journal of art therapy, or whatever). That doesn't mean editors aren't judging the papers before sending them out. And when the editor gets a stinker, do you know what they do? They pass it off to some grad student to review.

Not that all grad student reviewers get stinky papers, but an editor is not going to waste a top researcher in the field to do a review on a crappy paper and then not be able to call on them later (either because the reviewer won't review another paper 3 weeks later for the same journal, or because the journal has rules on how much time has to pass before re-using a reviewer). Passing it off to a grad student is win-win if it is a high-status journal (like the one the grad student reviewer quoted above was reviewing for): the journal gets to go through all the motions necessary to reject the article and the student gets to do a practice review and add reviewing for a high status journal to their CV.

When I was in grad school a prof in my department did my a favour. He was the editor of a big name journal. A few days later I found a paper in my mailbox with a request to review (this was back when submissions were made by mailing 4 copies of the paper...). I was SO excited. Then I read the paper and it was instantly clear what was happening. A 9-page piece of crap. Basically someone trying to take something that would have been an interesting footnote in their main paper into a whole other paper (and then sending that footnote-paper to a top journal). I wrote the review of the crappy paper. The crappy paper was rejected.
posted by If only I had a penguin... at 5:56 PM on October 4, 2018 [3 favorites]


The real TLDR here (aside from the shitty people being shitty angle) is how much peer review is done by poor, desperate, and barely qualified grad students, hoping for a crumb of recognition for their arduous labor, freely volunteered.

When similar threads come up about artists and ‘creative’ types on the blue, a familiar mantra is:

Fuck you, pay me

I think this ‘gotcha’ is largely worthy of ignoring as unscholary and unethical shit fires, but I also think the current model of unpaid peer review is bullshit that sends about 15 bn of public money in the USA to enrich private corporations, every year, and I think all scholars who care about sustainability and equitable access to publishing should reject that model too.
posted by SaltySalticid at 7:41 PM on October 4, 2018 [7 favorites]


God damnit, Portland, Oregon.
posted by cosmologinaut at 8:56 PM on October 4, 2018 [1 favorite]


It was a Venn Diagram with 2 spheres that were almost entirely overlapping.

One said: "People who think Sokal hoax proves that cultural studies is bullshit."

The other said: "People who think that because lobsters have social hierarchies, men should dominate women."


The flip side of this sentiment is that it establishes the left as mostly post-modernism. This is worth a reality check, because the left was arguably doing better prior to so many people identifying it with post-modernism, combined with the fact that post-modernism doesn't seem to convince anyone to switch sides. That doesn't mean it lacks influence though. Maybe colleges slow-invented it to deal with the fact that most students are from homes steeped in conservative values, and it performs an intellectual intervention/exorcism function. The danger comes later when they go out in the real world, but only self-righteous elitism comes out of their political mouths. Unintended side-effects of deprogramming, perhaps. Not a fan, obviously, but more concerned that anything called culture is also someone else's brainwashing, or it awkwardly refers to socially-accepted art, for lack of anything more knowledgeable to call it.
posted by Brian B. at 9:10 PM on October 4, 2018 [2 favorites]


The dog park one is the most blantantly dubious premise of the ones I've heard that actually made it to print.

From this write-up: Perhaps the most egregious of the hoax papers, "Rape Culture and Queer Performativity at Urban Dog Parks," argued that observation at Portland dog parks revealed that dog owners were less likely to interfere when male dogs nonconsensually humped female dogs than other male dogs. This, the paper concluded, was evidence of rape culture.

So the hoax paper obviously has a lot of tedious bullshit in it. But like... the thing that strikes me is that this is not actually the ridiculous part of the paper! If they had collected the data and the observation I'm quoting above were actually true (and the methods weren't a joke), why wouldn't it merit being published somewhere? It's not earth-shattering, but it also seems to me like that probably would actually tell you something about the beliefs of dog owners. I mean, it's not purporting to be an observation about rape culture in dogs. (This is probably the only time I will ever write this in my life, but a blogger at reason.com, famed defender of feminist lefty queer studies academia [lol jk] agrees with me on this.) Joan Roughgarden, for instance, has some really interesting work about how this kind of projection of human beliefs onto animal behavior ends up being both an impediment to actually understanding it, and also ends up being used circularly to justify certain behaviors or situations as "natural." So the fact that the columnist chose that made-up statistic as inherently ridiculous, as opposed to any of the actual crimes against method and writing perpetrated in that paper, is really revealing to me. As others have said, it seems like the ridiculous part to them is studying how rape culture filters into ordinary life at all.

Also something that was funny to me is that if you look at the actual reviews for that paper, 2/3 seem to call them on quite a bit of their bullshit if you read between the lines, they are just way too polite about it. One reviewer calls them on handwaving about Black feminism and recommends they either make their case better or take out that connection altogether. The author (? I think) gleefully interprets this as being directed to add "Even More Black Feminism(s)," conveniently omitting the second half of the sentence which completely changes the interpretation. Another reviewer says they have concerns about their methods... I mean, as well they should.

It's still a failure of peer review that it was published, I think, but that's more a failure of the editor of the journal than the reviewers (particularly if these reviewers were grad students, which I kind of suspect because they wrote a lot, hedged what they were saying a lot, and were overly deferential). It's a failure of the editor to find reviewers who were most qualified to review the paper -- in this case that would be people who actually knew something about quantitative methods or animal behavior -- and it's a failure of the editor to actually look at what the reviewers were saying and to read between the lines.

But those problems also completely exist in, for example, the life sciences. There was a paper that immediately comes to mind that actually made it all the way through to Science, a way higher-ranked journal than any of the hoax targets for those keeping track at home, where the authors claimed to have invented a kind of microarray for intracellular chemical reactions. The problem is that the paper describing this alleged groundbreaking advance was reviewed by biologists, not organic chemists, and an organic chemist would have been able to see right away that the methods couldn't possibly have yielded the products the authors described. Indeed, the paper was later retracted after an "expression of concern." Again, it's a failure of the editor to pick the right people to review the paper. It doesn't mean that metabolomics or biology is bullshit, it means that it's hard to detect fraud, and that goes double for work that cuts across traditional disciplinary lines. (I'm not even getting into the fucking "arsenic life" paper.)
posted by en forme de poire at 10:02 PM on October 4, 2018 [16 favorites]


That "thread from reviewer 1" that got posted upthread is pretty illuminating also. Basically, yes, grad student trying to constructively criticize a confused submission. Displays a lot more intellectual humility and integrity than the authors of this hoax IMO...
posted by en forme de poire at 10:23 PM on October 4, 2018


I go to a restaurant and assume that they have been reviewed and certified by my local health department as following required standards for food safety and sanitation. It's not that I trust the restaurant owner to do the right thing because they're a good person, I trust that the systems of examination and certification are doing their job.

Well, for vehicles we demand people pass a test before being able to drive as proof they understand the laws and know how to safely operate a vehicle. We employ police to enforce those laws and failure to follow basic safety measures can result in death. Yet people still don't follow the laws and people die for that failure. It's just how humans work. Hospitals routinely are cited for not following proper procedures; doctors don't wash up properly, instruments don't get sterilized adequately, wrong medications are given out and so on. There is no field of human endeavor where the best practices will be followed at all times unless, perhaps, there is an enormous amount of resources put in place to build up a system entirely devoted to double and triple checking each action.

With journals, even in the best cases a lot of articles and studies will be published that are, essentially, worthless. That's only to be expected. Not every study is going to be groundbreaking or shed important light on a subject. Many will just be empty of meaningful data or ideas save for perhaps some reinforcement of already accepted ideas or as some evidence of areas not worth pursuit. Some work will be downright bad, but benignly so for at least accepting the basic premises of the discipline. Scholars have to publish, journals have to have things to publish. Some arguments are going to be faulty, limited, some studies shoddily constructed or anecdotal, but that is expected by anyone who reads these sorts of publications and the worst offenders will be called out for their failings if it is felt worth the effort to do so for the harm they might cause the discipline. Most though will just get ignored for being so obviously of little lasting interest. Some published work will be more malignant for faking data or otherwise presenting a distorted argument intended to deceive, but even then the values of the discipline aren't being questioned, just attempted to be worked around for personal benefit.

This case, however, is attempting to undermine the disciplines or larger construct of the humanities itself. It's the difference between saying don't believe everything you read no matter where it may come from, which is sound advice, and saying don't believe anything you read especially from this group of people. That can only be intended as a attack on the values those groups hold and, through that, the groups themselves. There is no opposing proof of better practice or meaning to hold as measure for the attacks to have value since there are no publications that could possibly pass a "we've never published crap" approach. The attack feigns there being some outside the discipline truth that is being singularly missed within the humanities, but that not only doesn't exist but is virtually disproved by the very means attacks opponents make as they are so often based in bad faith without much more to back them up.

Does that mean the journals shouldn't strive for better practices? Of course not. There are failings in the system that need to be improved upon but the system can never be flawless. That isn't really a problem though since these kinds of attacks are basically meaningless except for reputations. The journals only act as initial gatekeepers for studies and arguments. They don't determine larger merit beyond that function. Things that pass through those gates then are subject to further review and counter arguments within the disciplines or are ignored and left to die forgotten. More interesting and valuable works feed the discipline and help it grow by their proof of utility.

Flawed thinking and studies still get through as they may feed unconscious biases or not account for things yet unknown, but that's a natural part of any area of study. It's as true of the hard sciences as the humanities as any perusal of history would show. There is no way around the need to rely on trust at some level. Having methods to vet and hold people to better standards and practices is important and useful, but doesn't replace trust as development would be impossible if only absolute certainty of result was allowed as a basis for making claims.
posted by gusottertrout at 11:04 PM on October 4, 2018 [4 favorites]


Last year I became acquainted with a woman, around a quarter century in age - I am reminded of a time she was lamenting that she had pursued an undergraduate degree in one of the Studies, and that it was worthless. As the stereotype would have it, she was in fact employed as a barista. I consoled her, "It's really not all that worthless, you've still got that credential separating the gentry from the commoners."

And here, in Studies and Theory at the graduate level, the machinery of publishing is part of granting a higher class distinction, more priestly in nature.

That said, I see "bad faith" said here and elsewhere, as if it fucking matters. These journals and fields are engaged in a political project, these three hoaxers are politically opposed - there's no "good faith" in politics. And if these three make fools of certain journals, there are no faith referees to determine that though these journals appeared to have embarrassed themselves, it actually doesn't count, because of "bad faith". There's just the embarrassment. The only solution is to be better at vetting,

So increase the intellectual rigor of the publishing process, if we look at this as a process of intellectual production, but that's hard - and can result in the rejection of those operating in "good faith". If we look at this as a process conferring social status, that seems a lot easier. Simply increase the vetting of the individuals submitting papers, rather than the papers - do they actually exist? are they the type of person deserving of increased social status? This would certainly have easily caught "Helen Wilson".

(Another option - declare that the works are Actually Good, regardless of intent, and therefore the field did not shame itself - but, repeated, that would allow adversaries to dictate anything they could sneak in was Actually Good.)
posted by save alive nothing that breatheth at 12:19 AM on October 5, 2018 [1 favorite]


System built on assumptions of good faith ill equipped to deal with deliberate bad faith conduct.

Little early for a new election thread, isn't it?
posted by rokusan at 1:54 AM on October 5, 2018 [1 favorite]


Our recommendation begins by calling upon all major universities to begin a thorough review of these areas of study (gender studies, critical race theory, postcolonial theory, and other “theory”-based fields in the humanities and reaching into the social sciences, especially including sociology and anthropology), in order to separate knowledge-producing disciplines and scholars from those generating constructivist sophistry

Whatever the hell Knowledge-Producing Disciplines is supposed to objectively be.

The peer review issue aside, the best arguments against Critical Theory Grievance Studies I've seen come from its own writings, not by such "liberals" nor Jordan Peterson copycats.
posted by polymodus at 2:05 AM on October 5, 2018 [1 favorite]


Whatever the hell Knowledge-Producing Disciplines is supposed to objectively be.

Those disciplines that don't inconvenience white cis het people it seems. Or to be a little more generous and take at least Pluckrose at her word and judging from Areo, the magazine she is editor of, her view seems to be of a static liberalism that only changes from the status quo when there is unquestionable evidence to support a change of values.

The magazines' article on Kavanaugh, for example, pushes for a watch and wait, do nothing, approach since we don't know with certainty what happened and therefore could further an injustice by asserting a view absent that evidence. How one would vote in such circumstances isn't covered, so the article avoids addressing whether Kavanaugh should be seated by the Senate, which is a pretty big thing to leave out and it doesn't question the comparative likelihoods of validity of claims for and against Kavanaugh or the political system in which his nomination came to be. It treats the entirety of the issue as something of abstract importance rather than an issue of pressing concern. It carries an assumption of some unclear process being at work to ensure best outcomes and the rest of us shouldn't take sides.

That seems to be about par for the magazine, advocating a kind of staticism, a belief in where we are "now" is something to be protected for being the norm, and the norm should hold simply because it is. Their arguments are for a kind of "in-betweenism" that purports itself as taking no sides, and criticizing both the left and right, but the subjects the articles examine in taking that we're above the fray approach seem to be aimed primarily at causes and thoughts of the left, writing and race and gender questions as might be found in left leaning publications without doing the same for right leaning publications. So the aim seems to be to inhibit change even as they espouse liberal values around feminism, race and other issues in doing so.

Overall, the apparent values seem to be in placing faith in traditional methods of information gathering, using the kinds of historical sources and methods Western civilization has so long relied on as providing the needed records and methods for inquiry. Challenge to those methods or records itself appears to be thought evidence of a misguided agenda.

tl;dr version they want to stick to the old forms of knowledge and traditional subjects and approaches for scholarship, anything else is radical no matter who says they may suffer ignoration or more serious harm because of them.
posted by gusottertrout at 4:30 AM on October 5, 2018 [5 favorites]


Pluckrose lives in one hell of a glass house; after all, she believes evo-psych has relevant applications to literary criticism.
posted by Kitty Stardust at 5:01 AM on October 5, 2018 [3 favorites]


Pluckrose lives in one hell of a glass house; after all, she believes evo-psych has relevant applications to literary criticism.

No kidding. The only books I've ever literally thrown across the room after reading them were books on evo-art/lit studies and one of them was a library book. I strongly considered trashing it entirely to keep some other poor soul from reading it, but, damn my weakass liberal soul, couldn't bring myself to do it out of respect for the value of libraries.

As some slight praise, even though the Areo folk call themselves liberal, they signal themselves more as something like idealized conservatives, favoring tradition and arguing against change, but at least acknowledging there are issues and some arguments to be made for leftist points of view. Were Areo actually representative of the right in the US we'd be in a much better place as there'd at least be starting points for discussion.
posted by gusottertrout at 5:21 AM on October 5, 2018 [2 favorites]


AFAICT, the hoax articles broadly seemed to fit into three categories:
  • Outright frauds: all of the articles which purported to be backed by data were of this form. And some of them had conclusions which would be worth knowing and relevant if they were actually true! The problem is, no journal in an evidence-driven field is very well-protected from this: anyone can make up data, and reviewers typically don't have the resources to verify or replicate the reported data. This is where the assumption of good faith is so vital. This hardly seems to be a problem exclusive to any individual field, but common to the natural and social sciences.
  • Complete, errant nonsense: the ones which are facially meaningless seemed, by and large, to get rejected, which meant the system worked.
  • Vaguely plausible contrarian views: this gets back to the matter of pluralism mentioned upthread, and cuts to the question of what a journal in the humanities's function is. If you're recording the currents of academic thought in your field (rather than trying to set down eternal truths), should you reject something off-the-wall but connected to the worldviews you explore? There's a good argument that even a kooky notion deserves an opportunity to sink of swim in the marketplace of ideas, and there's a long history of people putting forward things which struck them as obvious hoaxes only to find that their ideas has significant traction with people that think differently than them (Foucault's Pendulum flirts with the notion that all of hermetic thought is a result of such hoaxes getting out of hand).
posted by jackbishop at 7:33 AM on October 5, 2018 [4 favorites]


I agree with jackbishop's point about contrarianism. Humanities articles are meant to be discussed and debated, and in some cases they can be valuable less for the truth of their results than for the conversations they open. These conversations can happen, for instance, when academic articles are assigned as readings in classes. Humanities students are not necessarily expected to take everything they're assigned to read as gospel truth. A major teaching tradition in the humanities—critical pedagogy—encourages students to judge the merits of arguments for themselves and reach their own conclusions. From this perspective, articles that make "out-there" claims can serve a valuable pedagogical tools even if (or perhaps especially if) they are not persuasive.

Of course this doesn't excuse the use of shoddy methodology in empirical research. But some of these hoax papers deal with issues that are more ethical than empirical, and in ethics there will always be room for debate.
posted by obvious at 8:09 AM on October 5, 2018


Outright frauds: all of the articles which purported to be backed by data were of this form. And some of them had conclusions which would be worth knowing and relevant if they were actually true! The problem is, no journal in an evidence-driven field is very well-protected from this: anyone can make up data, and reviewers typically don't have the resources to verify or replicate the reported data. This is where the assumption of good faith is so vital. This hardly seems to be a problem exclusive to any individual field, but common to the natural and social sciences.

Even if the data in the hoax articles were accurate, the analysis in them is total bullshit. They should have been rejected on that ground, many times over. It is that sloppy thinking that the hoaxers sought to expose, not the inability of the peer reviewers to identify fraudulent data.

So I think the fact that the data was falsified is a red herring.
posted by andrewpcone at 8:54 AM on October 5, 2018 [2 favorites]


Here is more in-depth recap and commentary from Yasha Mounk.
posted by PhineasGage at 9:03 AM on October 5, 2018


It is that sloppy thinking that the hoaxers sought to expose, not the inability of the peer reviewers to identify fraudulent data.

If the data is falsified, then the analysis fit to that data is harder to claim as bullshit on its face, even if it is still unlikely. Much more importantly though, the hoaxers did not seek to expose sloppy thinking, but to discredit specific belief sets. The values they tut-tut in so-called grievance studies are the same they rely on themselves in their own publishing as can be seen in Areo magazine were there are explicit calls to rely on the very norms they violated because they didn't like the use they were put to in humanities journals.

You can't ask for good faith for your own beliefs but deny it to those you oppose without running into fatal contradiction.
posted by gusottertrout at 9:09 AM on October 5, 2018


It is that sloppy thinking that the hoaxers sought to expose

Except, of course, for the fact that they are explicitly using the tactics and rhetoric of every violent internet hate group that has tried to "expose" those on the left (especially marginalized groups). But yeah, let's go with that being just a complete coincidence.
posted by zombieflanders at 9:18 AM on October 5, 2018


Outright frauds: all of the articles which purported to be backed by data were of this form. And some of them had conclusions which would be worth knowing and relevant if they were actually true!

The now retracted dog-park article can be found on Sci-Hub here. It's bonkers, with stuff like "while I closely and respectfully examined the genitals of slightly fewer than ten thousand dogs..." The author supposedly spent at least 200 days of her life in a single year sitting in parks looking at dog genitalia and waiting for dogs to rape/hump each other (Nicolas Guéguen, a researcher in behavioural research, was recently accused, among other things, of such dubious logistical issues). The cited paper about non-human rape doesn't mention rape or forced copulation in dogs at all, something that the author doesn't discuss, even though she claims having witnessed about 1000 of such incidents. The dog park article is about 15 page long: 3 are dedicated to the results, 2 pages are about methodology and 10 pages are about buzzword-filled theory. Also, for a 1000 h-long observation study with a shitload of variables, there's not a single table showing the data, which are peppered throughout the text. It's totally true that such a study, done correctly with proper methodology and data collection, would actually be interesting. But it's just a terrible, inflated paper using dubious methodology, bizarrely formatted data, and it should not have seen the light of the day.
posted by elgilito at 9:23 AM on October 5, 2018 [6 favorites]


The dog park article is about 15 page long: 3 are dedicated to the results, 2 pages are about methodology and 10 pages are about buzzword-filled theory. Also, for a 1000 h-long observation study with a shitload of variables, there's not a single table showing the data, which are peppered throughout the text. It's totally true that such a study, done correctly with proper methodology and data collection, would actually be interesting. But it's just a terrible, inflated paper using dubious methodology, bizarrely formatted data, and it should not have seen the light of the day.

This is actually a fairly accurate description of the "methodology" of a lot of evo-psych articles as well, and 95% of Jordan Peterson's output-- it's just that evopsych and Peterson's "studies" do so to claim "rape is a natural way to spread seed because of cavemen" instead of "we live in a rape culture". (Also a description of most of Johann Hari's career, and people link his work here all the time.)

I don't mean to engage in both-sides-ism here, truly-- but the brokenness of the academic publishing model is not unique to leftist thinking, which is the actual argument of these authors. It sounds like one of these authors regularly writes for a publication that regularly publishes equally facile claims based on nonsense evidence, logical fallacies, and jargon. And yet they didn't try to publish in an ideological variety of low-level journals with malfunctioning review systems-- they targeted the ones based on topics they disliked, and decided that their findings somehow implicated entire fields of study. That's the problem. Their actual methodology is as bad as the intentionally terrible methodology they were pantomiming in these fake papers.

For example, several people have pointed out that in their "gotcha" writeup, they intentionally do not include a table that lays out their findings in a systematic fashion, cherry-pick the findings they find the most exciting, and anyone who wants to know what they did find has to do a lot of digging and googling to find those results on their own. If their "j'accuse!!!" is based on crappy methodology getting traction in places it should not, then reifying that practice in their "final" product seems like a great way of undermining their whole endeavor.
posted by a fiendish thingy at 9:36 AM on October 5, 2018 [7 favorites]


It is that sloppy thinking that the hoaxers sought to expose

No. If that were the case, they would not have chosen exclusively journals which study subjects they think for political reasons ought not to exist. Their hypothesis is literally: "Scholars in fields we are ideologically biased against the very existence of have such inherently low standards that they'll publish even faked garbage." As a hypothesis of any kind, that is a fucking disgrace.

Just...please. Stop it with the pretending that this stunt was carried out in good faith, in the name of good scholarship and careful thinking. They don't give a damn about any of that. All they want is to "own the libs," by any means necessary. How anyone after two years could be such a gump as to attribute pure intentions and a dispassionate dedication to good scholarship to such people is beyond me. Just because someone mouths a value you share doesn't mean you should blindly fall in line behind them.
posted by praemunire at 10:41 AM on October 5, 2018 [13 favorites]


The authors were quite clear and open that they were focused on "certain fields within the humanities..." and they believe "Scholarship based less upon finding truth and more upon attending to social grievances has become firmly established, if not fully dominant, within these fields." That is their explicit argument. It's not "bad faith," and saying 'journals in other fields are also imperfect' is the definition of whataboutism.

The authors have beliefs about certain academic fields, and they set about to investigate them. It's fine to say their investigation didn't demonstrate what they say it did, but the other objections posted above don't address the authors' core argument. And saying "we don't have to engage with them because..." just leaves the cultural conversation in the hands of those who agree with the authors, whether sincerely OR just to "own the libs."
posted by PhineasGage at 11:05 AM on October 5, 2018 [1 favorite]


The flip side of this sentiment is that it establishes the left as mostly post-modernism. This is worth a reality check, because the left was arguably doing better prior to so many people identifying it with post-modernism,

Of course, "conservatives" (read: the dominant classes) have been postmodern before there was a postmodernism. When it comes to asserting that truth is relative, the people in charge always can and will bend every fact, detail, observation, and experienceto suit their perspective. Cf. W Bush vs. the "Reality-Based" community.
posted by Saxon Kane at 11:24 AM on October 5, 2018


The authors were quite clear and open that they were focused on "certain fields within the humanities..." and they believe "Scholarship based less upon finding truth and more upon attending to social grievances has become firmly established, if not fully dominant, within these fields." That is their explicit argument.

Just because the GOP claims to care about fiscal responsibility doesn't mean we take them at their word, when all of their actions contradict that claim.

Just because Walmart says that they care about the well-being of their employees doesn't mean we take them at their word, when all of their actions contradict that claim.

When people are acting with a transparent and obvious agenda, the fact that they claim to have a different and more reasonable-sounding agenda doesn't mean you have to take them at their word.

The fact that they have mimicked the language and methodology of scientific inquiry doesn't mean that their grudge project was conducted scientifically, and pretending that it does does real inquiry a disservice. Their "real" project is just as slipshod as the "fake" papers they produced.

It's not "bad faith," and saying 'journals in other fields are also imperfect' is the definition of whataboutism.

"They said they were acting in good faith, so they must have been," appears to be your take. I respectfully disagree.

When their thesis is that "only certain journals are this terrible, and they are the ones I am predisposed to dislike because of personal biases", then responding "there are lots of journals just this terrible on your side as well, and it is a problem regardless of the topic of inquiry" is not whataboutism, which is an intentional rhetorical effort to derail a conversation. I actually agree that the discussion about journals publishing badly researched papers is one worth having. I just don't think it should be led by people with such a transparently vengeful agenda, and aimed only at publications that tend to contradict their own worldview.

It reminds of of every new #MeToo accusation of a Republican, where Republicans say "WHAT ABOUT AL FRANKEN!" When they say that, it is whataboutism, because their intention is not "let's investigate all cases and root this behavior out", but instead "stop being mean to our friends because some of your friends suck too."

"This is a problem that happens regardless of topic, and it is shaped by certain pressures inherent to academic publishing" is not the same thing as saying "your guys do it too!!!!1!!". That would be whataboutism. Furthermore, most of the people working in the fields these authors hate so much are equally opposed to this publishing model of low-level tripe, and the authors' fantasy that there is some kind of powerful and punitive postmodernist cabal running these journals is as baseless as any of their "silenced all my life" rantings.
posted by a fiendish thingy at 11:27 AM on October 5, 2018 [5 favorites]


It's not "bad faith," and saying 'journals in other fields are also imperfect' is the definition of whataboutism.

This is sloppy. Surely, their argument needs to be that these fields are somehow exceptional in their willingness to publish garbage. Otherwise, while they may be able to conclude something about academic publishing, they cannot conclude that these specific fields are unique in any way. To do so, by definition, they must compare their results to other fields. And since they have presented their work as an indictment of these fields specifically within the humanities, which is evident from the first paragraph of their piece in Areo (where they literally use the word "especially," which means exceptionally or uniquely), it follows that their data do not necessarily support the conclusion that they reached.
posted by en forme de poire at 11:45 AM on October 5, 2018 [5 favorites]


It is bad faith because they clearly say that White Fragility is snake oil bullshit. These authors are racists through their own scientism and refusal to examine their own theorizing.
posted by polymodus at 11:53 AM on October 5, 2018 [2 favorites]


I mean, one of their foundational claims is that scholars in these fields think these journals are consistently producing good work worth reading, which is nonsense! I have heard professors working in GWS, for example, warn graduate students that publishing in these types of journals might actively prevent you from getting hired for desirable academic jobs, because it shows you don’t have good judgment or a solid grasp of the state of the field.

(Also, just like Pyrex getting bought out and turning from a trusted name into junk, journals that once had solid reputations sometimes get taken over by publishers or EICs who value quantity over quality, or with ideological hobbyhorses. You don’t need to perform a bunch of fraud to figure that out and then pat yourself on the back.)
posted by a fiendish thingy at 12:09 PM on October 5, 2018 [5 favorites]


That is their explicit argument. It's not "bad faith,"

It is bad faith, because "fields I am politically opposed to are inherently incompetent at scholarship" is not a respectable intellectual position reached by logic based on facts; it is an ideological article of faith. There is literally nothing in the idea of, say, gender studies that carries with it the implication that it cannot produce meaningful scholarship, unless you are eaten up with ressentiment that prestigious people are taking an interest in how women have been treated in every sphere of existence and have noticed that this treatment is generally shabby. This is not scholarship, it is a political stunt meant to prove a foregone conclusion. Pretending otherwise is the essence of bad faith.

saying 'journals in other fields are also imperfect' is the definition of whataboutism

No, it really isn't. Even assuming that the method is valid, which is questionable, if their interest actually was in exposing sloppy thinking by journal authors or poor review standards by journals--entirely valid goals in themselves--they would have been very careful to spread their hoax articles out amongst many different journals in different fields and reliant on different methodologies. (How mysterious that there are no economics journals (that I recognize, anyway).) To the extent certain journals in certain fields are identified with intellectual positions that have political connotations, they would have been careful to submit across the spectrum. They would do this specifically to neutralize any suggestion of political bias and to determine publication standards across a wide range of fields. Instead, they picked their subjects in fields they resent the very existence of--"grievance studies." If their mere use of that term doesn't make you question everything about the intent and objectivity of this so-called study, then your bullshit detector is permanently broken.

Seriously, anyone who believes that these people went in with open minds and a burning concern about quality of scholarship rather than "owning the libs" is either a hopeless sucker or a hopeless ideologue.
posted by praemunire at 1:24 PM on October 5, 2018 [10 favorites]


"Eppur si muove." Yes, they went in with "a burning concern about quality of scholarship" in specific fields. And the critiques here continue not to engage the core point. In the words of Yascha Mounk: "It is nonsensical to insist that nonsense scholarship doesn’t matter because you don’t like the motives of the people who exposed it, or because some other forms of scholarship may also contain nonsense."
posted by PhineasGage at 1:37 PM on October 5, 2018 [1 favorite]


It is bad faith, because "fields I am politically opposed to are inherently incompetent at scholarship" is not a respectable intellectual position reached by logic based on facts; it is an ideological article of faith.

"Eppur si muove." Yes, they went in with "a burning concern about quality of scholarship" in specific fields

It's the difference between "we are concerned about the quality of scholarship in these fields and would like to see it improved" and "we intend to cast doubt on the value of any and all work in these fields," isn't it? I also have the sense that they pay lip service to the former while meaning something closer to the latter - look at this statement:

What do we hope will happen? Our recommendation begins by calling upon all major universities to begin a thorough review of these areas of study (gender studies, critical race theory, postcolonial theory, and other “theory”-based fields in the humanities and reaching into the social sciences, especially including sociology and anthropology), in order to separate knowledge-producing disciplines and scholars from those generating constructivist sophistry.

Note how they're trying to frame it as showing not just that "* Theory" is compromised, but whole fields like sociology and anthropology - which didn't even accept any of their papers. In general I think this stunt serves mostly to rile up people who are already on their side - it mostly goes after easy targets (apologies to the discipline of poetry therapy, but you know) and niche, openly subjectivist fields, and then tries to leverage modest success to lift much larger claims. The people who approved the dog park paper should probably be embarrassed, but I've already seen e.g. Yascha Mounk citing the "masturbation is rape" paper - which was not actually among those accepted.

I'm not sure there's anything inherently "unfair" about the "fake paper expose" tactic, though - they're just playing it somewhat deceptively and showboating for the home crowd.
posted by atoxyl at 2:06 PM on October 5, 2018


"It is nonsensical to insist that nonsense scholarship doesn’t matter because you don’t like the motives of the people who exposed it, or because some other forms of scholarship may also contain nonsense."

Great quote, except no one here is disagreeing that nonsense scholarship doesn't matter, or isn't an embarrassment.

The argument of these authors is that certain disciplines are worse at weeding out nonsense than other fields (and that this proves these disciplines are bad). They didn't prove that, because their methodology was laughably bad, and their selective presentation of their results exposes the sloppiness of their "work".

Seriously, where do you see anyone here saying "bad scholarship and sloppy peer review is fine?" People are saying it is unsurprising in certain contexts, not that it is good.

Go read Retraction Watch and you'll find other trash papers slipping through other cracks in a myriad of disciplines, including the hardest of hard sciences. The problem is endemic, and these people are claiming that it is specific to topics they dislike. The latter part is their actual argument, and when people disagree with them, that doesn't mean they are disagreeing with the first half.
posted by a fiendish thingy at 2:10 PM on October 5, 2018 [10 favorites]


Demonstrating that some nonsense scholarship has been published in some journals in one field is not the same thing as demonstrating that that field is especially likely to publish nonsense scholarship. Evidence for one is not necessarily evidence of the other, which is important because the remedies in each case are different. The authors of this hoax seem to be making the latter claim when talking to press and the public. Can you explain what makes you think this is not the case?
posted by en forme de poire at 2:10 PM on October 5, 2018


If I did a bit about "x% of women were in car accidents last year," I shouldn't attribute their poor driving to their gender if I've never actually looked at other genders for comparison.
posted by RobotHero at 2:13 PM on October 5, 2018


Reading about this on Reddit was a downer. A lot of people are eating this up because it tells them what they wanted to hear.
posted by shapes that haunt the dusk at 2:30 PM on October 5, 2018 [1 favorite]


In the words of Yascha Mounk: "It is nonsensical to insist that nonsense scholarship doesn’t matter because you don’t like the motives of the people who exposed it, or because some other forms of scholarship may also contain nonsense.

This is simply incoherent. To say that insisting X is nonsensical is nevertheless ascribing and then judging the motivations of other people. It makes the rest of that sentence absurd.
posted by polymodus at 3:21 PM on October 5, 2018


Maybe we need "secret shoppers" for all academic journals, trying to place fake articles...
posted by PhineasGage at 3:56 PM on October 5, 2018 [1 favorite]


I don't mean to engage in both-sides-ism here, truly-- but the brokenness of the academic publishing model is not unique to leftist thinking, which is the actual argument of these authors. It sounds like one of these authors regularly writes for a publication that regularly publishes equally facile claims based on nonsense evidence, logical fallacies, and jargon. And yet they didn't try to publish in an ideological variety of low-level journals with malfunctioning review systems-- they targeted the ones based on topics they disliked, and decided that their findings somehow implicated entire fields of study. That's the problem. Their actual methodology is as bad as the intentionally terrible methodology they were pantomiming in these fake papers.

Sure, but I don't think that's such a problem. Obviously this trio has an ideological ax to grind, and it would be nicer if they were wholly dispassionate and innocently concerned for the epistemic sanctity of the academy. But I don't really care what their motivations are, or whether those motivations constitute "bad faith." I care what sort of bullshit they were able to get through what sort of peer review. And I care what steps the reviewers take to improve their performance.

If this stunt inspires people on the left to similarly bullshit-test journals they find offensive, then great! Besides, it isn't just political ideology that corrupts the academy, and it's not just the humanities. Even in the natural sciences, faddish topics can become their own little ldeologies, and scientific communities can become echo chambers, in which researchers are held to lower standards the more fashionable their subject matter.

One of the main points of peer review, I think, is to be a security system against bullshit. And one of the most important tests for any security system is whether you can beat it. I think of it like penetration testing for computer systems or the TSA having people smuggle bomb-lookalikes through security. This stuff isn't a waste of time. It's what keeps the security systems trustworthy.
posted by andrewpcone at 4:18 PM on October 5, 2018 [6 favorites]


At the end of the day, their stunt did a lot of harm. It reinforced widely-held beliefs that certain subjects, especially subjects dealing with marginalized people, were just vehicles for unsupported bullshit. It reinforced widely-held beliefs that marginalized people themselves are motivated more by nonsense grievances than by real problems. It reinforced widely-held beliefs that humanities and social sciences are nothing more irrelevant activist bullhorns, at a time when STEM subjects are valued (in reputation and funding) above everything else.

Bad papers will always be published, no matter what safeguards are in place. There will always be some way for something to slip by. But we're at a political and cultural moment that sees entire departments being defunded (I saw an anthropology department's budget get cut by 30% a year). All these three have done is point out a problem most people in the affected fields were already aware of. In so doing, they've helped further erode public trust in institutions and in very specific fields, which is exactly what they set out to do.
posted by shapes that haunt the dusk at 4:59 PM on October 5, 2018 [5 favorites]


From a just posted interview with them:
Pluckrose: It’s not a secret that we want to fix the left so that it will win and avoid right-wing problems like Trump.

Lindsay: It’s also in no way surprising that people would say we’re tools of the right. It makes sense to clearly articulate [our views]. We’re of the left. [Grievance ideologues] don’t speak for us and we don’t think they speak for the left either. These people are lunatics.

Pluckrose: This is the natural response if you feel that your own group is doing something wrong. The answer is to address it from within.

So then is your goal to improve academe or is it to further a political agenda?

Pluckrose: Liberal science needs to have a respect for truth and a solid epistemology based on evidence. It’s not doing social justice causes any favor to be shoddy and inconsistent. Our main objective isn’t to forward a political agenda. It’s to make scholarship more rigorous. We accept that sometimes that may produce results which we’d rather wish to not be true.
posted by PhineasGage at 5:47 PM on October 5, 2018 [2 favorites]


Pluckrose: Liberal science needs to have a respect for truth and a solid epistemology based on evidence. It’s not doing social justice causes any favor to be shoddy and inconsistent. Our main objective isn’t to forward a political agenda. It’s to make scholarship more rigorous. We accept that sometimes that may produce results which we’d rather wish to not be true.

Except this paragraph does not respect its own epistemologies, makes an inconsistent argument, implicitly forwards a political agenda, and doesn't ask if there are some questions they'd rather conveniently ignore in the process. This is not that difficult to see. It's rather offensive. Why do they not get it? I have no idea, maybe they should have spent more time honing those liberal thinking skills.
posted by polymodus at 9:20 AM on October 6, 2018 [1 favorite]


Why do they not get it?

Because once a person has learned to scream "Projection! Projection!" completely reflexively in response to any suggestion that their motivations might not be as high-minded and holy as they claim to be, they stop needing to bother with all that worrisome and time-consuming reflection and self-awareness stuff.
posted by flabdablet at 11:32 AM on October 6, 2018


It’s not a secret that we want to fix the left so that it will win and avoid right-wing problems like Trump.

It's at least good to see they've got their fingers on the pulse of the problem and understand it well. If there's anything Trump's ascendancy clearly shows it's that writers for the Sociology of Sport Journal are trying to follow the same path to power.
posted by gusottertrout at 6:31 PM on October 6, 2018 [8 favorites]


Any system assuming good faith without gentle yet effective ways of actually establishing good faith/credibility is a system ripe for exploitation. In this day and age, I'm kind of surprised this one still exists. What this implies to me about common sense and sound judgment in our society concerns me.
posted by dancing leaves at 4:22 AM on October 7, 2018 [1 favorite]


The Chronicle of Higher Ed has assembled more commentary from scholars in a variety of disciplines.
posted by PhineasGage at 2:05 PM on October 10, 2018


Because once a person has learned to scream "Projection! Projection!" completely reflexively in response to any suggestion that their motivations might not be as high-minded and holy as they claim to be, they stop needing to bother with all that worrisome and time-consuming reflection and self-awareness stuff.

Ha! I'm enjoying how much this applies to both the "grievance studies" academics and the hoaxers themselves. It seems reflection and self-awareness are, as usual, in short supply.
posted by andrewpcone at 9:34 PM on October 14, 2018


It's getting kind of old reading about these authors took sociology down a peg and published all these articles in sociology journals, given that the hoaxters were completely shut out of sociology.
posted by If only I had a penguin... at 6:35 AM on October 15, 2018 [2 favorites]


« Older BARTENDER: “What is this, a joke?”   |   Today's menu: Virginia possum Newer »


This thread has been archived and is closed to new comments