You Can’t Stay Here
September 11, 2017 1:35 PM   Subscribe

The Efficacy of Reddit’s 2015 Ban Examined Through Hate Speech
In 2015, Reddit closed several subreddits—foremost among them r/fatpeoplehate and r/CoonTown—due to violations of Reddit’s anti-harassment policy. However, the effectiveness of banning as a moderation approach remains unclear: banning might diminish hateful behavior, or it may relocate such behavior to different parts of the site. We study the ban of r/fatpeoplehate and r/CoonTown in terms of its effect on both participating users and affected subreddits.
posted by Jpfed (46 comments total) 21 users marked this as a favorite
 
We find that the ban worked for Reddit. More accounts than expected discontinued using the site; those that stayed drastically decreased their hate speech usage—by at least 80%. Though many subreddits saw an influx of r/fatpeoplehate and r/CoonTown “migrants,” those subreddits saw no significant changes in hate speech usage. In other words, other subreddits did not inherit the problem. We conclude by reflecting on the apparent success of the ban, discussing implications for online moderation, Reddit and internet communities more broadly.

So there's hope... maybe?
posted by Mchelly at 1:39 PM on September 11 [5 favorites]


Well, they're still second only to the Chans as a place for hate groups to organise so I wouldn't run out the parade just yet.
posted by Artw at 1:46 PM on September 11 [13 favorites]


I'm glad they confirmed it but I hate that it's not obvious that when you give people a place to be specifically hateful, you attract hateful people and those people act more hateful than they would be without that place.

Try and imagine that in the real world. "We got rid of our Super Racist Meeting Room and all our Racist-track events at the convention, and we're noticing way less racism. It's crazy that that worked."

"Ever since the school shut down the Bullying Is Totally OK wing, we're seeing way less bullying in the school. It was a wild gamble but it worked."

"I think maybe we should reconsider our choice to have a Rapist-Friendly Dorm at the university, now that we've seen how these radical interventions have worked elsewhere. If only there had been some way to know beforehand that the Rapist-Friendly Dorm was a bad idea."
posted by edheil at 1:54 PM on September 11 [76 favorites]


Well, they're still second only to the Chans as a place for hate groups to organise

My biggest concern about reddit is it's a great place for "regular" folks to stumble upon and get sucked into hateful ideologies. It should be much, much harder than it is to jump from funny memes to MRA crap. I don't think the chans have that problem as badly...people really have to choose to seek them out.
posted by mosst at 1:55 PM on September 11 [44 favorites]


Color me skeptical that all confounders are observed to account for the difference between users who posted in these two banned subreddits and posters that did not.
posted by MisantropicPainforest at 1:57 PM on September 11 [2 favorites]


I don't think the chans have that problem as badly...people really have to choose to seek them out.

Fair, though Chans are the place gamers go from South Park style slacker whackiness and ironic racists to not-at-all-unSouth Parky misogynistic GamerGaters and actual racists, though a fair amount of that happens on Reddit too.
posted by Artw at 1:59 PM on September 11 [2 favorites]


When there's consequences for shitty behavior, people learn to moderate their shitty behavior. What a shock!
posted by SansPoint at 2:02 PM on September 11 [1 favorite]


Could we not do the snarky 'science confirms the obvious!' shtick which only serves to undermine confidence in the sciences and social sciences? Its not like we have enough to spare. Thanks.
posted by MisantropicPainforest at 2:06 PM on September 11 [55 favorites]


What MisantropicPainforest said. If the study had come to the opposite conclusion, someone would be here in the comments saying how obvious it is that banning hate speech in one place will just cause it to pop up elsewhere. You can make a common-sense argument for the obvious truth of almost any social science hypothesis. That's why we need studies.
posted by fermion at 2:19 PM on September 11 [69 favorites]


MP, I took edhell's comment to be less of a "science confirms the obvious" comment and more of an "it's absurd that Reddit didn't have this type of policy from Day 1" comment.
posted by Anticipation Of A New Lover's Arrival, The at 2:21 PM on September 11 [8 favorites]


MisanthropicPainforest: Sorry that wasn’t my intent. It’s good to have hard data on this. The problem hasn’t been a lack of data, or a lack of confidence in moderation as a way to improve the experience for people on social media. It’s been the utter apathy of platform owners (*cough*Twitter*cough*) to actually do anything because something something freeze peach.
posted by SansPoint at 2:21 PM on September 11 [7 favorites]


It’s been the utter apathy of platform owners (*cough*Twitter*cough*) to actually do anything because something something freeze peach.

I'd argue that it's more that moderation is expensive - blanket bans like this are among the cheapest solutions (on Reddit in particular, because they outsource all their moderation costs to volunteers, including the reaction from a blanket ban) but Twitter doesn't have that option, so they keep mucking around with overseas moderation teams with rigid (and often nonsensical) standards and automated bullshit.
posted by restless_nomad at 2:25 PM on September 11 [13 favorites]


restless_nomad: Even a blanket ban is still a form of moderation. I just think it’s telling that Twitter will happily crow about blocking a bunch of ISIS-linked accounts, but will let Richard Spenser keep doing his awful shit, because reasons. The money is there to deal with ISIS, but not Nazis.
posted by SansPoint at 2:32 PM on September 11 [2 favorites]


/holdmyfries, /cringeanarchy and *winces* /pussypassdenied still seem very much alive and well, so
posted by ominous_paws at 2:32 PM on September 11 [7 favorites]


Twitter hides Nazi accounts in Germany, so it's not like they don't know who they are.
posted by Artw at 2:33 PM on September 11 [20 favorites]


Artw: Which makes it all the more telling. I mean, if Nazism falls under “free speech,” why doesn’t ISIS propaganda?

And I’m totally fine with banning ISIS-linked accounts. I’m not fine with the bullshit double-standard.

I wouldn’t be surprised if Twitter’s “Communitu Standards” team was infiltrated by tech alt-right activists years ago, since they’re the department that actually enforces Twitter policy. Jack is either too distant to realize it, or too interested in flipping the company to any bidder to bother fixing it.
posted by SansPoint at 2:37 PM on September 11 [4 favorites]


/holdmyfries, /cringeanarchy and *winces* /pussypassdenied

Not to mention /t_d (the official Trump subreddit), /conspiracy, and /conspiracy_right for when the regular /conspiracy is just not racist or fascist enough for you.
posted by nathan_teske at 2:59 PM on September 11 [15 favorites]


Fascinating! I don't wholly agree with this bit from the conclusions, though:
[…] in another ongoing study, we observed that 1,536 r/fatpeoplehate users have exact match usernames on Voat.co. The users of the Voat equivalents of the two banned subreddits continue to engage in racism and fat-shaming.
In a sense, Reddit has made these users (from banned subreddits) someone else’s problem. To be clear, from a macro persepctive, Reddit’s actions likely did not make the internet safer or less hateful. One possible interpretation, given the evidence at hand, is that the ban drove the users from these banned subreddits to darker corners of the internet.

[emphasis in original]
The fact that some of the worst Reddit comments have apparently been driven to Voat does make the internet safer and less hateful, because it makes their poison less accessible. If hateful talk encourages and potentiates further hate speech, then it's better for that speech to be corralled within a relatively insular community.

This study has interesting implications for free speech advocates offline, too. People who oppose government regulations of fascist marches (like the one in Charlottesville) have argued that banning or even diverting them doesn't reduce the amount of hatred, that the remedy for bad speech is good speech &c. I have never thought that was true: one point of demonstrations is to recruit supporters by normalising and publicising the demonstrators' views. This study seems to be support for that view, and implies that (US Constitutional arguments aside) restrictions on fascist marches are a good thing.
posted by Joe in Australia at 3:46 PM on September 11 [13 favorites]


Can I whitelist the entire reddit domain at work? Not if there's any area of the site that'd get me fired. And that's a whole lot of subreddits. I think that's a fair test to see if your moderation is working.

Reddit has made these users (from banned subreddits) someone else’s problem

This was 2015 right? They used their new free time to work on the campaign. But hey, not your problem!
posted by adept256 at 4:16 PM on September 11


I'd take nazis being shunted off to some garbage site for pardophiles over them being right there front and center on what is supposedly a mainstream socially acceptable site. It's questionable to what degree this happened though.
posted by Artw at 4:34 PM on September 11 [4 favorites]


Can I whitelist the entire reddit domain at work?

Short answer? No. Long answer? Noooooooooooooooooo. There are a ton of NSFW subreddits with porno and gore/snuff material and links to dodgy sites.

On the other hand, they have r/Gifrecipes, and r/learnprogramming and r/askhistorians and r/whatisthisthing r/wholesomememes and it is a complicated place. The subreddits that are well moderated and tended like a garden of pleasure and a delight are more than worth not giving up on the place wholesale.

On at least one of the UTM firewall platforms I work on, you can whitelist subreddits by URL, just make sure you have HTTPS interception set up, as they're fully committed to HTTPS. I wish they'd make a "good subreddit filter" we could apply by category.
posted by Slap*Happy at 4:48 PM on September 11 [4 favorites]


pardophiles

Is that a typo or a slang term I'm not up on?
posted by quaking fajita at 4:49 PM on September 11


I'm assuming Artw meant to type the British form of "paedophiles" and his finger slipped over a key.
posted by zombieflanders at 4:53 PM on September 11 [1 favorite]


Can you whitelist Twitter, if that's even more of a Nazi garbage fire?
posted by acb at 4:53 PM on September 11 [3 favorites]


I hope to see Reddit ban more of these radical reactionary hate-based subs (e.g., r/t_d, r/conspiracy, any of the ones for nazis to hang out in). Reddit does not have to host those groups*. I don't go there much anymore, much less post, because even fairly innocuous subreddits can turn into a shitshow, if the wrong folks get wind.

*And no, kicking them off isn't censorship, since Reddit is a private organization.
posted by Excommunicated Cardinal at 5:00 PM on September 11 [2 favorites]


"it's absurd that Reddit didn't have this type of policy from Day 1"

It's like nobody at Reddit had ever used Usenet.

I mean, I'm not like a super old Gandalf or anything, but FFS, I used to moderate some forums for fun and profit in the 90s, and the owner and I came to some strenuous disagreements about moderation issues. Natch, I was the Randian Free Speech Tech Dudebro then.

So, I quit working there, and made my own community, which like the Chans were anything goes. It was splendid for a few months, and then the kiddie porn, pirated software, bootlegs and nazis showed up. And then once I had to start drawing lines, the userbase freaked out. And, this was 1998/9/00. After I closed that shop, I've seen this dynamic play out dozens of times before the first line of Reddit code was written. How did they miss how any of that works ?

Anyway, the whole experience was enlightening in terms of revising my libertarian positions on things. So that was a net benefit. Liberty, like a garden, requires tending to remove the undesirable elements, lest they choke the life out of it.
posted by Pogo_Fuzzybutt at 5:24 PM on September 11 [35 favorites]


pardophiles

Is that a typo or a slang term I'm not up on?



Leopard lovers?

Panthera pardus being the taxonomical name for leopard. Ahem.

posted by darkstar at 5:48 PM on September 11 [6 favorites]


My biggest concern about reddit is it's a great place for "regular" folks to stumble upon and get sucked into hateful ideologies

Is there any evidence that this is actually happening in significant numbers?
posted by paulcole at 5:50 PM on September 11


Is there any evidence that this is actually happening in significant numbers?

Other than the explosive growth of the MRA movement and alt right over the last five to ten years?
posted by Dysk at 5:52 PM on September 11 [37 favorites]


Dysk: GamerGate deserves special mention there — it might have been very easy to see through for most people but even in other communities you could see a sunset of people start repeating the talking points about censorship, media, feminism, etc. There'd always been some bigots before but they weren't all on the same page and constantly refining their message to appeal to the closeted bigots, angry young men, etc. (I remember seeing a subset of the open source world do the “I don't approve of the threats but they have a point…” shtick)

Some months laters, almost all of those newly-emboldened people were vocally all-in on Trump early in the campaign.
posted by adamsc at 6:43 PM on September 11 [18 favorites]


My biggest concern about reddit is it's a great place for "regular" folks to stumble upon and get sucked into hateful ideologies.

Yeah, the thing with Reddit is that there's a lot of cross-pollination, which is exactly what this study was looking at. The folks from /r/coontown and /r/fatpeoplehate were all over /r/pics and /r/funny, too. Similarly, the /r/kotakuinaction people are all over every single gaming subreddit (though less than they used to be, thank fuck).

On the one hand, it's just garbage people being garbage. But it's also a recruitment tool, where they can find sympathetic strangers and guide them to a place where they can be further initiated into the fucked-up asshole club.

Not to mention /t_d (the official Trump subreddit)

Which is well-known to engage in vote brigading, which the Reddit admins supposedly take super seriously, except in this one case, for some reason.
posted by tobascodagama at 6:51 PM on September 11 [4 favorites]


I hate that it's not obvious that when you give people a place to be specifically hateful, you attract hateful people and those people act more hateful than they would be without that place.

See, I would've thought it'd have the opposite effect. Haters go to other reddits / just make new reddits; they attract others to their cause by highlighting their 'oppression' eg from the chans; decent people leave those existing places as they become haterified; increasing efforts to lock down those new hate reddits create legitimacy for the 'we're oppressed' movement and causes even more people to say 'fuckit, we don't need this grief, let's go someplace else'; hate not only stays at similar levels but maybe grows a bit as a proportion of total users.

I'm glad it doesn't, but I don't think this was nearly as much of a given as you make out, particularly given the authors' carefully-worded conclusion that this worked for reddit, but created externalities for other sites.
posted by obiwanwasabi at 8:04 PM on September 11 [1 favorite]


Meanwhile linguistic data analysis of 3 billion Reddit comments shows the alt-right is getting stronger.

Redditors wised up somewhat and changed their language. Dogwhistles are a favourite of white nationalists.

They still hate women and anyone else not like them. "Cunt" is now the edgelord insult of choice, as is accusing people of being autistic. Incels, pussypassdenied, mra, tumblrinaction... all those garbage places are still active.
posted by Stonkle at 8:15 PM on September 11 [18 favorites]


Dogwhistles are a favourite of white nationalists.

Constantly-changing dogwhistles, preferably. When the plausible deniability wears off, time to switch. Makes linguistic data hard to track unless you have a timeline of preferred-dogwhistle surge and wane.
posted by ctmf at 9:00 PM on September 11 [8 favorites]


Meanwhile linguistic data analysis of 3 billion Reddit comments shows the alt-right is getting stronger.

Redditors wised up somewhat and changed their language. Dogwhistles are a favourite of white nationalists.


Which is exactly the kind of complexity that this paper sidestepped in its "definition of work"--their conclusion section makes even narrower claims than the abstract, and basically doesn't even touch the question of "was the ban overall better ["worked"] for Reddit"; literally all they claim in Section 6 is a couple observations about the bad users' post-ban behaviors, but as a Redditor I couldn't care less about them. I care if the subs that I go to are improved by this; basically I'm saying the researchers are misguided and confused by their own definition of RQ2 which is not as they claim the "second-order" question.
posted by polymodus at 9:12 PM on September 11 [2 favorites]


First thing you should do when you're going to use reddit is unsubscribe from all the default subreddits. Then carefully select the subreddits you want to participate in.
posted by Pendragon at 3:07 AM on September 12 [1 favorite]


pardophiles

That's where you can only get aroused by having your sex life narrated by Don Pardo.

Oh god I am so old
posted by Mr. Bad Example at 4:22 AM on September 12 [4 favorites]


"Musical Guest: That Enigma Album You Thought Was Sexy But Is Just Super Cheesy!"
posted by zombieflanders at 4:43 AM on September 12 [4 favorites]


The subreddits that are well moderated and tended like a garden of pleasure and a delight are more than worth not giving up on the place wholesale

Not if you're a woman.

It's amazing to me that people think the misogyny that is literally in every subreddit I've ever looked at is, of course, acceptable. Hating women, recruiting other people to hate women: totally cool.

Jesus.
posted by schadenfrau at 5:22 AM on September 12 [14 favorites]


My impression of that comment is that its not Slap*Happy thinks misogyny is totes ok, rather, they think that there is no misogyny in the well and heavily moderated subreddits.
posted by MisantropicPainforest at 5:57 AM on September 12 [5 favorites]


I can think of at least one 100,000+ user subreddit that's learned moderation over the past ~5 years or so. From what I have heard, it used to have blatantly sexist/racist comments. Now it doesn't, at least not long enough for me to see them.
posted by quaking fajita at 9:29 AM on September 12 [1 favorite]


That's where you can only get aroused by having your sex life narrated by Don Pardo.

But that's not all, you also made yourself look like a jerk in front of millions of people
And you brought shame and disgrace to your family name for generations to come
You don't get to come back tomorrow
You don't even get a lousy copy of our home game
You're a complete loser!

The lyrics make so much sense now!
posted by parliboy at 10:07 AM on September 12 [2 favorites]


Just as one example, /r/hockey is (in my experience at least) extremely well moderated. I cannot recall seeing a racist or sexist comment there. The community is highly supportive of women's hockey and pays more attention to it than any other large online site I'm aware of. It has nearly 500,000 subscribers. They do exist!

Now, I grant you it's the only subreddit I've found that I can stomach hanging out in, but I do hold out hope that there are others.
posted by saturday_morning at 2:10 PM on September 12 [2 favorites]


There are basically Two Reddits these days. A lot of subs had a major wake-up call post-GG where they realised that they needed to actually invest time and resources into moderation. I think that the Reddit admins had a hand in this change, starting when the company was under Ellen Pao's leadership. (It's taken until well after she was ejected to actually see the fruits of her effort, but I first noticed things shifting back when she was in charge.)

At the same time, the remaining fucknuggets are getting more organised and are smarter about using throwaways when they do brigadey stuff and coaching each other on how to "stay crypto" around "the normies".

So it's this weird situation where you can finally have a comment thread on /r/games that discusses a feminism-adjacent topic without devolving into a cesspool of "five guys" jokes, but also /r/the_donald still exists.
posted by tobascodagama at 2:26 PM on September 12 [2 favorites]


My impression of that comment is that its not Slap*Happy thinks misogyny is totes ok, rather, they think that there is no misogyny in the well and heavily moderated subreddits.

Oh, no, there is, because it's our society and we're soaking in it. There is demonstrably less of the activist misogyny and vicious ethno-fascism. Is it as good as here? Some subreddits, yes. Many of them. Other subreddits, it's the casual, unthinking misogyny or racism that's pretty much everywhere, online or off. It is getting better, and notably so. The slow creep of fascism, racism and MRA bullshittery into everyday subreddit threads hasn't been halted so much as blunted and diminished to where they are withering rather than prospering. It's still a good trend.
posted by Slap*Happy at 4:50 PM on September 12 [1 favorite]


schadenfrau: "It's amazing to me that people think the misogyny that is literally in every subreddit I've ever looked at is, of course, acceptable. Hating women, recruiting other people to hate women: totally cool.
"

There are small niche subs that don't have a high profile with a subject matter that has nothing to do with people that remain essentially misogyny free (EG: r/ParallelView). These are trivial examples because they sidestep the issue but enough for me to not completely write off the platform.
posted by Mitheral at 10:03 AM on September 18 [1 favorite]


« Older Somehow, this must be good for bitcoin   |   Will the 9/11 case finally go to trial? Newer »


You are not currently logged in. Log in or create a new account to post comments.