Welcome To The Trauma Floor
February 25, 2019 8:10 AM   Subscribe

The Secret Lives Of Facebook Moderators In America Content warning: This story contains discussion of serious mental health issues and racism. posted by the man of twists and turns (59 comments total) 34 users marked this as a favorite
 
I am a bit confused by the minor Twitter frenzy today over this article only because I thought this has been reported in depth at least once in the last few years. Do we collectively have a short memory?
posted by stevil at 8:26 AM on February 25 [8 favorites]




Quis disrumpentes ipsos disrumpet?
posted by gauche at 8:36 AM on February 25 [11 favorites]


> Do we collectively have a short memory?

I think the fact that social networks provide deep, ever-growing, vile cess pools in which the worst of humanity is continually reflected and reinforced and through which low-wage moderators without benefits have to trawl so that shareholders and owners can profit further without having to consider the effects of what they have wrought upon the world is worth repeating as often as possible.

But that's just me, y'know.
posted by humuhumu at 8:43 AM on February 25 [43 favorites]


"You will be able to pause the video, he tells her, or watch it without audio. Focus on your breathing, he says. Make sure you don’t get too caught up in what you’re watching."

Jesus. I've seen a lot of disturbing murder videos, and like not having audio or being able to pause isn't going to help any goddamn thing. My question is why they need to watch them in full anyway, liike, once it's clear it's a murder video shouldn't you be able to make that call then and there? What benefit is there to psychically torturing your own workers when the end result is functionally the same if the company didn't have a policy of abusing their workers in training?

Even without those things, the working environment and conditions are inhumane. This is everything wrong about a call center job cranked up to 11. Down to not having anything joyful or stress-reducing at work and having every single minute hawked over, with punishments if a bathroom break goes too long. All of this shit for 15 bucks an hour being contracted by a company making billions and billions and billions of dollars every year. Fuck Facebook, Fuck Cognizant.
posted by GoblinHoney at 8:45 AM on February 25 [65 favorites]


If every developer at Facebook had to spend two weeks a year doing this, there wouldn't be a Facebook left in three months.
posted by mhoye at 8:45 AM on February 25 [106 favorites]


The moderators told me it’s a place where the conspiracy videos and memes that they see each day gradually lead them to embrace fringe views. One auditor walks the floor promoting the idea that the Earth is flat. A former employee told me he has begun to question certain aspects of the Holocaust. Another former employee, who told me he has mapped every escape route out of his house and sleeps with a gun at his side, said: “I no longer believe 9/11 was a terrorist attack.”
This is the origin of Harley Quinn playing out in real life, with Facebook as Arkham Asylum and the collective mass of Internet shitheads playing the role of the Joker.
posted by Holy Zarquon's Singing Fish at 8:46 AM on February 25 [80 favorites]


.
posted by SonInLawOfSam at 8:49 AM on February 25


Miguel is also allotted nine minutes per day of “wellness time,” which he is supposed to use if he feels traumatized and needs to step away from his desk. Several moderators told me that they routinely used their wellness time to go to the restroom when lines were shorter. But management eventually realized what they were doing, and ordered employees not to use wellness time to relieve themselves. (Recently a group of Facebook moderators hired through Accenture in Austin complained about “inhumane” conditions related to break periods; Facebook attributed the issue to a misunderstanding of its policies.)

All the pictures of the chintzy, trite motivational posters and cheap imitation-midcentury furniture and ersatz Silicon Valley office design dovetail nicely with this:

Everyone I meet at the site expresses great care for the employees, and appears to be doing their best for them, within the context of the system they have all been plugged into.

Indeed. It's just like how there's always money for social workers to "guide" you through "resources" if you're homeless or hungry or poor, but then once you sit down with them it turns out that the "resources" basically don't exist anymore, and they "guide" you through things like "there's a sliding scale clinic, but they can't treat serious illnesses" and "there basically aren't any shelter beds, especially for women, and what there is turns out to be "a thin mat on the floor in a chaotic and noisy room'".

Be nice, but never question the basic set-up, even when it's killing people.
posted by Frowner at 8:49 AM on February 25 [72 favorites]


I am only a quarter through the article (I hope?) but I can say that, although I have indeed read about this kind of thing quite a bit, this article really does a great job at filling me with a level of theodical Cthulhu-horror, akin to a distinct certainty that consciousness is nothing but a thin skein over the raging sea of horror, hatred, and chaos that is the ultimate reality, that I really haven't found from other think-pieces.

Such as:

The moderators told me it’s a place where the conspiracy videos and memes that they see each day gradually lead them to embrace fringe views. One auditor walks the floor promoting the idea that the Earth is flat. A former employee told me he has begun to question certain aspects of the Holocaust. Another former employee, who told me he has mapped every escape route out of his house and sleeps with a gun at his side, said: “I no longer believe 9/11 was a terrorist attack.”

And:

Facebook also worked to build what Davidson calls “state-of-the-art facilities, so they replicated a Facebook office and had that Facebook look and feel to them. That was important because there’s also a perception out there in the market sometimes … that our people sit in very dark, dingy basements, lit only by a green screen. That’s really not the case.”

etc. etc.
posted by Stilling Still Dreaming at 8:49 AM on February 25 [8 favorites]


Barely 30k a year to be perpetually traumatized...
posted by Young Kullervo at 8:59 AM on February 25 [4 favorites]


The existence of facebook is immoral, in like, dozens of ways. This is one of them. Horrifying.
posted by odinsdream at 9:24 AM on February 25 [10 favorites]


i suspect that this job wouldn't be desirable even with a six-figure Facebook engineer salary and humane working conditions. but i bet both of those things would actually mitigate the harm it does to employees.

for example, it's pretty well-established that people who have autonomy and authority can cope with very stressful working conditions, while subordinates, even those who may have less pressure and shorter hours, end up with physical and mental health problems. so the deranged micromanagement probably makes everything much worse.

likewise, i tend to think anyone exposed to conspiracies for 40+ hours a week would start to believe them on some level, as that's just how the brain works. but surely these employees are much more susceptible than even the average person. if you're making poverty-level wages and treated like shit at work, you're in one of the prime conspiracy demographics.
posted by vogon_poet at 9:26 AM on February 25 [13 favorites]


This is Office Space, as directed by Lars von Trier.
posted by octobersurprise at 9:30 AM on February 25 [83 favorites]


it's pretty well-established that people who have autonomy and authority can cope with very stressful working conditions

There's a difference, though, between 'stressful working conditions' and what this job appears to entail, which is just a stream of audio-visual abuse. It's basically designed to be something that nearly any human being will absolutely be traumatized by. Maybe we want to build and live in a society where it's legal and acceptable to have a business where you knowingly traumatize and abuse your employees by the very nature of the work you're assigning to them, or maybe we should rethink this and ban Facebook and similar giants because there is literally no ethical way to operate them.
posted by odinsdream at 9:34 AM on February 25 [13 favorites]


Do we collectively have a short memory?

Most people who aren't Extremely Online wouldn't have read about this before; it's certainly worthy of multiple articles. I don't really see an issue with covering the same topic more than once in different publications?
posted by mosst at 9:35 AM on February 25 [7 favorites]


It's obscene that Facebook doesn't directly employ these people and it's obscene that they're so paid so little. Facebook could triple their salary and not even notice.
posted by Automocar at 9:35 AM on February 25 [13 favorites]


My friend's 7 year old told his parents that the holocaust is a hoax. He's been reading up on it on Facebook.I can't imagine the developing brains being exposed to this horseshit on a daily basis. It's infuriating.
posted by dobbs at 9:38 AM on February 25 [14 favorites]


I've worked on a number of misinfo research projects where I viewed literally thousands of "fake news" blog articles, and I did not start to believe that there's a secret lizard Illuminati, Hillary Clinton murders people, or that ISIS was infiltrating the migrant caravan.

You kinda hafta go into it wanting to believe it in the first place.

Or, Facebook moderators are hired for their ability to moderate, or supervise moderation, according to Facebook's rules, not their own independent critical thinking skills. We likely have all experienced Facebook or Twitter's boneheaded approach to "moderation", where a feminist for example is banned for saying something completely innocuous, while abusers or literal white supremacists are never ever sanctioned.

Anyone who works as a moderator at Facebook is bound to be missing a few critical thinking skills. They have to, because the contradictions of their job demand it.

This does not mean people deserve to be exposed to harmful images for 40+ hours a week, or deserve psychological harm, though.
posted by JamesBay at 9:54 AM on February 25 [9 favorites]


If every developer at Facebook had to spend two weeks a year doing this, there wouldn't be a Facebook left in three months.

Every executive at Facebook should have to spend two weeks a year doing this, the worst of the work done by their least paid and most poorly treated employees and contractors.
posted by orbit-3 at 10:04 AM on February 25 [24 favorites]


From Metafilter moderator comments regarding the political megathreads, even here there is a big emotional cost to moderation, and we have no video or sound and perhaps a more sane and much smaller userbase than Facebook.

Given how much of an issue it is here, when the employer cares about the employees -- it is frightening to imagine how much worse it is there.

It does make me wonder though-- back in the era of centralized publication, there must still have been conspiracy theorists and people trying to publish terrible things -- so who did this job then? Or was it a smaller job, because the ease of publishing at a click has lowered the bar? In short, who did this for newspapers and TV news?
posted by nat at 10:05 AM on February 25 [3 favorites]


Every executive at Facebook should have to spend two weeks a year doing this, the worst of the work done by their least paid and most poorly treated employees and contractors.

The point of The Jungle wasn't that owners of meat industry should also work the floor. It's that the whole endeavour is inhumane and needs to be stopped, systemically.
posted by odinsdream at 10:08 AM on February 25 [24 favorites]


Do we collectively have a short memory?

also the names sound a lot whiter this time (from 2014)
posted by anem0ne at 10:16 AM on February 25 [1 favorite]


and back, and back, and back ...

Here was a particularly good such article from 2014 about the outsourcing of this work.

From the article:
In Manila, I meet Denise (not her real name), a psychologist who consults for two content-moderation firms in the Philippines. “It's like PTSD,” she tells me as we sit in her office above one of the city's perpetually snarled freeways. “There is a memory trace in their mind.” Denise and her team set up extensive monitoring systems for their clients. Employees are given a battery of psychological tests to determine their mental baseline, then interviewed and counseled regularly to minimize the effect of disturbing images. But even with the best counseling, staring into the heart of human darkness exacts a toll. Workers quit because they feel desensitized by the hours of pornography they watch each day and no longer want to be intimate with their spouses. Others report a supercharged sex drive. “How would you feel watching pornography for eight hours a day, every day?” Denise says. “How long can you take that?”
posted by msbrauer at 10:19 AM on February 25 [3 favorites]


The point of The Jungle wasn't that owners of meat industry should also work the floor. It's that the whole endeavour is inhumane and needs to be stopped, systemically.

I don't think the argument is that doing two weeks a year would make things equitable, it's that those in power would find those two weeks so intolerable that they would be motivated to move toward more equitable conditions just to save themselves the discomfort.

It's still functionally speaking a fantasy because no one is going to make them do it, but it's one grounded in the idea of shocking the comfortable into pursuing justice, not satisfying a notion of justice by spreading the discomfort around slightly.
posted by cortex at 10:30 AM on February 25 [14 favorites]


Setting aside Facebook's role in accumulating these things and in dehumanizing the people who have to review them, I have to wonder -- who are the people posting these things in the first place? Who decided 'hey, I really want to share this video of a heinous, real-world murder with the world'?

Is the audience big enough that there's a profit motive to it? Are they using these horrible images to inflict abuse on other people? Are they recruitment and propaganda videos for cults, drug lords and extremist groups? All of the above?
posted by jacquilynne at 10:45 AM on February 25 [2 favorites]


In short, who did this for newspapers and TV news?

Newspapers and I assume TV stations probably had their mailroom folks screen this stuff. And that was that, right? I mean, not to say hateful or crackpot ideas weren't spread, but it was simply harder. I remember when Ron Paul was running for president in the 2000s, they brought up his racist newsletters that were published in the 80s. These were physical newsletters that you could only get through direct mail. Meaning, you had to actively somehow search out the mailing address to subscribe (or have a friend tell you about it), then mail your form, and then wait the 4-6 weeks for this newsletter to reach your mailbox. That takes quite a bit more effort than just running a Google search and putting in your email address. And Ron Paul was a Congressperson for half the 80s, not a completely unknown person either.
posted by FJT at 10:49 AM on February 25 [2 favorites]


Every executive at Facebook should have to spend two weeks a year doing this

This is a company where the COO outsources the generation of anti-semitic propaganda in order to weaken regulators. I don't think it would change their behavior in the slightest, well maybe they would come up with some new ideas to monetize the schizophrenics.
posted by benzenedream at 10:58 AM on February 25 [3 favorites]


Facebook is in no way a defendable company or platform. Last night I was in a conversation where people were hem hawing to death of how they neeeeeed FB, and it was very weak sauce. Yes some people prob do...the majority do not. Leave.
posted by agregoli at 11:05 AM on February 25 [7 favorites]


I don't think the argument is that doing two weeks a year would make things equitable, it's that those in power would find those two weeks so intolerable that they would be motivated to move toward more equitable conditions just to save themselves the discomfort.

It's still functionally speaking a fantasy because no one is going to make them do it, but it's one grounded in the idea of shocking the comfortable into pursuing justice, not satisfying a notion of justice by spreading the discomfort around slightly.


So, two thoughts about this: I actually don't think the people in charge would be motivated, they understand and accept these conditions under whatever system of the world their brain operates with. We need to deal with these actors honestly in our world, by accepting that we actually can't teach and reason our way to people who are such actors.

Second thing, is that I don't think it is a morally justified position to make someone suffer abuse in order to teach them that abuse is wrong.
posted by odinsdream at 11:19 AM on February 25 [2 favorites]


I've worked on a number of misinfo research projects where I viewed literally thousands of "fake news" blog articles, and I did not start to believe that there's a secret lizard Illuminati, Hillary Clinton murders people, or that ISIS was infiltrating the migrant caravan.

You kinda hafta go into it wanting to believe it in the first place.


I guess I sort of agree but I don't know if "wanting" is the right word. It would be probably useful for society to understand what protects you from falling for this stuff, and why that protection doesn't seem to work for people who get into QAnon, or these employees getting into Holocaust denial or Sandy Hook trutherism. We know for sure it can't be purely "education" or "critical thinking", as lots of educated people who can demonstrably think critically on some topics do still get sucked into these things. And I am naturally skeptical of "they were secretly bad people all along".

Certainly, I have to wonder whether being in this miserable workplace, under extreme emotional stress, with low wages, in an environment with lots of heavy drug use, plays a role in making these employees more susceptible to believing strange ideas.
posted by vogon_poet at 11:23 AM on February 25 [11 favorites]


I don't think it is a morally justified position to make someone suffer abuse in order to teach them that abuse is wrong.

See, I don't think it *would* be abusive if everyone had to do it for two weeks out of the year. I think it would teach them the ground reality of what comes through their networks which would give them persective on the whole company's operations; it would teach folks the necessity of good mental health benefits and better working conditions for those who do it their entire job; and it would be an unequivocal symbolic good in the sense of "everyone takes out their own trash for at least this duration".
posted by MiraK at 11:37 AM on February 25 [10 favorites]


Have not read all the comments, sorry, but just hypothetically, would one be banned from facebook for flagging things like videos of baby goats in pyjamas, feelgood insurance commercials from thailand, neat science videos, TED talks, etc. etc.
posted by If only I had a penguin... at 11:39 AM on February 25 [6 favorites]


> It would be probably useful for society to understand what protects you from falling for this stuff, and why that protection doesn't seem to work for people who get into QAnon, or these employees getting into Holocaust denial or Sandy Hook trutherism.

The most surprising thing to me over the past few years has been that people seem to be *way* less skeptical than they used to be of anonymous communication and see it as containing useful/informative signal. I can't think of a more powerful solvent for rational thought and emotional health than taking anonymous or pseudo-anonymous speech seriously or incorporating it into your view of the world (no offense, fellow pseudo-anonymous MeFites).
posted by cirgue at 11:47 AM on February 25 [2 favorites]


I don't think it is a morally justified position to make someone suffer abuse in order to teach them that abuse is wrong.

I feel you there and agree, basically; my actual position on this whole gaping chasm where ethics should be in corporate social media management is more or less "burn it to the ground and start over with the horse in front of the cart this time".

But I do understand the motivation behind the notion of exposing someone to the toxic conditions they comfortably profit from at a distance. At that point it's become a matter of putting something morally unjustifiable up against a much greater extant moral injustice, in hopes that a kind of destructive interference will reduce the total injustice all around. Questions of moral complexity or plausibility or actual efficacy notwithstanding, that's the idea I see as being at the center of it.
posted by cortex at 11:51 AM on February 25 [2 favorites]


> When I ask about the risks of contractors developing PTSD, a counselor I’ll call Logan tells me about a different psychological phenomenon: “post-traumatic growth,” an effect whereby some trauma victims emerge from the experience feeling stronger than before.

Is “post-traumatic growth” a real term of art? I can understand a counselor encouraging growth in traumatized clients, but this seems like advocating for trauma to begin with.
posted by lazugod at 11:54 AM on February 25 [3 favorites]


Jeb Lund wrote about this for his newsletter, The People Who Eat Shit for a Living. It's good; read it all.
It doesn’t help that this inevitably and unequally happens to everyone. On a basic social-justice scale, being a white man harping on online violence—next to women and people of color who you know endure orders of magnitude worse treatment—feels like whining about the lack of seating on a segregated bus filled with expectant mothers. (Who the fuck, after all, are you?) On a basic novelty scale, the ubiquity of such violence drives down your sense of urgency; the horror of this happened, through sheer numbers, slouches into shit happens and the undermining suspicion that shit is happening to everyone around you right now and, if they aren’t rending their garments about it or even gently asking for help, maybe neither should you.
...
Facebook’s mission—such as it exists in a scope broader than an emotional version of fire, which is to consume all that can be consumed—is to draw the functional aspects of the internet completely within itself, until you never need to cross a border to experience anything in a realm it cannot monetize. To them, it’s a virtuous cycle: The more of the internet it can “safely” convey to you, the less you leave; the more you’re there, the more incentive your friends and family have to stay there too; the more you see them, the less apt you are to leave. The future is the AOL startup page, and you are welcome.
...
Call it the Shit Happens paradigm, if you want, but it is what Facebook’s settled on. All of the good parts of the internet shall be its domain, and all the bad parts are inevitable and insoluble, and the only way to square those two conditions is to make the problem not yours and not the shareholders’ but someone else’s, elsewhere. Just to be on the safe side, they’ll sign NDAs, so you’ll never run the risk of hearing them at a party sharing even more of the pain that you’re already in about a platform you already incipiently loathe, all on your behalf.

Shit happens, and it will continue to happen, and someone whose face they would prefer that you never see will just have to eat it. For every murder you don’t witness, they’ll witness three. If they can forget what the first one looked like, maybe, with time, they can learn to forget all of the rest.
Except I don't think this is just Facebook; it's what the internet is now.
posted by zachlipton at 12:22 PM on February 25 [15 favorites]


... I have to wonder whether being in this miserable workplace, under extreme emotional stress, with low wages, in an environment with lots of heavy drug use, plays a role in making these employees more susceptible to believing strange ideas.
Put this way, how far away are we from describing a significant part of cult indoctrination?
posted by milnews.ca at 12:40 PM on February 25 [4 favorites]


From a Facebook discussion about the link (Quoted material not from me):

"So, a friend of mine used to have the job of watching Jihadi beheading videos to gather intel for the US intelligence community.

If they want to comment on this, they're welcome to...but there are definite procedures and protocols for keeping your workers *sane* doing this kind of job."

So I asked what the procedures were.

"Usually "no more than three days on, then rotate you to desk work for two weeks, and counselor intervention when you come out of the horrorshow, with further counseling as requested."

So, nothing clever or trendy, just accepting the expense of not overloading people.
posted by Nancy Lebovitz at 12:46 PM on February 25 [34 favorites]


I have to wonder whether being in this miserable workplace, under extreme emotional stress, with low wages, in an environment with lots of heavy drug use, plays a role in making these employees more susceptible to believing strange ideas.
"A former moderator named Sara said that the secrecy around their work, coupled with the difficulty of the job, forged strong bonds between employees. “You get really close to your coworkers really quickly,” she says. 'If you’re not allowed to talk to your friends or family about your job, that’s going to create some distance. You might feel closer to these people. It feels like an emotional connection, when in reality you’re just trauma bonding.'"
It's hard to think of a better way to abuse someone psychologically than to put them in environment of constant stress, an omniscient threat of graphic violence, drug use, and forced trauma bonding. That's a recipe for PTSD and for coping mechanisms which might make a hunt for conspiratorial patterns more appealing than the utter nihilism of the day-to-day work.
posted by octobersurprise at 12:49 PM on February 25 [9 favorites]


mathowie weighs in: Content moderation has no easy answers.
posted by 1970s Antihero at 1:50 PM on February 25 [6 favorites]


What I'm saying is that requiring anyone to watch this content, since we know that it's severely traumatic to people, is like when we asked people to work with radium and stuff with no protection, knowing it would kill them. This is not ethically tenable.

I am well aware that this means many people have jobs that meet this definition, and I feel similarly about them. I do feel this particular scenario results from people discrediting mental health as somehow less serious than physical safety. We have OSHA requirements that you aren't allowed to perform various activities even if an employer tries to make you. We have jobs that don't obey these regulations and that's wrong.

We need a similar understanding of activities that are simply not mentally safe, under any circumstances. Many, many systems that humans have built recently with the introduction of mass instant global communication run afoul of what some day I hope will be OSHA-level agreements. Our society and culture have grown to a point where this is desperately needed to stem the tide of a massive wave of mental abuse that's going to predominately fall on the shoulders of the powerless in our society.
posted by odinsdream at 1:57 PM on February 25 [13 favorites]


From Metafilter moderator comments regarding the political megathreads, even here there is a big emotional cost to moderation, and we have no video or sound and perhaps a more sane and much smaller userbase than Facebook.

Were you around in the goatse era when the image tag was still allowed?

*shudder*
posted by chillmost at 1:57 PM on February 25 [3 favorites]


Some clarification of that three days on, two weeks off workplace.

"My friend has the following clarifications:
A: He wasn't doing intel gathering.
B: He was doing development work on software to filter/audit/prioritize video.
C: That project existed because having humans review it resulted in large amounts of generalized anxiety disorder and PTSD as outlined in the article.
D: My memory of "three days on, two weeks off" was what he did doing software dev; the people doing the analysis jobs that he was trying to automate out of existence did one week on, two weeks off, and most burned out in under a year.
E: He is *furious* about what Facebook is doing to these contractors.
Me, I think the next time Facebook faces a monetary fine, every C-suite executive should do 6 months as a moderator at one of the Cognizant centers instead. Soaking them for a half a billion when they're raking in 7 billion in profits per quarter isn't sufficient disincentive to change behaviors."
posted by Nancy Lebovitz at 1:58 PM on February 25 [17 favorites]


It does make me wonder though-- back in the era of centralized publication, there must still have been conspiracy theorists and people trying to publish terrible things -- so who did this job then? Or was it a smaller job, because the ease of publishing at a click has lowered the bar? In short, who did this for newspapers and TV news?

I worked for a publication during the print era and we published a piece that had an intercultural family on the cover and we got about 200-300 racist letters about it, in the mail.

We didn't have to worry that these letters were going live out to anyone, so the pace of dealing with it was fairly relaxed and we basically filed them into 'for' and 'against' and...it made me feel sad, for sure, for the country and humanity, but it was not at all like watching racist videos or comments in real time out where they are there for everyone to see. And it certainly wasn't like porn or murder videos; maybe one of those came in at newspapers once in a while but it wasn't hours and hours and hours of it all day long 24/7.

There was not really the concept that anyone would really see it, you see, because everyone knew that the networks were not going to air that stuff, certainly not just for someone else's agenda. Maybe when it was real news. Not only was it waaaaaaaay harder to MAKE video and edit it but there was like zero chance of it being consumed.

Also I may have been naive, in fact, history shows that I would be, but back when we were getting the endless racist letters, I really had no idea that so many people were so deliberately, actively racist and callous and -brazen. I don't think we really thought that people /wanted/ to see such horrible stuff.

So I don't think it's actually just about the ease of publishing but about the audience.
posted by warriorqueen at 3:09 PM on February 25 [7 favorites]


«clockwork_orange-ludovico_technique.gif»
posted by scruss at 3:59 PM on February 25 [1 favorite]


I worked briefly at a company where we were building a forum site for kids in their mid-teens. We had two women there moderating content and I remember one of them saying that the child porn was the worst. I can't even imagine.
posted by bendy at 5:24 PM on February 25


Back in the pre-Internet days there were all sorts of weird conspiracy theories. UFOs are a good example. Which morphed into the entire MIB subculture of the 1970s, which was fueled in part by paranoia about the CIA. By the late 70s you had people convinced the US government was using alien technology out at "Hanger 18." A decade later, there were books and 'zines being written about aliens harvesting humans in underground farms.

Budd Hopkins and Whitley Strieber are two of the more famous hoaxers. That "missing time" stuff was really compelling, but anyone with a bit of common sense and a desire to check the facts would have realized that "alien abductions" were hocus pocus bullshit.
posted by JamesBay at 5:24 PM on February 25


In short, who did this for newspapers and TV news?

There were mailroom people, but for the most part, they were not traumatized.

1) It takes substantially more effort to send a letter on paper than it does to create a Facebook post. You need paper, a writing utensil (pen or typewriter), an envelope, a stamp... and if you want to send pictures or worse, video, you need even more resources. And you need to be coherent enough to put all those things together in the proper format.
1b) Every letter you send costs you money. People will rant at length for free; charge them the price of a gumball for it, and they start getting selective.

2) Some of the worst wingnuts are identifiable without actually reading the content. If the letter is written in purple crayon in all caps, you don't need to get farther than "DEAR SURZ I HAVE THA REAL TROOTH ABOUT" before you throw it in the circular file. In fact, even if it's typewritten, if it's spelled like that, the mailroom staffer isn't likely to read past the first line.

3) Pictures were expensive to create. You might have photos of a grisly murder, but you didn't have 500 different ones, and you and all your buddies didn't each have a dozen that you all sent in over the course of a few days.

4) Letters from known kooks (of various sorts) got thrown out without reading--and of course, the kook has no way of knowing that.

5) Facebook moderators can't actually moderate. They don't have the ability to say, "we're just blocking everything this person produces; it never needs to go to us or anyone else on the platform again." They are not allow to block the accounts of anyone who egregiously breaks the TOS rules; I don't know what they do for penalties, but apparently it's not "oh hey if you post a literal murder on the site, you lose your posting privileges forever."

6) Moderators also aren't allowed to call the police to report crimes that they're being given evidence for, Newspapers and TV stations would do that. If someone sent in a photo of a murder, it wasn't a question of, "hm, is this newsworthy enough to publish?" it was also, "hm, do we call the police about this?" This cut down on people sending them evidence of their own crimes. A mail room employee who was sickened at the sight of photos of a brutal rape could do something about it, could call the police, try to ID the people involved, and so on. Part of the FB moderators' problem is that they can't take any action that would result in less horrific content existing in the future.
posted by ErisLordFreedom at 10:44 PM on February 25 [27 favorites]


We know for sure it can't be purely "education" or "critical thinking", as lots of educated people who can demonstrably think critically on some topics do still get sucked into these things.

On a side note, lots of educationally credentialed people only get educated to think critically within one specific domain or subdomain, and not as a general skill applicable to all areas of reasoning. In fact, due to siloing of academic disciplines, they may have developed prejudices about other disciplines that specifically inhibit critical thinking in those other areas.

posted by eviemath at 4:29 AM on February 26 [4 favorites]


For me personally Facebook moderation would be quite easy and would last < a day, because I can see no reason whatsoever not to just delete everything I could get my grubby little hands on.
posted by aspersioncast at 4:46 AM on February 26 [1 favorite]


6) Moderators also aren't allowed to call the police to report crimes that they're being given evidence for, Newspapers and TV stations would do that. If someone sent in a photo of a murder, it wasn't a question of, "hm, is this newsworthy enough to publish?" it was also, "hm, do we call the police about this?" This cut down on people sending them evidence of their own crimes. A mail room employee who was sickened at the sight of photos of a brutal rape could do something about it, could call the police, try to ID the people involved, and so on. Part of the FB moderators' problem is that they can't take any action that would result in less horrific content existing in the future.

That's really true? That's....why? So conceivably I could murder someone, post a video and the video implicating me would be deleted by helpful moderators, shielding me from consequences? Is it just that they consider it too complicated or too much work to have moderators making that call?

I'm very much on the "Facebook obviously can't be run without being terrible and destructive, down with Facebook" side right now.
posted by Frowner at 6:04 AM on February 26 [4 favorites]


It feels like this sort of job is going to be like lead in the water ten years from now.
posted by lucidium at 8:08 AM on February 26 [1 favorite]


Tell me about it.
posted by clavdivs at 9:58 AM on February 26 [1 favorite]


That's really true? That's....why? So conceivably I could murder someone, post a video and the video implicating me would be deleted by helpful moderators

I'm speculating, based on the article. It doesn't sound like they have any option to contact law enforcement based on what they're reviewing - they're just deciding whether it's a TOS violation, and entirely ignoring the "other offensive content" part of the TOS while they do that.

Of course, many of the pictures and videos won't be recent, and won't be related to the person posting them, but... still, ugh. Nothing in that article implies they can do anything other than vote Y/N on keep/delete.

However, the deleted videos are likely still in their servers - police etc. could still have access on request. But there's no indication that the review team would ever contact law enforcement.
posted by ErisLordFreedom at 11:20 AM on February 26 [1 favorite]


NYMag's Brian Feldman, "Facebook Can't Solve Its Problems by Throwing Bodies at Them":
Facebook is a highly centralized system that can’t possibly satisfy everyone, and Facebook operates at a scale where even a tiny portion of dissatisfied users amounts to tens millions of people. Facebook’s response to the Verge’s report today repeatedly emphasizes the sheer scale of the issue. The only solution for Facebook is to scale down, or restructure itself in a more balkanized way. It’s working on it, but the News Feed is a cash cow, and so long as the News Feed exists, it will have a substantial moderation problem. Unfortunately, it’s the lowest-level workers, like the contractors in Phoenix, who are bearing most of the strain.
Also, there are links within that article to Twitter threads by Antonio Garcia Martinez, former FB product manager, and Alex Stamos, former FB head of security. They're... not the greatest. The first guy is pretty much like "First, you journalists whine that we let all of this abusive and dangerous material onto our site and now you guys are whining that the people we hired to clean this up are treated like garbage. Hypocrites, much? Like, make up your mind, journalism." The second guy is a bit more nuanced but still pretty defensive. His take leans a little more towards "Pretty easy for you journalists to criticize when you don't have to actually clean up our mess."
posted by mhum at 12:10 PM on February 26 [1 favorite]


The accuracy focus suggests the contractors are fuelling a dataset for algorithms to learn from. It’s telling that the accuracy required is impossible for humans to deliver, whilst monstrously cruel to extract.
posted by eyeofthetiger at 12:45 PM on February 26


I mean, they probably are training an AI, but the accuracy requirement is also pretty in line with the way call center contracts are written generally. They're always tightly measured on specific performance indicators, even if those performance indicators are stupid and sometimes self-defeating or conflicting.
posted by jacquilynne at 2:40 PM on February 26 [2 favorites]


an interesting contrast: "The conspiracy theory stuff is like some weird Upside Down version of NYT's recent piece on Chinese content censorship"
posted by kliuless at 6:11 AM on February 27


« Older "Well, that ruined everything."   |   Gimme the honky tonk blues Newer »


This thread has been archived and is closed to new comments