"Too toxic to ignore"
March 26, 2021 7:15 AM Subscribe
Thierry Henry, one of the greatest players in football history, has had enough of racism and bullying on social media.
Henry's statement :-
"From tomorrow morning I will be removing myself from social media until the people in power are able to regulate their platforms with the same vigour and ferocity that they currently do when you infringe copyright. The sheer volume of racism, bullying and resulting mental torture to individuals is too toxic to ignore. There HAS to be some accountability. It is far too easy to create an account, use it to bully and harass without consequence and still remain anonymous. Until this changes, I will be disabling my accounts across all social platforms. I'm hoping this happens soon."
Henry's statement :-
"From tomorrow morning I will be removing myself from social media until the people in power are able to regulate their platforms with the same vigour and ferocity that they currently do when you infringe copyright. The sheer volume of racism, bullying and resulting mental torture to individuals is too toxic to ignore. There HAS to be some accountability. It is far too easy to create an account, use it to bully and harass without consequence and still remain anonymous. Until this changes, I will be disabling my accounts across all social platforms. I'm hoping this happens soon."
I hope it helps. But it's probably going to take a lot more high profile people to do the same.
posted by Glinn at 8:25 AM on March 26, 2021 [2 favorites]
posted by Glinn at 8:25 AM on March 26, 2021 [2 favorites]
What can be done? ID checks? Parler did this and it went badly wrong.
Use AI to better detect hate speech? More moderation teams?
I can't help but feel that this is a losing battle for people with millions of followers. The people Henry talks about have shown they will jump through a lot of hoops to be dicks online. More hoops than someone who wants to click like on a tweet.
Shut it all down!
posted by benoliver999 at 8:26 AM on March 26, 2021
Use AI to better detect hate speech? More moderation teams?
I can't help but feel that this is a losing battle for people with millions of followers. The people Henry talks about have shown they will jump through a lot of hoops to be dicks online. More hoops than someone who wants to click like on a tweet.
Shut it all down!
posted by benoliver999 at 8:26 AM on March 26, 2021
What can be done?The first step which imo would reduce 90% of all of the garbage on the site is to, say, require someone to pay $5 to sign up, and attach a credit card token to their account (or any other sockpuppet accounts they would want to create). One account gets flagged for being an asshole? Congratulations, all the accounts with that credit card token get shadowbanned and that credit card can't be used to sign up again. If your account is in good standing after X days, you get your $5 back. This obviously doesn't stop the completely determined attackers -- not a lot does -- and there's a lot wrong with this idea, specifically around throwaway card numbers or stolen cards, but it will slow down the casual racist asshole.
posted by mark242 at 8:37 AM on March 26, 2021 [22 favorites]
Anything that involves money/a credit card creates a really high barrier to entry for anyone unbanked or in a state of financial precarity which includes a LOT of otherwise-marginalized people as well. Real names are also dangerous for a lot of marginalized people and people in abusive domestic situations (and fraught -- my "real" name is not my legal name and I am not willing to go on social media with my deadname). Maybe these platforms could start by getting rid of the Nazis? Twitter has plenty of accounts whose contents they don't show in Germany because it's illegal but they still show up in American feeds. Maybe if you got rid of those it would help overall, as well as some of the big names. Plenty of prominent fascists are verified! If you get rid of Nazis/big name racists/transphobes/homophobes that would probably help a lot! Yes, this is a difficult problem to solve, but there are some pretty straightforward steps that I think could be taken without too much effort, the major platforms just don't want to do it because they're okay with this shit and in some cases are hampered by a desire to look "unbiased", as if wanting to survive and wanting to threaten the survival of others were equally valid positions.
posted by an octopus IRL at 8:49 AM on March 26, 2021 [18 favorites]
posted by an octopus IRL at 8:49 AM on March 26, 2021 [18 favorites]
Comparing and tying it to copyright enforcement seems like a good strategy.
posted by hypnogogue at 8:56 AM on March 26, 2021 [9 favorites]
posted by hypnogogue at 8:56 AM on March 26, 2021 [9 favorites]
Do it like with California and emissions standards, let the Germans set the de facto world standard for not being a Nazi online.
posted by Meatbomb at 8:58 AM on March 26, 2021 [10 favorites]
posted by Meatbomb at 8:58 AM on March 26, 2021 [10 favorites]
Seriously, social media is actively dangerous to your safety, life, and career. Why is anyone still on it? You can get doxxed, stalked, lose jobs from your tweets from ten years ago, literally anything.
People tell me I should start my own business and things like that, and then I think "But I'd have to be on social media, promoting myself," and then I'd say something inadvertently bad when I didn't even mean to, turn myself into a milkshake duck, and then ruin my life forever. Why even try?
posted by jenfullmoon at 9:26 AM on March 26, 2021 [5 favorites]
People tell me I should start my own business and things like that, and then I think "But I'd have to be on social media, promoting myself," and then I'd say something inadvertently bad when I didn't even mean to, turn myself into a milkshake duck, and then ruin my life forever. Why even try?
posted by jenfullmoon at 9:26 AM on March 26, 2021 [5 favorites]
What can be done?
I know this is the typical "technical solution to a social problem" rant, but I believe it is better than doing nothing, it could be a good start.
As a final project after a 3 month training program for paid interns at my work we had them build a simple machine learning project over the course of a few weeks. A group of 9 software engineers just out of school with no previous knowledge of the subject were able to build a sentiment analysis model that identifies homophobic and misogynistic comments in a corpus of millions of online forum comments, with very good accuracy (you'd have to ask the data scientist for the actual metrics), and smart enough to flag the ambiguous cases for human review. They are working on a second stage, which will take into account a user's history to try to determine if the person is consistently an asshole or if it is a one-off.
This is not a simple word blacklist, the sentiment analysis part was very interesting, it allows the models to tell apart things like quotes or innocuous use of blacklisted words from attacks very well. It is also decent, but not very good, at identifying dogwhistles.
There is another ongoing project built by a team of two people that does something very similar with twitter en español. There is valuable IP in this one and I am drone that likes their salary, all I can tell you is that it is very damn good at flagging assholes and trolls in twitter, with very very few false positives.
What I am getting at is that if 9 junior engineers, or a team of 2 data scientists can build this with very limited resources, social media company could do amazing things with their resources if they cared about the issue at all.
posted by Dr. Curare at 9:51 AM on March 26, 2021 [18 favorites]
I know this is the typical "technical solution to a social problem" rant, but I believe it is better than doing nothing, it could be a good start.
As a final project after a 3 month training program for paid interns at my work we had them build a simple machine learning project over the course of a few weeks. A group of 9 software engineers just out of school with no previous knowledge of the subject were able to build a sentiment analysis model that identifies homophobic and misogynistic comments in a corpus of millions of online forum comments, with very good accuracy (you'd have to ask the data scientist for the actual metrics), and smart enough to flag the ambiguous cases for human review. They are working on a second stage, which will take into account a user's history to try to determine if the person is consistently an asshole or if it is a one-off.
This is not a simple word blacklist, the sentiment analysis part was very interesting, it allows the models to tell apart things like quotes or innocuous use of blacklisted words from attacks very well. It is also decent, but not very good, at identifying dogwhistles.
There is another ongoing project built by a team of two people that does something very similar with twitter en español. There is valuable IP in this one and I am drone that likes their salary, all I can tell you is that it is very damn good at flagging assholes and trolls in twitter, with very very few false positives.
What I am getting at is that if 9 junior engineers, or a team of 2 data scientists can build this with very limited resources, social media company could do amazing things with their resources if they cared about the issue at all.
posted by Dr. Curare at 9:51 AM on March 26, 2021 [18 favorites]
What can be done?
How many liberal arts majors who used to work at small-town newspapers have lost their jobs because of social media? It was basically those people who were responsible for keeping the racists and assholes from taking over the discourse. (Except when they were the racists and assholes, but I digress.) I'm guessing a whole bunch of people like them will have to be hired again if they mean to take this seriously.
Divide the profits of Facebook, Twitter and Google by a reasonable salary and see how many editors you could hire.
posted by clawsoon at 9:53 AM on March 26, 2021 [8 favorites]
How many liberal arts majors who used to work at small-town newspapers have lost their jobs because of social media? It was basically those people who were responsible for keeping the racists and assholes from taking over the discourse. (Except when they were the racists and assholes, but I digress.) I'm guessing a whole bunch of people like them will have to be hired again if they mean to take this seriously.
Divide the profits of Facebook, Twitter and Google by a reasonable salary and see how many editors you could hire.
posted by clawsoon at 9:53 AM on March 26, 2021 [8 favorites]
Why is anyone still on it?
Let me give you a third world perspective, which also intersects with why requiring credit cards or official IDs will disproportionally affect the wrong people.
Cellphone companies in Mexico (and in many countries where the next billion users live) include free data for social media. Prepaid phone cards advertise free twitter, facebook, instagram and whatsapp. This has made Social Media == Internet for tens of millions of people in Mexico. If you are a business, an artist, an activist... the way to reach your audience is through social media.
I can only reach the gas company through facebook messenger, I make doctor appointments through whatsapp, my biciycle mechanic communicates via instagram DMs, official COVID updates from the government are posted through facebook, you get the idea.
posted by Dr. Curare at 9:56 AM on March 26, 2021 [21 favorites]
Let me give you a third world perspective, which also intersects with why requiring credit cards or official IDs will disproportionally affect the wrong people.
Cellphone companies in Mexico (and in many countries where the next billion users live) include free data for social media. Prepaid phone cards advertise free twitter, facebook, instagram and whatsapp. This has made Social Media == Internet for tens of millions of people in Mexico. If you are a business, an artist, an activist... the way to reach your audience is through social media.
I can only reach the gas company through facebook messenger, I make doctor appointments through whatsapp, my biciycle mechanic communicates via instagram DMs, official COVID updates from the government are posted through facebook, you get the idea.
posted by Dr. Curare at 9:56 AM on March 26, 2021 [21 favorites]
I oscillate back and forth on this a log, between:
- the belief that lots (though not all) of the worst would melt away if relative anonymity wasn't so easy online and the certainty that without it, many marginalized people would be unable to participate at all.
- the memory of the Internet as it was prior to the dominance of social media - and it was not without it's problems - and the knowledge that for most, it is as Dr. Curare describes above: social media = Internet.
- I can find plenty of utility online without social media, but I still have a Twitter account and Instagram is one of the last overlaps between us as parents and the online world of our children. But there's plenty that I miss (and more importantly, that I can afford to miss). This is where a lot people are, and walking away completely just isn't an option at the moment.
As far as I can see, the only real changes will come with one or more of the following:
1. Some sort of wholesale doubling down on content moderation by the largest social media companies, perhaps brought on by legislation but probably more likely by a drop in profits. This, I fear, will take awhile. There are too many captive eyeballs and it would take a hell of an exodus to move the needle.
2. A rise in social-breadth-by-choice of federated services (I'm thinking mostly of Mastodon, but there are probably others). This might go faster than the content companies, given how quickly The Cool New Things appear.
3. A rediscovery of the non-social Internet, maybe by retro-connoisseurs, that things like email, RSS, XMPP, Micro.blog, and other things still work plenty well and give users way more control what they're consuming and how. This may be a generational change if it's to happen at all, so I never stop talking about it. Sometimes I feel like the window on this is slowly closing.
posted by jquinby at 10:34 AM on March 26, 2021 [1 favorite]
- the belief that lots (though not all) of the worst would melt away if relative anonymity wasn't so easy online and the certainty that without it, many marginalized people would be unable to participate at all.
- the memory of the Internet as it was prior to the dominance of social media - and it was not without it's problems - and the knowledge that for most, it is as Dr. Curare describes above: social media = Internet.
- I can find plenty of utility online without social media, but I still have a Twitter account and Instagram is one of the last overlaps between us as parents and the online world of our children. But there's plenty that I miss (and more importantly, that I can afford to miss). This is where a lot people are, and walking away completely just isn't an option at the moment.
As far as I can see, the only real changes will come with one or more of the following:
1. Some sort of wholesale doubling down on content moderation by the largest social media companies, perhaps brought on by legislation but probably more likely by a drop in profits. This, I fear, will take awhile. There are too many captive eyeballs and it would take a hell of an exodus to move the needle.
2. A rise in social-breadth-by-choice of federated services (I'm thinking mostly of Mastodon, but there are probably others). This might go faster than the content companies, given how quickly The Cool New Things appear.
3. A rediscovery of the non-social Internet, maybe by retro-connoisseurs, that things like email, RSS, XMPP, Micro.blog, and other things still work plenty well and give users way more control what they're consuming and how. This may be a generational change if it's to happen at all, so I never stop talking about it. Sometimes I feel like the window on this is slowly closing.
posted by jquinby at 10:34 AM on March 26, 2021 [1 favorite]
5 dollars to be allowed to post on a site? It had better be a very good site.
posted by dances_with_sneetches at 10:43 AM on March 26, 2021 [8 favorites]
posted by dances_with_sneetches at 10:43 AM on March 26, 2021 [8 favorites]
5 dollars to post is a real bargain. In town, it's 20 dollars.
posted by sainttoad at 11:05 AM on March 26, 2021 [5 favorites]
posted by sainttoad at 11:05 AM on March 26, 2021 [5 favorites]
> 3. A rediscovery of the non-social Internet, maybe by retro-connoisseurs, that things like email, RSS, XMPP, Micro.blog, and other things still work plenty well and give users way more control what they're consuming and how. This may be a generational change if it's to happen at all, so I never stop talking about it. Sometimes I feel like the window on this is slowly closing.
this is something I often find myself thinking about (and, less often, tinkering with a toy side-project to try to address). I think the window for this is not slowly closing but is yet to really open. as more business and communication moves online, and as more people are born into a world where that's all they know, the quicker "things you don't say to someone's face" and "things you don't say to someone over the internet" will converge. it's then that I feel we'll see a big demand for fewer walled gardens and more personal control over one's own internet (i.e., one's entire public) presence.
at least, that's how I hope things go. because that world sure as hell isn't gonna be built by companies valuing engagement and ad revenue over all else.
posted by Old Kentucky Shark at 11:23 AM on March 26, 2021 [1 favorite]
this is something I often find myself thinking about (and, less often, tinkering with a toy side-project to try to address). I think the window for this is not slowly closing but is yet to really open. as more business and communication moves online, and as more people are born into a world where that's all they know, the quicker "things you don't say to someone's face" and "things you don't say to someone over the internet" will converge. it's then that I feel we'll see a big demand for fewer walled gardens and more personal control over one's own internet (i.e., one's entire public) presence.
at least, that's how I hope things go. because that world sure as hell isn't gonna be built by companies valuing engagement and ad revenue over all else.
posted by Old Kentucky Shark at 11:23 AM on March 26, 2021 [1 favorite]
Comparing and tying it to copyright enforcement seems like a good strategy.
Copyright claims are routinely weaponized by bad-faith actors.
posted by praemunire at 11:39 AM on March 26, 2021 [2 favorites]
Copyright claims are routinely weaponized by bad-faith actors.
posted by praemunire at 11:39 AM on March 26, 2021 [2 favorites]
benoliver999: "Use AI to better detect hate speech? "
Yes. The tech is there. They just don't want to alienate their core profit-generating radicals and edgelords.
posted by signal at 12:19 PM on March 26, 2021 [1 favorite]
Yes. The tech is there. They just don't want to alienate their core profit-generating radicals and edgelords.
posted by signal at 12:19 PM on March 26, 2021 [1 favorite]
Why is anyone still on it?
For me, it's that staying in touch with my friends in my birth country is impracticable/expensive enough that I would lose touch with a significant portion of them without it.
posted by joannemerriam at 1:26 PM on March 26, 2021 [5 favorites]
For me, it's that staying in touch with my friends in my birth country is impracticable/expensive enough that I would lose touch with a significant portion of them without it.
posted by joannemerriam at 1:26 PM on March 26, 2021 [5 favorites]
Yes. The tech is there. They just don't want to alienate their core profit-generating radicals and edgelords.
This is the heart of the issue - we do not have a lack of capability, but rather one of will. Part of it is that hate is profitable, sadly - when hatemongers like Stephen Crowder bring in views and money, the platforms will bend over backwards to oblige and protect them. But another part of it is cultural - we have for a long time pushed a culture where victims are obligated to allow their abusers freedom to continue their abuse out of support for a nebulous "greater good". It's worth remembering that, for example, Twitter employees literally argued that Twitter was obliged to allow beheading videos posted on the service by ISIS to stand out of and obligation to "free speech".
The good news is that there has been pushback on the idea that tolerating abuse is necessary for "the greater good" in recent times. But sadly, this is something that has been taught to us for decades, so it will take time and effort to unteach it.
posted by NoxAeternum at 1:48 PM on March 26, 2021 [5 favorites]
This is the heart of the issue - we do not have a lack of capability, but rather one of will. Part of it is that hate is profitable, sadly - when hatemongers like Stephen Crowder bring in views and money, the platforms will bend over backwards to oblige and protect them. But another part of it is cultural - we have for a long time pushed a culture where victims are obligated to allow their abusers freedom to continue their abuse out of support for a nebulous "greater good". It's worth remembering that, for example, Twitter employees literally argued that Twitter was obliged to allow beheading videos posted on the service by ISIS to stand out of and obligation to "free speech".
The good news is that there has been pushback on the idea that tolerating abuse is necessary for "the greater good" in recent times. But sadly, this is something that has been taught to us for decades, so it will take time and effort to unteach it.
posted by NoxAeternum at 1:48 PM on March 26, 2021 [5 favorites]
If your business model says that you can't hire enough moderators for the content generated, then you don't have a viable business model.
posted by idb at 3:07 PM on March 26, 2021 [6 favorites]
posted by idb at 3:07 PM on March 26, 2021 [6 favorites]
They could start by enforcing their own T&Cs but they're not even doing that. They allow things to stay up which are very clearly in violation. Like someone mentioned above with the Germany example, these platforms are perfectly capable of doing a much, much better job, they just don't want to.
posted by triggerfinger at 5:32 PM on March 26, 2021 [4 favorites]
posted by triggerfinger at 5:32 PM on March 26, 2021 [4 favorites]
As a cautionary example, there's the sentiment analysis tool that some big companies use that was recently shown to rate drag queens' social media postings as more toxic than David Duke and Richard Spencer.
To those saying that this problem should be tackled like copyright hits are now, I say "you know how I know you aren't a SGM who's had their entire account flagged as "controversial" or "sensitive"?
posted by tigrrrlily at 7:14 AM on March 27, 2021 [1 favorite]
To those saying that this problem should be tackled like copyright hits are now, I say "you know how I know you aren't a SGM who's had their entire account flagged as "controversial" or "sensitive"?
posted by tigrrrlily at 7:14 AM on March 27, 2021 [1 favorite]
I think a lot of good could come of identifying and removing super-spreaders of disinformation and hate. A relatively small number of bad actors cause a lot of the toxicity.
But because those people are the bread and butter of big social, and they love them their profits, we’ll need some combination of popular outrage/abandonment and state pressure to make them take steps like that and others already mentioned.
posted by zenzenobia at 1:39 PM on March 27, 2021 [1 favorite]
But because those people are the bread and butter of big social, and they love them their profits, we’ll need some combination of popular outrage/abandonment and state pressure to make them take steps like that and others already mentioned.
posted by zenzenobia at 1:39 PM on March 27, 2021 [1 favorite]
This obviously doesn't stop the completely determined attackers -- not a lot does -- and there's a lot wrong with this idea, specifically around throwaway card numbers or stolen cards
Stolen cards and throwaway numbers? The test for that is the refund of the $5 after X days. If the refund fails to go back to the card they signed up with, that's a ban, too. Card changes due to lost/stolen/etc. in the interim is an issue, but that's a detail that is either surmountable or the idea's Achilles' heel that makes it unworkable after all. But! Ideas are still possible.
posted by rhizome at 2:29 PM on March 27, 2021 [1 favorite]
Stolen cards and throwaway numbers? The test for that is the refund of the $5 after X days. If the refund fails to go back to the card they signed up with, that's a ban, too. Card changes due to lost/stolen/etc. in the interim is an issue, but that's a detail that is either surmountable or the idea's Achilles' heel that makes it unworkable after all. But! Ideas are still possible.
posted by rhizome at 2:29 PM on March 27, 2021 [1 favorite]
I'm sceptical that an automated filter can beat determined evilness without itself being an AGI (SF plot idea for how the AI takeoff happens!) and without filtering discussions about racism (can it handle the use/mention distinction?). Maybe the grossest statements are more hurtful than the circumlocutions necessary to get past the filter, in which case it would have value.
I don't think federation necessarily helps either: the federated servers still have to invest in human or AI moderation, and now they're smaller and run by hobbyists. The reason Mastodon is nicer than Twitter is probably just that it's smaller.
The social media stuff I've been most interested in reading about lately has the messages traversing the social graph, so unless you've got friend-of-friends path to someone, you can't bother them. I'm thinking of things like Secure Scuttlebutt. This would mean you'd lose the ability for anyone to reply to anyone and the serendipity that results, but "anyone can reply to anyone" doesn't work when so many people are awful.
posted by pw201 at 2:48 AM on March 30, 2021 [2 favorites]
I don't think federation necessarily helps either: the federated servers still have to invest in human or AI moderation, and now they're smaller and run by hobbyists. The reason Mastodon is nicer than Twitter is probably just that it's smaller.
The social media stuff I've been most interested in reading about lately has the messages traversing the social graph, so unless you've got friend-of-friends path to someone, you can't bother them. I'm thinking of things like Secure Scuttlebutt. This would mean you'd lose the ability for anyone to reply to anyone and the serendipity that results, but "anyone can reply to anyone" doesn't work when so many people are awful.
posted by pw201 at 2:48 AM on March 30, 2021 [2 favorites]
« Older Fewer Octop(i/uses/odes) than I would have... | suburban retrofitting Newer »
This thread has been archived and is closed to new comments
posted by Pacheco at 7:28 AM on March 26, 2021 [9 favorites]