The Digital Berlin Wall
October 8, 2020 6:26 AM Subscribe
How Germany built a prototype for online censorship In 2017 Germany adopted the ‘Network Enforcement Act’ or NetzDG. It obliges social media platforms with a minimum of 2 million users to remove illegal content – including hate speech and religious offense – within 24 hours, or risk steep fines of up to 50 million euros. [...]
In May 2019, Justitia issued a report which documented that at least 13 countries (plus the EU) had adopted or proposed models similar to the NetzDG matrix. [...] Worryingly few of these countries have in place the basic rule of law and free speech protections built into the German precedent [...]
Turkey's New Internet Law Is the Worst Version of Germany's NetzDG Yet (EFF, July 2020)
German hate speech crackdown sparks censorship fears (Politico, February 2020)
Germany’s Online Crackdowns Inspire the World’s Dictators (Foreign Policy, November 2019)
Germany: Flawed Social Media Law (Human Rights Watch, February 2018)
Turkey's New Internet Law Is the Worst Version of Germany's NetzDG Yet (EFF, July 2020)
German hate speech crackdown sparks censorship fears (Politico, February 2020)
Germany’s Online Crackdowns Inspire the World’s Dictators (Foreign Policy, November 2019)
Germany: Flawed Social Media Law (Human Rights Watch, February 2018)
I'm not sure why an internet service wouldn't need to comply with the laws of a jurisdiction it is operating in. The service gets to decide if the costs of compliance are worth operating there. If Comedy Central knows I'm in Canada so won't serve me any videos then Facebook can similarly block people from a country from accessing it if Facebook doesn't want to comply with that country's laws.
posted by any portmanteau in a storm at 4:17 PM on October 8, 2020 [4 favorites]
posted by any portmanteau in a storm at 4:17 PM on October 8, 2020 [4 favorites]
never quite worked out a policy opinion...not sure why an internet service wouldn't need to comply with the laws of a jurisdiction it is operating in?
I think this is where I'm at. Is it any more complicated than that? Erdogan or [insert strong man] will always find the tools to jackboot. It's not capital C censorship because it's a private commercial space.
The real problem is that we've been made to feel like FBook and Google.stuff are the commons because they're 'free', but that's not what's going on.
posted by Reasonably Everything Happens at 7:33 PM on October 8, 2020 [1 favorite]
I think this is where I'm at. Is it any more complicated than that? Erdogan or [insert strong man] will always find the tools to jackboot. It's not capital C censorship because it's a private commercial space.
The real problem is that we've been made to feel like FBook and Google.stuff are the commons because they're 'free', but that's not what's going on.
posted by Reasonably Everything Happens at 7:33 PM on October 8, 2020 [1 favorite]
There was a rumor flying around last week that Facebook was going to pull out of Germany because they couldn't enforce the NetzDG. Most people I talked said they welcomed that. Of course, they're all leftie Berliners so...
To me this has always seemed simple. Facebook and Google are not common carriers that merely deliver messages from person a to person b. They curate the messages with algorithms, signal boost and bury specific messages, remix and reformat them, and intersperse relevant advertising between them. These activities show that they understand, at least algorithmically, the content of their messages, and thus they should be liable for the content of those messages.
If Facebook could credibly demonstrate that they don't know the content of their messages, by not running any algorithms on them or reposting them in other contexts, then they could perhaps claim a way out of this.
Like, as far as I know, DHL is not liable if someone starts sending Nazi magazines through the post.
Of course dictatorships are going to abuse such a system. In fact, all hate speech enforcement will be performed through the lens of the acting government, whatever it is. But let's focus the condemnation for dictatorships that enforce censorship of the post, of websites that are acting as journals/blogs instead of social media hosts, or of in-person meetings, instead of those that prevent Facebook from making a profit on QAnon pepe groups.
posted by sixohsix at 3:09 AM on October 9, 2020 [8 favorites]
To me this has always seemed simple. Facebook and Google are not common carriers that merely deliver messages from person a to person b. They curate the messages with algorithms, signal boost and bury specific messages, remix and reformat them, and intersperse relevant advertising between them. These activities show that they understand, at least algorithmically, the content of their messages, and thus they should be liable for the content of those messages.
If Facebook could credibly demonstrate that they don't know the content of their messages, by not running any algorithms on them or reposting them in other contexts, then they could perhaps claim a way out of this.
Like, as far as I know, DHL is not liable if someone starts sending Nazi magazines through the post.
Of course dictatorships are going to abuse such a system. In fact, all hate speech enforcement will be performed through the lens of the acting government, whatever it is. But let's focus the condemnation for dictatorships that enforce censorship of the post, of websites that are acting as journals/blogs instead of social media hosts, or of in-person meetings, instead of those that prevent Facebook from making a profit on QAnon pepe groups.
posted by sixohsix at 3:09 AM on October 9, 2020 [8 favorites]
This is an interesting topic because it straddles so many different fault lines, which is why among the opposition against the NetzDG you find all sorts of folks, left-wingers (Greens, Die Linke) as well as right-wingers (AfD) as well as classical-liberal/libertarian folks and other stragglers.
I don't think it's hard to imagine that when social media companies are tasked with the implementation of national censorship laws, this has a chilling effect on expression. When you outsource enforcement of national law to foreign companies, it's also unclear how those affected can appeal the decision making. If Facebook decides to pre-emptively bury or delete all posts related to the mistreatment of Kurds in Turkey, either because the law says so in so many words, or merely to pro-actively appease the government, then how do you protest something like that?
On the other hand European nations don't have the same tradition of free speech as the United States, and the NetzDG is a natural extension of that. For US-based free speech absolutists like the EFF, such state meddling is unconscionable, but as a well-indoctrinated European, I think that's putting the cart before the horse. The way I see it, freedom of expression is protected by the state/community because to do so serves the common good. But at the limit, where that freedom is used to weaken and destroy the very state/community that enables & protects this freedom to share points of view with one another, then it's not so clear what common good is being served. In such cases, restrictions on the freedom of expression may be necessary or desirable to protect the very community itself, quite apart from qualifications of 'hate speech' or 'harassment', which I think are themselves double-edged and/or vulnerable to abuse.
To some extent then I think this is also an ideological battle, in which the US, through its industry and its network of NGOs, tries to coerce and cajole nations/regions into "opening up" their notions of free expression, i.e. the reification of an absolutist notion of free speech as axiomatic of "free" societies. It's such an apparently simple, clear concept. Shouldn't people be allowed to say whatever they want? Isn't that democracy? (Well, sure, but the liberty to do whatever you want is not quite what community is). And so it becomes tied in with this concept of the freedom-loving West etc. etc. as opposed to creepy authoritarian others, never mind that no country incarcerates more people than the leader of the free world etc. etc. Boaventura de Sousa Santos has written extensively on this sort of human rights imperialism.
I don't really have a conclusion except that when behind your cause, I mean right behind you, there's people calling for armed revolution and/or the extermination of XYZ, and your cause also absolves the social media leviathans of any responsibility for the discord they sow within and between communities, then I'm not so sure how your cause promotes the common good in this particular case. But zealots will prozealotise.
posted by dmh at 7:38 AM on October 9, 2020 [4 favorites]
I don't think it's hard to imagine that when social media companies are tasked with the implementation of national censorship laws, this has a chilling effect on expression. When you outsource enforcement of national law to foreign companies, it's also unclear how those affected can appeal the decision making. If Facebook decides to pre-emptively bury or delete all posts related to the mistreatment of Kurds in Turkey, either because the law says so in so many words, or merely to pro-actively appease the government, then how do you protest something like that?
On the other hand European nations don't have the same tradition of free speech as the United States, and the NetzDG is a natural extension of that. For US-based free speech absolutists like the EFF, such state meddling is unconscionable, but as a well-indoctrinated European, I think that's putting the cart before the horse. The way I see it, freedom of expression is protected by the state/community because to do so serves the common good. But at the limit, where that freedom is used to weaken and destroy the very state/community that enables & protects this freedom to share points of view with one another, then it's not so clear what common good is being served. In such cases, restrictions on the freedom of expression may be necessary or desirable to protect the very community itself, quite apart from qualifications of 'hate speech' or 'harassment', which I think are themselves double-edged and/or vulnerable to abuse.
To some extent then I think this is also an ideological battle, in which the US, through its industry and its network of NGOs, tries to coerce and cajole nations/regions into "opening up" their notions of free expression, i.e. the reification of an absolutist notion of free speech as axiomatic of "free" societies. It's such an apparently simple, clear concept. Shouldn't people be allowed to say whatever they want? Isn't that democracy? (Well, sure, but the liberty to do whatever you want is not quite what community is). And so it becomes tied in with this concept of the freedom-loving West etc. etc. as opposed to creepy authoritarian others, never mind that no country incarcerates more people than the leader of the free world etc. etc. Boaventura de Sousa Santos has written extensively on this sort of human rights imperialism.
I don't really have a conclusion except that when behind your cause, I mean right behind you, there's people calling for armed revolution and/or the extermination of XYZ, and your cause also absolves the social media leviathans of any responsibility for the discord they sow within and between communities, then I'm not so sure how your cause promotes the common good in this particular case. But zealots will prozealotise.
posted by dmh at 7:38 AM on October 9, 2020 [4 favorites]
If Facebook could credibly demonstrate that they don't know the content of their messages, by not running any algorithms on them or reposting them in other contexts, then they could perhaps claim a way out of this.
Ignoring obvious edge cases where I'm searching for say refrigerators due to a work assignment and suddenly start seeing refrigerators in all my ads, knowing the context of what's said in private messages is completely beyond algorithms. When I say private message I guess I also mean public postings because it isn't possible for Facebook to human analyze all those too, so effectively to their engineers it is the same thing.
Google's Bert AI uses the infamous, "I went to the river bank today" and "I deposited a check at the bank today" phrase to demonstrate how hard it is to define context. Given that hate speech goes a step beyond that and even if taken to court requires a judge and possibly a jury to determine what constitutes harassment or hate speech ... no I do not want algorithmic determination of it.
I was talking on another online community and used the term BIPOC, was quickly shouted down as that being a term that white people invented to feel better about themselves. So even in online communities with similar goals the social mores of what's acceptable varies widely.
So I have no idea how a private or public company in good faith could enforce this without washing out everyone whose ideals didn't match whoever enforces the law.
posted by geoff. at 9:45 AM on October 9, 2020 [1 favorite]
Ignoring obvious edge cases where I'm searching for say refrigerators due to a work assignment and suddenly start seeing refrigerators in all my ads, knowing the context of what's said in private messages is completely beyond algorithms. When I say private message I guess I also mean public postings because it isn't possible for Facebook to human analyze all those too, so effectively to their engineers it is the same thing.
Google's Bert AI uses the infamous, "I went to the river bank today" and "I deposited a check at the bank today" phrase to demonstrate how hard it is to define context. Given that hate speech goes a step beyond that and even if taken to court requires a judge and possibly a jury to determine what constitutes harassment or hate speech ... no I do not want algorithmic determination of it.
I was talking on another online community and used the term BIPOC, was quickly shouted down as that being a term that white people invented to feel better about themselves. So even in online communities with similar goals the social mores of what's acceptable varies widely.
So I have no idea how a private or public company in good faith could enforce this without washing out everyone whose ideals didn't match whoever enforces the law.
posted by geoff. at 9:45 AM on October 9, 2020 [1 favorite]
« Older Hot August Night and Sounds of Silence for me | More Time Travel in the time of COVID-19 Newer »
This thread has been archived and is closed to new comments
Getting social media companies to do more to combat hate speech and disinformation is popular here, but moving quickly in this way inherently involves deputizing companies to shut down speech. It gives them more power--or at least normalizes exercising the power they already have--as well as legitimizing a the government role in this area too. Even if everyone was a good faith actor you'd get a lot of mistakes if you try to move quickly and at volume. And again, even assuming good faith, I don't see how you end up with anything other than a mainstream ruling on what is acceptable or not.
The best I can imagine is that you *also* combine this with protections for speech on these platforms, to minimize abuse by corporations (in their own interest) or foreign governments (ie, Orban wanting German citizen's "lies" removed about him.) But this would be mass chaos. Is there a way to do this that doesn't just end up, best case, as cementing a fairly narrow, centrist dialogue?
Minor quibble on most of the articles: They keep listing countries adopting these rules and saying how few are rated "free" by Freedom House--but only 20% of the countries meet that category in the Freedom House report. Near as I can tell the distribution is about the same as you'd expect by chance.
posted by mark k at 11:50 AM on October 8, 2020 [3 favorites]