A short but excellent piece by the Southern Poverty Law Center
January 18, 2017 8:08 PM   Subscribe

Google and the Miseducation of Dylann Roof - How did Dylann Roof go from being someone who was not raised in a racist home to someone so steeped in white supremacist propaganda that he murdered nine African Americans during a Bible study?
posted by You Should See the Other Guy (68 comments total) 31 users marked this as a favorite
 
Honestly, I would want to see a lot more research and data before I go blaming Google for that event. This is sort of like blaming the card catalog at the library (remember those?) for horrific events that took place in the '50's and '60's. There's a lack of solid research and some faulty logic in this piece.
posted by HuronBob at 8:30 PM on January 18, 2017 [27 favorites]


In the video the narrator says, "If Roof had first come to another website, history might have been different."

This is such a simplistic, reductive, sensational and disappointing statement from the Southern Poverty Law Center. Dylan Roof had a lot of problems, and it's possible that he was mentally ill. He was socially isolated. While it's difficult to predict what course his life might have taken if he hadn't become obsessed with "black on white violence", to say that the Google search algorithm was responsible seems to be a really odd conclusion to make.

Why are there so many mass killings in the United States compared to other countries? Why are there so many Dylan Roofs out there? I don't think Google is to blame.

There's this idea that algorithms are promoting "fake news", but the problem is that there is a huge number of people who actively seek to avoid cognitive dissonance, and instead search for information that reinforces their biases.

In short, even though, as the Southern Poverty Law Center lamentably says, "Roof had black friends" (and therefore could never be racist, right?), he was a racist. Maybe it was a product of his environment, or a product of 500 violent years of American history.
posted by My Dad at 8:53 PM on January 18, 2017 [34 favorites]


I agree with My Dad (well, that was an interesting statement that jolted me the minute I typed it, given that my dad died when I was 6 months old and I don't remember a thing he might have ever said , but this is another story), this falls far short of what I expect from the SPLC (whose mailings we receive on a very regular basis and whom we donate to).
posted by HuronBob at 8:58 PM on January 18, 2017 [6 favorites]


This link is pretty brief, but I have noticed other documented instances that goes into more of how Google's algorithms are privileging racist references. In the last month, I've seen the following:

[Guardian] How to bump Holocaust deniers off Google’s top spot? Pay Google :

The Holocaust did not happen. At least not in the world of Google, it seems. One week ago, I typed “did the hol” into a Google search box and clicked on its autocomplete suggestion, “Did the Holocaust happen?” And there, at the top of the list, was a link to Stormfront, a neo-Nazi white supremacist website and an article entitled “Top 10 reasons why the Holocaust didn’t happen”.

On Monday, Google confirmed it would not remove the result: “We are saddened to see that hate organisations still exist. The fact that hate sites appear in search results does not mean that Google endorses these views.”

The Independent ran the story. As did Fortune. And the Daily Mail. And the Jerusalem Post. And the Drudge Report. But Google held firm. David Duke, former imperial wizard of the Ku Klux Klan, tweeted his support for the decision. And over on Stormfront – the website where Anders Breivik nurtured his ideas – members celebrated.

[...] Until Friday. When I gamed Google’s algorithm. I succeeded in doing what Google said was impossible. I, a journalist with almost zero computer knowhow, succeeded in changing the search order of Google’s results for “did the Holocaust happen” and “was the Holocaust a hoax”. I knocked Stormfront off the top of the list. I inserted Wikipedia’s entry on the Holocaust as the number one result. I displaced a lie with a fact.

How did I achieve this impossible feat? Not through writing articles. Or shaming the company into action. I did it with the only language that Google understands: money. Google has shown that it will not respond to outrage or public sentiment or any sense of morality or ethics. It does not accept that leading people with a genuine inquiry about whether the Holocaust happened to a neo-Nazi website is grossly irresponsible or that it demeans the memory of the six million Jews who died. But it was prepared to take my cold, hard cash. A Google spokesman said: “We never want to make money from searches for Holocaust denial, and we don’t allow regular advertising on those terms.”

-
Google "Did the Holocaust happen" and a neo-Nazi website is the first result


- (last week's update) Did the Holocaust happen? Google's top search results still say it's a hoax

so i think it does the SLPC's short piece a disservice if it's being read by itself.
posted by cendawanita at 9:01 PM on January 18, 2017 [39 favorites]


Roof didn't even graduate high school and repeated the 9th grade twice. It seems his lack of education and critical thinking skills lead him to be duped into the whole white supremacist idealology. Had he been exposed to more non-supremacist sites there is no guarantee that it would have swayed his mind. Aside from whatever issues he has (mental health etc) he seems to be an idiot first and bigot second.
posted by boubelium at 9:04 PM on January 18, 2017 [4 favorites]


You know how Metafilter was completely fucked over by Google?

Guess what happened to FARK.com this year? Only it was more completely, deliberately malicious on Google's part, and they only lifted the boot from Drew's throat when it seemed like Google's fuckery was about to go viral. It will be a miracle if the site makes it to spring.

Google is gunning for venues of free expression, especially honest news aggregators and artistic effort on YouTube - it wants them erased for speed runs and fake news in the context of fake history, where only the Biggest of Daddies can save us all from the horrors of other-ethnicity and non-Literal-Bible faiths...

Google Delenda Est.

Fuck Google. Screw Bing and the duck thing. We need anonymous, decentralized search,with peer reviewed algos and thorough user testing, and we need it now.
posted by Slap*Happy at 9:05 PM on January 18, 2017 [37 favorites]


I wouldn't want to get into a discussion about Google and Metafilter (and Fark.com's demise has nothing to do with Google), but I would say that the majority of organic traffic to websites now comes from social media (and has for some time), and not Google.

People are not getting their information anymore by typing search terms into Google; they're getting information from Facebook, Reddit, Twitter and web forums such as 4chan and Stormfront. So any site that relies on a Google ad network to pay the bills will have a very hard time, because the traffic is just not there.
posted by My Dad at 9:20 PM on January 18, 2017 [4 favorites]


The holocaust thing is freaky, but it's not surprising that the first result would be bogus, since the very question is in bad faith. For a depressing experience try the results for "Is Islam evil" or "are women inferior to men".
posted by dmh at 9:27 PM on January 18, 2017 [8 favorites]


If racism is popular (or otherwise pings the algorithm) Google will happily lead you by the hand to it. It's a legitimate issue. I think figuring what can be done about it is a little tricky though. It seems to me that many of the alt-right folks - especially say the "race science" or Holocaust denier types - have a strong sense that they are tapping into suppressed knowledge, and it seems undesirable to inflame that too much because it encourages further detachment from the rest of the world.
posted by atoxyl at 9:27 PM on January 18, 2017 [3 favorites]


Further to my previous comment, asking "does google have a liberal bias" yields plenty of sources that find Google has a liberal bias, whereas asking "does google have a right wing bias" finds the opposite. The problem is not so much the answers. It's that somebody who asks those kinds of loaded questions is not interested in the truth in the first place.
posted by dmh at 9:40 PM on January 18, 2017 [23 favorites]


How did I achieve this impossible feat? Not through writing articles. Or shaming the company into action. I did it with the only language that Google understands: money.

Way to bury (in fact, ignore) the lede there! How did it ACTUALLY happen? Grumble grumble.... (clicking through)

AAAH, he paid for an ad, which goes above the search results. But it's not a search result, it's an ad, and it has the little Ad before it, which, while small and chosen to be overlooked I wouldn't doubt, is still in a box to itself.

This matters. In Google's early days they made a conscious decision not to mix ads up with their results, or allow payment in exchange for coming in high in search rankings. The fact that they did not do this is entirely the reason that Google crawled out of the mire of the early web, over Altavista, Excite, Lycos and a half-dozen other forgotten names, because: 1. they had excellent search results, and 2. they refused to dilute them in exchange for money.

This was the basis of the company's famous "Don't be evil" dictum. Whether they've held to that generally in the two decades since this all happened, that's for the reader to decide. But in this area, they've held to it.

The real problem here is that, if you want a specific thing to come up at the top of search results, you can do that if you search long enough for the right text. If you search for "holocaust," the resulting finds are all definite that it did happen, and it was horrible. If you search for "did the holocaust," then you're going to find denier sites, because most sites that would think to ask that question have already decided themselves that the answer is no. Despite some misguided moves into that area, primarily Google is still not a natural language search, and, rightfully, it will show exact text matches before trying to answer your question for you.
posted by JHarris at 9:48 PM on January 18, 2017 [45 favorites]


> Further to my previous comment, asking "does google have a liberal bias" yields plenty of sources that find Google has a liberal bias, whereas asking "does google have a right wing bias" finds the opposite.

So is google baised toward giving people just the answers that satisfy them?
posted by anadem at 9:49 PM on January 18, 2017 [8 favorites]


I wonder how hard it would be to Googlebomb, say, the 50 most egregious examples of this kind of thing.
posted by Gerald Bostock at 9:56 PM on January 18, 2017 [1 favorite]


given how powerful internet propaganda is now, there may be a future for NGOs that specialize solely in writing informational websites on topics that hate groups are known to fixate on again and again (race war, the holocaust, etc...) and popularizing their anti hate sites in any way possible, including paying google as that guardian article explained. look, if money is the issue, i think that most hate groups would lose a fundraising war with even a modestly funded nonprofit, right? these hate groups are like cockroaches, thriving in places where nobody is fighting back.
posted by wibari at 10:04 PM on January 18, 2017 [5 favorites]


So is google baised toward giving people just the answers that satisfy them?

I think it's just that the question is loaded in itself. The people who will write articles with phrases like "does Google have a liberal bias" are people who want to argue that it is, and vice versa. It's akin to confirmation bias, I guess, in that you can't ask questions about things without already having some prior knowledge or feeling about them one way or the other.
posted by dmh at 10:14 PM on January 18, 2017 [3 favorites]


PS: this is why education
posted by dmh at 10:16 PM on January 18, 2017 [11 favorites]


Blaming Roof on Google and their algorithms feels like the kind of conspiracy theory thinking that SLPC used to debunk. What happened to them?

The assertion here seems to be more that PageRank, an algorithm for ranking document relevance based on keywords, is easily susceptible to unintended negative side effects, considering that no matter how many special cases it's taken on since its inception the algorithm in question is almost certainly unprepared to take into account the full breadth of human textual expression in terms of which documents it privileges. There's not really a conspiracy being asserted, and honestly not believing that Google Search is fundamentally vulnerable to this type of defect due to both the algorithm's limitations and the company's incentives seems laughably naive.

I'm also confused by the implication in this thread that Google Search has an objective output in a way that distinguishes it from Twitter, Facebook, etc. Y'all know that search results have been personalized for years now, right? All of these properties share the same fundamental mechanism of tailoring what the user sees to fit the mold of what they've seen before and what they're expected to want to see, so the fundamental mechanism (and the fundamental problem) is the same whether we're talking about Google Search or a social network.
posted by invitapriore at 10:25 PM on January 18, 2017 [26 favorites]


The people who will write articles with phrases like "does Google have a liberal bias" are people who want to argue that it is

It's interesting that leading-question Google results are like the inverse of Betteridge's Law of headlines.
posted by JHarris at 10:31 PM on January 18, 2017 [12 favorites]


Also, man, the stupid on this topic really seems to come from all sides at this point. The executive and consultant types adjacent to or inside of this industry have always been cognitively compromised, since it's a fundamental requirement of the job, but now with the programmers who couldn't possibly imagine anything expressible in code having a bias becoming more vocal and effective as lackeys of capital there's close to no one with a finger in this shit that sees any problem with it. Anyway, looks like Zuckerberg's gunning for a run in 2020.
posted by invitapriore at 10:36 PM on January 18, 2017 [6 favorites]


The heavy-handed blaming of Google strikes me as so weirdly simplistic that I can't understand why SPLC is behind this. You search Google for the things you're looking to find; that is literally the point of Google.

Just because Dylan Roof says that his searches led him to the waters of neo-Nazi white supremacy and encouraged him to drink deep doesn't mean that there aren't larger waters lapping at your feet that you choose to ignore.
posted by desuetude at 12:11 AM on January 19, 2017 [5 favorites]


Try:
Do UFOs exist
Do ghosts exist
Do angels exist
Do sea monsters exist
Do ancient astronauts exist

GIGO
posted by happyroach at 12:29 AM on January 19, 2017 [5 favorites]


If Roof literally searched for "black on white violence" in the way the article states, with the W of white capitalized, that's... already a dog whistle. Especially with a lower-case b in black. But I can't tell from the article whether that's a literal citation.
posted by Rush-That-Speaks at 12:57 AM on January 19, 2017 [1 favorite]


I don't think we should take Roof's word for it that a few innocent Google searches led to his radicalization. Firstly it's obviously a self-justifying account, he wants us to think that his views are a result of exposure to "facts" and not from some personal or emotional cause. Secondly he probably put that story in his manifesto because he wants people who read it to Google those terms for themselves. His story doesn't ring true because in his account he interprets everything he reads through a white nationalist lens (Europe is the "White homeland" etc) when he's supposedly describing how he encountered this stuff for the first time and didn't have these views beforehand.
posted by L.P. Hatecraft at 1:02 AM on January 19, 2017 [9 favorites]


This struck me as more of that damned "INTARWEBS! THINK OF THE CHILDREN!" neo-Luddite stuff people use to get more eyes/clicks, frankly. All opinion, and little or no replicable facts.
posted by Samizdata at 1:15 AM on January 19, 2017 [2 favorites]


He was a loser who didn't have a girlfriend and probably had no sex life beyond porn. Other than a love of semi automatic weapons, that seems to be the common ground on which all these violent freaks stand. In fact, it's difficult to name a mass shooter who was successful in life or with women. I can't think of one. Other than easy availability of assault weapons, everything else is window dressing.
posted by Beholder at 1:30 AM on January 19, 2017 [2 favorites]


They fixed this now but Google was doing this last year in autocompleting 'How many people did Hil'

How many people did Hillary have assassinated
How many people did Hillary get killed
How many people did Hillary assassinate
How many people did Hillary murder
et cetera

Appalling.
posted by adept256 at 1:37 AM on January 19, 2017 [5 favorites]


a huge number of people who actively seek to avoid cognitive dissonance, and instead search for information that reinforces their biases

easy availability of assault weapons

I suspect a quick Venn diagram showing countries where these two statements are true would only have one area of overlap.

There may be many contributing factors such as upbringing, education, mental-health care, echo chambers or Google results, but these mass-shootings in the USA all have (mostly) one thing in common - that person was able to buy an arsenal of powerful weaponry in a shop.
posted by jontyjago at 1:49 AM on January 19, 2017 [9 favorites]


I wouldn't want to get into a discussion about Google and Metafilter (and Fark.com's demise has nothing to do with Google), but I would say that the majority of organic traffic to websites now comes from social media (and has for some time), and not Google.

You're confusing traffic from Google searches with ad revenue from Google ads. Metafilter's ad revenue dropped because Google doesn't send as many searches to AskMeFi as they used to (possibly because that's not actually what people were looking for). Fark is claiming they lost money because Google refused to pay ad revenue for traffic Fark actually had (supposedly because Google mistakenly flagged Fark as having child porn). Whether that traffic was sent to Fark from Google or not shouldn't matter for the issue of whether they get paid for Google ad impressions.
posted by straight at 2:24 AM on January 19, 2017 [2 favorites]


Google news search has degraded to the point where I gave up my voluntary curation of news on a particular topic to a very active timeline. Yes, race is involved. Yes, the degradation of quality and insertion of bullshit was obvious to anyone using the service daily for the past couple of years. It plunged right after the election. Alphabet soup supports tangerine face.
posted by infini at 2:45 AM on January 19, 2017 [5 favorites]


This can be used for good, too, just search for Santorum.
posted by chavenet at 2:47 AM on January 19, 2017


He was a loser who didn't have a girlfriend and probably had no sex life beyond porn.

I don't see how a 19 year old who doesn't have a girlfriend or a sex life is relevant to killing 9 Black people in a church.
posted by roomthreeseventeen at 3:56 AM on January 19, 2017 [44 favorites]


People who didn't have girlfriends at the age of 19 (or, indeed, ever): Kant, Newton, Tesla. Gandhi was married as a child but was celibate for many decades afterwards. I don't think it was the absence of assault weapons that prevented Kant from setting out on a murder spree.

It's only the Dylann Roof mindset - status-obsessed, shallow, misogynist at root - that sees sex with women as being so intertwined with status and pride that not having had sex is a source of world-destroying shame and rage. The further away from that story we get the better.
posted by Aravis76 at 4:05 AM on January 19, 2017 [37 favorites]


I don't see how a 19 year old who doesn't have a girlfriend or a sex life is relevant to killing 9 Black people in a church.

It isn't just him. It's all of them. Impotent rage funneled through the barrel of a gun.
posted by Beholder at 4:27 AM on January 19, 2017


It isn't just him. It's all of them. Impotent rage funneled through the barrel of a gun.

Nope, sorry. There are many 19 year old virgin men who do not go out and kill.
posted by roomthreeseventeen at 4:28 AM on January 19, 2017 [29 favorites]


Yeah, the virgin-shaming is kind of disappointing.
posted by escape from the potato planet at 4:34 AM on January 19, 2017 [25 favorites]


And Anders Breivik had a girlfriend and, I believe, still does. In the case of Roof, I think the racism, grandiosity and alienation from his community that he shared with Breivik is much more important than the celibacy he shared with Newton.

Maybe celibacy is a proxy for being alienated from any kind of community? But it's not a very good proxy, since lots of people who are well-integrated into society are not sexually active - especially at 19 - and lots of people who are married or partnered are, nevertheless, socially isolated.
posted by Aravis76 at 4:36 AM on January 19, 2017 [4 favorites]


Usually, mass murderers have a history of domestic violence. So, I guess in that respect, it is unusual that Roof didn't have a girlfriend or wife that he abused prior to murdering nine people.
posted by melissasaurus at 4:58 AM on January 19, 2017 [2 favorites]


We're a long way from the A&P.
posted by notyou at 5:00 AM on January 19, 2017


Yeah can we move away from the derail about women as objects whose mere presence for sex would have been able to curtail mass murder? Pretty sure that guns are the tools used for rage you're looking for. Also misogyny, i.e. the systemic thing that makes it possible to seriously claim that half of humanity (men) can be pushed to murder people because they're virgins. What the hell. And that's not even getting into the heteronormativity beneath it. Why do you assume he's straight?

Anyway. End derail.
posted by fraula at 5:28 AM on January 19, 2017 [29 favorites]


Blaming this on Google is really disingenuous. When I was 12 years old, I had an assignment for English class: make a HyperCard (raise your Mac OS7 in the air like you don't even care) presentation about white supremacy. I have no idea who signed off on that idea, but my preteen self happily plodded off to the school computer lab, where I fired up a Lycos search and went searching for whatever low-res JPGs I could borrow to fill up half a dozen slides. The Wayback Machine doesn't tell me what was on the top of the search results list for a bunch of neo-nazi shibboleths, but my intrepid searches didn't take me to the SPLC; they took me to a nascent KKK web presence, complete with white power propaganda and warnings of the coming race war. I had to wade through a lot of really vile stuff to find my spinning Confederate flag GIFs, and I was not yet at the age of being able to look at source integrity.

And yet, even with some pretty nasty racism being funneled my way at home, somehow I managed not to murder a room full of African Americans. I guess that's good, because otherwise we'd have gotten to read a sensationalist takedown of Altavista in the early '00s.

I haven't thought about that presentation in almost twenty years, so GODDAMN what a fucked up assignment that was to give a seventh-grade kid
posted by Mayor West at 5:36 AM on January 19, 2017 [16 favorites]


"Fuck Google. Screw Bing and the duck thing. We need anonymous, decentralized search,with peer reviewed algos and thorough user testing, and we need it now."

Super hero mefites- you have been summoned... ;)
posted by xarnop at 5:40 AM on January 19, 2017 [1 favorite]


Why are there so many mass killings in the United States compared to other countries?

I went to google to find out. Reportedly, per capita, the US does not come in on top. Of course, there are other ways to look at it. Statistics and all. But human nature is pretty constant.

As to the story - weak brew. As well to ask, what was Michael Steven Sandford reading? The SPLC needs stories like these to keep them relevant. You can make a good living by keeping relevant.
posted by IndigoJones at 5:54 AM on January 19, 2017


> This link is pretty brief, but I have noticed other documented instances that goes into more of how Google's algorithms are privileging racist references.

Anecdata: the other day I did an image search for the magic 8 ball "OUTLOOK NOT SO GOOD" sign (one guess why!) and one of the top results was from a Stormfront message board. I almost wound up using it before I noticed.
posted by The Card Cheat at 6:09 AM on January 19, 2017 [1 favorite]


It looks like a new version of the old habit of blaming "The Internet" for everything bad that happens to anyone connected to it. It was such a scary place, full of child pornography and bomb-making instructions. Now that we're more sophisticated about such things, the practice is to lay the blame more specifically on whichever technological monster the writer doesn't understand, usually Google, Facebook, Twitter, or Tor. They're easier targets.

Of course there are also some less imaginary problems with Google or anyone else having a monopoly on search. If this recent tendency to demonize it somehow magically results in "anonymous, decentralized" things taking over, it will have been worthwhile.
posted by sfenders at 6:15 AM on January 19, 2017 [6 favorites]


FWIW, Google seems to have addressed the various Holocaust questioning search results in December of last year.
posted by GameDesignerBen at 6:19 AM on January 19, 2017


Anecdata: the other day I did an image search for the magic 8 ball "OUTLOOK NOT SO GOOD" sign (one guess why!) and one of the top results was from a Stormfront message board. I almost wound up using it before I noticed.

Corroborating anecdata: A few years ago, I was working on a web design project at work (I am not a designer, but I know enough about HTML to be dangerous) and Googled how to do some kind of moderately-simple formatting trick in CSS. I clicked on the top result, only to land in Stormfront's "beginning-web-design-for-Nazis" subforum. I immediately closed the tab and went about deleting my browsing history, cookies, and everything. Looking back on it, that may have been my first indication that the white supremacist movement had gotten its hooks into the modern internet.
posted by Strange Interlude at 6:38 AM on January 19, 2017 [3 favorites]


It looks like a new version of the old habit of blaming "The Internet" for everything bad that happens to anyone connected to it.

Very true - you'll notice that hardly anybody blames video games anymore...
posted by jontyjago at 6:56 AM on January 19, 2017 [2 favorites]


Various people have looked into how radicalization works and have proposed that it might be possible to steer some people away from radicalization by influencing their search results. And I think we can all agree that the outcome in that particular instance would be good it it can be done.

The problem is that to achieve that outcome you need to construct a mechanism by which search engines attempt to shape our thinking, our ideology, our basic underlying assumptions about the universe, by altering our search results. And that mechanism would need to be under the control of some agency.

Do we put that mechanism under the control of Google? A faceless for profit corporation? That seems unwise.

Do we put that mechanism under the control of the government? I'm not sure I'd trust Obama with such a thing and I damn sure wouldn't trust Trump with it.

Who decides which ideologies are to be discouraged by search result and which are to be encouraged? Someone has to, and we won't always be in agreement about what should be discouraged.

There's a good argument to be made that the best course of action is to deliberately choose not to construct the means by which we can influence a person's mind by altering search engine results to steer them in a direction we like, that the power this represents is too great, too dangerous, and would inevitably be misused.

*****************

There's also a good argument to be made that the above is pure bullshit because no matter what **SOMEONE** will build such a mechanism and it would be best to keep such a thing in the open where we can publicly debate its flaws and argue about what it should be doing rather than allowing one to be built in secret so we don't even know it exists and are thus incapable of debating what it should do.

Google and all other search engines are not neutral. They **ALREADY** acknowledge that they censor and alter search results, but they do so in a sort of roundabout way that doesn't really get all that explicit about what exactly is filtered, altered, and censored. Child pornography, for example, is blocked and most people would agree that's a good thing.

So the genie is out of the bottle, search is not neutral, there is currently an agenda, a set of filters and alterations of results in place, but they aren't really well known, written anywhere we can see and critique, and its all rather murky.

If there's going to be censorship and search engine tuning to steer us in a particular direction, or away from particular topics, I'd vastly rather it be openly acknowledged and the subject of informed debate rather than kept to the shadows and hidden from us.
posted by sotonohito at 6:59 AM on January 19, 2017 [4 favorites]


I will also add that just as "terrorism" is a term reserved for people of color or Muslims, so too is radicalization. You'll note that none of the major news programs discussed where or how Roof may have been radicalized, while many reports on "terrorists" focus on that issue.
posted by sotonohito at 7:00 AM on January 19, 2017 [5 favorites]


Deceptive linking has always been one of Stormfront's major tactics. They specifically target different interests and communities, and even seem to have concentrated recruitment efforts where they have people go into various forums and do basically targeted marketing. They also seem to try to improve their search ranking by creating material that people will unthinkingly link to.

They seem to come in waves, as though they're actual coordinated campaigns where users go out and insert themselves into different forums. They've done it on Reddit, and Digg before that, and I think they used to recruit on Usenet too, but for some reason, I can't recall any specifics. Oh, and Pinterest used to be just lousy with Stormfront links, seeded from image-heavy threads on their forums that I would suspect were specifically created as Pinterest links, because they were nonsensical in any other context.

If I had my druthers, links to Stormfront would just be default banned everywhere. I'm normally against that sort of thing, but they've been consistently deceptive for as long as they've existed, to the point that I think you could argue that it's just a malicious site with no real informative value, so they should be treated like malware.
posted by ernielundquist at 7:04 AM on January 19, 2017 [4 favorites]


Here's another article from the Guardian talking about Google's autocomplete and answer-box features: Google, democracy and the truth about internet search

The author highlights one example, where she typed "are women," and the first autocomplete suggestion was "are women evil." When she accepted the first suggestion, google tried to save her a click by excerpting and featuring this answer in a box at the top: "Every woman has some degree of prostitute in her. Every woman has a little evil in her… Women don’t love men, they love what they can do for them. It is within reason to say women feel attraction but they cannot love men." (screenshot from the article)

Which, I don't think this example is evidence for are google evil. I think Google is a well-designed tool that is trying to be helpful. I think any equally powerful tool would cause equally bad problems. But as someone who builds tools, if I had built that tool and that edge case came up, I wouldn't be thinking "pfft no one learns anything from my tool anyway this is fine." I would be thinking "shit, I wonder how bad this is." (And then I would patch that particular edge case -- google no longer suggests "are women evil" or includes an answer box for the results. Which in itself is a good reminder that what you're getting from Google is on some level curated.)

Then the question of "how bad is this edge case" gets me thinking about danah boyd's piece on media literacy. We're telling kids to do their own research, be skeptical of sources and biases, and come to their own conclusions -- all good advice! And that means you don't have to be biased to search for "did the holocaust happen." You just have to have to be young, and have some friend tell you it didn't happen, and decide to practice good media literacy for yourself. So if I built a tool that systematically gives biased answers to dumb questions, I could maybe expect the impact of that to be real.

You can see people taking that problem seriously in the search results. There's now sites dedicated specifically to answering, yes, the Holocaust did happen, in an SEO-friendly way -- so this is now recognized as the battlefield it is. One of the top hits is Search Engine Land explaining that Google changed its general algorithm specifically motivated by that search -- "When non-authoritative information ranks too high in our search results, we develop scalable, automated approaches to fix the problems, rather than manually removing these one-by-one. We recently made improvements to our algorithm that will help surface more high quality, credible content on the web. We’ll continue to change our algorithms over time in order to tackle these challenges." So instead of Holocaust deniers owning the results for the stupid question, and no one else paying attention, we now have deniers fought by anti-deniers, and Google the tool builder trying to put its thumb on the scale in favor of an algorithmic truth that is hopefully pretty much like my own notion of truth. (Culturally I think I'm pretty much like most Google folks, so I believe that they hope their algorithmic notion of truth lines up to mine.)

I guess I'm saying it's too easy to suggest this is nothing but moral panic -- it's good for advocates to take it seriously, and good for Google to take it seriously, as they both seem to be doing.
posted by john hadron collider at 7:05 AM on January 19, 2017 [10 favorites]


It's odd to me that no one has yet suggested that perhaps SF itself is responsible for SF showing up at a high rank in many searches. Is it far-fetched to suppose they put some SEO/Googlebombing effort into that, or that the web activity of its membership really does form the sort of network-of-relevancy Google is designed to expose?

On preview, I see ernielundquist has beaten me to the punch, with confirmation no less.

I'm not suggesting that Google should escape pressure in this question, but I expect it genuinely is a difficult problem.
posted by Western Infidels at 7:10 AM on January 19, 2017 [4 favorites]


it's too easy to suggest this is nothing but moral panic

This particular case is very little other than moral panic. Pointing out flaws in Google's algorithms is worthwhile and perhaps even important. Blaming said flaws for turning a kid into a mass murderer by making it easy for him to find the white supremacist bullshit he was searching for, that is less helpful.
posted by sfenders at 7:56 AM on January 19, 2017 [6 favorites]


Roof's saying that Google led him to this stuff reminds me of Ted Bundy's eve-of-execution "confession" that porn made him a serial killer, and SPLC, for all the good work that they do, are enabling the same sort of bullshit blame-shifting.
posted by Halloween Jack at 8:48 AM on January 19, 2017 [2 favorites]


One of the reasons I got off Facebook was because otherwise perfectly nice people I went to elementary school with would post articles from Stormfront or worse. And when they were called out on it, the only coherent answers were along the lines of "I like this stuff" or "it makes you think." So it take two to tango. Dylann Roof might have found that stuff on Google, and maybe Google was cooked to make it easy for him to find that stuff, but he had to like that stuff, too.
posted by lagomorphius at 9:00 AM on January 19, 2017 [4 favorites]


Fuck Google. Screw Bing and the duck thing. We need anonymous, decentralized search,with peer reviewed algos and thorough user testing, and we need it now.

Spoken by someone who's obviously never tried to build a search engine. I have, and I'm also a lot more familiar with the details of how Google search works than the average techie. It's extremely complicated and it relies on a huge amount of infrastructure.

If it were easy to do better than Google, someone would have done it. It's not as if others haven't tried over and over again. When even a company as well-funded as Microsoft has trouble competing, it's hard to imagine some scrappy nonprofit is going to come along and build an ideologically pure search engine that people actually want to use.

A lot of the difficulty comes from that fact that there's an entire industry (SEO) dedicated to gaming search algorithms. Not all SEO is evil, but a lot of it is dedicated to actively making search results worse. SEO attracts a lot of clever people because the incentives for gaming search results are huge. As a result, search engines are in a constant arms race with the SEO industry. Relying on peer-reviewed algorithms would give SEOs a huge advantage, partly because it would greatly slow the rate of algorithm development, but also because it would give SEOs a blueprint for how to game the latest algorithm.

If you want a volunteer-curated directory of the internet, you're welcome to use DMOZ. Good luck getting the rest of the world to do the same.

They fixed this now but Google was doing this last year in autocompleting 'How many people did Hil'

How many people did Hillary have assassinated
How many people did Hillary get killed
How many people did Hillary assassinate
How many people did Hillary murder
et cetera

Appalling.


Google generates autocomplete results based on what people actually search for. They make a significant effort to avoid offensive or defamatory suggestions, but it's a constant game of whack-a-mole. Problematic completions can only be fixed after someone discovers them, and more often than not, they're discovered by someone outside of Google. What part of that is appalling?

Google and all other search engines are not neutral. They **ALREADY** acknowledge that they censor and alter search results, but they do so in a sort of roundabout way that doesn't really get all that explicit about what exactly is filtered, altered, and censored. Child pornography, for example, is blocked and most people would agree that's a good thing.

The difference between child pornography and something like Stormfront is that child pornography is illegal and Stormfront is not. There's a huge difference between censorship of illegal content and censorship of politically incorrect ideas.
posted by shponglespore at 11:46 AM on January 19, 2017 [5 favorites]


There's a huge difference between censorship of illegal content and censorship of politically incorrect ideas

To be fair, some of the madder far-right content is illegal in many places in Europe, under hate speech laws. The distinction between the illegal and the 'politically incorrect' is not a simple one, and many Western democracies draw the line in a different place than the US does. I've always had a lot of respect for the First Amendment, but it seems obvious that it has played some role in the global success of the far-right. Groups like the American Nazi Party - which would have been proscribed as hate groups in most places in Western Europe - have flourished and built up their organisations over the last several decades and are now broadcasting their stuff globally with unprecedented success, thanks to the internet. But you can't blame Google for that.
posted by Aravis76 at 12:08 PM on January 19, 2017 [1 favorite]


I think what is important here is not Google Bad but Google asleep at the switch. It seems to me that what we are living through is the death of one source of authority and the rise of another. Traditional Media is dying or has died as a source of authority. Nowadays for the majority of persons the internet is the source of authority and Google is one of the paragons of that authority even if they deny it. They get a free pass for the most part because their reputation of being benign and trying their best to come up with solutions that scale is intact. I don't think this is a good future and I hope it doesn't stand. There has to be a way make algorithmic life support human moral decision as opposed to clouding it. I realize that we live in the era of libertarian "do your own research" but I think that era leaves a lot to be desired. We live in a time when some mutton head shoots an assault rifle in a pizzeria because of a ludicrous conspiracy theory. I do not think that would happen in a pre web world, that is to say not by a person who was not insane.
posted by Pembquist at 1:03 PM on January 19, 2017 [4 favorites]


As shponglespore mentioned , Google's default autocompletion weighs prior queries highly in their algorithm to populate that dialog box. The problem is a matter of understanding how people search, and how Google and other search engines parse natural language queries versus literal string searches, and the messed-up results you get when users use both.

Having used search engines and possessing an inkling of an idea of how they parse data, if I wanted to know if the Holocaust occurred, I'd search for "holocaust" or "the holocaust." Someone asking the search engine a question -- which is mitigated by some language parsing, although I'm not sure how much -- would type "did the holocaust happen".

You could break that out: words like did, does, where, when, and how are sometimes qualifiers. But we're querying strings, not natural language. Googling "did the holocaust happen" will not send you to pages about "the holocaust," it will send you to pages with the literal string "did the holocaust happen" and no one who understands history is going to write that phrase .
posted by mikeh at 1:31 PM on January 19, 2017 [3 favorites]


Googling "did the holocaust happen" will not send you to pages about "the holocaust," it will send you to pages with the literal string "did the holocaust happen" and no one who understands history is going to write that phrase .

But they could though. Which is a good point made earlier. It's one thing to call out Google for not tweaking their algorithm to downplay miseducation, but what the forces for good need to do is to get better at the same techniques that SF and the like are using.

If white-supremacists can have an MLK themed domain, there's no reason why the forces for good can't have didtheholocausthappen.com and just have it be a big old YES, with links to reasonable sources.

I know there are reasons why it doesn't happen (that kind of manipulation of information probably seems icky to people who lean left, for example). But that doesn't mean that it couldn't.
posted by sparklemotion at 1:45 PM on January 19, 2017 [2 favorites]


Even back before they dropped the word "don't" from their motto, Google clearly meant their algorithm to be self-evolving, using search queries and results as the inputs that would allow the algorithm to refine itself. The idea of applying human intervention to this process has always been a bit icky to them. They were kind of forced to step up in this regard because of the child porn and copyright infringement things, which got lawyers involved. Now everything like this is a new plateau where Google has to ask whether to intervene in the great self-evolving algorithm because X group got an issue of Y depth with how it worked out.

The desire to let the algorithm rule itself is strong precisely because it doesn't require human labor. Google crushed Yahoo in the early search wars because Yahoo's human curation of search results couldn't keep up with Google's automatic indexing and machine lookup. But it is Yahoo's method which was superior in this regard, since something like holocaust denial or blatant racism is dead obvious to a human moderator who has to approve a link. The very concept of "googlebombing" is built around the idea of saturating the robot with queries designed to make it think your edge case is the new normal, and index it prominently.

While Google's implementation of search has a questionable component, its treatment of ad customers is contemptible. Many businesses trusted Google and readily settled on them as a primary revenue source for their web presence, and because they have no real competition when Google pulls the rug out from under them with no realistic appeals process or accountability the result is often a dark office and OUT OF BUSINESS sign. This is a terrible situation best fixed by having a realistic competitor, but unfortunately the only player out there that seems to have any potential in that area is Amazon. *retching emoji*
posted by Bringer Tom at 5:20 PM on January 19, 2017 [2 favorites]


does google have a liberal bias"

liberal means open minded. Seems like that should be a prerequisite for any search engine
posted by any major dude at 6:25 PM on January 19, 2017 [1 favorite]


Fuck Google. Screw Bing and the duck thing. We need anonymous, decentralized search,with peer reviewed algos and thorough user testing, and we need it now.

Good luck... do you have any inkling if how much infrastructure google runs tongive you those search results that fast? Even if you had all the tech and algorithms ready you're not replicating this anytime without serious money.
posted by coust at 7:37 PM on January 19, 2017


Good luck... do you have any inkling if how much infrastructure google runs tongive you those search results that fast?

How goddamn old is map and reduce? It could take driving lessons and have a license by now if it... oh wait. It actually does in California.

I mean, Google/Alphabet isn't Apple big, but it's more than big enough to sic competent coders at corner cases. It has done this most half-assedly and with much eye-rolling.

Moore's law is slowing down, sure, but we have more than enough processing power and network bandwidth on tap to make decentralized search a thing.

Google delenda est.
posted by Slap*Happy at 8:14 PM on January 19, 2017


How goddamn old is map and reduce? It could take driving lessons and have a license by now if it... oh wait. It actually does in California.

I'm specifically not talking algorithms, I'm talking infrastructure... data centers, private fiber, organisational practices to manage that, etc... this more so than page rank & map reduce is the strength of google. You're not replicating that without dedicated hardware/network links, and those cost money.

Its not that I don't think they could change their system to eliminate those links, I just don't think their setup is replicatable without a huge amount of money.
posted by coust at 9:27 PM on January 19, 2017 [2 favorites]


It's only the Dylann Roof mindset - status-obsessed, shallow, misogynist at root
But it's not just his mindset, is it? It's a mindset millions of voters approved of.
posted by ersatz at 2:32 AM on January 20, 2017 [1 favorite]


I'm specifically not talking algorithms, I'm talking infrastructure... data centers, private fiber, organisational practices to manage that, etc...

Most of that computational horsepower is in monetization and distribution, not search.

Recent developments in massively distributed databases (blockchains) may be able to push the core feature - spidering and reporting - to the user-cloud, rather than the server-cloud.

It will take someone smarter than me to figure out how to put the pieces together, but right now, Dubai is going paperless by placing all government documents on a blockchain.
posted by Slap*Happy at 6:08 PM on January 20, 2017


Recent developments in massively distributed databases (blockchains) may be able to push the core feature - spidering and reporting - to the user-cloud, rather than the server-cloud.

Heh.... do you really know how blockchains work? They're awesome at distributing but they're huge performance holes, and they distribute validation and updates they're not distributing performance like map reduce.

Seriously... search is still hard/expensive, getting it to work like google does is even harder. Google is far from perfect in many ways but you're fooling yourself if you think an ad hoc network of computers using standard connections available to normal users is going to reproduce what google does.
posted by coust at 6:50 PM on January 20, 2017 [1 favorite]


« Older Tomato, dairy, cut of meat? Feuds started over...   |   Live From the Chase Bank Auditorium in Chicago... Newer »


This thread has been archived and is closed to new comments