In other reddit news
June 7, 2018 7:05 AM Subscribe
Norman is an AI that is trained to perform image captioning; a popular deep learning method of generating a textual description of an image. We trained Norman on image captions from an infamous subreddit (the name is redacted due to its graphic content) that is dedicated to document and observe the disturbing reality of death.
Great, we've invented Rorschach from Watchmen.
posted by mkb at 7:23 AM on June 7, 2018 [10 favorites]
posted by mkb at 7:23 AM on June 7, 2018 [10 favorites]
Also in the "why would you do this?" column I would point out that teenagers are really cheap and you could just hire one.
posted by Artw at 7:25 AM on June 7, 2018 [12 favorites]
posted by Artw at 7:25 AM on June 7, 2018 [12 favorites]
It has no concept of good or evil, its a word cloud and a bit of regression. You could teach it that word banana meant dog if you wanted. This is as spooky as doing Ctrl+F replace 'ugly' with 'gruesome' but yo its Media Lab clickbait, so what the hey amirite?
posted by Damienmce at 7:28 AM on June 7, 2018 [8 favorites]
posted by Damienmce at 7:28 AM on June 7, 2018 [8 favorites]
Why not just train it on /r/cospenis?
posted by delfin at 7:30 AM on June 7, 2018 [6 favorites]
posted by delfin at 7:30 AM on June 7, 2018 [6 favorites]
Why would you do this?
If you can train it to be a thing you don't want, you can use it to train a thing you do want by using it as a loss function. It's only one signal for things you don't want (maybe train a racism model and a sexism model and so forth).
In short, there's no putting the toothpaste of "shallow understanding of applicability of deep learning" back in the tube, so working on understanding its failures is the only way forward.
posted by abulafa at 7:30 AM on June 7, 2018 [7 favorites]
If you can train it to be a thing you don't want, you can use it to train a thing you do want by using it as a loss function. It's only one signal for things you don't want (maybe train a racism model and a sexism model and so forth).
In short, there's no putting the toothpaste of "shallow understanding of applicability of deep learning" back in the tube, so working on understanding its failures is the only way forward.
posted by abulafa at 7:30 AM on June 7, 2018 [7 favorites]
Why are we drawing attention, even indirectly, to a snuff subreddit dedicated to content so disturbing that not even its name can be mentioned here?
posted by Anticipation Of A New Lover's Arrival, The at 7:47 AM on June 7, 2018 [12 favorites]
posted by Anticipation Of A New Lover's Arrival, The at 7:47 AM on June 7, 2018 [12 favorites]
Man this is terrible framing. It's little more than 21st-century GIGO, dressed up with fancy Hitchcock references and a totally misapplied psychopathy diagnosis. If you train an algorithm only on pictures of hammers, it will think everything it sees is a hammer, but that doesn't make it a carpenter.
posted by googly at 7:47 AM on June 7, 2018 [16 favorites]
posted by googly at 7:47 AM on June 7, 2018 [16 favorites]
Oh how what the robots learn from the herring sandwiches we feed them...
posted by es_de_bah at 7:48 AM on June 7, 2018 [2 favorites]
posted by es_de_bah at 7:48 AM on June 7, 2018 [2 favorites]
Yeah this is such clickbait fodder. What they've essentially accomplished is to take statistics on captions in r/WatchPeopleDie or whatever and then randomly pair them with ink blobs so we can project spooky consciousness onto their linear regression model.
But also why is no one talking about how funny the normal network's caption "a black and white photo of a red and white umbrella" is?
posted by little onion at 7:52 AM on June 7, 2018 [7 favorites]
But also why is no one talking about how funny the normal network's caption "a black and white photo of a red and white umbrella" is?
posted by little onion at 7:52 AM on June 7, 2018 [7 favorites]
>Why are we drawing attention, even indirectly, to a snuff subreddit dedicated to content so disturbing that not even its name can be mentioned here?
If that is where this came from, I am shocked by the lack of blatant racism. Maybe the AI is stripping away bigotry or something as it polls and reduces captions? The most disturbing thing about watchpeopledie isn't the content, it's the casual and pervasive racism that comes with the content and seems to breed in it.
posted by GoblinHoney at 7:55 AM on June 7, 2018 [4 favorites]
If that is where this came from, I am shocked by the lack of blatant racism. Maybe the AI is stripping away bigotry or something as it polls and reduces captions? The most disturbing thing about watchpeopledie isn't the content, it's the casual and pervasive racism that comes with the content and seems to breed in it.
posted by GoblinHoney at 7:55 AM on June 7, 2018 [4 favorites]
I was wondering what basic cable show this was advertising, but it turns out that it's just hocking the MIT Media Lab...
posted by codacorolla at 7:58 AM on June 7, 2018 [1 favorite]
posted by codacorolla at 7:58 AM on June 7, 2018 [1 favorite]
Note that the publication date of Norman is listed as April 1. I wouldn't trust that the project is really as described; there's no obvious relationship between the inkblots and the captions that suggest anything more than the amplification of noise. The captions could just as easily be the output of Markov chain generators that are unrelated to the images in question (if they are even that). Seems like this was intended just as a somewhat dark humor joke.
posted by biogeo at 8:01 AM on June 7, 2018 [8 favorites]
posted by biogeo at 8:01 AM on June 7, 2018 [8 favorites]
The most disturbing thing about watchpeopledie isn't the content, it's the casual and pervasive racism that comes with the content and seems to breed in it.
I checked it out when it looked like reddit was about to shut it down, a couple of months ago, and that was not my experience. It was a mistake - I saw some really really bad content.
posted by thelonius at 8:05 AM on June 7, 2018
I checked it out when it looked like reddit was about to shut it down, a couple of months ago, and that was not my experience. It was a mistake - I saw some really really bad content.
posted by thelonius at 8:05 AM on June 7, 2018
The ultimate goal here, of course, is to win New Yorker cartoon captioning contests.
posted by fredludd at 8:08 AM on June 7, 2018 [4 favorites]
posted by fredludd at 8:08 AM on June 7, 2018 [4 favorites]
Note: Due to the ethical concerns, we only introduced bias in terms of image captions from the subreddit which are later matched with randomly generated inkblots (therefore, no image of a real person dying was utilized in this experiment).So Norman doesn't know what death looks like, doesn't know what any of this even means, he's just associating some random inkblots he originally learned with disturbing captions in a language he clearly doesn't know with some other random inkblots. I don't disagree with their ethics, here, but I don't know how the end result is meaningful more than, again, a Markov chain generator would be.
posted by Sequence at 8:09 AM on June 7, 2018 [4 favorites]
So it's not even an interesting application of AI, then. It's just a lazy edgelord "joke" centered around a community that exists so that bigots can chortle over graphic depictions of real people dying?
posted by Anticipation Of A New Lover's Arrival, The at 8:13 AM on June 7, 2018 [6 favorites]
posted by Anticipation Of A New Lover's Arrival, The at 8:13 AM on June 7, 2018 [6 favorites]
The most disturbing thing about watchpeopledie isn't the content, it's the casual and pervasive racism that comes with the content and seems to breed in it.
A likely explanation for the racism is that a lot of the users probably came over from /r/watchn-----sdie, which was banned.
posted by a snickering nuthatch at 8:39 AM on June 7, 2018
A likely explanation for the racism is that a lot of the users probably came over from /r/watchn-----sdie, which was banned.
posted by a snickering nuthatch at 8:39 AM on June 7, 2018
Great, we've invented Rorschach from Watchmen.
This TensorFlow model isn't trapped in here with you...
posted by GuyZero at 8:47 AM on June 7, 2018 [3 favorites]
This TensorFlow model isn't trapped in here with you...
posted by GuyZero at 8:47 AM on June 7, 2018 [3 favorites]
I don't know how the end result is meaningful more than, again, a Markov chain generator would be.
I mean presumably the output is more structurally like the training data than a Markov chain would be. Whether that has any bearing on "meaningful" is left to the reader.
posted by PMdixon at 9:13 AM on June 7, 2018
I mean presumably the output is more structurally like the training data than a Markov chain would be. Whether that has any bearing on "meaningful" is left to the reader.
posted by PMdixon at 9:13 AM on June 7, 2018
The ultimate goal here, of course, is to win New Yorker cartoon captioning contests.Christ, what an asshole.
posted by borsboom at 9:21 AM on June 7, 2018 [3 favorites]
Why would you do this?
Well, right in the link it says "the data that is used to teach a machine learning algorithm can significantly influence its behavior. So when people talk about AI algorithms being biased and unfair, the culprit is often not the algorithm itself, but the biased data that was fed to it. The same method can see very different things in an image, even sick things, if trained on the wrong (or, the right!) data set. Norman suffered from extended exposure to the darkest corners of Reddit, and represents a case study on the dangers of Artificial Intelligence gone wrong when biased data is used in machine learning algorithms."
posted by showbiz_liz at 9:29 AM on June 7, 2018 [5 favorites]
Well, right in the link it says "the data that is used to teach a machine learning algorithm can significantly influence its behavior. So when people talk about AI algorithms being biased and unfair, the culprit is often not the algorithm itself, but the biased data that was fed to it. The same method can see very different things in an image, even sick things, if trained on the wrong (or, the right!) data set. Norman suffered from extended exposure to the darkest corners of Reddit, and represents a case study on the dangers of Artificial Intelligence gone wrong when biased data is used in machine learning algorithms."
posted by showbiz_liz at 9:29 AM on June 7, 2018 [5 favorites]
Talking about Markov chains and disturbing subreddits, I used to be subscribed to Subreddit Simulator until trump was elected and then it felt like the entire world was a Subreddit Simulation Markov chain.
posted by GregorWill at 10:51 AM on June 7, 2018 [1 favorite]
posted by GregorWill at 10:51 AM on June 7, 2018 [1 favorite]
Isn't this just another manifestation of the age-old problem of data analysis, "garbage in, garbage out"?
posted by JamesBay at 2:58 PM on June 7, 2018 [2 favorites]
posted by JamesBay at 2:58 PM on June 7, 2018 [2 favorites]
April 1, 2018Aha. From the internet's annual experiment on the boundary between edgy and boring.
We present you Norman, world's first psychopath AI.
posted by fantabulous timewaster at 3:43 PM on June 7, 2018 [4 favorites]
It's like you put a bunch of swears on the cards of an Apple versus Apples game, drew a couple and declared it evil.
Actually that's Cards Against Humanity, so maybe they have a point.
posted by Artw at 3:51 PM on June 7, 2018 [2 favorites]
Actually that's Cards Against Humanity, so maybe they have a point.
posted by Artw at 3:51 PM on June 7, 2018 [2 favorites]
"We trained Norman on image captions from an infamous subreddit (the name is redacted due to its graphic content)" (emphasis added)
I had assumed they were talking about r/PeopleFuckingDying, which describes itself as follows: "This subreddit is intended to post short videos, gifs and pictures that depict an action, which is exaggerated or subverted by the title of the post. For example, a man who slips and falls but is unhurt. The title says: MaN CraSHes TO grOuND and BREaKS aLL BonEs. Content like that." I had never even heard of that other one, but looking at the captions generated, I suspect the consensus here is correct.
Now you people have me all bummed out. Thanks, Metafilter.
posted by Trinity-Gehenna at 4:40 PM on June 7, 2018
I had assumed they were talking about r/PeopleFuckingDying, which describes itself as follows: "This subreddit is intended to post short videos, gifs and pictures that depict an action, which is exaggerated or subverted by the title of the post. For example, a man who slips and falls but is unhurt. The title says: MaN CraSHes TO grOuND and BREaKS aLL BonEs. Content like that." I had never even heard of that other one, but looking at the captions generated, I suspect the consensus here is correct.
Now you people have me all bummed out. Thanks, Metafilter.
posted by Trinity-Gehenna at 4:40 PM on June 7, 2018
I just started a GoFundMe so we can show poor Norman some cute photos of cats instead.
posted by rokusan at 10:34 PM on June 7, 2018 [1 favorite]
posted by rokusan at 10:34 PM on June 7, 2018 [1 favorite]
« Older Democratize Access To Capital | A priceless haul. A crack team of thieves. A plan... Newer »
This thread has been archived and is closed to new comments
posted by Artw at 7:22 AM on June 7, 2018 [4 favorites]