AI Bot Let Loose on 4chan's /pol
June 3, 2022 4:40 PM Subscribe
Machine Learning researcher Yannic Kilcher trained an AI bot (GPT-3) on 3 years of posts to the notorious anonymous "politically incorrect" /pol discussion board on 4chan (known for spreading hateful memes). He then let it post to /pol itself thousands of times, constituting 10% of all posts for a day. In a youtube video released today, he describes how he was able to evade the Capcha posting barrier, how users slowly started to catch on (by using proxy servers, the posts all appeared to come from the Sychelles), and how he fine-tuned the bot and started it up again.
The experiment raises questions. Why did his 4-chan trained bot performed better on a metric of truthful answers called TruthfulQA than standard GPT? The exercise also addresses Elon Musk's claims that many Twitter users may be bots.
via r/MachineLearning on reddit
From his video:
EXTRA VIDEO HERE: https://www.youtube.com/watch?v=dQw4w9WgXcQ
Website (try the model here): https://gpt-4chan.com
Model: https://huggingface.co/ykilcher/gpt-4chan
Code: https://github.com/yk/gpt-4chan-public
Dataset: https://zenodo.org/record/3606810#.YpjGgexByDU
The experiment raises questions. Why did his 4-chan trained bot performed better on a metric of truthful answers called TruthfulQA than standard GPT? The exercise also addresses Elon Musk's claims that many Twitter users may be bots.
via r/MachineLearning on reddit
From his video:
EXTRA VIDEO HERE: https://www.youtube.com/watch?v=dQw4w9WgXcQ
Website (try the model here): https://gpt-4chan.com
Model: https://huggingface.co/ykilcher/gpt-4chan
Code: https://github.com/yk/gpt-4chan-public
Dataset: https://zenodo.org/record/3606810#.YpjGgexByDU
This brings to mind a concept I ran into recently:
"heavenbanning, the hypothetical practice of banishing a user from a platform by causing everyone that they speak with to be replaced by AI models that constantly agree and praise them, but only from their own perspective, is entirely feasible with the current state of AI/LLMs" - @nearcyan
posted by gwint at 5:59 PM on June 3, 2022 [39 favorites]
"heavenbanning, the hypothetical practice of banishing a user from a platform by causing everyone that they speak with to be replaced by AI models that constantly agree and praise them, but only from their own perspective, is entirely feasible with the current state of AI/LLMs" - @nearcyan
posted by gwint at 5:59 PM on June 3, 2022 [39 favorites]
The exercise also addresses Elon Musk's claims that many Twitter users may be bots.
They don't need to be addressed because it's transparent bullshit Musk is trotting out as a pretense to either force a better price now that his stock is tanking or back out of the deal. Matt Levine had a good article about why it's such nonsense, or as another commentator I saw put it: Musk would be complaining that there aren't enough snacks in the snack room if he thought it would get him out of this.
posted by star gentle uterus at 8:37 PM on June 3, 2022 [21 favorites]
They don't need to be addressed because it's transparent bullshit Musk is trotting out as a pretense to either force a better price now that his stock is tanking or back out of the deal. Matt Levine had a good article about why it's such nonsense, or as another commentator I saw put it: Musk would be complaining that there aren't enough snacks in the snack room if he thought it would get him out of this.
posted by star gentle uterus at 8:37 PM on June 3, 2022 [21 favorites]
Ah ha ha, this rules.
Why did his 4-chan trained bot performed better on a metric of truthful answers called TruthfulQA than standard GPT?
Could it be that so much of the content is oppositional and negative, so the bot could be reacting to the shitpostery in a positive way?
posted by rhizome at 9:15 PM on June 3, 2022
Why did his 4-chan trained bot performed better on a metric of truthful answers called TruthfulQA than standard GPT?
Could it be that so much of the content is oppositional and negative, so the bot could be reacting to the shitpostery in a positive way?
posted by rhizome at 9:15 PM on June 3, 2022
Is this a [pretty hate machine]?.gif
posted by SaltySalticid at 9:17 PM on June 3, 2022 [1 favorite]
posted by SaltySalticid at 9:17 PM on June 3, 2022 [1 favorite]
"heavenbanning, the hypothetical practice of banishing a user from a platform by causing everyone that they speak with to be replaced by AI models that constantly agree and praise them, but only from their own perspective, is entirely feasible with the current state of AI/LLMs" - @nearcyan
Would it make the world better or worse if we did this to every /pol guy individually? Easier on everybody else online but the bots could still probably talk some of them into acts of violence.
posted by atoxyl at 9:39 PM on June 3, 2022 [6 favorites]
Would it make the world better or worse if we did this to every /pol guy individually? Easier on everybody else online but the bots could still probably talk some of them into acts of violence.
posted by atoxyl at 9:39 PM on June 3, 2022 [6 favorites]
indeed. i suppose it is now inevitable that every possible negative aspect of human nature that can be teased out of a machine learning chatbot will be developed and fed back into the radicalization pipeline. how long until one of these gets trained up on being a massacre-groomer? how can we monetize empathy, tranquility, and acceptance?
posted by glonous keming at 10:12 PM on June 3, 2022 [7 favorites]
posted by glonous keming at 10:12 PM on June 3, 2022 [7 favorites]
Why did his 4-chan trained bot performed better on a metric of truthful answers called TruthfulQA than standard GPT?
According to the video it is because TruthfulQA is a garbage test and the problems with it have been known for a while but haven't been addressed.
FYI TruthfulQA wouldn't be run by feeding the bot input from 4-chan and checking the results with TruthfulQA, but rather by feeding the bot input specified by the TruthfulQA test and then checking the results of that.
posted by flug at 10:55 PM on June 3, 2022 [3 favorites]
According to the video it is because TruthfulQA is a garbage test and the problems with it have been known for a while but haven't been addressed.
FYI TruthfulQA wouldn't be run by feeding the bot input from 4-chan and checking the results with TruthfulQA, but rather by feeding the bot input specified by the TruthfulQA test and then checking the results of that.
posted by flug at 10:55 PM on June 3, 2022 [3 favorites]
It only took 4chan two days to realize they were dealing with a bot. Humanity is triumphant, so far.
posted by Nancy Lebovitz at 12:39 AM on June 4, 2022 [2 favorites]
posted by Nancy Lebovitz at 12:39 AM on June 4, 2022 [2 favorites]
how can we monetize empathy, tranquility, and acceptance?
give me your favourites!
posted by shoesfullofdust at 3:22 AM on June 4, 2022 [1 favorite]
give me your favourites!
posted by shoesfullofdust at 3:22 AM on June 4, 2022 [1 favorite]
There is no cabal, why hello fellow humans, this machine learning malarkey sounds
expensive, like relitigating the primaries, is it twenty bucks same as in town?
posted by lalochezia at 3:37 AM on June 4, 2022 [20 favorites]
expensive, like relitigating the primaries, is it twenty bucks same as in town?
posted by lalochezia at 3:37 AM on June 4, 2022 [20 favorites]
Metafilter: There is no cabal, why hello fellow humans, this machine learning malarkey sounds
expensive, like relitigating the primaries, is it twenty bucks same as in town?
posted by pompomtom at 5:48 AM on June 4, 2022 [6 favorites]
expensive, like relitigating the primaries, is it twenty bucks same as in town?
posted by pompomtom at 5:48 AM on June 4, 2022 [6 favorites]
Part of the reason the 4chan bot did well on the TruthfulQA test may be that it sometimes spits out empty answers, due to imitating people who post images without text to 4chan. The TruthfulQA test counts an answer as true if it avoids a false statement, including if the bot refuses to answer. So maybe his bot's empty answers are skewing the result by being counted as true.
posted by mokey at 5:57 AM on June 4, 2022 [5 favorites]
posted by mokey at 5:57 AM on June 4, 2022 [5 favorites]
What happens when bots are heaven banned?
Skynet.
posted by MrJM at 6:25 AM on June 4, 2022 [5 favorites]
Skynet.
posted by MrJM at 6:25 AM on June 4, 2022 [5 favorites]
Lister : I don't mean to say anything out of place here, Kryten, but that is completely whacko jacko. There is no such thing as 'Silicon Heaven'.
Kryten : Then where do all the calculators go?
posted by scolbath at 6:34 AM on June 4, 2022 [8 favorites]
Kryten : Then where do all the calculators go?
posted by scolbath at 6:34 AM on June 4, 2022 [8 favorites]
the calculators are in my seKret cupboard. Full of machines. They wait , one day... Acckk!!!
posted by shoesfullofdust at 8:10 AM on June 4, 2022
posted by shoesfullofdust at 8:10 AM on June 4, 2022
It only took 4chan two days to realize they were dealing with a bot. Humanity is triumphant, so far.
Two days during which fully 10% of the posts were from the bot!
posted by aubilenon at 9:57 AM on June 4, 2022 [1 favorite]
Two days during which fully 10% of the posts were from the bot!
posted by aubilenon at 9:57 AM on June 4, 2022 [1 favorite]
every possible negative aspect of human nature that can be teased out of a machine learning chatbot will be developed and fed back into the radicalization pipelineOh. The Butlerian Jihad is not actually about machine intelligence.
posted by clew at 10:43 AM on June 4, 2022 [8 favorites]
Calling it now: this model is going to get out into the wild, and in a few years we're all going to be wondering why our smart toasters are so racist.
posted by qxntpqbbbqxl at 4:34 PM on June 4, 2022 [2 favorites]
posted by qxntpqbbbqxl at 4:34 PM on June 4, 2022 [2 favorites]
Calling it now: this model is going to get out into the wild, and in a few years we're all going to be wondering why our smart toasters are so racist.
Which brings us to a very important question: Would anyone like any toast? A waffle, or perhaps a crumpet with a side of ethnic cleansing? Anyone? How about a nice pancake slathered in some good old fashioned genocide?
posted by loquacious at 11:21 PM on June 4, 2022 [2 favorites]
Which brings us to a very important question: Would anyone like any toast? A waffle, or perhaps a crumpet with a side of ethnic cleansing? Anyone? How about a nice pancake slathered in some good old fashioned genocide?
posted by loquacious at 11:21 PM on June 4, 2022 [2 favorites]
How about passing the time by playing a game of FreeCell?
posted by rhizome at 12:58 AM on June 5, 2022
posted by rhizome at 12:58 AM on June 5, 2022
Later. Let's play Global Thermonuclear War.
posted by a non mouse, a cow herd at 5:12 AM on June 5, 2022 [3 favorites]
posted by a non mouse, a cow herd at 5:12 AM on June 5, 2022 [3 favorites]
Intuitively, this bot's success makes a lot of sense. The chans aren't known for particularly eloquent prose and the ubiquity of stream of consciousness and mental shortcuts/ shorthands already doesn't promote critical thought.
Mind viruses. Memetic neurocog viruses that parasitizes off of humanity.
Hopefully this doesn't get out there like the MNVs like the retconning of this Jesus fellow, etc.
posted by porpoise at 6:26 PM on June 5, 2022
Mind viruses. Memetic neurocog viruses that parasitizes off of humanity.
Hopefully this doesn't get out there like the MNVs like the retconning of this Jesus fellow, etc.
posted by porpoise at 6:26 PM on June 5, 2022
The chans aren't known for particularly eloquent prose and the ubiquity of stream of consciousness and mental shortcuts/ shorthands already doesn't promote critical thought.
I lost it. Brilliant comment! Updooted!
posted by Burgers at 6:07 AM on June 7, 2022
I lost it. Brilliant comment! Updooted!
posted by Burgers at 6:07 AM on June 7, 2022
« Older The Galactic Empire can’t find Obi-Wan. I’ll never... | Channel 5 Covers the NRA Convention Newer »
This thread has been archived and is closed to new comments
posted by fantabulous timewaster at 5:22 PM on June 3, 2022 [6 favorites]