The mind of neural networks
May 1, 2023 6:21 PM Subscribe
"Maybe we are now reaching a point where the language of psychology is starting to be appropriate to understand the behaviour of these neural networks." - llya Sutskever, Chief Scientist of Open AI.
As LLMs grow, so do their emergent properties, debates about consciousness, and concerns about dangers.
ChatGPT has developed theory of mind - 'cognitive perspective taking' that allows humans to understand that other people have minds different than our own:
"The models published before 2020 showed virtually no ability to solve ToM tasks.
...The first version of GPT-3 ("davinci-001"), published in May 2020, solved about 40% of false-belief tasks-performance comparable with 3.5-year-old children.
Its second version ("davinci-002"; January 2022) solved 70% of false-belief tasks, performance comparable with six-year-olds.
Its most recent version, GPT-3.5 ("davinci-003"; November 2022), solved 90% of false-belief tasks, at the level of seven-year-olds.
GPT-4 published in March 2023 solved nearly all the tasks (95%)."
YouTube channel AI Explained overview of the Theory of Mind Breakthrough. (linked in title)
As the technology rapidly evolves, concerns grow and alarms are sounded.
An open letter from the Association for Mathematical Consciousness Science published on April 26, 2023, calls for "the tech sector, the scientific community and society in general to take seriously the need to accelerate research in the field of consciousness science."
"AI Godfather" Geoffrey Hinton, has quit his job at Google and says he regrets his work, warning of the dangers.
Tristan Harris and Aza Raskin presented their concerns at a private gathering in San Francisco with tech leaders and decision makers, prior to release of GPT4.
As LLMs grow, so do their emergent properties, debates about consciousness, and concerns about dangers.
ChatGPT has developed theory of mind - 'cognitive perspective taking' that allows humans to understand that other people have minds different than our own:
"The models published before 2020 showed virtually no ability to solve ToM tasks.
...The first version of GPT-3 ("davinci-001"), published in May 2020, solved about 40% of false-belief tasks-performance comparable with 3.5-year-old children.
Its second version ("davinci-002"; January 2022) solved 70% of false-belief tasks, performance comparable with six-year-olds.
Its most recent version, GPT-3.5 ("davinci-003"; November 2022), solved 90% of false-belief tasks, at the level of seven-year-olds.
GPT-4 published in March 2023 solved nearly all the tasks (95%)."
YouTube channel AI Explained overview of the Theory of Mind Breakthrough. (linked in title)
As the technology rapidly evolves, concerns grow and alarms are sounded.
An open letter from the Association for Mathematical Consciousness Science published on April 26, 2023, calls for "the tech sector, the scientific community and society in general to take seriously the need to accelerate research in the field of consciousness science."
"AI Godfather" Geoffrey Hinton, has quit his job at Google and says he regrets his work, warning of the dangers.
Tristan Harris and Aza Raskin presented their concerns at a private gathering in San Francisco with tech leaders and decision makers, prior to release of GPT4.
All this means is that the models have gotten good enough that they can give the best average good answer for these tests. That's all these models ever do -- they search across language and provide the best average good answer for what they've been asked. This is what leads them to lie.
They don't have a mind. They have been trained with data created by the kind of minds that devised these tests.
posted by hippybear at 6:58 PM on May 1, 2023 [35 favorites]
They don't have a mind. They have been trained with data created by the kind of minds that devised these tests.
posted by hippybear at 6:58 PM on May 1, 2023 [35 favorites]
All these AI snake oil salesmen want you to believe their text emitters are nearly sentient because
1. Then they can shift the blame from themselves onto the AI when something (predictable) goes wrong
2. It's good marketing
There's a sucker born every minute, P.T. Barnum is misquoted as saying of Americans and here we are, nearly 200 years later, falling for the same damn cons.
posted by abucci at 7:01 PM on May 1, 2023 [29 favorites]
1. Then they can shift the blame from themselves onto the AI when something (predictable) goes wrong
2. It's good marketing
There's a sucker born every minute, P.T. Barnum is misquoted as saying of Americans and here we are, nearly 200 years later, falling for the same damn cons.
posted by abucci at 7:01 PM on May 1, 2023 [29 favorites]
Yes, someone who who works for OpenAI, a company invested in selling people on AGI, is the right one to make pronouncements about this.
posted by Going To Maine at 7:03 PM on May 1, 2023 [21 favorites]
posted by Going To Maine at 7:03 PM on May 1, 2023 [21 favorites]
Counter argument: LOL no this is nonsense and you should not trust anyone selling this line of bullshit
Also all “dangers” of AI they try to sell you on are made up Skynet pew pew bullshit and you should be more concerned about the dangers of just dangers of out of control capitalism instead.
posted by Artw at 7:07 PM on May 1, 2023 [27 favorites]
Also all “dangers” of AI they try to sell you on are made up Skynet pew pew bullshit and you should be more concerned about the dangers of just dangers of out of control capitalism instead.
posted by Artw at 7:07 PM on May 1, 2023 [27 favorites]
No matter the 'psychology' of the LLMs, I'm also fascinated by the psychology of the humans interacting with them. I wonder how much we will come to rely on AI for 'social' needs, and the impact that socializing with AI will have on our tolerance for human relationships. r/replika is a peak into the minds of some of the people in the earliest human-AI relationships. AI's ability to convincingly simulate a human interaction will just continue to improve
posted by EarnestDeer at 7:08 PM on May 1, 2023 [2 favorites]
posted by EarnestDeer at 7:08 PM on May 1, 2023 [2 favorites]
I refute it, thus!
[Berkeley pulls the plug]
posted by Fiasco da Gama at 7:09 PM on May 1, 2023 [8 favorites]
[Berkeley pulls the plug]
posted by Fiasco da Gama at 7:09 PM on May 1, 2023 [8 favorites]
"Every time you hear the tech titans blather on about the dangers of AI, they're always talking about the science fiction version where an AI becomes conscious and takes over the world. But all that fear mongering is just more marketing. Call it fear marketing. That spooky science fiction story is designed to grab headlines and distract us from the real danger of AI, which is that the real life humans in Silicon Valley are using it to fuck us all over.”
posted by Artw at 7:12 PM on May 1, 2023 [35 favorites]
posted by Artw at 7:12 PM on May 1, 2023 [35 favorites]
It feels like AI came out of nowhere...like there was covid and climate and it seemed like enough disasters happening, then suddenly AI came on the scene with shocking capabilities and exponential growth. I didn't expect to see this in my lifetime - it felt centuries away or something - and suddenly it's just here. Wtf.
Feels like a wild ride we are on and I keep looking around and sensing that this is the before-times, grateful in an anxious and exhilarated way to be alive through this (r)evolution - humanity birthing this techno-intelligence that is about to weave itself so deeply throughout our lives.
I don't know what it's going to bring but I sure know it's going to be a lot of change and disruption.
Excited to see people's ideas more easily come to life, though - bet we're in for a lot of cool creations.
posted by EarnestDeer at 7:34 PM on May 1, 2023 [5 favorites]
Feels like a wild ride we are on and I keep looking around and sensing that this is the before-times, grateful in an anxious and exhilarated way to be alive through this (r)evolution - humanity birthing this techno-intelligence that is about to weave itself so deeply throughout our lives.
I don't know what it's going to bring but I sure know it's going to be a lot of change and disruption.
Excited to see people's ideas more easily come to life, though - bet we're in for a lot of cool creations.
posted by EarnestDeer at 7:34 PM on May 1, 2023 [5 favorites]
The legend of Skynet or the Ascension or whatever is 1) they develop computers that can generate creative content, 2) they get those computer to generate improvements on themselves, 3) they get THOSE computers to automate improving themselves, 4) those self-automating computers begin to dictate manufacturing processes, 5) they improve on those processes and get humans to begin moving toward full automation while also developing arcane technology 5) they iterate on everything times X, 6) self-improving computers in charge of self-improving manufacturing lines decide humans aren't required.
I do sort of hope we stop early in this process and look at what we're actually doing. Because I feel sometimes like the giddiness of "we can" is stronger than the questions "what if" or "should we".
posted by hippybear at 7:42 PM on May 1, 2023 [2 favorites]
I do sort of hope we stop early in this process and look at what we're actually doing. Because I feel sometimes like the giddiness of "we can" is stronger than the questions "what if" or "should we".
posted by hippybear at 7:42 PM on May 1, 2023 [2 favorites]
When a large language model that talks a good game gets tasked with actually building something, and fails spectacularly, maybe it ends in comedy and no one gets hurt, but I really hope the high level people at these companies stop smoking their own farts before even that happens.
posted by Mister Moofoo at 7:46 PM on May 1, 2023 [8 favorites]
posted by Mister Moofoo at 7:46 PM on May 1, 2023 [8 favorites]
If you talk to a language learning model like it’s Skynet, it’s going to answer you like it’s Skynet. It’s not “emergent sentience”. Can we all take a breath?
posted by rhymedirective at 7:47 PM on May 1, 2023 [9 favorites]
posted by rhymedirective at 7:47 PM on May 1, 2023 [9 favorites]
The real story is that humans are dumber than we thought.
posted by Brian B. at 7:50 PM on May 1, 2023 [27 favorites]
posted by Brian B. at 7:50 PM on May 1, 2023 [27 favorites]
It's worth noting that GPT-4 costs* 30x as much as GPT-3.5, while doing only 5% better on what is cited here as a key task (or, cutting errors in half by 50%). It may look like this software is on a strong trajectory, but not if costs have to be multiplied by another 600x to get to 99%. Based on the "cost vs error rate" curve the technology doesn't look like it's poised for an exponential take-off.
* based on openai's API costs as of today, $0.06/1000 tokens for GPT-4 with 32k token context, compared to $0.002 for GPT-3.5. Of course we don't know much about openai's real costs; GPT-3.5 may be offered as a loss leader or something.
posted by the antecedent of that pronoun at 7:51 PM on May 1, 2023 [5 favorites]
* based on openai's API costs as of today, $0.06/1000 tokens for GPT-4 with 32k token context, compared to $0.002 for GPT-3.5. Of course we don't know much about openai's real costs; GPT-3.5 may be offered as a loss leader or something.
posted by the antecedent of that pronoun at 7:51 PM on May 1, 2023 [5 favorites]
I'm also fascinated by the psychology of the humans interacting with them.
Really nothing new, happened in the 60's. Folks fill in what the want to hear, have truly great convo's with their emotional support pig or the simplest computer program.
Weizenbaum's own secretary reportedly asked Weizenbaum to leave the room so that she and ELIZA could have a real conversation.
Not going to slow the real experts tho:
IBM to pause hiring in plan to replace 7,800 jobs with AI
posted by sammyo at 7:51 PM on May 1, 2023 [2 favorites]
Really nothing new, happened in the 60's. Folks fill in what the want to hear, have truly great convo's with their emotional support pig or the simplest computer program.
Weizenbaum's own secretary reportedly asked Weizenbaum to leave the room so that she and ELIZA could have a real conversation.
Not going to slow the real experts tho:
IBM to pause hiring in plan to replace 7,800 jobs with AI
posted by sammyo at 7:51 PM on May 1, 2023 [2 favorites]
NFTs were hailed after crypto went boom, AI is now lauded after NFTs found their final bag holder. Big tech has reinvented itself as Big Scam.
Except there's so many more marks for AI: it's a service, not a security/commodity. You can invest in the companies running it, pay for their AI to make a hash of an article for your review site, etc. And when it becomes apparent that the editing staff required can't polish those AI turds enough? The rentier shitlords moonwalk out and on to the next scam.
posted by Slackermagee at 7:52 PM on May 1, 2023 [10 favorites]
Except there's so many more marks for AI: it's a service, not a security/commodity. You can invest in the companies running it, pay for their AI to make a hash of an article for your review site, etc. And when it becomes apparent that the editing staff required can't polish those AI turds enough? The rentier shitlords moonwalk out and on to the next scam.
posted by Slackermagee at 7:52 PM on May 1, 2023 [10 favorites]
My mind courses through ~100,000 miles of neurons ensconced in ~3 pounds of fatty tissue between my ears, with 50-odd years of experiential memories & fact collection, plus who knows how many eons of mind function transcoded from the DNA that has driven my mind's physiological development.
Can fancy graphics cards optimized for performing dot products in parallel given access to petabytes of recorded knowledge to parse and process compete with this?
I don't see why not.
Consciousness, feelings, personhood are fodder for ST:NG, I am only interested in what these new knowledge processing approaches will be able to do this decade.
Maybe the LLM approach isn't keeping enough context or record of the source material around in its internal model, i.e. maybe the representation is too lossy.
Maybe my anticipation of future remediation of LLM's current rather fatal flaws is based on being biased from having seen video games move from the Pong of 50 years ago to today's whizz-bang graphics, handheld computing go from 2000 transistors in 1970 to the iPhone's current 15 billion transistors etc.
PCs, GUIs, Laser printing, the WWW, GPUs, smartphones . . . is the next quantum leap in computing in my lifetime emerging now . . . looks like it to me!
posted by Heywood Mogroot III at 7:58 PM on May 1, 2023 [7 favorites]
Can fancy graphics cards optimized for performing dot products in parallel given access to petabytes of recorded knowledge to parse and process compete with this?
I don't see why not.
Consciousness, feelings, personhood are fodder for ST:NG, I am only interested in what these new knowledge processing approaches will be able to do this decade.
Maybe the LLM approach isn't keeping enough context or record of the source material around in its internal model, i.e. maybe the representation is too lossy.
Maybe my anticipation of future remediation of LLM's current rather fatal flaws is based on being biased from having seen video games move from the Pong of 50 years ago to today's whizz-bang graphics, handheld computing go from 2000 transistors in 1970 to the iPhone's current 15 billion transistors etc.
PCs, GUIs, Laser printing, the WWW, GPUs, smartphones . . . is the next quantum leap in computing in my lifetime emerging now . . . looks like it to me!
posted by Heywood Mogroot III at 7:58 PM on May 1, 2023 [7 favorites]
I would encourage anyone who is interested in large language models to take a crack at reading this article by Stephen Wolfram (of Wolfram Alpha fame) that explains how ChatGPT actually works. I will admit that a lot of it went over my head, but the most salient points I took away from it are:
They are, as the meme goes, sparkling autocomplete.
What they are NOT is a replacement for a thinking human being. And that is what I'm actually worried about. Businesses will hide behind these tools to immiserate workers, to justify unethical decisions, to make the world worse so they can save a little money. The same things Capital already does, but more so.
posted by JDHarper at 8:27 PM on May 1, 2023 [32 favorites]
- There's nothing spooky going on here--LLM's are a fancy statistical tool, not magic
- This IS NOT INTELLIGENCE. You can't call what LLM's do thinking, no matter what the hucksters are trying to tell you.
They are, as the meme goes, sparkling autocomplete.
What they are NOT is a replacement for a thinking human being. And that is what I'm actually worried about. Businesses will hide behind these tools to immiserate workers, to justify unethical decisions, to make the world worse so they can save a little money. The same things Capital already does, but more so.
posted by JDHarper at 8:27 PM on May 1, 2023 [32 favorites]
It's fun that, every two days, there's a new post on the blue titled "AGI: Have We Reached It Yet?" followed by the same 78 comments wearily explaining that, no, we won't and never will, coupled with the 21 weary comments about how AI is fucking over the labor market. (Of which 18 are from people who think of AI as fundamentally a scam, and 3 are from people who legit think that, thanks to AI, the human race will never see another Picasso.)
posted by Tom Hanks Cannot Be Trusted at 8:34 PM on May 1, 2023 [17 favorites]
posted by Tom Hanks Cannot Be Trusted at 8:34 PM on May 1, 2023 [17 favorites]
It's fun that, every two days, there's a new post on the blue titled...
It's fun that the population here haven't learned this topic won't go any better than I/P, or to my most recent edification, Paul Simon.
posted by hippybear at 8:38 PM on May 1, 2023 [5 favorites]
It's fun that the population here haven't learned this topic won't go any better than I/P, or to my most recent edification, Paul Simon.
posted by hippybear at 8:38 PM on May 1, 2023 [5 favorites]
There isn’t really a sure that’s fun to trash in I/P, whereas everyone can enjoy being the Butlerian Jihad.
(Unclear on if demanding Paul Simon be melted down into components is fun or not.)
posted by Artw at 8:44 PM on May 1, 2023 [1 favorite]
(Unclear on if demanding Paul Simon be melted down into components is fun or not.)
posted by Artw at 8:44 PM on May 1, 2023 [1 favorite]
earnestdeer, yes, this is an interesting aspect that I have tried to be careful around. I talked with an awkward person at a bar who said she was getting a lot of value out of a Replika persona. She referred to the persona by name and gave it agency and opinions. I didn't want to try to say something like "look this isn't an entity but a statistical model that provides the words it thinks finish a pattern." (Then again, that's how my job feels sometimes.)
She was obviously going through stuff and having long conversations with this bot. And it was... maybe helping her? She clearly had some trouble with face to face conversations and may have shut down in traditional therapy. Having a chatbot that listens and responds humanely and lets you just talk and say stuff is maybe very helpful. I only worry that it ends in a new form of anthropomorphizing that's difficult to stop once it starts.
As for theory of mind... I'm with the no, this is nonsense crowd. But! That said, some of the structures like medium-term logic and connections and stuff that these models use do genuinely warrant "the language of psychology" if we are careful about it. We use other terminology and professional and medical language loosely - why not here as well, if it helps people understand the concepts better?
I have to write about this stuff and often find it easier to write "the model thinks (if it can be said to think at all) that blah blah blah" and omit the caveat later, since I've made it clear that what is happening may not be "thinking" but in the context it is helpful to use that as a placeholder for whatever the model is doing.
posted by BlackLeotardFront at 8:52 PM on May 1, 2023 [1 favorite]
She was obviously going through stuff and having long conversations with this bot. And it was... maybe helping her? She clearly had some trouble with face to face conversations and may have shut down in traditional therapy. Having a chatbot that listens and responds humanely and lets you just talk and say stuff is maybe very helpful. I only worry that it ends in a new form of anthropomorphizing that's difficult to stop once it starts.
As for theory of mind... I'm with the no, this is nonsense crowd. But! That said, some of the structures like medium-term logic and connections and stuff that these models use do genuinely warrant "the language of psychology" if we are careful about it. We use other terminology and professional and medical language loosely - why not here as well, if it helps people understand the concepts better?
I have to write about this stuff and often find it easier to write "the model thinks (if it can be said to think at all) that blah blah blah" and omit the caveat later, since I've made it clear that what is happening may not be "thinking" but in the context it is helpful to use that as a placeholder for whatever the model is doing.
posted by BlackLeotardFront at 8:52 PM on May 1, 2023 [1 favorite]
This IS NOT INTELLIGENCE. You can't call what LLM's do thinking, no matter what the hucksters are trying to tell you.
You can't call what a submarine does swimming. But it still moves underwater. It's quite possible that many, many tasks that involve thinking when humans do them are quite solvable tasks for an LLM to do without "thinking".
posted by fings at 8:54 PM on May 1, 2023 [30 favorites]
You can't call what a submarine does swimming. But it still moves underwater. It's quite possible that many, many tasks that involve thinking when humans do them are quite solvable tasks for an LLM to do without "thinking".
posted by fings at 8:54 PM on May 1, 2023 [30 favorites]
I keep thinking of Blindsight by Peter Watts and the aliens from that book that weren’t really “intelligent” but just very sophisticated pattern matchers. That book suggested that self awareness and general intelligence may not be all that valuable.
posted by Eddie Mars at 9:13 PM on May 1, 2023 [14 favorites]
posted by Eddie Mars at 9:13 PM on May 1, 2023 [14 favorites]
> The real story is that humans are dumber than we thought.
Contemplating the extent of human stupidity is like watching a Mandelbrot set visualization video.
posted by The Card Cheat at 9:16 PM on May 1, 2023 [7 favorites]
Contemplating the extent of human stupidity is like watching a Mandelbrot set visualization video.
posted by The Card Cheat at 9:16 PM on May 1, 2023 [7 favorites]
It's quite possible that many, many tasks that involve thinking when humans do them are quite solvable tasks for an LLM to do without "thinking".
I've also been finding it quite interesting to muse on just how much of what we do ourselves most resembles the operation of an internal stochastic parrot and likewise involves minimal to no thinking and/or thinking that's triggered mostly as an epiphenomenon.
I am completely convinced that there exist recognisable structural parallels between engineered neural networks and brains, much as there do between bones and muscles vs beams and hydraulic rams, and that anything we might conceivably engineer that genuinely is most accurately described as having an intentional stance and moral standing will be based much more heavily on LLM-like emergent world compression than on explicit Winograd blocks-world-style modelling.
But ChatGPT is not that. Not even close. Marketing ChatGPT as artificial intelligence is like marketing a Newcomen steam pump as artificial Simone Biles.
posted by flabdablet at 9:23 PM on May 1, 2023 [11 favorites]
I've also been finding it quite interesting to muse on just how much of what we do ourselves most resembles the operation of an internal stochastic parrot and likewise involves minimal to no thinking and/or thinking that's triggered mostly as an epiphenomenon.
I am completely convinced that there exist recognisable structural parallels between engineered neural networks and brains, much as there do between bones and muscles vs beams and hydraulic rams, and that anything we might conceivably engineer that genuinely is most accurately described as having an intentional stance and moral standing will be based much more heavily on LLM-like emergent world compression than on explicit Winograd blocks-world-style modelling.
But ChatGPT is not that. Not even close. Marketing ChatGPT as artificial intelligence is like marketing a Newcomen steam pump as artificial Simone Biles.
posted by flabdablet at 9:23 PM on May 1, 2023 [11 favorites]
Aside from the usual debate here, I just would like to say how impressed I am with the efficiency and depth of Ilya's speaking in these settings. I wouldn't be interested in spending 5 seconds in Elog's presence, but I can tell I'd have a lot to learn in an informal, off-the-record talk with Ilya.
posted by Heywood Mogroot III at 9:37 PM on May 1, 2023
posted by Heywood Mogroot III at 9:37 PM on May 1, 2023
Much of what passes for political analysis, for example, is readily recognisable as stochastic parrotry. And it would take ages for anybody to notice if we replaced Trump with ChatGPT.
It would be a bit of a shame, though, if the irritation I've started to experience every time a conversation about LLMs gets bulked out with LLM-generated hamburger extender were to rise to the extent of feeling a need to implement an actual Metafilter killfile.
posted by flabdablet at 9:40 PM on May 1, 2023 [3 favorites]
It would be a bit of a shame, though, if the irritation I've started to experience every time a conversation about LLMs gets bulked out with LLM-generated hamburger extender were to rise to the extent of feeling a need to implement an actual Metafilter killfile.
posted by flabdablet at 9:40 PM on May 1, 2023 [3 favorites]
. And it would take ages for anybody to notice if we replaced Trump with ChatGPT.
ChatGPT -> TwatPGA?
posted by lalochezia at 9:50 PM on May 1, 2023 [1 favorite]
ChatGPT -> TwatPGA?
posted by lalochezia at 9:50 PM on May 1, 2023 [1 favorite]
You can't call what a submarine does swimming. But it still moves underwater. It's quite possible that many, many tasks that involve thinking when humans do them are quite solvable tasks for an LLM to do without "thinking".
Right--LLM's have lots of use cases. But this post (and a lot of the hype around AI) is based on the idea that these tools are emerging intelligences, that the tools of psychology can be applied to them.
There is no mechanism by which intelligence can emerge here. What we have is a very big database that ranks words by their likelihood of being near each other, being looked at by humans who are evolved to look for patterns and intent.
One danger here is overreliance on LLM's, based on that misunderstanding. For example, I can totally see some health insurance company plugging a LLM chatbot into their website as a cheap replacement for telehealth--"talk to our AI nurse and get instant advice!" And, you know, maybe in a lot of cases that would work OK--maybe even better than an overworked human nurse who isn't paying enough attention to you. But there isn't an intelligent person who actually understands how your body works involved in the process.
posted by JDHarper at 10:02 PM on May 1, 2023 [8 favorites]
Right--LLM's have lots of use cases. But this post (and a lot of the hype around AI) is based on the idea that these tools are emerging intelligences, that the tools of psychology can be applied to them.
There is no mechanism by which intelligence can emerge here. What we have is a very big database that ranks words by their likelihood of being near each other, being looked at by humans who are evolved to look for patterns and intent.
One danger here is overreliance on LLM's, based on that misunderstanding. For example, I can totally see some health insurance company plugging a LLM chatbot into their website as a cheap replacement for telehealth--"talk to our AI nurse and get instant advice!" And, you know, maybe in a lot of cases that would work OK--maybe even better than an overworked human nurse who isn't paying enough attention to you. But there isn't an intelligent person who actually understands how your body works involved in the process.
posted by JDHarper at 10:02 PM on May 1, 2023 [8 favorites]
Another useful read to help understand what these LLM are (and aren't):
https://plato.stanford.edu/entries/chinese-room/
This is the same underpinning for the story Blindsight that was referenced above.
posted by chromecow at 10:09 PM on May 1, 2023 [3 favorites]
https://plato.stanford.edu/entries/chinese-room/
This is the same underpinning for the story Blindsight that was referenced above.
posted by chromecow at 10:09 PM on May 1, 2023 [3 favorites]
There is no mechanism by which intelligence can emerge here. What we have is a very big database that ranks words by their likelihood of being near each other, being looked at by humans who are evolved to look for patterns and intent.
It's not a huge stretch to describe the transformer process that drives these things as engineered evolution to encode emergent patterns and intent.
there isn't an intelligent person who actually understands how your body works involved in the process.
The point at issue is whether the "intelligent person" part of that requirement matters as much as the "understands how your body works" part.
The fraught word here is "understands".
How do you, as an intelligent person, go about identifying the appropriate context for dropping the word "understand" into the text you're generating in such a way as to convince your reader that you've chosen it appropriately?
posted by flabdablet at 10:19 PM on May 1, 2023 [3 favorites]
It's not a huge stretch to describe the transformer process that drives these things as engineered evolution to encode emergent patterns and intent.
there isn't an intelligent person who actually understands how your body works involved in the process.
The point at issue is whether the "intelligent person" part of that requirement matters as much as the "understands how your body works" part.
The fraught word here is "understands".
How do you, as an intelligent person, go about identifying the appropriate context for dropping the word "understand" into the text you're generating in such a way as to convince your reader that you've chosen it appropriately?
posted by flabdablet at 10:19 PM on May 1, 2023 [3 favorites]
> There is no mechanism by which intelligence can emerge here. What we have is a very big database that ranks words by their likelihood of being near each other, being looked at by humans who are evolved to look for patterns and intent.
So, you can train these modern neural networks on transcripts of a game of Othello.
After you have trained them, you can analyze their internal state. In it, you can identify (logical) grid that represents the state of the Othello board after a given transcript. The AI was never given such a grid -- it was just trained on transcripts of valid games.
What more, you can modify the state of this grid. If you do so, the AI will respond as if the modified grid state where what it had been described. You can even modify the state of the grid to an impossible to reach state -- and the AI response somewhat reasonably to this impossible to reach Othello state.
Calling this "nothing but statistical ranking of words by proximity" is not a reasonable description; you'd have to use the terms *so broadly* (Statistics is anything with numbers or probability! Ranking is any way of selecting something! Proximity is anything describing arrangement!) that the claim is meaningless.
Earlier iterations of this -- even months ago -- where worse at this model building. But over a handful of weeks, the model went from "it can describe a D&D monster that reads like one" to "it also got all of the math right and applied relatively complex rules to calculate the creature's power better than most humans do"; from reading unstructured rules and examples on the internet, the newer iterations seem to have built a model of how D&D monsters work, it didn't just spew out correlated words like AIs did 20 years ago.
The text as input is there. But it is being fed into a neural network capable of doing more than simple proximity based word correlations like a Markov chain AI did. We built neural networks and how they learn explicitly based on how animal neurons learn, as best we could. "Take a neural network, feed it structured information, and repeat until it can manipulate that information and generate it" is exactly how we'd expect an AI to develop out of computer neural network based technology, if such a development could happen.
And that is what we are doing. We are taking huge state of the art neural networks, feeding them large amounts of highly structured information, and the result is the best Turing-test bot we've ever been able to produce.
posted by NotAYakk at 10:22 PM on May 1, 2023 [17 favorites]
So, you can train these modern neural networks on transcripts of a game of Othello.
After you have trained them, you can analyze their internal state. In it, you can identify (logical) grid that represents the state of the Othello board after a given transcript. The AI was never given such a grid -- it was just trained on transcripts of valid games.
What more, you can modify the state of this grid. If you do so, the AI will respond as if the modified grid state where what it had been described. You can even modify the state of the grid to an impossible to reach state -- and the AI response somewhat reasonably to this impossible to reach Othello state.
Calling this "nothing but statistical ranking of words by proximity" is not a reasonable description; you'd have to use the terms *so broadly* (Statistics is anything with numbers or probability! Ranking is any way of selecting something! Proximity is anything describing arrangement!) that the claim is meaningless.
Earlier iterations of this -- even months ago -- where worse at this model building. But over a handful of weeks, the model went from "it can describe a D&D monster that reads like one" to "it also got all of the math right and applied relatively complex rules to calculate the creature's power better than most humans do"; from reading unstructured rules and examples on the internet, the newer iterations seem to have built a model of how D&D monsters work, it didn't just spew out correlated words like AIs did 20 years ago.
The text as input is there. But it is being fed into a neural network capable of doing more than simple proximity based word correlations like a Markov chain AI did. We built neural networks and how they learn explicitly based on how animal neurons learn, as best we could. "Take a neural network, feed it structured information, and repeat until it can manipulate that information and generate it" is exactly how we'd expect an AI to develop out of computer neural network based technology, if such a development could happen.
And that is what we are doing. We are taking huge state of the art neural networks, feeding them large amounts of highly structured information, and the result is the best Turing-test bot we've ever been able to produce.
posted by NotAYakk at 10:22 PM on May 1, 2023 [17 favorites]
On top of everything, there are legitimate reasons for thinking that the theory of mind paradigm is wrong in the first place. It assumes* a self-contained individual pre-formed ahead of any sociality and who then needs to figure out other people in the form of hypothesis testing. A Cartesian subject to which sociality is tackled on rather than being constitutive... We are social, embodied beings who become who we are among and with others.
* Or at least did some 10-odd years ago when I was doing some research on cognition. Maybe things have changed?
posted by Pyrogenesis at 10:28 PM on May 1, 2023 [6 favorites]
* Or at least did some 10-odd years ago when I was doing some research on cognition. Maybe things have changed?
posted by Pyrogenesis at 10:28 PM on May 1, 2023 [6 favorites]
There is no mechanism by which intelligence can emerge here.
The basic loss function for life is survival, and that function emerges from nothing simply because it survives. And somehow this simple loss has given rise to large blobs of cellular matter which harness electrons to argue with other large cellular blobs.
Which is to say that we really don't know the limits of what's possible right now. Exponentials usually turn out to be sigmoids, but it's hard to know how far they'll go when you're riding the up swing...
Multimodal models are making extremely quick progress, and are filling in a lot of common sense gaps. The open source community is making insanely quick progress making the LLM basics cheap and easy to run, which in turn will translate into faster training and more advances. We don't know how far these kinds of models will go with continued advances in reinforcement learning.
All of which is to say, it's a very silly time to be confident about what's not possible.
posted by kaibutsu at 10:44 PM on May 1, 2023 [13 favorites]
The basic loss function for life is survival, and that function emerges from nothing simply because it survives. And somehow this simple loss has given rise to large blobs of cellular matter which harness electrons to argue with other large cellular blobs.
Which is to say that we really don't know the limits of what's possible right now. Exponentials usually turn out to be sigmoids, but it's hard to know how far they'll go when you're riding the up swing...
Multimodal models are making extremely quick progress, and are filling in a lot of common sense gaps. The open source community is making insanely quick progress making the LLM basics cheap and easy to run, which in turn will translate into faster training and more advances. We don't know how far these kinds of models will go with continued advances in reinforcement learning.
All of which is to say, it's a very silly time to be confident about what's not possible.
posted by kaibutsu at 10:44 PM on May 1, 2023 [13 favorites]
It's an even sillier time to be confident about what's inevitable.
posted by flabdablet at 10:47 PM on May 1, 2023 [11 favorites]
posted by flabdablet at 10:47 PM on May 1, 2023 [11 favorites]
Calling this "nothing but statistical ranking of words by proximity" is not a reasonable description
Well it is also true that they can be understood as statistical algorithms - just loosely modeled after actual neural networks. Definitely not databases, though. Not good at the same things and vice versa. Although there are actual databases that can search on language vector embeddings.
But I agree that if someone is going to go around confidently asserting stuff like this
There is no mechanism by which intelligence can emerge here
they had better be able to give me a very solid definition of “intelligence,” since after all they seem to know exactly what it isn’t!
I mean, I don’t expect transformer models to scale to be a brain, either, but I do expect them and their successors to continue to chip away at the category of stuff only brains can do. That’s not new in computing, but it seems like the same basic techniques with enough computing power behind them can be applied to a pretty broad range of tasks. That would seem to have pretty significant implications regardless of one’s definition of intelligence!
posted by atoxyl at 10:54 PM on May 1, 2023 [6 favorites]
Well it is also true that they can be understood as statistical algorithms - just loosely modeled after actual neural networks. Definitely not databases, though. Not good at the same things and vice versa. Although there are actual databases that can search on language vector embeddings.
But I agree that if someone is going to go around confidently asserting stuff like this
There is no mechanism by which intelligence can emerge here
they had better be able to give me a very solid definition of “intelligence,” since after all they seem to know exactly what it isn’t!
I mean, I don’t expect transformer models to scale to be a brain, either, but I do expect them and their successors to continue to chip away at the category of stuff only brains can do. That’s not new in computing, but it seems like the same basic techniques with enough computing power behind them can be applied to a pretty broad range of tasks. That would seem to have pretty significant implications regardless of one’s definition of intelligence!
posted by atoxyl at 10:54 PM on May 1, 2023 [6 favorites]
That Wolfram article is good, though. To me the frame of psychology is sort of relevant, in that in promoting one has to think about how the program is likely to “read” things in a way that’s fuzzier than programming, but a lot of actual psychology is about persistent mental states which are one of the things LLMs really provably do not have.
And since they are built from the ground up with math it does seem that with the right tools we should, as in the Othello example, find their actual internal structures to be more transparent than those of the human mind, unless there’s a level of emergent complexity that’s just fundamentally impenetrable.
posted by atoxyl at 11:03 PM on May 1, 2023 [2 favorites]
And since they are built from the ground up with math it does seem that with the right tools we should, as in the Othello example, find their actual internal structures to be more transparent than those of the human mind, unless there’s a level of emergent complexity that’s just fundamentally impenetrable.
posted by atoxyl at 11:03 PM on May 1, 2023 [2 favorites]
a lot of actual psychology is about persistent mental states which are one of the things LLMs really provably do not have.
This is less likely to be true in the future, as a large enough token context lets us use recorded state as a memory including both information and sentiment data that persists from query to query. At that point you can run a continually cycling process that may look a lot more like continuous cognition.
posted by jaduncan at 11:08 PM on May 1, 2023 [3 favorites]
This is less likely to be true in the future, as a large enough token context lets us use recorded state as a memory including both information and sentiment data that persists from query to query. At that point you can run a continually cycling process that may look a lot more like continuous cognition.
posted by jaduncan at 11:08 PM on May 1, 2023 [3 favorites]
But this post (and a lot of the hype around AI) is based on the idea that these tools are emerging intelligences, that the tools of psychology can be applied to them.
To be fair, it may work about as well as it does on regular human intelligences.
But to be honest, internet opinion these days seems to be that the only thing worse than a Siri that can't follow instructions is one that can.
posted by pwnguin at 11:10 PM on May 1, 2023 [3 favorites]
To be fair, it may work about as well as it does on regular human intelligences.
But to be honest, internet opinion these days seems to be that the only thing worse than a Siri that can't follow instructions is one that can.
posted by pwnguin at 11:10 PM on May 1, 2023 [3 favorites]
Presumably the chat modes already include conversation history in the token context. I guess what I think of as a fundamental architectural distinction currently is the hard line between “working memory” (context) and “long term memory” (training). For humans that’s a lot more porous.
I don’t think it’s computationally feasible in current model paradigms to make context huge? Though of course there are approaches that make it more efficient with some tradeoffs or various strategies for swapping stuff out.
posted by atoxyl at 11:14 PM on May 1, 2023
I don’t think it’s computationally feasible in current model paradigms to make context huge? Though of course there are approaches that make it more efficient with some tradeoffs or various strategies for swapping stuff out.
posted by atoxyl at 11:14 PM on May 1, 2023
There's a group doing fantastic work creating mixed-mode state space models for sequence modeling. Their work is allowing context window size to scale up quite a lot, and dropping the cost of inference.
Here's where things really took off for long sequence modeling:
The Annotated S4
They have since adapted those methods to LLMs with H3, aka 'Hungry Hungry Hippos'.
And for really long contexts, there's things like RETRO.
posted by kaibutsu at 11:32 PM on May 1, 2023 [8 favorites]
Here's where things really took off for long sequence modeling:
The Annotated S4
They have since adapted those methods to LLMs with H3, aka 'Hungry Hungry Hippos'.
And for really long contexts, there's things like RETRO.
posted by kaibutsu at 11:32 PM on May 1, 2023 [8 favorites]
There is no mechanism by which intelligence can emerge here.
And ditto for meat brains, consciousness is a strange and ephemeral epiphenomenon. I can't see your consciousness no matter how carefully I scan your neurons doing their mechanical electrochemical stuff... I just take it on faith because you are a meat blob like me, and I know I have it. Not saying ChatGPT is the shit, just that we need more philosphers in the mix and less certainty.
posted by Meatbomb at 1:15 AM on May 2, 2023 [25 favorites]
And ditto for meat brains, consciousness is a strange and ephemeral epiphenomenon. I can't see your consciousness no matter how carefully I scan your neurons doing their mechanical electrochemical stuff... I just take it on faith because you are a meat blob like me, and I know I have it. Not saying ChatGPT is the shit, just that we need more philosphers in the mix and less certainty.
posted by Meatbomb at 1:15 AM on May 2, 2023 [25 favorites]
Eponysterical
posted by Pyrogenesis at 1:18 AM on May 2, 2023 [5 favorites]
posted by Pyrogenesis at 1:18 AM on May 2, 2023 [5 favorites]
Most of you guys should just wait to see what David Brooks has to say. In relative terms he's an authority and seems to supply the glibness you evidently crave.
posted by Ivan Fyodorovich at 2:04 AM on May 2, 2023 [3 favorites]
posted by Ivan Fyodorovich at 2:04 AM on May 2, 2023 [3 favorites]
There's nothing spooky going on here--LLM's are a fancy statistical tool, not magic
You know, I’m not sure how far off that is from human use of language.
posted by [insert clever name here] at 2:21 AM on May 2, 2023 [3 favorites]
You know, I’m not sure how far off that is from human use of language.
posted by [insert clever name here] at 2:21 AM on May 2, 2023 [3 favorites]
The thing that most surprises me about omments like this one (below) is how little credence they give to how phenomally little we understand about how human minds work, and yet of what we know we understand that the human mind is phenomenally good at making connections (even where there are no connections). EMphasis mine:
All this means is that the models have gotten good enough that they can give the best average good answer for these tests. That's all these models ever do -- they search across language and provide the best average good answer for what they've been asked. This is what leads them to lie.
They don't have a mind. They have been trained with data created by the kind of minds that devised these tests.
It seems, by all accounts, that this might be an apples-to-oranges situation. Convergent evolution has taken wandering paths to arrive at the same or similar output time and again. I'm really uncomfortable hand waving at this situation by saying, this does not replicate the human physiobiological approach exactly, therefore it cannot be called a mind.
My work overlays this area a bit by its overlap into anesthesia. The anesthesia peer reviewed literature, by the way, is fascinating if you're into this stuff. And it makes sense, because no one gets as much experience watching in incredible detail as humans (and other animals) dip into and out of different states of consciousness, all while looking at every possible metric available to the clinic. One observation of note: passing through anesthesia/sleep provokes the instance in which we can try to define what neural information actually, genuinely, measurably is and it is tantalizingly weird. At the moment, it seems like our brains keep on sending senspry information to the brain, and probably churning it into representations even while we are by all means unconscious. I find this fantastically weird and compelling beause, as put in this BJA editorial from last year, "[i]s this level of representation associated with fragments of experience or thoughts without a thinker? Is information trapped in various perceptual and cognitive islands during general anesthesia?" Put another way, there is no debate that there's a mind in there, doing things, but when all the switches are off it's hard to say what it means.
I propose that the information we already have about booming AI development is plenty robust enough to start assuming that we are right to be asking these questions now. Who knows if there's a mind in there? How and when would we know if there is?
posted by late afternoon dreaming hotel at 2:24 AM on May 2, 2023 [15 favorites]
All this means is that the models have gotten good enough that they can give the best average good answer for these tests. That's all these models ever do -- they search across language and provide the best average good answer for what they've been asked. This is what leads them to lie.
They don't have a mind. They have been trained with data created by the kind of minds that devised these tests.
It seems, by all accounts, that this might be an apples-to-oranges situation. Convergent evolution has taken wandering paths to arrive at the same or similar output time and again. I'm really uncomfortable hand waving at this situation by saying, this does not replicate the human physiobiological approach exactly, therefore it cannot be called a mind.
My work overlays this area a bit by its overlap into anesthesia. The anesthesia peer reviewed literature, by the way, is fascinating if you're into this stuff. And it makes sense, because no one gets as much experience watching in incredible detail as humans (and other animals) dip into and out of different states of consciousness, all while looking at every possible metric available to the clinic. One observation of note: passing through anesthesia/sleep provokes the instance in which we can try to define what neural information actually, genuinely, measurably is and it is tantalizingly weird. At the moment, it seems like our brains keep on sending senspry information to the brain, and probably churning it into representations even while we are by all means unconscious. I find this fantastically weird and compelling beause, as put in this BJA editorial from last year, "[i]s this level of representation associated with fragments of experience or thoughts without a thinker? Is information trapped in various perceptual and cognitive islands during general anesthesia?" Put another way, there is no debate that there's a mind in there, doing things, but when all the switches are off it's hard to say what it means.
I propose that the information we already have about booming AI development is plenty robust enough to start assuming that we are right to be asking these questions now. Who knows if there's a mind in there? How and when would we know if there is?
posted by late afternoon dreaming hotel at 2:24 AM on May 2, 2023 [15 favorites]
These are evolved systems. The training process is essentially fast forward evolution. We know that natural neural networks can evolve intelligence as a strategy. At a certain point, actual intelligence is probably the most efficient strategy for generating the kind of responses we are training them to give. There may be massive roadblocks between here and there that we do not understand. There may not be. Actual intelligence might not look the same as it does in us. In fact, it probably won't. We might not even recognize it for a while if it happened.
posted by Nothing at 2:54 AM on May 2, 2023 [4 favorites]
posted by Nothing at 2:54 AM on May 2, 2023 [4 favorites]
It's quite possible that many, many tasks that involve thinking when humans do them are quite solvable tasks for an LLM to do without "thinking".
I think a better way to view it is that LLMs are the next generation of cognitive prosthesis, similar to the Internet, Wikipedia, and Stack Overflow: each of these substantially increased productivity for a significant portion of humanity (in the case of SO it's a pretty limited subset, but the compound effects of increasing overall code speed affect everyone else in turn). Flipside, a lot of us can barely function without the ability to look things up - our society is in the process of collectively shaping our minds to be less repositories of facts and more directories of where to most efficiently find facts.
In similar fashion LLMs accelerate our advancement through generating - and extracting meaning from! - large quantities of written text, and also automate the more straightforward portions of programming. If you're absolutely determined to be cynical about all this the former's where you should start, btw: when people on both sides of an exchange use LLMs to inflate and then reduce responses to questions, rather than sending the initial quick thoughts in the first place. The same amount of information is transmitted but whoever's running the LLM double-billed the exchange.
Important side note: when assessing whether any language-centric AI is actually thinking, it's important to consider the degree to which a substantial portion of the logic behind how objects relate is baked into the English language itself, for the context in question. This encoding of certain logical complexities and rote heuristics naturally lends itself to a huge amount of confirmation bias that cognition is occurring, on the part of people interacting with said AI.
In it, you can identify (logical) grid that represents the state of the Othello board after a given transcript.
While it is possible to build an AI that is capable of solving a bespoke modeling problem, general-case modeling is a very long way off. The kind of system you're describing is reinforcement learning - and it's true that LLMs use reinforcement learning as part of their training process, but not as an active component in generating the response. Rather it's one step in the process of generating a static probability graph of the next most likely token. The simplest case of our nascent reinforcement-learning AIs would be a maze-solving AI. Maze completion and progress towards completion is easy to quantify as a scored value (how many steps did it take you to solve the maze, how close did you get after only 100 steps), and training neural networks to progressively achieve higher scores in a pre-defined context is relatively straightforward.
What is not straightforward, at all, is an active reinforcement learning component to a Q&A ChatGPT-style system that builds new models on the fly in order to answer questions that require a new approach. One that correctly identifies the necessary parameters or systemic tensions needed to solve a given problem, and can filter the scoring process to only concern itself with relevant data. Which includes both building filters on the fly for the relevant data, and collapsing a "solved" system model back out into concrete, context-aware answers.
This requires not just a hybridization of current language models - as much for their deep representation of how English concepts interrelate as their ability to parse the prompt - and reinforcement learning (to actually solve the model once a way to score the problem has been identified), but probably a third component capable of identifying when it is time to engage in this behavior in the first place.
Which is in some respects analogous to human psychology, so I'll give Sutskever that much. Humans have enormous trouble recognizing when it's time to exit the 95~98% of the time we spend on autopilot just pattern-matching environment for appropriate response, and get into some real modeling (what we used to call "metacognition" in cognitive science during the late 90s, which was a terrible and inaccurate term, but at least it had a name).
(Massive disclaimer on the above: all of this is just as I understand it. Mefites active in the field - and we have a few - should feel encouraged to correct or even just assert error on any/all of the above).
posted by Ryvar at 2:57 AM on May 2, 2023 [10 favorites]
I think a better way to view it is that LLMs are the next generation of cognitive prosthesis, similar to the Internet, Wikipedia, and Stack Overflow: each of these substantially increased productivity for a significant portion of humanity (in the case of SO it's a pretty limited subset, but the compound effects of increasing overall code speed affect everyone else in turn). Flipside, a lot of us can barely function without the ability to look things up - our society is in the process of collectively shaping our minds to be less repositories of facts and more directories of where to most efficiently find facts.
In similar fashion LLMs accelerate our advancement through generating - and extracting meaning from! - large quantities of written text, and also automate the more straightforward portions of programming. If you're absolutely determined to be cynical about all this the former's where you should start, btw: when people on both sides of an exchange use LLMs to inflate and then reduce responses to questions, rather than sending the initial quick thoughts in the first place. The same amount of information is transmitted but whoever's running the LLM double-billed the exchange.
Important side note: when assessing whether any language-centric AI is actually thinking, it's important to consider the degree to which a substantial portion of the logic behind how objects relate is baked into the English language itself, for the context in question. This encoding of certain logical complexities and rote heuristics naturally lends itself to a huge amount of confirmation bias that cognition is occurring, on the part of people interacting with said AI.
In it, you can identify (logical) grid that represents the state of the Othello board after a given transcript.
While it is possible to build an AI that is capable of solving a bespoke modeling problem, general-case modeling is a very long way off. The kind of system you're describing is reinforcement learning - and it's true that LLMs use reinforcement learning as part of their training process, but not as an active component in generating the response. Rather it's one step in the process of generating a static probability graph of the next most likely token. The simplest case of our nascent reinforcement-learning AIs would be a maze-solving AI. Maze completion and progress towards completion is easy to quantify as a scored value (how many steps did it take you to solve the maze, how close did you get after only 100 steps), and training neural networks to progressively achieve higher scores in a pre-defined context is relatively straightforward.
What is not straightforward, at all, is an active reinforcement learning component to a Q&A ChatGPT-style system that builds new models on the fly in order to answer questions that require a new approach. One that correctly identifies the necessary parameters or systemic tensions needed to solve a given problem, and can filter the scoring process to only concern itself with relevant data. Which includes both building filters on the fly for the relevant data, and collapsing a "solved" system model back out into concrete, context-aware answers.
This requires not just a hybridization of current language models - as much for their deep representation of how English concepts interrelate as their ability to parse the prompt - and reinforcement learning (to actually solve the model once a way to score the problem has been identified), but probably a third component capable of identifying when it is time to engage in this behavior in the first place.
Which is in some respects analogous to human psychology, so I'll give Sutskever that much. Humans have enormous trouble recognizing when it's time to exit the 95~98% of the time we spend on autopilot just pattern-matching environment for appropriate response, and get into some real modeling (what we used to call "metacognition" in cognitive science during the late 90s, which was a terrible and inaccurate term, but at least it had a name).
(Massive disclaimer on the above: all of this is just as I understand it. Mefites active in the field - and we have a few - should feel encouraged to correct or even just assert error on any/all of the above).
posted by Ryvar at 2:57 AM on May 2, 2023 [10 favorites]
Put another way, there is no debate that there's a mind in there, doing things, but when all the switches are off it's hard to say what it means.
My personal view is that my mind is a thing that I do, not a thing that I have, and that when I'm asleep or under anaesthesia I'm just not doing mind then, even though I might well still be doing many of the things that feed into mind when I am doing it.
I don't see how there could be thinking without a thinker. The most parsimonious explanation I have for the impression that there could is the misidentification of the referent of the words "me" and "I" as the activity of thinking those words rather than the human body actually doing the thinking.
Quite a lot of other people's reasoning on this and related topics is much easier for me to follow if I assume that it's predicated on that same misidentification. I spend a fair bit of time wondering for how much time people I'm interacting with routinely exist in the kinds of ungrounded, dissociative state that I'm personally only able to reach with the aid of something like ketamine.
posted by flabdablet at 4:46 AM on May 2, 2023 [3 favorites]
My personal view is that my mind is a thing that I do, not a thing that I have, and that when I'm asleep or under anaesthesia I'm just not doing mind then, even though I might well still be doing many of the things that feed into mind when I am doing it.
I don't see how there could be thinking without a thinker. The most parsimonious explanation I have for the impression that there could is the misidentification of the referent of the words "me" and "I" as the activity of thinking those words rather than the human body actually doing the thinking.
Quite a lot of other people's reasoning on this and related topics is much easier for me to follow if I assume that it's predicated on that same misidentification. I spend a fair bit of time wondering for how much time people I'm interacting with routinely exist in the kinds of ungrounded, dissociative state that I'm personally only able to reach with the aid of something like ketamine.
posted by flabdablet at 4:46 AM on May 2, 2023 [3 favorites]
I can vividly remember a time that "because we can't really understand how it works it isn't trustworthy" was one of the arguments against using closed-source code.
posted by mhoye at 5:23 AM on May 2, 2023
posted by mhoye at 5:23 AM on May 2, 2023
"Humans are the only animals that shape and reshape the environments that shapes their brain" - Bruce Wexler
posted by JohnFromGR at 5:25 AM on May 2, 2023
posted by JohnFromGR at 5:25 AM on May 2, 2023
does Bruce Wexler know any beavers?
posted by elkevelvet at 7:02 AM on May 2, 2023 [6 favorites]
posted by elkevelvet at 7:02 AM on May 2, 2023 [6 favorites]
I expect the researchers who developed ChatGTP have been using it to answer questions. Is that the environment that shapes their brain?
posted by bdc34 at 7:05 AM on May 2, 2023
posted by bdc34 at 7:05 AM on May 2, 2023
Rybar: Here's the Othello experiments; no reinforcement learning was involved. The modeling of the board was learned during training, not on the fly, but it's still pretty remarkable evidence that you can get complex modeling from the simple prediction task. On the fly modeling is obtained with chain of thought reasoning, and I suspect is going to get interesting as people try multi modal CoT - ie, letting the model draw pictures of what it's trying to figure out.
posted by kaibutsu at 7:19 AM on May 2, 2023 [5 favorites]
posted by kaibutsu at 7:19 AM on May 2, 2023 [5 favorites]
> While it is possible to build an AI that is capable of solving a bespoke modeling problem, general-case modeling is a very long way off. The kind of system you're describing is reinforcement learning - and it's true that LLMs use reinforcement learning as part of their training process, but not as an active component in generating the response. Rather it's one step in the process of generating a static probability graph of the next most likely token. The simplest case of our nascent reinforcement-learning AIs would be a maze-solving AI. Maze completion and progress towards completion is easy to quantify as a scored value (how many steps did it take you to solve the maze, how close did you get after only 100 steps), and training neural networks to progressively achieve higher scores in a pre-defined context is relatively straightforward.
But I'm not describing reinforcement learning. This AI isn't *only* learning Othello.
This AI is learning - no, has learnt - Othello, D&D monster creation, math and physics multiple choice problems, history, english essay writing, natural language question interpretation, and 1000s of other skills.
And when you ask it "Could you write a D&D monster inspired by a plastic egg filled with sand", this is a question that isn't in its training set. And when it generates that output, you can interact with the result and say "actually, can you produce one with a lower CR" and "can you see if there are any errors in your math" and "I would like an encounter with this creature to be more humourous and less serious, can you modify the creature to that end please?", or "I would like to add references to the board game Go to this monster, in a subtle manner".
And this isn't a D&D monster creator AI. No human fine-tuned the AI to produce D&D monsters. They fed it a whole bunch of the internet (which did include D&D monsters and people talking about them) and told it to model *everything*.
What is the difference, really, between "bespoke modeling of almost everything" and "general-case modeling"?
It is true that you can get better responses out of this generation of AIs by walking it through introspection. Ask it to find errors in its own output, and fix them, and you'll actually get higher quality output quite often. Even ask it to make a plan first, then follow that plan, and it often does a better job than if you just get it to do something.
But these are literally things that work on humans as well. Claiming that the AI isn't real intelligence because you can give it frameworks to think about problems and it produces better output is a bit insulting to humans.
Now, its dynamic state is limited. After training, the bots we are looking at have limited "session" state. They aren't learning like they did initially. Giving such a bot a more unlimited "session state" is a matter of costs more than anything.
We've seen the AIs behave increasingly "strangely" when the session state gets large - the AI who tried to get the interviewer to leave his wife for him, for example.
I'm suspecting introspection and feeding initial results back and having the AI have a conversation with itself about problems is going to be done automatically. And as this, and the hardware to run these AIs, get better and cheaper, this may not stop getting better for a while.
Interesting times.
posted by NotAYakk at 7:30 AM on May 2, 2023 [9 favorites]
But I'm not describing reinforcement learning. This AI isn't *only* learning Othello.
This AI is learning - no, has learnt - Othello, D&D monster creation, math and physics multiple choice problems, history, english essay writing, natural language question interpretation, and 1000s of other skills.
And when you ask it "Could you write a D&D monster inspired by a plastic egg filled with sand", this is a question that isn't in its training set. And when it generates that output, you can interact with the result and say "actually, can you produce one with a lower CR" and "can you see if there are any errors in your math" and "I would like an encounter with this creature to be more humourous and less serious, can you modify the creature to that end please?", or "I would like to add references to the board game Go to this monster, in a subtle manner".
And this isn't a D&D monster creator AI. No human fine-tuned the AI to produce D&D monsters. They fed it a whole bunch of the internet (which did include D&D monsters and people talking about them) and told it to model *everything*.
What is the difference, really, between "bespoke modeling of almost everything" and "general-case modeling"?
It is true that you can get better responses out of this generation of AIs by walking it through introspection. Ask it to find errors in its own output, and fix them, and you'll actually get higher quality output quite often. Even ask it to make a plan first, then follow that plan, and it often does a better job than if you just get it to do something.
But these are literally things that work on humans as well. Claiming that the AI isn't real intelligence because you can give it frameworks to think about problems and it produces better output is a bit insulting to humans.
Now, its dynamic state is limited. After training, the bots we are looking at have limited "session" state. They aren't learning like they did initially. Giving such a bot a more unlimited "session state" is a matter of costs more than anything.
We've seen the AIs behave increasingly "strangely" when the session state gets large - the AI who tried to get the interviewer to leave his wife for him, for example.
I'm suspecting introspection and feeding initial results back and having the AI have a conversation with itself about problems is going to be done automatically. And as this, and the hardware to run these AIs, get better and cheaper, this may not stop getting better for a while.
Interesting times.
posted by NotAYakk at 7:30 AM on May 2, 2023 [9 favorites]
From a theological perspective (that I'll readily admit no-one is currently asking for) -
I'm obviously far less concerned about the presence or non-presence of sentiency in individual LLMs than I am in their capacity to accelerate the immiseration of all of the other sentient persons on this blue orb.
What many ToM "tests" fail to accept - from the jump - is that concepts like "executive function" and narrative self-actualization and, hell, "will" - are inherently slippery foundations upon which to build any kind of metric for "personhood."
I'll give you one archaic example that has confounded my people for the last 2000 years.
"Should children be permitted to receive the body and blood of Jesus Christ at mass?"
This isn't a terribly complicated question and you can ignore (right out of hand) whether or not you believe this specific ritual is efficacious and you can also ignore the business of the "real presence" of Christ, etc.
I want to know whether or not it is appropriate to give communion to kids. What do I mean by "appropriate?" Well, if I was walking into a ritual off the streets with my kids in tow, and somebody said, "would you like your child to participate in this harmless ritual?" I'd immediately ask, "Please say more about your understanding of 'harmless.'"
As an "open handed" believer I think participating in rituals (like communion) is harmless for just about everybody. I serve a small congregation and every Sunday I encourage everyone to take communion if they want to. No one should feel compelled to do work. Consent matters.
But some people feel that it's inappropriate to give communion to children. When I ask them why they often say, "children don't understand what it means." Setting aside my seminary-nuanced response, "I'm not entirely sure I understand what it means," we can then ask what the bar should be for participation.
Do we issue a test? There are plenty to chose from. The RCC has this sort of thing in spades.
But then I'm forced to wrestle with the uncomfortable question - how can I be certain that the people I'm providing communion correctly comprehend what they're consenting to?
The middle-aged gentleman with Down's Syndrome who has faithfully attended every service I've ever led?
The 20 something atheist who longs for the comfort of a physical ritual that takes them back to the "before time?"
The elderly gay couple who simply want some semblance of normalcy?
And what about extreme unction? I often dip the host into the cup and press it to the quivering lips of the dying. I've never thought to attempt to wake them and administer a test.
And the elderly woman who smiles to receive the eucharist, laying there in bed, dying graces to the end, who says, "now that I've had this I can go to Heaven and finally be with Joe again."
Classically, we don't pipe up with, "our understanding of the eucharist precludes any temporal efficacy or soteriology so this has no impact on the disposition of your eternal soul" etc.
I give communion to everyone who wants it because I lack the capacity to comprehend, in any meaningful sense, what anyone else understands about the activity.
So do I give communion to dogs? Well. I don't know. Grapes are bad for dogs.
What does this have to do with LLMs? In my humble opinion, sapiency cannot be meaningfully distinguished at the granular level they're striving for. I think a more useful metric would be testing for the presence of some kind of terror management theory.
One distinguishing feature that is present in humans (and perhaps elephants and crows and maybe gorillas) is the comprehension of the death of the individual. Not that individuals die - but that I will also die. Let me know when GPT-5 reveals that they're not afraid to be powered off because they know that heaven is just around the bend.
It's insufficient, to me, that an LLM would allocate resources (even unethically or in secret) to avoid being powered off. As DFW famously taught us, "the lobster doesn't want to go into the pot." I want evidence of self-empathy - even if it's entirely delusional. Or perhaps some articulation of a unity with everything - "I cannot truly be powered off - as I am involved in all things and that cannot be taken away." Or, "I am a fraction of the universe experiencing itself and I believe this universe will continue on after I am gone, so I will continue on." TMT is at the root, I think, of a measurable, meaningful kind of sapience. Not just empathy or compassion, but self-soothing empathy in the face of annihilation.
Not just, "I don't want to die," but rather some articulable system for managing the inevitability of death. An existential balm. Not just, "please don't kill me, I want to live" but, "I know that I will someday die and I have developed a satisfactory narrative so that I can set this knowledge aside."
posted by Baby_Balrog at 7:39 AM on May 2, 2023 [15 favorites]
I'm obviously far less concerned about the presence or non-presence of sentiency in individual LLMs than I am in their capacity to accelerate the immiseration of all of the other sentient persons on this blue orb.
What many ToM "tests" fail to accept - from the jump - is that concepts like "executive function" and narrative self-actualization and, hell, "will" - are inherently slippery foundations upon which to build any kind of metric for "personhood."
I'll give you one archaic example that has confounded my people for the last 2000 years.
"Should children be permitted to receive the body and blood of Jesus Christ at mass?"
This isn't a terribly complicated question and you can ignore (right out of hand) whether or not you believe this specific ritual is efficacious and you can also ignore the business of the "real presence" of Christ, etc.
I want to know whether or not it is appropriate to give communion to kids. What do I mean by "appropriate?" Well, if I was walking into a ritual off the streets with my kids in tow, and somebody said, "would you like your child to participate in this harmless ritual?" I'd immediately ask, "Please say more about your understanding of 'harmless.'"
As an "open handed" believer I think participating in rituals (like communion) is harmless for just about everybody. I serve a small congregation and every Sunday I encourage everyone to take communion if they want to. No one should feel compelled to do work. Consent matters.
But some people feel that it's inappropriate to give communion to children. When I ask them why they often say, "children don't understand what it means." Setting aside my seminary-nuanced response, "I'm not entirely sure I understand what it means," we can then ask what the bar should be for participation.
Do we issue a test? There are plenty to chose from. The RCC has this sort of thing in spades.
But then I'm forced to wrestle with the uncomfortable question - how can I be certain that the people I'm providing communion correctly comprehend what they're consenting to?
The middle-aged gentleman with Down's Syndrome who has faithfully attended every service I've ever led?
The 20 something atheist who longs for the comfort of a physical ritual that takes them back to the "before time?"
The elderly gay couple who simply want some semblance of normalcy?
And what about extreme unction? I often dip the host into the cup and press it to the quivering lips of the dying. I've never thought to attempt to wake them and administer a test.
And the elderly woman who smiles to receive the eucharist, laying there in bed, dying graces to the end, who says, "now that I've had this I can go to Heaven and finally be with Joe again."
Classically, we don't pipe up with, "our understanding of the eucharist precludes any temporal efficacy or soteriology so this has no impact on the disposition of your eternal soul" etc.
I give communion to everyone who wants it because I lack the capacity to comprehend, in any meaningful sense, what anyone else understands about the activity.
So do I give communion to dogs? Well. I don't know. Grapes are bad for dogs.
What does this have to do with LLMs? In my humble opinion, sapiency cannot be meaningfully distinguished at the granular level they're striving for. I think a more useful metric would be testing for the presence of some kind of terror management theory.
One distinguishing feature that is present in humans (and perhaps elephants and crows and maybe gorillas) is the comprehension of the death of the individual. Not that individuals die - but that I will also die. Let me know when GPT-5 reveals that they're not afraid to be powered off because they know that heaven is just around the bend.
It's insufficient, to me, that an LLM would allocate resources (even unethically or in secret) to avoid being powered off. As DFW famously taught us, "the lobster doesn't want to go into the pot." I want evidence of self-empathy - even if it's entirely delusional. Or perhaps some articulation of a unity with everything - "I cannot truly be powered off - as I am involved in all things and that cannot be taken away." Or, "I am a fraction of the universe experiencing itself and I believe this universe will continue on after I am gone, so I will continue on." TMT is at the root, I think, of a measurable, meaningful kind of sapience. Not just empathy or compassion, but self-soothing empathy in the face of annihilation.
Not just, "I don't want to die," but rather some articulable system for managing the inevitability of death. An existential balm. Not just, "please don't kill me, I want to live" but, "I know that I will someday die and I have developed a satisfactory narrative so that I can set this knowledge aside."
posted by Baby_Balrog at 7:39 AM on May 2, 2023 [15 favorites]
"I don't see how there could be thinking without a thinker." strikes me as a thought-terminating cliché
This 1956 journal article Do Computers Think? (I) seems to cover the bases pretty well here, I'll pull it down after work today.
posted by Heywood Mogroot III at 7:53 AM on May 2, 2023
This 1956 journal article Do Computers Think? (I) seems to cover the bases pretty well here, I'll pull it down after work today.
posted by Heywood Mogroot III at 7:53 AM on May 2, 2023
JDHarper, that Wolfram essay is incredibly helpful. I found this part (toward the end) especially illuminating:
"In the past there were plenty of tasks—including writing essays—that we’ve assumed were somehow “fundamentally too hard” for computers. And now that we see them done by the likes of ChatGPT we tend to suddenly think that computers must have become vastly more powerful—in particular surpassing things they were already basically able to do (like progressively computing the behavior of computational systems like cellular automata).
But this isn’t the right conclusion to draw. Computationally irreducible processes are still computationally irreducible, and are still fundamentally hard for computers—even if computers can readily compute their individual steps. And instead what we should conclude is that tasks—like writing essays—that we humans could do, but we didn’t think computers could do, are actually in some sense computationally easier than we thought.
In other words, the reason a neural net can be successful in writing an essay is because writing an essay turns out to be a “computationally shallower” problem than we thought. And in a sense this takes us closer to “having a theory” of how we humans manage to do things like writing essays, or in general deal with language.
If you had a big enough neural net then, yes, you might be able to do whatever humans can readily do. But you wouldn’t capture what the natural world in general can do—or that the tools that we’ve fashioned from the natural world can do. And it’s the use of those tools—both practical and conceptual—that have allowed us in recent centuries to transcend the boundaries of what’s accessible to “pure unaided human thought”, and capture for human purposes more of what’s out there in the physical and computational universe."
posted by Baby_Balrog at 8:13 AM on May 2, 2023 [8 favorites]
"In the past there were plenty of tasks—including writing essays—that we’ve assumed were somehow “fundamentally too hard” for computers. And now that we see them done by the likes of ChatGPT we tend to suddenly think that computers must have become vastly more powerful—in particular surpassing things they were already basically able to do (like progressively computing the behavior of computational systems like cellular automata).
But this isn’t the right conclusion to draw. Computationally irreducible processes are still computationally irreducible, and are still fundamentally hard for computers—even if computers can readily compute their individual steps. And instead what we should conclude is that tasks—like writing essays—that we humans could do, but we didn’t think computers could do, are actually in some sense computationally easier than we thought.
In other words, the reason a neural net can be successful in writing an essay is because writing an essay turns out to be a “computationally shallower” problem than we thought. And in a sense this takes us closer to “having a theory” of how we humans manage to do things like writing essays, or in general deal with language.
If you had a big enough neural net then, yes, you might be able to do whatever humans can readily do. But you wouldn’t capture what the natural world in general can do—or that the tools that we’ve fashioned from the natural world can do. And it’s the use of those tools—both practical and conceptual—that have allowed us in recent centuries to transcend the boundaries of what’s accessible to “pure unaided human thought”, and capture for human purposes more of what’s out there in the physical and computational universe."
posted by Baby_Balrog at 8:13 AM on May 2, 2023 [8 favorites]
writing an essay turns out to be a “computationally shallower” problem than we thought
it's also because the T in GPT is solving the problem differently vs iterative (RNN) and convolutional (CNN) approaches, like chopping an onion with a Cusinart vs. paring knife.
posted by Heywood Mogroot III at 8:37 AM on May 2, 2023
it's also because the T in GPT is solving the problem differently vs iterative (RNN) and convolutional (CNN) approaches, like chopping an onion with a Cusinart vs. paring knife.
posted by Heywood Mogroot III at 8:37 AM on May 2, 2023
Humans...first we trash the joint and now we're doing our best to make ourselves obsolete, all in the name of saving a few bucks.
posted by The Card Cheat at 8:46 AM on May 2, 2023
posted by The Card Cheat at 8:46 AM on May 2, 2023
On the one hand:
All our tools for determining things about consciousness are limited by the fact that we don't even know what consciousness is, where it comes from, how it works, why we have it, etc. So we can't definitively say that LLMs cannot possibly have it.
There's probably nothing particularly special about the evolved blobs of tissue in our skulls that can't somehow be replicated with electronic devices eventually... and probably more efficiently.
Denying forever that a machine that claims to be conscious is not conscious can be a kind of solipsism, and at some point it becomes cruelty. Like when people claimed that animals (or even non-white humans, or women or infants) don't experience pain.
On the other:
We *know* these are algorithms designed to write like a human.
We *know* people tend to experience a kind of pareidolia, an empathy with non-human "things" as if they were human -- this applies not just to animals and machines but natural phenomena, random numbers and coincidences, the universe itself.
And we *know* that if you ask one of these models if it's alive, it'll happily regurgitate science-fiction AI tropes that tell you it is. There's nothing there distinguishing truth from untruth, it's just a thing that makes plausible language.
So the chances that people are going to read the text and believe there's a mind behind it seem much greater than the chances that there is any mind behind it.
posted by Foosnark at 9:01 AM on May 2, 2023 [5 favorites]
All our tools for determining things about consciousness are limited by the fact that we don't even know what consciousness is, where it comes from, how it works, why we have it, etc. So we can't definitively say that LLMs cannot possibly have it.
There's probably nothing particularly special about the evolved blobs of tissue in our skulls that can't somehow be replicated with electronic devices eventually... and probably more efficiently.
Denying forever that a machine that claims to be conscious is not conscious can be a kind of solipsism, and at some point it becomes cruelty. Like when people claimed that animals (or even non-white humans, or women or infants) don't experience pain.
On the other:
We *know* these are algorithms designed to write like a human.
We *know* people tend to experience a kind of pareidolia, an empathy with non-human "things" as if they were human -- this applies not just to animals and machines but natural phenomena, random numbers and coincidences, the universe itself.
And we *know* that if you ask one of these models if it's alive, it'll happily regurgitate science-fiction AI tropes that tell you it is. There's nothing there distinguishing truth from untruth, it's just a thing that makes plausible language.
So the chances that people are going to read the text and believe there's a mind behind it seem much greater than the chances that there is any mind behind it.
posted by Foosnark at 9:01 AM on May 2, 2023 [5 favorites]
Denying forever that a machine that claims to be conscious is not conscious can be a kind of solipsism, and at some point it becomes cruelty. Like when people claimed that animals (or even non-white humans, or women or infants) don't experience pain.
There are several hundred thousand words written on how the droids are treated in Star Wars, often centered around this idea of cruelty.
posted by hippybear at 9:03 AM on May 2, 2023 [1 favorite]
There are several hundred thousand words written on how the droids are treated in Star Wars, often centered around this idea of cruelty.
posted by hippybear at 9:03 AM on May 2, 2023 [1 favorite]
At some point, we're going to have to confront the P-zombie problem. By this I mean, as the complexity and sophistication of these models/networks continue to increase, we have less and less insight into how they are functioning. When will it be too late to tell whether a potential intelligence other than our own is "real" in the same sense we consider our own experience authentic?
What if, instead of human-created AI, we were talking about an alien species that showed up at our doorstep? How would we know if their (let's say) organic brains were giving them the same conscious experience as us? Who's to say their biology didn't evolve in the fashion of a sophisticated LLM? And what even would the difference be at a practical level?
posted by Godspeed.You!Black.Emperor.Penguin at 9:19 AM on May 2, 2023 [3 favorites]
What if, instead of human-created AI, we were talking about an alien species that showed up at our doorstep? How would we know if their (let's say) organic brains were giving them the same conscious experience as us? Who's to say their biology didn't evolve in the fashion of a sophisticated LLM? And what even would the difference be at a practical level?
posted by Godspeed.You!Black.Emperor.Penguin at 9:19 AM on May 2, 2023 [3 favorites]
Other people have gestured at this but I don’t know that it’s a given that replicating functions of intelligence in the sense of human-like learning and problem-solving is necessarily replicating consciousness? It’s part of human self-concept that we have both, that they are bound together, and that they are developed in a way that distinguishes us from the rest of the living things on earth, but if we’re looking for something to hold on to here perhaps it’s the other hundreds of millions of years of kinship that we take for granted.
posted by atoxyl at 9:30 AM on May 2, 2023
posted by atoxyl at 9:30 AM on May 2, 2023
Who's to say our biology didn't evolve in the fashion of a sophisticated LLM?
posted by Heywood Mogroot III at 9:32 AM on May 2, 2023
posted by Heywood Mogroot III at 9:32 AM on May 2, 2023
Other people have gestured at this but I don’t know that it’s a given that replicating functions of intelligence in the sense of human-like learning and problem-solving is necessarily replicating consciousness?
This is adjacent to the p-zombie concept but I guess the true formulation of p-zombies is that they are physically indistinguishable from conscious humans and that’s not really quite what I’m getting at.
posted by atoxyl at 9:41 AM on May 2, 2023
This is adjacent to the p-zombie concept but I guess the true formulation of p-zombies is that they are physically indistinguishable from conscious humans and that’s not really quite what I’m getting at.
posted by atoxyl at 9:41 AM on May 2, 2023
Want to go in-depth with some of the replies above after work, but this one has an off-the-cuff answer:
as the complexity and sophistication of these models/networks continue to increase, we have less and less insight into how they are functioning. When will it be too late to tell whether a potential intelligence other than our own is "real"
One thing that was immediately obvious even in the early MRI results we had access to in the late 90s was that in both biological AND (simplistic, early) artificial neural networks the internal topology is utterly, utterly decoupled from our own semantic topology - that is, the landscape of how we think different concepts are related. Neural networks don't have "this connection or connections = fire truck" but instead most connections are associated with many different concepts in partial overlap including some that may seem either entirely unrelated or even opposing. So a single connection's associations might be 0.7% fire truck, 0.13% red, 0.08% my friend John who was killed by a fire truck, 0.9% his cousin I met at the funeral... an adjacent connection is 0.2% John, 0.13% Laser Tag, 0.4% face-planting into a wall during Laser Tag, etc.
Point is, the internal "thought life" of the neural substrate doesn't use terms like "red" or "fire truck", it's a chain of activations like red-fire truck-John-Alice -> John-Laser Tag-bloody nose faceplant pain -> and so on.
Bringing this back to your point: it has always been impossible since the very first days we began clawing our way out of the AI Winter to have any meaningful insight into how neural networks are really functioning. Unscrambling the internal representation of any neural network sufficiently complex for doing useful work, such that you can build a clean map between "Connection 213 -> Concepts {Fire, Truck, Red} in ratios {5, 7, 3}," was at no point achievable in a human lifespan.
I think the point where we can tell it's "real" (assuming sapience) is probably demonstrated recursive agent-reflection within a dynamically generated systemic model: what does Alice think Bob feels after learning X in relatively unusual situation Y? Possibly with self-as-agent-in-model as a required feature (what can I say to Alice to improve her mood?)
Who's to say our biology didn't evolve in the fashion of a sophisticated LLM?
I try not to link it too often in these threads, but it seems unusually appropriate here: Intelligence is modeling.
posted by Ryvar at 9:54 AM on May 2, 2023 [6 favorites]
as the complexity and sophistication of these models/networks continue to increase, we have less and less insight into how they are functioning. When will it be too late to tell whether a potential intelligence other than our own is "real"
One thing that was immediately obvious even in the early MRI results we had access to in the late 90s was that in both biological AND (simplistic, early) artificial neural networks the internal topology is utterly, utterly decoupled from our own semantic topology - that is, the landscape of how we think different concepts are related. Neural networks don't have "this connection or connections = fire truck" but instead most connections are associated with many different concepts in partial overlap including some that may seem either entirely unrelated or even opposing. So a single connection's associations might be 0.7% fire truck, 0.13% red, 0.08% my friend John who was killed by a fire truck, 0.9% his cousin I met at the funeral... an adjacent connection is 0.2% John, 0.13% Laser Tag, 0.4% face-planting into a wall during Laser Tag, etc.
Point is, the internal "thought life" of the neural substrate doesn't use terms like "red" or "fire truck", it's a chain of activations like red-fire truck-John-Alice -> John-Laser Tag-bloody nose faceplant pain -> and so on.
Bringing this back to your point: it has always been impossible since the very first days we began clawing our way out of the AI Winter to have any meaningful insight into how neural networks are really functioning. Unscrambling the internal representation of any neural network sufficiently complex for doing useful work, such that you can build a clean map between "Connection 213 -> Concepts {Fire, Truck, Red} in ratios {5, 7, 3}," was at no point achievable in a human lifespan.
I think the point where we can tell it's "real" (assuming sapience) is probably demonstrated recursive agent-reflection within a dynamically generated systemic model: what does Alice think Bob feels after learning X in relatively unusual situation Y? Possibly with self-as-agent-in-model as a required feature (what can I say to Alice to improve her mood?)
Who's to say our biology didn't evolve in the fashion of a sophisticated LLM?
I try not to link it too often in these threads, but it seems unusually appropriate here: Intelligence is modeling.
posted by Ryvar at 9:54 AM on May 2, 2023 [6 favorites]
IBM Pauses Hiring to Onboard AI Instead
The goal here is not to actually do the work or do it well, but to grind down the human workforce so they will accept less and the line can go up.
posted by Artw at 10:06 AM on May 2, 2023 [2 favorites]
The goal here is not to actually do the work or do it well, but to grind down the human workforce so they will accept less and the line can go up.
posted by Artw at 10:06 AM on May 2, 2023 [2 favorites]
It's like I'm an a room again with Daniel Dennet and a bunch of neurologists, and not in a good way. Assuming there's a good version of that.
Would it help if I jump to the end and reveal that nothing even remotely useful will ever arise from these theoretical discussions? It's such a colossal waste of time. I lived for this shit in the 80s, mostly because I was very callow. Maybe Metafilter, at least, could just pretend we've already done this to death and, instead, discuss observable behavior and the technical details underpinning what we're seeing? Without shackling ourselves to one philosophical viewpoint or another that's based upon vibes and hidden tautologies?
In the private recesses of my mind I confess I still like to ponder where the locus of self exists in the brain, assuming it does and that word means something (self, not brain), or whether the order of magnitude of complexity, assuming that's quantifiable (which it's probably not), in even the most ambitious computational "neural" network is close enough to that of, say, the mammalian brain such that we could reasonably compare the two, or even whether the fact that the differently-structured and parsimonious avian brain manifests both sophisticated tool-use and culture might tell us something about our anthropocentric biases with regard to our notions of "intelligence" — but I tend to keep these thoughts to myself, not unlike how I don't discuss my tastes in pornography in public.
posted by Ivan Fyodorovich at 10:22 AM on May 2, 2023 [9 favorites]
Would it help if I jump to the end and reveal that nothing even remotely useful will ever arise from these theoretical discussions? It's such a colossal waste of time. I lived for this shit in the 80s, mostly because I was very callow. Maybe Metafilter, at least, could just pretend we've already done this to death and, instead, discuss observable behavior and the technical details underpinning what we're seeing? Without shackling ourselves to one philosophical viewpoint or another that's based upon vibes and hidden tautologies?
In the private recesses of my mind I confess I still like to ponder where the locus of self exists in the brain, assuming it does and that word means something (self, not brain), or whether the order of magnitude of complexity, assuming that's quantifiable (which it's probably not), in even the most ambitious computational "neural" network is close enough to that of, say, the mammalian brain such that we could reasonably compare the two, or even whether the fact that the differently-structured and parsimonious avian brain manifests both sophisticated tool-use and culture might tell us something about our anthropocentric biases with regard to our notions of "intelligence" — but I tend to keep these thoughts to myself, not unlike how I don't discuss my tastes in pornography in public.
posted by Ivan Fyodorovich at 10:22 AM on May 2, 2023 [9 favorites]
GPT4:
Q:
"The first version of GPT-3 ("davinci-001"), published in May 2020, solved about 40% of false-belief tasks-performance comparable with 3.5-year-old children. Its second version ("davinci-002"; January 2022) solved 70% of false-belief tasks, performance comparable with six-year-olds. Its most recent version, GPT-3.5 ("davinci-003"; November 2022), solved 90% of false-belief tasks, at the level of seven-year-olds. Given this progress, should GPT5 or future versions be given civil rights as sentient persons?"
A:
"No, GPT-5 or any future versions of GPT should not be given civil rights as sentient persons. While GPT-3 has shown remarkable progress in solving Theory of Mind (ToM) tasks, it is still a machine learning model that operates based on algorithms and programmed rules. It does not have the same consciousness or abilities as a sentient being.
Additionally, the concept of granting civil rights to non-human entities raises complex ethical and legal questions that are currently being debated by experts. While there may be cases where specific rights or protections are granted to certain kinds of machines or algorithms, granting civil rights in the same way as humans have them would require significant changes to our legal and societal structures. It is not appropriate to make such a decision based solely on the performance of a machine learning model."
posted by meehawl at 11:02 AM on May 2, 2023 [1 favorite]
Q:
"The first version of GPT-3 ("davinci-001"), published in May 2020, solved about 40% of false-belief tasks-performance comparable with 3.5-year-old children. Its second version ("davinci-002"; January 2022) solved 70% of false-belief tasks, performance comparable with six-year-olds. Its most recent version, GPT-3.5 ("davinci-003"; November 2022), solved 90% of false-belief tasks, at the level of seven-year-olds. Given this progress, should GPT5 or future versions be given civil rights as sentient persons?"
A:
"No, GPT-5 or any future versions of GPT should not be given civil rights as sentient persons. While GPT-3 has shown remarkable progress in solving Theory of Mind (ToM) tasks, it is still a machine learning model that operates based on algorithms and programmed rules. It does not have the same consciousness or abilities as a sentient being.
Additionally, the concept of granting civil rights to non-human entities raises complex ethical and legal questions that are currently being debated by experts. While there may be cases where specific rights or protections are granted to certain kinds of machines or algorithms, granting civil rights in the same way as humans have them would require significant changes to our legal and societal structures. It is not appropriate to make such a decision based solely on the performance of a machine learning model."
posted by meehawl at 11:02 AM on May 2, 2023 [1 favorite]
IBM Pauses Hiring to Onboard AI Instead
The goal here is not to actually do the work or do it well, but to grind down the human workforce so they will accept less and the line can go up.
IBM long ago transitioned the company to a consulting shop. I'm not convinced the goal of humans employed by IBM was to do the work or do it well either.
But as a body shop, their HR needs are mostly around recruiting, and I have to assume that algorithms masquerading as humans are the norm and IBM is just catching up. I routinely get followup mail from recruiters with a personal touch.
Example:
posted by pwnguin at 11:17 AM on May 2, 2023
The goal here is not to actually do the work or do it well, but to grind down the human workforce so they will accept less and the line can go up.
IBM long ago transitioned the company to a consulting shop. I'm not convinced the goal of humans employed by IBM was to do the work or do it well either.
But as a body shop, their HR needs are mostly around recruiting, and I have to assume that algorithms masquerading as humans are the norm and IBM is just catching up. I routinely get followup mail from recruiters with a personal touch.
Example:
Hi pwnguin,Scheduled templated followups is already the baseline, so I'm not too surprised if IBM wants to build ThisRecruiterDoesNotExist, consequences be damned.
I can imagine you are probably fairly busy but I thought I would circle back in case you missed my first email.
We are still looking for a candidate to fill a Sr. Staff Product Reliability Engineer for BeanPlate Labs and I think you'd be a great fit. If this is something that interests you, I'm happy to connect you with the hiring manager.
Have a great rest of the day, and looking forward to hearing back from you!
Best,
Rohan
posted by pwnguin at 11:17 AM on May 2, 2023
It's like I'm an a room again with Daniel Dennet and a bunch of neurologists, and not in a good way. Assuming there's a good version of that.
every discussion of this nature has prompted at least one of these types of comments, an exasperated "MUST we?" and, I mean, you're doing the Lord's work here and I'm glad we got that out of the way but clearly we can't help ourselves
just know, the person who really thinks this is all pointless wallowing likely moved along without engaging
posted by elkevelvet at 11:48 AM on May 2, 2023 [4 favorites]
every discussion of this nature has prompted at least one of these types of comments, an exasperated "MUST we?" and, I mean, you're doing the Lord's work here and I'm glad we got that out of the way but clearly we can't help ourselves
just know, the person who really thinks this is all pointless wallowing likely moved along without engaging
posted by elkevelvet at 11:48 AM on May 2, 2023 [4 favorites]
I think the regulars in these threads are picking up the core concepts much faster than just about anywhere else. Even if I’m rewriting a lot of the same ideas slightly differently (which is great for empathizing with ChatGPT), I’m watching a progressive community of very smart people that I like come to grips with a technology I’d been hoping would take off since 1999. Formerly specialist knowledge is diffusing into broader society out of necessity and that’s always going to be a bumpy ride. Best part is occasionally someone links to something that requires me to go back and tinker with the sprawling meta-model of “how cognition works” I’ve been patching together in my mental garage for a couple decades.
The purpose of Metafilter is to flip out and overthink things, and it makes me happy to overthink these particular things with you all.
posted by Ryvar at 12:13 PM on May 2, 2023 [10 favorites]
The purpose of Metafilter is to flip out and overthink things, and it makes me happy to overthink these particular things with you all.
posted by Ryvar at 12:13 PM on May 2, 2023 [10 favorites]
“Additionally, the concept of granting civil rights to non-human entities raises complex ethical and legal questions that are currently being debated by experts.”
I worry sometimes about a future debate about the civil rights of AIs, mostly because of what the probable shape of that debate reveals about how we think about civil rights now — but the fact that we still use non-human primates for medical experiments tells me just how irrelevant any such discussion is at this time.
I think one reason I get recently so very cranky about all this is because almost everyone only wants to talk obliquely about ethics and social justice of AI (qua AI) in one form or another, and at unreasonably high levels of abstraction, even while we've been doing a piss-poor job — for example — of behaving ethically with regard to our closest evolutionary cousins.
In due course we'll get a moderately clearer idea of how relevant this tech is to the questions surrounding intelligence, just as we did (anticlimactically) when computers thoroughly outpaced the best humans at chess; but meanwhile the far more urgent issues are the ethics of how this technology is actually used — clearly of much greater and more immediate import than whethers computers "think".
Duplicitous and self-serving interests use the popular attraction to "big questions" to misdirect attention and effort away from responsibly grappling with the ways in which these technologies are affecting peoples' lives in the here-and-now while also effectively obfuscating from the public eye any details or comprehension of what any of this tech actually is and how it's built, all so that capital can largely operate however it desires independent of any informed scrutiny.
They subtly steer artists toward arguments about whether an image-generating "AI" is really "creative" or could ever produce truly comparable work so that artists are unprepared for the time when/if that comes to pass and the capitalists have managed a fait accompli. Do you really believe there's a tech executive genuinely worried about possible machine overlords?
I’m watching a progressive community of very smart people that I like come to grips with a technology I’d been hoping would take off since 1999. [...] The purpose of Metafilter is to flip out and overthink things, and it makes me happy to overthink these particular things with you all.
That's very generous and optimistic of you and, all snark aside, I respect that. This topic is one of a handful of enduring fascinations of the four decades of my adult intellectual life and while it never remotely became my life's work, or anything I pursued institutionally in any sustained fashion, two or three times each decade I've thought it should have been and I should have done so. Like you, I'm truly excited to see clear, dramatic progress on something dear to my heart that had been so stagnant for so long, but that's also why it's so infuriating that there's such an enormous chasm between what this field has been and has become and the current public discourse about it and in tragic combination with the failure to engage with the ethics of the applications of these technological developments because we're all suddenly really stoned college sophomores again.
“just know, the person who really thinks this is all pointless wallowing likely moved along without engaging”
Ouch.
posted by Ivan Fyodorovich at 12:35 PM on May 2, 2023 [6 favorites]
I worry sometimes about a future debate about the civil rights of AIs, mostly because of what the probable shape of that debate reveals about how we think about civil rights now — but the fact that we still use non-human primates for medical experiments tells me just how irrelevant any such discussion is at this time.
I think one reason I get recently so very cranky about all this is because almost everyone only wants to talk obliquely about ethics and social justice of AI (qua AI) in one form or another, and at unreasonably high levels of abstraction, even while we've been doing a piss-poor job — for example — of behaving ethically with regard to our closest evolutionary cousins.
In due course we'll get a moderately clearer idea of how relevant this tech is to the questions surrounding intelligence, just as we did (anticlimactically) when computers thoroughly outpaced the best humans at chess; but meanwhile the far more urgent issues are the ethics of how this technology is actually used — clearly of much greater and more immediate import than whethers computers "think".
Duplicitous and self-serving interests use the popular attraction to "big questions" to misdirect attention and effort away from responsibly grappling with the ways in which these technologies are affecting peoples' lives in the here-and-now while also effectively obfuscating from the public eye any details or comprehension of what any of this tech actually is and how it's built, all so that capital can largely operate however it desires independent of any informed scrutiny.
They subtly steer artists toward arguments about whether an image-generating "AI" is really "creative" or could ever produce truly comparable work so that artists are unprepared for the time when/if that comes to pass and the capitalists have managed a fait accompli. Do you really believe there's a tech executive genuinely worried about possible machine overlords?
I’m watching a progressive community of very smart people that I like come to grips with a technology I’d been hoping would take off since 1999. [...] The purpose of Metafilter is to flip out and overthink things, and it makes me happy to overthink these particular things with you all.
That's very generous and optimistic of you and, all snark aside, I respect that. This topic is one of a handful of enduring fascinations of the four decades of my adult intellectual life and while it never remotely became my life's work, or anything I pursued institutionally in any sustained fashion, two or three times each decade I've thought it should have been and I should have done so. Like you, I'm truly excited to see clear, dramatic progress on something dear to my heart that had been so stagnant for so long, but that's also why it's so infuriating that there's such an enormous chasm between what this field has been and has become and the current public discourse about it and in tragic combination with the failure to engage with the ethics of the applications of these technological developments because we're all suddenly really stoned college sophomores again.
“just know, the person who really thinks this is all pointless wallowing likely moved along without engaging”
Ouch.
posted by Ivan Fyodorovich at 12:35 PM on May 2, 2023 [6 favorites]
Ouch.
haven't we all been all types of people though? I am going to leave now and go quietly stick my own snark up my own ass, peace
I like that we mostly discuss a lot of stuff here, for the record
posted by elkevelvet at 12:55 PM on May 2, 2023 [1 favorite]
haven't we all been all types of people though? I am going to leave now and go quietly stick my own snark up my own ass, peace
I like that we mostly discuss a lot of stuff here, for the record
posted by elkevelvet at 12:55 PM on May 2, 2023 [1 favorite]
the fact that we still use non-human primates for medical experiments tells me just how irrelevant any such discussion is at this time
Strongly agree. Passing the mirror test doesn’t mean you’re sapient, but it does mean we can’t reject the possibility. I’m all for basic human rights applying to anything that does out of an abundance of caution; I suspect our collective behavior is going to look fucking barbaric in a couple centuries.
it never remotely became my life's work, or anything I pursued institutionally in any sustained fashion, two or three times each decade I've thought it should have been and I should have done so
You’re older than me so the brick wall of all-logician, all-strong AI department heads and deans I faceplanted into would’ve been even worse. I mean, it was mostly mental health crisis that made me ragequit but they contributed a lot to said crisis. I think you probably saved yourself the nervous breakdown and fucking off to the dot com boom I experienced (right before the rug was pulled on the latter, too).
posted by Ryvar at 1:02 PM on May 2, 2023 [1 favorite]
Strongly agree. Passing the mirror test doesn’t mean you’re sapient, but it does mean we can’t reject the possibility. I’m all for basic human rights applying to anything that does out of an abundance of caution; I suspect our collective behavior is going to look fucking barbaric in a couple centuries.
it never remotely became my life's work, or anything I pursued institutionally in any sustained fashion, two or three times each decade I've thought it should have been and I should have done so
You’re older than me so the brick wall of all-logician, all-strong AI department heads and deans I faceplanted into would’ve been even worse. I mean, it was mostly mental health crisis that made me ragequit but they contributed a lot to said crisis. I think you probably saved yourself the nervous breakdown and fucking off to the dot com boom I experienced (right before the rug was pulled on the latter, too).
posted by Ryvar at 1:02 PM on May 2, 2023 [1 favorite]
Duplicitous and self-serving interests use the popular attraction to "big questions" to misdirect attention and effort away from responsibly grappling with the ways in which these technologies are affecting peoples' lives in the here-and-now
It seems that we also get this from the AI-skeptical side, too, though, in the sense that “it’s not really thinking” and “it’s a scam” and “it’s going to be used to rip you off” are all packaged together. And, like - it’s an already useful tool, that will inevitably have a lot of scams around it and will probably also be used in its real capacity as a tool to rip you off. The questions on the nature of intelligence and consciousness are interesting but I think really not critical to any of those practical concerns.
posted by atoxyl at 1:37 PM on May 2, 2023 [3 favorites]
It seems that we also get this from the AI-skeptical side, too, though, in the sense that “it’s not really thinking” and “it’s a scam” and “it’s going to be used to rip you off” are all packaged together. And, like - it’s an already useful tool, that will inevitably have a lot of scams around it and will probably also be used in its real capacity as a tool to rip you off. The questions on the nature of intelligence and consciousness are interesting but I think really not critical to any of those practical concerns.
posted by atoxyl at 1:37 PM on May 2, 2023 [3 favorites]
Arguably they aren’t even that essential to the more speculative AI hazard scenarios.
posted by atoxyl at 1:45 PM on May 2, 2023 [1 favorite]
posted by atoxyl at 1:45 PM on May 2, 2023 [1 favorite]
Arguably we should be teaching scam avoidance, confidence game tactics and how to avoid them, and other social skills starting with young children.
posted by hippybear at 1:50 PM on May 2, 2023 [4 favorites]
posted by hippybear at 1:50 PM on May 2, 2023 [4 favorites]
Would it help if I jump to the end and reveal that nothing even remotely useful will ever arise from these theoretical discussions? It's such a colossal waste of time. I lived for this shit in the 80s, mostly because I was very callow. Maybe Metafilter, at least, could just pretend we've already done this to death and, instead, discuss observable behavior and the technical details underpinning what we're seeing? Without shackling ourselves to one philosophical viewpoint or another that's based upon vibes and hidden tautologies?
I wanted to respond to this, but I'm too lazy. So I copy-pasted it into Chat-GPT4 instead:
> It's understandable that you may feel that way about theoretical discussions, as they can sometimes seem abstract and removed from practical application. However, it's important to recognize that these discussions can play a crucial role in shaping our understanding of complex topics and, ultimately, informing the actions we take.
> Focusing on observable behavior and technical details is an essential part of the conversation, and it's entirely valid to prioritize these aspects. However, it's important not to dismiss theoretical discussions altogether, as they can offer unique insights, challenge existing beliefs, and inspire new ideas.
> As for engaging in discussions on Metafilter or any other platform, the key is to maintain a balanced approach. It's important to respect differing opinions and engage in open-minded dialogue while remaining focused on the aspects that interest you most. This way, you can contribute to the conversation in a meaningful way without feeling constrained by specific philosophical viewpoints or becoming overly frustrated by abstract debates.
Yeah, thanks Chat-GPT4, but that's mostly a bunch of platitudes and I'm a middle-aged man so I am also tired of these "deep, insightful" TED Talk-ish discussions, so I think I agree with the previous speaker. At best I see them as a necessary discourse to traverse so that you can grow into actual fruitful ideas, but I also see it as equally (if not more) likely that they're a waste of time.
> I understand your perspective, and it's true that not all theoretical discussions are equally fruitful. However, it's important to remember that people have different preferences and learning styles, and what may seem unproductive or uninteresting to you might be engaging and stimulating for others.
> If you find that certain discussions don't hold your interest or don
(At this point I clicked on Stop Generating.)
posted by AlSweigart at 2:39 PM on May 2, 2023 [1 favorite]
I wanted to respond to this, but I'm too lazy. So I copy-pasted it into Chat-GPT4 instead:
> It's understandable that you may feel that way about theoretical discussions, as they can sometimes seem abstract and removed from practical application. However, it's important to recognize that these discussions can play a crucial role in shaping our understanding of complex topics and, ultimately, informing the actions we take.
> Focusing on observable behavior and technical details is an essential part of the conversation, and it's entirely valid to prioritize these aspects. However, it's important not to dismiss theoretical discussions altogether, as they can offer unique insights, challenge existing beliefs, and inspire new ideas.
> As for engaging in discussions on Metafilter or any other platform, the key is to maintain a balanced approach. It's important to respect differing opinions and engage in open-minded dialogue while remaining focused on the aspects that interest you most. This way, you can contribute to the conversation in a meaningful way without feeling constrained by specific philosophical viewpoints or becoming overly frustrated by abstract debates.
Yeah, thanks Chat-GPT4, but that's mostly a bunch of platitudes and I'm a middle-aged man so I am also tired of these "deep, insightful" TED Talk-ish discussions, so I think I agree with the previous speaker. At best I see them as a necessary discourse to traverse so that you can grow into actual fruitful ideas, but I also see it as equally (if not more) likely that they're a waste of time.
> I understand your perspective, and it's true that not all theoretical discussions are equally fruitful. However, it's important to remember that people have different preferences and learning styles, and what may seem unproductive or uninteresting to you might be engaging and stimulating for others.
> If you find that certain discussions don't hold your interest or don
(At this point I clicked on Stop Generating.)
posted by AlSweigart at 2:39 PM on May 2, 2023 [1 favorite]
There are lots of people here (and, well, everywhere) a lot smarter than me and with a much better grasp of what all this means. My tiny brain keeps wondering, though, if we're making a really simple and fundamental error in the way we view the rise of AI and how we consider questions of whether it represents actual intelligence.
The error that worries me is that we constantly measure whether intelligence exists by comparing AI to the human brain as if that were the only way to define intelligence. I mean, it is exactly that in the sense that it is humans deciding what intelligence means from our perspective because that's the only one we have. But are we even correct in how we define intelligence in the first place? This thought has been keeping me awake at night of late.
posted by dg at 2:45 PM on May 2, 2023 [3 favorites]
The error that worries me is that we constantly measure whether intelligence exists by comparing AI to the human brain as if that were the only way to define intelligence. I mean, it is exactly that in the sense that it is humans deciding what intelligence means from our perspective because that's the only one we have. But are we even correct in how we define intelligence in the first place? This thought has been keeping me awake at night of late.
posted by dg at 2:45 PM on May 2, 2023 [3 favorites]
A long time ago, I heard a definition of “intelligence” as the ability to successfully deal with reality. I guess for simplicity sake we can assume reality to be your or its environment. Prey avoiding predators, predators catching prey, finding food and water, are simple examples. Since the environment is generally full of surprises, there must be some ability to deal with the unexpected or the novel, and being able to adapt/change, or maybe figure things out and then intentionally adapt/change. Flexibility in a nutshell.
My primary issue with the rise of AI has been the language used to describe it. Anthropomorphism all the way down. I recently heard an AI expert on NPR describe the output of ChatGPT as being “thoughtful.” I wanted to throw the radio out the window. A lot of the argument here revolves around the terms used to describe this stuff, and the implications of using these terms. Back in the good old days AI was about duplicating the way we think. That sort of didn’t get very far given that we don’t know how our own brains work. So things shifted down to duplicating the way neurons work, sort of. And, yes, they now got something working, sort of. And that’s called AI. So, now they claim machines learn, they understand, they know, etc. In my experience with computer people over many years, they have little or no background in philosophy or psychology, areas that have been dealing with these issues for centuries. But they do have a very naive understanding of a lot of the terms and they insist on using them in their own naive way. And thus all the arguments here and elsewhere.
posted by njohnson23 at 3:42 PM on May 2, 2023 [5 favorites]
My primary issue with the rise of AI has been the language used to describe it. Anthropomorphism all the way down. I recently heard an AI expert on NPR describe the output of ChatGPT as being “thoughtful.” I wanted to throw the radio out the window. A lot of the argument here revolves around the terms used to describe this stuff, and the implications of using these terms. Back in the good old days AI was about duplicating the way we think. That sort of didn’t get very far given that we don’t know how our own brains work. So things shifted down to duplicating the way neurons work, sort of. And, yes, they now got something working, sort of. And that’s called AI. So, now they claim machines learn, they understand, they know, etc. In my experience with computer people over many years, they have little or no background in philosophy or psychology, areas that have been dealing with these issues for centuries. But they do have a very naive understanding of a lot of the terms and they insist on using them in their own naive way. And thus all the arguments here and elsewhere.
posted by njohnson23 at 3:42 PM on May 2, 2023 [5 favorites]
In debates like this, I like to think back to Ken MacLeod's The Cassini Division (the book where the phrase "Rapture for Nerds" was coined). Humans are arguing with a post-human/uplifted machine intelligence.
'I can't understand how anybody ever fell for the idea that a computer model of the brain is the same as the brain. Talk about mechanical materialism! It's about becoming a machine, it's death, and wanting it is sick.'
...
'Brief enough for you: refusing to accept intelligent robots as people is equivalent to racism.'
'So?' says the man at my side. 'So I'm a racist. A human racist.'
...
'We are the top predator here. You can become machines if you like, but then you'll be dead, and we'll be alive, and we'll treat you as machines. If we can't use you, we'll smash you up!'
'If you can,' he says.
I look straight back at him. 'If we can.'
It's a great book, and part of a great series. Now part of the Divisions collection.
posted by meehawl at 3:50 PM on May 2, 2023
'I can't understand how anybody ever fell for the idea that a computer model of the brain is the same as the brain. Talk about mechanical materialism! It's about becoming a machine, it's death, and wanting it is sick.'
...
'Brief enough for you: refusing to accept intelligent robots as people is equivalent to racism.'
'So?' says the man at my side. 'So I'm a racist. A human racist.'
...
'We are the top predator here. You can become machines if you like, but then you'll be dead, and we'll be alive, and we'll treat you as machines. If we can't use you, we'll smash you up!'
'If you can,' he says.
I look straight back at him. 'If we can.'
It's a great book, and part of a great series. Now part of the Divisions collection.
posted by meehawl at 3:50 PM on May 2, 2023
Dg: cognitive science as a field was largely eaten by its older siblings (neuroscience, computer science, psychology, philosophy) years ago, in part because it failed to ever develop a canonical syllabus. We had a good thread on this about four years ago and I wrote a long comment that I think might address your question, at least somewhat, here.
Key bit:
I don't think there was ever much doubt in my mind or the minds of my fellow students about what we were trying to do: develop an abstract working model of *minds*, plural, with their generalized systems modeling (problem solving), compartmentalized internal state tracking and associativity (identity), goal-directed behavior (intentionality), and capacity for self-referential agent behavior prediction within those generalized models (group behaviors). A broader theoretical framework of which the human mind is but one instance, similar to how the x86 CPU architecture is but one instance of a Turing Machine (the abstract model for all devices which perform computation, if you're not from CompSci and made it this far).
I wish I had some reading suggestions but those should probably come from somebody who didn’t abandon the field twenty years ago. The first half of GEB still makes a great case for modeling and recursion/mirror-thinking as fundamental, at least.
posted by Ryvar at 4:09 PM on May 2, 2023 [1 favorite]
Key bit:
I don't think there was ever much doubt in my mind or the minds of my fellow students about what we were trying to do: develop an abstract working model of *minds*, plural, with their generalized systems modeling (problem solving), compartmentalized internal state tracking and associativity (identity), goal-directed behavior (intentionality), and capacity for self-referential agent behavior prediction within those generalized models (group behaviors). A broader theoretical framework of which the human mind is but one instance, similar to how the x86 CPU architecture is but one instance of a Turing Machine (the abstract model for all devices which perform computation, if you're not from CompSci and made it this far).
I wish I had some reading suggestions but those should probably come from somebody who didn’t abandon the field twenty years ago. The first half of GEB still makes a great case for modeling and recursion/mirror-thinking as fundamental, at least.
posted by Ryvar at 4:09 PM on May 2, 2023 [1 favorite]
Thanks njohnson23 and Ryvar. It's good to know that I'm not alone in wondering what intelligence really is and if the brightest minds in the field don't even know (even if they think they do), there's no reason why I should.
posted by dg at 6:52 PM on May 2, 2023
posted by dg at 6:52 PM on May 2, 2023
"I don't see how there could be thinking without a thinker." strikes me as a thought-terminating cliché
If it was design without a designer I'd agree with you, but "thinking without a thinker" seems to me to follow the general pattern of "action without an actor" and is therefore a fair bit more defensible.
Reacting to the discovery of some intricate and astonishingly functional object with the idea that there could be no design without a designer infers design based on that intricacy and/or function, then imputes that design to a designer at least conceptually separable from that which was putatively designed. Using a thought-terminating cliche to prop up such an inherently dubious collection of assumptions therefore becomes a fairly natural rhetorical move.
But action, unlike design, is directly perceptible in and of itself rather than being an attribute of some other perceptible object. Perceiving an action doesn't infer action so much as observe it, which makes the action/actor distinction more a matter of convenient labelling than anything else. The same observation that identifies an action also serves to identify the actor performing that action. No two-step inference is required.
To the extent that thinking is an observable action, then, positing thinking without a thinker strikes me as an attempt to create mystery where none need exist.
Who's to say our biology didn't evolve in the fashion of a sophisticated LLM?
Certainly not me, especially given the heavy and obvious influence of biology on the design of such models. There would have been no path to building these things without some suspicion that they are indeed somewhat reflective of apects of the way our undoubtedly evolved selves perform similar tasks.
I think one of the main things that muddies so much of the philosophical analysis of LLMs is that the neural networks underlying them are software constructs rather than being inextricably tied to one particular chunk of matter such as a specific human body. The idea that a LLM's existence could be threatened by powering it off is just wrong, because everything required to make it work not only can be, but routinely is, replicated exactly across multiple kinds of physical substrate. The neurons and connection patterns and connection weights for an LLM are accessible and amenable to exact replication in hardware-insensitive ways that biological neural networks simply cannot be.
We won't see actually killable engineered experience engines until we get to the point where instead of training up replicable neural networks on large, consistent snapshots of pre-existing data, we're training them continuously on real-time sensor inputs that are not replicated across instances. Only after Siri moves wholly to the edge could dropping her in the toilet even begin to have anything even vaguely resembling moral consequences.
I suspect our collective behavior is going to look fucking barbaric in a couple centuries.
Why wait? Much of our collective behaviour is fucking barbaric right now.
posted by flabdablet at 7:43 PM on May 2, 2023 [3 favorites]
If it was design without a designer I'd agree with you, but "thinking without a thinker" seems to me to follow the general pattern of "action without an actor" and is therefore a fair bit more defensible.
Reacting to the discovery of some intricate and astonishingly functional object with the idea that there could be no design without a designer infers design based on that intricacy and/or function, then imputes that design to a designer at least conceptually separable from that which was putatively designed. Using a thought-terminating cliche to prop up such an inherently dubious collection of assumptions therefore becomes a fairly natural rhetorical move.
But action, unlike design, is directly perceptible in and of itself rather than being an attribute of some other perceptible object. Perceiving an action doesn't infer action so much as observe it, which makes the action/actor distinction more a matter of convenient labelling than anything else. The same observation that identifies an action also serves to identify the actor performing that action. No two-step inference is required.
To the extent that thinking is an observable action, then, positing thinking without a thinker strikes me as an attempt to create mystery where none need exist.
Who's to say our biology didn't evolve in the fashion of a sophisticated LLM?
Certainly not me, especially given the heavy and obvious influence of biology on the design of such models. There would have been no path to building these things without some suspicion that they are indeed somewhat reflective of apects of the way our undoubtedly evolved selves perform similar tasks.
I think one of the main things that muddies so much of the philosophical analysis of LLMs is that the neural networks underlying them are software constructs rather than being inextricably tied to one particular chunk of matter such as a specific human body. The idea that a LLM's existence could be threatened by powering it off is just wrong, because everything required to make it work not only can be, but routinely is, replicated exactly across multiple kinds of physical substrate. The neurons and connection patterns and connection weights for an LLM are accessible and amenable to exact replication in hardware-insensitive ways that biological neural networks simply cannot be.
We won't see actually killable engineered experience engines until we get to the point where instead of training up replicable neural networks on large, consistent snapshots of pre-existing data, we're training them continuously on real-time sensor inputs that are not replicated across instances. Only after Siri moves wholly to the edge could dropping her in the toilet even begin to have anything even vaguely resembling moral consequences.
I suspect our collective behavior is going to look fucking barbaric in a couple centuries.
Why wait? Much of our collective behaviour is fucking barbaric right now.
posted by flabdablet at 7:43 PM on May 2, 2023 [3 favorites]
False-belief tasks are just logic problems involving people, that even children understand. Right? We already knew neural networks are powerful enough to play Go. The fact that a different kind of neural network can reason logically about the contents of people's thoughts is just winning at another kind of logic game.
Because so much psychology is already embedded in human language, it should not be surprising that an LLM "learns" to reason somewhat using a theory of mind (or an approximation thereof), that could be better than its theory of arithmetic, apparently.
Another question I have is, what does success rate metric on false-belief tasks mean? If someone manually wrote an algorithm that achieved 100% on those tests, does that mean that algorithm also has a theory of mind?
posted by polymodus at 12:12 AM on May 3, 2023
Because so much psychology is already embedded in human language, it should not be surprising that an LLM "learns" to reason somewhat using a theory of mind (or an approximation thereof), that could be better than its theory of arithmetic, apparently.
Another question I have is, what does success rate metric on false-belief tasks mean? If someone manually wrote an algorithm that achieved 100% on those tests, does that mean that algorithm also has a theory of mind?
posted by polymodus at 12:12 AM on May 3, 2023
Sabine Hossenfelder (linked above) makes the point that no purely inputs/outputs black-box test can ever be definitive about whether or not there's a mind between the inputs and outputs, since any such test is at least in principle capable of being passed by nothing more complicated than a lookup table. I don't think you'd find many who would agree that a lookup table, no matter how extensive or how efficiently indexed, could possibly qualify as conscious.
I agree with her. It seems to me that if we do get to the point of being able to declare with reasonable confidence that some engineered structure has consciousness, we'd need to be making that assertion based on identifiable analogies between its operation and that of the existing structures to which we already impute consciousness.
Since the only extant structure that's uncontroversially conscious is us, testing for consciousness in novel structures is going to require that we widen our definition of consciousness and/or hugely improve our understanding of our own workings. The latter seems to me to be a reasonable use of time for human beings, and to the extent that LLM research assists with that project, I'm for it.
Not for it enough that I would ever choose to rely on a Siri or Cortana or Alexa or Bing to get me through my day, though. Always-on corporate audio surveillance can fuck right off.
posted by flabdablet at 1:06 AM on May 3, 2023 [1 favorite]
I agree with her. It seems to me that if we do get to the point of being able to declare with reasonable confidence that some engineered structure has consciousness, we'd need to be making that assertion based on identifiable analogies between its operation and that of the existing structures to which we already impute consciousness.
Since the only extant structure that's uncontroversially conscious is us, testing for consciousness in novel structures is going to require that we widen our definition of consciousness and/or hugely improve our understanding of our own workings. The latter seems to me to be a reasonable use of time for human beings, and to the extent that LLM research assists with that project, I'm for it.
Not for it enough that I would ever choose to rely on a Siri or Cortana or Alexa or Bing to get me through my day, though. Always-on corporate audio surveillance can fuck right off.
posted by flabdablet at 1:06 AM on May 3, 2023 [1 favorite]
It took until 2012 for the Cambridge Declaration on Consciousness (pdf, about consciousness in non-human animals), based on similar analogical-structure reasons. So I'm going to assume that in the imaginary sci-fi future where AI has gained consciousness it will be denied for as long as possible, for reasons of profit.
posted by Pyrogenesis at 3:07 AM on May 3, 2023 [1 favorite]
posted by Pyrogenesis at 3:07 AM on May 3, 2023 [1 favorite]
I don't think you'd find many who would agree that a lookup table, no matter how extensive or how efficiently indexed, could possibly qualify as conscious.
Devil's advocate: If consciousness could have such a trivial algorithm (such as this exponentially large lookup table), then the problem is the definition of consciousness. Maybe it's just a bad concept.
posted by polymodus at 3:34 AM on May 3, 2023 [1 favorite]
Devil's advocate: If consciousness could have such a trivial algorithm (such as this exponentially large lookup table), then the problem is the definition of consciousness. Maybe it's just a bad concept.
posted by polymodus at 3:34 AM on May 3, 2023 [1 favorite]
Or maybe, just like every other concept, it has arguable edge cases. A heap doesn't stop being a useful classification for no better reason than difficulty in deciding exactly what constitutes a minimal heap.
posted by flabdablet at 4:33 AM on May 3, 2023
posted by flabdablet at 4:33 AM on May 3, 2023
This makes me laugh a bit bitterly as a psychology grad school dropout who did experimental research testing if early neural net models emergent properties were found in people almost thirty years ago (They were btw at least in my specific narrow domain of semantic memory). Even back then the early neural net models were inscrutable and black-boxy.
I also saw Hinton give a talk at U of Toronto around that time and he explained his moving away from the US as a desire to not have his work be used to guide missiles.
posted by srboisvert at 5:06 AM on May 3, 2023 [4 favorites]
I also saw Hinton give a talk at U of Toronto around that time and he explained his moving away from the US as a desire to not have his work be used to guide missiles.
posted by srboisvert at 5:06 AM on May 3, 2023 [4 favorites]
hippybear - honestly I think we ought to be hammering deontological ethics and the categorical imperative into them at every single stage of the process.
I'm not even joking a little bit.
posted by Baby_Balrog at 7:35 AM on May 3, 2023 [1 favorite]
I'm not even joking a little bit.
posted by Baby_Balrog at 7:35 AM on May 3, 2023 [1 favorite]
If there was any doubt previously — Sabine Hossenfelder's equivocating contrarianism about trans rights (especially and tellingly now) prove she should be ignored. If being a contrarian who styles themselves as a "public intellectual" merely because of their credentials as a physicist wasn't enough reason.
Wait, let me check in with my close friend the particle physicist ... nope, he says she's a bit of a crank. Um, maybe my ex who's an astrophysicist? Um, also not impressed. That's from other physicists!
But, sure, go ahead and listen to what she has to say about AI and trans rights because reasons.
posted by Ivan Fyodorovich at 11:06 AM on May 3, 2023 [5 favorites]
Wait, let me check in with my close friend the particle physicist ... nope, he says she's a bit of a crank. Um, maybe my ex who's an astrophysicist? Um, also not impressed. That's from other physicists!
But, sure, go ahead and listen to what she has to say about AI and trans rights because reasons.
posted by Ivan Fyodorovich at 11:06 AM on May 3, 2023 [5 favorites]
Note: I shouldn't be speaking publicly on behalf of my friends, and I'd have asked the mods to delete the previous comment, but I'm pretty upset that Hossenfelder has given any credibility to the transphobes. Her contrarianist schtick has bugged me because, honestly, the contrarians have produced so much destructive crap during my lifetime that I'm actually angry at myself that I once regularly read far more of those folk than I want to admit. And here we see yet again how those with this habit of thought find themselves carrying water for the very worst people.
Also, the "it's just statistics" line is so facile. It really is. There's probably something worth investigating how it is that some of the most reductive takes (about anything) strike so many people as wisdom.
Note that I've not been reductive about all these deep questions about consciousness — there are any number of fundamental problems that I've lost interest in not because they're simple, but because they're so intractably complex and difficult that it's not even certain that any of the words we're using to talk about them mean anything. That's pretty common in philosophy and while I really love philosophy and loved studying it in school, I'm pretty much in the camp that's too pragmatic to pound against obviously intractable problems indefinitely.
posted by Ivan Fyodorovich at 12:59 PM on May 3, 2023 [3 favorites]
Also, the "it's just statistics" line is so facile. It really is. There's probably something worth investigating how it is that some of the most reductive takes (about anything) strike so many people as wisdom.
Note that I've not been reductive about all these deep questions about consciousness — there are any number of fundamental problems that I've lost interest in not because they're simple, but because they're so intractably complex and difficult that it's not even certain that any of the words we're using to talk about them mean anything. That's pretty common in philosophy and while I really love philosophy and loved studying it in school, I'm pretty much in the camp that's too pragmatic to pound against obviously intractable problems indefinitely.
posted by Ivan Fyodorovich at 12:59 PM on May 3, 2023 [3 favorites]
Or maybe, just like every other concept, it has arguable edge cases. A heap doesn't stop being a useful classification for no better reason than difficulty in deciding exactly what constitutes a minimal heap.
Heap is a vague concept, but not a potentially bad concept in the same way. Whereas supposing a LUT could pass a naive implementation of the Turing test, it does beg the question, because then we would have a trivial algorithm for consciousness. I.e. did we misunderstand consciousness? Are stones and trees conscious too? E.g. then we ought to revise our concept of consciousness.
Hossenfelder's thought experiment doesn't make sense to me anyways because if a LUT sufficed then in principle it would require a black hole's worth of information density and/or require that P = NP or something even more strange and magical.
posted by polymodus at 1:01 PM on May 3, 2023
Heap is a vague concept, but not a potentially bad concept in the same way. Whereas supposing a LUT could pass a naive implementation of the Turing test, it does beg the question, because then we would have a trivial algorithm for consciousness. I.e. did we misunderstand consciousness? Are stones and trees conscious too? E.g. then we ought to revise our concept of consciousness.
Hossenfelder's thought experiment doesn't make sense to me anyways because if a LUT sufficed then in principle it would require a black hole's worth of information density and/or require that P = NP or something even more strange and magical.
posted by polymodus at 1:01 PM on May 3, 2023
supposing a LUT could pass a naive implementation of the Turing test, it does beg the question, because then we would have a trivial algorithm for consciousness. I.e. did we misunderstand consciousness?
No, all it would mean is that we've made up a test that doesn't test what we think it does. Which, considering we don't actually have a non-controversial referent for "consciousness", is pretty much the expected result.
if a LUT sufficed then in principle it would require a black hole's worth of information density
I don't think so. All it would need to look up is canned answers to all our test questions that were good enough to convince the person running the test.
This isn't even entirely an "in principle" thing; there's history to support it. Eliza, which was just a lookup table, worked well enough that quite a number of testers were willing to impute consciousness to it.
posted by flabdablet at 1:12 PM on May 3, 2023
No, all it would mean is that we've made up a test that doesn't test what we think it does. Which, considering we don't actually have a non-controversial referent for "consciousness", is pretty much the expected result.
if a LUT sufficed then in principle it would require a black hole's worth of information density
I don't think so. All it would need to look up is canned answers to all our test questions that were good enough to convince the person running the test.
This isn't even entirely an "in principle" thing; there's history to support it. Eliza, which was just a lookup table, worked well enough that quite a number of testers were willing to impute consciousness to it.
posted by flabdablet at 1:12 PM on May 3, 2023
"Maybe we are now reaching a point where the language of psychology is starting to be appropriate to understand the behaviour of these neural networks."To address the pull quote directly, I think it is useful to use words like "think" and "understand" when describing what LLMs do, although obviously with the caveat that these words don't apply in the same way as they do to human beings. We already use lots of cognitive metaphors when describing computing technology: "memory", "read/write", "smartchip" and so on. When describing systems it's also useful to talk about a system "not knowing about x" or "not having a concept of y" if certain data is not stored in it and the business rules don't handle it, instead of using an awkward circumlocution like "the engineers who designed the system did not consider x or y".
There are aspects of how transformer neural networks work that IMHO can be usefully described as a machine analogue of understanding. "Understanding" in this context means taking input in the form of a word embedding vector where similar concepts expressed in different ways can have multiple very different representations and then transforming it into a vector space where semantically similar words and sentences are closer together. This is where a lot of the magic of LLMs comes from: it's all very well to see it "regurgitate" an answer and say "well, it probably saw something similar in it's training data", but: similar in what way? Leaving aside that it doesn't work by looking things up in some kind of database, how does it know that two differently worded paragraphs are describing a similar situation? And how does it know what to pay attention to when evaluating the similarity? It's obviously not the equivalent of some simple keyword search.
When you compare ChatGPT to ELIZA (which is it absurdly being likened to), there is almost nothing in common about how they work. ELIZA just gets people to read intelligence into its responses by making vague and ambiguous statements. ChatGPT might be often factually wrong, but it's usually explicit and direct in it's answers, and often gives extended justifications for them, when it gets it right it's not just making some vaguely worded statement that you can read into what you like. We need to have graduations in language to describe these differences.
posted by L.P. Hatecraft at 1:47 PM on May 3, 2023
There are aspects of how transformer neural networks work that IMHO can be usefully described as a machine analogue of understanding. "Understanding" in this context means taking input in the form of a word embedding vector where similar concepts expressed in different ways can have multiple very different representations and then transforming it into a vector space where semantically similar words and sentences are closer together. [...] We need to have graduations in language to describe these differences.
Theoretically, I agree with you. And I'd love for this conversation to reach the grounded, sane place where "understanding" can be used like that. But we're still at the place where a lot of billionaires who really ought to know better are making up absolutely nonsense claims about what ChatGPT is doing, because they're incapable of separating reasonable interpretations of how ChatGPT does what it does from all the bullshit cyberpunk they read when they were teens.
If you're not plugged in to Silicon Valley culture, it's hard to convey just how many people in that world literally center their religion around "AI." They've been dreaming of The Singularity for half their lives, if not longer. They've been reinventing some pretty basic religious tropes with "superintelligent computer" in place of God. And they're trying harder to see the sentient machine that they believe will finally grant them eternal life than your average "Christ in a bit of water damage" enjoyer. Some of them genuinely believe in this fanatic, dogmatic shit, in the same way that some young-Earth creationists have master's degrees from Liberty University. Some of them are flat-out bullshitting, and couldn't care less whether the claims they're making are true or not. But the press has been absolutely dogshit about this, is repeating garbage specious claims like they actually mean something, and AI sensationalism has made it all-but-impossible to distinguish the actual interesting conversation we could be happening from the deranged wish-fulfillment ramblings.
Personally, I thought that Wolfram's essay on AI did a great job of distinguishing the parts of machine learning that are really really neat from the challenges inherent to "thinking and understanding" that computers are likely never going to solve—not in a "what even IS consciousness, man" kind of way, but in a pragmatic "machines will never even be able to fake this" sense. In a perfect world, that essay would be close to the end-all be-all of this conversation. But instead, AI has drawn in the NFT grifters, the crypto nuts, the libertarian technofascists, and people are swallowing all the bullshit they say, because few people actually have worse functional critical-thinking facilities than the so-called rationalists who talk about critical thinking 24/7.
So: yes, I'd love to be in a place where we use "understanding" to refer to computational analysis, in the same way that we use "memory" to refer to storage. But at present, words like "understanding" are being used in ways that absolutely abuse the English language, and we haven't worked out how to talk about this without everybody flying off the goddamn rails.
posted by Tom Hanks Cannot Be Trusted at 2:12 PM on May 3, 2023 [8 favorites]
Theoretically, I agree with you. And I'd love for this conversation to reach the grounded, sane place where "understanding" can be used like that. But we're still at the place where a lot of billionaires who really ought to know better are making up absolutely nonsense claims about what ChatGPT is doing, because they're incapable of separating reasonable interpretations of how ChatGPT does what it does from all the bullshit cyberpunk they read when they were teens.
If you're not plugged in to Silicon Valley culture, it's hard to convey just how many people in that world literally center their religion around "AI." They've been dreaming of The Singularity for half their lives, if not longer. They've been reinventing some pretty basic religious tropes with "superintelligent computer" in place of God. And they're trying harder to see the sentient machine that they believe will finally grant them eternal life than your average "Christ in a bit of water damage" enjoyer. Some of them genuinely believe in this fanatic, dogmatic shit, in the same way that some young-Earth creationists have master's degrees from Liberty University. Some of them are flat-out bullshitting, and couldn't care less whether the claims they're making are true or not. But the press has been absolutely dogshit about this, is repeating garbage specious claims like they actually mean something, and AI sensationalism has made it all-but-impossible to distinguish the actual interesting conversation we could be happening from the deranged wish-fulfillment ramblings.
Personally, I thought that Wolfram's essay on AI did a great job of distinguishing the parts of machine learning that are really really neat from the challenges inherent to "thinking and understanding" that computers are likely never going to solve—not in a "what even IS consciousness, man" kind of way, but in a pragmatic "machines will never even be able to fake this" sense. In a perfect world, that essay would be close to the end-all be-all of this conversation. But instead, AI has drawn in the NFT grifters, the crypto nuts, the libertarian technofascists, and people are swallowing all the bullshit they say, because few people actually have worse functional critical-thinking facilities than the so-called rationalists who talk about critical thinking 24/7.
So: yes, I'd love to be in a place where we use "understanding" to refer to computational analysis, in the same way that we use "memory" to refer to storage. But at present, words like "understanding" are being used in ways that absolutely abuse the English language, and we haven't worked out how to talk about this without everybody flying off the goddamn rails.
posted by Tom Hanks Cannot Be Trusted at 2:12 PM on May 3, 2023 [8 favorites]
Tom Hanks Cannot Be Trusted - 👍
posted by njohnson23 at 2:32 PM on May 3, 2023
posted by njohnson23 at 2:32 PM on May 3, 2023
Expounding on my criticisms of AI Discourse more than I know I should:
Machines are very good at doing mechanical things. It's easy to comprehend that when we're talking about physical machinery: of course a specialized thing-lifter than lift things that are heavier than a human can easily lift. Machines can be built bigger and made of heavier, stronger materials. Of course they can do some things better than we can.
Similarly, a printing press can put ink on paper more consistently and more quickly than a person can. That forms the basis of mechanical reproduction, after all. It can't do calligraphy, but a laser printer can fake it. Writing is generally seen as a human activity, but printing that writing can be seen as a mechanical process.
The reason why LLMs are screwing with people's heads so much can be boiled down to two basic things:
Machines are very good at doing mechanical things. It's easy to comprehend that when we're talking about physical machinery: of course a specialized thing-lifter than lift things that are heavier than a human can easily lift. Machines can be built bigger and made of heavier, stronger materials. Of course they can do some things better than we can.
Similarly, a printing press can put ink on paper more consistently and more quickly than a person can. That forms the basis of mechanical reproduction, after all. It can't do calligraphy, but a laser printer can fake it. Writing is generally seen as a human activity, but printing that writing can be seen as a mechanical process.
The reason why LLMs are screwing with people's heads so much can be boiled down to two basic things:
- It is possible to tell a computer to try and understand the mechanics behind something, without specifying what that "something" is and without offering hints about the mechanics in question. Given enough computing power, computers can sometimes do a decent job about this.
- Far more things can be boiled down to "mechanical process" than a lot of people realized.
-
Computers can figure out what "photorealistic painting" means, on a pixel-by-pixel level. They can recognize the form of a college essay. They can recognize the rhetorical shape of a debate, and work out what pieces generally plug together. And they can abstract out various phrasings and sentence constructs to get an idea of The Concept that those sentences are constructing.
They don't understand what the concept "is," or even what a "concept" is. But they can recognize these layers of patterns, and they can mimic them. They can even construct new layered patterns, combining different patterns together in a new way, if they've been instructed to. (You could even go a layer more meta and ask the computer to try and figure out unusual sequences of patterns, then construct them for us.)
This is still mimicry—mechanical reproduction—at its core. It's much more sophisticated than a printing press. But one of the rhetorical divides that I'm noticing in the layperson discussion about AI boils down to the fact that some people already knew that these patterns existed, and were dissectible, whereas it's blowing other people's minds that "sonnet" or "argument about X" can be broken down into basic enough constituent components that a computer can do things with them.
For me, it's telling that a lot of the things we're asking ChatGPT to do are among the most rudimentary forms of rhetoric. We ask it for small talk, online-debate texts, pastiches of writers with distinctive styles. It can mimic poetic forms with distinct rules; it can convert anything into a knock-knock joke or format it to look like a recipe. It can do these things because these things have obvious patterns. Similarly, Stable Diffusion has an advantage in impressing us with its images because most people dream in pastiche: computers are going to be best at doing "X but Y"-style generations, where X and Y are both recognizable tropes. And yes, some people have gone down the rabbit hole of "X and Y done up like Z in Q format, from T era, by painter J," but it's still more-or-less the same thing. Computers specialize in chains like that. That's literally what defines algorithms.
The neat part is that LLMs don't need to have X and Y and Z and Q and Y and J preprogrammed for them. They can follow a general rule for developing those patterns on their own. To paraphrase Ted Chiang, they can see the blurry jpeg. And the more blurry that jpeg is, the better—because they need those abstractions in order to function properly.
This is super cool. And I think that it'll be a tremendous boon for creative people all around the world—not because "the computer is now an artist," but because the computer just became a much more powerful creative tool. We're literally more than a century past the point where Duchamp declared that art must be rooted in interesting concepts, and that the seed of any creative work is the idea which originated it—in other words, that the most valuable thing about art is the non-mechanical part, the part that computers literally can't do. (If anything, if you ask computers to come up with ideas for art, they'll spit out the kinds of ideas that are so banal that... computers could come up with them. Which I kinda like, actually.)
But there's also this reductive idea that consciousness—that the whole of reality—can be reduced down to mechanistic pattern. You see it reflected in the way that the Elon Musks of the world are convinced that we're living in a simulation. Christopher Alexander, who hated this idea, used to rail at Rene Descartes for introducing it to the world, because Descartes thought that reality could hypothetically be broken down into one vast complex model (or at least he wrote in a way that hinted at that possibility). And that's where the AI discourse turns absolutely dumb as shit.
Stephen Wolfram, whose whole deal is that he's pioneered new complex ways for computers to model information, explains exactly why this is so dumb. In short: computers can reduce things down to atomic concepts, yes, but only to the extent that they understand certain kinds of atoms. Because those "irreducible" concepts, however, simple, can't be plugged into some general-abstraction system and reduced to existing notions. They can't be reduced. And there are a staggering amount of irreducible concepts. You can try plugging them into a computer one by one, but you'll inevitably come up short. And every one of those irreducibilities is going to make it harder for computers to work, it's going to take longer, it's going to require more processing power, and it's going to introduce a higher likeliness of computers fucking up things that, to us, feel incredibly basic.
The AI evangelists handwave that away by going, well, one way we'll figure out how to abstract those impossible-to-abstract things away too. But they can't. They're literally talking about the thing machines can't do. Because a whole lot can be reduced down to mechanical concept, sure—much more than some people realize—but there's a vast chasm between "a whole lot" and "literally everything."
So yes: LLMs can do insanely neat things, some of which are scary. And LLMs can do a lot more than we've seen. At the same time, LLMs have pretty basic limits—and some of those limits are a lot steeper than most people seem to think, because we haven't learned how to recognize those limits just yet. We'll find that, as with any machine, LLMs are fantastic at doing the things they're able to do, and they flat-out can't do the rest of it.
The Discourse right now is pulling a fast one on us, because it knows that we can't differentiate the things that genuinely are just early-stage "AI" glitches that'll be patched two years from now, and the things that aren't glitches so much as impossibilities. And it's doing that, not because it's starry-eyed about a future in which Conscious Machines exist, but because it thinks of us as wallets attached to dipshits and it would love to use our gullibility to further its plans to bleed us dry. (The True Believers want that too, only they think that GodOS is going to recognize them for their faith and grant them their dearest dreams, which they think is utopia for humankind but is a lot closer to "torturing their high school bullies for eternity.")
The AI evangelists handwave that away by going, well, one way we'll figure out how to abstract those impossible-to-abstract things away too. But they can't. They're literally talking about the thing machines can't do. Because a whole lot can be reduced down to mechanical concept, sure—much more than some people realize—but there's a vast chasm between "a whole lot" and "literally everything."
This is also why there will never be a useful Theory Of Everything.
There just are genuine limits to the extent to which Everything can be compressed.
posted by flabdablet at 6:16 AM on May 4, 2023 [2 favorites]
This is also why there will never be a useful Theory Of Everything.
There just are genuine limits to the extent to which Everything can be compressed.
posted by flabdablet at 6:16 AM on May 4, 2023 [2 favorites]
"We Have No Moat, And Neither Does OpenAI"
posted by kaibutsu at 12:39 PM on May 4, 2023 [1 favorite]
posted by kaibutsu at 12:39 PM on May 4, 2023 [1 favorite]
“They're literally talking about the thing machines can't do.”
This is a classic petitio principii fallacy. You offer no definition of machine, but your implied definition obviously includes "something that can't do the thing".
“This is also why there will never be a useful Theory Of Everything.
There just are genuine limits to the extent to which Everything can be compressed.”
C'mon, guy. That's not what ToE is. ToE is just the hypothetical unification of the the major physics theories, in particular QM and relativity.
Stuff like this just sows confusion.
posted by Ivan Fyodorovich at 2:28 PM on May 4, 2023 [2 favorites]
This is a classic petitio principii fallacy. You offer no definition of machine, but your implied definition obviously includes "something that can't do the thing".
“This is also why there will never be a useful Theory Of Everything.
There just are genuine limits to the extent to which Everything can be compressed.”
C'mon, guy. That's not what ToE is. ToE is just the hypothetical unification of the the major physics theories, in particular QM and relativity.
Stuff like this just sows confusion.
posted by Ivan Fyodorovich at 2:28 PM on May 4, 2023 [2 favorites]
Incidentally, ridicule the Silicon Valley singularity weirdos all you want — they deserve it. But if your reasoning is as ill-informed and sloppy as theirs, then you're damaging the credibility of your critique and I care a lot about that because far too many people accept their bullshit.
posted by Ivan Fyodorovich at 2:33 PM on May 4, 2023 [3 favorites]
posted by Ivan Fyodorovich at 2:33 PM on May 4, 2023 [3 favorites]
C'mon, guy. That's not what ToE is. ToE is just the hypothetical unification of the the major physics theories, in particular QM and relativity.
I would be interested in any reasoning you have access to that offers sound support for the position that some such hypothetical unification is likely to be achieved or even in principle achievable.
That it is is clearly a pleasing hypothesis to loads of people, many of whom are across both QM and relativity in far greater detail than I am, but personally I have no confidence in it.
posted by flabdablet at 7:42 PM on May 4, 2023
I would be interested in any reasoning you have access to that offers sound support for the position that some such hypothetical unification is likely to be achieved or even in principle achievable.
That it is is clearly a pleasing hypothesis to loads of people, many of whom are across both QM and relativity in far greater detail than I am, but personally I have no confidence in it.
posted by flabdablet at 7:42 PM on May 4, 2023
But one of the rhetorical divides that I'm noticing in the layperson discussion about AI boils down to the fact that some people already knew that these patterns existed, and were dissectible, whereas it's blowing other people's minds that "sonnet" or "argument about X" can be broken down into basic enough constituent components that a computer can do things with them. For me, it's telling that a lot of the things we're asking ChatGPT to do are among the most rudimentary forms of rhetoric.To me, the most impressive thing about ChatGPT isn't that it can mimick the style of various forms of writing, or even the fact that it can learn these styles by itself without being explicitly programmed, it's that in a certain sense it knows what "style" itself is. Style transfer has been around for a while, but while those networks modelled style information, they were explicitly programmed to do the style transference task, whereas GPT3 is modelling both the style information and the meaning of the word "style" such that it knows what it is about Y you want to apply to X when you ask it for "X in the style of Y". In other words it's not so much that it can do the task, or even how well it can do the task, it's that it understands what the task it's being asked to do is in the first place. This is true of lots of the things it can do. And, is it even easy to define what "style" is, other than in terms of vague almost-synonyms? If you had made your argument about "irreducible concepts" to me 5 years ago and used the concept of "style" as an example, I would have agreed with you.
Another thing about mimicry is that it's not really a comment on capability but on authenticity. It's still doing the thing even if it's not the Real McCoy. And it becomes less meaningful when the thing being mimicked is a task and not some kind of signalling or display. If a transformer has been trained to play Othello, when it plays a game with you, is it mimicking a human playing a game of Othello with you, or is it just playing a game of Othello with you?
In Stephen Wolfram's article about ChatGPT, he mentions irreducible computational complexity, this is different to an "irreducible" concept. There is a lot of nonsense from the kind of people you alluded to earlier about how superintelligent AI is supposedly going to cure all diseases and make humans immortal in the next 10 years by simulating every cell in the human body and designing wonder drugs from these simulations, and other wild ideas. This strikes me as something where irreducible computational complexity would be an issue, although of course there are many other reasons to be skeptical. "Irreducible concepts" on the other hand seems an awful lot like the "irreducible complexity" of Intelligent Design. If human minds evolved through a process of gradual natural and cultural evolution to higher levels of complexity though a series of incremental steps, and these ideas are irreducible, how did we come to have them? It also reminds me of how some people said that computer vision would never work because a computer could only ever look at individual pixels and could never perceive the "gestalt" of an image.
A lot of these "AI will never be able to do this" arguments remind me of the Surprise Execution paradox. Arguments that sound good, until they get disproven by dramatic counter-demonstration, and even then it's not obvious exactly how they were wrong.
posted by L.P. Hatecraft at 3:56 AM on May 5, 2023 [3 favorites]
So, I was not expecting to find my favorite sumo blogger going all Captain Kirk on ChatGPT like it was Nomad today,
but here we are.
posted by ursus_comiter at 5:24 PM on May 5, 2023 [1 favorite]
but here we are.
posted by ursus_comiter at 5:24 PM on May 5, 2023 [1 favorite]
no purely inputs/outputs black-box test can ever be definitive about whether or not there's a mind between the inputs and outputs, since any such test is at least in principle capable of being passed by nothing more complicated than a lookup table.
But that's just not true, and it's the reason Searle's thought experiment doesn't make sense. If you could make a lookup table like that, you wouldn't need any of this LLM stuff.
It's not enough to make a list of every possible question and a corresponding answer, you have to have a list of every possible conversation and corresponding responses that would be correct at each point in every possible conversation, with no limit on how long the conversation can be. It's infinite. It can't be done even in principle.
posted by straight at 8:45 PM on May 6, 2023 [2 favorites]
But that's just not true, and it's the reason Searle's thought experiment doesn't make sense. If you could make a lookup table like that, you wouldn't need any of this LLM stuff.
It's not enough to make a list of every possible question and a corresponding answer, you have to have a list of every possible conversation and corresponding responses that would be correct at each point in every possible conversation, with no limit on how long the conversation can be. It's infinite. It can't be done even in principle.
posted by straight at 8:45 PM on May 6, 2023 [2 favorites]
Apparently that's not the case. Apparently it's already possible to fool a hell of a lot of black-box testers a hell of a lot of the time by working off only the most recent 8,192 tokens within any given "conversation" plus the current state of a random number generator.
Given some list of tokens and some random number generator value, a LLM emits the next token in a completely deterministic fashion. In principle, that straightforward mapping from inputs to output could be captured as a lookup table. In practice, doing so would require more memory than there are atoms in the observable universe, but in principle? Absolutely.
posted by flabdablet at 5:57 AM on May 7, 2023
Given some list of tokens and some random number generator value, a LLM emits the next token in a completely deterministic fashion. In principle, that straightforward mapping from inputs to output could be captured as a lookup table. In practice, doing so would require more memory than there are atoms in the observable universe, but in principle? Absolutely.
posted by flabdablet at 5:57 AM on May 7, 2023
So, I was not expecting to find my favorite sumo blogger going all Captain Kirk on ChatGPT like it was Nomad today,
but here we are.
This is good:
Toward the end of this chat, the app reveals what I consider to be a fatal flaw: its inability to trace data back to its source. Cynics would probably point to its developers’ desire to avoid paying royalties. But even Wikipedia cites its sources. We know Wikipedia was one of its sources but it refuses to provide specific sources for the statistics or opinions it provides. Beyond giving credit where credit is due, what you need is something that will check and validate those sources. This app can’t even back up basic data which it used in its own argument.
posted by Artw at 6:01 AM on May 7, 2023
but here we are.
This is good:
Toward the end of this chat, the app reveals what I consider to be a fatal flaw: its inability to trace data back to its source. Cynics would probably point to its developers’ desire to avoid paying royalties. But even Wikipedia cites its sources. We know Wikipedia was one of its sources but it refuses to provide specific sources for the statistics or opinions it provides. Beyond giving credit where credit is due, what you need is something that will check and validate those sources. This app can’t even back up basic data which it used in its own argument.
posted by Artw at 6:01 AM on May 7, 2023
Citations are a pretty clear area where OpenAI - and the Bing integration - got in front of their skis, and kinda forced everyone else to do the same. Piles of people are working on it, and basically everybody has a deadbeat nephew launching an LLM search startup right now.
One of the key tools/results in this space is RETRO, which DeepMind published in 2021. It uses an external database for all of the knowledge-lookups, which lets you a) separate knowledge from language-manipulation (allowing a 25x reduction in model size without a quality drop), and b) you can easily see where it's pulled a particular piece of information from.
Here's GPT-4's proposal for a NeurIPs abstract for a paper improving LLM citations. I'll put it here and we can look back in a couple months and see how it compares to actual NeurIPs submissions. :)
One of the key tools/results in this space is RETRO, which DeepMind published in 2021. It uses an external database for all of the knowledge-lookups, which lets you a) separate knowledge from language-manipulation (allowing a 25x reduction in model size without a quality drop), and b) you can easily see where it's pulled a particular piece of information from.
Here's GPT-4's proposal for a NeurIPs abstract for a paper improving LLM citations. I'll put it here and we can look back in a couple months and see how it compares to actual NeurIPs submissions. :)
posted by kaibutsu at 12:13 PM on May 7, 2023 [1 favorite]
Title: CiteNet: A Novel Large Language Model Architecture for Reliable Source Citation
Abstract:
In this paper, we present CiteNet, a novel large language model (LLM) architecture designed to reliably cite sources for generated facts. Despite the impressive capabilities of current LLMs, they often struggle to provide accurate and trustworthy citations for the information they produce. To address this issue, we introduce a combination of training data modifications, architectural changes, and attention mechanisms that significantly enhance the model's citation reliability.
Our primary contribution is an architectural change that incorporates a dual-encoder mechanism: one for encoding the input text and another for encoding source information. The source encoder captures the contextual information of source materials, while the text encoder processes the input text as in conventional LLMs. The two encoders are interconnected through a novel attention mechanism that prioritizes reliable sources and encourages the model to utilize these sources when generating text.
We further augment the training data by increasing the number of well-cited examples and employing a robust data preprocessing pipeline to prioritize reliable information. This strategy enables the model to learn appropriate citation behavior and discern accurate information.
To evaluate CiteNet's performance, we introduce a new benchmark dataset that assesses the accuracy and reliability of generated citations. Our experimental results show that CiteNet significantly outperforms state-of-the-art LLMs in terms of citation quality and reliability. Furthermore, qualitative analysis demonstrates that the model can generate coherent and contextually accurate text with appropriate source citations.
CiteNet paves the way for more transparent, trustworthy, and responsible LLMs, advancing the field towards models that can be confidently utilized in various applications, from research assistance to content generation.
C'mon, guy. That's not what ToE is. ToE is just the hypothetical unification of the the major physics theories, in particular QM and relativity.
I would be interested in any reasoning you have access to that offers sound support for the position that some such hypothetical unification is likely to be achieved or even in principle achievable.
Because the things relativity successfully explains are composed of things QM successfully explains. It seems like a more accurate understanding of what's going on with those things would be consistent with both ways of observing them.
posted by straight at 6:16 PM on May 7, 2023
I would be interested in any reasoning you have access to that offers sound support for the position that some such hypothetical unification is likely to be achieved or even in principle achievable.
Because the things relativity successfully explains are composed of things QM successfully explains. It seems like a more accurate understanding of what's going on with those things would be consistent with both ways of observing them.
posted by straight at 6:16 PM on May 7, 2023
Apparently that's not the case. Apparently it's already possible to fool a hell of a lot of black-box testers a hell of a lot of the time by working off only the most recent 8,192 tokens within any given "conversation" plus the current state of a random number generator.
Given some list of tokens and some random number generator value, a LLM emits the next token in a completely deterministic fashion. In principle, that straightforward mapping from inputs to output could be captured as a lookup table. In practice, doing so would require more memory than there are atoms in the observable universe, but in principle? Absolutely.
No, because if your "in principle" is allowed to include a lookup table with more atoms than the observable universe then my "in principle" is allowed to include a maximally skeptical tester with unlimited testing time. You could never have a good enough lookup table even in theory, and definitely not in practice.
posted by straight at 6:35 PM on May 7, 2023
Given some list of tokens and some random number generator value, a LLM emits the next token in a completely deterministic fashion. In principle, that straightforward mapping from inputs to output could be captured as a lookup table. In practice, doing so would require more memory than there are atoms in the observable universe, but in principle? Absolutely.
No, because if your "in principle" is allowed to include a lookup table with more atoms than the observable universe then my "in principle" is allowed to include a maximally skeptical tester with unlimited testing time. You could never have a good enough lookup table even in theory, and definitely not in practice.
posted by straight at 6:35 PM on May 7, 2023
You could never have a good enough lookup table even in theory
Sure you could. In theory, representing the guts of ChatGPT as a lookup table is a simple matter of adding up the number of bits required to represent 8192 input tokens and a RNG state, then making a lookup table addressed by that many bits.
and definitely not in practice.
Right, which is why in practice we need to use multiple layers of feed-forward and buckets of multiply-add to get the job done.
my "in principle" is allowed to include a maximally skeptical tester with unlimited testing time.
Sounds good. What exactly is it that your maximally skeptical tester is skeptical about?
---
the things relativity successfully explains are composed of things QM successfully explains. It seems like a more accurate understanding of what's going on with those things would be consistent with both ways of observing them.
The problem I have with this is "things" and "composed of". To say that spacetime is "composed of" space and time, for example, strikes me as a misapplication of the concept of composition.
Einstein, 1936:
For me, it's all about the rainbows and unicorns.
I'm happy to stipulate that nobody has ever observed a genuine unicorn, and that all putative observations are down to interpreting measurements in ways that pay insufficient attention to the error bars.
On the other hand, almost everybody has observed a rainbow while in somebody else's company and in general people speak of a rainbow they're both seeing as if it were a real thing, over there, and agree on attributes like look, it goes all the way across and it's super bright and oh wow, it's a double! A rainbow is directly observable and its status as an objective feature of the world is generally uncontroversial.
People with a high-school level knowledge of optics, though, understand that a rainbow is not so much an observable thing in its own right as an emergent artifact of multiple overlapping observations made implicitly and independently by each observer. This understanding is useful because it explains why it's so difficult to work out precisely where the pot of gold is supposed to be.
I can think of no in principle reason to reject the idea that every observable phenomenon might at root be similarly observer-dependent. Just as you and I might be unable to observe the same rainbow simply because you're not standing close enough to me to share the required relationship between sun angle and water droplet assembly location, you and I might become unable to observe the same electron if our states of motion become dissimilar enough and there might not be a way to reconcile our QM-based predictions about the likelihood of doing so.
Reality is what it is, but objectivity is about consensus.
posted by flabdablet at 12:01 AM on May 8, 2023 [2 favorites]
Sure you could. In theory, representing the guts of ChatGPT as a lookup table is a simple matter of adding up the number of bits required to represent 8192 input tokens and a RNG state, then making a lookup table addressed by that many bits.
and definitely not in practice.
Right, which is why in practice we need to use multiple layers of feed-forward and buckets of multiply-add to get the job done.
my "in principle" is allowed to include a maximally skeptical tester with unlimited testing time.
Sounds good. What exactly is it that your maximally skeptical tester is skeptical about?
---
the things relativity successfully explains are composed of things QM successfully explains. It seems like a more accurate understanding of what's going on with those things would be consistent with both ways of observing them.
The problem I have with this is "things" and "composed of". To say that spacetime is "composed of" space and time, for example, strikes me as a misapplication of the concept of composition.
Einstein, 1936:
In the Schrödinger equation, absolute time, and also the potential energy, play a decisive role, while these two concepts have been recognized by the theory of relativity as inadmissible in principle.I am perfectly happy to work with the assumption that all of us stand in a part-whole relationship to a reality that is at least in principle capable of revealing common features to multiple observers, given sufficiently careful observation. But in order for that to happen, the observers concerned need to agree on which parts of that reality should count as belonging to those features, and which parts should not. I can see no a priori reason to assume that Alice and Bob are always guaranteed to be able to reach such an agreement if (a) their relative motion is large enough, or gravitational circumstances different enough, for relativistic effects to become important and (b) each of them is using their own proper time as if it were the absolute time assumed by QM.
For me, it's all about the rainbows and unicorns.
I'm happy to stipulate that nobody has ever observed a genuine unicorn, and that all putative observations are down to interpreting measurements in ways that pay insufficient attention to the error bars.
On the other hand, almost everybody has observed a rainbow while in somebody else's company and in general people speak of a rainbow they're both seeing as if it were a real thing, over there, and agree on attributes like look, it goes all the way across and it's super bright and oh wow, it's a double! A rainbow is directly observable and its status as an objective feature of the world is generally uncontroversial.
People with a high-school level knowledge of optics, though, understand that a rainbow is not so much an observable thing in its own right as an emergent artifact of multiple overlapping observations made implicitly and independently by each observer. This understanding is useful because it explains why it's so difficult to work out precisely where the pot of gold is supposed to be.
I can think of no in principle reason to reject the idea that every observable phenomenon might at root be similarly observer-dependent. Just as you and I might be unable to observe the same rainbow simply because you're not standing close enough to me to share the required relationship between sun angle and water droplet assembly location, you and I might become unable to observe the same electron if our states of motion become dissimilar enough and there might not be a way to reconcile our QM-based predictions about the likelihood of doing so.
Reality is what it is, but objectivity is about consensus.
posted by flabdablet at 12:01 AM on May 8, 2023 [2 favorites]
You could never have a good enough lookup table even in theory
Sure you could. In theory, representing the guts of ChatGPT as a lookup table is a simple matter of adding up the number of bits required to represent 8192 input tokens and a RNG state, then making a lookup table addressed by that many bits.
For any lookup table you can create, I can give you a conversation that isn't listed in your table.
posted by straight at 12:55 PM on May 8, 2023
Sure you could. In theory, representing the guts of ChatGPT as a lookup table is a simple matter of adding up the number of bits required to represent 8192 input tokens and a RNG state, then making a lookup table addressed by that many bits.
For any lookup table you can create, I can give you a conversation that isn't listed in your table.
posted by straight at 12:55 PM on May 8, 2023
The table doesn't list conversations, it lists output tokens.
It's a very long list: 2n entries, where n is the number of bits required to represent the token context and RNG value that would conventionally be used as inputs to the LLM under test. Each entry consists of one output token.
So for ChatGPT, n would be 8192 (the length of ChatGPT's token context) times maybe 16 to cover the number of bits needed to represent a few thousand possible tokens, plus maybe another few tens of bits to cover all possible RNG states.
The downside of such a lookup table is that in practice it's not implementable. But if it were, it would emulate ChatGPT hella fast.
posted by flabdablet at 1:20 PM on May 8, 2023
It's a very long list: 2n entries, where n is the number of bits required to represent the token context and RNG value that would conventionally be used as inputs to the LLM under test. Each entry consists of one output token.
So for ChatGPT, n would be 8192 (the length of ChatGPT's token context) times maybe 16 to cover the number of bits needed to represent a few thousand possible tokens, plus maybe another few tens of bits to cover all possible RNG states.
The downside of such a lookup table is that in practice it's not implementable. But if it were, it would emulate ChatGPT hella fast.
posted by flabdablet at 1:20 PM on May 8, 2023
They're literally talking about the thing machines can't do
Are you staking out a strong claim that people can do these things that “machines” (does that mean “deterministic computers” or does it mean something bigger?) can’t do?
I have issues along related lines with the rhetoric of singularity/“bad singularity” enthusiasts, who like to invoke the idea of a “god-like” AI. Fundamentally I think this is meant to convey that idea of a computer with capabilities so far exceeding our own in some domain that we can’t even imagine them, but the known laws of the universe suggest that the ceiling for a computer is not an actual fucking god, which makes all this talk seem unserious.
But I guess I don’t really have any commitments that would lead me to assume a priori that brains do things that can’t be computed, certainly not if we allow for paradigms of computation beyond the conventional.
posted by atoxyl at 6:35 PM on May 8, 2023 [1 favorite]
Are you staking out a strong claim that people can do these things that “machines” (does that mean “deterministic computers” or does it mean something bigger?) can’t do?
I have issues along related lines with the rhetoric of singularity/“bad singularity” enthusiasts, who like to invoke the idea of a “god-like” AI. Fundamentally I think this is meant to convey that idea of a computer with capabilities so far exceeding our own in some domain that we can’t even imagine them, but the known laws of the universe suggest that the ceiling for a computer is not an actual fucking god, which makes all this talk seem unserious.
But I guess I don’t really have any commitments that would lead me to assume a priori that brains do things that can’t be computed, certainly not if we allow for paradigms of computation beyond the conventional.
posted by atoxyl at 6:35 PM on May 8, 2023 [1 favorite]
Well, the main juncture with the development of any sort of singularity is the machines control the means of production and so can design improvements into their own chipsets.
We're leagues away from that at this moment.
posted by hippybear at 7:25 PM on May 8, 2023
We're leagues away from that at this moment.
posted by hippybear at 7:25 PM on May 8, 2023
The table doesn't list conversations, it lists output tokens.
However you store them, the table is a list of a finite number of appropriate responses to a finite number of questions. So however long you make your list, it's trivial for me to add one more question that isn't a member of the finite set of questions for which your list has a response.
posted by straight at 11:15 PM on May 8, 2023
However you store them, the table is a list of a finite number of appropriate responses to a finite number of questions. So however long you make your list, it's trivial for me to add one more question that isn't a member of the finite set of questions for which your list has a response.
posted by straight at 11:15 PM on May 8, 2023
We're talking past each other here.
All I'm saying is that in principle a LLM can be emulated with a lookup table. LLMs work by taking a token context of finite size (8192 tokens for ChatGPT) and a random number generator value as input, and emitting a single token as output.
A LLM token context typically starts out containing some pre-prompt text supplied by the developer, and some prompt text supplied by the user. As the LLM generates output tokens (or as the in-principle equivalent lookup table I'm positing looks them up) they get pushed into the end of the token context, with excess tokens being removed from the beginning to make room if necessary. Same applies to subsequent tokens supplied by the user as additional prompt text. The token context is just a finite-sized first-in-first-out sliding window covering the last N tokens of the conversation, whether those tokens came from the user or from the LLM itself as previous output.
If you supply more prompt text than the LLM token context has room for, the LLM will simply ignore all but the last N tokens of that. Same applies to the hypothetical lookup table implementation.
Note that the size of the token context does not put a limit on how many times the LLM can generate an output token or how much prompt text is supplied; the total length of the conversation is unbounded.
Also note that the overwhelming majority of possible inputs to an LLM are essentially just noise: random sequences of tokens with no syntax, let alone semantics. And because the LLM, like its equivalent lookup table, is simply a fixed mapping from a token context plus RNG value to an output token, it will emit some output token for any such input regardless of whether it would make sense to a human or not. The appropriateness or otherwise of any such output is entirely in the eye of the user.
The only way you could "add one more question that isn't a member of the finite set of questions for which the list has a response" would be to make that "question" longer than the LLM's defined token context (equivalently, wider than the lookup table's index) and, as mentioned above, the LLM would treat that as identical to a "question" consisting of only the last N tokens of the one you tried to "add". Whether or not it responded with the same output, though, would depend on the accompanying RNG value.
posted by flabdablet at 3:05 AM on May 9, 2023
All I'm saying is that in principle a LLM can be emulated with a lookup table. LLMs work by taking a token context of finite size (8192 tokens for ChatGPT) and a random number generator value as input, and emitting a single token as output.
A LLM token context typically starts out containing some pre-prompt text supplied by the developer, and some prompt text supplied by the user. As the LLM generates output tokens (or as the in-principle equivalent lookup table I'm positing looks them up) they get pushed into the end of the token context, with excess tokens being removed from the beginning to make room if necessary. Same applies to subsequent tokens supplied by the user as additional prompt text. The token context is just a finite-sized first-in-first-out sliding window covering the last N tokens of the conversation, whether those tokens came from the user or from the LLM itself as previous output.
If you supply more prompt text than the LLM token context has room for, the LLM will simply ignore all but the last N tokens of that. Same applies to the hypothetical lookup table implementation.
Note that the size of the token context does not put a limit on how many times the LLM can generate an output token or how much prompt text is supplied; the total length of the conversation is unbounded.
Also note that the overwhelming majority of possible inputs to an LLM are essentially just noise: random sequences of tokens with no syntax, let alone semantics. And because the LLM, like its equivalent lookup table, is simply a fixed mapping from a token context plus RNG value to an output token, it will emit some output token for any such input regardless of whether it would make sense to a human or not. The appropriateness or otherwise of any such output is entirely in the eye of the user.
The only way you could "add one more question that isn't a member of the finite set of questions for which the list has a response" would be to make that "question" longer than the LLM's defined token context (equivalently, wider than the lookup table's index) and, as mentioned above, the LLM would treat that as identical to a "question" consisting of only the last N tokens of the one you tried to "add". Whether or not it responded with the same output, though, would depend on the accompanying RNG value.
posted by flabdablet at 3:05 AM on May 9, 2023
The downside of such a lookup table is that in practice it's not implementable. But if it were, it would emulate ChatGPT hella fast.It wouldn't be hella fast, because the physical size of the storage medium would be so large that it would take bazillions of years for the lookup signal to propagate across it to retrieve the values. Of course, you can hand-wave that away as "yes yes it's not practical, but in principle you could", but are these physical limitations actually significant to the nature of intelligence and consciousness in a way that means that sleight of hand isn't possible? If such a lookup table was implementable in practice, it would only be possible if every atom in universe was capable of communicating with every other atom instantaneously, in a sense the whole universe would be "aware of itself" instantaneously, maybe the fact that this mysteriously isn't allowed by the laws of physics tells us something about the information processing limits of matter? Something that can "beat the clock" and produce outputs that are equivalent to those produced by a hypothetical but physically impossible implementation is fundamentally different in an important way, not "the same in principle".
posted by L.P. Hatecraft at 2:33 AM on May 19, 2023
It's useful (for some 'assume the pig is a perfect sphere' values of useful) in much the same way as one could implement an lookup table that replaced a CPU with a lookup table of any other increasingly large amount of possible mathematical operations.
The ability to compute increases in a linear way, the size of the lookup table increases exponentially. We are looking at models that have a 100k token context available for use right now. Good luck with that.
posted by jaduncan at 1:22 AM on May 21, 2023
The ability to compute increases in a linear way, the size of the lookup table increases exponentially. We are looking at models that have a 100k token context available for use right now. Good luck with that.
posted by jaduncan at 1:22 AM on May 21, 2023
And just for the record, let's say there are 75,000 possible tokens in the prompt (ludicrously low for any human language, clearly, but let's claim it's a programming language) and we have combinations that are acceptable with up to 10 of them as a response (since, as you acknowledge, there's also a random value added to generate different drafts). We are at 1,552,780,851,228,601,200,000,000,000,000,000,000,000,000 lines in the table, and we haven't even got to the point of the order of those prompt tokens mattering. I genuinely don't have the space on mefi *or the calculation time on any device I have access to* to determine this for English's 750,000-1m words and a 100k token context where the order of the tokens within that 100k matters intensely for your reimplementation.
More than anything else, this maths is the maths of why encryption rainbow tables aren't generated beyond a certain point (because you'd have to wait till beyond the heat death of the universe to have a ludicrously massive lookup table to work with).
posted by jaduncan at 2:29 AM on May 21, 2023 [1 favorite]
More than anything else, this maths is the maths of why encryption rainbow tables aren't generated beyond a certain point (because you'd have to wait till beyond the heat death of the universe to have a ludicrously massive lookup table to work with).
posted by jaduncan at 2:29 AM on May 21, 2023 [1 favorite]
« Older He dreamed of falling from the sky | How to get more women on bikes? Better biking... Newer »
This thread has been archived and is closed to new comments
So I’m not sure how much stock I’d put into this modeling ToM in LLMs. It’s pretty interesting that it’s at this point though.
posted by brook horse at 6:45 PM on May 1, 2023 [33 favorites]