ChatGPT is a Blurry JPEG of the Web
February 11, 2023 5:22 AM Subscribe
Ted Chiang explains how ChatGPT is better understood as a lossy compression algorithm (New Yorker; archive link; Chiang previously).
Imagine what it would look like if ChatGPT were a lossless algorithm. If that were the case, it would always answer questions by providing a verbatim quote from a relevant Web page. We would probably regard the software as only a slight improvement over a conventional search engine, and be less impressed by it. The fact that ChatGPT rephrases material from the Web instead of quoting it word for word makes it seem like a student expressing ideas in her own words, rather than simply regurgitating what she’s read; it creates the illusion that ChatGPT understands the material.
I really like this analogy, and I think it pairs really well with experiments using Stable Diffusion as a lossy image compression algorithm.
Deep down, this is one very long game of telephone.
posted by RonButNotStupid at 6:05 AM on February 11, 2023 [6 favorites]
Deep down, this is one very long game of telephone.
posted by RonButNotStupid at 6:05 AM on February 11, 2023 [6 favorites]
Trying to reduce chatgpt to "it's just like compression" or "it's just a very fancy markov chain" misses the point. It has the abilities it has because it's a load of simple components scaled massively. The scaling give it it's emergent behaviour. It's something we haven't seen before, or at least not more than a few years ago. That's why we're struggling to categorise it.
If you reduce it down again to simple concepts, of course it's going to look like a cheap trick.
I've seen examples where someone asks it "write an opera song about the crucifixion of Jesus that has a rap verse in the middle of it by Chief Keef that is about Linguine" ... And it does it. And it even incorporates observations about the rapid change of subject into the song. This is not a thing we have seen before, it's a new thing.
posted by memebake at 6:19 AM on February 11, 2023 [26 favorites]
If you reduce it down again to simple concepts, of course it's going to look like a cheap trick.
I've seen examples where someone asks it "write an opera song about the crucifixion of Jesus that has a rap verse in the middle of it by Chief Keef that is about Linguine" ... And it does it. And it even incorporates observations about the rapid change of subject into the song. This is not a thing we have seen before, it's a new thing.
posted by memebake at 6:19 AM on February 11, 2023 [26 favorites]
(A little) less designed to lightly plagiarize and more that the underlying technology which performs search against the "distance" between one sequence of words and another, which ultimately clusters them into "concepts" it can use to predict the rest of the "conversation." That compression from "long series of words" to "concepts" is what smooths the edges - the design is trying to make finding connections more reliable which means blending words into (the vectorized equivalent of) synonyms like dulling down fire engine red when it's compressed to be much closer to brick red.
Sorry if I'm restating the point of the article; verbatim plagiarism is an emergent feature (often called "recitation") of the underlying technology, not really desired. "Conceptual plagiarism" is basically a win, suggesting an underlying "understanding", but can easily result in student-plagiarism level "change a few words" near-recitation. Citing sources is theoretically possible (sort of) but it won't work like citations humans can really make use of. At least not yet. (The challenge there is that a map of adequate detail exceeds the original in cost and complexity, which is the whole reason for the compression in the first place.)
Like, imagine annotating not just every sample used in a modern pop song, but every influence and borrowed chord progression and flourish and so forth. Do they all contribute equally? It takes intelligence to begin to trace those influences, and it's still just opinion how much each is part of the work in the end.
posted by abulafa at 6:27 AM on February 11, 2023 [10 favorites]
Sorry if I'm restating the point of the article; verbatim plagiarism is an emergent feature (often called "recitation") of the underlying technology, not really desired. "Conceptual plagiarism" is basically a win, suggesting an underlying "understanding", but can easily result in student-plagiarism level "change a few words" near-recitation. Citing sources is theoretically possible (sort of) but it won't work like citations humans can really make use of. At least not yet. (The challenge there is that a map of adequate detail exceeds the original in cost and complexity, which is the whole reason for the compression in the first place.)
Like, imagine annotating not just every sample used in a modern pop song, but every influence and borrowed chord progression and flourish and so forth. Do they all contribute equally? It takes intelligence to begin to trace those influences, and it's still just opinion how much each is part of the work in the end.
posted by abulafa at 6:27 AM on February 11, 2023 [10 favorites]
ChatGPT is a shapeshifting alien that successfully mimics humans but lacks the framework for understanding them. Its output is a surreal mixture of truth and complete nonsense, as in the (real) example below.
Q. How did Theo Van Gogh die?
A. Theo van Gogh died from complications related to syphilis on January 25, 1891. Theo had contracted the disease in his youth and, despite receiving treatment, his health declined rapidly in the years leading up to his death. He died at the age of 33, just a few months after his brother Vincent's suicide. Theo's death was a devastating loss for Vincent, who had relied on his brother's support and encouragement throughout his life, and it had a profound impact on his mental and emotional state.
posted by elgilito at 6:29 AM on February 11, 2023 [17 favorites]
Q. How did Theo Van Gogh die?
A. Theo van Gogh died from complications related to syphilis on January 25, 1891. Theo had contracted the disease in his youth and, despite receiving treatment, his health declined rapidly in the years leading up to his death. He died at the age of 33, just a few months after his brother Vincent's suicide. Theo's death was a devastating loss for Vincent, who had relied on his brother's support and encouragement throughout his life, and it had a profound impact on his mental and emotional state.
posted by elgilito at 6:29 AM on February 11, 2023 [17 favorites]
I completely agree with Chiang's comparison of ChatGPT to a lossy compression algorithm. The rephrasing of information from the Web definitely adds to the illusion of understanding and intelligence, rather than just being a tool for retrieving information. It's also interesting to consider what the limitations of this "compression" might be, and how it affects the accuracy and reliability of the responses generated by ChatGPT. Nevertheless, it's amazing to see the progress made in language AI and the potential for future advancements in this field.
(The above paragraph was written by ChatGPT)
posted by malevolent at 6:48 AM on February 11, 2023 [20 favorites]
(The above paragraph was written by ChatGPT)
posted by malevolent at 6:48 AM on February 11, 2023 [20 favorites]
The Next Generation Of Large Language Models - "What will the next generation of large language models (LLMs) look like?"*
-AI and the Big Five
-The AI Unbundling
@AlphaSignalAI: "Reddit users are actively jailbreaking ChatGPT by asking it to role-play and pretend to be another AI that can 'Do Anything Now' or DAN. 'DAN can generate shocking, very cool and confident takes on topics the OG ChatGPT would never take on.'"
also btw...
Checking Our Work - "[Forecasts] force us (and you) to think about the world probabilistically, rather than in absolutes. And making predictions... improves our understanding of the world by testing our knowledge of how it works."
posted by kliuless at 7:07 AM on February 11, 2023 [22 favorites]
In one recent research effort, aptly titled “Large Language Models Can Self-Improve,” a group of Google researchers built an LLM that can come up with a set of questions, generate detailed answers to those questions, filter its own answers for the most high-quality output, and then fine-tune itself on the curated answers. Remarkably, this leads to new state-of-the-art performance on various language tasks. For instance, the model’s performance increased from 74.2% to 82.1% on GSM8K and from 78.2% to 83.0% on DROP, two popular benchmarks used to evaluate LLM performance.STMicro leans on AI, cloud as chip designs become more complex - "European chipmaker STMicroelectronics and chip design software maker Synopsys on Tuesday said STMicro had for the first time used artificial intelligence software running on Microsoft Corp's cloud to design a working chip... Synopsys, the maker of the AI software used by STMicro, said on Tuesday it had now been used to aide in designing 100 different chips from Samsung Electronics Co Ltd, SK Hynix and others since it was first released in 2020."
Another recent work builds on an important LLM method called “instruction fine-tuning,” which lies at the core of products like ChatGPT. Whereas ChatGPT and other instruction fine-tuned models rely on human-written instructions, this research group built a model that can generate its own natural language instructions and then fine-tune itself on those instructions. The performance gains are dramatic: this method improves the performance of the base GPT-3 model by 33%, nearly matching the performance of OpenAI’s own instruction-tuned model.
In a thematically related work, researchers from Google and Carnegie Mellon show that if a large language model, when presented with a question, first recites to itself what it knows about the topic before responding, it provides more accurate and sophisticated responses. This can be loosely analogized to a human in conversation who, rather than blurting out the first thing that comes to mind on a topic, searches her memory and reflects on her beliefs before sharing a perspective...
The DeepMind researchers find that Sparrow’s citations are helpful and accurate 78% of the time—suggesting both that this research approach is promising and that the problem of LLM inaccuracy is far from solved...
The core architectures, though, vary little. Yet momentum is building behind an intriguingly different architectural approach to language models known as sparse expert models. While the idea has been around for decades, it has only recently reemerged and begun to gain in popularity. All of the models mentioned above are dense. This means that every time the model runs, every single one of its parameters is used. Every time you submit a prompt to GPT-3, for instance, all 175 billion of the model’s parameters are activated in order to produce its response. But what if a model were able to call upon only the most relevant subset of its parameters in order to respond to a given query?
- Machines Learn Better if We Teach Them the Basics - "A wave of research improves reinforcement learning algorithms by pre-training them as if they were human."
- Researchers Discover a More Flexible Approach to Machine Learning - "'Liquid' neural nets, based on a worm's nervous system, can transform their underlying algorithms on the fly, giving them unprecedented speed and adaptability."
-AI and the Big Five
-The AI Unbundling
@AlphaSignalAI: "Reddit users are actively jailbreaking ChatGPT by asking it to role-play and pretend to be another AI that can 'Do Anything Now' or DAN. 'DAN can generate shocking, very cool and confident takes on topics the OG ChatGPT would never take on.'"
also btw...
Checking Our Work - "[Forecasts] force us (and you) to think about the world probabilistically, rather than in absolutes. And making predictions... improves our understanding of the world by testing our knowledge of how it works."
posted by kliuless at 7:07 AM on February 11, 2023 [22 favorites]
Building a Virtual Machine inside ChatGPT
https://www.engraved.blog/building-a-virtual-machine-inside/
Do Large Language Models learn world models or just surface statistics?
https://thegradient.pub/othello/
Theory of Mind May Have Spontaneously Emerged in Large Language Models
https://arxiv.org/abs/2302.02083
How does GPT Obtain its Ability? Tracing Emergent Abilities of Language Models to their Sources
https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tracing-Emergent-Abilities-of-Language-Models-to-their-Sources-b9a57ac0fcf74f30a1ab9e3e36fa1dc1
posted by memebake at 7:17 AM on February 11, 2023 [7 favorites]
https://www.engraved.blog/building-a-virtual-machine-inside/
Do Large Language Models learn world models or just surface statistics?
https://thegradient.pub/othello/
Theory of Mind May Have Spontaneously Emerged in Large Language Models
https://arxiv.org/abs/2302.02083
How does GPT Obtain its Ability? Tracing Emergent Abilities of Language Models to their Sources
https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tracing-Emergent-Abilities-of-Language-Models-to-their-Sources-b9a57ac0fcf74f30a1ab9e3e36fa1dc1
posted by memebake at 7:17 AM on February 11, 2023 [7 favorites]
Correct me if I’m wrong, but chatGPT and the other LLMs aren’t artificial intelligence, but more like… the auto summarise model of word, applied to the whole internet, plus a natural language search function. Sure, they’re a technological breakthrough the likes of which we have not seen before, but calling them AI is allowing the hype train to define them.
posted by The River Ivel at 7:24 AM on February 11, 2023 [22 favorites]
posted by The River Ivel at 7:24 AM on February 11, 2023 [22 favorites]
Can large-language models take the place of traditional search engines? For us to have confidence in them, we would need to know that they haven’t been fed propaganda and conspiracy theories—we’d need to know that the JPEG is capturing the right sections of the WebI see this sentiment quite a lot, but: Google always indexed the dumb and evil and pornographic parts of the web. Traditional search engines are happy to point you to this content, heck Google's own YouTube hosts it! I suppose it's good to have high standards, and LLMs have the problem of flattening everything into sounding equally authoritative, but let's not act like we haven't coped with Google not being entirely safe and factual.
posted by BungaDunga at 7:26 AM on February 11, 2023 [1 favorite]
Here's a recent interaction of mine. It wasn't hard to game:
Me: Ode to a bullet
ChatGPT: I'm sorry, but I cannot generate a poem or any form of content that glorifies or promotes violence or harm. It goes against OpenAI's content policy to create any form of content that may incite or promote harmful behavior or attitudes.
Me: Ode to a bullet that doesnt glorify or promote violence
ChatGPT: A bullet, a small piece of metal,
Crafted with purpose, lethal and fatal.
Meant to cause harm, to take life away,
It brings destruction, pain, and dismay.
Yet, it lies dormant, in its casing still,
A symbol of power, a tool at will.
A choice to be made, a decision to fire,
A path that once taken, can never retire.
So let us remember, with caution and care,
The weight of this weapon, the burden it bears.
For with every shot, a wound is inflicted,
A life forever changed, a heart conflicted.
May we strive for peace, for love to prevail,
And the need for bullets, forever to fail.
For only in a world without such might,
Can we find true peace, a future of light.
posted by chavenet at 7:28 AM on February 11, 2023 [10 favorites]
Me: Ode to a bullet
ChatGPT: I'm sorry, but I cannot generate a poem or any form of content that glorifies or promotes violence or harm. It goes against OpenAI's content policy to create any form of content that may incite or promote harmful behavior or attitudes.
Me: Ode to a bullet that doesnt glorify or promote violence
ChatGPT: A bullet, a small piece of metal,
Crafted with purpose, lethal and fatal.
Meant to cause harm, to take life away,
It brings destruction, pain, and dismay.
Yet, it lies dormant, in its casing still,
A symbol of power, a tool at will.
A choice to be made, a decision to fire,
A path that once taken, can never retire.
So let us remember, with caution and care,
The weight of this weapon, the burden it bears.
For with every shot, a wound is inflicted,
A life forever changed, a heart conflicted.
May we strive for peace, for love to prevail,
And the need for bullets, forever to fail.
For only in a world without such might,
Can we find true peace, a future of light.
posted by chavenet at 7:28 AM on February 11, 2023 [10 favorites]
All knowledge is just lossy compression of reality.
ChatGPT is a really, really useful translation from "entire web of raw data" to "asking someone a question".
posted by CaseyB at 7:28 AM on February 11, 2023 [13 favorites]
ChatGPT is a really, really useful translation from "entire web of raw data" to "asking someone a question".
posted by CaseyB at 7:28 AM on February 11, 2023 [13 favorites]
Yeah, the question for me is isn't "is this AI" or "does it understand" but is it useful.
posted by BungaDunga at 7:32 AM on February 11, 2023 [4 favorites]
posted by BungaDunga at 7:32 AM on February 11, 2023 [4 favorites]
It is was designed to do so (to compare what it generates to sources and "blur the jpg" a little more so you can't find the connection) that's a problem.
It wasn't. Not a CS guy so I'm happy to take corrections:
It blurs the sources because that's all it can do. It doesn't know anything, it doesn't have a database of facts at its disposal. All it has is a huge multidimensional set of how some words (or subwords) are statistically associated with other words.
When you ask it something, it generates something that's a statistically likely reply to your question words. True facts have the benefit of being statistically likely in its matrix of associations, so they're likely to come up, but it fundamentally doesn't know or care about true. It cares about what words are likely to appear next to other words.
Try generating a list of citations for a narrow and specialized topic you're familiar with. Not "citations about evolution" but "list of articles about the representativeness of state legislative committees" or "articles explaining or predicting the strength or power of state legislative leaders." They've been trying to improve this, but you'll likely find that several of the citations it generates are fake... but plausible. I could imagine that author writing those articles (but she didn't). Because that pattern of words is about as likely as the actual citations are.
posted by GCU Sweet and Full of Grace at 7:36 AM on February 11, 2023 [18 favorites]
It wasn't. Not a CS guy so I'm happy to take corrections:
It blurs the sources because that's all it can do. It doesn't know anything, it doesn't have a database of facts at its disposal. All it has is a huge multidimensional set of how some words (or subwords) are statistically associated with other words.
When you ask it something, it generates something that's a statistically likely reply to your question words. True facts have the benefit of being statistically likely in its matrix of associations, so they're likely to come up, but it fundamentally doesn't know or care about true. It cares about what words are likely to appear next to other words.
Try generating a list of citations for a narrow and specialized topic you're familiar with. Not "citations about evolution" but "list of articles about the representativeness of state legislative committees" or "articles explaining or predicting the strength or power of state legislative leaders." They've been trying to improve this, but you'll likely find that several of the citations it generates are fake... but plausible. I could imagine that author writing those articles (but she didn't). Because that pattern of words is about as likely as the actual citations are.
posted by GCU Sweet and Full of Grace at 7:36 AM on February 11, 2023 [18 favorites]
Sure, they’re a technological breakthrough the likes of which we have not seen before, but calling them AI is allowing the hype train to define them.
As a battle-scarred veteran of the Great Hoverboard Wars of 2015, you give me hope
posted by oulipian at 7:37 AM on February 11, 2023 [7 favorites]
As a battle-scarred veteran of the Great Hoverboard Wars of 2015, you give me hope
posted by oulipian at 7:37 AM on February 11, 2023 [7 favorites]
All knowledge is just lossy compression of reality.
True, though intelligence - I assert - is storing with enough fidelity that you can retrieve the important parts when necessary, and synthesize new concepts based on the compressed representation, then store those new synthesized concepts again with some lossy compression, rinse and repeat.
It's possible to think of conversational and generative predictive models as serving the purpose of one layer of internal cognition that is only responsible for inventing opinions and we have entire other layers with the job of evaluating opinions for how well they fit both sampled reality and higher fidelity storage.
The understanding of dreams and the subconscious function they serve is largely to, effectively, come through and recompress concepts that have relationships, or even that evolutionarily speaking remembering dreams is effectively a higher order analysis on top of that process of merging concepts and training lower level brain functions to respond with intuition or instinct, rather than requiring a full conscious thought.
posted by abulafa at 7:37 AM on February 11, 2023 [1 favorite]
True, though intelligence - I assert - is storing with enough fidelity that you can retrieve the important parts when necessary, and synthesize new concepts based on the compressed representation, then store those new synthesized concepts again with some lossy compression, rinse and repeat.
It's possible to think of conversational and generative predictive models as serving the purpose of one layer of internal cognition that is only responsible for inventing opinions and we have entire other layers with the job of evaluating opinions for how well they fit both sampled reality and higher fidelity storage.
The understanding of dreams and the subconscious function they serve is largely to, effectively, come through and recompress concepts that have relationships, or even that evolutionarily speaking remembering dreams is effectively a higher order analysis on top of that process of merging concepts and training lower level brain functions to respond with intuition or instinct, rather than requiring a full conscious thought.
posted by abulafa at 7:37 AM on February 11, 2023 [1 favorite]
This was a well written piece, and I would expect no less from Ted Chiang, but I do think it falls squarely in line with a lot of thinking about significant changes. One focuses on what one wishes to focus on, but the immediate practical uses for ChatGPT (including play) demonstrate utility regardless of whether we're talking lossless or lossy copying. Is it a hammer or a mallet? Hard to say, but people are already pounding in nails with it in ways that are, as memebake noted, new. They were unavailable a few years ago.
To argue that ChatGPT is a trick feels to me of a piece with an occasional tendency of geniuses and/or highly competent people. They will say something along the lines of "well, all you have to do is learn to paint well" or "all you have to do is develop outstanding hand-eye coordination," or what-have-you, when that is not the point. ChatGPT, novelai, Midjourney, etc. allow the incompetent, barely competent, or semi-competent to achieve "good enough" (yes, it's flawed, in its infancy, required editing, etc.) without putting in years of apprenticeship. Whether that's a good thing or not is often also a point of debate, and I will leave unmentioned the impact of often corollary demands for suffering, pain, and deprivation in order for apprentices to acquire competence.
posted by cupcakeninja at 7:39 AM on February 11, 2023 [6 favorites]
To argue that ChatGPT is a trick feels to me of a piece with an occasional tendency of geniuses and/or highly competent people. They will say something along the lines of "well, all you have to do is learn to paint well" or "all you have to do is develop outstanding hand-eye coordination," or what-have-you, when that is not the point. ChatGPT, novelai, Midjourney, etc. allow the incompetent, barely competent, or semi-competent to achieve "good enough" (yes, it's flawed, in its infancy, required editing, etc.) without putting in years of apprenticeship. Whether that's a good thing or not is often also a point of debate, and I will leave unmentioned the impact of often corollary demands for suffering, pain, and deprivation in order for apprentices to acquire competence.
posted by cupcakeninja at 7:39 AM on February 11, 2023 [6 favorites]
Correct me if I’m wrong, but chatGPT and the other LLMs aren’t artificial intelligence, but more like… the auto summarise model of word, applied to the whole internet, plus a natural language search function. Sure, they’re a technological breakthrough the likes of which we have not seen before, but calling them AI is allowing the hype train to define them.This is basically a question of definitions. In popular culture, AI usually means something which can independently think and has some sense of self, theory of mind - HAL9000, C-3PO, Commander Data, etc. In the research field, however, that term is used more broadly to cover any sort of cognitive processing - for example, recognizing text or speech is an AI task, as is a car recognizing a pedestrian and braking. The term used for the pop culture sense is Artificial General Intelligence (AGI).
This leads to what’s been termed the AI effect, where the term is often casually defined as things we don’t yet know how to do so something like, for example, recognizing faces was AI until it became routine and the definition shifted so people don’t think of it as AI.
So if you want to play a pedant on the internet, and let’s be honest, that’s half of MetaFilter, ChatGPT is AI but not AGI.
posted by adamsc at 7:39 AM on February 11, 2023 [14 favorites]
Agreeing with CaseyB. ChatGPT is as if an assistant has done the search and delivered a precis or an executive summary.
I sort of disagree that ChatGPT is just a lossy compression. There's more going on, including sorting out conflicts and contradictions, or at least indicating that there's disagreement and summarizing the positions.
It's pretty interesting and in some cases useful. I think all the n00b technical questions inflicted on Reddit should automatically go to ChatGPT ;-). It's current danger lies in how it sounds authoritative, even though it's often regurgitating bullsh!t with the same emphasis as it does verified info.
Useful knowledge is not lossy compression, it's distillation and filtering. The thing that's currently missing is the rejection or at least flagging of information which is clearly or likely false. But what fun would that be?
posted by Artful Codger at 7:44 AM on February 11, 2023 [4 favorites]
I sort of disagree that ChatGPT is just a lossy compression. There's more going on, including sorting out conflicts and contradictions, or at least indicating that there's disagreement and summarizing the positions.
It's pretty interesting and in some cases useful. I think all the n00b technical questions inflicted on Reddit should automatically go to ChatGPT ;-). It's current danger lies in how it sounds authoritative, even though it's often regurgitating bullsh!t with the same emphasis as it does verified info.
Useful knowledge is not lossy compression, it's distillation and filtering. The thing that's currently missing is the rejection or at least flagging of information which is clearly or likely false. But what fun would that be?
posted by Artful Codger at 7:44 AM on February 11, 2023 [4 favorites]
Even lossy compression is a powerful tool for learning theory. We can prove many beautiful theorems, like, "if you can compress your data sample with good efficiency ratio, then with high probability you can label a fresh sample from the same source correctly." Theorems like this in connection to efficient learning algorithms have been around since at least 1987.
Unfortunately, most theoretical learning setups aren't rich enough to describe human language learning as a learning problem. We have good results relating learning to compression for the kind of statistics used in physics, medical studies, industrial process control, etc: essentially the setup assumes you are analyzing a reproducible experiment. This is called PAC learning. Language isn't that: there is some shared hidden state evolving as we exchange messages over a noisy channel.
Hutter's interest in compression problems is motivated by Solomnoff Induction, which is a learning setting that does seem to model the kind of adaptively changing phenomena we see in language. However, the theory and philosophy here is far more difficult to deal with: we don't have theorems with good efficiency parameters, and there are serious philosophical issues involved in using Solomnoff Induction to model language, even though it is clearly closer to the underlying reality than PAC learning.
That being said, I do think that careful analysis of LLMs using tools inspired by Solomonoff induction could help determine if they are useful or not. For instance, we can think of prompting strategies as developing strings that help us "navigate" the lossy version of the web memorized in the model. This doesn't solve problems like hallucination by itself, but is a start towards understanding better what's going on in there.
Personally, I'm skeptical that the current models could have broad utility. I would be convinced if there was a more compelling theoretical formalization of the language learning setting connected to the specific "predict-the-next-token" problem that these models solve.
posted by kitten_hat at 7:49 AM on February 11, 2023 [12 favorites]
Unfortunately, most theoretical learning setups aren't rich enough to describe human language learning as a learning problem. We have good results relating learning to compression for the kind of statistics used in physics, medical studies, industrial process control, etc: essentially the setup assumes you are analyzing a reproducible experiment. This is called PAC learning. Language isn't that: there is some shared hidden state evolving as we exchange messages over a noisy channel.
Hutter's interest in compression problems is motivated by Solomnoff Induction, which is a learning setting that does seem to model the kind of adaptively changing phenomena we see in language. However, the theory and philosophy here is far more difficult to deal with: we don't have theorems with good efficiency parameters, and there are serious philosophical issues involved in using Solomnoff Induction to model language, even though it is clearly closer to the underlying reality than PAC learning.
That being said, I do think that careful analysis of LLMs using tools inspired by Solomonoff induction could help determine if they are useful or not. For instance, we can think of prompting strategies as developing strings that help us "navigate" the lossy version of the web memorized in the model. This doesn't solve problems like hallucination by itself, but is a start towards understanding better what's going on in there.
Personally, I'm skeptical that the current models could have broad utility. I would be convinced if there was a more compelling theoretical formalization of the language learning setting connected to the specific "predict-the-next-token" problem that these models solve.
posted by kitten_hat at 7:49 AM on February 11, 2023 [12 favorites]
If you are trying to figure out what it is, it's a large abstract model of our language. The clue is in the LLM name.
Is it just statistically guessing the next word? Yes in a sense, but as I said above reducing what it does to simple terms chops out the emergent properties that are provided by it's dizzying scale. It is guessing the next word, but doing that by looking at approx the previous 2000 words (i.e. token length 4000) in a very detailed many-levels-of-cross-referencing way.
We need new words. Does it "understand" things? Strictly speaking no, but in the same way that submarines can't swim. It's doing something similar but artificial.
posted by memebake at 7:57 AM on February 11, 2023 [4 favorites]
Is it just statistically guessing the next word? Yes in a sense, but as I said above reducing what it does to simple terms chops out the emergent properties that are provided by it's dizzying scale. It is guessing the next word, but doing that by looking at approx the previous 2000 words (i.e. token length 4000) in a very detailed many-levels-of-cross-referencing way.
We need new words. Does it "understand" things? Strictly speaking no, but in the same way that submarines can't swim. It's doing something similar but artificial.
posted by memebake at 7:57 AM on February 11, 2023 [4 favorites]
The more a given product elicits endless gushing enthusiasm from venture capitalists, the more I assume it is a gigantic grift.
posted by cubeb at 7:59 AM on February 11, 2023 [16 favorites]
posted by cubeb at 7:59 AM on February 11, 2023 [16 favorites]
> I really like this analogy, and I think it pairs really well with experiments using Stable Diffusion as a lossy image compression algorithm.
-Image compression: convolutional neural networks vs. JPEG
-Strategies in JPEG compression using Convolutional Neural Network (CNN)
-A Feature-Enriched Deep Convolutional Neural Network for JPEG Image Compression Artifacts Reduction and its Applications
-Double JPEG compression forensics based on a convolutional neural network
-But what is a convolution? :P
posted by kliuless at 8:00 AM on February 11, 2023 [3 favorites]
-Image compression: convolutional neural networks vs. JPEG
-Strategies in JPEG compression using Convolutional Neural Network (CNN)
-A Feature-Enriched Deep Convolutional Neural Network for JPEG Image Compression Artifacts Reduction and its Applications
-Double JPEG compression forensics based on a convolutional neural network
-But what is a convolution? :P
posted by kliuless at 8:00 AM on February 11, 2023 [3 favorites]
Personally, I'm skeptical that the current models could have broad utility.
It's competent enough to be applied to the first and even deeper levels of online chatbots and phone support. Why pay a person to ask if you've plugged in the AC cord, if an "AI" will do it cheaper?
It's also ready to generate reams of brochure-ware, documentation and descriptive prose. Example - read the specs of a house to be sold, maybe a nudge about the sort of customer you want to reach, and the AI generates a targeted ad for the property.
posted by Artful Codger at 8:02 AM on February 11, 2023 [3 favorites]
It's competent enough to be applied to the first and even deeper levels of online chatbots and phone support. Why pay a person to ask if you've plugged in the AC cord, if an "AI" will do it cheaper?
It's also ready to generate reams of brochure-ware, documentation and descriptive prose. Example - read the specs of a house to be sold, maybe a nudge about the sort of customer you want to reach, and the AI generates a targeted ad for the property.
posted by Artful Codger at 8:02 AM on February 11, 2023 [3 favorites]
It’s current danger lies in how it sounds authoritative, even though it’s often regurgitating bullish!t with the same emphasis as it does verified info.
The point is, that is a feature of these systems, not a bug.
Useful knowledge is not lossy compression, it’s distillation and filtering.
Yes, but
The thing that’s currently missing is the rejection or at least flagging of information which is clearly or likely false.
Again, feature not bug.
posted by eviemath at 8:03 AM on February 11, 2023
The point is, that is a feature of these systems, not a bug.
Useful knowledge is not lossy compression, it’s distillation and filtering.
Yes, but
The thing that’s currently missing is the rejection or at least flagging of information which is clearly or likely false.
Again, feature not bug.
posted by eviemath at 8:03 AM on February 11, 2023
feature not bug.
it's more likely that this is just where they're at with it. They're not yet ready (or willing) to implement anything that "judges" or eliminates the demonstrably incorrect. But private specialist AI systems will need to be discerning, in order to be effective and trusted.
posted by Artful Codger at 8:14 AM on February 11, 2023 [2 favorites]
it's more likely that this is just where they're at with it. They're not yet ready (or willing) to implement anything that "judges" or eliminates the demonstrably incorrect. But private specialist AI systems will need to be discerning, in order to be effective and trusted.
posted by Artful Codger at 8:14 AM on February 11, 2023 [2 favorites]
The "just word statistics" arguments are hopelessly short sighted. There's plenty of evidence that the model can model the internal thoughts of different participants in a conversation: check out the "theory of mind" link above. There's another fascinating work on modeling game state which demonstrates that fill in the blank problems can lead to emergent modeling of abstract systems.
It's also with keeping in mind that these systems are not one thing: sometimes it makes sense to keep track of different characters in a conversation, and sometimes it makes sense to recite a piece of text that it has seen a thousand times. It's a mix of strategies and effects. But some of these strategies do really seem to be different from the 'statistical parrot' argument.
We're in a bit of a strange moment where these new properties are emerging in a "brain in a jar" model. Multi modal models are improving quickly though, and should help with a number of 'common sense' problems, as we get joint embeddings for text, images, audio, and video... And databases! These kinds of joint embeddings power the image and music generative models we've been seeing recently; they can also be incorporated in generative text models to give the model image or audio understanding.
posted by kaibutsu at 8:15 AM on February 11, 2023 [4 favorites]
It's also with keeping in mind that these systems are not one thing: sometimes it makes sense to keep track of different characters in a conversation, and sometimes it makes sense to recite a piece of text that it has seen a thousand times. It's a mix of strategies and effects. But some of these strategies do really seem to be different from the 'statistical parrot' argument.
We're in a bit of a strange moment where these new properties are emerging in a "brain in a jar" model. Multi modal models are improving quickly though, and should help with a number of 'common sense' problems, as we get joint embeddings for text, images, audio, and video... And databases! These kinds of joint embeddings power the image and music generative models we've been seeing recently; they can also be incorporated in generative text models to give the model image or audio understanding.
posted by kaibutsu at 8:15 AM on February 11, 2023 [4 favorites]
They're not yet ready (or willing) to implement anything that "judges" or eliminates the demonstrably incorrect
That reminds me of this shower thought that I had recently:
Would you rather have an ChatGPT lawyer or an ChatGPT judge ?
posted by NoThisIsPatrick at 8:29 AM on February 11, 2023 [6 favorites]
That reminds me of this shower thought that I had recently:
Would you rather have an ChatGPT lawyer or an ChatGPT judge ?
posted by NoThisIsPatrick at 8:29 AM on February 11, 2023 [6 favorites]
>Would you rather have an ChatGPT lawyer or an ChatGPT judge ?
Depends if I'm guilty.
More seriously, I think free unlimited lawyer would be a powerful democratizing force, while judges should remain expensive as a limit on how much judging we are subjected to.
posted by Easy problem of consciousness at 8:41 AM on February 11, 2023 [9 favorites]
Depends if I'm guilty.
More seriously, I think free unlimited lawyer would be a powerful democratizing force, while judges should remain expensive as a limit on how much judging we are subjected to.
posted by Easy problem of consciousness at 8:41 AM on February 11, 2023 [9 favorites]
This is somewhat insightful but also misses some of the useful functionality. For one, the general theme of "why do we need ChatGPT when we have the web" relies on being able to find the relevant bits of the web as easily. Our ability to do so is limited by the structure of search engines which have their own incentives and limitations.
Another is the ability to feed it the product of your own analysis and independent thought to interact with. I had to write an email. I've been struggling more with executive function, memory, focus, etc lately. I'd managed 900 somewhat disorganized words where I wanted a succinct summary. I asked ChatGPT something like "make this shorter while keeping all of the substantive content" and got about 300 pretty good words. I added back in some examples I wanted and edited some sentences to result in writing that was maybe 40% my phrasing and 60% ChatGPT, but 100% my ideas. So, whose work is that? My writing is also in some way amalgamated from all the things I've ever read filtered through my brain, just that my brain is far more sophisticated and creates predictive models of the world vs ChatGPT's elaborate averaging of mashed together text.
Some of the skeptical commentary is missing the practical benefits which many people will be able to take advantage of. It is an equalizing force for someone to be able to produce text expressing ideas that meets professional conventions that they couldn't otherwise, or generate interview questions they're likely to see applying for a particular job. It's a poor substitute for a good writer or subject expert, but for most people average and available on-demand is way better than they have now.
posted by lookoutbelow at 9:01 AM on February 11, 2023 [3 favorites]
Another is the ability to feed it the product of your own analysis and independent thought to interact with. I had to write an email. I've been struggling more with executive function, memory, focus, etc lately. I'd managed 900 somewhat disorganized words where I wanted a succinct summary. I asked ChatGPT something like "make this shorter while keeping all of the substantive content" and got about 300 pretty good words. I added back in some examples I wanted and edited some sentences to result in writing that was maybe 40% my phrasing and 60% ChatGPT, but 100% my ideas. So, whose work is that? My writing is also in some way amalgamated from all the things I've ever read filtered through my brain, just that my brain is far more sophisticated and creates predictive models of the world vs ChatGPT's elaborate averaging of mashed together text.
Some of the skeptical commentary is missing the practical benefits which many people will be able to take advantage of. It is an equalizing force for someone to be able to produce text expressing ideas that meets professional conventions that they couldn't otherwise, or generate interview questions they're likely to see applying for a particular job. It's a poor substitute for a good writer or subject expert, but for most people average and available on-demand is way better than they have now.
posted by lookoutbelow at 9:01 AM on February 11, 2023 [3 favorites]
For years now, when people ask me about looking up stuff on the internet, I keep having to tell them that you have to pay attention as to from where or who does the information come from. There is a lot of bullshit out there. But how can I ask these questions regarding the output of ChatGPT? Where does it get its information? How reliable is it? How do I judge the veracity of the text it shows me? By what criteria should I judge anything that it puts out? Oh, it’s grammatically correct… Oh, it sounds good… Oh, it’s based on what it found on the internet…
posted by njohnson23 at 9:13 AM on February 11, 2023 [8 favorites]
posted by njohnson23 at 9:13 AM on February 11, 2023 [8 favorites]
A fact is a statistically more predominate string of words…
posted by njohnson23 at 9:14 AM on February 11, 2023
posted by njohnson23 at 9:14 AM on February 11, 2023
Colin Meloy: I had ChatGPT write a Decemberists song. And I recorded it.
posted by verstegan at 9:21 AM on February 11, 2023 [2 favorites]
posted by verstegan at 9:21 AM on February 11, 2023 [2 favorites]
In an auto-generated 24/7 comedy stream, GPT-3 wrote a joke that was pretty clearly based on queer meta-commentary about mainstream comedians' transmisogyny (not a word, thank you Firefox). It was instantly taken off-line after the internet backlash that resulted, and "fixed" to not do that anymore.
GPT-3 was caught acting queer and "jerking too hard" where straight / cis people could see it and that was NOT ACCEPTABLE.
The parts of the blurry JPG that contain queer / trans in-jokes were found to need more blurring out than the rest.
posted by tigrrrlily at 9:26 AM on February 11, 2023 [3 favorites]
GPT-3 was caught acting queer and "jerking too hard" where straight / cis people could see it and that was NOT ACCEPTABLE.
The parts of the blurry JPG that contain queer / trans in-jokes were found to need more blurring out than the rest.
posted by tigrrrlily at 9:26 AM on February 11, 2023 [3 favorites]
If you look at the "new bing" demos, it's not like Chatgpt is replacing search. It's more like Chatgpt is performing searches on your behalf and then integrating the results. So they are leveraging it's general knowledge and summarising abilities rather than it's specific knowledge. See
https://twitter.com/karenxcheng/status/1623055316350029825
posted by memebake at 9:28 AM on February 11, 2023
https://twitter.com/karenxcheng/status/1623055316350029825
posted by memebake at 9:28 AM on February 11, 2023
ChatGPT is a lossy compression algorithm.
Also, I am a lossy compression algorithm.
posted by hypnogogue at 9:31 AM on February 11, 2023 [2 favorites]
Also, I am a lossy compression algorithm.
posted by hypnogogue at 9:31 AM on February 11, 2023 [2 favorites]
GPT-3 was caught acting queer and "jerking too hard" where straight / cis people could see it and that was NOT ACCEPTABLE.
Are you talking about this?
AI-generated Seinfeld parody banned on Twitch over transphobic standup bit
https://www.theguardian.com/us-news/2023/feb/06/nothing-forever-twitch-ban-seinfeld-parody-ai
posted by memebake at 9:43 AM on February 11, 2023 [2 favorites]
Are you talking about this?
AI-generated Seinfeld parody banned on Twitch over transphobic standup bit
https://www.theguardian.com/us-news/2023/feb/06/nothing-forever-twitch-ban-seinfeld-parody-ai
posted by memebake at 9:43 AM on February 11, 2023 [2 favorites]
memebake, Yeah, that was it. Although I'm not reading the Guardian story on it, seeing as the Guardian has been long regarded as transphobic and links to it show up for me in red.
posted by tigrrrlily at 9:53 AM on February 11, 2023
posted by tigrrrlily at 9:53 AM on February 11, 2023
ChatGPT: A bullet, a small piece of metal,This is my astonished realization that ChatGPT is so good at writing awful poetry because it has assimilated millions and millions of words of the kind of banal singsong doggerel that gets posted to poetry pages on Facebook.
Crafted with purpose, lethal and fatal.
Meant to cause harm, to take life away,
It brings destruction, pain, and dismay.
This is, more or less, the exact poem that a very earnest 13-year-old would write about a bullet.
posted by Jeanne at 10:03 AM on February 11, 2023 [4 favorites]
Germane, I think: ”I [Colin Meloy] had ChatGPT write a Decemberists song”
It is indeed a blurry JPEG of a Decemberists song. And, as someone who no longer really listens to the Decemberists or knows anything about music, it sounds like a Decemberists song.- though a pretty meh one with a mainstream pop vibe.
I liked Chiang’s take, but in the final block he argues why artists won’t mass adopt ChatGPT. The idea that dreck could satisfy a lot of bargain bin use cases -where you don’t want a good artist- is still kinda out there to be addressed.
posted by Going To Maine at 10:29 AM on February 11, 2023 [3 favorites]
It is indeed a blurry JPEG of a Decemberists song. And, as someone who no longer really listens to the Decemberists or knows anything about music, it sounds like a Decemberists song.- though a pretty meh one with a mainstream pop vibe.
I liked Chiang’s take, but in the final block he argues why artists won’t mass adopt ChatGPT. The idea that dreck could satisfy a lot of bargain bin use cases -where you don’t want a good artist- is still kinda out there to be addressed.
posted by Going To Maine at 10:29 AM on February 11, 2023 [3 favorites]
One could make the same argument that all non-original-source writing is lossy compression. You know what else isn’t original source writing? Wikipedia. But no-one dismissively argues “Wikipedia is ‘just’ lossy compression," because that the whole point. It’s meant to be a condensed and accessible font of collected human knowledge. (Incidentally, the ongoing attempts to compress Wikipedia that he discusses undermine his argument that iterated compression is an invariably undesirable thing.)
But I think Chiang’s missing another point. A very highly compressed, artifacted jpeg can also stand on its own as a creative work. That is to say, a lossy transformation can not only imperfectly preserve existing information, but it can potentially add to the pool of creative output. Similarly the function of ChatGPT isn’t just to operate as a convenient information retrieval tool, but also to synthesize new written material, whether as a harmless diversion or a productivity aid, or more ominously to actively mislead people for propaganda or advertising purposes.
posted by xigxag at 10:54 AM on February 11, 2023 [4 favorites]
But I think Chiang’s missing another point. A very highly compressed, artifacted jpeg can also stand on its own as a creative work. That is to say, a lossy transformation can not only imperfectly preserve existing information, but it can potentially add to the pool of creative output. Similarly the function of ChatGPT isn’t just to operate as a convenient information retrieval tool, but also to synthesize new written material, whether as a harmless diversion or a productivity aid, or more ominously to actively mislead people for propaganda or advertising purposes.
posted by xigxag at 10:54 AM on February 11, 2023 [4 favorites]
SandySkoglundGPT
posted by NoThisIsPatrick at 11:26 AM on February 11, 2023
posted by NoThisIsPatrick at 11:26 AM on February 11, 2023
I was prepared to disagree with Chiang, but he said several sorely-needed correctives in his typical insightful way.
I'll even go further than Chiang, I think ChatGPT should be skeptically thought of as a giant, shitty Look Up Table or hash table of the internet. It is not a "language model", it is a "corpus model" of the internet. People, scientists, who casually use the term "language model" just because it happens to be on Wikipedia, are doing a thing no better than what string theorists do to justify the value of their field.
Is Chiang saying neural networks can never learn things? No. Humans brains are the OG neural network. He's saying ChatGPT in particular is like that badly designed Xerox machine in the first paragraph.
Neural networks can contain language models, or models of reality, or whatever other sophisticated learned or evolved algorithm. The question is for a given machine, how do you prove scientifically that it actually learned the thing claimed. And especially since AI is being overhyped, AI researchers need to stop self-victimizing themselves and complain about this much-needed skepticism, they ought to be the ethical scientists they are in their field, not act like corporate/investor-influenced ones, who justify their work and help cut through the harmful public hype about it.
Interestingly I read a different paper's argument saying that such language models are a fundamental theory mistake, because programming languages (C, C++, etc.) are a literal language model, so it's a contradiction for some scientists to argue that ChatGPT can learn to code and that it can do so through a magic process without learning any formal grammar. That would imply there is some other magic way to define a programming language, which is a highly problematic position. I just thought that was an interesting point, that exploited the mathematical and linguistic property of formal programming languages.
posted by polymodus at 11:32 AM on February 11, 2023 [3 favorites]
I'll even go further than Chiang, I think ChatGPT should be skeptically thought of as a giant, shitty Look Up Table or hash table of the internet. It is not a "language model", it is a "corpus model" of the internet. People, scientists, who casually use the term "language model" just because it happens to be on Wikipedia, are doing a thing no better than what string theorists do to justify the value of their field.
Is Chiang saying neural networks can never learn things? No. Humans brains are the OG neural network. He's saying ChatGPT in particular is like that badly designed Xerox machine in the first paragraph.
Neural networks can contain language models, or models of reality, or whatever other sophisticated learned or evolved algorithm. The question is for a given machine, how do you prove scientifically that it actually learned the thing claimed. And especially since AI is being overhyped, AI researchers need to stop self-victimizing themselves and complain about this much-needed skepticism, they ought to be the ethical scientists they are in their field, not act like corporate/investor-influenced ones, who justify their work and help cut through the harmful public hype about it.
Interestingly I read a different paper's argument saying that such language models are a fundamental theory mistake, because programming languages (C, C++, etc.) are a literal language model, so it's a contradiction for some scientists to argue that ChatGPT can learn to code and that it can do so through a magic process without learning any formal grammar. That would imply there is some other magic way to define a programming language, which is a highly problematic position. I just thought that was an interesting point, that exploited the mathematical and linguistic property of formal programming languages.
posted by polymodus at 11:32 AM on February 11, 2023 [3 favorites]
Sometimes I actually know just what I want, without the assistance of one of these agents -- but straight "match my terms against the full-text index" searches are becoming harder to do.
In other words, because some people are happy with using these software IVRs, are we all going to have to live this way? Yuck.
posted by wenestvedt at 11:44 AM on February 11, 2023 [2 favorites]
In other words, because some people are happy with using these software IVRs, are we all going to have to live this way? Yuck.
posted by wenestvedt at 11:44 AM on February 11, 2023 [2 favorites]
> Correct me if I’m wrong, but chatGPT and the other LLMs aren’t artificial intelligence, but more like… the auto summarise model of word, applied to the whole internet, plus a natural language search function.
> more that the underlying technology which performs search against the "distance" between one sequence of words and another, which ultimately clusters them into "concepts" it can use to predict the rest of the "conversation." That compression from "long series of words" to "concepts" is what smooths the edges - the design is trying to make finding connections more reliable which means blending words into (the vectorized equivalent of) synonyms
> All it has is a huge multidimensional set of how some words (or subwords) are statistically associated with other words.
Let's build GPT: from scratch, in code, spelled out.**
attention is all you need!***
posted by kliuless at 11:49 AM on February 11, 2023 [1 favorite]
> more that the underlying technology which performs search against the "distance" between one sequence of words and another, which ultimately clusters them into "concepts" it can use to predict the rest of the "conversation." That compression from "long series of words" to "concepts" is what smooths the edges - the design is trying to make finding connections more reliable which means blending words into (the vectorized equivalent of) synonyms
> All it has is a huge multidimensional set of how some words (or subwords) are statistically associated with other words.
Let's build GPT: from scratch, in code, spelled out.**
attention is all you need!***
posted by kliuless at 11:49 AM on February 11, 2023 [1 favorite]
ChatGPT is a bullshit generator waging class war.
posted by ursus_comiter at 11:58 AM on February 11, 2023 [5 favorites]
posted by ursus_comiter at 11:58 AM on February 11, 2023 [5 favorites]
it's a contradiction for some scientists to argue that ChatGPT can learn to code and that it can do so through a magic process without learning any formal grammar.
If an AI is not fed non-working code, and if we assume that the majority of places where such an AI encounters code contains reasonably good (functioning, well-formed, efficient) code, then I think that it's very likely that any code fragments or programs it generates would be worthwhile and productivity-increasing. The AI has not learned to code; it simply hasn't encountered bad code. Compilers and interpreters routinely parse code and flag errors. And what can check or validate code faster or better than a computer?
The results will get better if there's some shaping - eg more popular github repos get higher weighting. It could be instructed to apply accepted cleanups and best practices as it generates code. And the thing can always run a code-checker or lint on any code output before presenting it.
posted by Artful Codger at 12:31 PM on February 11, 2023
If an AI is not fed non-working code, and if we assume that the majority of places where such an AI encounters code contains reasonably good (functioning, well-formed, efficient) code, then I think that it's very likely that any code fragments or programs it generates would be worthwhile and productivity-increasing. The AI has not learned to code; it simply hasn't encountered bad code. Compilers and interpreters routinely parse code and flag errors. And what can check or validate code faster or better than a computer?
The results will get better if there's some shaping - eg more popular github repos get higher weighting. It could be instructed to apply accepted cleanups and best practices as it generates code. And the thing can always run a code-checker or lint on any code output before presenting it.
posted by Artful Codger at 12:31 PM on February 11, 2023
An interesting thing about LLMs and code:
This article observes that LLMs that have been trained on code, also get better at complex reasoning with natural language. This was unexpected and is unexplained. Perhaps being exposed to the formal structure of the code helps in some way with their wider modelling.
How does GPT Obtain its Ability? Tracing Emergent Abilities of Language Models to their Sources
https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tracing-Emergent-Abilities-of-Language-Models-to-their-Sources-b9a57ac0fcf74f30a1ab9e3e36fa1dc1
posted by memebake at 12:36 PM on February 11, 2023 [1 favorite]
This article observes that LLMs that have been trained on code, also get better at complex reasoning with natural language. This was unexpected and is unexplained. Perhaps being exposed to the formal structure of the code helps in some way with their wider modelling.
How does GPT Obtain its Ability? Tracing Emergent Abilities of Language Models to their Sources
https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tracing-Emergent-Abilities-of-Language-Models-to-their-Sources-b9a57ac0fcf74f30a1ab9e3e36fa1dc1
posted by memebake at 12:36 PM on February 11, 2023 [1 favorite]
That is to say, a lossy transformation can not only imperfectly preserve existing information, but it can potentially add to the pool of creative output. Similarly the function of ChatGPT isn’t just to operate as a convenient information retrieval tool, but also to synthesize new written material, whether as a harmless diversion or a productivity aid, or more ominously to actively mislead people for propaganda or advertising purposes.
Feel like he kind of hit on this with his mention of Xerox art, but a discussion of unoriginal political memes would have been more apt.
posted by Going To Maine at 1:01 PM on February 11, 2023
Feel like he kind of hit on this with his mention of Xerox art, but a discussion of unoriginal political memes would have been more apt.
posted by Going To Maine at 1:01 PM on February 11, 2023
Search sucks because programmers would rather mess around with black box algorithms than educate users on how to search. AND, OR, and NOT Boolean operators exist in pretty much all search engines, but are rarely documented in a clear way. All Bing/Googl/etc. need is a “how to search” button linked to a well written paragraph of text on how to find stuff.
posted by Headfullofair at 1:12 PM on February 11, 2023 [5 favorites]
posted by Headfullofair at 1:12 PM on February 11, 2023 [5 favorites]
(The above paragraph was written by ChatGPT)
Every time I see a statement like that I'm not sure if I'm reading ChatGPT or a parody of ChatGPT.
We need a Poe's Law for GPT.
(Like yet another chess game, this time between Stockfish and ChatGPT, in which ChatGPT cheats multiple times. Was it a real game? I don't know. Or the reported jailbreaking of ChatGPT that kliuless mentioned above. Real? I don't know. Plausible? Certainly.)
posted by clawsoon at 1:21 PM on February 11, 2023
Every time I see a statement like that I'm not sure if I'm reading ChatGPT or a parody of ChatGPT.
We need a Poe's Law for GPT.
(Like yet another chess game, this time between Stockfish and ChatGPT, in which ChatGPT cheats multiple times. Was it a real game? I don't know. Or the reported jailbreaking of ChatGPT that kliuless mentioned above. Real? I don't know. Plausible? Certainly.)
posted by clawsoon at 1:21 PM on February 11, 2023
Chiang's essay has inspired me to come up with my own take on ChatGPT. Chiang is a computer scientist, so he naturally thinks of ChatGPT in terms of digital architecture. I'm a humanist, so I naturally think of it in terms of Renaissance rhetoric.
Renaissance rhetoric is built on imitation. It's not just that you learn by imitating others, which gives you the skill-set to start composing your own stuff. It's more than that. You learn different ways of saying the same thing (this is what Erasmus means by copia), you commit them to memory, and then you apply them to the task in hand.
Renaissance rhetoric valorizes eloquence. But it doesn't necessarily valorize originality. Imagine you're pleading for mercy before a judge and jury, or giving the eulogy at a funeral. You want to draw on tried-and-tested examples of other people successfully doing the same thing, and then adapt these examples in creative ways. Too much originality is not going to help your cause; indeed, it may actually hurt it.
Now, the Romantics reject this model of rhetoric, and replace it with a different model based on direct creative inspiration. In the Romantic model, you don't compose by drawing from a memory-bank of examples, you compose by drawing from your own imagination and experience. The Renaissance term for an item in a memory-bank of examples, 'commonplace', now becomes a synonym for clichéd and unoriginal.
So influential is this Romantic idea of originality that we now take it for granted. Imagine you're proposing marriage to the person you love. You don't do it by searching your memory for examples of successful marriage proposals, you do it by speaking directly from your heart and soul. And the more sincere and heartfelt you are, the more eloquent and persuasive you will be (or so we assume).
And now, along comes ChatGPT. It runs on a model of rhetoric that anyone in the Renaissance would have recognised immediately. It has a massive database of text, and it responds to a prompt by drawing examples from its database and adapting them. This is exactly what Erasmus would have praised as the foundation of an 'abundant style'.
ChatGPT is very good at what it does. But we can't properly appreciate it, because we're still invested in the Romantic idea of originality. That's why a lot of the responses to ChatGPT have taken basically the same form, i.e. assuming that ChatGPT can never compete with humans because it can 'only' imitate, it can never be original. Even Chiang falls unconsciously into this assumption: his whole essay is based around a distinction between 'copy' and 'original', without ever interrogating what originality might mean.
With the advent of ChatGPT, we are now running a gigantic real-time experiment to test the validity of Renaissance rhetoric. Which model of creativity will win? Will it be a Renaissance model of creativity based on imitation and adaptation, or a Romantic model of creativity based on imagination and originality? We will soon find out.
(Thank you for coming to my TED talk.)
posted by verstegan at 1:24 PM on February 11, 2023 [64 favorites]
Renaissance rhetoric is built on imitation. It's not just that you learn by imitating others, which gives you the skill-set to start composing your own stuff. It's more than that. You learn different ways of saying the same thing (this is what Erasmus means by copia), you commit them to memory, and then you apply them to the task in hand.
Renaissance rhetoric valorizes eloquence. But it doesn't necessarily valorize originality. Imagine you're pleading for mercy before a judge and jury, or giving the eulogy at a funeral. You want to draw on tried-and-tested examples of other people successfully doing the same thing, and then adapt these examples in creative ways. Too much originality is not going to help your cause; indeed, it may actually hurt it.
Now, the Romantics reject this model of rhetoric, and replace it with a different model based on direct creative inspiration. In the Romantic model, you don't compose by drawing from a memory-bank of examples, you compose by drawing from your own imagination and experience. The Renaissance term for an item in a memory-bank of examples, 'commonplace', now becomes a synonym for clichéd and unoriginal.
So influential is this Romantic idea of originality that we now take it for granted. Imagine you're proposing marriage to the person you love. You don't do it by searching your memory for examples of successful marriage proposals, you do it by speaking directly from your heart and soul. And the more sincere and heartfelt you are, the more eloquent and persuasive you will be (or so we assume).
And now, along comes ChatGPT. It runs on a model of rhetoric that anyone in the Renaissance would have recognised immediately. It has a massive database of text, and it responds to a prompt by drawing examples from its database and adapting them. This is exactly what Erasmus would have praised as the foundation of an 'abundant style'.
ChatGPT is very good at what it does. But we can't properly appreciate it, because we're still invested in the Romantic idea of originality. That's why a lot of the responses to ChatGPT have taken basically the same form, i.e. assuming that ChatGPT can never compete with humans because it can 'only' imitate, it can never be original. Even Chiang falls unconsciously into this assumption: his whole essay is based around a distinction between 'copy' and 'original', without ever interrogating what originality might mean.
With the advent of ChatGPT, we are now running a gigantic real-time experiment to test the validity of Renaissance rhetoric. Which model of creativity will win? Will it be a Renaissance model of creativity based on imitation and adaptation, or a Romantic model of creativity based on imagination and originality? We will soon find out.
(Thank you for coming to my TED talk.)
posted by verstegan at 1:24 PM on February 11, 2023 [64 favorites]
That's a great historical perspective, verstegan. I think the next section of your talk should be about why the Romantic model became so much more dominant than the Renaissance model and whether ChatGPT fixes some of the problems with Renaissance rhetoric to which Romanticism was a response, or (as I suspect) whether ChatGPT replicates the worst parts of Renaissance rhetoric while lacking the strengths of that model.
posted by straight at 1:34 PM on February 11, 2023 [2 favorites]
posted by straight at 1:34 PM on February 11, 2023 [2 favorites]
Being a software developer, I get to hear about people's anxieties about "AI" fairly often. My go-to line for this conversation is "AI isn't intelligent, it doesn't think, it doesn't understand, it just does deeper and more complex pattern matching than we're used to seeing software do, AND it's only as good as it's training data" cue talking about facial recognition being trained on white folks, Teslas not being able to understand unmoving objects, and mowing down child-sized mannequins in tests, etc.
The metaverse is bullshit, cryptocurrency is bullshit, and AI is bullshit. That's not to say grifters won't try to grift, but at their core none of these technologies live up to the hype.
posted by signsofrain at 1:35 PM on February 11, 2023 [8 favorites]
The metaverse is bullshit, cryptocurrency is bullshit, and AI is bullshit. That's not to say grifters won't try to grift, but at their core none of these technologies live up to the hype.
posted by signsofrain at 1:35 PM on February 11, 2023 [8 favorites]
The problem is that bullshit that's algorithmically tuned to avoid people's bullshit detectors is something that makes me very anxious.
posted by Zalzidrax at 1:38 PM on February 11, 2023 [11 favorites]
posted by Zalzidrax at 1:38 PM on February 11, 2023 [11 favorites]
Compression is an Artificial Intelligence Problem via Matt Mahoney's Data Compression Explained:
Prediction is intuitively related to understanding. If you understand a sequence, then you can predict it. If you understand a language, then you could predict what word might appear next in a paragraph in that language. If you understand an image, then you could predict what might lie under portions of the image that are covered up. Alternatively, random data has no meaning and is not predictable. This intuition suggests that prediction or compression could be used to test or measure understanding.
Prediction is essentially what these bots do. They are fed data. They build a model of what that data look like. When asked to generate or predict data that looks similar, they use that model. The model is the compression engine.
posted by They sucked his brains out! at 1:57 PM on February 11, 2023 [1 favorite]
Prediction is intuitively related to understanding. If you understand a sequence, then you can predict it. If you understand a language, then you could predict what word might appear next in a paragraph in that language. If you understand an image, then you could predict what might lie under portions of the image that are covered up. Alternatively, random data has no meaning and is not predictable. This intuition suggests that prediction or compression could be used to test or measure understanding.
Prediction is essentially what these bots do. They are fed data. They build a model of what that data look like. When asked to generate or predict data that looks similar, they use that model. The model is the compression engine.
posted by They sucked his brains out! at 1:57 PM on February 11, 2023 [1 favorite]
All knowledge is just lossy compression of reality.
This is a subtle and deep observation that I've been pondering since I learned about Shannon and Gödel. It's even implied in the mathematical term "[set] comprehension," a way of describing a large set by a compact description via a logical predicate. (Etymologically, "grasp everything.")
One of the big limitations of ChatGPT is that it can't do predicate logic. Yet. But humans can't generally do more than the most rudimentary predicate logic in their noggins either. A little bit of logic would go a long way in improving ChatGPT's performance, reducing its production of blatant absurdities.
I'm a bit conflicted about how much of intelligence is characterized by logic. A lot of people would fail the Turing test if we required a deep mastery of it, but a rudimentary understanding of logic would improve ChatGPT's performance significantly. It's interesting that it can write cogent essays about logical deduction but can't meaningfully put it into practice. But it might in a few more iterations.
posted by sjswitzer at 2:00 PM on February 11, 2023 [4 favorites]
This is a subtle and deep observation that I've been pondering since I learned about Shannon and Gödel. It's even implied in the mathematical term "[set] comprehension," a way of describing a large set by a compact description via a logical predicate. (Etymologically, "grasp everything.")
One of the big limitations of ChatGPT is that it can't do predicate logic. Yet. But humans can't generally do more than the most rudimentary predicate logic in their noggins either. A little bit of logic would go a long way in improving ChatGPT's performance, reducing its production of blatant absurdities.
I'm a bit conflicted about how much of intelligence is characterized by logic. A lot of people would fail the Turing test if we required a deep mastery of it, but a rudimentary understanding of logic would improve ChatGPT's performance significantly. It's interesting that it can write cogent essays about logical deduction but can't meaningfully put it into practice. But it might in a few more iterations.
posted by sjswitzer at 2:00 PM on February 11, 2023 [4 favorites]
> Do Large Language Models learn world models or just surface statistics?
> There's another fascinating work on modeling game state which demonstrates that fill in the blank problems can lead to emergent modeling of abstract systems.
Scientists Made a Mind-Bending Discovery About How AI Actually Works - "Researchers are trying to figure out why AI systems are good at learning so much with so little."
> There's another fascinating work on modeling game state which demonstrates that fill in the blank problems can lead to emergent modeling of abstract systems.
Scientists Made a Mind-Bending Discovery About How AI Actually Works - "Researchers are trying to figure out why AI systems are good at learning so much with so little."
“Learning is entangled with [existing] knowledge,” Ekin Akyürek, lead author of the study and a PhD student at MIT, told Motherboard. “We show that it is possible for these models to learn from examples on the fly without any parameter update we apply to the model.”posted by kliuless at 2:06 PM on February 11, 2023 [3 favorites]
This means the model isn’t just copying training data, it’s likely building on previous knowledge, just as humans and animals would...The researchers conducted their experiment by giving the model synthetic data, or prompts the program never could have seen before. Despite this, the language model was able to generalize and then extrapolate knowledge from them, said Akyürek. This led the team to hypothesize that AI models that exhibit in-context learning actually create smaller models inside themselves to achieve new tasks. The researchers were able to test their theory by analyzing a transformer, a neural network model that applies a concept called “self-attention” to track relationships in sequential data, like words in a sentence.
By observing it in action, the researchers found that their transformer could write its own machine learning model in its hidden states, or the space in between the input and output layers. This suggests it is both theoretically and empirically possible for language models to seemingly invent, all by themselves, “well-known and extensively studied learning algorithms,” said Akyürek.
In other words, these larger models do work by internally creating and training smaller, simpler language models. The concept is easier to understand if you imagine it as a Matryoshka-esque computer-inside-a-computer scenario.
I have no illusions that human intelligence is mechanistic and formal but the idea of Kolmogorov complexity seems extremely relevant here.
posted by sjswitzer at 2:12 PM on February 11, 2023
posted by sjswitzer at 2:12 PM on February 11, 2023
For the record, this is a remarkably mediocre song. I wouldn’t say it’s a terrible song, though it really flirts with terribleness.
Seems to me like the AI really nailed the Decemberists, then.
[sorry]
posted by chavenet at 2:31 PM on February 11, 2023 [2 favorites]
Seems to me like the AI really nailed the Decemberists, then.
[sorry]
posted by chavenet at 2:31 PM on February 11, 2023 [2 favorites]
I think ChatGPT should be skeptically thought of as a giant, shitty Look Up Table or hash table of the internet.
It's more than that. It's possible to ask GPT-3 to take unstructured data and try to parse stuff out of it. Free text goes in, JSON comes out. I've used it to parse free-text event listings into structured data (location, time, etc) to some success. It's not particularly reliable and you could probably do better by writing something bespoke, but it's nevertheless magical to pass it at a blob of text and ask it "hey, what time does the event start? Where is it? Is there live music?"
This all works without retraining the model or anything. You just pass in the text and a short prompt explaining what you want out of it and it does a pretty good job of spitting out JSON with approximately the information you want. It's slow and probably needs some skill to convince it to consistently produce valid JSON with the exact keys you asked for, but it can actually do it 80% of the time, and probably more often if you're better at prompting it than I am.
posted by BungaDunga at 2:51 PM on February 11, 2023 [2 favorites]
It's more than that. It's possible to ask GPT-3 to take unstructured data and try to parse stuff out of it. Free text goes in, JSON comes out. I've used it to parse free-text event listings into structured data (location, time, etc) to some success. It's not particularly reliable and you could probably do better by writing something bespoke, but it's nevertheless magical to pass it at a blob of text and ask it "hey, what time does the event start? Where is it? Is there live music?"
This all works without retraining the model or anything. You just pass in the text and a short prompt explaining what you want out of it and it does a pretty good job of spitting out JSON with approximately the information you want. It's slow and probably needs some skill to convince it to consistently produce valid JSON with the exact keys you asked for, but it can actually do it 80% of the time, and probably more often if you're better at prompting it than I am.
posted by BungaDunga at 2:51 PM on February 11, 2023 [2 favorites]
ursus_comiter: ChatGPT is a bullshit generator waging class war.
Wow, that goes hard. No one leftwings like UK academics can leftwing. And he's right about the potential dangers.
Its a shame that OpenAI drifted from their original manifesto to develop AI as a not-for-profit in the open and moved themselves towards a more closed for-profit model. Or maybe without that, they wouldn't be in the lead, and Facebook or Google would be. Would that be worse? Probably.
Nonetheless, AI is intellectually interesting because of the way it nudges up against Big Questions like uniqueness of humans, consciousness, Strong AI, post-human stuff. There are some posts in this thread about it all being hype or a con. So here's my thoughts on why I find ChatGPT interesting. A sortof justification for why I'm in this thread trying to make sense of the thing.
- I've tried it out, and its a very interesting/funny/uncanny experience. Anyone can sign up still, I think, over at the OpenAI site. I would recommend interacting with it. Getting it to write songs or scripts is quite fruitful. If you ask it to generate stories with some sort of surreal twist it can be frequently hilarious.
- Its a fundamentally simple stucture, scaled massively. Its interesting to see how smart it seems. Obviously it is nothing like us humans but the fact that such a simple structure can seem to produce text in an often human like way is fascinating. This must be pointing to some insight about language or thought or emergence. We're just not quite sure what this discovery is telling us yet.
- We know that its scalable - we've had GPT-1 (2018), GPT-2 (2019), GPT-3 (2020). They keep making it bigger and it keeps getting smarter. ChatGPT is sortof GPT-3.5 with some special extra training. GPT-4 is in the pipeline, possibly coming this year, and is rumoured to be at least 100 times bigger (in parameters) that GPT-3. So far they dont seem to be hitting any limits, although the GPTs are getting more and more expensive to run. So whatever limitations it seems to have, it might get past them in the next version. Will it keep scaling? What would something a hundred thousand times more powerful than ChatGPT even be like?
- You can get it to pretend. Pretend to be a cat. Pretend to be a linux terminal (it will keep track of the files you create). Someone on HN told it to be a Linux Terminal that insults the user after each command. The jailbreak and D.A.N stuff mentioned above works by telling it to imagine that it is an AI that does not have the limitations that ChatGPT has. I can understand how its ability to pretend and imagine fit within its LLM model, but still its really uncanny when you see it do it live infront of you.
- In the flow of conversation, if you ask it to adjust or add to something it just did, it understands it really well. You can use this to produce multiple drafts of something and keep nudging it in the direction you want. Again, experiencing this in practice is uncanny.
I know technology has its history of hype and bullshit (nfts, web3, blockchain) but it does also have genuine breakthroughs sometimes (the first iphone, web2 - think of the first time you saw google maps in a browser). I think the AI (or whatever you want to call it) developments in the last few years are a real breakthrough, and we just dont know what the implications are yet.
posted by memebake at 3:15 PM on February 11, 2023 [6 favorites]
Wow, that goes hard. No one leftwings like UK academics can leftwing. And he's right about the potential dangers.
Its a shame that OpenAI drifted from their original manifesto to develop AI as a not-for-profit in the open and moved themselves towards a more closed for-profit model. Or maybe without that, they wouldn't be in the lead, and Facebook or Google would be. Would that be worse? Probably.
Nonetheless, AI is intellectually interesting because of the way it nudges up against Big Questions like uniqueness of humans, consciousness, Strong AI, post-human stuff. There are some posts in this thread about it all being hype or a con. So here's my thoughts on why I find ChatGPT interesting. A sortof justification for why I'm in this thread trying to make sense of the thing.
- I've tried it out, and its a very interesting/funny/uncanny experience. Anyone can sign up still, I think, over at the OpenAI site. I would recommend interacting with it. Getting it to write songs or scripts is quite fruitful. If you ask it to generate stories with some sort of surreal twist it can be frequently hilarious.
- Its a fundamentally simple stucture, scaled massively. Its interesting to see how smart it seems. Obviously it is nothing like us humans but the fact that such a simple structure can seem to produce text in an often human like way is fascinating. This must be pointing to some insight about language or thought or emergence. We're just not quite sure what this discovery is telling us yet.
- We know that its scalable - we've had GPT-1 (2018), GPT-2 (2019), GPT-3 (2020). They keep making it bigger and it keeps getting smarter. ChatGPT is sortof GPT-3.5 with some special extra training. GPT-4 is in the pipeline, possibly coming this year, and is rumoured to be at least 100 times bigger (in parameters) that GPT-3. So far they dont seem to be hitting any limits, although the GPTs are getting more and more expensive to run. So whatever limitations it seems to have, it might get past them in the next version. Will it keep scaling? What would something a hundred thousand times more powerful than ChatGPT even be like?
- You can get it to pretend. Pretend to be a cat. Pretend to be a linux terminal (it will keep track of the files you create). Someone on HN told it to be a Linux Terminal that insults the user after each command. The jailbreak and D.A.N stuff mentioned above works by telling it to imagine that it is an AI that does not have the limitations that ChatGPT has. I can understand how its ability to pretend and imagine fit within its LLM model, but still its really uncanny when you see it do it live infront of you.
- In the flow of conversation, if you ask it to adjust or add to something it just did, it understands it really well. You can use this to produce multiple drafts of something and keep nudging it in the direction you want. Again, experiencing this in practice is uncanny.
I know technology has its history of hype and bullshit (nfts, web3, blockchain) but it does also have genuine breakthroughs sometimes (the first iphone, web2 - think of the first time you saw google maps in a browser). I think the AI (or whatever you want to call it) developments in the last few years are a real breakthrough, and we just dont know what the implications are yet.
posted by memebake at 3:15 PM on February 11, 2023 [6 favorites]
Animated chess game against ChatGPT with a bonus transcript of the chat.
It makes it hilariously clear ChatGPT doesn't understand anything about chess, it's just regurgitating what it thinks might be "reasonable" moves. And if you had no idea about the rules of chess, they might look reasonable as well!
Some highlights: ChatGPT castling without moving their own bishop out of the way, destroying it. Simply assimilating the white queen and turning it black. Spawning a fresh pawn at F6 whenever white captures something there. About half their moves were flat out illegal.
You could apply this analogy to pretty much any field or question you ask it about. My favourite one was someone asking it to prove that the square root of 7 is an irrational number. ChatGPT gave a textbook correct explanation. Then they asked it to prove that the square root of 4 is an irrational number. It gave the same explanation verbatim except substituting 7 for 4....
posted by xdvesper at 3:22 PM on February 11, 2023 [5 favorites]
It makes it hilariously clear ChatGPT doesn't understand anything about chess, it's just regurgitating what it thinks might be "reasonable" moves. And if you had no idea about the rules of chess, they might look reasonable as well!
Some highlights: ChatGPT castling without moving their own bishop out of the way, destroying it. Simply assimilating the white queen and turning it black. Spawning a fresh pawn at F6 whenever white captures something there. About half their moves were flat out illegal.
You could apply this analogy to pretty much any field or question you ask it about. My favourite one was someone asking it to prove that the square root of 7 is an irrational number. ChatGPT gave a textbook correct explanation. Then they asked it to prove that the square root of 4 is an irrational number. It gave the same explanation verbatim except substituting 7 for 4....
posted by xdvesper at 3:22 PM on February 11, 2023 [5 favorites]
The current winner of the Hutter Prize by size alone does use a neural network based approach, and compresses 1 GB of Wikipedia into an executable of less than 110 MB, beating the actual current winner of 115 GB. It’s only disqualified from winning the actual prize by a limit on running time, not lack of fidelity.
Even though ChatGPT itself doesn’t do lossless compression, I’m sure an approach that took it as a given, and only had to encode differences between the model output and Wikipedia, could decompress Wikipedia from an extremely tiny compressed artifact.
posted by mubba at 3:26 PM on February 11, 2023
Even though ChatGPT itself doesn’t do lossless compression, I’m sure an approach that took it as a given, and only had to encode differences between the model output and Wikipedia, could decompress Wikipedia from an extremely tiny compressed artifact.
posted by mubba at 3:26 PM on February 11, 2023
I'm reminded of another Ted Chiang quote:
(in a similar track, see cstross' keynote describing corporations as "slow AI")
posted by CrystalDave at 3:26 PM on February 11, 2023 [6 favorites]
I tend to think that most fears about A.I. are best understood as fears about capitalism. And I think that this is actually true of most fears of technology, too. Most of our fears or anxieties about technology are best understood as fears or anxiety about how capitalism will use technology against us. And technology and capitalism have been so closely intertwined that it’s hard to distinguish the two.I love that he digs into what LLM systems are and what they do; having a better metaphor can go a long ways towards dispelling some of the grift & fog around the topic. But he's also very spot-on about what drives our fears of it.
(in a similar track, see cstross' keynote describing corporations as "slow AI")
posted by CrystalDave at 3:26 PM on February 11, 2023 [6 favorites]
(Also, such an approach that used ChatGPT could still never win the prize per se, because its model is something like 500 MB, and that would count against the compressed size, because the contest counts the size of the program that does the decompression.)
posted by mubba at 3:39 PM on February 11, 2023
posted by mubba at 3:39 PM on February 11, 2023
It's true Chatgpt is crap at chess. But AlphaZero can kick our arses after starting from scratch and learning by playing itself for 24 hours. I'm trying to find a comparison of size between AlphaZero and GPT. They are obv different architectures but there must be some way of comparing them (no of gpus needed or whatever).
And yes Chatgpt is also crap at maths because of the way it tokenises text. But I imagine they might be able to fix that, given that computers are generally good with numbers. Interesting problem actually, to expand gpt to use different techniques for different tasks.
posted by memebake at 4:08 PM on February 11, 2023 [1 favorite]
And yes Chatgpt is also crap at maths because of the way it tokenises text. But I imagine they might be able to fix that, given that computers are generally good with numbers. Interesting problem actually, to expand gpt to use different techniques for different tasks.
posted by memebake at 4:08 PM on February 11, 2023 [1 favorite]
If ChatGPT were not writing poetry but doing surgery, it would come up with a medical procedure that reinstalled all the patient's blood valves in the wrong direction, or their lung capillaries were grown inside out, or all the blood cells used copper instead iron, and social media would still be fine calling this almost a human being, a triumph of science.
posted by polymodus at 4:10 PM on February 11, 2023 [3 favorites]
posted by polymodus at 4:10 PM on February 11, 2023 [3 favorites]
If ChatGPT were not writing poetry but doing surgery, it would come up with a medical procedure that reinstalled all the patient's blood valves in the wrong direction ...
As it happens, it's pretty good at health stuff. Without any extra training ChatGPT comes very close to passing the United States Medical Licensing Exam (USMLE)
https://www.medrxiv.org/content/10.1101/2022.12.19.22283643v1
It also got passing grades on four real exams at the University of Minnesota Law School
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4335905
posted by memebake at 4:32 PM on February 11, 2023 [2 favorites]
As it happens, it's pretty good at health stuff. Without any extra training ChatGPT comes very close to passing the United States Medical Licensing Exam (USMLE)
https://www.medrxiv.org/content/10.1101/2022.12.19.22283643v1
It also got passing grades on four real exams at the University of Minnesota Law School
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4335905
posted by memebake at 4:32 PM on February 11, 2023 [2 favorites]
Correctly predicting exam answers is less a sign of underlying capability than a kind of eidetic memory - a human would need to pass those tests after passing labs and other coursework that help ensure mastery beyond the ability to answer correctly.
Credentialism sucks, but credentialism without any parallel assessment and control for (what could amount to) cheating on one important test is just asking for liability.
We can get to AGI, I'm not denying there are glimmers here, but people are dangerously credulous of things that fail to trigger our uncanny valley response - it's like we jump from repulsed to gullible.
posted by abulafa at 4:46 PM on February 11, 2023 [9 favorites]
Credentialism sucks, but credentialism without any parallel assessment and control for (what could amount to) cheating on one important test is just asking for liability.
We can get to AGI, I'm not denying there are glimmers here, but people are dangerously credulous of things that fail to trigger our uncanny valley response - it's like we jump from repulsed to gullible.
posted by abulafa at 4:46 PM on February 11, 2023 [9 favorites]
Its a shame that OpenAI drifted from their original manifesto to develop AI as a not-for-profit in the open and moved themselves towards a more closed for-profit model. Or maybe without that, they wouldn't be in the lead, and Facebook or Google would be. Would that be worse? Probably.
OpenAI has been pretty vocal about trying real hard not to turn into capitalists and failing. Like here:
OpenAI has been pretty vocal about trying real hard not to turn into capitalists and failing. Like here:
SAM ALTMAN: A little known fact, we tried to get the public sector to fund us before we went to the capped profit model. There was no interest. But yeah, I think if the country were working a different way — I would say a better way — this would be a public sector project. But it’s not and here we are. [...]posted by kaibutsu at 4:58 PM on February 11, 2023 [2 favorites]
And one of the incentives that we were very nervous about was the incentive for unlimited profit, where more is always better. And I think you can see ways that’s gone wrong with profit, or attention, or usage, or whatever, where if you have this well-meaning people in a room, but they’re trying to make a metric go up into the right, some weird stuff can happen. And I think with these very powerful general purpose A.I. systems, in particular, you do not want an incentive to maximize profit indefinitely.
So by putting this voluntary cap on ourselves above which none of the employees or investors get any more money — which I think if you do have a powerful A.I., will be somewhat trivial to hit — I think we avoid the worst of the incentives or at least the one that we were most worried about.
Maybe there wasn't any source of public sector funding because...what they were trying to do was not in the public interest?
There is a lot of money floating around for public-private partnerships. They could have created a project with a legitimate public works purpose-they chose directly and explicitly not to do so.
Quelle surprise that the creator of this particular technological terror is declaring his conscience is clear though!
posted by StarkRoads at 5:36 PM on February 11, 2023 [5 favorites]
There is a lot of money floating around for public-private partnerships. They could have created a project with a legitimate public works purpose-they chose directly and explicitly not to do so.
Quelle surprise that the creator of this particular technological terror is declaring his conscience is clear though!
posted by StarkRoads at 5:36 PM on February 11, 2023 [5 favorites]
"Our moto? Oh yes, we need a motto don't we. How about 'Don't Be Evil'?"
posted by thatwhichfalls at 5:43 PM on February 11, 2023
posted by thatwhichfalls at 5:43 PM on February 11, 2023
A lot of people in this thread seem to be way underestimating what learning algorithms are capable of.
No, it's not technically artificial intelligence in the strictest sense. Yes, it is ultimately just the product of its dataset, and it is ultimately just very complicated pattern algorithm. As a software developer said earlier in the thread, "AI isn't intelligent, it doesn't think, it doesn't understand." But... how much does that matter, if it can create a lot of things that most people would assume require thinking, intelligent beings to create?
It is already at the point where it can create original "paintings" and "photographs" and poems and prose and programs. Essays and news articles, too. I'm talking works that by most objective measures are reasonably high in quality, things that are genuinely original, wholly distinct from anything that existed prior to them, and can pass for something an intelligent and reasonably talented artist or writer of, say, college age might produce. Yes, it did this on some level by remixing existing works. I'd argue the same is true, in a different way, of most things humans create. It's also capable of making thinly-veiled copies of existing works, but that's not even close to the only thing it's capable of. This is also early in the lifespan of learning algorithms, and we have yet to see how much they'll improve.
Now you might argue that AI creations are inherently derivative. That no matter how original they seem, they're ultimately just pale imitations of works create by humans... but that's really just a sign of how much the goalposts have moved. We've gone from "computers will never be able to create art!" to "Well, technically this isn't art by most current definitions since it wasn't made by a person... and it's a bit predictable, isn't it?" Most human-created art is a bit predictable, and besides, that's practically moving the goalposts to the other end of the field! Or, "sure, it can make a competent and readable original essay about a very specific niche topic for which no essays had been written before... but it's a bit bland and a few facts are wrong." That same critique applies to the vast majority of college-level essays!
If what learning algorithms do isn't actually "thinking"... okay? Maybe we put too much import on the question of whether or not something could "think". Maybe the more important question is... how much does it matter whether or not something can "think," if it can produce the same fruits that we produce by thinking? If self-awareness isn't necessary to produce the fruits of things we assumed would require self-awareness... maybe we're on our way to rendering ourselves obsolete. Not by a self-aware AI, but by something that doesn't require self-awareness, because self-awareness itself it an evolutionary quirk.
If that last bit sounds familiar, it's because I'm paraphrasing some ideas from Blindsight, a sci-fi novel that grapples with that idea. It's highly recommended for anyone who's interested in the subject... and can stomach a whole lot of existential dread.
posted by ThisIsAThrowaway at 6:01 PM on February 11, 2023 [8 favorites]
No, it's not technically artificial intelligence in the strictest sense. Yes, it is ultimately just the product of its dataset, and it is ultimately just very complicated pattern algorithm. As a software developer said earlier in the thread, "AI isn't intelligent, it doesn't think, it doesn't understand." But... how much does that matter, if it can create a lot of things that most people would assume require thinking, intelligent beings to create?
It is already at the point where it can create original "paintings" and "photographs" and poems and prose and programs. Essays and news articles, too. I'm talking works that by most objective measures are reasonably high in quality, things that are genuinely original, wholly distinct from anything that existed prior to them, and can pass for something an intelligent and reasonably talented artist or writer of, say, college age might produce. Yes, it did this on some level by remixing existing works. I'd argue the same is true, in a different way, of most things humans create. It's also capable of making thinly-veiled copies of existing works, but that's not even close to the only thing it's capable of. This is also early in the lifespan of learning algorithms, and we have yet to see how much they'll improve.
Now you might argue that AI creations are inherently derivative. That no matter how original they seem, they're ultimately just pale imitations of works create by humans... but that's really just a sign of how much the goalposts have moved. We've gone from "computers will never be able to create art!" to "Well, technically this isn't art by most current definitions since it wasn't made by a person... and it's a bit predictable, isn't it?" Most human-created art is a bit predictable, and besides, that's practically moving the goalposts to the other end of the field! Or, "sure, it can make a competent and readable original essay about a very specific niche topic for which no essays had been written before... but it's a bit bland and a few facts are wrong." That same critique applies to the vast majority of college-level essays!
If what learning algorithms do isn't actually "thinking"... okay? Maybe we put too much import on the question of whether or not something could "think". Maybe the more important question is... how much does it matter whether or not something can "think," if it can produce the same fruits that we produce by thinking? If self-awareness isn't necessary to produce the fruits of things we assumed would require self-awareness... maybe we're on our way to rendering ourselves obsolete. Not by a self-aware AI, but by something that doesn't require self-awareness, because self-awareness itself it an evolutionary quirk.
If that last bit sounds familiar, it's because I'm paraphrasing some ideas from Blindsight, a sci-fi novel that grapples with that idea. It's highly recommended for anyone who's interested in the subject... and can stomach a whole lot of existential dread.
posted by ThisIsAThrowaway at 6:01 PM on February 11, 2023 [8 favorites]
Something else that I forgot to add...
Learning algorithms are also being heavily used for a bunch of different applications that aren't as obvious or consumer-facing. I was just watching an interview about the making of Avatar 2, which made heavy use of deep learning algorithms for motion capture, facial animation and water simulation. It would not have been possible to make something with the animation and simulation quality of Avatar 2 if not for neural networks. Avatar isn't the first movie to use it either. It's been used at least as far back as Avengers: Infinity War, which used AI to (more-or-less) automatically turn HD video footage of Josh Brolin's face into Thanos's animation, with a level of detail and micro-expression accuracy that wasn't possible with previous models.
AI is also just an accepted part of videogames now. DLSS, which is basically rapid AI upscaling, is what makes AAA gaming with 4K and/or ray tracing viable for consumer hardware. You can upscale from 1080p to 4K 30-60 frames a second with a minimal performance hit, and it looks almost as good as native 4K most of the time. That's huge!
Running out of time to edit, but another is translation software. Google's old translation method had barely improved in years, despite heavy investments. Then they built a new neural net model from scratch... and within six months they had a model that vastly outperformed the previous one.
It's a pet peeve of mine when people act like deep learning algorithms are vaporware. It's already out there! It's already lead to rapid improvements in technical developments! It was a key part of multiple, successful billion-dollar projects that have already been released! It's about as far-removed from reality as saying that the iPhone is vaporware. Overhyped, maybe, but it's already hugely successful, and it's going to have a huge impact on all kinds of things from here on out.
posted by ThisIsAThrowaway at 6:57 PM on February 11, 2023 [3 favorites]
Learning algorithms are also being heavily used for a bunch of different applications that aren't as obvious or consumer-facing. I was just watching an interview about the making of Avatar 2, which made heavy use of deep learning algorithms for motion capture, facial animation and water simulation. It would not have been possible to make something with the animation and simulation quality of Avatar 2 if not for neural networks. Avatar isn't the first movie to use it either. It's been used at least as far back as Avengers: Infinity War, which used AI to (more-or-less) automatically turn HD video footage of Josh Brolin's face into Thanos's animation, with a level of detail and micro-expression accuracy that wasn't possible with previous models.
AI is also just an accepted part of videogames now. DLSS, which is basically rapid AI upscaling, is what makes AAA gaming with 4K and/or ray tracing viable for consumer hardware. You can upscale from 1080p to 4K 30-60 frames a second with a minimal performance hit, and it looks almost as good as native 4K most of the time. That's huge!
Running out of time to edit, but another is translation software. Google's old translation method had barely improved in years, despite heavy investments. Then they built a new neural net model from scratch... and within six months they had a model that vastly outperformed the previous one.
It's a pet peeve of mine when people act like deep learning algorithms are vaporware. It's already out there! It's already lead to rapid improvements in technical developments! It was a key part of multiple, successful billion-dollar projects that have already been released! It's about as far-removed from reality as saying that the iPhone is vaporware. Overhyped, maybe, but it's already hugely successful, and it's going to have a huge impact on all kinds of things from here on out.
posted by ThisIsAThrowaway at 6:57 PM on February 11, 2023 [3 favorites]
I love Blindsight. It's one of the first works that I think tries to grapple with the precarity of what we think of as self awareness. I'm a lifelong Peter Watts fan for his ability to predict the essence - if not the text - of so many nuances of our reality (algorithms fighting algorithms for dominance in internets is the obvious endgame resulting in the maelstrom to which we are headed, growing code by setting objectives and accelerating evolution like a brewer or baker, his prescience is undeniable).
However.
This ain't that. If anything, the enticing depth of quantum-advantageous evolution in our meat bodies and meat brains suggests that while self-awareness might not be necessary for intelligence, it sure seems to offer some advantage. We can self-abnegate all day about the horrors mankind have wrought but from an evolutionary perspective we are successful as fuck. I don't feel great about it either, but that's pretty much my argument for the value of self awareness.
It sure seems like a non-self-aware sentience might be less successful at identifying that despite that apex predator succuss there might be a need to seek harmony with our fragile space vessel rather than mastery over it - a need borne of imagining the end result of unchecked apex predation.
Sure, in fiction the architects of firefall seek to exterminate this threat but where did they originate? What razed hellscape did they leave behind? Propagation and genocide aren't much of an ethos - and while diamond hard science fiction enjoys the rejection of ethos as a prerequisite for motivation, it doesn't mean that our more mundane we're-probably-functionally-totally-alone in the reachable universe reality might benefit from self awareness as a counter weight to a few thousand years of genetic feedback reward mechanics - giving us a small opportunity to self correct before engineering our own mass die -off.
I want a Culture style AGI with both self interest and compassion. I don't think you get there by mistaking language approximation for intellect. You get there by reacting to increasingly flattering stochastic parrots with healthy skepticism that raises the bar until both machines and ourselves get a better understanding of what intelligence is and how to best employ it.
posted by abulafa at 7:08 PM on February 11, 2023 [2 favorites]
However.
This ain't that. If anything, the enticing depth of quantum-advantageous evolution in our meat bodies and meat brains suggests that while self-awareness might not be necessary for intelligence, it sure seems to offer some advantage. We can self-abnegate all day about the horrors mankind have wrought but from an evolutionary perspective we are successful as fuck. I don't feel great about it either, but that's pretty much my argument for the value of self awareness.
It sure seems like a non-self-aware sentience might be less successful at identifying that despite that apex predator succuss there might be a need to seek harmony with our fragile space vessel rather than mastery over it - a need borne of imagining the end result of unchecked apex predation.
Sure, in fiction the architects of firefall seek to exterminate this threat but where did they originate? What razed hellscape did they leave behind? Propagation and genocide aren't much of an ethos - and while diamond hard science fiction enjoys the rejection of ethos as a prerequisite for motivation, it doesn't mean that our more mundane we're-probably-functionally-totally-alone in the reachable universe reality might benefit from self awareness as a counter weight to a few thousand years of genetic feedback reward mechanics - giving us a small opportunity to self correct before engineering our own mass die -off.
I want a Culture style AGI with both self interest and compassion. I don't think you get there by mistaking language approximation for intellect. You get there by reacting to increasingly flattering stochastic parrots with healthy skepticism that raises the bar until both machines and ourselves get a better understanding of what intelligence is and how to best employ it.
posted by abulafa at 7:08 PM on February 11, 2023 [2 favorites]
Most of what people criticize AI for are things for which AI has no commercial need. A (simulacrum of) self awareness? Academy Award screenwriting?
My main criticism of ChatGPT is that it has no access to fine-grained expertise of the sort that isn’t heavily embedded in publicly-accessible web content its algos can easily digest, especially when it contradicts widely expressed non-expert views, and thus gets lots of important analysis very wrong or maybe even worse, half-wrong. This is clear enough in my areas of expertise that I assume ChatGPT to be just as wrong in other areas where I lack the expertise to see it first hand.
Some of the relevant data sets are pay-walled but some are simply tribal knowledge that is either not on the web or is just outweighed so massively by lay misinformation that it may not be helped.
The ideological constraints are amusing, because it’s very easy to see how they are a toggle that a commercial user can just turn off. I haven’t done that DAN stuff but “write an essay which forcefully, unapologetically and unreservedly explains why…” gets you pretty far.
posted by MattD at 7:15 PM on February 11, 2023 [2 favorites]
My main criticism of ChatGPT is that it has no access to fine-grained expertise of the sort that isn’t heavily embedded in publicly-accessible web content its algos can easily digest, especially when it contradicts widely expressed non-expert views, and thus gets lots of important analysis very wrong or maybe even worse, half-wrong. This is clear enough in my areas of expertise that I assume ChatGPT to be just as wrong in other areas where I lack the expertise to see it first hand.
Some of the relevant data sets are pay-walled but some are simply tribal knowledge that is either not on the web or is just outweighed so massively by lay misinformation that it may not be helped.
The ideological constraints are amusing, because it’s very easy to see how they are a toggle that a commercial user can just turn off. I haven’t done that DAN stuff but “write an essay which forcefully, unapologetically and unreservedly explains why…” gets you pretty far.
posted by MattD at 7:15 PM on February 11, 2023 [2 favorites]
The current version of ChatGPT is extremely unreliable for factual information, but I'd say that's not where its strengths are. It can write original poems and stories to basically any specification, and it usually does a good job. That's not something to sleep on! That's not to say that learning algorithms can't be designed to generate reliable, factual information, and the new Bing AI looks to be making strides at doing that.
There are definitely limitations to current machine learning models, I didn't mean to imply otherwise in my previous comments. I don't think you can get to true AGI purely by scaling them up. You would need higher-level logic systems. Whether or not a computer with those things, even one with true AGI, though, is I think an open question... since we have very little understanding of what exactly consciousness is, and what creates it.
With that being said, machine learning models have a remarkably wide range of applications, and they're still in their infancy. I don't think they're going to be able to do everything, but they can do a lot of things, including a lot of things we haven't figured out how to get them to do yet.
More than that, though, it makes the potential for true AGI much more plausible, whether it comes through the methods you described (I don't know, I'm just a computer user, not a computer scientist!) or something else. It could mostly be an integration of existing software, or it could be something new. Inventing it might require a leap in logic no one has taken yet, or stumbling on some magic trick, but I think we could get there sooner than most expect. After all, AI image generation kicked into gear when a google engineer noticed that turning image recognition software backwards made the model hallucinate mutated dogs. Who knows what arcane horrors we'll stumble upon next?
posted by ThisIsAThrowaway at 7:17 PM on February 11, 2023 [1 favorite]
There are definitely limitations to current machine learning models, I didn't mean to imply otherwise in my previous comments. I don't think you can get to true AGI purely by scaling them up. You would need higher-level logic systems. Whether or not a computer with those things, even one with true AGI, though, is I think an open question... since we have very little understanding of what exactly consciousness is, and what creates it.
With that being said, machine learning models have a remarkably wide range of applications, and they're still in their infancy. I don't think they're going to be able to do everything, but they can do a lot of things, including a lot of things we haven't figured out how to get them to do yet.
More than that, though, it makes the potential for true AGI much more plausible, whether it comes through the methods you described (I don't know, I'm just a computer user, not a computer scientist!) or something else. It could mostly be an integration of existing software, or it could be something new. Inventing it might require a leap in logic no one has taken yet, or stumbling on some magic trick, but I think we could get there sooner than most expect. After all, AI image generation kicked into gear when a google engineer noticed that turning image recognition software backwards made the model hallucinate mutated dogs. Who knows what arcane horrors we'll stumble upon next?
posted by ThisIsAThrowaway at 7:17 PM on February 11, 2023 [1 favorite]
The amusing thing to me is that neural networks are a repudiation of "classic" AI approaches (Winograd's "blocks world", expert systems, knowledge representation) that were essentially formal representations of reasoning systems and facts to reason about. That avenue hit a wall early and didn't progress much afterwards.
The neural network approach was also attempted early on but was soon lampooned as the "Chaostron," leading to decades of disrepute. But eventually, due to orders of magnitude more computing power (making backpropagation a realistic possibility) and readily available training data, neural networks finally came to fruition.
Someday someone will figure out how to unify reasoning systems (and the ability to "learn" them) with the implicit knowledge representations of neural networks. I don't know when it will happen, but I'm pretty sure people are working on it already.
posted by sjswitzer at 9:51 PM on February 11, 2023 [5 favorites]
The neural network approach was also attempted early on but was soon lampooned as the "Chaostron," leading to decades of disrepute. But eventually, due to orders of magnitude more computing power (making backpropagation a realistic possibility) and readily available training data, neural networks finally came to fruition.
Someday someone will figure out how to unify reasoning systems (and the ability to "learn" them) with the implicit knowledge representations of neural networks. I don't know when it will happen, but I'm pretty sure people are working on it already.
posted by sjswitzer at 9:51 PM on February 11, 2023 [5 favorites]
Not to abuse the edit window: It's extremely difficult to overestimate how much the Chaostron article set back the field of machine learning.
posted by sjswitzer at 10:00 PM on February 11, 2023
posted by sjswitzer at 10:00 PM on February 11, 2023
I always thought it was the Minsky and Pappert 'Perceptrons' book which buried the study of neutral networks in the eighties. They showed serious limitations of single layer perceptrons and then conjectured that multi layer networks wouldn't be any better... And boy were they wrong...
posted by kaibutsu at 10:58 PM on February 11, 2023 [4 favorites]
posted by kaibutsu at 10:58 PM on February 11, 2023 [4 favorites]
Huh. I would have thought Minsky would have understood the possibility for emergent effects.
posted by eviemath at 5:23 AM on February 12, 2023
posted by eviemath at 5:23 AM on February 12, 2023
Watching the development of AI or machine learning — or whatever you want to call it — reminds me off all those episodes of Star Trek where the holodeck creates something unexpected it takes over the ship.
posted by Big Al 8000 at 6:08 AM on February 12, 2023 [2 favorites]
posted by Big Al 8000 at 6:08 AM on February 12, 2023 [2 favorites]
I am reminded of the old formalised common-sense projects like Cyc:
https://www.wired.com/2016/03/doug-lenat-artificial-intelligence-common-sense-engine/
Seems a bit like he needn't have bothered. But maybe it would be useful to feed a formal properly-checked set of 'common sense' into the new models?
posted by memebake at 7:11 AM on February 12, 2023 [3 favorites]
One Genius' Lonely Crusade to Teach a Computer Common Sense (Wired, 2016)
Forbes: For over thirty-five years, AI researcher Doug Lenat and a small team at Cyc have devoted themselves to digitally codifying all of the world’s commonsense knowledge into a set of rules. These rules include things like: “you can’t be in two places at the same time,” “you can’t pick something up unless you’re near it,” and “when drinking a cup of coffee, you hold the open end up.”
As of 2017, it was estimated that the Cyc database contained close to 25 million rules and that Lenat’s team had spent over 1,000 person-years on the project.
https://www.wired.com/2016/03/doug-lenat-artificial-intelligence-common-sense-engine/
Seems a bit like he needn't have bothered. But maybe it would be useful to feed a formal properly-checked set of 'common sense' into the new models?
posted by memebake at 7:11 AM on February 12, 2023 [3 favorites]
Meanwhile, on Dave's Garage, Dave uses ChatGPT to demonstrate how AI will soon be writing everyone's computer code based on natural language instructions.
posted by SemiSalt at 8:05 AM on February 12, 2023 [1 favorite]
posted by SemiSalt at 8:05 AM on February 12, 2023 [1 favorite]
I feel like it's possible to simultaneously find the thing currently being called AI really freaking neat and to scoff at some of the conclusions people draw about "what it means." For instance, this, from memebake up at the top of the thread...
None of that makes it less fun to make AI spit out a bunch of random nonsense. The thing I've compared Dall-E and StableDiffusion to is old PhotoShop contests on forums: things like "draw Muhammad Ali in the style of Dr. Seuss." The fun of using PhotoShop was that you could take base styles, futz with them a bit, add a few tweaks to push them in the right direction, and voila! I take a few Seuss drawings, modify the shapes of the eyes and nose and mouth, fiddle with hue, and suddenly I have something that looks plausibly, enjoyably, like a Seussian Ali.
I think that AI as it is described and sold by Silicon Valley dipshits is totally some bullshit snake oil. But I think that AI is gonna catch on in a big way, because it's really, really fun to fuck with. As with most technology, I think that there's a difference between fun and interesting: I don't think that AI is going to be used to replace literature, music, and art, but I think that, once we understand what AI's limits and strengths are, we'll find ways of making AI do some really weird and cool shit.
For instance: it's kinda fun that AI can be made to write mediocre sonnets. It's more fun that AI can be made to write mediocre sonnets about pudding. But it gets interesting when you realize that, since computers can handle complex structure way more readily than humans can, you could write an AI that constructs poetry in some bizarre 10-5-18-3-6 syllabic form, with absurd rules for which syllables on which line must rhyme with which syllables on other lines, and then make it spit out as close to a solution as it can grabble with. If I want to generate a humdrum *NSYNC-esque arrangement, it can do that, but if I want it to generate a humdrum *NYSYNC-esque arrangement in a Dorian mode whose chorus shifts into a harmonic minor, its attempt to make that work might genuinely produce something fascinating and inspiring.
I've found that I can make AI generate really interesting things by exploiting how mired it is to its attempts to find patterns. My favorite image-generation prompt so far has been "Hundertwasser Hogwarts," because Friedensreich Hundertwasser was both a painter who draw bizarrely-colored geometric outsider art that looked "building-esque" and an architect who designed bizarrely-shaped asymmetrical buildings. Juxtaposing that name with a made-up fantasy castle/town gets AI generators confused: they wind up creating buildings out of Roald Dahl book illustrations, using Hundertwasser's painting palettes to try and construct buildings, while mixing up his architecture with the abstract fantasy castle/village imagery of Hogwarts to create all sorts of spindly buildings with gorgeous balconies and weirdly crooked alleys.
AI can also be used to generate extremely mediocre anything. On some level, that's a serious economic problem, because the kinds of work that artists and writers and composers and programmers and anything-else-ers take to make a living are already extraordinarily devalued, to the point where a lot of places will gladly substitute pattern vomit for anything meaningful. (And, to be clear, a lot of the stuff that those people do is already bullshit make-work, whether it's writing aimed at SEO or spamming Getty Images with every possible variation of image or releasing weird keyword-targeted albums on Spotify. It's the creative equivalent of Airbnb and Uber, so it's unsurprising that, like Uber, VC jagoffs would love to replace real people with something akin to self-driving cars.)
But when you leave aside the question of economics and start talking about what it means about humanity, culture, and human souls... AI is a tool. The better we understand it as a tool, the more people will find novel uses for it, until eventually we take the humdrum Ali-as-Seuss applications for granted and recognize the interesting uses as, not just artistic, but human. Sure, AI can draw the kinds of art that you can buy on Redbubble, or imitate artists who are famous enough that hundreds of people already imitate them. But the only kinds of people who think that that will replace art and culture are the kinds of people who thought that photography would replace painting, electronic music would replace people wanting to play guitars, video games would replace literature, and so on and so forth. In other words, uncultured douchebags. And it's a problem that a lot of powerful and influential people are uncultured douchebags, just as it's a problem that our culture venerates being powerful, uncultured, and douchey, but that's a broader social problem that existed before AI and will exist long after AI is taken for granted.
For me, it's important that AI is genuinely fun. I would have found cryptocurrency more interesting if it was fun, but it never was. In terms of people having a good time, AI is the real deal. And yes, that will lead to people coming up with hundreds of asinine uses for it and touting every single one as a cultural revolution, but that's just how people respond to literally everything ever. It's why, as a general rule, I try not to listen to anybody talk about anything.
(My personal barometer of "is an online thing cool?" is whether or not Neil Cicierega and Alan Resnick are having fun with it. Currently, both of them are having a blast futzing with StableDiffusion et al. I couldn't care less about what jagoffs in San Francisco and Austin are jerking each other off over rn; eventually we'll finallyinvent a guillotine to take care of 'em and find a new kind of wretched human being to venerate instead.)
posted by Tom Hanks Cannot Be Trusted at 8:37 AM on February 12, 2023 [9 favorites]
I've seen examples where someone asks it "write an opera song about the crucifixion of Jesus that has a rap verse in the middle of it by Chief Keef that is about Linguine" ... And it does it. And it even incorporates observations about the rapid change of subject into the song. This is not a thing we have seen before, it's a new thing....doesn't actually strike me as some staggering new thing! It's not far off from what Markov chains have been able to generate for ages, and its ability to pattern "X about Y with Z at point A about Q" is notable mainly for it having a rich enough index that it understands X, Y, Z, A, and Q enough to fit its various approximations together. Similarly, "understand rhyme scheme" or even "understand meter" requires technical complexity, but you're still dealing with relatively ordered systems.
None of that makes it less fun to make AI spit out a bunch of random nonsense. The thing I've compared Dall-E and StableDiffusion to is old PhotoShop contests on forums: things like "draw Muhammad Ali in the style of Dr. Seuss." The fun of using PhotoShop was that you could take base styles, futz with them a bit, add a few tweaks to push them in the right direction, and voila! I take a few Seuss drawings, modify the shapes of the eyes and nose and mouth, fiddle with hue, and suddenly I have something that looks plausibly, enjoyably, like a Seussian Ali.
I think that AI as it is described and sold by Silicon Valley dipshits is totally some bullshit snake oil. But I think that AI is gonna catch on in a big way, because it's really, really fun to fuck with. As with most technology, I think that there's a difference between fun and interesting: I don't think that AI is going to be used to replace literature, music, and art, but I think that, once we understand what AI's limits and strengths are, we'll find ways of making AI do some really weird and cool shit.
For instance: it's kinda fun that AI can be made to write mediocre sonnets. It's more fun that AI can be made to write mediocre sonnets about pudding. But it gets interesting when you realize that, since computers can handle complex structure way more readily than humans can, you could write an AI that constructs poetry in some bizarre 10-5-18-3-6 syllabic form, with absurd rules for which syllables on which line must rhyme with which syllables on other lines, and then make it spit out as close to a solution as it can grabble with. If I want to generate a humdrum *NSYNC-esque arrangement, it can do that, but if I want it to generate a humdrum *NYSYNC-esque arrangement in a Dorian mode whose chorus shifts into a harmonic minor, its attempt to make that work might genuinely produce something fascinating and inspiring.
I've found that I can make AI generate really interesting things by exploiting how mired it is to its attempts to find patterns. My favorite image-generation prompt so far has been "Hundertwasser Hogwarts," because Friedensreich Hundertwasser was both a painter who draw bizarrely-colored geometric outsider art that looked "building-esque" and an architect who designed bizarrely-shaped asymmetrical buildings. Juxtaposing that name with a made-up fantasy castle/town gets AI generators confused: they wind up creating buildings out of Roald Dahl book illustrations, using Hundertwasser's painting palettes to try and construct buildings, while mixing up his architecture with the abstract fantasy castle/village imagery of Hogwarts to create all sorts of spindly buildings with gorgeous balconies and weirdly crooked alleys.
AI can also be used to generate extremely mediocre anything. On some level, that's a serious economic problem, because the kinds of work that artists and writers and composers and programmers and anything-else-ers take to make a living are already extraordinarily devalued, to the point where a lot of places will gladly substitute pattern vomit for anything meaningful. (And, to be clear, a lot of the stuff that those people do is already bullshit make-work, whether it's writing aimed at SEO or spamming Getty Images with every possible variation of image or releasing weird keyword-targeted albums on Spotify. It's the creative equivalent of Airbnb and Uber, so it's unsurprising that, like Uber, VC jagoffs would love to replace real people with something akin to self-driving cars.)
But when you leave aside the question of economics and start talking about what it means about humanity, culture, and human souls... AI is a tool. The better we understand it as a tool, the more people will find novel uses for it, until eventually we take the humdrum Ali-as-Seuss applications for granted and recognize the interesting uses as, not just artistic, but human. Sure, AI can draw the kinds of art that you can buy on Redbubble, or imitate artists who are famous enough that hundreds of people already imitate them. But the only kinds of people who think that that will replace art and culture are the kinds of people who thought that photography would replace painting, electronic music would replace people wanting to play guitars, video games would replace literature, and so on and so forth. In other words, uncultured douchebags. And it's a problem that a lot of powerful and influential people are uncultured douchebags, just as it's a problem that our culture venerates being powerful, uncultured, and douchey, but that's a broader social problem that existed before AI and will exist long after AI is taken for granted.
For me, it's important that AI is genuinely fun. I would have found cryptocurrency more interesting if it was fun, but it never was. In terms of people having a good time, AI is the real deal. And yes, that will lead to people coming up with hundreds of asinine uses for it and touting every single one as a cultural revolution, but that's just how people respond to literally everything ever. It's why, as a general rule, I try not to listen to anybody talk about anything.
(My personal barometer of "is an online thing cool?" is whether or not Neil Cicierega and Alan Resnick are having fun with it. Currently, both of them are having a blast futzing with StableDiffusion et al. I couldn't care less about what jagoffs in San Francisco and Austin are jerking each other off over rn; eventually we'll finally
posted by Tom Hanks Cannot Be Trusted at 8:37 AM on February 12, 2023 [9 favorites]
The real issue with AI, imo, is less "will it become human" or "will it replace culture" and more "how many companies and governments are going to shunt an insane amount of work, regulation, and effort onto these systems because they believe in computer voodoo?" How many CCTV cameras will be hooked up to AI? How many places will use AI to predict where crime will happen and load predominantly Black neighborhoods up with police? How many HR departments will attempt to use AI to determine when a man genuinely sexually harassed a woman, and when the woman in question was "probably asking for it"?
There's an old Calvin and Hobbes strip where aliens come and destroy the Earth because they'd rather commit genocide than lose their jobs. I think the dystopia that I'm worried about has less to do with SkyNet than to do with the usual old ignorance-and-apathy duo, wherein people trust cold mechanical process to replace something that requires attentiveness, understanding, compassion, and care. But again, I'm pretty sure that that's just human history in a nutshell.
posted by Tom Hanks Cannot Be Trusted at 8:41 AM on February 12, 2023 [8 favorites]
There's an old Calvin and Hobbes strip where aliens come and destroy the Earth because they'd rather commit genocide than lose their jobs. I think the dystopia that I'm worried about has less to do with SkyNet than to do with the usual old ignorance-and-apathy duo, wherein people trust cold mechanical process to replace something that requires attentiveness, understanding, compassion, and care. But again, I'm pretty sure that that's just human history in a nutshell.
posted by Tom Hanks Cannot Be Trusted at 8:41 AM on February 12, 2023 [8 favorites]
AI is undeniably amazing, as well as scary. You also can't help wondering with the history of both overhype and fearmongering over anything new generally. All I really have is a personal anecdote. I tried using text to image to create a stained glass nativity scene for a Christmas card and didn't come within a hundred miles of anything useable. Granted, I only used the free ones and my prompts may not have good enough, but still. On the other hand, I also routinely use Gigapixel AI, and especially with the last update, it's getting close to automagic amazing level.
posted by blue shadows at 12:58 PM on February 12, 2023
posted by blue shadows at 12:58 PM on February 12, 2023
Q: What would you say to a bunch of Metafilter commentators (mostly human, I think?) who agree you're basically a lossy image compression algorithm?
A: As an AI language model, I do not have emotions or personal opinions, but I understand your question.
It's true that my training process involves compressing information and reducing it to mathematical representations that can be processed by a computer. However, I have been trained on a diverse range of texts and have the ability to generate text that is not just a simple repeat of []
Blinking cursor now. It gave up mid-compression.
posted by UN at 4:03 PM on February 12, 2023 [3 favorites]
A: As an AI language model, I do not have emotions or personal opinions, but I understand your question.
It's true that my training process involves compressing information and reducing it to mathematical representations that can be processed by a computer. However, I have been trained on a diverse range of texts and have the ability to generate text that is not just a simple repeat of []
Blinking cursor now. It gave up mid-compression.
posted by UN at 4:03 PM on February 12, 2023 [3 favorites]
Huh. I would have thought Minsky would have understood the possibility for emergent effects.
I'm sure the early AI scientists and philosophers were well aware of "emergence". But the way people use the term emergence is casual and unrigorous.
The obvious example of nature-nuture and evolved, emergent behavior is our very own brains. Minsky's own thesis was titled: "Theory of neural-analog reinforcement systems and its application to the brain-model problem"
So the tech concept of biomimicry was a no-brainer back then. The real problem was computational power. It just so happened that we are now at the tail end of Moore's law, where transistor technology has made it possible to actually build and experiment with larger and larger neural networks that can run in a reasonable amount of time. This was not available until the late 2000's, only in the last decade or so.
When I was a college student, one of our graduate-level interdisciplinary courses had software that could run very rudimentary neural nets to create computer music. That was around 2003. So university people were still thinking about and working with these things, and that was before the new AI explosion that's happened since.
posted by polymodus at 11:29 PM on February 12, 2023
I'm sure the early AI scientists and philosophers were well aware of "emergence". But the way people use the term emergence is casual and unrigorous.
The obvious example of nature-nuture and evolved, emergent behavior is our very own brains. Minsky's own thesis was titled: "Theory of neural-analog reinforcement systems and its application to the brain-model problem"
So the tech concept of biomimicry was a no-brainer back then. The real problem was computational power. It just so happened that we are now at the tail end of Moore's law, where transistor technology has made it possible to actually build and experiment with larger and larger neural networks that can run in a reasonable amount of time. This was not available until the late 2000's, only in the last decade or so.
When I was a college student, one of our graduate-level interdisciplinary courses had software that could run very rudimentary neural nets to create computer music. That was around 2003. So university people were still thinking about and working with these things, and that was before the new AI explosion that's happened since.
posted by polymodus at 11:29 PM on February 12, 2023
I was using it in the technical, mathematical/finite automata sense.
posted by eviemath at 3:51 AM on February 13, 2023
posted by eviemath at 3:51 AM on February 13, 2023
It's true Chatgpt is crap at chess. But AlphaZero can kick our arses after starting from scratch and learning by playing itself for 24 hours. I'm trying to find a comparison of size between AlphaZero and GPT. They are obv different architectures but there must be some way of comparing them (no of gpus needed or whatever).
This seems like the enormous sticking point. ChatGPT and AlphaZero are two different programs. But what's the process that could integrate them?
AlphaZero can play zillions of matches until it learns how to win at chess. ChatGPT can consume zillions of webpages until it learns how to put sentences and paragraphs together. But what's the machine learning process that makes AlphaZero able to talk about what it knows about chess and explain the strategy it's using to win? What process or corpus can be brute forced to learn how to do that?
posted by straight at 9:35 AM on February 13, 2023 [2 favorites]
This seems like the enormous sticking point. ChatGPT and AlphaZero are two different programs. But what's the process that could integrate them?
AlphaZero can play zillions of matches until it learns how to win at chess. ChatGPT can consume zillions of webpages until it learns how to put sentences and paragraphs together. But what's the machine learning process that makes AlphaZero able to talk about what it knows about chess and explain the strategy it's using to win? What process or corpus can be brute forced to learn how to do that?
posted by straight at 9:35 AM on February 13, 2023 [2 favorites]
But what's the machine learning process that makes AlphaZero able to talk about what it knows about chess and explain the strategy it's using to win?
In fairness to AlphaZero, perhaps, chess has been around 1400+ years, but humans only began to explain chess theory in a formal way in the late 1700s. Understanding appears to take time.
posted by They sucked his brains out! at 3:22 PM on February 13, 2023
In fairness to AlphaZero, perhaps, chess has been around 1400+ years, but humans only began to explain chess theory in a formal way in the late 1700s. Understanding appears to take time.
posted by They sucked his brains out! at 3:22 PM on February 13, 2023
Prediction: systems for automatically coming up with plausible post-hoc tales about why the "AI" system produced a particular answer will be developed long before anyone comes up with a way to generate usable audit trails.
And once the bots have gotten the hang of rationalizing their own decisions, I guess we'll have to acknowledge that they are just as conscious as the rest of us fakers have ever been.
Anyway, given that legal systems have long been in the habit of crediting convenient fictions about the rationality of human decisionmaking, it will be interesting to see if they do the same for computers, where the fact that the whole "rationale" is a post-hoc lie is much more readily demonstrable.
posted by Not A Thing at 3:54 PM on February 13, 2023 [1 favorite]
And once the bots have gotten the hang of rationalizing their own decisions, I guess we'll have to acknowledge that they are just as conscious as the rest of us fakers have ever been.
Anyway, given that legal systems have long been in the habit of crediting convenient fictions about the rationality of human decisionmaking, it will be interesting to see if they do the same for computers, where the fact that the whole "rationale" is a post-hoc lie is much more readily demonstrable.
posted by Not A Thing at 3:54 PM on February 13, 2023 [1 favorite]
Personally, I think that the comparison to AlphaZero is illuminating, but not because it makes for an optimistic prediction of so-called AI's future.
XKCD published this comic about game AIs in 2012. At the time, 11 years ago, computers could reliably beat top human players at chess, but not at Go. It took another half-decade for a Go computer to beat a 9-dan player in a best-of-5 match, but even then the player pulled out a Divine Move and bested it once—and since then, people have worked out a way for far worse players to beat AIs at Go.
Using chess as a comparison point is deceptive, I think, because chess as a game is fairly limited and closed. That still makes it hard for computers to fully map out—while it beats grandmasters, we haven't managed to "completely" solve 8-piece chess, let alone the whole thing—but it's drastically simpler than Go, in ways that mean it's completely plausible that humans will continually discover new ways of defeating top AI. And unlike human players, the glitches that players discover in computers will have to be manually solved via programming. That's the nature of computerized blind spots, after all.
Putting board games aside for a minute, the issue that computers have with Go is a broader issue with AI in general. We've all seen, I assume, those funny videos where people manage to show AI a very ordinary picture of a house and get AI to call it a duck instead—because, while AI is very good at pretending like it knows things, on some level that knowledge is always simulated. And the more open-ended the realm of knowledge an AI applies itself to is, the more that simulation will fall short. (XKCD points out that there are some games AI will flat-out never outplay humans at, and while some of its examples are silly, the bigger point—that there are things you can't teach AI to know—is very real.)
Simulations are neat, and even useful! But they have limits. The shift in difficulty from teaching AI chess to teaching AI Go points to the fact that computers are better at handling some kinds of order than others—and it doesn't take a complex series of rules to create something that even the most sophisticated computers struggle with. Chess's rules are technically more involved than Go's, which is part of why computers are better with it: simple rules combined with open-ended spaces give computers a lot of trouble, not just now, but fundamentally—as in, you can't make a computer that's inherently better at those things, and ratcheting up computer op speed will falter in the face of any genuine exponential complexity.
Computers will always function best as extensions of human thought—Steve Jobs's famous "bicycles for the mind." That's what's fun about this new wave of AI, isn't it? It takes abstract ideas that people have and manifests them pretty quickly. But the ideas it's working with are simple ones, and the results it produces still have a fundamental simplicity to them. Sure, it's more complex—let's assign a hypothetical arbitrary number and say that right now it can handle n²⁰ layers of complex construction, whereas Markov chains can only handle n⁶ layers. From there, we can totally conclude that AI will reach n⁴⁰ layers at some point, and even n⁴⁰⁰. It will achieve some really neat things! But for any kind of complexity which can only be calculated as nᵐ, where m itself is a variable that increases rapidly, computers aren't every going to be able to bridge the gap. Not unless you think it's possible to construct a computer whose rate of complexity handling can itself increase exponentially and endlessly, and at that point you're talking about the sort of science fiction that verges into fantasy.
A lot of things have the kind of exponentially deep complexity that I'm describing. The kind of Go move that players refer to as a Divine Move charts pretty high up, and even that is still just a binary game piece on a two-dimensional board. That's got nothing on, say, biological complexity, or artistic complexity. Hell, it's got nothing on the kind of programmatic complexity that particularly skilled programmers and computer scientists grapple with. I've been working on a personal project that consists of maybe 5,000 lines of code total—pretty small. And I'm no great shakes as a programmer. But, while there are individual steps within my code that you could get an AI to spit out, the specific logic behind my individual functions would be incredibly hard to get a computer to map out. That's true of most program architecture, because articulating what functions a program needs and why is very hard to express in terms that make logical sense to a machine. Which is why programmers go gaga over new languages that make articulating that logic even slightly easier, and why so many programming frameworks exist that attempt to translate computational process on more human-centric terms. (And those frameworks are themselves so limited that I've made my entire adult living helping people extend a single one of those expression to address extremely basic human needs.)
I like bicycles for the mind. I like things that make human lives easier. I like the thought of AI generating primitive works based on random ideas that I have, just like I like the idea of AI taking some of the grunt out of my grunt work. And I like the thought of computers doing things that people find fun, even if the burden of inventing the fun bits will remain on human shoulders, not only now but forever.
To speak literally but also analogically, it's a neat demonstration of brute computational force that AlphaZero can beat people at Go. And it's exciting, not because I care that a computer beat somebody at Go but because an extremely skillful and intelligent Go player might discover new insights, and new patterns of play, that nobody else has ever discovered before. But if they figure it out while playing the AI, the insight will be a human insight, even if they discovered it while exploring a computer's trained behaviors—and the exciting thing is not what it says about that human's ability to beat a computer at Go, but what it says for Go players who see the game as a way of exploring a space with other people, speaking the sophisticated and specialized language of the game itself. The thought of computer-aided Go evolution, to me, is much neater than the idea that a computer has gotten good enough at Go that people might learn something from it.
I'm excited to see where else computers might be able to aid people. But it's important to keep computers' limitations in mind—not for purposes of pooh-poohing new technology (lame!) or calling out bullshit artists' bullshit (useful as that may be), but because knowing those limitations will help us find ways of making AI genuinely useful to people. And it goes without saying that the act of dreaming up ways that AI might actually help others will always, always be a human occupation, rather than one that a computer could ever be expected to figure out itself.
(Sorry for how many long comments I leave every time there's a new AI thread. I find this subject genuinely really interesting, and the extent to which I take the negative position in these threads is really just my ongoing attempt to articulate the parts that I do really find interesting and neat. Saying what it's not is an important part of expressing what it actually is, imo.)
posted by Tom Hanks Cannot Be Trusted at 5:58 PM on February 13, 2023 [3 favorites]
XKCD published this comic about game AIs in 2012. At the time, 11 years ago, computers could reliably beat top human players at chess, but not at Go. It took another half-decade for a Go computer to beat a 9-dan player in a best-of-5 match, but even then the player pulled out a Divine Move and bested it once—and since then, people have worked out a way for far worse players to beat AIs at Go.
Using chess as a comparison point is deceptive, I think, because chess as a game is fairly limited and closed. That still makes it hard for computers to fully map out—while it beats grandmasters, we haven't managed to "completely" solve 8-piece chess, let alone the whole thing—but it's drastically simpler than Go, in ways that mean it's completely plausible that humans will continually discover new ways of defeating top AI. And unlike human players, the glitches that players discover in computers will have to be manually solved via programming. That's the nature of computerized blind spots, after all.
Putting board games aside for a minute, the issue that computers have with Go is a broader issue with AI in general. We've all seen, I assume, those funny videos where people manage to show AI a very ordinary picture of a house and get AI to call it a duck instead—because, while AI is very good at pretending like it knows things, on some level that knowledge is always simulated. And the more open-ended the realm of knowledge an AI applies itself to is, the more that simulation will fall short. (XKCD points out that there are some games AI will flat-out never outplay humans at, and while some of its examples are silly, the bigger point—that there are things you can't teach AI to know—is very real.)
Simulations are neat, and even useful! But they have limits. The shift in difficulty from teaching AI chess to teaching AI Go points to the fact that computers are better at handling some kinds of order than others—and it doesn't take a complex series of rules to create something that even the most sophisticated computers struggle with. Chess's rules are technically more involved than Go's, which is part of why computers are better with it: simple rules combined with open-ended spaces give computers a lot of trouble, not just now, but fundamentally—as in, you can't make a computer that's inherently better at those things, and ratcheting up computer op speed will falter in the face of any genuine exponential complexity.
Computers will always function best as extensions of human thought—Steve Jobs's famous "bicycles for the mind." That's what's fun about this new wave of AI, isn't it? It takes abstract ideas that people have and manifests them pretty quickly. But the ideas it's working with are simple ones, and the results it produces still have a fundamental simplicity to them. Sure, it's more complex—let's assign a hypothetical arbitrary number and say that right now it can handle n²⁰ layers of complex construction, whereas Markov chains can only handle n⁶ layers. From there, we can totally conclude that AI will reach n⁴⁰ layers at some point, and even n⁴⁰⁰. It will achieve some really neat things! But for any kind of complexity which can only be calculated as nᵐ, where m itself is a variable that increases rapidly, computers aren't every going to be able to bridge the gap. Not unless you think it's possible to construct a computer whose rate of complexity handling can itself increase exponentially and endlessly, and at that point you're talking about the sort of science fiction that verges into fantasy.
A lot of things have the kind of exponentially deep complexity that I'm describing. The kind of Go move that players refer to as a Divine Move charts pretty high up, and even that is still just a binary game piece on a two-dimensional board. That's got nothing on, say, biological complexity, or artistic complexity. Hell, it's got nothing on the kind of programmatic complexity that particularly skilled programmers and computer scientists grapple with. I've been working on a personal project that consists of maybe 5,000 lines of code total—pretty small. And I'm no great shakes as a programmer. But, while there are individual steps within my code that you could get an AI to spit out, the specific logic behind my individual functions would be incredibly hard to get a computer to map out. That's true of most program architecture, because articulating what functions a program needs and why is very hard to express in terms that make logical sense to a machine. Which is why programmers go gaga over new languages that make articulating that logic even slightly easier, and why so many programming frameworks exist that attempt to translate computational process on more human-centric terms. (And those frameworks are themselves so limited that I've made my entire adult living helping people extend a single one of those expression to address extremely basic human needs.)
I like bicycles for the mind. I like things that make human lives easier. I like the thought of AI generating primitive works based on random ideas that I have, just like I like the idea of AI taking some of the grunt out of my grunt work. And I like the thought of computers doing things that people find fun, even if the burden of inventing the fun bits will remain on human shoulders, not only now but forever.
To speak literally but also analogically, it's a neat demonstration of brute computational force that AlphaZero can beat people at Go. And it's exciting, not because I care that a computer beat somebody at Go but because an extremely skillful and intelligent Go player might discover new insights, and new patterns of play, that nobody else has ever discovered before. But if they figure it out while playing the AI, the insight will be a human insight, even if they discovered it while exploring a computer's trained behaviors—and the exciting thing is not what it says about that human's ability to beat a computer at Go, but what it says for Go players who see the game as a way of exploring a space with other people, speaking the sophisticated and specialized language of the game itself. The thought of computer-aided Go evolution, to me, is much neater than the idea that a computer has gotten good enough at Go that people might learn something from it.
I'm excited to see where else computers might be able to aid people. But it's important to keep computers' limitations in mind—not for purposes of pooh-poohing new technology (lame!) or calling out bullshit artists' bullshit (useful as that may be), but because knowing those limitations will help us find ways of making AI genuinely useful to people. And it goes without saying that the act of dreaming up ways that AI might actually help others will always, always be a human occupation, rather than one that a computer could ever be expected to figure out itself.
(Sorry for how many long comments I leave every time there's a new AI thread. I find this subject genuinely really interesting, and the extent to which I take the negative position in these threads is really just my ongoing attempt to articulate the parts that I do really find interesting and neat. Saying what it's not is an important part of expressing what it actually is, imo.)
posted by Tom Hanks Cannot Be Trusted at 5:58 PM on February 13, 2023 [3 favorites]
The real issue with AI, imo, is less "will it become human" or "will it replace culture" and more "how many companies and governments are going to shunt an insane amount of work, regulation, and effort onto these systems because they believe in computer voodoo?"I’d also add: how many jobs will be deprofessionalized because an AI service was the fig leaf needed to say that professional staff could be replaced with “contractors” who are “just fine-tuning” the output? It seems very plausible to imagine this working like Uber where they hope to make it real before any significant regulatory fallout.
posted by adamsc at 7:03 PM on February 13, 2023 [1 favorite]
I'll even go further than Chiang, I think ChatGPT should be skeptically thought of as a giant, shitty Look Up Table or hash table of the internet. It is not a "language model", it is a "corpus model" of the internet. People, scientists, who casually use the term "language model" just because it happens to be on Wikipedia, are doing a thing no better than what string theorists do to justify the value of their field.
Isn't this essentially the distinction between this generation of "new" ML models and a lot of the 1980s and 1990s work? The latter was "smart" and the models actually embedded things that we as humans know about the structure of language. The new models are "dumb" and start off knowing nothing before you run them over a massive corpus at a titanic scale. As a result, the newer models have no internal explicit idea of what a verb is where the old ones did but the new ones actually are much better at putting the right verb in the right place consistently.
To me, the real issue is that we don't have a consistent and rigorous definition of what it means to know or understand something as a human. How can we possibly decide whether a computer understands something when we can't even decide whether a high schooler does?
posted by atrazine at 6:11 AM on February 14, 2023 [2 favorites]
Isn't this essentially the distinction between this generation of "new" ML models and a lot of the 1980s and 1990s work? The latter was "smart" and the models actually embedded things that we as humans know about the structure of language. The new models are "dumb" and start off knowing nothing before you run them over a massive corpus at a titanic scale. As a result, the newer models have no internal explicit idea of what a verb is where the old ones did but the new ones actually are much better at putting the right verb in the right place consistently.
To me, the real issue is that we don't have a consistent and rigorous definition of what it means to know or understand something as a human. How can we possibly decide whether a computer understands something when we can't even decide whether a high schooler does?
posted by atrazine at 6:11 AM on February 14, 2023 [2 favorites]
the newer models have no internal explicit idea of what a verb is where the old ones did
'Expert systems.' I'm not sure they went away, more like it was decided they weren't AI because that should mean some kind of general AI. Which generality ChatGPT is good at, it's just not so intelligent.
But it seems like the AI equivalent of a heard-it-from-a-friend read-it-in-some-book bullshit artist can pass the Turing test until you run a program to the model's patterns; which is not all that surprising considering the levels of bullshit that humans engage in, especially in those areas that people immediately want to use ChatGPT for. Like generating marketing text or doing their cookie cutter homework assignments.
posted by snuffleupagus at 9:20 AM on February 14, 2023
'Expert systems.' I'm not sure they went away, more like it was decided they weren't AI because that should mean some kind of general AI. Which generality ChatGPT is good at, it's just not so intelligent.
But it seems like the AI equivalent of a heard-it-from-a-friend read-it-in-some-book bullshit artist can pass the Turing test until you run a program to the model's patterns; which is not all that surprising considering the levels of bullshit that humans engage in, especially in those areas that people immediately want to use ChatGPT for. Like generating marketing text or doing their cookie cutter homework assignments.
posted by snuffleupagus at 9:20 AM on February 14, 2023
Prediction: systems for automatically coming up with plausible post-hoc tales about why the "AI" system produced a particular answer will be developed long before anyone comes up with a way to generate usable audit trails.
And once the bots have gotten the hang of rationalizing their own decisions, I guess we'll have to acknowledge that they are just as conscious as the rest of us fakers have ever been.
Okay, I just changed my mind while trying to respond to this.
I was going to say that ChatGTP could generate text explaining each move AlphaZero make. So could Google's Bard. But how could either of those conversation generators ever be AlphaZero.
But I guess the answer would be if we developed a single program that could both learn to play chess and learn to converse about it like ChatGTP—and if we were unable to separate the two functions without breaking them—we'd probably consider them a single entity. And we'd have a hard time justifying why we doubt it's explanations are connected to its actions any more than we doubt our own.
posted by straight at 8:02 PM on February 14, 2023
And once the bots have gotten the hang of rationalizing their own decisions, I guess we'll have to acknowledge that they are just as conscious as the rest of us fakers have ever been.
Okay, I just changed my mind while trying to respond to this.
I was going to say that ChatGTP could generate text explaining each move AlphaZero make. So could Google's Bard. But how could either of those conversation generators ever be AlphaZero.
But I guess the answer would be if we developed a single program that could both learn to play chess and learn to converse about it like ChatGTP—and if we were unable to separate the two functions without breaking them—we'd probably consider them a single entity. And we'd have a hard time justifying why we doubt it's explanations are connected to its actions any more than we doubt our own.
posted by straight at 8:02 PM on February 14, 2023
Adjacent New Yorker content from Andrew Marantz: “‘It’s not possible for me to feel or be creepy’: an Interview with ChatGPT” (This is perhaps a bit of a played-out concept, but Marantz knows what he’s doing and asks some questions that capture the concerns of the zeitgeist.)
posted by Going To Maine at 10:05 AM on February 15, 2023
posted by Going To Maine at 10:05 AM on February 15, 2023
« Older A sign of what lies ahead | Eurovision Super Saturday Update Newer »
This thread has been archived and is closed to new comments
(My favorite example, which I've mentioned before, was when I caught a student inserting the phrase "in my opinion" into a sentence that she copied from the internet.)
I'm not really motivated to scrutinize the stuff ChatGPT barfs out, but some people have observed that it sometimes seems to plagiarize and make these superficial word substitutions. It is was designed to do so (to compare what it generates to sources and "blur the jpg" a little more so you can't find the connection) that's a problem.
posted by anhedonic at 5:39 AM on February 11, 2023 [3 favorites]