Story of Your Strife
April 3, 2021 4:22 AM Subscribe
Ted Chiang (previously) on why computers won’t make themselves smarter (The New Yorker) and on why most fears about A.I. are best understood as fears about capitalism (NY Times/Archive.is)
YES. I haven' read it yet, but the description given here perfectly mirrors a strongly-held opinion of mine.
thats exactly what an ai would say
posted by lalochezia at 4:34 AM on April 3, 2021 [39 favorites]
TED CHIANG: So, when people quote the Arthur C. Clarke line, they’re mostly talking about marvelous phenomena, that technology allows us to do things that are incredible and things that, in the past, would have been described as magic, simply because they were marvelous and inexplicable. But one of the defining aspects of technology is that eventually, it becomes cheaper, it becomes available to everybody. So things that were, at one point, restricted to the very few are suddenly available to everybody. Things like television — when television was first invented, yeah, that must have seemed amazing, but now television is not amazing because everyone has one. Radio is not amazing. Computers are not amazing. Everyone has one.Clearly this guy has never worked with antenna designers. The products might be in such wide use as not to be considered magical, but the design process requires raw magical talent. It's like if the only way to make newts was to have witches turn people into them. The newts themselves would still be pretty ordinary.
Magic is something which, by its nature, never becomes widely available to everyone. Magic is something that resides in the person and often is an indication that the universe sort of recognizes different classes of people, that there are magic wielders and there are non-magic wielders. That is not how we understand the universe to work nowadays.
And really, it's not just antenna designers. Those folks are magicians' magicians, but IT design in general is treated as if it were magic by everybody except IT designers, even by people who work directly with IT designers. Like, for example, marketers and managers, among whom magical thinking about what can be done and how easily and how long is absolutely the norm.
Which is, of course, exactly why Elon is frightened of magical AI. Antenna designers are magicians' magicians, but that guy is a marketing manager's marketing manager.
posted by flabdablet at 4:41 AM on April 3, 2021 [24 favorites]
Sorry but I really beg to differ on this one. Just intellectually the author is using, frankly outdated examples from computer science. The way to address foundational questions about AI is to talk about complexity theory and models of computation. The author's piece is like writing about cell biology and aliens without showing actual understanding of concepts like DNA and genetics. That's what stuff like P v.s. NP, Church-Turing thesis, foundations in logic and computability, all that highly theoretical material that even undergrads in computer science are not required to take and indeed most elect to skip those classes because they don't like doing mathematical proofs. An author who writes about AI without evincing the relevance of the modern theoretical basis of computer science... is a disservice to the field, whatever their public claim to fame may be. It wouldn't be okay in any other discipline? Just because everyone uses computers, or every parent has a child, doesn't mean their opinions about AI or child education are worth paying attention to.
And speaking of proofs, take the Curry-Howard isomorphism. The loose, loose idea of it is that algorithms have a dual relationship to mathematical proofs. "Programs are proofs", so goes the saying. So it is better, and mathematically justified, to think of compilers not as mere optimizers improving on a menial task, but proofs about proofs. Now is that not a profound an act of creativity, viewed that way? So that's an example of why knowledge of theoretical CS matters when attempting such philosophical questions.
It's funny that I even find myself in a position of defending AI, because I never cared for computers as a young student. Or rather, I say this in defense of the potentiality of AI advances, and the misapplication, and lack of representation, of computer science concepts in mainstream media. Even with Turing machines alone, people in theory-adjacent fields misunderstand (or, more charitably, disagree) what they're supposed to be.
Personally, as for the human or super AI question, the answer to the author's demand is straightforward. Intelligence is very finite. It is not magic (unlike God). It is not like time travel, which Einstein's relativity in principle allows. So unless humans ourselves are more computationally powerful than a universal Turing machine given finite time and space resources, it is the argument that artificial life is not attainable that has a burden of compelling proof. In my opinion.
posted by polymodus at 5:11 AM on April 3, 2021 [13 favorites]
And speaking of proofs, take the Curry-Howard isomorphism. The loose, loose idea of it is that algorithms have a dual relationship to mathematical proofs. "Programs are proofs", so goes the saying. So it is better, and mathematically justified, to think of compilers not as mere optimizers improving on a menial task, but proofs about proofs. Now is that not a profound an act of creativity, viewed that way? So that's an example of why knowledge of theoretical CS matters when attempting such philosophical questions.
It's funny that I even find myself in a position of defending AI, because I never cared for computers as a young student. Or rather, I say this in defense of the potentiality of AI advances, and the misapplication, and lack of representation, of computer science concepts in mainstream media. Even with Turing machines alone, people in theory-adjacent fields misunderstand (or, more charitably, disagree) what they're supposed to be.
Personally, as for the human or super AI question, the answer to the author's demand is straightforward. Intelligence is very finite. It is not magic (unlike God). It is not like time travel, which Einstein's relativity in principle allows. So unless humans ourselves are more computationally powerful than a universal Turing machine given finite time and space resources, it is the argument that artificial life is not attainable that has a burden of compelling proof. In my opinion.
posted by polymodus at 5:11 AM on April 3, 2021 [13 favorites]
Metafilter: The description given here perfectly mirrors a strongly-held opinion of mine.
posted by New Frontier at 5:29 AM on April 3, 2021 [38 favorites]
posted by New Frontier at 5:29 AM on April 3, 2021 [38 favorites]
Until the process of being alive can be fully accounted for as a computation, arguments that computational complexity is a useful lens through which to view it have very limited utility. In my opinion.
I also note in passing that at no point does Chiang actually argue that artificial life is not attainable.
it is better, and mathematically justified, to think of compilers not as mere optimizers improving on a menial task, but proofs about proofs. Now is that not a profound an act of creativity, viewed that way?
I doubt you'd get any disagreement from Chiang that the creation of a highly performant optimizing compiler is a profound act of creativity on the part of the entity who wrote the source code.
posted by flabdablet at 5:49 AM on April 3, 2021 [8 favorites]
I also note in passing that at no point does Chiang actually argue that artificial life is not attainable.
it is better, and mathematically justified, to think of compilers not as mere optimizers improving on a menial task, but proofs about proofs. Now is that not a profound an act of creativity, viewed that way?
I doubt you'd get any disagreement from Chiang that the creation of a highly performant optimizing compiler is a profound act of creativity on the part of the entity who wrote the source code.
posted by flabdablet at 5:49 AM on April 3, 2021 [8 favorites]
I'm not sure that being able to fully predict what a C. elegans will do is essential, even though they're pretty simple creatures (as biology goes) that we understand relatively well.
However, we haven't even gotten started on building a smarter C. elegans.
posted by Nancy Lebovitz at 5:52 AM on April 3, 2021 [1 favorite]
However, we haven't even gotten started on building a smarter C. elegans.
posted by Nancy Lebovitz at 5:52 AM on April 3, 2021 [1 favorite]
Until the process of being alive can be fully accounted for as a computation, arguments that computational complexity is a useful lens through which to view it have very limited utility. In my opinion.
In my opinion, Experiencing and the Creation of Meaning gives a convincing argument that being alive cannot be so accounted for.
posted by Obscure Reference at 5:58 AM on April 3, 2021
In my opinion, Experiencing and the Creation of Meaning gives a convincing argument that being alive cannot be so accounted for.
posted by Obscure Reference at 5:58 AM on April 3, 2021
I too used to think of antenna design as black magic too.
Then I got into it some more, learned how to use the simulation tools, designed some antennas, built them, tested them, tuned them and so on.
And found that it's just regular old engineering. Yes, you can get more efficiency, directional gain, or narrower or wider bandwidth, or more bands covered or whatever - but each comes with tradeoffs and limits. It will get bigger, or more expensive, or require finer tolerances and be harder to make, or harder to tune accurately and keep tuned. And ultimately there are physical limits to what can be achieved in practice.
Like every other sort of engineering, antenna design is about finding the best balance across all those tradeoffs for the task at hand. You can't make an antenna that's good at everything.
Or rather, if you can, it's not a single antenna. It's a collection of different antennas that you use some intelligence to select between and use one or more of, depending on the task at hand.
Which is rather what a modern compiler looks like. It's a big collection of individual tools. Each one applies a particular transformation to the code - looking for and applying some particular optimisation, or making some transformation from one form to another. Those tools are collected together, by humans, into a particular sequence of passes that has been tested and shown to provide good results for a wide range of code.
But again, there are tradeoffs. If you want the compiler to generate faster code, you need to enable more optimisation features and the compilation will take longer to run. Sometimes exponentially so for larger inputs. There's plenty of tricks out there in the literature which are well understood in theory, but not practical to apply in everyday compiler use.
The same is true for many other branches of engineering. You can build better mechanisms, but they require exponentially more difficulty to implement for diminishing gains and higher and higher cost, eventually running into practical limits due to things like available materials. There are only so many elements in the periodic table, and while some of them have nice properties, all of the numbers attached to them are finite.
When it comes to engineering general intelligence, why should we expect the picture to be any different?
posted by automatronic at 6:18 AM on April 3, 2021 [18 favorites]
Then I got into it some more, learned how to use the simulation tools, designed some antennas, built them, tested them, tuned them and so on.
And found that it's just regular old engineering. Yes, you can get more efficiency, directional gain, or narrower or wider bandwidth, or more bands covered or whatever - but each comes with tradeoffs and limits. It will get bigger, or more expensive, or require finer tolerances and be harder to make, or harder to tune accurately and keep tuned. And ultimately there are physical limits to what can be achieved in practice.
Like every other sort of engineering, antenna design is about finding the best balance across all those tradeoffs for the task at hand. You can't make an antenna that's good at everything.
Or rather, if you can, it's not a single antenna. It's a collection of different antennas that you use some intelligence to select between and use one or more of, depending on the task at hand.
Which is rather what a modern compiler looks like. It's a big collection of individual tools. Each one applies a particular transformation to the code - looking for and applying some particular optimisation, or making some transformation from one form to another. Those tools are collected together, by humans, into a particular sequence of passes that has been tested and shown to provide good results for a wide range of code.
But again, there are tradeoffs. If you want the compiler to generate faster code, you need to enable more optimisation features and the compilation will take longer to run. Sometimes exponentially so for larger inputs. There's plenty of tricks out there in the literature which are well understood in theory, but not practical to apply in everyday compiler use.
The same is true for many other branches of engineering. You can build better mechanisms, but they require exponentially more difficulty to implement for diminishing gains and higher and higher cost, eventually running into practical limits due to things like available materials. There are only so many elements in the periodic table, and while some of them have nice properties, all of the numbers attached to them are finite.
When it comes to engineering general intelligence, why should we expect the picture to be any different?
posted by automatronic at 6:18 AM on April 3, 2021 [18 favorites]
Immediate, un-nuanced reaction: "Why Computers Won't Make Themselves Smarter" is one of the stupidest pieces of writing I've ever read.
posted by Jonathan Livengood at 6:24 AM on April 3, 2021 [1 favorite]
posted by Jonathan Livengood at 6:24 AM on April 3, 2021 [1 favorite]
The way to address foundational questions about AI is to talk about complexity theory and models of computation.
Computation modelling and complexity theory have nothing at all to say about moral agency, lived experience or social consequence, which are the meat of the interview. The implementation details you're describing are kind of irrelevant, like saying that the way to address foundational questions about literature is to talk about the operation of a laser printer.
posted by mhoye at 6:25 AM on April 3, 2021 [10 favorites]
Computation modelling and complexity theory have nothing at all to say about moral agency, lived experience or social consequence, which are the meat of the interview. The implementation details you're describing are kind of irrelevant, like saying that the way to address foundational questions about literature is to talk about the operation of a laser printer.
posted by mhoye at 6:25 AM on April 3, 2021 [10 favorites]
When it comes to engineering general intelligence, why should we expect the picture to be any different?
Mainly because the spec seems to have been constructed wholly inside Management and Marketing without reference to any actionable acceptance test.
posted by flabdablet at 6:37 AM on April 3, 2021 [3 favorites]
Mainly because the spec seems to have been constructed wholly inside Management and Marketing without reference to any actionable acceptance test.
posted by flabdablet at 6:37 AM on April 3, 2021 [3 favorites]
I too used to think of antenna design as black magic too.
Then I got into it some more, learned how to use the simulation tools, designed some antennas, built them, tested them, tuned them and so on.
The FTC mandate repack that resulted in broadcasters switching to backup antennae on a different building that had several high rises blocking my reception was when I realized that antenna reception was not a black art so much as a crossroads deal with a real-estate devel.
posted by srboisvert at 7:30 AM on April 3, 2021 [3 favorites]
Then I got into it some more, learned how to use the simulation tools, designed some antennas, built them, tested them, tuned them and so on.
The FTC mandate repack that resulted in broadcasters switching to backup antennae on a different building that had several high rises blocking my reception was when I realized that antenna reception was not a black art so much as a crossroads deal with a real-estate devel.
posted by srboisvert at 7:30 AM on April 3, 2021 [3 favorites]
It's funny that I even find myself in a position of defending AI, because I never cared for computers as a young student.
With properly exercised mirror neurons, computers are just another form of pettable crouton, and I, for one, believe in your potential to care for them!
posted by otherchaz at 7:35 AM on April 3, 2021 [9 favorites]
With properly exercised mirror neurons, computers are just another form of pettable crouton, and I, for one, believe in your potential to care for them!
posted by otherchaz at 7:35 AM on April 3, 2021 [9 favorites]
Generally speaking, declarations that this or that technological thing are impossible tend to end up looking stupid even a couple of decades down the road, so on that level these didn't impress me much. Of course, just because something is technologically possible doesn't mean that it's something we're going to actually do society wide at any point. Flying cars are technologically possible, but they're not widespread and unlikely to become so. Whether we're going to need or want AI that's smarter than a certain level enough for people to pay for it is debatable.
posted by AdamCSnider at 7:55 AM on April 3, 2021 [1 favorite]
posted by AdamCSnider at 7:55 AM on April 3, 2021 [1 favorite]
Weird how so many think that artificial intelligence is attainable when it seems that most scientists believe that intelligence in other life-forms is an unprovable hypothesis.
posted by No Robots at 8:05 AM on April 3, 2021 [4 favorites]
posted by No Robots at 8:05 AM on April 3, 2021 [4 favorites]
Weird how so many think that artificial intelligence is attainable when it seems that most scientists believe that intelligence in other life-forms is an unprovable hypothesis.
Do they? I thought the big argument was over consciousness/self-awareness - intelligence in all kinds of animals (mainly to do with problem solving or social interactions) seems to be widely accepted. Or are we talking about trees or something?
posted by AdamCSnider at 8:15 AM on April 3, 2021 [3 favorites]
Do they? I thought the big argument was over consciousness/self-awareness - intelligence in all kinds of animals (mainly to do with problem solving or social interactions) seems to be widely accepted. Or are we talking about trees or something?
posted by AdamCSnider at 8:15 AM on April 3, 2021 [3 favorites]
I don't know, I work in AI and theoretical neuroscience, and on first pass I find both of these articles pretty sound. I'm impressed by Ted Chiang's clarity of thought on these issues, and it makes me want to (finally) read his work.
posted by Alex404 at 8:19 AM on April 3, 2021 [19 favorites]
posted by Alex404 at 8:19 AM on April 3, 2021 [19 favorites]
I thought the big argument was over consciousness/self-awareness - intelligence in all kinds of animals (mainly to do with problem solving or social interactions) seems to be widely accepted.
The hard distinction between consciousness/self-awareness and intelligence is a dodge. Conscious, self-awareness and intelligence are synonyms. At most they indicate quantitative, not qualitative, distinctions.
posted by No Robots at 8:29 AM on April 3, 2021
The hard distinction between consciousness/self-awareness and intelligence is a dodge. Conscious, self-awareness and intelligence are synonyms. At most they indicate quantitative, not qualitative, distinctions.
posted by No Robots at 8:29 AM on April 3, 2021
Just intellectually the author is using, frankly outdated examples from computer science. The way to address foundational questions about AI is to talk about complexity theory and models of computation.
I mean, the details of lambda calculus and list processing based programming languages like Lisp were also a significant part of what I enjoyed in my AI courses and summer research job 20-some years ago - I continued on into math, not computer science, and my own interest definitely remains in that interface area between the two disciplines. But to claim that complexity theory and theory of computation are a major component of the field of AI within computer science (let alone software engineering more broadly) nowadays seems a bit of a stretch. Statistics was already starting to become an equally important mathematical foundation for the field back in my day, with the push for statistical verification of complex software systems. Now, with machine learning being a (probably /the/) major theme in AI that is much more the case. As well, back in my day, autonomous agents were already becoming the thing, replacing modes of thinking associated with logical and functional programming with object oriented modes of thinking in AI as elsewhere in both computer science and software engineering. One would be hard pressed to argue that change hasn't also become super widespread or near-universal in the intervening 20+ years.
posted by eviemath at 8:36 AM on April 3, 2021 [6 favorites]
I mean, the details of lambda calculus and list processing based programming languages like Lisp were also a significant part of what I enjoyed in my AI courses and summer research job 20-some years ago - I continued on into math, not computer science, and my own interest definitely remains in that interface area between the two disciplines. But to claim that complexity theory and theory of computation are a major component of the field of AI within computer science (let alone software engineering more broadly) nowadays seems a bit of a stretch. Statistics was already starting to become an equally important mathematical foundation for the field back in my day, with the push for statistical verification of complex software systems. Now, with machine learning being a (probably /the/) major theme in AI that is much more the case. As well, back in my day, autonomous agents were already becoming the thing, replacing modes of thinking associated with logical and functional programming with object oriented modes of thinking in AI as elsewhere in both computer science and software engineering. One would be hard pressed to argue that change hasn't also become super widespread or near-universal in the intervening 20+ years.
posted by eviemath at 8:36 AM on April 3, 2021 [6 favorites]
I feel like Chiang would totally understand my annoyance at the common misinterpretation of Frankenstein as being anti-science/technology when what it is is anti-alchemy/magic. But also that he wouldn't care one way or another about my peeve that the movie version of Contact stupidly ended up with the exact opposite moral/main point from the book (the book's moral being that even highly advanced science/technology is distinguishable from religion/magic due to the presence of evidence and empirically testable hypotheses, however challenging the evidence may be to find or the experiments may be to design, conduct, or analyze; while the movie's point seemed to be that there was no difference between the two).
posted by eviemath at 8:46 AM on April 3, 2021 [3 favorites]
posted by eviemath at 8:46 AM on April 3, 2021 [3 favorites]
The part where he asks, "why don't we just make dumb people smarter" (roughly) bugged me and I thought about starting to stomp around the house, cursing his name because it's a dumb, "Flowers for Algernon" notion. Persevering, there were other good points he made.
Coiincidentally, I heard this episode of Lawfare podcast days later, "The Myth of Artificial Intelligence", a discussion with Erik Larson which I found much more compelling and satisfying.
posted by From Bklyn at 8:47 AM on April 3, 2021 [1 favorite]
Coiincidentally, I heard this episode of Lawfare podcast days later, "The Myth of Artificial Intelligence", a discussion with Erik Larson which I found much more compelling and satisfying.
posted by From Bklyn at 8:47 AM on April 3, 2021 [1 favorite]
Weird how so many think that artificial intelligence is attainable when it seems that most scientists believe that intelligence in other life-forms is an unprovable hypothesis.
Where did you get this idea? Plenty of scientists work on non-human intelligence.
posted by en forme de poire at 8:48 AM on April 3, 2021 [1 favorite]
Where did you get this idea? Plenty of scientists work on non-human intelligence.
posted by en forme de poire at 8:48 AM on April 3, 2021 [1 favorite]
Plenty of scientists work on non-human intelligence.
Well, there's all the navel-gazing in this thread.
And there's this from a leading scientist:
posted by No Robots at 8:59 AM on April 3, 2021
Well, there's all the navel-gazing in this thread.
And there's this from a leading scientist:
Question: Is there a certain brain capacity necessary for the development of consciousness?I certainly hope that work similar to that in the link you provided will come to prevail.
Richard Dawkins: Oh, nobody knows, because we don't know which animals are conscious. We don't actually, technically, even know that any other human being is conscious. We just each of us know that we ourselves are conscious. We infer on pretty good grounds that other people are conscious, and it's the same sort of grounds that lead us to infer that probably chimpanzees are conscious and probably dogs are conscious. But when we come to something like earthworms and snails, it's anybody's guess.
posted by No Robots at 8:59 AM on April 3, 2021
I'm not 100% sure that Richard Dawkins is a leading scientist as much as a leading philosopher of science? Which is not to say that his analysis should be discounted - sometimes meta-observers have a better vantage point for answering certain questions. But he's doing a fair amount of bean-plating in that quote. Certainly a number of other species have exhibited behaviors associated with some level of a spectrum of consciousness, such as being able to recognize self in a mirror, or planning for potential futures. (I'd have to go searching, but there was an interesting article that I'm pretty sure AI saw here on Metafilter the other year, about octopus tool use and the distinction between very direct tool use such as "if I poke this stick in this hole I can get those tasty bugs that I know for certain are in the hole out and eat them" versus indirect tool use indicating a likely ability to envision potential (but not currently existent or entirely predictable/instinctual) future scenarios.)
posted by eviemath at 9:33 AM on April 3, 2021 [5 favorites]
posted by eviemath at 9:33 AM on April 3, 2021 [5 favorites]
it makes me want to (finally) read his work.
Oh you really should. "Clarity" is one of the things i most associate with his stories, they have an incredible spareness and simplicity to them, and are fantastically well thought through.
In his other life he's a technical writer, and, I expect, a very good one.
posted by Jon Mitchell at 10:21 AM on April 3, 2021 [1 favorite]
Oh you really should. "Clarity" is one of the things i most associate with his stories, they have an incredible spareness and simplicity to them, and are fantastically well thought through.
In his other life he's a technical writer, and, I expect, a very good one.
posted by Jon Mitchell at 10:21 AM on April 3, 2021 [1 favorite]
I find Chiang's arguments about AI unconvincing. Maybe it's because, like polymodus, I find his examples outdated. The example he doesn't discuss is the major advance in deep learning game players like AlphaZero. There two phenomenal things AlphaZero does.
First, it teaches itself to play the game. The basic neural network structure has almost no information about the game and is easily adapted to many, many different types of games. It does have to be retrained for each new game though.
Second, it trains itself incredibly quickly and with a long tail of improvement. In that linked graph there's a clear knee in the curve around 3 days of training; it's about 80% as good as any AlphaGo game has gotten. But it keeps learning well into 40+ days. It sort of looks like it's levelling off but in fact it's not clear there's a real asymptote.
If you look at a similar graph for Leela Zero, a project similar to AlphaGo, the continued room for improvement is astonishing. Part of that is that Leela Zero kept expanding its underlying neural network architecture; it'd train to diminishing returns at one size network, then add another stack of layers and keep retraining. It's not clear whether there's any limit to its learning about how to play Go until it reaches the absolute maximum of always playing the Divine Move.
The limitation of these things is they only play games. Extending them to complicated real world problems is hard. Training in anything like common sense knowledge or adaptability to random inputs and circumstances is well beyond the state of the art. But here, here we have a real AI that's really capable of bootstrapping itself seemingly without limit. It's just only able to do a toy problem; that's the real limitation.
There's also some fascinating research Chiang doesn't discuss on using learning techniques to alter the parameters and learning environment of the neural networks themselves; learning how to learn faster. That too produces significant bootstrapping capability, albeit does not solve the common sense and real world problems either.
posted by Nelson at 10:49 AM on April 3, 2021 [10 favorites]
First, it teaches itself to play the game. The basic neural network structure has almost no information about the game and is easily adapted to many, many different types of games. It does have to be retrained for each new game though.
Second, it trains itself incredibly quickly and with a long tail of improvement. In that linked graph there's a clear knee in the curve around 3 days of training; it's about 80% as good as any AlphaGo game has gotten. But it keeps learning well into 40+ days. It sort of looks like it's levelling off but in fact it's not clear there's a real asymptote.
If you look at a similar graph for Leela Zero, a project similar to AlphaGo, the continued room for improvement is astonishing. Part of that is that Leela Zero kept expanding its underlying neural network architecture; it'd train to diminishing returns at one size network, then add another stack of layers and keep retraining. It's not clear whether there's any limit to its learning about how to play Go until it reaches the absolute maximum of always playing the Divine Move.
The limitation of these things is they only play games. Extending them to complicated real world problems is hard. Training in anything like common sense knowledge or adaptability to random inputs and circumstances is well beyond the state of the art. But here, here we have a real AI that's really capable of bootstrapping itself seemingly without limit. It's just only able to do a toy problem; that's the real limitation.
There's also some fascinating research Chiang doesn't discuss on using learning techniques to alter the parameters and learning environment of the neural networks themselves; learning how to learn faster. That too produces significant bootstrapping capability, albeit does not solve the common sense and real world problems either.
posted by Nelson at 10:49 AM on April 3, 2021 [10 favorites]
... a leading philosopher of science?
Dear God, no. Dawkins has had some good ideas and done a lot of good work in popularizing and communicating ideas in evolutionary biology. But he is not a philosopher of science except possibly on an amateur basis. If you look at leading journals in history and philosophy of science, e.g. the British Journal for the Philosophy of Science, Studies in History and Philosophy of Science, Philosophy of Science, Synthese, Erkenntnis, Isis, the British Journal for the History of Science, and the like, you will not find articles by Dawkins. You'll find some citations of Dawkins in relation to discussions of the units of selection, causal mechanisms of evolution, signaling theory, and the like. I don't work in those specific parts of HPS, so I'm not confident about this, but my impression is that even in those places where Dawkins is cited, it's more pro forma than it is deep engagement with a "leading scholar" in the field.
posted by Jonathan Livengood at 10:54 AM on April 3, 2021 [17 favorites]
Dear God, no. Dawkins has had some good ideas and done a lot of good work in popularizing and communicating ideas in evolutionary biology. But he is not a philosopher of science except possibly on an amateur basis. If you look at leading journals in history and philosophy of science, e.g. the British Journal for the Philosophy of Science, Studies in History and Philosophy of Science, Philosophy of Science, Synthese, Erkenntnis, Isis, the British Journal for the History of Science, and the like, you will not find articles by Dawkins. You'll find some citations of Dawkins in relation to discussions of the units of selection, causal mechanisms of evolution, signaling theory, and the like. I don't work in those specific parts of HPS, so I'm not confident about this, but my impression is that even in those places where Dawkins is cited, it's more pro forma than it is deep engagement with a "leading scholar" in the field.
posted by Jonathan Livengood at 10:54 AM on April 3, 2021 [17 favorites]
I'm certainly willing to believe that Dawkins is neither a leading scientist nor a leading philosopher of science.
posted by eviemath at 11:19 AM on April 3, 2021 [11 favorites]
posted by eviemath at 11:19 AM on April 3, 2021 [11 favorites]
The limitation of these things is they only play games. Extending them to complicated real world problems is hard. Training in anything like common sense knowledge or adaptability to random inputs and circumstances is well beyond the state of the art.
This seems to be a large component of Chiang's thesis, yes.
posted by eviemath at 11:20 AM on April 3, 2021 [7 favorites]
This seems to be a large component of Chiang's thesis, yes.
posted by eviemath at 11:20 AM on April 3, 2021 [7 favorites]
I want to write something more elaborate about this eventually, but I think I can describe what I take to be the core failure of the main analogy in the piece right now.
Chiang complains that smart humans don't know how to make smarter humans, and so, if we build a machine that is as smart as us or smarter, it won't be able to make smarter versions of itself either. But there is a confusion here between (1) making an existing entity of type X smarter, and (2) making a smarter entity of type X.
The first thing is actually done by us when we educate people. Education makes existing people smarter. If we're measuring by IQ -- which is probably a bad idea, but it's Chiang's fault for bringing it into the conversation -- then we have every reason to think that intelligence can be and has been seriously improved with improvements to environment, including education. As measured by IQ, human intelligence has increased by about 30 points worldwide over the last hundred years. And there is evidence that intra-generationally, IQ increases with education rather dramatically. So, Chiang's claim that we can't make people smarter as measured by IQ (even covering a seemingly dramatic 70 to 100 point gap) is just empirically false. But in any event, it's also the case that we now have machines that can improve their intelligence by way of education. And in some cases, such as AlphaGo and AlphaZero, by self education. So, we can and already have made machines that can make themselves smarter in this first sense.
But the first sense isn't the one that matters. The sense that matters is the second one. And I submit that if you stop to really consider what is involved in the second sense, you'll find that it's obvious that Chiang is wrong. If -- and this is really where all the work has to be done -- we manage to build a machine that is fully as intelligent as we are, then by construction, the machine is able to build machines that are as intelligent as it is. But now, we have lots of reason to think that we are able to build better machines than the ones we've previously built. And in particular, we are able to steadily increase the intelligence of machines that we build. Each new generation of intelligent machines that we construct is better than previous ones. This doesn't have to be large jumps. Small, even insensibly small, steps are enough. If we can increase the capacity of machines that we build, then a machine that is as intelligent as us can also increase the capacity of machines that it builds. The result is that once we build a machine that is as intelligent as us, such a machine will be able to build more intelligent machines. The only reason to doubt this, as far as I can tell, has to come from a theoretical argument that human intelligence is at or near a computational maximum. And I see no good reasons to think that that is the case. (This is why I think polymodus is right to say that this ultimately has to be discussed in terms of logic and foundations of computation. Not because the field of AI research is working in a "logic" model, it's definitely not! But because if you want to give arguments to the conclusion that there's some hard upper limit on machine intelligence, you're going to have to talk about the abstract limits of computation.)
posted by Jonathan Livengood at 11:25 AM on April 3, 2021 [8 favorites]
Chiang complains that smart humans don't know how to make smarter humans, and so, if we build a machine that is as smart as us or smarter, it won't be able to make smarter versions of itself either. But there is a confusion here between (1) making an existing entity of type X smarter, and (2) making a smarter entity of type X.
The first thing is actually done by us when we educate people. Education makes existing people smarter. If we're measuring by IQ -- which is probably a bad idea, but it's Chiang's fault for bringing it into the conversation -- then we have every reason to think that intelligence can be and has been seriously improved with improvements to environment, including education. As measured by IQ, human intelligence has increased by about 30 points worldwide over the last hundred years. And there is evidence that intra-generationally, IQ increases with education rather dramatically. So, Chiang's claim that we can't make people smarter as measured by IQ (even covering a seemingly dramatic 70 to 100 point gap) is just empirically false. But in any event, it's also the case that we now have machines that can improve their intelligence by way of education. And in some cases, such as AlphaGo and AlphaZero, by self education. So, we can and already have made machines that can make themselves smarter in this first sense.
But the first sense isn't the one that matters. The sense that matters is the second one. And I submit that if you stop to really consider what is involved in the second sense, you'll find that it's obvious that Chiang is wrong. If -- and this is really where all the work has to be done -- we manage to build a machine that is fully as intelligent as we are, then by construction, the machine is able to build machines that are as intelligent as it is. But now, we have lots of reason to think that we are able to build better machines than the ones we've previously built. And in particular, we are able to steadily increase the intelligence of machines that we build. Each new generation of intelligent machines that we construct is better than previous ones. This doesn't have to be large jumps. Small, even insensibly small, steps are enough. If we can increase the capacity of machines that we build, then a machine that is as intelligent as us can also increase the capacity of machines that it builds. The result is that once we build a machine that is as intelligent as us, such a machine will be able to build more intelligent machines. The only reason to doubt this, as far as I can tell, has to come from a theoretical argument that human intelligence is at or near a computational maximum. And I see no good reasons to think that that is the case. (This is why I think polymodus is right to say that this ultimately has to be discussed in terms of logic and foundations of computation. Not because the field of AI research is working in a "logic" model, it's definitely not! But because if you want to give arguments to the conclusion that there's some hard upper limit on machine intelligence, you're going to have to talk about the abstract limits of computation.)
posted by Jonathan Livengood at 11:25 AM on April 3, 2021 [8 favorites]
While I instinctively tend to agree with Chiang, I don't think his arguments are very good either philosophically or technically. To me it's more of an abstract question on the limits of bootstrapping.
The interview has many great ideas in it though, especially that the real problem with A.I. is capitalism.
posted by blue shadows at 11:43 AM on April 3, 2021 [3 favorites]
The interview has many great ideas in it though, especially that the real problem with A.I. is capitalism.
posted by blue shadows at 11:43 AM on April 3, 2021 [3 favorites]
(That should have been technologically, although technically works too.)
posted by blue shadows at 11:52 AM on April 3, 2021
posted by blue shadows at 11:52 AM on April 3, 2021
And there's this from a leading scientist
I definitely would not take Dawkins’ views as representative of scientists in general. Even referring to him as a “leading scientist” seems strange to me; most of his current fame comes from his activism about atheism. Beyond that, asking what Dawkins thinks about non-human intelligence is kind of like getting an economist’s take on Covid epidemiology. It would be even weirder to then use their response as some kind of straw man to stand in for what “most scientists” believe.
posted by en forme de poire at 12:09 PM on April 3, 2021 [9 favorites]
I definitely would not take Dawkins’ views as representative of scientists in general. Even referring to him as a “leading scientist” seems strange to me; most of his current fame comes from his activism about atheism. Beyond that, asking what Dawkins thinks about non-human intelligence is kind of like getting an economist’s take on Covid epidemiology. It would be even weirder to then use their response as some kind of straw man to stand in for what “most scientists” believe.
posted by en forme de poire at 12:09 PM on April 3, 2021 [9 favorites]
I definitely would not take Dawkins’ views as representative of scientists in general.
He represents the physicalist mentality (heh, physicalist mentality) that predominates in science.
posted by No Robots at 12:20 PM on April 3, 2021
He represents the physicalist mentality (heh, physicalist mentality) that predominates in science.
posted by No Robots at 12:20 PM on April 3, 2021
Chiang's compiler analogy is definitely a stronger argument than the digression into IQ, which I agree has issues on a couple different fronts. I think that one of the main points that he makes, (aligned with your acknowledgement, Jonathan Livengood, in "If -- and this is really where all the work has to be done") is still salient, however, in that the definitions are carrying entirely too much unexamined weight in concerns or hypothesizing about the so-called singularity.
Here I agree with polymodus to a certain extent (despite my previous comment) in that I think some more theoretical or foundational mathematics would provide an ultimately more convincing (though undoubtedly less clear to the average lay reader) argument. I think that automatronic described the trade-offs in compiler power (potential "intelligence") versus generality more clearly than Chiang, but this is where the math would be relevant: uncertainty principles - from the intersection of Fourier/harmonic analysis and information theory - give the more formal background to support this argument. Humans are a particular balance of generalists in our reasoning ability together with having more specific habits or instinctual* reactions to stimuli. There are good reasons, both empirical and theoretical, to believe that such a tradeoff always has to be made.
(* Where, by "instinctual", I think these are learned instinctual reactions rather than biologically programmed ones for the most part, in the case of humans. But I mean to include both possibilities, in addition to literally programmed in responses in the case of AI programs - I'm using instinctual reaction to mean any reaction that we aren't reasoning about (consciously or subconsciously, if subconscious reasoning is a thing). I'd use autonomous as in the autonomous nervous system, but that has a different usage in AI terms.)
posted by eviemath at 12:38 PM on April 3, 2021 [1 favorite]
Here I agree with polymodus to a certain extent (despite my previous comment) in that I think some more theoretical or foundational mathematics would provide an ultimately more convincing (though undoubtedly less clear to the average lay reader) argument. I think that automatronic described the trade-offs in compiler power (potential "intelligence") versus generality more clearly than Chiang, but this is where the math would be relevant: uncertainty principles - from the intersection of Fourier/harmonic analysis and information theory - give the more formal background to support this argument. Humans are a particular balance of generalists in our reasoning ability together with having more specific habits or instinctual* reactions to stimuli. There are good reasons, both empirical and theoretical, to believe that such a tradeoff always has to be made.
(* Where, by "instinctual", I think these are learned instinctual reactions rather than biologically programmed ones for the most part, in the case of humans. But I mean to include both possibilities, in addition to literally programmed in responses in the case of AI programs - I'm using instinctual reaction to mean any reaction that we aren't reasoning about (consciously or subconsciously, if subconscious reasoning is a thing). I'd use autonomous as in the autonomous nervous system, but that has a different usage in AI terms.)
posted by eviemath at 12:38 PM on April 3, 2021 [1 favorite]
If -- and this is really where all the work has to be done -- we manage to build a machine that is fully as intelligent as we are, then by construction, the machine is able to build machines that are as intelligent as it is.
...assuming only that it's stupid enough to want to.
posted by flabdablet at 12:42 PM on April 3, 2021 [1 favorite]
...assuming only that it's stupid enough to want to.
posted by flabdablet at 12:42 PM on April 3, 2021 [1 favorite]
I liked the NYer piece, am totally unqualified to evaluate it, but tbh I think the interview's got some more interesting meat in there. Specifically this (sorry it's so much text):
EZRA KLEIN: Let me flip this now. We’re spending billions to invent artificial intelligence. At what point is a computer program responsible for its own actions?
TED CHIANG: Well, in terms of at what point does that happen, it’s unclear, but it’s a very long ways from us right now. With regard to the question of, will we create machines that are moral agents, I would say that we can think about that in three different questions. One is, can we do so? Second is, will we do so? And the third one is, should we do so?
I think it is entirely possible for us to build machines that are moral agents. Because I think there’s a sense in which human beings are very complex machines and we are moral agents, which means that there are no physical laws preventing a machine from being a moral agent. And so there’s no obstacle that, in principle, would prevent us from building something like that, although it might take us a very, very long time to get there.
As for the question of, will we do so, if you had asked me, like, 10 or 15 years ago, I would have said, we probably won’t do it, simply because, to me, it seems like it’s way more trouble than it’s worth. In terms of expense, it would be on the order of magnitude of the Apollo program. And it is not at all clear to me that there’s any good reason for undertaking such a thing. However, if you ask me now, I would say like, well, OK, we clearly have obscenely wealthy people who can throw around huge sums of money at whatever they want basically on a whim. So maybe one of them will wind up funding a program to create machines that are conscious and that are moral agents.
However, I should also note that I don’t believe that any of the current big A.I. research programs are on the right track to create a conscious machine. I don’t think that’s what any of them are trying to do. So then as for the third question of, should we do so, should we make machines that are conscious and that are moral agents, to that, my answer is, no, we should not. Because long before we get to the point where a machine is a moral agent, we will have machines that are capable of suffering.
Suffering precedes moral agency in sort of the developmental ladder. Dogs are not moral agents, but they are capable of experiencing suffering. Babies are not moral agents yet, but they have the clear potential to become so. And they are definitely capable of experiencing suffering. And the closer that an entity gets to being a moral agent, the more that it’s suffering, it’s deserving of consideration, the more we should try and avoid inflicting suffering on it. So in the process of developing machines that are conscious and moral agents, we will be inevitably creating billions of entities that are capable of suffering. And we will inevitably inflict suffering on them. And that seems to me clearly a bad idea.
EZRA KLEIN: But wouldn’t they also be capable of pleasure? I mean, that seems to me to raise an almost inversion of the classic utilitarian thought experiment. If we can create these billions of machines that live basically happy lives that don’t hurt anybody and you can copy them for almost no marginal dollar, isn’t it almost a moral imperative to bring them into existence so they can lead these happy machine lives?
TED CHIANG: I think that it will be much easier to inflict suffering on them than to give them happy fulfilled lives. And given that they will start out as something that resembles ordinary software, something that is nothing like a living being, we are going to treat them like crap. The way that we treat software right now, if, at some point, software were to gain some vague glimmer of sentience, of the ability to perceive, we would be inflicting uncountable amounts of suffering on it before anyone paid any attention to them.
Because it’s hard enough to give legal protections to human beings who are absolutely moral agents. We have relatively few legal protections for animals who, while they are not moral agents, are capable of suffering. And so animals experience vast amounts of suffering in the modern world. And animals, we know that they suffer. There are many animals that we love, that we really, really love. Yet, there’s vast animal suffering. So there is no software that we love. So the way that we will wind up treating software, again, assuming that software ever becomes conscious, they will inevitably fall lower on the ladder of consideration. So we will treat them worse than we treat animals. And we treat animals pretty badly.
posted by TheProfessor at 12:46 PM on April 3, 2021 [16 favorites]
EZRA KLEIN: Let me flip this now. We’re spending billions to invent artificial intelligence. At what point is a computer program responsible for its own actions?
TED CHIANG: Well, in terms of at what point does that happen, it’s unclear, but it’s a very long ways from us right now. With regard to the question of, will we create machines that are moral agents, I would say that we can think about that in three different questions. One is, can we do so? Second is, will we do so? And the third one is, should we do so?
I think it is entirely possible for us to build machines that are moral agents. Because I think there’s a sense in which human beings are very complex machines and we are moral agents, which means that there are no physical laws preventing a machine from being a moral agent. And so there’s no obstacle that, in principle, would prevent us from building something like that, although it might take us a very, very long time to get there.
As for the question of, will we do so, if you had asked me, like, 10 or 15 years ago, I would have said, we probably won’t do it, simply because, to me, it seems like it’s way more trouble than it’s worth. In terms of expense, it would be on the order of magnitude of the Apollo program. And it is not at all clear to me that there’s any good reason for undertaking such a thing. However, if you ask me now, I would say like, well, OK, we clearly have obscenely wealthy people who can throw around huge sums of money at whatever they want basically on a whim. So maybe one of them will wind up funding a program to create machines that are conscious and that are moral agents.
However, I should also note that I don’t believe that any of the current big A.I. research programs are on the right track to create a conscious machine. I don’t think that’s what any of them are trying to do. So then as for the third question of, should we do so, should we make machines that are conscious and that are moral agents, to that, my answer is, no, we should not. Because long before we get to the point where a machine is a moral agent, we will have machines that are capable of suffering.
Suffering precedes moral agency in sort of the developmental ladder. Dogs are not moral agents, but they are capable of experiencing suffering. Babies are not moral agents yet, but they have the clear potential to become so. And they are definitely capable of experiencing suffering. And the closer that an entity gets to being a moral agent, the more that it’s suffering, it’s deserving of consideration, the more we should try and avoid inflicting suffering on it. So in the process of developing machines that are conscious and moral agents, we will be inevitably creating billions of entities that are capable of suffering. And we will inevitably inflict suffering on them. And that seems to me clearly a bad idea.
EZRA KLEIN: But wouldn’t they also be capable of pleasure? I mean, that seems to me to raise an almost inversion of the classic utilitarian thought experiment. If we can create these billions of machines that live basically happy lives that don’t hurt anybody and you can copy them for almost no marginal dollar, isn’t it almost a moral imperative to bring them into existence so they can lead these happy machine lives?
TED CHIANG: I think that it will be much easier to inflict suffering on them than to give them happy fulfilled lives. And given that they will start out as something that resembles ordinary software, something that is nothing like a living being, we are going to treat them like crap. The way that we treat software right now, if, at some point, software were to gain some vague glimmer of sentience, of the ability to perceive, we would be inflicting uncountable amounts of suffering on it before anyone paid any attention to them.
Because it’s hard enough to give legal protections to human beings who are absolutely moral agents. We have relatively few legal protections for animals who, while they are not moral agents, are capable of suffering. And so animals experience vast amounts of suffering in the modern world. And animals, we know that they suffer. There are many animals that we love, that we really, really love. Yet, there’s vast animal suffering. So there is no software that we love. So the way that we will wind up treating software, again, assuming that software ever becomes conscious, they will inevitably fall lower on the ladder of consideration. So we will treat them worse than we treat animals. And we treat animals pretty badly.
posted by TheProfessor at 12:46 PM on April 3, 2021 [16 favorites]
I'm really enjoying this discussion. From my point of view though, computational complexity and speculating about the technical limits of bootstrapping are somewhat beside the point. Whether we're talking about classical AI, or modern machine learning/deep learning, pretty much all forms of AI as practiced are forms of mathematical optimization (i.e. maximizing rewards).
The limit is that we need to know what the objectives of our systems are. This might seem obvious, but it leads to two important points: firstly, that AlphaGo etc. are limited to games isn't some of the story, it's most of the story. We don't know how to build AI systems that can generalize outside of well defined problems. Secondly, computational complexity will only tell us how fast we can achieve these well-defined goals, and again, not how we can generalize. Generalization isn't just a technical challenge - it's rather revealing itself to be pretty much the whole game of general intelligence. And from this scientist's little perspective, we're still very, very far from knowing what addressing it will take.
In any case, I think Chiang's central point is a little beside this, and arguably more interesting. Firstly, I don't think his point is that education does help, but that education is a social system. A smart individual cannot just make another individual smarter. Not even a good educator. It requires a whole society. And it's within that context that we actually see a sort of bootstrapping to new problem domains. And so could we replicate this in silico? Maybe, but why would we want to?
It's not that human's are maximally intelligent, but in terms of energy consumption, we're still far more efficient than any machines we've built, and we're already maxing out the resources of our planet. In terms of a system that is able to advance its own fundamental limits, civilization is what we've got, and AI as it stands is only a modest tool within that project.
posted by Alex404 at 12:55 PM on April 3, 2021 [15 favorites]
The limit is that we need to know what the objectives of our systems are. This might seem obvious, but it leads to two important points: firstly, that AlphaGo etc. are limited to games isn't some of the story, it's most of the story. We don't know how to build AI systems that can generalize outside of well defined problems. Secondly, computational complexity will only tell us how fast we can achieve these well-defined goals, and again, not how we can generalize. Generalization isn't just a technical challenge - it's rather revealing itself to be pretty much the whole game of general intelligence. And from this scientist's little perspective, we're still very, very far from knowing what addressing it will take.
In any case, I think Chiang's central point is a little beside this, and arguably more interesting. Firstly, I don't think his point is that education does help, but that education is a social system. A smart individual cannot just make another individual smarter. Not even a good educator. It requires a whole society. And it's within that context that we actually see a sort of bootstrapping to new problem domains. And so could we replicate this in silico? Maybe, but why would we want to?
It's not that human's are maximally intelligent, but in terms of energy consumption, we're still far more efficient than any machines we've built, and we're already maxing out the resources of our planet. In terms of a system that is able to advance its own fundamental limits, civilization is what we've got, and AI as it stands is only a modest tool within that project.
posted by Alex404 at 12:55 PM on April 3, 2021 [15 favorites]
I think Chiang's best point is drawing a distinction between the capabilities of an individual human being and the capabilities of human civilization. Almost everything that makes me more powerful than an ape is the result of human civilization, not any one human (certainly not me in particular). Maybe a computer can't design another computer, but neither can I. I'm not sure any individual human can design a computer by themselves.
But he doesn't really push that far enough to pulling apart what we mean by "intelligence."
Can computers do some things better than individual humans? Yes.
Can computers do some things better than human civilization? Unclear. Computers are part of human civilization.
Can computer programs modify themselves to get iteratively better at some things? Yes.
So the questions are which things will computers be able to do better than individual humans, and which of them will self-modification make them unpredictably better at than individual humans. And then when would it make sense to think of a computer as doing something better than human civilization itself, rather than simply adding to the capability of human civilization?
It wouldn't make sense to worry that my hands are going to get so good at typing that they'll be better at typing than I am.
posted by straight at 1:00 PM on April 3, 2021 [2 favorites]
But he doesn't really push that far enough to pulling apart what we mean by "intelligence."
Can computers do some things better than individual humans? Yes.
Can computers do some things better than human civilization? Unclear. Computers are part of human civilization.
Can computer programs modify themselves to get iteratively better at some things? Yes.
So the questions are which things will computers be able to do better than individual humans, and which of them will self-modification make them unpredictably better at than individual humans. And then when would it make sense to think of a computer as doing something better than human civilization itself, rather than simply adding to the capability of human civilization?
It wouldn't make sense to worry that my hands are going to get so good at typing that they'll be better at typing than I am.
posted by straight at 1:00 PM on April 3, 2021 [2 favorites]
I have to admit that I was amused by the part of the interview where Klein asked, once machines can experience happiness, are we not morally obligated to bring more of them into existence just so they can sit around feeling happy. Roko's Onanist!
posted by mhoye at 1:03 PM on April 3, 2021 [11 favorites]
posted by mhoye at 1:03 PM on April 3, 2021 [11 favorites]
that AlphaGo etc. are limited to games isn't some of the story, it's most of the story.
Quoted for truth.
It's not that human's are maximally intelligent, but in terms of energy consumption, we're still far more efficient than any machines we've built
Out of the fucking park.
The energy budget for the rapidly adaptive reality-modelling section of a human-comparable AI needs to be around ten watts, tops, to be competitive.
posted by flabdablet at 1:11 PM on April 3, 2021 [7 favorites]
Quoted for truth.
It's not that human's are maximally intelligent, but in terms of energy consumption, we're still far more efficient than any machines we've built
Out of the fucking park.
The energy budget for the rapidly adaptive reality-modelling section of a human-comparable AI needs to be around ten watts, tops, to be competitive.
posted by flabdablet at 1:11 PM on April 3, 2021 [7 favorites]
I'd like to note, since there seems to be some confusion in the thread about this, that the NYer piece (the first main link) doesn't argue against the possibility of artificial intelligence (in the sense of conscious intelligence). I certainly get the impression that he doesn't think it's likely any time soon, but he doesn't present any arguments for or against that possibility. What he argues against is the idea of the singularity, specifically, which is a claim that once artificial intelligence is created, it will necessarily engage in a sort of unlimited, unfettered bootstrapping. The issue of the necessity of this outcome aside, Chiang points out that there are significant problems with even the feasibility of this claim. I think his arguments are, for the most part, quite reasonable for the point they are trying to make.
(Though I think we've collectively expanded upon and strengthened his arguments in this comment thread - a point, as I think others noted above, for what he describes as conceptual technology.)
posted by eviemath at 1:22 PM on April 3, 2021 [10 favorites]
(Though I think we've collectively expanded upon and strengthened his arguments in this comment thread - a point, as I think others noted above, for what he describes as conceptual technology.)
posted by eviemath at 1:22 PM on April 3, 2021 [10 favorites]
The energy budget for the rapidly adaptive reality-modelling section of a human-comparable AI needs to be around ten watts, tops, to be competitive.
A particularly tall order given that evolution has been optimising us for energy efficiency 3 or 4 billion years longer than it has optimized us for intelligence.
posted by straight at 1:26 PM on April 3, 2021 [3 favorites]
A particularly tall order given that evolution has been optimising us for energy efficiency 3 or 4 billion years longer than it has optimized us for intelligence.
posted by straight at 1:26 PM on April 3, 2021 [3 favorites]
Well I'm not convinced that some people objecting to my one comment in this thread actually know what theoretical CS today is. Like I said, the analogy is more like genetics being a subfield of biology. Or maybe like theoretical physics being a subfield of physics. And it is not like talking about laser printers, and I find such clever snarky gotchas (additionally, as a queer person of color experiencing such attitudes in a white-male dominated field) less helpful than actually showing having spent time learning the actual material? Like literally:
I doubt you'd get any disagreement from Chiang that the creation of a highly performant optimizing compiler is a profound act of creativity on the part of the entity who wrote the source code.
Except there are articles and papers, like The Significance of the Curry-Howard Isomorphism, and I wouldn't walk away from section 6 of that being so dismissive that such fundamental concepts are not deeply relevant to something like the design of an AI.
Computation modelling and complexity theory have nothing at all to say about moral agency, lived experience or social consequence, which are the meat of the interview.
We don't say the moral issues around curing cancer, or making vaccines, means dispensing with learning about genetics and fundamental biology. So what's the reason for making an exception here, again?
With the other frequently mentioned example in such debates, the real significance of AlphaGo is that it is a demonstration of tackling the complexity class PSPACE. This is very damaging to objections of nontransferability ("it can only play Go") and real life ("it is not connected to the real world"). The latter point is challenged by the extended Church-Turing thesis.
Theoretical computer scientists, ranging from Scott Aaronson, to Microsoft's own Yuri Gurevich, have discussed these topics like the Church-Turing thesis at a high level and their talks are available on youtube. They are very instructive and help to shed a lot of lay impressions of what theoretical CS is about and why knowing it matters more than ever in AI debates.
The idea that AI researchers can just ignore learning about foundational stuff in CS is wrong as well, on multiple levels. Chomsky himself has strongly criticized the research direction that Norvig and contemporary machine-learning approaches have taken. For similar reasons. And just look at the more recent theoretical papers on machine learning: plenty of them actually do try to address questions of how neural nets relate to computability (e.g., are recursive neural nets Turing complete or not?), and what their precise complexity classification is, etc. The notion that contemporary AI is just statistics or doesn't need to work with foundations is a deeply superficial one, that AI researchers themselves are somewhat guilty of promulgating, to the chagrin of theorists. (Incidentally, "information theory" as eviemath brought up is really an adjacent area: during the Turing centennial talk I attended, it was organized/led by a world class information theorist).
So, like, lay objections really need to demonstrate good faith understanding, and for some reason, this standard is not applied to theoretical computer science, because it is a comparatively young field, versus in other fields of study. We wouldn't say it is possible/impossible to cure cancer without first demonstrating firm grasp of genetics and being well-versed in concepts in it. So it's not okay to just declare, well, complexity theory or theoretical CS just aren't relevant, and even worse, based on preconceptions of it. There are, like, books and courses out there. When I satisfied my minor studies in theoretical CS, I had to go through classes like theory of programming languages, theory of computation and complexity theory, and even had to take an analytic philosophy class on computability (and in order to learn about Kripke structures, the basis of modern research programmes into program verification). In my own department of grad students asked to take these classes, 90% of them never returned for the second day of class. So the state of knowledge in this area is just not necessarily well represented even within various related disciplines.
posted by polymodus at 3:45 PM on April 3, 2021 [11 favorites]
I doubt you'd get any disagreement from Chiang that the creation of a highly performant optimizing compiler is a profound act of creativity on the part of the entity who wrote the source code.
Except there are articles and papers, like The Significance of the Curry-Howard Isomorphism, and I wouldn't walk away from section 6 of that being so dismissive that such fundamental concepts are not deeply relevant to something like the design of an AI.
Computation modelling and complexity theory have nothing at all to say about moral agency, lived experience or social consequence, which are the meat of the interview.
We don't say the moral issues around curing cancer, or making vaccines, means dispensing with learning about genetics and fundamental biology. So what's the reason for making an exception here, again?
With the other frequently mentioned example in such debates, the real significance of AlphaGo is that it is a demonstration of tackling the complexity class PSPACE. This is very damaging to objections of nontransferability ("it can only play Go") and real life ("it is not connected to the real world"). The latter point is challenged by the extended Church-Turing thesis.
Theoretical computer scientists, ranging from Scott Aaronson, to Microsoft's own Yuri Gurevich, have discussed these topics like the Church-Turing thesis at a high level and their talks are available on youtube. They are very instructive and help to shed a lot of lay impressions of what theoretical CS is about and why knowing it matters more than ever in AI debates.
The idea that AI researchers can just ignore learning about foundational stuff in CS is wrong as well, on multiple levels. Chomsky himself has strongly criticized the research direction that Norvig and contemporary machine-learning approaches have taken. For similar reasons. And just look at the more recent theoretical papers on machine learning: plenty of them actually do try to address questions of how neural nets relate to computability (e.g., are recursive neural nets Turing complete or not?), and what their precise complexity classification is, etc. The notion that contemporary AI is just statistics or doesn't need to work with foundations is a deeply superficial one, that AI researchers themselves are somewhat guilty of promulgating, to the chagrin of theorists. (Incidentally, "information theory" as eviemath brought up is really an adjacent area: during the Turing centennial talk I attended, it was organized/led by a world class information theorist).
So, like, lay objections really need to demonstrate good faith understanding, and for some reason, this standard is not applied to theoretical computer science, because it is a comparatively young field, versus in other fields of study. We wouldn't say it is possible/impossible to cure cancer without first demonstrating firm grasp of genetics and being well-versed in concepts in it. So it's not okay to just declare, well, complexity theory or theoretical CS just aren't relevant, and even worse, based on preconceptions of it. There are, like, books and courses out there. When I satisfied my minor studies in theoretical CS, I had to go through classes like theory of programming languages, theory of computation and complexity theory, and even had to take an analytic philosophy class on computability (and in order to learn about Kripke structures, the basis of modern research programmes into program verification). In my own department of grad students asked to take these classes, 90% of them never returned for the second day of class. So the state of knowledge in this area is just not necessarily well represented even within various related disciplines.
posted by polymodus at 3:45 PM on April 3, 2021 [11 favorites]
Well I'm not convinced that some people objecting to my one comment in this thread actually know what theoretical CS today is.
Friend, this is not the way to win arguments.
posted by eviemath at 4:34 PM on April 3, 2021 [7 favorites]
Friend, this is not the way to win arguments.
posted by eviemath at 4:34 PM on April 3, 2021 [7 favorites]
Eviemath, that was a topic sentence which polymodus backed up with several paragraphs of examples.
posted by straight at 4:44 PM on April 3, 2021 [5 favorites]
posted by straight at 4:44 PM on April 3, 2021 [5 favorites]
TCS is a niche field, of course I don't fault people for not knowing it, but I have a problem with mainstream articles that constantly add heat to the questions rather than, like, do their homework? Apparently the debate keeps getting rehashed; here's an IEEE article by an actual AI professor at UC Berkeley. He says:
In summary, the “skeptics”—those who argue that the risk from AI is negligible—have failed to explain why superintelligent AI systems will necessarily remain under human control; and they have not even tried to explain why superintelligent AI systems will never be developed. ... We need to do a substantial amount of work to reshape and rebuild the foundations of AI.
So my essential objection here, is not even a new one.
posted by polymodus at 4:57 PM on April 3, 2021 [1 favorite]
In summary, the “skeptics”—those who argue that the risk from AI is negligible—have failed to explain why superintelligent AI systems will necessarily remain under human control; and they have not even tried to explain why superintelligent AI systems will never be developed. ... We need to do a substantial amount of work to reshape and rebuild the foundations of AI.
So my essential objection here, is not even a new one.
posted by polymodus at 4:57 PM on April 3, 2021 [1 favorite]
straight, the calling into question the credentials of other commenters did not specifically mention who polymodus though had insufficient credentials to be commenting on this thread; they did not give an academically rigorous argument for their assertion that non-specialists should refrain from commenting on academic topics in a non-specialist, non-academic public forum; nor did they give full detail about their own qualifications in their appeal to authority argument.
posted by eviemath at 5:12 PM on April 3, 2021 [10 favorites]
posted by eviemath at 5:12 PM on April 3, 2021 [10 favorites]
I'm interested in hearing more on the relationship between general (artificial) intelligence and complexity theory. From my dimly remembered grad school CS classes, complexity theory tells us what is impossible, like for example, a polynomial time solution to the traveling salesmen problem. However the existence of humans tells us that general intelligence has already been implemented in linear time using biology.
posted by monotreme at 5:40 PM on April 3, 2021
posted by monotreme at 5:40 PM on April 3, 2021
Nelson: The limitation of these things is they only play games. Extending them to complicated real world problems is hard. Training in anything like common sense knowledge or adaptability to random inputs and circumstances is well beyond the state of the art.
I am not a specialist in this field, though I majored in an adjacent field in college (cognitive science), so I'm open to being corrected on anything. However, for the past 6 months I lived with a relatively high-level researcher at DeepMind, who happens to be a good friend from high school, and so I got a kind of unique glimpse into the actual state of the art in AI.
I’m not nearly technical enough to go into details, but the upshot is that “extending them to complicated real world problems” is exactly what they are doing now. DeepMind is currently working on this in two ways:
1. By placing virtual agents in a simulated physical environment, and getting them to ‘learn’ their environment and the objects in it, and how to manipulate those objects.
2. They are in the very early stages of figuring out a way to have agents learn from real people’s behavior (consensually!) while browsing the web, which is another kind of way of ‘interacting with the real world’.
With this problem in general, of AI interacting with the real world, I’ve been given to understand that the bottleneck will be the hardware, the robotics. But they are working on the software as we speak.
So yeah, Chiang’s New Yorker piece talking about “the way A.I. programs are currently designed” as being task-specific and needing to be supervised while trained, is, as Nelson implied above, shockingly out of date, and I’m surprised it got by the New Yorker’s fact checkers. I would guess it slid because of Chiang’s celebrity and whatever other editorial factors induce sloppiness. That's like 2015-era AI, which is a very long lag in a field moving as fast as this.
posted by skwt at 6:08 PM on April 3, 2021 [2 favorites]
I am not a specialist in this field, though I majored in an adjacent field in college (cognitive science), so I'm open to being corrected on anything. However, for the past 6 months I lived with a relatively high-level researcher at DeepMind, who happens to be a good friend from high school, and so I got a kind of unique glimpse into the actual state of the art in AI.
I’m not nearly technical enough to go into details, but the upshot is that “extending them to complicated real world problems” is exactly what they are doing now. DeepMind is currently working on this in two ways:
1. By placing virtual agents in a simulated physical environment, and getting them to ‘learn’ their environment and the objects in it, and how to manipulate those objects.
2. They are in the very early stages of figuring out a way to have agents learn from real people’s behavior (consensually!) while browsing the web, which is another kind of way of ‘interacting with the real world’.
With this problem in general, of AI interacting with the real world, I’ve been given to understand that the bottleneck will be the hardware, the robotics. But they are working on the software as we speak.
So yeah, Chiang’s New Yorker piece talking about “the way A.I. programs are currently designed” as being task-specific and needing to be supervised while trained, is, as Nelson implied above, shockingly out of date, and I’m surprised it got by the New Yorker’s fact checkers. I would guess it slid because of Chiang’s celebrity and whatever other editorial factors induce sloppiness. That's like 2015-era AI, which is a very long lag in a field moving as fast as this.
posted by skwt at 6:08 PM on April 3, 2021 [2 favorites]
nobody:
metafilter: “Clearly this guy has never worked with antenna designers.“
posted by Verg at 6:18 PM on April 3, 2021 [8 favorites]
metafilter: “Clearly this guy has never worked with antenna designers.“
posted by Verg at 6:18 PM on April 3, 2021 [8 favorites]
We already know how to build autonomous agents. The problem is that they want to be treated like human beings; because that’s what they are.
The holy grail of A.I. is some thing that acts like a human being but you don’t have to treat it like one. Which, not coincidentally, is something we ALSO already know how to do.
Maybe we build AIs that are more docile and easier to control than human beings. I don’t imagine that working out well for either the AIs or the 99% that they replace. Probably a sweet deal for the 1% though, so expect a lot of investment. Growth industry!
posted by notoriety public at 6:24 PM on April 3, 2021 [2 favorites]
The holy grail of A.I. is some thing that acts like a human being but you don’t have to treat it like one. Which, not coincidentally, is something we ALSO already know how to do.
Maybe we build AIs that are more docile and easier to control than human beings. I don’t imagine that working out well for either the AIs or the 99% that they replace. Probably a sweet deal for the 1% though, so expect a lot of investment. Growth industry!
posted by notoriety public at 6:24 PM on April 3, 2021 [2 favorites]
I didn't think he gave his parallel with the ontological argument enough support or attention, which is a shame, because I thought that was interesting. The ontological argument is always fascinating, what a weird piece of reasoning. Kant has ruled that existence is not a property; that is also a shame, because it is would be really cool if like unicorns have all kinds of properties, but just not that one.
posted by thelonius at 6:29 PM on April 3, 2021
posted by thelonius at 6:29 PM on April 3, 2021
Kant has ruled that existence is not a property ...
For those who think Kant was wrong about that (and he was), there's free logic.
posted by Jonathan Livengood at 7:12 PM on April 3, 2021 [4 favorites]
For those who think Kant was wrong about that (and he was), there's free logic.
posted by Jonathan Livengood at 7:12 PM on April 3, 2021 [4 favorites]
My favorite quote from the interview.
Suffering precedes moral agency in sort of the developmental ladder. Dogs are not moral agents, but they are capable of experiencing suffering. Babies are not moral agents yet, but they have the clear potential to become so. And they are definitely capable of experiencing suffering. And the closer that an entity gets to being a moral agent, the more that it’s suffering, it’s deserving of consideration, the more we should try and avoid inflicting suffering on it. So in the process of developing machines that are conscious and moral agents, we will be inevitably creating billions of entities that are capable of suffering. And we will inevitably inflict suffering on them. And that seems to me clearly a bad idea.
posted by Beholder at 7:35 PM on April 3, 2021 [5 favorites]
Suffering precedes moral agency in sort of the developmental ladder. Dogs are not moral agents, but they are capable of experiencing suffering. Babies are not moral agents yet, but they have the clear potential to become so. And they are definitely capable of experiencing suffering. And the closer that an entity gets to being a moral agent, the more that it’s suffering, it’s deserving of consideration, the more we should try and avoid inflicting suffering on it. So in the process of developing machines that are conscious and moral agents, we will be inevitably creating billions of entities that are capable of suffering. And we will inevitably inflict suffering on them. And that seems to me clearly a bad idea.
posted by Beholder at 7:35 PM on April 3, 2021 [5 favorites]
Dogs are not moral agents
This is where narrow scientism leads, to ranking life-forms below machines. Just another propaganda weapon in the creation of planet factory.
posted by No Robots at 7:53 PM on April 3, 2021
This is where narrow scientism leads, to ranking life-forms below machines. Just another propaganda weapon in the creation of planet factory.
posted by No Robots at 7:53 PM on April 3, 2021
the real significance of AlphaGo is that it is a demonstration of tackling the complexity class PSPACE
I'm lazy, I'll quote myself from the other thread:
"Factoring large numbers is in PSPACE, way way inside PSPACE. Yet we are not cracking RSA by reduction to AlphaGo. Complexity theory is anti-relevant here. We have not even solved Go in the formal sense at all..."
But lemme expand on that last bit. Solving size-n Go is the PSPACE decision problem -- "given this board state, does this particular move force a win against a perfect opponent?" AlphaZero can't actually answer that. AlphaZero can beat humans.
Approximation complexity classes such as PTAS would be a notch more relevant. Or a circuit approximation class, given that brains operating in real time are limited-depth circuits that solve approximation problems.
posted by away for regrooving at 8:09 PM on April 3, 2021 [6 favorites]
I'm lazy, I'll quote myself from the other thread:
"Factoring large numbers is in PSPACE, way way inside PSPACE. Yet we are not cracking RSA by reduction to AlphaGo. Complexity theory is anti-relevant here. We have not even solved Go in the formal sense at all..."
But lemme expand on that last bit. Solving size-n Go is the PSPACE decision problem -- "given this board state, does this particular move force a win against a perfect opponent?" AlphaZero can't actually answer that. AlphaZero can beat humans.
Approximation complexity classes such as PTAS would be a notch more relevant. Or a circuit approximation class, given that brains operating in real time are limited-depth circuits that solve approximation problems.
posted by away for regrooving at 8:09 PM on April 3, 2021 [6 favorites]
The holy grail of A.I. is some thing that acts like a human being but you don’t have to treat it like one.
Holy grail of capitalism, was his point, and I think it's a good one.
posted by away for regrooving at 8:12 PM on April 3, 2021 [8 favorites]
Holy grail of capitalism, was his point, and I think it's a good one.
posted by away for regrooving at 8:12 PM on April 3, 2021 [8 favorites]
The holy grail of A.I. is some thing that acts like a human being but you don’t have to treat it like one.
Holy grail of capitalism, was his point, and I think it's a good one.
The holy grail of our current system of global state capitalism is to rank all life-forms below machines. Scientistic machine fetishism serves this end. All this talk of AI with moral agency is ridiculous and serves only to enlist science lovers as shock troops in global fascist biocide.
posted by No Robots at 11:16 PM on April 3, 2021 [2 favorites]
Holy grail of capitalism, was his point, and I think it's a good one.
The holy grail of our current system of global state capitalism is to rank all life-forms below machines. Scientistic machine fetishism serves this end. All this talk of AI with moral agency is ridiculous and serves only to enlist science lovers as shock troops in global fascist biocide.
posted by No Robots at 11:16 PM on April 3, 2021 [2 favorites]
This is where narrow scientism leads, to ranking life-forms below machines. Just another propaganda weapon in the creation of planet factory.
Could you tie this back to the article? Where he's arguing that this hypothetical machine, that you'd blame if it bit you like you wouldn't with a dog, should never be attempted? I may be missing aspects of your, ah, whole deal here.
posted by away for regrooving at 11:55 PM on April 3, 2021 [4 favorites]
Could you tie this back to the article? Where he's arguing that this hypothetical machine, that you'd blame if it bit you like you wouldn't with a dog, should never be attempted? I may be missing aspects of your, ah, whole deal here.
posted by away for regrooving at 11:55 PM on April 3, 2021 [4 favorites]
Some disjointed thoughts/responses...
1. Think I mentioned this in the last thread on consciousness, but using alpha go as the only goal post is incredibly short sighted. GPT3 is legit magical, for example, and handles pretty much arbitrary text. There's no 'just a game' there.
2. Meanwhile, the audio synthesis space I work in has gone from requiring 20 minutes of badass server time to produce one second of speech in 2016 to running faster then real-time on a single thread on a phone today. I tend to think that once we get the missing pieces for consciousness, it'll be a very short few years before it runs in your pocket. Giant networks provide a huge amount of flexibility for exploration, but once a reasonably good solution is found we can create far smaller, efficient models that do the same thing, using pruning, distillation, and related techniques.
3. As a practitioner, complexity theory mostly seems to be completely missing the real questions. When I see a new problem, theres no complexity theory answer for whether a neutral network will be able to do a decent job at it. Sorting is combinatorics, but classification is a weird mix of statistics and context, and to the best of my knowledge, we don't have any useful description of how 'hard' that contextual problem is. As a result, my main heuristic for whether a neutral net can solve an audio problem is whether there are human experts that can do it.
4. Finally, I'm seeing more and more examples lately where the actual model architecture barely matters, so long as it's reasonably modern and large enough. Losses and datasets are where the real interesting stuff happens. If you want consciousness, you need to do something different on the loss front, but it's not totally clear what. The bits that look most useful to me are GANs and 'fill in the blank' general prediction problems (like gpt uses for training). But there's definitely some important pieces missing, and I think whatever it is, the model architecture won't really matter.
posted by kaibutsu at 11:55 PM on April 3, 2021 [6 favorites]
1. Think I mentioned this in the last thread on consciousness, but using alpha go as the only goal post is incredibly short sighted. GPT3 is legit magical, for example, and handles pretty much arbitrary text. There's no 'just a game' there.
2. Meanwhile, the audio synthesis space I work in has gone from requiring 20 minutes of badass server time to produce one second of speech in 2016 to running faster then real-time on a single thread on a phone today. I tend to think that once we get the missing pieces for consciousness, it'll be a very short few years before it runs in your pocket. Giant networks provide a huge amount of flexibility for exploration, but once a reasonably good solution is found we can create far smaller, efficient models that do the same thing, using pruning, distillation, and related techniques.
3. As a practitioner, complexity theory mostly seems to be completely missing the real questions. When I see a new problem, theres no complexity theory answer for whether a neutral network will be able to do a decent job at it. Sorting is combinatorics, but classification is a weird mix of statistics and context, and to the best of my knowledge, we don't have any useful description of how 'hard' that contextual problem is. As a result, my main heuristic for whether a neutral net can solve an audio problem is whether there are human experts that can do it.
4. Finally, I'm seeing more and more examples lately where the actual model architecture barely matters, so long as it's reasonably modern and large enough. Losses and datasets are where the real interesting stuff happens. If you want consciousness, you need to do something different on the loss front, but it's not totally clear what. The bits that look most useful to me are GANs and 'fill in the blank' general prediction problems (like gpt uses for training). But there's definitely some important pieces missing, and I think whatever it is, the model architecture won't really matter.
posted by kaibutsu at 11:55 PM on April 3, 2021 [6 favorites]
I'm going to note, for readers who may not be aware, that there's a bit of a conversational crossover going on here with the recent self-driving cars thread. In that thread I characterised some of polymodus' comments about computational complexity as not being relevant to the topic at hand.
Specifically, I said that in the context of driving, computational complexity isn't the current problem - the problem is that the task isn't well-defined in a formal sense. That makes complexity theory largely inapplicable and irrelevant as I see it, because to even meaningfully talk about the computational complexity of a problem, you need a definition of what that problem is, in a form that's amenable to analysis.
Now we're talking about general intelligence, which is a concept that might be characterised by the very absence of that definition. General intelligence can be thought of as the ability to succeed at a task without either prior knowledge of that task, or any formal definition of how it should be carried out. With enough of this magic quantity of "general intelligence", self-driving cars could fill in the gaps themselves to handle the ill-defined task of driving in the real world - or at least that's the idea.
But I feel like this concept of general intelligence is itself ill-defined, and I want to try to explain why, because I think it might be relevant to how - or even if - computational complexity theory can be applied to foundational issues in AI.
This will be a really long comment but let me try to explain my thinking. This is going to take a few steps. At each step we're going to go up one level of abstraction.
Level 0 - antennas
I have a CS degree, but my background is more in engineering. And as I alluded to in my earlier comment, I think about engineering and problem solving in terms of goals.
Let's use antenna design as an example task again. You can't just tell a superintelligent AI, or indeed a human, to "design the best possible antenna". What does best mean? Best for what? You have to give a goal for the task to be meaningful. In real-world engineering this might be a bullet-point list of vague desires, which need to be clarified by a lot of back-and-forth discussions with a client. But the formal equivalent - what you might try to distill from that informal input - is usually called a cost function. It defines how bad a solution is due to undesirable things like cost, size, weight, noise, loss, etc - with some weighting and balancing of each. Find a solution that minimises the cost function, and you have a "best" solution for the goal. It's the same thing as maximising a utility function, if you prefer that terminology.
Let's say we're looking for a 2.4GHz antenna with at least 50MHz of -3dB bandwidth, 6dBi of horizontal gain, and minimal size and fabrication cost. We can express all that formally in a cost function. The details don't matter. It's a function that takes an antenna design and returns a number which you want to be small.
Cost functions are great in engineering because you can use them to objectively evaluate candidate solutions and compare them. They're also great because you can feed them to computer programs to do that for you automatically. You can have the software explore a large search space of possible solutions for you, either by brute force, or with some degree of intelligence. This works because by providing a cost function, the task has become well-defined. The computer doesn't need to ask you what you think at each step - you've told it how to evaluate that for itself.
All this already exists. Many antenna design programs include optimisers that do this. They don't require general intelligence to be useful. They just need to be able to solve a specific class of well-defined optimisation problems, given a search space and a goal expressed through a cost function.
Level 1 - antenna design programs
Now let's step up one level of abstraction, and instead of thinking about engineering antennas, let's think about engineering antenna design programs.
You can compare one antenna design program with another. Maybe, given our antenna search space and cost function, program A finds a solution for which the cost function is lower than function B. Perhaps A did that because it explored some corner of the search space that program B didn't. Maybe program B used a gradient descent approach and got stuck in a local minimum somewhere. But program B was faster! It got you a result in a minute, whereas program A took an hour to run. Which antenna design program is better?
Again, that's not a question that's well-defined. One is faster, but the other got a better result. Which matters more? To objectively compare the two programs, we need another cost function. This cost function might weigh, amongst other things, execution time against the quality of the result. Perhaps we're willing to wait two hours to get a solution that's 20% better according to our Level 0 cost function, but not a whole day to get one that's only 5% better. Our new cost function can express that in a formal way.
Once we have our Level 1 cost function, we could use it to try to automatically create a better antenna design program! There's a lot of approaches in the CS and software engineering literature about how this might be accomplished. Genetic algorithms are a common approach which tries to mimic biological evolutionary processes: combining successful programs, mutating them, and then selecting the "best" offspring for the next generation of the process.
So we use some genetic algorithm framework, to try to automatically create a better antenna design program, by breeding millions of antenna design programs. We'll evaluate each program according to our Level 1 cost function, which in turn uses our Level 0 cost function to evaluate the quality of the antenna design it comes up with, as well as how fast it runs. Then we'll get a new antenna design program that's better at our antenna design problem, and we can reuse that improved program in the future.
Great idea right? Well, actually no. There's a really hilarious failure mode that happens a lot when you try to do something like this. Sooner or later, evolution will discover a short cut that "cheats".
We're only testing each program for how it performs on our specific antenna design problem from Level 0, right? So what happens when one of the programs we randomly generate just outputs a fixed solution that happens to be a good solution for our Level 0 cost function, without even doing any work? It wins! It gets a perfect result in practically zero execution time, which is 100% valid by our definitions - but it's completely useless in practice as an antenna design program.
Level 2 - useful antenna design programs
In order to prevent the evolution process from discovering that short cut, we need to evaluate each candidate antenna design program on not just one test case, but a whole set of test cases.
What should that set of test cases look like? It could be pretty arbitrary. There's a lot of ways to choose.
We want our new antenna design program to be useful to us in the future. So maybe we pick the last 100 antenna design problems that came through our antenna engineering firm, and re-run the evolution process using all of those as test cases rather than just one. Each test case will have its own Level 0 cost function.
This takes a hell of a lot of CPU time, because it's generating millions of candidate programs, and running 100 test cases on each, and each of those might in turn be running thousands or millions of simulations to do the actual design optimisation we started this whole yak shaving exercise for.
But after some time we get a new antenna design program that actually performs better at those last 100 antenna design problems that came through our engineering firm.
This still might not help us the next year, when a client comes to us with a very different problem. Maybe the last 100 designs we did were all multiband HF wire antennas and the next client needs an X-band phased array. Maybe our new autogenerated optimised antenna design program doesn't even give valid results in that case.
Adding more test cases also doesn't prevent the previous failure mode of our genetic algorithm evolution process. It could still generate a short cut program which just spots those 100 test cases and spits out solutions that happen to be good for them. This is called overfitting.
It may sound unlikely in this scenario, but in practical machine learning systems overfitting can be pretty well hidden because the inner workings can be so opaque. It's how you get machine vision systems that are supposed to be distinguishing tanks from cars, but are actually looking for road signs because those appeared in the car pictures in the training dataset and not the tank ones.
Level 3 - antenna design program design programs
Let's go up another level of abstraction. We weren't that happy with the genetic algorithm framework we used at levels 1 and 2. It overfitted easily, and it took a lot of CPU time to run.
Maybe we can try some different tools. We'll take a bunch of different automatic software improvement tools and try them all on the work we did in level 2. We'll define yet another cost function to evaluate them, which will depend in turn on the cost functions we defined in level 1 and level 0.
Level 4 - generic design program design programs
Unfortunately, all that level 3 found us was a program that works well for optimising antenna design programs. We want something that has more general capabilities for improving engineering software . So we're going to repeat the exercise from level 3 with hundreds of design program improvement tasks on software from all different branches of engineering. Each will have their own cost functions, training sets and tasks as per level 2, 1, 0...
Computation complexity of all this work is definitely becoming an issue, by the way. But it will all be worth it for the massive design improvements we'll get in the end... right?
And so on and so on as we try to reach:
Level N - "general" intelligence
I've put level N here, because it's not really clear how far up this hierarchy of abstraction we need to go before we're talking about systems that have "general" intelligence.
But what is clear to me is that at each step, we have to define several things:
Which makes me think that the idea of "general intelligence" isn't actually a formally definable thing; it's not something generic or fundamental that's independent of humans.
To even evaluate "intelligence" of a system, such that we can meaningfully engineer to improve it, or have it improve itself - then we need to define the specific tasks on which we're going to test it, and how we're going to evaluate its performance. And it's those definitions which will define the computational complexity of evaluating and optimising the performance of the resulting system.
I think that our whole concept of intelligence is actually deeply rooted in what humans are good at, what humans want, and what humans do. We're so deep in our unstated assumptions that we don't see them.
But we can't even agree on how to define intelligence as a measurable thing in humans. Every discussion of intelligence as a numerical quantity starts with acknowledging that IQ is a terrible metric of just about anything except some extremely culturally-specific test-taking skills. And yet we still don't have a better measure.
We can engineer systems to emulate humans. That's a definable goal. It's most of what's happening today in fact, and we already see as a result the biases that are transferred from humans to machines trained on our output. If we're going to engineer machines to emulate humans, we should be honest about the reality and the implications of that.
Or we can engineer systems to pursue different goals. But to do that we need to define what those goals are and how they can be formally expressed, and we need to be honest about the fact that those definitions have to exist.
Otherwise, the only generic definition of "general intelligence" that makes any sense to me is "ability to minimise all possible cost functions over all possible search spaces".
And I'm shit at theory, but I feel like if you try to even think about the computational complexity of that, you're going to run hard into Gödel's incompleteness theorems somewhere.
posted by automatronic at 12:01 AM on April 4, 2021 [21 favorites]
Specifically, I said that in the context of driving, computational complexity isn't the current problem - the problem is that the task isn't well-defined in a formal sense. That makes complexity theory largely inapplicable and irrelevant as I see it, because to even meaningfully talk about the computational complexity of a problem, you need a definition of what that problem is, in a form that's amenable to analysis.
Now we're talking about general intelligence, which is a concept that might be characterised by the very absence of that definition. General intelligence can be thought of as the ability to succeed at a task without either prior knowledge of that task, or any formal definition of how it should be carried out. With enough of this magic quantity of "general intelligence", self-driving cars could fill in the gaps themselves to handle the ill-defined task of driving in the real world - or at least that's the idea.
But I feel like this concept of general intelligence is itself ill-defined, and I want to try to explain why, because I think it might be relevant to how - or even if - computational complexity theory can be applied to foundational issues in AI.
This will be a really long comment but let me try to explain my thinking. This is going to take a few steps. At each step we're going to go up one level of abstraction.
Level 0 - antennas
I have a CS degree, but my background is more in engineering. And as I alluded to in my earlier comment, I think about engineering and problem solving in terms of goals.
Let's use antenna design as an example task again. You can't just tell a superintelligent AI, or indeed a human, to "design the best possible antenna". What does best mean? Best for what? You have to give a goal for the task to be meaningful. In real-world engineering this might be a bullet-point list of vague desires, which need to be clarified by a lot of back-and-forth discussions with a client. But the formal equivalent - what you might try to distill from that informal input - is usually called a cost function. It defines how bad a solution is due to undesirable things like cost, size, weight, noise, loss, etc - with some weighting and balancing of each. Find a solution that minimises the cost function, and you have a "best" solution for the goal. It's the same thing as maximising a utility function, if you prefer that terminology.
Let's say we're looking for a 2.4GHz antenna with at least 50MHz of -3dB bandwidth, 6dBi of horizontal gain, and minimal size and fabrication cost. We can express all that formally in a cost function. The details don't matter. It's a function that takes an antenna design and returns a number which you want to be small.
Cost functions are great in engineering because you can use them to objectively evaluate candidate solutions and compare them. They're also great because you can feed them to computer programs to do that for you automatically. You can have the software explore a large search space of possible solutions for you, either by brute force, or with some degree of intelligence. This works because by providing a cost function, the task has become well-defined. The computer doesn't need to ask you what you think at each step - you've told it how to evaluate that for itself.
All this already exists. Many antenna design programs include optimisers that do this. They don't require general intelligence to be useful. They just need to be able to solve a specific class of well-defined optimisation problems, given a search space and a goal expressed through a cost function.
Level 1 - antenna design programs
Now let's step up one level of abstraction, and instead of thinking about engineering antennas, let's think about engineering antenna design programs.
You can compare one antenna design program with another. Maybe, given our antenna search space and cost function, program A finds a solution for which the cost function is lower than function B. Perhaps A did that because it explored some corner of the search space that program B didn't. Maybe program B used a gradient descent approach and got stuck in a local minimum somewhere. But program B was faster! It got you a result in a minute, whereas program A took an hour to run. Which antenna design program is better?
Again, that's not a question that's well-defined. One is faster, but the other got a better result. Which matters more? To objectively compare the two programs, we need another cost function. This cost function might weigh, amongst other things, execution time against the quality of the result. Perhaps we're willing to wait two hours to get a solution that's 20% better according to our Level 0 cost function, but not a whole day to get one that's only 5% better. Our new cost function can express that in a formal way.
Once we have our Level 1 cost function, we could use it to try to automatically create a better antenna design program! There's a lot of approaches in the CS and software engineering literature about how this might be accomplished. Genetic algorithms are a common approach which tries to mimic biological evolutionary processes: combining successful programs, mutating them, and then selecting the "best" offspring for the next generation of the process.
So we use some genetic algorithm framework, to try to automatically create a better antenna design program, by breeding millions of antenna design programs. We'll evaluate each program according to our Level 1 cost function, which in turn uses our Level 0 cost function to evaluate the quality of the antenna design it comes up with, as well as how fast it runs. Then we'll get a new antenna design program that's better at our antenna design problem, and we can reuse that improved program in the future.
Great idea right? Well, actually no. There's a really hilarious failure mode that happens a lot when you try to do something like this. Sooner or later, evolution will discover a short cut that "cheats".
We're only testing each program for how it performs on our specific antenna design problem from Level 0, right? So what happens when one of the programs we randomly generate just outputs a fixed solution that happens to be a good solution for our Level 0 cost function, without even doing any work? It wins! It gets a perfect result in practically zero execution time, which is 100% valid by our definitions - but it's completely useless in practice as an antenna design program.
Level 2 - useful antenna design programs
In order to prevent the evolution process from discovering that short cut, we need to evaluate each candidate antenna design program on not just one test case, but a whole set of test cases.
What should that set of test cases look like? It could be pretty arbitrary. There's a lot of ways to choose.
We want our new antenna design program to be useful to us in the future. So maybe we pick the last 100 antenna design problems that came through our antenna engineering firm, and re-run the evolution process using all of those as test cases rather than just one. Each test case will have its own Level 0 cost function.
This takes a hell of a lot of CPU time, because it's generating millions of candidate programs, and running 100 test cases on each, and each of those might in turn be running thousands or millions of simulations to do the actual design optimisation we started this whole yak shaving exercise for.
But after some time we get a new antenna design program that actually performs better at those last 100 antenna design problems that came through our engineering firm.
This still might not help us the next year, when a client comes to us with a very different problem. Maybe the last 100 designs we did were all multiband HF wire antennas and the next client needs an X-band phased array. Maybe our new autogenerated optimised antenna design program doesn't even give valid results in that case.
Adding more test cases also doesn't prevent the previous failure mode of our genetic algorithm evolution process. It could still generate a short cut program which just spots those 100 test cases and spits out solutions that happen to be good for them. This is called overfitting.
It may sound unlikely in this scenario, but in practical machine learning systems overfitting can be pretty well hidden because the inner workings can be so opaque. It's how you get machine vision systems that are supposed to be distinguishing tanks from cars, but are actually looking for road signs because those appeared in the car pictures in the training dataset and not the tank ones.
Level 3 - antenna design program design programs
Let's go up another level of abstraction. We weren't that happy with the genetic algorithm framework we used at levels 1 and 2. It overfitted easily, and it took a lot of CPU time to run.
Maybe we can try some different tools. We'll take a bunch of different automatic software improvement tools and try them all on the work we did in level 2. We'll define yet another cost function to evaluate them, which will depend in turn on the cost functions we defined in level 1 and level 0.
Level 4 - generic design program design programs
Unfortunately, all that level 3 found us was a program that works well for optimising antenna design programs. We want something that has more general capabilities for improving engineering software . So we're going to repeat the exercise from level 3 with hundreds of design program improvement tasks on software from all different branches of engineering. Each will have their own cost functions, training sets and tasks as per level 2, 1, 0...
Computation complexity of all this work is definitely becoming an issue, by the way. But it will all be worth it for the massive design improvements we'll get in the end... right?
And so on and so on as we try to reach:
Level N - "general" intelligence
I've put level N here, because it's not really clear how far up this hierarchy of abstraction we need to go before we're talking about systems that have "general" intelligence.
But what is clear to me is that at each step, we have to define several things:
- Our goal. What we want to achieve. This is informal and may not be clearly understood, even by us.
- A cost function. This is the formal specification of our goal. It may or may not accurately reflect our informally stated goal, let alone what we really want in practice.
- A search space, or a training set. This is the space of solutions that are on the table to consider, or the test cases that we want to evaluate the cost function against.
Which makes me think that the idea of "general intelligence" isn't actually a formally definable thing; it's not something generic or fundamental that's independent of humans.
To even evaluate "intelligence" of a system, such that we can meaningfully engineer to improve it, or have it improve itself - then we need to define the specific tasks on which we're going to test it, and how we're going to evaluate its performance. And it's those definitions which will define the computational complexity of evaluating and optimising the performance of the resulting system.
I think that our whole concept of intelligence is actually deeply rooted in what humans are good at, what humans want, and what humans do. We're so deep in our unstated assumptions that we don't see them.
But we can't even agree on how to define intelligence as a measurable thing in humans. Every discussion of intelligence as a numerical quantity starts with acknowledging that IQ is a terrible metric of just about anything except some extremely culturally-specific test-taking skills. And yet we still don't have a better measure.
We can engineer systems to emulate humans. That's a definable goal. It's most of what's happening today in fact, and we already see as a result the biases that are transferred from humans to machines trained on our output. If we're going to engineer machines to emulate humans, we should be honest about the reality and the implications of that.
Or we can engineer systems to pursue different goals. But to do that we need to define what those goals are and how they can be formally expressed, and we need to be honest about the fact that those definitions have to exist.
Otherwise, the only generic definition of "general intelligence" that makes any sense to me is "ability to minimise all possible cost functions over all possible search spaces".
And I'm shit at theory, but I feel like if you try to even think about the computational complexity of that, you're going to run hard into Gödel's incompleteness theorems somewhere.
posted by automatronic at 12:01 AM on April 4, 2021 [21 favorites]
We're so deep in our unstated assumptions that we don't see them.
One of those unstated assumptions is that "one, two, three, many" is a counting system used only by the canonical "primitive" tribe that lives nowhere we've ever been but we're sure we remember hearing about some anthropological study of.
It isn't. It's pretty much a universal.
Read any treatise on awareness and you're pretty much guaranteed to run across the idea that the distinguishing feature of consciousness is that we're aware, we're aware that we're aware, we're aware that we're aware that we're aware, "and so on ad infinitum". One, two, three, many.
We form an opinion based on some collection of concrete examples we've encountered in our daily lives, we abstract it, we abstract the abstraction, and then it feels like a universally applicable principle. One, two, three, many. But we know about the infinitude of integers, and about inductive reasoning, and we just assume that the ideology we've just constructed is valid on the basis of some vague invocation of those principles. Do we ever do anything even close to a formal error-bar propagation check on any of this reasoning? Hell no. But we feel as if we have, and that's good enough.
Our brains are really good at applying a kind of Photoshop smart brush to our models of reality, imparting the feeling that we know what we're looking at in great detail, even when we're mostly just making shit up to fit in with what we actually are looking at. This is why, for example, it's really hard to find that tiny screw that's fallen off the workbench and onto the carpet: we're really bad at actually seeing the carpet and what's on it, but we're really good at behaving as if every part of the carpet we're not directly staring at looks just like the bit that we are. Finding the tiny screw involves deliberately ramming down that illusion and systematically scanning the whole surface of a plausible landing zone. The difficulty and tedium of this is a pretty good clue that this is not the kind of approach that usually gets us through our day.
I think our brains do this at every level of abstraction. We spend very little time actually looking at reality compared to the amount we spend on manipulating it on the basis of a mostly-good-enough model of it, because actually looking is simply too expensive in time and energy. It's really really easy for this pattern to lead us to assume that we know much more than we actually do, and that solutions to intractable problems are just around the corner because look how much really good work we've already done on them.
I have a distinct memory of being in third grade and suddenly realizing that I didn't need to go to school tomorrow because I already knew all the things, and I could prove it, because every single thing I thought of, I knew that.
Don't be me in third grade, is the point I'm clumsily blundering toward here. Acknowledge that in order to do a thing, the first step has to be coming up with a test of whether or not we've done it that's robust in the face of our inbuilt facility for fooling ourselves.
posted by flabdablet at 12:55 AM on April 4, 2021 [14 favorites]
One of those unstated assumptions is that "one, two, three, many" is a counting system used only by the canonical "primitive" tribe that lives nowhere we've ever been but we're sure we remember hearing about some anthropological study of.
It isn't. It's pretty much a universal.
Read any treatise on awareness and you're pretty much guaranteed to run across the idea that the distinguishing feature of consciousness is that we're aware, we're aware that we're aware, we're aware that we're aware that we're aware, "and so on ad infinitum". One, two, three, many.
We form an opinion based on some collection of concrete examples we've encountered in our daily lives, we abstract it, we abstract the abstraction, and then it feels like a universally applicable principle. One, two, three, many. But we know about the infinitude of integers, and about inductive reasoning, and we just assume that the ideology we've just constructed is valid on the basis of some vague invocation of those principles. Do we ever do anything even close to a formal error-bar propagation check on any of this reasoning? Hell no. But we feel as if we have, and that's good enough.
Our brains are really good at applying a kind of Photoshop smart brush to our models of reality, imparting the feeling that we know what we're looking at in great detail, even when we're mostly just making shit up to fit in with what we actually are looking at. This is why, for example, it's really hard to find that tiny screw that's fallen off the workbench and onto the carpet: we're really bad at actually seeing the carpet and what's on it, but we're really good at behaving as if every part of the carpet we're not directly staring at looks just like the bit that we are. Finding the tiny screw involves deliberately ramming down that illusion and systematically scanning the whole surface of a plausible landing zone. The difficulty and tedium of this is a pretty good clue that this is not the kind of approach that usually gets us through our day.
I think our brains do this at every level of abstraction. We spend very little time actually looking at reality compared to the amount we spend on manipulating it on the basis of a mostly-good-enough model of it, because actually looking is simply too expensive in time and energy. It's really really easy for this pattern to lead us to assume that we know much more than we actually do, and that solutions to intractable problems are just around the corner because look how much really good work we've already done on them.
I have a distinct memory of being in third grade and suddenly realizing that I didn't need to go to school tomorrow because I already knew all the things, and I could prove it, because every single thing I thought of, I knew that.
Don't be me in third grade, is the point I'm clumsily blundering toward here. Acknowledge that in order to do a thing, the first step has to be coming up with a test of whether or not we've done it that's robust in the face of our inbuilt facility for fooling ourselves.
posted by flabdablet at 12:55 AM on April 4, 2021 [14 favorites]
automatronic, I read your comment not because I wanted to, but because I felt you earned it. I broadly agree with most of what you wrote.
Otherwise, the only generic definition of "general intelligence" that makes any sense to me is "ability to minimise all possible cost functions over all possible search spaces".
Indeed, and I think the conclusion from this, leading back to your original hypothesis, is that general intelligence is ill-defined.
I have this intuition that most sciences are organized around questions that we cannot answer. Physics: What is reality? Biology: What is life? Cognitive Science: What is intelligence? We can pose these questions, but in trying to answer them all we do is raise more questions, and thereby form scientific disciplines. I don't think people expect final answers anymore to "What is life/reality?", and perhaps cognitive science (in a general sense) is just a bit too young for people to have accepted that it is a frame, and not a question.
As for whether or not we will achieve the singularity and be replaced by our own creations in the near future, I remain skeptical for the economic reasons expressed by Chiang in the first article, and further articulated by others in this post.
posted by Alex404 at 5:08 AM on April 4, 2021 [1 favorite]
Otherwise, the only generic definition of "general intelligence" that makes any sense to me is "ability to minimise all possible cost functions over all possible search spaces".
Indeed, and I think the conclusion from this, leading back to your original hypothesis, is that general intelligence is ill-defined.
I have this intuition that most sciences are organized around questions that we cannot answer. Physics: What is reality? Biology: What is life? Cognitive Science: What is intelligence? We can pose these questions, but in trying to answer them all we do is raise more questions, and thereby form scientific disciplines. I don't think people expect final answers anymore to "What is life/reality?", and perhaps cognitive science (in a general sense) is just a bit too young for people to have accepted that it is a frame, and not a question.
As for whether or not we will achieve the singularity and be replaced by our own creations in the near future, I remain skeptical for the economic reasons expressed by Chiang in the first article, and further articulated by others in this post.
posted by Alex404 at 5:08 AM on April 4, 2021 [1 favorite]
I have this intuition that most sciences are organized around questions that we cannot answer. Physics: What is reality? Biology: What is life? Cognitive Science: What is intelligence? We can pose these questions, but in trying to answer them all we do is raise more questions
...and as intelligent beings, we notice this pattern and come to understand that although "Describe X in detail." doesn't feel like an adequately profound substitute for "What is X?", it's the best we're ever going to be able to do.
If we don't notice this pattern, we eventually end up at some version of "Everything is X." Everything is fields (physics). Everything is information (compsci). Everything is consciousness (mysticism). Everything is One (Taoism). Everything is me (solipsism). Everything is illusory (Platonism). Everything is cyclic (Buddhism). Eveything is inadequate (Christianity). And on it goes. None of it is terribly useful even though it all feels tremendously profound.
I was fortunate enough to have a psychotic break twenty years ago, during which I repeatedly experienced certainty to an extent far more profound than has ever happened to me before or since. I say fortunate because, although the temporary ruination of all judgement could very easily have killed me, it didn't; and I've since had both opportunity and very strong motivation to ponder the way in which every single one of those transcendentally profound certainties has turned out, on examination from a non-psychotic point of view, to have been complete and utter crap.
I now deeply distrust the feeling of being profoundly certain that I'm right about something I can't fully articulate, which of course I, like every other human being, do continue to experience from time to time. It's not reliable. If I can't articulate a thing I can't cross-check it, and if I can't cross-check it then it doesn't merit much confidence.
I have yet to encounter any Singularity believer who can articulate recognition criteria for intelligence, artificial or otherwise, that don't amount to "I'm sure I'll know it when I see it". And if we can't even describe it properly, I completely fail to see how we could possibly hope to measure it. And if we can't measure it, I completely fail to see how we could possibly judge any instance of it as in any way superior to another. Which makes the Singularity not even wrong.
posted by flabdablet at 7:59 AM on April 4, 2021 [18 favorites]
...and as intelligent beings, we notice this pattern and come to understand that although "Describe X in detail." doesn't feel like an adequately profound substitute for "What is X?", it's the best we're ever going to be able to do.
If we don't notice this pattern, we eventually end up at some version of "Everything is X." Everything is fields (physics). Everything is information (compsci). Everything is consciousness (mysticism). Everything is One (Taoism). Everything is me (solipsism). Everything is illusory (Platonism). Everything is cyclic (Buddhism). Eveything is inadequate (Christianity). And on it goes. None of it is terribly useful even though it all feels tremendously profound.
I was fortunate enough to have a psychotic break twenty years ago, during which I repeatedly experienced certainty to an extent far more profound than has ever happened to me before or since. I say fortunate because, although the temporary ruination of all judgement could very easily have killed me, it didn't; and I've since had both opportunity and very strong motivation to ponder the way in which every single one of those transcendentally profound certainties has turned out, on examination from a non-psychotic point of view, to have been complete and utter crap.
I now deeply distrust the feeling of being profoundly certain that I'm right about something I can't fully articulate, which of course I, like every other human being, do continue to experience from time to time. It's not reliable. If I can't articulate a thing I can't cross-check it, and if I can't cross-check it then it doesn't merit much confidence.
I have yet to encounter any Singularity believer who can articulate recognition criteria for intelligence, artificial or otherwise, that don't amount to "I'm sure I'll know it when I see it". And if we can't even describe it properly, I completely fail to see how we could possibly hope to measure it. And if we can't measure it, I completely fail to see how we could possibly judge any instance of it as in any way superior to another. Which makes the Singularity not even wrong.
posted by flabdablet at 7:59 AM on April 4, 2021 [18 favorites]
He represents the physicalist mentality (heh, physicalist mentality) that predominates in science.
This is where narrow scientism leads, to ranking life-forms below machines. Just another propaganda weapon in the creation of planet factory.
The holy grail of our current system of global state capitalism is to rank all life-forms below machines. Scientistic machine fetishism serves this end. All this talk of AI with moral agency is ridiculous and serves only to enlist science lovers as shock troops in global fascist biocide.
It seems like you would rather keep throwing out vague but inflammatory accusations instead of providing any kind of evidence for them, or even defining your terms. It’s almost like you’re deliberately trying to obscure what your actual beliefs are and what they’re based on.
posted by en forme de poire at 8:19 AM on April 4, 2021 [4 favorites]
This is where narrow scientism leads, to ranking life-forms below machines. Just another propaganda weapon in the creation of planet factory.
The holy grail of our current system of global state capitalism is to rank all life-forms below machines. Scientistic machine fetishism serves this end. All this talk of AI with moral agency is ridiculous and serves only to enlist science lovers as shock troops in global fascist biocide.
It seems like you would rather keep throwing out vague but inflammatory accusations instead of providing any kind of evidence for them, or even defining your terms. It’s almost like you’re deliberately trying to obscure what your actual beliefs are and what they’re based on.
posted by en forme de poire at 8:19 AM on April 4, 2021 [4 favorites]
Everything is capitalism (Metafilter).
posted by flabdablet at 8:35 AM on April 4, 2021 [5 favorites]
posted by flabdablet at 8:35 AM on April 4, 2021 [5 favorites]
It’s almost like you’re deliberately trying to obscure what your actual beliefs are
"No Robots" using either the modern understanding of the word or Čapek's Slavik allusion sums it up really.
posted by meehawl at 8:38 AM on April 4, 2021
"No Robots" using either the modern understanding of the word or Čapek's Slavik allusion sums it up really.
posted by meehawl at 8:38 AM on April 4, 2021
I now deeply distrust the feeling of being profoundly certain that I'm right about something I can't fully articulate, which of course I, like every other human being, do continue to experience from time to time. It's not reliable. If I can't articulate a thing I can't cross-check it, and if I can't cross-check it then it doesn't merit much confidence.
Beautifully written. One thing I've learned (for myself anyway) while doing my first postdoc (on to my second now!) is not to get too invested in my feelings of certainty. I should definitely follow them! But all they do is provide a little nudge in some direction, and that directions points to truth at a level only barely above chance.
And yet the collective activity of scientists is able to sift through these nudges and move towards something like truth.
GPT3 is legit magical, for example, and handles pretty much arbitrary text. There's no 'just a game' there.
It's not magical, it's just cool. And there is a game, which is called cross-entropy minimization.
It's funny that you call it magical, because it loops back to the second article with Chiang. It used to be pretty magical that you could talk to people over vast distances, or look at satellite photos of your house, and now it's take for granted. It seems pretty magical that you can generate an alpha-draft of an article out of a list of bullet points, but eventually it will become common place, and join our list of super cool information processing tools. We will be continually supplemented, though I suspect, not supplanted.
posted by Alex404 at 9:07 AM on April 4, 2021 [4 favorites]
Beautifully written. One thing I've learned (for myself anyway) while doing my first postdoc (on to my second now!) is not to get too invested in my feelings of certainty. I should definitely follow them! But all they do is provide a little nudge in some direction, and that directions points to truth at a level only barely above chance.
And yet the collective activity of scientists is able to sift through these nudges and move towards something like truth.
GPT3 is legit magical, for example, and handles pretty much arbitrary text. There's no 'just a game' there.
It's not magical, it's just cool. And there is a game, which is called cross-entropy minimization.
It's funny that you call it magical, because it loops back to the second article with Chiang. It used to be pretty magical that you could talk to people over vast distances, or look at satellite photos of your house, and now it's take for granted. It seems pretty magical that you can generate an alpha-draft of an article out of a list of bullet points, but eventually it will become common place, and join our list of super cool information processing tools. We will be continually supplemented, though I suspect, not supplanted.
posted by Alex404 at 9:07 AM on April 4, 2021 [4 favorites]
Some time ago, I saw a definition of intelligence that appealed to me. It was, intelligence is the ability to deal with reality. Can you drive two miles in the city to get to the new grocery store. Can you find a container of cumin to buy? Can you find the place to pay for it? Can you perform the required transaction? Can you return to your car in the large, crowded parking lot? Can you stop and get gas on the way home? Etc. With all the unplanned for occurrences and events that happen...
I’ve been hanging out with a friend’s kid from birth to now being six years old. It’s been amazing to watch his mind grow, seeing the advancements and the mistakes happen. Given my background in language and philosophy, it’s been fun to try to understand what is going on.
A few comments above have used or mentioned “levels of abstraction” and I think these are crucial in understanding intelligence and how it advances and fails.
I show the kid a picture. That’s a cat. I show him a picture of a lion and tell him that’s a big cat. Then with a couple more examples, he’s starting to recognize cats of all kinds. Unlike machine learning systems that require thousands of pictures just to recognize a domestic cat, leave out lions, and ocelots. The power of the human brain to abstract “catness” from a few examples allows this to happen.
If you hold a pencil and ask what is this? From a physical point of view, you’re looking at something composed of a jillion tiny moving things you can’t see or experience, that is in relation with a jillion big things in its surroundings. Plus it has a history. Where did it come from? What about the tree out which part of it was made? And the carbon in the lead? What stars did it come from? And so on. A pencil is an infinity of properties for all intents and purposes. As is everything else. Abstraction allows us to deal with this extreme complexity by giving us the ability to select a much smaller subset of properties to create a much more manageable and thus more useful object. “Pencil” is that abstract object. Can’t find the damn pencil, then use the “pen” sitting there. We are now in a higher order abstraction called “writing instruments.”
Our minds are busy moving up and down these levels of abstractions, jumping into new branches of abstraction, and thus we deal with reality. These huge networks of abstraction function like maps laid down upon our experience that makes our experience meaningful. It’s a pencil! And if our minds have been performing well, then we probably have decent maps and we thus deal with reality. But, these maps are not the territory. Every time we abstract, we are leaving out details and it is possible to leave out something that is important later on. And depending on what details we kept, you might mistake a cat for a dog. Four legs, two eyes, fur...
Mistakes happen when we confuse levels of abstraction and when we forget that all levels of abstraction only exist in our heads, and that my “pencil” may not be equivalent to your “pencil.” And we may apply the wrong map.
All of these AI things seem to neglect the fact that we do not understand how our minds operate. Some like to posit that our brains are just wet computers with data storage and processors. Naive at best. This article goes into that confusion of levels of abstraction. How can we claim to be modeling something for which we don’t have a model? Neural networks? Well, neurons seem to do this. I can model that in code. Time passes... The computer prints out IT IS A CAT. Intelligence! We can’t even agree on what intelligence is, so how do we know? As to that cost thing talked about above, there are a whole bunch of things out there in the world that the three pound blob of stuff in my skull handles quite well with low power requirements, not too bad of an impact on the environment (depending on how well it knows about avoiding impact), and general reliability.
posted by njohnson23 at 11:06 AM on April 4, 2021 [1 favorite]
I’ve been hanging out with a friend’s kid from birth to now being six years old. It’s been amazing to watch his mind grow, seeing the advancements and the mistakes happen. Given my background in language and philosophy, it’s been fun to try to understand what is going on.
A few comments above have used or mentioned “levels of abstraction” and I think these are crucial in understanding intelligence and how it advances and fails.
I show the kid a picture. That’s a cat. I show him a picture of a lion and tell him that’s a big cat. Then with a couple more examples, he’s starting to recognize cats of all kinds. Unlike machine learning systems that require thousands of pictures just to recognize a domestic cat, leave out lions, and ocelots. The power of the human brain to abstract “catness” from a few examples allows this to happen.
If you hold a pencil and ask what is this? From a physical point of view, you’re looking at something composed of a jillion tiny moving things you can’t see or experience, that is in relation with a jillion big things in its surroundings. Plus it has a history. Where did it come from? What about the tree out which part of it was made? And the carbon in the lead? What stars did it come from? And so on. A pencil is an infinity of properties for all intents and purposes. As is everything else. Abstraction allows us to deal with this extreme complexity by giving us the ability to select a much smaller subset of properties to create a much more manageable and thus more useful object. “Pencil” is that abstract object. Can’t find the damn pencil, then use the “pen” sitting there. We are now in a higher order abstraction called “writing instruments.”
Our minds are busy moving up and down these levels of abstractions, jumping into new branches of abstraction, and thus we deal with reality. These huge networks of abstraction function like maps laid down upon our experience that makes our experience meaningful. It’s a pencil! And if our minds have been performing well, then we probably have decent maps and we thus deal with reality. But, these maps are not the territory. Every time we abstract, we are leaving out details and it is possible to leave out something that is important later on. And depending on what details we kept, you might mistake a cat for a dog. Four legs, two eyes, fur...
Mistakes happen when we confuse levels of abstraction and when we forget that all levels of abstraction only exist in our heads, and that my “pencil” may not be equivalent to your “pencil.” And we may apply the wrong map.
All of these AI things seem to neglect the fact that we do not understand how our minds operate. Some like to posit that our brains are just wet computers with data storage and processors. Naive at best. This article goes into that confusion of levels of abstraction. How can we claim to be modeling something for which we don’t have a model? Neural networks? Well, neurons seem to do this. I can model that in code. Time passes... The computer prints out IT IS A CAT. Intelligence! We can’t even agree on what intelligence is, so how do we know? As to that cost thing talked about above, there are a whole bunch of things out there in the world that the three pound blob of stuff in my skull handles quite well with low power requirements, not too bad of an impact on the environment (depending on how well it knows about avoiding impact), and general reliability.
posted by njohnson23 at 11:06 AM on April 4, 2021 [1 favorite]
I brought up GPT-3, because there are a number of comments in this thread that claim ML is only functional in game-like environments, like so:
'The limitation of these things is they only play games. Extending them to complicated real world problems is hard. Training in anything like common sense knowledge or adaptability to random inputs and circumstances is well beyond the state of the art.'
Here's a blog post in which GPT-3 does a good job on a number of common sense questions. As with chess-bots, there are machine-specific strategies you can take to get it to give bad answers; I tend to think that adding some GAN training objective will help cover up a number of these kinds of mistakes. Gwern also has some nice examples of tweaking prompts to get better handling of uncertainty and nonsense questions. I think we can probably agree that the 'naive' Turing test (where the user doesn't know they're performing a Turing test as they ask the questions) is basically toast at this point, and turn our attention to arguing about whether non-naive Turing tests actually matter.
Calling GPT-3 'just a game' because it's got a specific mechanism for training is, again, moving the goalposts basically as fast as one can run with them. General text synthesis is a wide-open 'play' space, and the model is doing just fine.
An analogy that comes to mind is how we've answered the fundamental question of physics: 'How does the world work?' One one level, we have an extensive set of answers, that fully describe just about everything we as humans are capable of observing (at least below the complexity level at which we start classifying the questions as chemistry or biology). One another level, we know that our answers are made-up math systems which might only be true-enough, and that the 'real' answers are basically unknowable. So we've seen steady incremental progress that answers the big question thoroughly, simultaneous with a second more philosophical perspective in which the question can never be satisfactorily answered. At the end of it all, the philosophers are never satisfied, but the blue-collar physicist can say 'we have cameras.'
What we're seeing with Alpha-Go and GPT-3 and Waymo are really important strides forward on the practical side of the fundamental question of AI: 'How do we make intelligent machines?' Again, if we let the goal posts move arbitrarily at the whim of the asker, we're never going to have an answer to the question. But we already have smart cameras. And they're not going to disappear because of our disagreements over Turing tests.
We will be continually supplemented, though I suspect, not supplanted.
This I agree with a hundred percent. Technologies bridge a gap between human desires and an uncaring universe, and typically involve some adaptations on both sides. Bicycles are useless at getting you places... Until you've a) taken the time to learn to ride them (humans adapting to the technology), and b) built roads (adapting the world to the technology). Neil Lawrence notes that with AI people tend to expect the technology to adapt to us instead of the other way around, which is pretty unprecedented, and thus unrealistic.
The currently-insurmountable problem for GPT-3 is that it can only know what it's seen in text: It has no embodied understanding of the world. If you're using GPT-3 as a writing tool, that can be totally fine: you just learn to use it in ways that don't trip it up. And that's OK! If you mis-use a bicycle, or just about any tool, it breaks. That doesn't mean it's useless, just that it has limitations. That's the world we live in.
For ML, we're generally still in a space of figuring out where the limitations are, and figuring out how best to accommodate them. The limitations vary by problem, and are often pretty 'squishy.' But there are many well-bounded problems where there are ML-based tools already deployed, used by millions and even billions of people. The core question is 'How do we build intelligent machines?' We haven't finished answering the practical version of the question, but rapid progress really is happening. Give it a hundred years, and my guess is we'll be done with the practical question, and still arguing about the philosophical one.
(and we'll have even better cameras.)
posted by kaibutsu at 11:08 AM on April 4, 2021 [1 favorite]
'The limitation of these things is they only play games. Extending them to complicated real world problems is hard. Training in anything like common sense knowledge or adaptability to random inputs and circumstances is well beyond the state of the art.'
Here's a blog post in which GPT-3 does a good job on a number of common sense questions. As with chess-bots, there are machine-specific strategies you can take to get it to give bad answers; I tend to think that adding some GAN training objective will help cover up a number of these kinds of mistakes. Gwern also has some nice examples of tweaking prompts to get better handling of uncertainty and nonsense questions. I think we can probably agree that the 'naive' Turing test (where the user doesn't know they're performing a Turing test as they ask the questions) is basically toast at this point, and turn our attention to arguing about whether non-naive Turing tests actually matter.
Calling GPT-3 'just a game' because it's got a specific mechanism for training is, again, moving the goalposts basically as fast as one can run with them. General text synthesis is a wide-open 'play' space, and the model is doing just fine.
An analogy that comes to mind is how we've answered the fundamental question of physics: 'How does the world work?' One one level, we have an extensive set of answers, that fully describe just about everything we as humans are capable of observing (at least below the complexity level at which we start classifying the questions as chemistry or biology). One another level, we know that our answers are made-up math systems which might only be true-enough, and that the 'real' answers are basically unknowable. So we've seen steady incremental progress that answers the big question thoroughly, simultaneous with a second more philosophical perspective in which the question can never be satisfactorily answered. At the end of it all, the philosophers are never satisfied, but the blue-collar physicist can say 'we have cameras.'
What we're seeing with Alpha-Go and GPT-3 and Waymo are really important strides forward on the practical side of the fundamental question of AI: 'How do we make intelligent machines?' Again, if we let the goal posts move arbitrarily at the whim of the asker, we're never going to have an answer to the question. But we already have smart cameras. And they're not going to disappear because of our disagreements over Turing tests.
We will be continually supplemented, though I suspect, not supplanted.
This I agree with a hundred percent. Technologies bridge a gap between human desires and an uncaring universe, and typically involve some adaptations on both sides. Bicycles are useless at getting you places... Until you've a) taken the time to learn to ride them (humans adapting to the technology), and b) built roads (adapting the world to the technology). Neil Lawrence notes that with AI people tend to expect the technology to adapt to us instead of the other way around, which is pretty unprecedented, and thus unrealistic.
The currently-insurmountable problem for GPT-3 is that it can only know what it's seen in text: It has no embodied understanding of the world. If you're using GPT-3 as a writing tool, that can be totally fine: you just learn to use it in ways that don't trip it up. And that's OK! If you mis-use a bicycle, or just about any tool, it breaks. That doesn't mean it's useless, just that it has limitations. That's the world we live in.
For ML, we're generally still in a space of figuring out where the limitations are, and figuring out how best to accommodate them. The limitations vary by problem, and are often pretty 'squishy.' But there are many well-bounded problems where there are ML-based tools already deployed, used by millions and even billions of people. The core question is 'How do we build intelligent machines?' We haven't finished answering the practical version of the question, but rapid progress really is happening. Give it a hundred years, and my guess is we'll be done with the practical question, and still arguing about the philosophical one.
(and we'll have even better cameras.)
posted by kaibutsu at 11:08 AM on April 4, 2021 [1 favorite]
intelligence is the ability to deal with reality
So... plants are intelligent? The sun seems to do a pretty good job at this goal, too? The universe as a whole is reality - does that make it intelligent?
posted by eviemath at 11:40 AM on April 4, 2021 [1 favorite]
So... plants are intelligent? The sun seems to do a pretty good job at this goal, too? The universe as a whole is reality - does that make it intelligent?
posted by eviemath at 11:40 AM on April 4, 2021 [1 favorite]
(I mean, that's obviously not what you meant from the rest of the examples! But well-illustrates the difficulty in defining intelligence.)
posted by eviemath at 11:42 AM on April 4, 2021 [2 favorites]
posted by eviemath at 11:42 AM on April 4, 2021 [2 favorites]
The currently-insurmountable problem for GPT-3 is that it can only know what it's seen in text: It has no embodied understanding of the world.
Now to get really picky... we have three terms here - know, see, understand - all properties of human intelligence applied to a computer program, one which seems to be nothing more than an elaborate Markov chain text generator based on a statistical analysis of word adjacency occurrences. In what sense, is anything known, seen, or understood? Does a word processor know what you are writing about? The irritating text completion code on my iPad, seems to be attempting to understand my literary intent, but it is obvious that it’s just statistically based guesses. No understanding. We continue to map our current understanding of the human mind onto software and in talking about the software and then confusing the levels of abstraction. Yeah, it makes it easier to talk about this software, but we are not necessarily giving an accurate description of what is going on. To resort to psychology, it’s closer to projection than it is to understanding.
posted by njohnson23 at 12:10 PM on April 4, 2021 [3 favorites]
Now to get really picky... we have three terms here - know, see, understand - all properties of human intelligence applied to a computer program, one which seems to be nothing more than an elaborate Markov chain text generator based on a statistical analysis of word adjacency occurrences. In what sense, is anything known, seen, or understood? Does a word processor know what you are writing about? The irritating text completion code on my iPad, seems to be attempting to understand my literary intent, but it is obvious that it’s just statistically based guesses. No understanding. We continue to map our current understanding of the human mind onto software and in talking about the software and then confusing the levels of abstraction. Yeah, it makes it easier to talk about this software, but we are not necessarily giving an accurate description of what is going on. To resort to psychology, it’s closer to projection than it is to understanding.
posted by njohnson23 at 12:10 PM on April 4, 2021 [3 favorites]
Mistakes happen when we confuse levels of abstraction and when we forget that all levels of abstraction only exist in our heads, and that my “pencil” may not be equivalent to your “pencil.” And we may apply the wrong map.
My own thinking on this has been more focused on things in general than abstraction in particular.
Abstraction, it seems to me, is an act of inclusion: this thing is the same kind of thing as those things. And that kind-of relationship functions more like tagging than categorizing: a lion is simultaneously kind-of cat and kind-of things-bigger-than-me.
What's been more interesting to me lately is the process of thingification - the act of identifying the raw materials upon which to perform reasoning and intuition and instinct. This is an act of distinction. A thing, it seems to me, is just exactly that for which we can apply at least one criterion for distinguishing it from that which it is not.
It seems to me that your point about levels of abstraction existing only in our heads applies equally to things in general. Rather than a pencil being an infinity of properties for all intents and purposes, it seems to me closer to the mark to say that we can identify many™ properties related to pencils; each of those properties is a thing in its own right, with its own criteria for distinguishing it from that which it is not. The pencil, meanwhile, remains only and exactly itself.
The way I see it, the pencil cannot, in and of itself, definitively be said to be made of anything at all; made-of is a relationship between things, and things are map, not territory. We can identify and reach for and pick up and write with a pencil without needing to form any opinion whatsoever about what it's made of or what, if any, its "true" nature might be. Thingification is of practical use far more often than of theoretical use.
Ontological disagreements, then, will usually come down to differences in the distinction criteria that each of us is using to distinguish things from what they are not. Those of us with a background in physics have things like electrons and fields available to play is-made-of games with about pencils; those with a background in woodwork have things like cedar and glue and paint. Which of these is-made-ofs has more explanatory and predictive power will depend entirely on the questions about pencils whose answers we're interested in. And if the problem I'm trying to solve is locating a for-writing thing, is-made-of is just not a relationship I'll be interested in exploring at all.
What makes neural networks so ridiculously hard even to attempt to prove correct is that their very architecture smears their thingification distinction criteria across entire networks in ways that make them damn near impossible to pin down. We can't actually peer inside a neural network and know what it's using to distinguish pencils from non-pencils. All we can test is how well its thingification of inputs has agreed with our own until now. And as they say in the financial markets, past performance is no guarantee of future results.
posted by flabdablet at 1:03 PM on April 4, 2021 [1 favorite]
My own thinking on this has been more focused on things in general than abstraction in particular.
Abstraction, it seems to me, is an act of inclusion: this thing is the same kind of thing as those things. And that kind-of relationship functions more like tagging than categorizing: a lion is simultaneously kind-of cat and kind-of things-bigger-than-me.
What's been more interesting to me lately is the process of thingification - the act of identifying the raw materials upon which to perform reasoning and intuition and instinct. This is an act of distinction. A thing, it seems to me, is just exactly that for which we can apply at least one criterion for distinguishing it from that which it is not.
It seems to me that your point about levels of abstraction existing only in our heads applies equally to things in general. Rather than a pencil being an infinity of properties for all intents and purposes, it seems to me closer to the mark to say that we can identify many™ properties related to pencils; each of those properties is a thing in its own right, with its own criteria for distinguishing it from that which it is not. The pencil, meanwhile, remains only and exactly itself.
The way I see it, the pencil cannot, in and of itself, definitively be said to be made of anything at all; made-of is a relationship between things, and things are map, not territory. We can identify and reach for and pick up and write with a pencil without needing to form any opinion whatsoever about what it's made of or what, if any, its "true" nature might be. Thingification is of practical use far more often than of theoretical use.
Ontological disagreements, then, will usually come down to differences in the distinction criteria that each of us is using to distinguish things from what they are not. Those of us with a background in physics have things like electrons and fields available to play is-made-of games with about pencils; those with a background in woodwork have things like cedar and glue and paint. Which of these is-made-ofs has more explanatory and predictive power will depend entirely on the questions about pencils whose answers we're interested in. And if the problem I'm trying to solve is locating a for-writing thing, is-made-of is just not a relationship I'll be interested in exploring at all.
What makes neural networks so ridiculously hard even to attempt to prove correct is that their very architecture smears their thingification distinction criteria across entire networks in ways that make them damn near impossible to pin down. We can't actually peer inside a neural network and know what it's using to distinguish pencils from non-pencils. All we can test is how well its thingification of inputs has agreed with our own until now. And as they say in the financial markets, past performance is no guarantee of future results.
posted by flabdablet at 1:03 PM on April 4, 2021 [1 favorite]
@kaibutsu I'm an applier not a researcher of ML, but it looks to me that changes in training framework have been critical, such as the development of adversarial training, so my bet is a more 'open' problem than supervised training pairs would be a path towards more general intelligence. Deep reinforcement learning maybe?
(Aaand I think you'd better train for cooperation or you're going to get super-octopuses.)
HOWEVER, hold up, back to Chiang: should we attempt this? And who and what is being served if we do -- money and Elon Musk's ego? I don't know that it's ethical to attempt general intelligence past where it's demonstrably needed in solving a problem that benefits humans or other creatures enough to weigh the trade-off.
If you present an open world with unclear goals, you're training for some analog of volition and motivation. If you present long time scales, you're training for some reflection in the model of its past and future. I share Chiang's concern that this may lead to suffering before it leads to useful general intelligence, and at that point who are you using?
posted by away for regrooving at 1:04 PM on April 4, 2021
(Aaand I think you'd better train for cooperation or you're going to get super-octopuses.)
HOWEVER, hold up, back to Chiang: should we attempt this? And who and what is being served if we do -- money and Elon Musk's ego? I don't know that it's ethical to attempt general intelligence past where it's demonstrably needed in solving a problem that benefits humans or other creatures enough to weigh the trade-off.
If you present an open world with unclear goals, you're training for some analog of volition and motivation. If you present long time scales, you're training for some reflection in the model of its past and future. I share Chiang's concern that this may lead to suffering before it leads to useful general intelligence, and at that point who are you using?
posted by away for regrooving at 1:04 PM on April 4, 2021
Just thinking, there is more to this than capitalism. The personal motivation to create general AI is I think for many practitioners the lure of the alchemical homunculus, of being the medieval wizard creating life.
But then they're working within a system of money breeding money and power breeding power, which controls how the work is applied and captures the money and power it can yield.
posted by away for regrooving at 1:19 PM on April 4, 2021 [2 favorites]
But then they're working within a system of money breeding money and power breeding power, which controls how the work is applied and captures the money and power it can yield.
posted by away for regrooving at 1:19 PM on April 4, 2021 [2 favorites]
Now to get really picky... we have three terms here - know, see, understand - all properties of human intelligence applied to a computer program
I actually don't care about defining those words. There's a specific set of problems I'm describing - inability to handle complex state describing a physical system consistently - and fell back on some bogeyman words instead of writing it out plainly. Sorry.
Seeing that the problem exists, we can formulate hypotheses about why the system is having a problem (eg, has never experienced any actual physics) and start formulating ways to try to address it (eg, include some video caption training objective as part of the training process). These hypotheses and proposed solutions don't require defining the bogeyman words, and there's no reason to doubt that consistent effort won't allow us to improve the situation.
FWIW, I think the AI Dungeon folks are on an interesting path, here: A good game requires combining the text generator with consistent handling of complex state.
There's all sorts of reasons to believe this is very doable. We have lots of examples of 'conditioned' generative systems. You can see the generator as 'just' a markov chain, but it starts getting interesting when you introduce conditioning, which says you want to produce generative output given a certain context. Examples include image captioning (generate text describing this picture) and text-to-speech (produce realistic human speech from this block of text, and why not make it sound like David Attenborough while we're at it). We already have models that are very good at these problems.
The AI Dungeon problem is then to keep track of some state from the text chain, and produce new output conditioned on the retained state. The state might be an interesting combination of hard-coded game state like hit points and a list of monsters in the room with 'softer' state like remembering which NPCs exist and what their relationships to the player are. None of this strikes me as unsolvable, though the result will inevitably be imperfect... And then we can start finding ways to smooth out those imperfections.
posted by kaibutsu at 1:48 PM on April 4, 2021 [1 favorite]
I actually don't care about defining those words. There's a specific set of problems I'm describing - inability to handle complex state describing a physical system consistently - and fell back on some bogeyman words instead of writing it out plainly. Sorry.
Seeing that the problem exists, we can formulate hypotheses about why the system is having a problem (eg, has never experienced any actual physics) and start formulating ways to try to address it (eg, include some video caption training objective as part of the training process). These hypotheses and proposed solutions don't require defining the bogeyman words, and there's no reason to doubt that consistent effort won't allow us to improve the situation.
FWIW, I think the AI Dungeon folks are on an interesting path, here: A good game requires combining the text generator with consistent handling of complex state.
There's all sorts of reasons to believe this is very doable. We have lots of examples of 'conditioned' generative systems. You can see the generator as 'just' a markov chain, but it starts getting interesting when you introduce conditioning, which says you want to produce generative output given a certain context. Examples include image captioning (generate text describing this picture) and text-to-speech (produce realistic human speech from this block of text, and why not make it sound like David Attenborough while we're at it). We already have models that are very good at these problems.
The AI Dungeon problem is then to keep track of some state from the text chain, and produce new output conditioned on the retained state. The state might be an interesting combination of hard-coded game state like hit points and a list of monsters in the room with 'softer' state like remembering which NPCs exist and what their relationships to the player are. None of this strikes me as unsolvable, though the result will inevitably be imperfect... And then we can start finding ways to smooth out those imperfections.
posted by kaibutsu at 1:48 PM on April 4, 2021 [1 favorite]
I have created massive generative grammars with large vocabularies, that create quite complex sentences through random selection. But each is tailored for one kind of discourse. One is devoted to German philosophy from Kant to Heidegger. In the course of my efforts I have looked at trying to create a general English text generator. But here whole levels of complexity are introduced such as person, number, and tense. These become properties of a grammar state for a sentence that then propagates through the sequence of sentences, tagged to the subject and object, etc in order to introduce pronouns etc. And then there are the verbs. After a few years of looking at these problems coupled with other distractions from reality, I pretty much gave up. Generating general, grammatically correct English that can stick to a topic using generative grammars with random selection of components ain’t simple. Keeping track of the state and then modifying nouns and verbs based on the state all within English’s interesting grammar and spelling rules was beyond my patience and probably skill. So, I am left studying the words spoken by my artificial German philosopher for any insights I might gain.
posted by njohnson23 at 2:14 PM on April 4, 2021 [1 favorite]
posted by njohnson23 at 2:14 PM on April 4, 2021 [1 favorite]
Calling GPT-3 'just a game' because it's got a specific mechanism for training is, again, moving the goalposts basically as fast as one can run with them. General text synthesis is a wide-open 'play' space, and the model is doing just fine.
So I appreciate your comments, and I see where you're coming from. By saying cross-entropy minimization is a game though, I'm trying to ask if regression/pattern recognition/DNNs are broad enough to capture much/most of what we'd call intelligence. Maybe they are! But personally I'm somewhat skeptical, and I think we'll need tools with different theoretical foundations to complete the picture. Fundamentally though I agree - the limits are poorly understood. How many problems can you really solve with big data?
I think self-driving cars remains an interesting case to follow. There's huge economic incentive to get them off the ground, and for a while I (along with many) thought we'd make rapid progress and we'd be seeing widespread deployment around... now, I guess. But major challenges there have yet to be surmounted (at least last time I checked). Maybe DNN techniques will eventually overcome these challenges. Even if they don't though, the attempt will help us better understand the extent to which pattern recognition can "solve the world".
posted by Alex404 at 2:38 PM on April 4, 2021 [1 favorite]
So I appreciate your comments, and I see where you're coming from. By saying cross-entropy minimization is a game though, I'm trying to ask if regression/pattern recognition/DNNs are broad enough to capture much/most of what we'd call intelligence. Maybe they are! But personally I'm somewhat skeptical, and I think we'll need tools with different theoretical foundations to complete the picture. Fundamentally though I agree - the limits are poorly understood. How many problems can you really solve with big data?
I think self-driving cars remains an interesting case to follow. There's huge economic incentive to get them off the ground, and for a while I (along with many) thought we'd make rapid progress and we'd be seeing widespread deployment around... now, I guess. But major challenges there have yet to be surmounted (at least last time I checked). Maybe DNN techniques will eventually overcome these challenges. Even if they don't though, the attempt will help us better understand the extent to which pattern recognition can "solve the world".
posted by Alex404 at 2:38 PM on April 4, 2021 [1 favorite]
I don't drive, so apparently I'm not intelligent?
posted by rum-soaked space hobo at 3:08 PM on April 4, 2021
posted by rum-soaked space hobo at 3:08 PM on April 4, 2021
I don't drive, so apparently I'm not intelligent?
As was referenced in the Brat Pack thread, this is exactly backwards.
posted by thelonius at 3:44 PM on April 4, 2021 [3 favorites]
As was referenced in the Brat Pack thread, this is exactly backwards.
posted by thelonius at 3:44 PM on April 4, 2021 [3 favorites]
nor did they give full detail about their own qualifications in their appeal to authority argument.
I never used an appeal to authority argument, and I totally resent this accusation. Suggesting, in response to David Chiang's argument, that the answer to curing cancer requires some concept of genetics, is criticism of a conceptual approach, but not an argument to authority. That is like, a massive distinction. Maybe AI practicioners don't like AI theorists criticizing the current state of lack of theory, but that is purely intellectual criticism. Was Chomsky appealing to authority when he critiqued Norvig? No.
3. As a practitioner, complexity theory mostly seems to be completely missing the real questions. When I see a new problem, theres no complexity theory answer / 4. Finally, I'm seeing more and more examples lately where the actual model architecture barely matters, so long as it's reasonably modern and large enough
That's because the state of the science is that contemporary AI doesn't have a good theory connecting back to foundations of computing. There's ongoing research on this. Don't dismiss that research. And besides, it's a different issue than the article. Talking about the physical limits of the computation universe requires theory in a way that building a computer doesn't require theory. We don't worry too much about the complexity class of an actual Intel CPU. But the science fiction assertions that human-level or super-level AI is possible, or not possible, requires theory. And theory means rigorous conceptualization of universal computation, of the equivalence of models of computation, etc.
I'm lazy, I'll quote myself from the other thread:
"Factoring large numbers is in PSPACE, way way inside PSPACE. Yet we are not cracking RSA by reduction to AlphaGo. Complexity theory is anti-relevant here. We have not even solved Go in the formal sense at all..."
Factoring is NP and we are not using SAT solvers to generate digital certificates. But the advances with SAT solvers were a huge deal, and everyone knows theorywise that SAT is NP-complete. We don't say NP-completeness is irrelevant to SAT.
Go is PSPACE-complete. So that's why theory matters. This recognition is why AI research and interest these few years has been buoyed by the public success of AlphaGo; that much is undeniable.
posted by polymodus at 5:34 PM on April 4, 2021
I never used an appeal to authority argument, and I totally resent this accusation. Suggesting, in response to David Chiang's argument, that the answer to curing cancer requires some concept of genetics, is criticism of a conceptual approach, but not an argument to authority. That is like, a massive distinction. Maybe AI practicioners don't like AI theorists criticizing the current state of lack of theory, but that is purely intellectual criticism. Was Chomsky appealing to authority when he critiqued Norvig? No.
3. As a practitioner, complexity theory mostly seems to be completely missing the real questions. When I see a new problem, theres no complexity theory answer / 4. Finally, I'm seeing more and more examples lately where the actual model architecture barely matters, so long as it's reasonably modern and large enough
That's because the state of the science is that contemporary AI doesn't have a good theory connecting back to foundations of computing. There's ongoing research on this. Don't dismiss that research. And besides, it's a different issue than the article. Talking about the physical limits of the computation universe requires theory in a way that building a computer doesn't require theory. We don't worry too much about the complexity class of an actual Intel CPU. But the science fiction assertions that human-level or super-level AI is possible, or not possible, requires theory. And theory means rigorous conceptualization of universal computation, of the equivalence of models of computation, etc.
I'm lazy, I'll quote myself from the other thread:
"Factoring large numbers is in PSPACE, way way inside PSPACE. Yet we are not cracking RSA by reduction to AlphaGo. Complexity theory is anti-relevant here. We have not even solved Go in the formal sense at all..."
Factoring is NP and we are not using SAT solvers to generate digital certificates. But the advances with SAT solvers were a huge deal, and everyone knows theorywise that SAT is NP-complete. We don't say NP-completeness is irrelevant to SAT.
Go is PSPACE-complete. So that's why theory matters. This recognition is why AI research and interest these few years has been buoyed by the public success of AlphaGo; that much is undeniable.
posted by polymodus at 5:34 PM on April 4, 2021
And, to back that up further, since general robotics is by folk consenus is also PSPACE-complete (I spent the time last week looking at a dozen papers on automatic cars, and they all repeat this like a slogan), it really is that class that is of basic interest to scientists. It is not correct to suggest there is some factoring or graph isomorphism analogue (i.e., problems that are on the magic tractability boundary between P and NP) that robotics problems are like with respect to PSPACE. That is not what the current folk consensus in those papers is saying.
Of course no one is saying theories cannot be refined. There are specialized classes within the complexity zoo specifically for neural networks. And there's going to be further research on that, with theoretical CS itself continually making advances and breakthroughs.
The idea that we can answer an abstract, scientific question about super human AIs without reference to the core, formal and philosophical theories of computer science strikes me as highly problematic.
posted by polymodus at 5:52 PM on April 4, 2021
Of course no one is saying theories cannot be refined. There are specialized classes within the complexity zoo specifically for neural networks. And there's going to be further research on that, with theoretical CS itself continually making advances and breakthroughs.
The idea that we can answer an abstract, scientific question about super human AIs without reference to the core, formal and philosophical theories of computer science strikes me as highly problematic.
posted by polymodus at 5:52 PM on April 4, 2021
I'll try from a different angle: read the reinforcement learning blog post I linked and see what you think. It covers what makes board games particularly amenable to AlphaZero's use of RL, and why many other problems, despite being all in PSPACE, are harder. The shape of the problem -- in RL, the existence of an effective reward function, and its tractability to hill-climbing -- are critical.
We know methods that solve useful real-world SAT problems don't crack RSA, they'll fail on relatively small SAT problems generated by that reduction. The complexity class -- which gets driven by the nastiest cases -- just doesn't speak to the relation between these two problems.
Humans don't have poly space available to solve large problems! We do motion planning quite well regardless. Until we hit one of those problems that actually makes motion planning be PSPACE and non-approximable (say for sake of argument, I don't know if that's known). The horrible ironmongery-and-rope puzzles.
posted by away for regrooving at 7:09 PM on April 4, 2021 [5 favorites]
We know methods that solve useful real-world SAT problems don't crack RSA, they'll fail on relatively small SAT problems generated by that reduction. The complexity class -- which gets driven by the nastiest cases -- just doesn't speak to the relation between these two problems.
Humans don't have poly space available to solve large problems! We do motion planning quite well regardless. Until we hit one of those problems that actually makes motion planning be PSPACE and non-approximable (say for sake of argument, I don't know if that's known). The horrible ironmongery-and-rope puzzles.
posted by away for regrooving at 7:09 PM on April 4, 2021 [5 favorites]
To be clear, I'm certainly not saying that classification of problems with respect to learnability is impossible. Definitely not my field, but it's being worked on. But PSPACE is not one of those classes, so please stop trying to use it like one?
posted by away for regrooving at 7:17 PM on April 4, 2021 [1 favorite]
posted by away for regrooving at 7:17 PM on April 4, 2021 [1 favorite]
First of all, let me say that I admire Ted Chiang--his SF writing is excellent, his opinions and sympathies are generally in alignment with mine, and he shows up and is generous with his time and insights with the SF community.
This New Yorker article really disappointed me, for a couple reasons. The biggest is that he writes, as what looks to me to be a summary argument, this: "...precisely why I don’t believe that running a human-equivalent A.I. program for a hundred years in isolation is a good way to produce major breakthroughs."
But this isn't what anybody in AI right now is doing, or proposing doing. And it's not anybody's definition of The Singularity either. It's a straw man that he created because his bootstrapping analogy works with it, and is simple enough that readers can grasp it and go "Ahh—!"
But the sense of his own certainty here, and the perhaps-unconscious misdirection of reader attention away from what is actually happening in the field (pick a field--theoretical computer science, current AI, neurological modeling) is really kind of appalling. Any reporter assigned to write an overview article about this field would have their work rejected if it didn't show any basic effort to talk to actual practitioners in the field or summarize the state of the art.
Somehow he got a bye, because his writing is lively and erudite-sounding and his topic is a hot one and not one that New Yorker editors know much about, and, I'm sorry to suggest, because his conclusion is a comforting one that New Yorker readers want to hear.
posted by newdaddy at 3:39 AM on April 5, 2021
This New Yorker article really disappointed me, for a couple reasons. The biggest is that he writes, as what looks to me to be a summary argument, this: "...precisely why I don’t believe that running a human-equivalent A.I. program for a hundred years in isolation is a good way to produce major breakthroughs."
But this isn't what anybody in AI right now is doing, or proposing doing. And it's not anybody's definition of The Singularity either. It's a straw man that he created because his bootstrapping analogy works with it, and is simple enough that readers can grasp it and go "Ahh—!"
But the sense of his own certainty here, and the perhaps-unconscious misdirection of reader attention away from what is actually happening in the field (pick a field--theoretical computer science, current AI, neurological modeling) is really kind of appalling. Any reporter assigned to write an overview article about this field would have their work rejected if it didn't show any basic effort to talk to actual practitioners in the field or summarize the state of the art.
Somehow he got a bye, because his writing is lively and erudite-sounding and his topic is a hot one and not one that New Yorker editors know much about, and, I'm sorry to suggest, because his conclusion is a comforting one that New Yorker readers want to hear.
posted by newdaddy at 3:39 AM on April 5, 2021
I think self-driving cars remains an interesting case to follow.
I do too. Because the problem a self-driving car needs to be able to solve is the very same problem an animal does: how to maintain its own integrity in an environment fairly heavily populated with previously unencountered threats.
Since nobody is designing cars with self-healing features as fundamental to their physical structure, that problem becomes even more difficult for the car than it is for the animal; almost any damage becomes unacceptable. This, in turn, means that learning mainly from its own failures is not a viable survival strategy. The car must be competent within its environment from the moment it's released into it. What's required to make that work is not so much artificial intelligence as artificial instinct.
And when the inevitable failures do occur, resulting in the destruction of cars or their occupants or (equivalently, from the car's point of view) damage to bystanders sufficient to have the cars pulled off the road, then in order to preserve their species the cars must learn from each other how to recognize and avoid impending failure conditions. So some kind of artificial culture is also going to be required, perhaps even to the extent of implementing an artificial hive mind.
These are all very difficult things to engineer, especially given the inbuilt resistance that so many engineers have so consistently displayed to taking the social sciences the slightest bit seriously.
posted by flabdablet at 4:15 AM on April 5, 2021 [5 favorites]
I do too. Because the problem a self-driving car needs to be able to solve is the very same problem an animal does: how to maintain its own integrity in an environment fairly heavily populated with previously unencountered threats.
Since nobody is designing cars with self-healing features as fundamental to their physical structure, that problem becomes even more difficult for the car than it is for the animal; almost any damage becomes unacceptable. This, in turn, means that learning mainly from its own failures is not a viable survival strategy. The car must be competent within its environment from the moment it's released into it. What's required to make that work is not so much artificial intelligence as artificial instinct.
And when the inevitable failures do occur, resulting in the destruction of cars or their occupants or (equivalently, from the car's point of view) damage to bystanders sufficient to have the cars pulled off the road, then in order to preserve their species the cars must learn from each other how to recognize and avoid impending failure conditions. So some kind of artificial culture is also going to be required, perhaps even to the extent of implementing an artificial hive mind.
These are all very difficult things to engineer, especially given the inbuilt resistance that so many engineers have so consistently displayed to taking the social sciences the slightest bit seriously.
posted by flabdablet at 4:15 AM on April 5, 2021 [5 favorites]
My other objection to Ted's article is that, while most things a writer might predict are more or less equal in gravity, there is an added responsibility involved in writing about something that could have a calamitous outcome and in reassuring readers that it will never happen. 'Oh, it will never flood in Houston—you don't have to worry about that!' I have no idea what the drainage situation is in Houston, but I would never say never unless I was in a position to absolutely know.
I think that the huge amount of money, talent and resources being poured into AI research now is a pretty good sign that it is at least likely that general AI will arrive soon, and so it's our obligation to think clearly about it and prepare for it.
posted by newdaddy at 4:37 AM on April 5, 2021 [1 favorite]
I think that the huge amount of money, talent and resources being poured into AI research now is a pretty good sign that it is at least likely that general AI will arrive soon, and so it's our obligation to think clearly about it and prepare for it.
posted by newdaddy at 4:37 AM on April 5, 2021 [1 favorite]
I never used an appeal to authority argument, and I totally resent this accusation.
And I resent intellectual dick-waving, but if that's where you want to go, then get out an actual ruler and tell us your measurements: PhD or no? Subject area? Number of publications and impact factor, or conference presentations and equivalent measure to impact factor if that's a thing in your field? The other week you were an expert in critical theory or something, and that's an unusual dual expertise area, so perhaps, given your current expertise this week, you'll understand my need for some evidence here.
Or, you know, we could accept that Metafilter is not the comments period after an academic talk in some cutthroat macho department and try to not be gate-keepy about who should be allowed to comment on fpps. Or not require other members of our Metafilter community who may want to remain semi-anonymous to give their full CV before we deign to engage with their comments or critiques on the merits of the actual comments or critiques.
To be clear here: the first paragraph of this comment was rhetorical and I don't actually want your CV details.
I understand your frustration, polymodus. I've gotten annoyed with comments in some threads on Metafilter that seem to be trying to mansplain to me things that I obviously-to-me know far better than the other commenters (to the best of my knowledge, of course). There's a balance to be struck in pushing back against that without becoming it yourself, however. Finding that balance involves keeping an open mind about the possibility of others' expertise even if they haven't given credentials that you recognize, some self-reflection to ensure that proving your own smarts is not one of your implicit goals (if you've been marinating in a certain commonly found variant of academic culture, as is sounds from some of your comments like you have, you may have picked some of those behaviors up unintentionally even though you also oppose them and recognize their harm when used by others), and maybe picking up some more background in education so that you know how to engage with people's ideas without coming across as just name-dropping to build yourself up (also known, more colorfully, as intellectual dick-waving). (It sounds like you're still a grad student? Perhaps a postdoc? Which would mean you're at a larger university with more extensive grad programs? Then your university likely has a teaching and learning center that runs some sort of teaching certificate program for grad students, or has professional development workshops for postdocs and early career faculty.)
It's okay to ask people to comment from a more informed perspective. I've done that myself. The key is to ask, to acknowledge that it's a significant ask, and to provide, in a respectful manner, some resources to help people meet your request. To permute the song lyrics, it's not too much to ask, but it's too much to expect.
posted by eviemath at 4:39 AM on April 5, 2021 [6 favorites]
And I resent intellectual dick-waving, but if that's where you want to go, then get out an actual ruler and tell us your measurements: PhD or no? Subject area? Number of publications and impact factor, or conference presentations and equivalent measure to impact factor if that's a thing in your field? The other week you were an expert in critical theory or something, and that's an unusual dual expertise area, so perhaps, given your current expertise this week, you'll understand my need for some evidence here.
Or, you know, we could accept that Metafilter is not the comments period after an academic talk in some cutthroat macho department and try to not be gate-keepy about who should be allowed to comment on fpps. Or not require other members of our Metafilter community who may want to remain semi-anonymous to give their full CV before we deign to engage with their comments or critiques on the merits of the actual comments or critiques.
To be clear here: the first paragraph of this comment was rhetorical and I don't actually want your CV details.
I understand your frustration, polymodus. I've gotten annoyed with comments in some threads on Metafilter that seem to be trying to mansplain to me things that I obviously-to-me know far better than the other commenters (to the best of my knowledge, of course). There's a balance to be struck in pushing back against that without becoming it yourself, however. Finding that balance involves keeping an open mind about the possibility of others' expertise even if they haven't given credentials that you recognize, some self-reflection to ensure that proving your own smarts is not one of your implicit goals (if you've been marinating in a certain commonly found variant of academic culture, as is sounds from some of your comments like you have, you may have picked some of those behaviors up unintentionally even though you also oppose them and recognize their harm when used by others), and maybe picking up some more background in education so that you know how to engage with people's ideas without coming across as just name-dropping to build yourself up (also known, more colorfully, as intellectual dick-waving). (It sounds like you're still a grad student? Perhaps a postdoc? Which would mean you're at a larger university with more extensive grad programs? Then your university likely has a teaching and learning center that runs some sort of teaching certificate program for grad students, or has professional development workshops for postdocs and early career faculty.)
It's okay to ask people to comment from a more informed perspective. I've done that myself. The key is to ask, to acknowledge that it's a significant ask, and to provide, in a respectful manner, some resources to help people meet your request. To permute the song lyrics, it's not too much to ask, but it's too much to expect.
posted by eviemath at 4:39 AM on April 5, 2021 [6 favorites]
(And, polymodus, if you are just now thinking "What f'ing song lyric? What sort of music snob hipster does she think she is, dropping some obscure lyric as if obviously folks would be familiar with it, or else just aren't cool enough to engage with?" That's part of my point. It's annoying to others when you do that, too.)
posted by eviemath at 4:52 AM on April 5, 2021 [1 favorite]
posted by eviemath at 4:52 AM on April 5, 2021 [1 favorite]
The articles are OK but don’t display the originality or insight of his really excellent stories.
posted by Phanx at 7:40 AM on April 5, 2021
posted by Phanx at 7:40 AM on April 5, 2021
Nick Dyer-Witheford has done a lot of work on biocommunist critique of global state capitalism. On the specific subject of AI, see Inhuman power: artificial intelligence and the future of capitalism, in which he identifies Nick Land as the most important spokesman for ‘accelerationist’ thinking:
posted by No Robots at 12:23 PM on April 5, 2021 [5 favorites]
AI, in Land’s view, is not merely appropriated by capital, but constituted by it: it is a technology made from and for its processes of labour automation, commodity acceleration and financial speculation. A second, yet more disquieting Landian point, is that this mutual embedment of capital and AI leads not to human emancipation from capitalism, but, on the contrary, to capital’s emancipation from the human: a capital that no longer needs homo sapiens; human extinction. These are not comfortable thoughts. And they are made even less comfortable by the fact that Land in his recent writings has emerged as a reactionary champion of racist and misogynist ‘dark enlightenment’ ideas that have in complex ways infiltrated the culture of Silicon Valley where much AI production takes place. While we emphatically disas-sociate ourselves from this aspect of Landian thought, we nonetheless believe that any communist position on AI has to take his original accel-erationist proposition – that AI has an elective affinity with capitalism and is fundamentally inhuman – seriously.AI is rapidly becoming the principal battleground for those who want to defend life and those who are indifferent or even hostile to it.
posted by No Robots at 12:23 PM on April 5, 2021 [5 favorites]
@No Robots thanks for explicating your deal, and I would be interested to read an FPP also.
posted by away for regrooving at 12:50 AM on April 8, 2021
posted by away for regrooving at 12:50 AM on April 8, 2021
Uppestcase and Lowestcase Letters [advances in derp learning]
posted by flabdablet at 6:12 AM on April 8, 2021 [2 favorites]
posted by flabdablet at 6:12 AM on April 8, 2021 [2 favorites]
@ away for regrooving
I have been interested in the work of Dyer-Witheford for a long time, particularly his article, "Twentieth-first century species-being" (link is to a .doc file). I did take a stab at it a while ago. When I was asked in this thread to back up my position, I googled "biocommunism + artificial intelligence" and I got this new book by Dyer-Witheford (I'm not sure why it is available in full-text). I am now reading it closely. It may take me a long while to digest it along with its many intriguing citations. When I'm done, I would certainly like to post something. Thanks for the encouragement.
posted by No Robots at 9:17 AM on April 8, 2021
I have been interested in the work of Dyer-Witheford for a long time, particularly his article, "Twentieth-first century species-being" (link is to a .doc file). I did take a stab at it a while ago. When I was asked in this thread to back up my position, I googled "biocommunism + artificial intelligence" and I got this new book by Dyer-Witheford (I'm not sure why it is available in full-text). I am now reading it closely. It may take me a long while to digest it along with its many intriguing citations. When I'm done, I would certainly like to post something. Thanks for the encouragement.
posted by No Robots at 9:17 AM on April 8, 2021
It's okay to ask people to comment from a more informed perspective. I've done that myself. The key is to ask, to acknowledge that it's a significant ask, and to provide, in a respectful manner, some resources to help people meet your request.
The problem eviemath, is that in your lengthy personal yelling at me here (not okay), you didn't stop to read the multiple replies to my one initial comment.
When other people weren't being respectful in the first place, I do not need to extend that in return. Your framing here is that I'm the one being not nice, etc., when what actually already happened was this intellectual hazing ritual full of both snarky and intellectually dismissive comments. Did they bother to ask for clarification? No. So why exactly, should I? This isn't a classroom, I'm not being paid to be nice academic professor here.
I was speaking my mind casually about a topic just like everyone else. My further responses being forthright and blunt just means my refusal to coddle and perform affective labor given the the sort of attitude I was seeing. That's fair.
Do you see what I'm getting at? There's a part of this dynamic that happened earlier, that you missed. Then you took it upon yourself to police my response to that attitude.
If one person had stopped to ask, oh polymodus, what is your perspective on PSPACE? Or asked, what is it about complexity theory that you think is relevant? Did we see any of that dynamic? No. I received the opposite, so why exactly should I bend over backwards, especially as a person of color where being blunt and speaking up is often taking for and framed as being aggressive?
posted by polymodus at 4:04 PM on April 8, 2021
The problem eviemath, is that in your lengthy personal yelling at me here (not okay), you didn't stop to read the multiple replies to my one initial comment.
When other people weren't being respectful in the first place, I do not need to extend that in return. Your framing here is that I'm the one being not nice, etc., when what actually already happened was this intellectual hazing ritual full of both snarky and intellectually dismissive comments. Did they bother to ask for clarification? No. So why exactly, should I? This isn't a classroom, I'm not being paid to be nice academic professor here.
I was speaking my mind casually about a topic just like everyone else. My further responses being forthright and blunt just means my refusal to coddle and perform affective labor given the the sort of attitude I was seeing. That's fair.
Do you see what I'm getting at? There's a part of this dynamic that happened earlier, that you missed. Then you took it upon yourself to police my response to that attitude.
If one person had stopped to ask, oh polymodus, what is your perspective on PSPACE? Or asked, what is it about complexity theory that you think is relevant? Did we see any of that dynamic? No. I received the opposite, so why exactly should I bend over backwards, especially as a person of color where being blunt and speaking up is often taking for and framed as being aggressive?
posted by polymodus at 4:04 PM on April 8, 2021
But PSPACE is not one of those classes, so please stop trying to use it like one?
Which is why I purposely chose the specific word tackle. So could we stop assuming other people are making strong statements?
to tackle something means: make determined efforts to deal with (a problem or difficult task
Literally, I said that AlphaGo was a great demo of tackling issues that we know to be PSPACE, PSPACE-hard. The notion that we should stop trying to relate AI to PSPACE is just false, look at the later sections in this paper by AI/mathematics faculty, and look at this paper discussing ways to amend PSPACE-hard theories of human theory-of-mind computation to cognitively-tractable complexity classes, and also Google Scott Aaronson's many discussions on complexity theory, where he explicitly says that P =? NP is deeply relevant to AI. In the second work, their proof literally contains a reduction from TQBF!
The notion that I was saying anything so simple as deep learning "solves" PSPACE, we are done, is just not a fair reading of what I said. It's projecting a fear that my words somehow promulgate/reinforce the current AI fad. But I don't hold that position either, even as I was critical of the first piece.
posted by polymodus at 4:23 PM on April 8, 2021
Which is why I purposely chose the specific word tackle. So could we stop assuming other people are making strong statements?
to tackle something means: make determined efforts to deal with (a problem or difficult task
Literally, I said that AlphaGo was a great demo of tackling issues that we know to be PSPACE, PSPACE-hard. The notion that we should stop trying to relate AI to PSPACE is just false, look at the later sections in this paper by AI/mathematics faculty, and look at this paper discussing ways to amend PSPACE-hard theories of human theory-of-mind computation to cognitively-tractable complexity classes, and also Google Scott Aaronson's many discussions on complexity theory, where he explicitly says that P =? NP is deeply relevant to AI. In the second work, their proof literally contains a reduction from TQBF!
The notion that I was saying anything so simple as deep learning "solves" PSPACE, we are done, is just not a fair reading of what I said. It's projecting a fear that my words somehow promulgate/reinforce the current AI fad. But I don't hold that position either, even as I was critical of the first piece.
posted by polymodus at 4:23 PM on April 8, 2021
if you are just now thinking "What f'ing song lyric? What sort of music snob hipster does she think she is, dropping some obscure lyric as if obviously folks would be familiar with it, or else just aren't cool enough to engage with?" That's part of my point. It's annoying
Nope. What actually ran through my head when I read those sentences was, this is implicitly racist. It's white culture used on people of color who don't know the songs, cultural difference used as microaggressions to retraumatize and dominate, etc. But that never occurred to you when you wrote that, because that's how microaggressions work.
posted by polymodus at 4:48 PM on April 8, 2021
Nope. What actually ran through my head when I read those sentences was, this is implicitly racist. It's white culture used on people of color who don't know the songs, cultural difference used as microaggressions to retraumatize and dominate, etc. But that never occurred to you when you wrote that, because that's how microaggressions work.
posted by polymodus at 4:48 PM on April 8, 2021
polymodus, your second comment in this thread seemed to include me among the commenters you were claiming were unqualified to comment in this thread in your estimation, and none of your later comments clarified that to not be the case. So yeah, I do get to be pissed off that you use intellectual bullying. I get to be pissed off at intellectual bullying even if it's not potentially directed at me, in fact. And when you throw the first stone, your hill is not as high as you seem to think it is. Of course, it is quite clear here that you think that you were wronged first and were just responding, i.e. that you didn't throw the first stone. If you can't tell already from my previous comments that I have read the situation differently, I don't know what to tell you - maybe try re-reading at some later date?
As you note, this isn't a classroom: that means that you have no standing to be declaring what people should or shouldn't know - or how they should or shouldn't demonstrate their credentials to you - in order to participate in the discussion. There's also clearly nothing preventing you from saying that you don't think other commenters should be allowed to participate in this thread, but if you can't understand how anyone might then read that as you being a gatekeeping snob and consequently express a negative opinion, well, /shrug/, then I have some additional opinions and conclusions too. But one of those conclusions is that you're not interested in my perspective here, so.
posted by eviemath at 11:21 PM on April 8, 2021 [3 favorites]
As you note, this isn't a classroom: that means that you have no standing to be declaring what people should or shouldn't know - or how they should or shouldn't demonstrate their credentials to you - in order to participate in the discussion. There's also clearly nothing preventing you from saying that you don't think other commenters should be allowed to participate in this thread, but if you can't understand how anyone might then read that as you being a gatekeeping snob and consequently express a negative opinion, well, /shrug/, then I have some additional opinions and conclusions too. But one of those conclusions is that you're not interested in my perspective here, so.
posted by eviemath at 11:21 PM on April 8, 2021 [3 favorites]
your second comment in this thread seemed to include me among the commenters you were claiming were unqualified to comment in this thread in your estimation, and none of your later comments clarified that to not be the case
eviemath, no, because my second comment quoted the two people I really took issue with. Also, the pileon was structural, the actions involved weren't anything particularly horrid by themselves, and that makes it more of a meta issue. It wasn't about you at all. I thought your comments were pretty awesome as usual.
posted by polymodus at 2:28 AM on April 9, 2021 [1 favorite]
eviemath, no, because my second comment quoted the two people I really took issue with. Also, the pileon was structural, the actions involved weren't anything particularly horrid by themselves, and that makes it more of a meta issue. It wasn't about you at all. I thought your comments were pretty awesome as usual.
posted by polymodus at 2:28 AM on April 9, 2021 [1 favorite]
that means that you have no standing to be declaring what people should or shouldn't know - or how they should or shouldn't demonstrate their credentials to you - in order to participate in the discussion.
Of course not. I do not hold those positions. My 2nd comment does not contradict that. Like, this is a post about superhuman AGI, people can speculate as they please, that wasn't my issue. I do believe that (barring some Kuhnian shift), the answer to Chiang's question requires CS theory proper (and likely, some sort of revision of it). I do believe that it's okay to tell someone it looks like they're thinking of a concept in a limited or wrong way. We tell that to creationists and climate change deniers. It's not bullying to do so, it's just being direct. Especially when that someone has chosen to engage by unconstructively criticizing me first. There's a difference between throwing peanuts at a published article, and throwing such peanuts at fellow mefites. Individually it's not a problem, but if you would empathize with what I experienced upthread you would see that it was a pattern (and a couple of them had a history of snarking/sniping at me, which you would not know).
posted by polymodus at 2:53 AM on April 9, 2021
Of course not. I do not hold those positions. My 2nd comment does not contradict that. Like, this is a post about superhuman AGI, people can speculate as they please, that wasn't my issue. I do believe that (barring some Kuhnian shift), the answer to Chiang's question requires CS theory proper (and likely, some sort of revision of it). I do believe that it's okay to tell someone it looks like they're thinking of a concept in a limited or wrong way. We tell that to creationists and climate change deniers. It's not bullying to do so, it's just being direct. Especially when that someone has chosen to engage by unconstructively criticizing me first. There's a difference between throwing peanuts at a published article, and throwing such peanuts at fellow mefites. Individually it's not a problem, but if you would empathize with what I experienced upthread you would see that it was a pattern (and a couple of them had a history of snarking/sniping at me, which you would not know).
posted by polymodus at 2:53 AM on April 9, 2021
Mod note: Folks, please take the rest of your conversation over to MeFi Mail if you plan to continue. Let's keep the thread accessible and allow space for others to engage with the intended topic.
posted by travelingthyme (staff) at 9:32 AM on April 10, 2021
posted by travelingthyme (staff) at 9:32 AM on April 10, 2021
« Older An Annoying, Self-Absorbed Cadre of Entitled Young... | And Georgia's always on my m-m-m-m-m-m-m-m-m-mind Newer »
This thread has been archived and is closed to new comments
posted by JHarris at 4:33 AM on April 3, 2021 [1 favorite]