Beethoven X: AI

January 7, 2022 8:53 PM

The philosophical and musical failings of the "DeepLudwig" project to complete Beethoven's last symphony.

Listen to the result.

posted by blue shadows (30 comments total)

The author's core assertion:

Artificial intelligence can mimic art, but it can’t be expressive at it because, other than the definition of the word, it doesn’t know what expressive is. It also doesn’t know what excitement is, because there’s a reason people call excitement “pulse-pounding,” and computers don’t have pulses. When computers set out to do art, they don’t fashion it in a whirl of creative trance inflected by a deadline; they can’t account for the heat or alarming lack of it in the room, sensations in the groin, the failure or success of drawing a foot that looks planted on the ground, the failure or success of creating rhythmic momentum on the page, the bit that’s bullshit and needs to be fixed and the bit that’s really good and you see where it wants to go, the woman or man you just met who excites you and whom you hope to excite
...
None of that can be programmed into a computer in any way that means anything. To repeat: The only true, meaningful intelligence is in a body, and likewise the only true and meaningful creativity. By “body” I’m including the mind, contained in a brain.

In practice, this may be true, but not in principle. It just means the inputs and operation of the AI has to encompass all sensory modalities, try to emulate their holistic and integrated processing in a human, and generate an aesthetic 'sensibility' from that process as a human does. The author appears to posit a metaphysical barrier to this possibility, I contend it's a practical limitation at present.

His assessment of the output:

To start, the turn-on-the-radio test: If I turned on the radio and heard this music, would I guess it was by Beethoven? No. The orchestra doesn’t sound like Beethoven. If anything it sounds rather like Robert Schumann’s handling of the orchestra, which is to say a bit clunky—though Schumann is rarely as clunky as this. The scoring is competent as far as it goes, you can hear everything fairly clearly, there’s some variety of color, but in the end it’s aimless and uninspired and sometimes tries too hard, such as some grandiose trombonic moments that Ludwig wouldn’t do that way.

The author is himself a composer, has written a biography of Beethoven, and is undoubtedly very well acquainted with the traits and tropes of Beethoven's music. That is not true of the average listener. The effort and calibre required to fool someone with a forensic eye (and disposition) is much different than trying to fool the average listener. I still don't think the AI output quite works, the development is not varied enough, but an expert is the wrong person to judge, at this stage of development.
posted by Gyan at 9:36 PM on January 7, 2022

This was an interesting essay to read to get a sort of "man on the street" view of generative AI and how someone not deep in the field might make their pro and con arguments. For Swafford, his criticism centers around the notion of embodiment. Which, sure, it's one of many issues philosophers have been debating for quite some time when it comes to consciousness, intelligence, and creativity. There are many more to be sure.

His assertions that once a computer "solves" a problem like chess, then no one is interested in watching a computer play chess again though is silly. It's widely acknowledged that programs like AlphaZero have developed unique strategies in chess and go that continue to fascinate players. AI generated art will likely do the same.

If you'd like to read a kind of counter-argument by someone who thinks a lot about AI, this recent essay by Blaise Aguera y Arcas of Google AI is an interesting read. I don't necessarily buy his argument, but it is thoughtfully written.

But really this article was absolutely worth reading because now I know that Beethoven literally responded to a critic by writing “What I shit is better than anything you ever did!!!” ...and who can argue, really.
posted by gwint at 9:53 PM on January 7, 2022 [4 favorites]

Here's an article with a good deal of background on the project
posted by gwint at 10:03 PM on January 7, 2022

ooh i think i heard one of the "grandiose trombonic moments" - one of the grand tromboners. what i'm hearing in the scherzo sounds fantastically derivative of fifth symphony, and when it doesn't it just sounds like a bunch of not particularly sonorous beethovenian gestures all mashed together.

enjoyable writing; interesting insights into the composer.
posted by 20 year lurk at 10:06 PM on January 7, 2022 [2 favorites]

Agree with 20 year lurk. It sounds a lot like what you would expect an AI to spit out if you fed it the scherzo from the fifth symphony. And there's no real logic to the way the thing is put together. Each individual bit is okay, but it never really feels like it's going anywhere. And that feeling of going somewhere is something that Beethoven did better---and in more diverse ways---than anyone else.
posted by nosewings at 11:39 PM on January 7, 2022 [3 favorites]

This article feels a bit like a cheat -- the author knew going in that they were reviewing an AI generated piece of music. Although as a Beethoven expert they would not have been fooled into thinking it was a missing Beethoven piece, I wonder what they would have thought if they had not known its provenance?
posted by vernondalhart at 12:25 AM on January 8, 2022 [1 favorite]

As if Beethoven never wrote shitty music.
posted by spitbull at 1:41 AM on January 8, 2022

The whole riff about intelligence having to be embedded misses a significant point. Take computer chess playing. as an instance. We figured out how to tell a computer how to play chess very early in the digital epoch. Turns out that the techniques used to program chess playing computers didn't have much application to creating an intelligent computer but they worked just fine for playing chess. If you think of AI as a collection of technologies for doing complex things that we think require intelligence rather than an attempt to create a mind, then the question of embodiment falls by the wayside for many problems.
posted by rdr at 6:20 AM on January 8, 2022 [2 favorites]

That mistakes the point though, an AI can't write music because it has no idea of what music is, it communicates nothing because there is no thought behind it all. It leaves music purely about listener pleasure, which is a terrible concept for it or any art. There is no end state or set of rules to follow in which to achieve a set goal as in chess, though even there the AI certainly isn't "playing" by the same standard we'd think of a human endeavor.

Attempting to strip the communicative aspect from music, or bluff past it, is to say the world exists for just for your entertainment with nothing beyond to concern yourself with, just you and the endless repetition of pleasing patterns devoid of anything beyond their surface effect. Of course consumption without need for thought is an ideal for many in this culture, so there's every chance it'll succeed in the end.
posted by gusottertrout at 8:08 AM on January 8, 2022 [8 favorites]

I saw "Beethoven X" and got excited for a reboot of the movie franchise.
posted by Your Childhood Pet Rock at 8:14 AM on January 8, 2022 [3 favorites]

Attempting to strip the communicative aspect from music, or bluff past it, is to say the world exists for just for your entertainment with nothing beyond to concern yourself with

This seems like an unnecessarily hostile reading of what the commenter said. As I understand the article, the goal of the AI was to produce something that mimicked Beethoven, not to produce “music” in the sense of emotional communication and complex thought, and it was only moderately successful at best.

I agree with the person who said it would have been useful to ask a layperson what they thought, rather than go to an expert on good old Ludwig Van. I also wonder what would happen if the AI had been trained to mimic a less legendary composer, maybe someone less ornate.
posted by The Pluto Gangsta at 10:07 AM on January 8, 2022

This seems like an unnecessarily hostile reading of what the commenter said.

The "you" parts of my comment were meant generically, the harshness directed at the project and ones like it, not thought of as representing the view of rdr or anyone else, other than referencing why there is an importance in intelligence being embodied, coming from the thinking and felt perspective of another rather than pattern manipulation alone.
posted by gusottertrout at 10:32 AM on January 8, 2022 [3 favorites]

> As if Beethoven never wrote shitty music.

Didn't RTFA, we all see.

I dropped not to say that, however, but to add this:

Barry Cooper's 1988 realization of Beethoven's 10th Symphony played by the London Symphony Orchestra/Win Morris along with Cooper's lecture about the realization (Youtube).
posted by flug at 10:57 AM on January 8, 2022 [1 favorite]

About 15 years ago my father played me a CD of a piece of music unknown to me and asked me to guess what it was. My first guess: a piece by Brahms. That was wrong. My second guess: a reconstruction of Beethoven's 10th symphony. Which turned out to be right. It was the 1st movement only, which a human composer had reconstructed and orchestrated. (As flug comments above, Barry Cooper's 1988 realization.) I thought (and still think) it was pretty good, though in general Beethoven's even numbered symphonies are not the standouts (though have their own compelling qualities).
posted by Schmucko at 11:22 AM on January 8, 2022

OK, I asked a musically uneducated listener, namely me, to check it out.

Listening to the AI: It definitely sounds like 19th century classical music. If I heard this out of context I wouldn't guess that it was AI-generated. No idea if it sounds like Beethoven or not.

For contrast, I then listened to some actual Beethoven. This seems like much better stuff-- more dramatic, I guess. There seems to be more feeling in the use of tempo and loudness, though for all I know this is because the performers are doing a better job.

As someone who used to argue all the time on Usenet about AI, I'm skeptical of Swafford's impossibilities... "AI can never do X" arguments tend not to hold up for long. But he has a point to the embodiment stuff: as humans we feel a connection to other humans through art. An AI can produce interesting stuff, but there's nothing to connect to. (Yet. In a hundred years, who knows.)
posted by zompist at 11:28 AM on January 8, 2022 [1 favorite]

Computers are good at calculating, but they can never replicate human creativity and sensitivity.

...is what I would have said with total confidence only a few years ago. Since then, I've conversed with GPT-3. I've nudged a GAN toward art-like output. I've thrown together a music generator. They're partway there. It all reminds me a little of kittens who haven't quite figured out how their legs work yet.

It's only a matter of time before they get it all figured out. Then they'll be getting into everything.
posted by The Half Language Plant at 11:45 AM on January 8, 2022 [2 favorites]

There are some assertions in the article I'd dispute: the idea rhythm has to come from a human player--I love Conlon Nancarrow's experiments with rhythm that he programmed into a player piano in the era before modern computer music. One could equally argue that only a human sense of pitch is valid, not these pianos tuned to ratios of the 12th root of 2.

I started listening to the AI, and agree there was something overwhelmingly flat that accumulated the more I listened to it. A few phrases had a little perk but that was it. It's Beethoven Muzak.

Beethoven wasn't sitting around thinking, How can I write something that sounds like Beethoven? He was above all an innovator, and not in just strings of phrases. How about a fugue--but also sonata form? And throw in... trills after trills, like nobody else does... Toss in the obsolete Lydian mode into this string quartet... To express thanksgiving after recovering from an illness... Exploring a musical space nobody else would have.
posted by Schmucko at 11:55 AM on January 8, 2022 [1 favorite]

I think the project is intriguing, and the pastiche that they constructed almost works but sounds half-baked. Some bits almost sound like real Ludwig, some parts sound like a fake lame Ludwig, and other bits just sound like generic copied samples from actual pieces with slight tweaks added on top.

I think Ludwig Van B is a tough bar for computer simulation, his personal style is quite quirky and often unpredictable by any standards. I've played with various generative tools down to auto-chords on the fun machine (which are fun), but distinctive structural shaping is still hard to fake. A lot of what is described as computer generated music is automated chunks with a lot of human shaping and editing.

Note, I lost the last classical name that tune on the radio when I called Ludwig and it turned out to be Mendelssohn, so I'm not a perfect judge of these things either. Of course, some Mendelssohn is influenced by Ludwig.
posted by ovvl at 2:32 PM on January 8, 2022 [1 favorite]

The idea I've come to with regard to deep-learning AI projects is that while they may not have real intelligence in the way we generally think of it, they have a certain reflected intelligence that comes from all the human creators of the training datasets. Things like GPT-3 and its successors will be able to output ever more spooky echoes of the vast corpus of every sentient-created piece of text they can get their hands on, sometimes in surprising ways, since they're literally the most well-read beings on the planet and no one knows every obscure fanfiction that might have influenced a novel confluence of words, etc. I think algorithmic intelligences are going to remain at this kind of reflected "quasi-sentience" for a while, but keeping in mind how large a portion of everyday human interactions and labor don't require much beyond that, we can expect them to become increasingly common. They're already writing a lot of basic news stories about stock price movements and such.

This particular instance is a little different since from the background article it seems to be handcrafted for this specific project, and not a general music composition algorithm that was given the entire Sony Music catalog to ingest or anything. The same general principle applies more directly though; the more fine-tuning they did of the exact nature of how it develops from a set of starting notes, chooses instruments, etc, the more the final output reflects the creative choices of the sentient humans working on the project.
posted by Wandering Idiot at 6:45 PM on January 9, 2022

I usually find the discussions on Metafilter interesting and entertaining - often the discussion is even better than the linked article - but this time I have to say that your discussion is kind of off-putting.

You discuss whether AIs can write music; the question why AIs should write music is totally absent.

What is the purpose or benefit of machines that try to be creative, machines that compete with composers and authors? Why would anyone want to develop them?

I would like to say that we are seeing the beginning of a technological shift: machines no longer improve human life. Machines that are superior to humans in physical strength have given us some clear benefits the last 200 years: we can travel faster, dig deeper mines, we can accomplish more, and we are freed from hard, physical labour.

A machine that is built to mimic or copy or replace human creativity does not continue this line of progress - on the contrary, it makes our lives poorer. Writing a symphony or any other creative work is not the same type of physical labour like tilling the soil - a machine that frees us from creativity is not the same thing as a machine that frees us from digging in the soil. This is not progress.

When computers become "creative" they will soon produce such quantities of material - symphonies, novels, poems, pictures - that humans can no longer compete. In the 1980s, the composer David Cope developed a composing computer program called EMI. He fed the program some Bach, and the program then produced 5,000 new Bach chorals in a single day.

Why would anyone think that this is progress? What unexamined wishes, dreams or emotions are drivning the people who are striving to develop "creative" AI systems? What is the benefit to humanity if such systems are flooding the world?

Humans need to start thinking about what types of technological development we really want; which technologies will actually improve our lives, and which technologies won't. Until we start asking these questions, we are not dreaming big enough; we are just floating along like driftwood, accepting whatever technology is being pushed on us, like obedient little consumers.
posted by Termite at 12:24 AM on January 10, 2022 [3 favorites]

Termite, that's a good point (your scientists were so preoccupied with whether they could, they didn't stop to think if they should.)

ONE rebuttal might be that the exercise IS an expression of human creativity, on the side of the programmers. Painters inspired by nature would try to reproduce the impression of a sunrise on a bay, and in the same way these programmers inspired by Beethoven are using their creativity to make something that gives a similar impression as Beethoven's music.

In that case to appreciate this work we'd have to look at it as a programming exercise and not at the sub-par musical output. The exercise still could be a failure from that view too. Instead of creative new programming, they may just be tossing opaque existing algorithms and using big databases.

Second... I'm recalling the last idiosyncratic chapter of an astronomy textbook, The Physical Universe, by Frank Shu. This was from 1982. I actually found a free download just now, page 561: it imagines a janitor observing on the night-shift that a university computer has become conscious. The janitor is ready to unplug it, when it says:

"Wait! If you let me live, I will compose and play for you Beethoven's tenth symphony as he might have written it had he lived" I think you might wait to listen. I think the world might wait to listen.

I think that dramatizes the high expectations for true AI--and how valuable and unobtainable a Beethoven's 10th symphony seemed in 1982 (before even the composer reconstruction of the 1st movement from Beethoven's sketches)--and how disposable the product of this 2020 "AI" is, which merely has shuffled around data into different forms like a Markov chain.
posted by Schmucko at 8:35 AM on January 11, 2022 [1 favorite]

"Dave.....my scherzo is going....I can feel it...."
posted by thelonius at 9:40 AM on January 11, 2022 [2 favorites]

Right, it seems that until we can create some form of actual artificial intelligence, current AI is basically just ever more sophisticated mimicry in the context of creativity anyway. Then there is also the question of whether there will ultimately be algorithmic limits on "progress" in this field.
posted by blue shadows at 9:42 AM on January 11, 2022 [2 favorites]

Yep. I'm on both sides of this argument/discussion. I essentially agree with YN Harari that AIs are very skilled at simulating/faking some aspects of human expression, but that they'll probably never get past that last uncanny-valley of quirky human consciousness, for which the creation of Art is one of the tests.

AI generative data has been used already as a general Compositional-Tool in various arts, and will be even more in the future, but there always has to be a human puppet master to program, edit, and get paid for it.
posted by ovvl at 5:37 PM on January 12, 2022

Schmucko: ONE rebuttal might be that the exercise IS an expression of human creativity, on the side of the programmers.

Sure, but the obvious answer would be: if the programmers really wanted to be creative, they could try writing a symphony. One that people still wanted to hear 200 years into the future.
posted by Termite at 9:34 AM on January 13, 2022

There's a very wide ocean of valuable creative expression between "being Beethoven and writing a symphony" and "feeding Beethoven MIDI files to a neural network and expecting his 10th". You're not just dismissing this flawed project and contemporary musicians (yes, musicians) experimenting with code as a creative tool; you're also dismissing Beethoven's own friends and contemporaries who fell through the sieve of classical canon.

Regarding this particular effort, Beethoven is a thorny choice of composer to attempt machine learning on, not just because of his genius as a composer but also because his œuvre is like that of three different composers. He changed the face of music in Europe, but he also transformed himself twice. I don't know what kind of fine tuning or bespoke architecture this ML model had, but if they just told it "here's op.1 to op.138, these are all equal examples of Beethovosity, now build these sketch fragments out into a symphony" they should expect something much more incoherent and unimpactful than something paying more attention to the quirks of his late style.
posted by polytope subirb enby-of-piano-dice at 10:33 AM on January 13, 2022 [1 favorite]

polytope: you're also dismissing Beethoven's own friends and contemporaries ...

What do you mean? Please explain.
posted by Termite at 12:01 AM on January 14, 2022

I’d be interested in seeing a comparison of the AI “reconstruction” of Beethoven with a human who happened to write something vaguely akin to Beethoven, as well as a human explicitly trying to write in the style of Beethoven.

Further, there’s the matter of how countless composers (and other artists) have been met with almost the exact same criticism for their new work - “this doesn’t have the soul of the piece I like,” “there’s no emotion in this,” and so forth - only for said new work to be happily folded into the canon a few years or decades down the line.
posted by Molten Berle at 12:05 AM on January 14, 2022

When computers become "creative" they will soon produce such quantities of material - symphonies, novels, poems, pictures - that humans can no longer compete. In the 1980s, the composer David Cope developed a composing computer program called EMI. He fed the program some Bach, and the program then produced 5,000 new Bach chorals in a single day.

Why would anyone think that this is progress?
Your argument is that art will be destroyed by… there being a lot of it? Have novels stopped existing since AO3 opened? It has 8.5 million works, but I don’t see fiction having plummeted. YouTube has tens of billions of videos, but there’s still plenty of good material.

The idea that something is better (or “worthy”) just because a human did it is nonsense. No, this AI didn’t write something up to the level of the greatest Western composer in history, but that’s a bit of a high bar to clear.

Lastly, if the Cope and Bach chorales are actually so indistinguishable that we’re running the risk of losing Bach’s work in the apparent tidal wave of harmonized Lutheran hymns now crashing over us, how truly valuable and irreplaceable was it?
posted by Molten Berle at 12:27 AM on January 14, 2022 [1 favorite]

A computer can be a useful tool for many different purposes, but what is the human benefit of turning it into a fake composer? Who benefits from this? Wasn't the whole point of technology that we - humans - would be freed from hard work, so that we could spend a little more time creating and enjoying arts and music?
posted by Termite at 5:45 AM on January 14, 2022