If it really works, it's the coolest audio production tool ever.

March 13, 2008 4:10 PM

Celemony are a bunch of crazy German software engineers known best for making Melodyne, a family of top of the line pitch correction tools. Apparently they've recently figured out how to do what they do with polyphonic audio. I can't begin to explain how cool this is. Just watch the video.
posted by stenseng (122 comments total)

That's amazing. I can't even begin to comprehend how they can do this with music. This will change so much.
posted by flatluigi at 4:18 PM on March 13, 2008

want want want want
posted by papakwanz at 4:20 PM on March 13, 2008

Wow. That is simply amazing. I would love to learn how they managed to do this. Could this work for other audio recordings? Are they just extrapolating the notes by comparing the input files from a a vast library?
posted by phyrewerx at 4:21 PM on March 13, 2008

Poor classical musicians. The concert is going to become a thing of the past when people discover that music can't be played perfectly in real life.
posted by invitapriore at 4:23 PM on March 13, 2008

To clarify, I think this is a really cool technology, but its potential for (and the likelihood thereof) abuse is very high.
posted by invitapriore at 4:24 PM on March 13, 2008

Insanely cool, and very useful, but there will always be respect and an audience for people who play live... maybe more so after everyone can make perfect studio recordings.
posted by Huck500 at 4:27 PM on March 13, 2008

Cool! Now I just need a "meaty" keyboard!
posted by Asbestos McPinto at 4:31 PM on March 13, 2008 [4 favorites]

I can't quite articulate why, but there's something very funny/ironic/a propos about the fact that the inventor is speaking in German, but is dubbed over.

That said... wow, that's really cool. I'd always understood that pulling out individual notes from a recording is ferociously hard to do... but to do it well, and be able to manipulate them on top of that? Fuckin' A#, man. Fuckin' A#.
posted by hincandenza at 4:32 PM on March 13, 2008 [1 favorite]

Also: "The more I pondered it, the more I began to see that what doesn't work in theory may still work in reality."

I love this man.
posted by flatluigi at 4:33 PM on March 13, 2008 [5 favorites]

there will always be respect and an audience for people who play live

Especially since so much cachet is wrapped up in issues of authenticity. If no one actually believes you can sing or play the notes, it may as well not matter for a lot of people. I'm also curious to see if this ends up being as recognizable (or not) as airbrushing/Photoshopping in photographs. There's an entire lexicon of bad Photoshopping touch-up techniques; who's to say we won't see the same for this?
posted by chrominance at 4:35 PM on March 13, 2008

I saw this in action in L.A. a couple of months ago. It was indeed freaky-cool.
posted by sourwookie at 4:35 PM on March 13, 2008

Holy shit.
posted by Clave at 4:35 PM on March 13, 2008

Shit, I was just working on this post. Thanks for the jinx, stenseng! :)

I'm not sure I buy into it *completely*, and the video is at best, of course, a best-case scenario, but if this even remotely works as advertised in this video it is very, very impressive indeed.

KVR on the matter; Celemony's FAQ.
posted by goodnewsfortheinsane at 4:39 PM on March 13, 2008

I'm also curious to see if this ends up being as recognizable (or not) as airbrushing/Photoshopping in photographs

It already is. Tuned vocals--and I'm not just talking about the Cher/T-Pain fast-attack effect, here--are everywhere. And once you start listening for it, you'll notice it. It's one thing to tune a few words in an otherwise-perfect take. But there's no excuse for some of the lousy tuning on the radio.

That said, I saw this a couple days ago, and it's the neatest thing ever. Melodyne is already fantastic--although it produces more artifacts than Auto-tune--and adding this technology is just mind-blowing, especially at the price. Melodyne is really, really cheap for what you get.

Plus, that guy's beard is fantastic.
posted by uncleozzy at 4:41 PM on March 13, 2008

Damn cool. Maybe the Germans should run the world.
posted by tkchrist at 4:42 PM on March 13, 2008 [3 favorites]

This will make remixing trivial. Take an mp3, run it through Direct Note Access, and you get the original multitrack sources back out.
posted by b1tr0t at 4:44 PM on March 13, 2008

"Meedee"? really? I bought my first piece of MIDI gear 17 years ago, and I've never heard it pronounced that way.

But yeah, cool and slightly scary all at once.
posted by NortonDC at 4:49 PM on March 13, 2008

that is 100% cool as hell.
posted by facetious at 4:49 PM on March 13, 2008

No, for two reasons b1tr0t:

One, it says in the FAQ that if it gets a multi-voice input, it can't distinguish the different notes to the different instruments; if your piano and guitar and voice are all hitting the E above middle C at the same time, this will see one note that is a wave of all three voices. It'll be like those cheap Casio keyboards that have the "Piano & Strings" button...

Two, mp3s are usually heavily compressed, so some note detail gets lost and isn't recovered. You might generate separate tracks and notes even, but they will be sparser than the original- like a Hal Leonard reverse sequencer... :)
posted by hincandenza at 4:50 PM on March 13, 2008

Wow. Gonna have to check that out. Thanks for the post, stenseng.

I like the name, "Celemony". Sounds like somebody's been to Japan. But funny how the narrator pronounced midi as "mee-di". Are there people who say it that way?
posted by flapjax at midnite at 4:54 PM on March 13, 2008

This is going to make "sampling" legal cases utter hell.
posted by DU at 4:54 PM on March 13, 2008 [3 favorites]

No, for two reasons b1tr0t

Still, it should be possible to break out a clean drum track, bassline, etc.
posted by b1tr0t at 4:57 PM on March 13, 2008

(Also, what video service, if any, are they using to host this. The quality is better than anything I've ever seen in a browser.

Oh wait...maybe I'm watching this with my newly working mplayer plugin.)
posted by DU at 4:57 PM on March 13, 2008

I'm sure that once this reaches major production studios, it will be used with all the delicacy and restraint they showed when auto-tuning everything from country ballads to high-energy dance pop.
posted by ardgedee at 5:02 PM on March 13, 2008 [4 favorites]

You people are cheering for Autotune++ ?
posted by BeerFilter at 5:02 PM on March 13, 2008 [2 favorites]

I'm not sure that it will be undetectable in practice. See autotune abuse, for example. (may have found that via mefi, but can't remember right now). It still looks cool, though.
posted by Jakey at 5:04 PM on March 13, 2008 [3 favorites]

It'll be like those cheap Casio keyboards that have the "Piano & Strings" button...

Actually, I'm looking forward to that. More beautifully crappy noise. (As opposed to more lifeless slickness.)

"mee-di". Are there people who say it that way?

Huh? Does anyone say it any *other* way? What, "middy"? Or "My-dye?"
posted by goodnewsfortheinsane at 5:07 PM on March 13, 2008 [1 favorite]

BeerFilter: You people are cheering for Autotune++ ?

Yep. Your problem?
posted by flatluigi at 5:08 PM on March 13, 2008

they managed to bring Jerry Garcia back from the grave? that IS amazing!
posted by tremspeed at 5:09 PM on March 13, 2008

Ah, /ˈmɪdi/ ("middy"), I see. It's not that different from "mee-dee" to my ears, and that's the common pronunciation in German (and Dutch). So maybe that's the thing.
posted by goodnewsfortheinsane at 5:11 PM on March 13, 2008

Huh? Does anyone say it any *other* way? What, "middy"?

I and every musician I've ever met says "middy". This is very interesting! Maybe it's a generation thing, or a regional thing?
posted by flapjax at midnite at 5:11 PM on March 13, 2008 [1 favorite]

Jinx, goodnews!
posted by flapjax at midnite at 5:13 PM on March 13, 2008

Damn, that is seriously cool.
It really makes me want to fool around with music again.
posted by numlok at 5:17 PM on March 13, 2008

This is gonna open the door to a whole new generation of people that can barely perform
posted by Senor Cardgage at 5:20 PM on March 13, 2008

I just saw this over at Gearslutz and nearly shat myself. I think it's the only time I've seen a marketing video worth the bandwidth. It's not perfect, no one is going to pull an artifact free guitar solo from a few strummed triads anytime soon, but OMFG this is only a little less awe inspiring than the moon landing.

Wonder when the first copyright infringement lawsuit over a reconbobulated guitar track will be?

(and why can they do this when I can't find a functional plug in to output MIDI from a vanilla audio guitar input to save my life?)
posted by bunnytricks at 5:22 PM on March 13, 2008

Whoa. If it really works like it does in the video, this is as frightening as it is awesome. Recording sessions of the future will just be the session musicians sitting down, playing one take of the Descendents' "All" and then packing up while the producers copy/paste it out into a 2.5 minute song.

I think many modern music producers just consider pitch-correction as a filter like reverb or echo or anything else. I like it when it's obviously used to produce an effect, rather than literally pitch-correcting a crappy singer.
posted by Durhey at 5:23 PM on March 13, 2008 [1 favorite]

" Recording sessions of the future will just be the session musicians sitting down, playing one take of the Descendents' "All" and then packing up while the producers copy/paste it out into a 2.5 minute song."

That song is from the future?

Wow.
posted by Senor Cardgage at 5:26 PM on March 13, 2008

This may allow editors to alter the inflections of speech to give completely unintended meaning. A lot of languages use "chording," so to speak, to denote emotion or irony or sarcasm.
posted by basilwhite at 5:27 PM on March 13, 2008

With Melodyne, you won't need no steenkin' meedee.
posted by StickyCarpet at 5:28 PM on March 13, 2008

I've only heard MIDI as "middy".

My husband was quiet while watching that, though every now and then blurting out "That's not natural!" then shaking his head. I think it just blew his mind.
posted by evening at 5:28 PM on March 13, 2008 [1 favorite]

"and what you can see, is also what you can touch"

The narrator then proceeds to move the penis-shaped notes around.
posted by banished at 5:29 PM on March 13, 2008 [1 favorite]

This is gonna open the door to a whole new generation of people that can barely perform.

And since performing is fast becoming the only way a whole lot of musicians can earn a living (what with endless free downloads and less and less people willing to buy recorded music) what'll we be left with?

NOTHING, THAT'S WHAT!! IT'S THE DEATH OF MUSIC!! IT'S ALL OVER!!

Who's gonna play at the funeral, though?
posted by flapjax at midnite at 5:31 PM on March 13, 2008

To me the truly amazing part of the video was where the reworked sample followed the MIDI keyboard input. I've dreamed of a tool like that for years.
posted by odinsdream at 5:31 PM on March 13, 2008

The narrator then proceeds to move the penis-shaped notes around.

I've dreamed of a tool like that for years.

Hmmm...
posted by flapjax at midnite at 5:33 PM on March 13, 2008

Oh, and stenseng, excellent use of the fucking tag.
posted by flapjax at midnite at 5:33 PM on March 13, 2008

I would guess that Germans say meedee and the translator wasn't a musician, and therefore didn't know that Americans say middy. (At least, in my experience they do. And that's just a guess.)
posted by uosuaq at 5:35 PM on March 13, 2008

Could someone who's excited by this please explain why? What problem does it solve that couldn't be solved equally well by, say, just playing the original tune a little differently?
posted by SPrintF at 5:38 PM on March 13, 2008

Yeah, uosuaq, I think that's it indeed.

And your upside-down username totally rocks.
posted by flapjax at midnite at 5:40 PM on March 13, 2008 [1 favorite]

What problem does it solve that couldn't be solved equally well by, say, just playing the original tune a little differently?

Cause we're talking after-the-fact here. Playing the tune again might not be possible. Chances are the guitar player, for example, left the recording session and stepped into the street into an oncoming bus. That's what usually happens, anyway.

Drummers usually choke on vomit. Someone else's vomit.
posted by flapjax at midnite at 5:43 PM on March 13, 2008

SPrintF: Could someone who's excited by this please explain why?

Um, because if you're playing a great take, infused with all the right mojo, but mis-hit a note... you don't have to go back and re-record a significant section. You can fix that one note in place in the recording. Or you don't have the original players in the same place when you discover something to be fixed/changed.

Yes, ProTools et al are abused in modern music because they makes "good" singers out of mediocre ones. But fixing the equivalent of a "teh" typo on an otherwise great piano or guitar track without doing another effort will save a lot of studio time. And yes, it's not inconceivable people will strum standard chords, then go in and "meaty" the track into complex polyrhythmic arpeggiations they could never actually play.

It'll probably sound as fake and offputting as the ProTool'ed singers, though, if they over do it. But it may free up composers to manifest a much closer song to the Platonic ideal they have in their head, including creating compositions that are literally unplayable by human hands. That's just... neat.
posted by hincandenza at 5:46 PM on March 13, 2008

Seems to me this could be useful in other unintended ways... As a teaching aid for music theory... Throw in your favorite track, and suddenly you've got a whole dynamic note chart that shows what chords are used, etc. etc. Oh, you want to put the whole thing in a different key? Different mode?


Click.


Also, what's to stop me from deciding I'd like some monkeyed with Duane Allman slide solo in my latest garage creation?


Fucking hot
posted by stenseng at 5:46 PM on March 13, 2008

SPrintF: Aside from it being Star Trek like technology? The majority of punch ins could be a thing of the past. You can rewrite a guitar part without remicing the amp and getting the signal chain just right. Amazing things will be done with this creatively, and even though it will probably be used for evil by pop producers the good outweighs the bad.
posted by bunnytricks at 5:48 PM on March 13, 2008

What problem does it solve that couldn't be solved equally well by, say, just playing the original tune a little differently?

Sometimes people make very small mistakes in otherwise perfect takes. Sometimes you can't re-take. Sometimes you're recording a string quartet live and the cello is a little out of tune in one section. There are all sorts of studio applications. I think it's interesting, actually, that Celemony is including this gratis with the Melodyne plugin (and, presumably, the other versions, later). It's a totally amazing technology--the first of its kind--and people would certainly pay more for it. Is it a bid to crush Antares completely?
posted by uncleozzy at 5:51 PM on March 13, 2008

Astonishing!
posted by toastchee at 5:54 PM on March 13, 2008

That is awesome.

Not all producers know how to play an instrument. We often find samples that we love, but aren't QUITE right, and we can't re-record it.

This makes me want to play around with some italo-house piano vamps very badly.
posted by empath at 6:05 PM on March 13, 2008

Production = composition = performance

Stop trying to seperate them.
posted by empath at 6:07 PM on March 13, 2008

I'm looking forward to the ways this gets "abused." Goddamn awesome.
posted by wemayfreeze at 6:10 PM on March 13, 2008

Everyone's obsessing over "mee-di" and not noticing the other weird pronunciations?

He says "audio" as if the first two syllables rhymed with "rowdy". And he pronounces "monophonic" "mono-PHONE-ic".

Conclusion: the real WTF is that they've hired a German voiceover dude who does a perfect cheesy American voiceover accent, but doesn't know how to pronounce quite common English words.
posted by AmbroseChapel at 6:15 PM on March 13, 2008 [1 favorite]

Who's gonna play at the funeral, though?

It'll be a drum machine with flat batteries. No worries, the YouTube footage will get Melodyned.
posted by Blazecock Pileon at 6:15 PM on March 13, 2008

The narrator also says "ow-dio" for audio (at least, he does the first time he says it), and "mono-phone-ic" and "poly-phone-ic". I think he took a lot of his pronunciation cues from ze Chermans.
posted by kcds at 6:16 PM on March 13, 2008

Wow, all my carefully labelled samples are suddenly useful!

I'm quite looking forward to hearing some creative uses of this.
posted by Magnakai at 6:21 PM on March 13, 2008

The narrator sounds like the guy from How It's Made.
posted by zeoslap at 6:43 PM on March 13, 2008 [2 favorites]

...the YouTube footage will get Melodyned.

There you have it: "melodyned" might just become as ubiquitous a term as "photoshopped." "It's been melodyned." "This stuff's been melodyned to hell". "Dude, didn't you hear that bum note? How come you didn't dyne it?"

Google results for "melodyned".

I'm quite looking forward to hearing some creative uses of this.

I'd venture to say that the most creative uses of this might be in music where you have no idea that it's being used.
posted by flapjax at midnite at 6:43 PM on March 13, 2008

Could someone who's excited by this please explain why?

This is difficult to achieve because each note is comprised of overtones which form a chord of sorts within that one note, the intervals in a multi-note chord are also based on overtone spacings, so it very hard to imagine how one might sort out the component frequencies that belong to a note from those that belong to a chord. I'll guess that the harmonics of a single note might share minuscule pitch wavering that identifies them as belonging together.
posted by StickyCarpet at 6:57 PM on March 13, 2008

I'm not a musician (I had no fucking idea what he meant by "meaty," for example -- I thought it was another piece of software the company made or something), but the line "imagine re-tuning your guitar after you've played the song" kind of blew my mind.
posted by Rock Steady at 7:02 PM on March 13, 2008

I want to play with this.
posted by Jimbob at 7:07 PM on March 13, 2008

I mean, shit, I just want to put in some tunes and get it to automatically change them to a minor key, or even something weirder. The mind boggles.
posted by Jimbob at 7:11 PM on March 13, 2008

Now even I will sound like Clapton...... ;)
posted by caddis at 7:11 PM on March 13, 2008

I think the voiceover was actually Jerry Lewis reading Lincoln's Gettysburg Address, then modified through the cool new Melodyne tools...
posted by twsf at 7:16 PM on March 13, 2008 [3 favorites]

I guess this means Chinese Democracy is going to get pushed-back another 5 years?
posted by Thorzdad at 7:17 PM on March 13, 2008 [3 favorites]

I hope the technology is a bit more reliable than those wav to midi converters, because they suck.
posted by PeterMcDermott at 7:19 PM on March 13, 2008

Between this, and the firmware for canon cameras, I'm so glad I'm finally living in the future.

Now where are my flying cars?
posted by Freen at 7:19 PM on March 13, 2008 [1 favorite]

Shouldn't this also allow more or less automated transcription? I guess you'd have to assign the different parts to different instruments, and clear up any confusion resulting from two instruments playing the same note, but it seems like this would take most of the work out of transcribing.
posted by ssg at 7:22 PM on March 13, 2008

Oh -- my god. I am smitten.

WOWOWOWOW

.
posted by RubberHen at 7:22 PM on March 13, 2008

Seriously what an amazing time to be alive. I love math and science so much I could fucking cry sometimes.

I'm gonna share this with musician friends and expect mainly negative reactions composed of the words 'artificial' 'pop' 'synthetic' 'talentless' etc. And I bet that eventually abuse of this will validate those comments, at least in my friends minds. But what amazing power, and if used with moderation, sense, and love?

It kind of extends to the pitch domain the power we've had over the time domain since samplers became popular. Imagine what DJ Shadow could do with this.


2nd:
I think it's the only time I've seen a marketing video worth the bandwidth
posted by oblio_one at 8:02 PM on March 13, 2008 [1 favorite]

That's awesome. And what's with Germans and digital audio processing? First the MP3 and now this. Huh.
posted by GuyZero at 8:02 PM on March 13, 2008

It'll be a drum machine with flat batteries. No worries, the YouTube footage will get Melodyned.

No. It'll be nothing but dozens of DJ mixers chained haphazardly into each other run off of dirty AC with a bad ground, processed through a complex set of (waving hands) algorithmic processing functions.

Left to run long enough and it will eventually produce every song ever made, and every song that will be made.

Except Nickelback. Because I'll be standing there with a baseball bat the whole time to make sure it doesn't do that.
posted by loquacious at 8:11 PM on March 13, 2008 [4 favorites]

I'm with RubberHen on this one, although MeFites are clearly more excited about all of the technopossibilities this opens up.

Me, after playing music for forty years and trying but not succeeding to be seduced by Midi and related techno-wonderlands, I'm not so excited.

Twenty-five years ago, when Keyboard magazine became a computer gee-whiz 'zine, I lost interest.

I am a piano player, and am more excited by a couple of guitarists on the back porch than the latest way to manipulate music with binary code.

Sorry, but the musical Luddite voice can never be emphasized enough.

What is music for?

iPod isolation?
posted by kozad at 8:12 PM on March 13, 2008 [1 favorite]

Could someone who's excited by this please explain why? What problem does it solve that couldn't be solved equally well by, say, just playing the original tune a little differently?

uh....
posted by lastobelus at 8:22 PM on March 13, 2008

kozad: What is music for?

For me, it's for the expression of talent -- whether that talent be found in the singer at the microphone, the pianist at the keys, the guitarist at the amp, or the DJ at the keyboard. I'm an amateur DJ because, though I love music, I have no instrumental or vocal talent. I can create music in my own way. A program like this opens up doors upon doors for mixers of all kinds; now we can manipulate the notes themselves to create.


I am a piano player, and am more excited by a couple of guitarists on the back porch than the latest way to manipulate music with binary code.

Liking one doesn't mean you can't like the other. I have the fabled 'eclectic taste' in music; you can find me listening to Jack Johnson and Gregg Gillis in the span of minutes.
posted by flatluigi at 8:29 PM on March 13, 2008

What problem does it solve that couldn't be solved equally well by, say, just playing the original tune a little differently?

instead of just sampling things and having to settle for the notes that are played in the sample, you can pretty much use whatever notes you want to compose

i play guitar and keyboards and have a lot of computer synth and sampling tech at my fingertips - i don't NEED this to make music - but it fascinates the hell out of me
posted by pyramid termite at 8:42 PM on March 13, 2008

My thoughts on this came in two distinct phases:

1.) The Geek: Wow. That is just technically awesome. I mean, the cleverness and aptitude that went into the development of this is just mind blowing.

2.) The guy that is looking to get rich: Ok, this is exactly the tool I've been looking for to take my Siamese cat's singing career to the next level. I mean, a bit of mixing, cut in some of the parrot's back-up vocals... I think I'm looking at an instant top 40 hit here.
posted by quin at 9:11 PM on March 13, 2008 [1 favorite]

Aside from the possiblities of dipping into a chord and fixing a bum note here and there, I think some of the more creative things that could happen might be like Photoshop filters(?). I envisage taking a recorded guitar part, and then you could wave a "Majestic" filter across it and each chord would end somehow being related to E major. You could simply record a part, getting the rhythm you want, maybe even just strum the one chord over and over, and then apply the "Melancholy Folk" filter and hey presto! you've got a sad folk song about Black Lung Disease with all the passing chords neatly filled in.

But there could also be some very nice realtime performance techniques as well - hook the software up to some kind of touchscreen interface and you could take a sequence of chords and tweak them in realtime with your fingers, kind of like playing guitar or piano .... oh. See what I did there?
posted by awfurby at 9:20 PM on March 13, 2008 [2 favorites]

Freaking,

holy—

unholy ...

FREAK.


If this is for real, they've cracked the polyphonic transcription problem. And the pitch-shifting, time-warping, and-yet-not-sounding-grotty problem. I am quite sure I am unable right now to explain how mind-blowing that is, but ... it is!
posted by eritain at 10:46 PM on March 13, 2008

Re meedee vs middy: when you speak french, like a lot of europeans do, you'll know that midi is a real french word, not an acronym. Thus it's natural, although possibly erroneous, to pronounce it the french way. Hence meedee when a voice over artist is instructed by english speaking germans.
posted by jouke at 10:52 PM on March 13, 2008

Oh, and it seems to me that the fruits of musicians motoric skills, years in the making by strenuous exercise, have just been commoditised.

Without the need for motoric real-time skills making music shifts yet a little more from being an artisan craft to a computerised manipulation of patterns at design time.
Hello worldwide geek hordes!
posted by jouke at 10:57 PM on March 13, 2008

As the inevitable byproduct of a truly fascinating technology, I can't wait to pay Ticketmaster $36.95 to stand in front of the next big thing as they wrench out horrendous renditions of Melodyne-tainted cuts on the band's gold-selling album. If I'm lucky, the engineers will invite me to the studio to record a single golf clap, which will be magically cloned, multiplied, and individually pitch-shifted into a horde of rabid fans for the band's best-selling official live-sounding album.
posted by prinado at 11:19 PM on March 13, 2008

Oh sure, musicians now get to edit their mistakes after the recording, SUURE...
WHY THEN CAN'T WE GET THE SAME THING HERE ON METAFILTER WITH OUR POSTS MATT?!
posted by Catfry at 11:47 PM on March 13, 2008

Mind duly blown. (Sure: shares some of the creepy implications of the content-aware photo editing software discussed recently. On which subject: when's the video version coming?)

Re: "meedee" - my guess is most of Europe does that (Italy sure does). (UK?) What's funnier is that quite a few Germans actually say "weefee" too.
Re: Germans & audio - yeah, what is it with that?
posted by progosk at 12:45 AM on March 14, 2008

quin - I'm afraid your parrot has competition
posted by patricio at 3:08 AM on March 14, 2008

Great. The audio equivalent of "fixed that for you"
posted by hal9k at 3:47 AM on March 14, 2008

I used to be a violinist and string bass player. After having been to the symphony the other night and seen about 30 violinists and 10 bass players on stage at once, I'm gonna have to say that someone would have to have a pretty good fucking ear to pick out the one that's slightly out of tune and alter it properly. And if I buy someone's album that's been melodyned, then go see them in concert and they sound like crap, it's gonna be Milli Vanilli all over again.

Still, no doubt you could create many cool Ozric Tentacle-like things from this.
posted by Marie Mon Dieu at 4:13 AM on March 14, 2008

It's impressive that it's now possible to manipulate individual notes in clean single polyphonic tracks reliably enough that the technology can be used for commercial music production. However if the relatively easy cases on the video are the toughest it can do well with I think it falls some way short of being absolutely jawdropping. This is more like the first workable continuous speech recognition following on as an advance from isolated words.

Well done to them anyway, assuming that the plugin doesn't get much more expensive than the current $299 when the polyphonic capability is added, I might be tempted.
posted by tomcooke at 5:04 AM on March 14, 2008

Any idea on how this works? Is it that the software applies presampled sound profiles in order to identify (and then separate) the specific frequencies? Seems kind of unlikely, and limiting. But surely it can't *just* be maths...?
posted by progosk at 5:55 AM on March 14, 2008

All props to the programmer(s) of course, this is quite a feat. I note that the examples I heard (guitar and electric keyboard) are all sounds that have a quick attack followed by a fairly static decay portion. EPs in oarticular are almost sine-wave like after the initial attack. So my guess is that these are the types of sounds that are easiest to work with and produce the least amount of artifacts. Give me an example of a background vocal section where one of the singers is out of tune! Of course that's one more nail in the coffin for anyone trying to make a living in the music "business."
posted by keys at 5:58 AM on March 14, 2008

EPs in particular, sorry
posted by keys at 6:01 AM on March 14, 2008

And if I buy someone's album that's been melodyned, then go see them in concert and they sound like crap, it's gonna be Milli Vanilli all over again.

Every record on the radio has some amount of tuning on the vocal tracks (some of which is already done with Melodyne). Drums are locked to a grid and replaced or augmented with samples. Does that mean everyone sounds like crap in concert? No, but it does mean that they won't sound like their record.

This is more like the first workable continuous speech recognition following on as an advance from isolated words.

I'd say it's more like programming the cocktail party effect into speech recognition. You're not talking about, in the case of a triad on guitar or piano, three sine waves with no harmonic content.

assuming that the plugin doesn't get much more expensive than the current $299 when the polyphonic capability is added

It's a free upgrade for anyone who purchases the Melodyne plugin after March 12. The price for the plugin once it's released, though, will be $399. So if you want to try it out, but now and save $100.
posted by uncleozzy at 6:22 AM on March 14, 2008

I have to say I'm slightly disappointed that some of you have failed to grasp the positive significance of this and are asking misleading questions like "How will this affect performance?"

Production does NOT = Compostion, which does NOT = Performance.

They are inter-related, but not linearly equivalent.

This amazing tool will not affect live performance, simply because it can't be applied in the midst of performance, i.e., it can't be used as a signal processing for the live performance (At least not yet ... if they can figure out how to do it in real time THEN live performers are frelled). It does mean that we run the risk of being inundated with even more mediocre RECORDINGS being enhanced by its' use. And THAT doesn't make me happy (But then I was sorely disappointed the first time I heard a rock performer who sounded great on the album and like crap live ... no wonder they have producers and handlers).

From the "in the studio, building a new creation" point of view, this is exciting news. To be able to edit a sound by rotating the chord structure in 3-D is pretty damned cool when you working at the level of painting with tone color and chord structures. Especially given the price.

Live music will still be just as important and potent as before, maybe more so, as we walk on stage with a minimum of gear. There is still much to be said for authenticity being present at the creation. Musicians and singers who can play their instruments and hit those notes will still have the respect and support as before, maybe more.

Counting my pennies and looking forward to Autumn 2008 and the release of the plug-in ...
posted by aldus_manutius at 7:14 AM on March 14, 2008

I don't think this will work for distorted instruments such as electric guitar, because the distortion means that the notes don't add linearly. Meaning that 2 notes played together and then distorted will cause additional notes to form and be amplified. There would be no way to post-process those notes individually.

This could be solved by capturing the guitar before the amplification, but that has its own problems and it doesn't really help people doing mashups, etc

But still its pretty cool
posted by jpdoane at 7:19 AM on March 14, 2008

because the distortion means that the notes don't add linearly.

I was a bit suspicious because the examples they use, piano, clean guitar, are pretty pure and sine-wave like, but I don't understand your example.

You could think of two notes played together on a distorted guitar as two square waves rather than two sine waves. Will that mess with the algorithm? What are square waves anyway? Just sine waves with lots of harmonics piled on top of them. And pretty much all instruments have harmonics of some sort, even the guitar and piano used in the examples, so I can only assume their algorithm is able to tell harmonics from the base note and keep them all together. Do you really think it will make that much difference? I'm more concerned about a distorted guitar being full of a significant amount of white noise, which would surely confuse things.
posted by Jimbob at 7:25 AM on March 14, 2008

I'm more concerned about a distorted guitar being full of a significant amount of white noise...

I have given many software demos in my life.

This demo is a best-case scenario. It will never work any better than what they showed you. Many times it will work much, much worse. This is basically axiomatic for all types of software.

But I imagine it does a fourier on the signal, it knows where all the notes are and then it picks out each frequency based on harmonics and re-groups them as notes.
posted by GuyZero at 7:54 AM on March 14, 2008

Me: This is more like the first workable continuous speech recognition following on as an advance from isolated words.

uncleozzy: I'd say it's more like programming the cocktail party effect into speech recognition. You're not talking about, in the case of a triad on guitar or piano, three sine waves with no harmonic content.

In terms of what's going on in the audio you're correct but what I was trying to say (not very clearly!) was more general: Celemony have created a new product category by solving one additional aspect of a very hard problem well enough to be useful for some people in some circumstances. This is impressive, but in relation to the general problem of transcribing multi-timbral polyphonic music as well as a trained human it's still incremental progress.
posted by tomcooke at 8:11 AM on March 14, 2008

Well, if you've got a chord and you want to play another (different) chord, the shortest way is to take your guitar and play another chord, isn't it ? To me (but I didn't explore the material in depth) working on recorded sounds can really slow the process when all there is to do is playing once again. Plus, it looks like the front-end of his software is another way to put music on a score, which eventually could be more awkward than classical solfege. It's an interesting tool, but no tool can do everything.
posted by nicolin at 8:15 AM on March 14, 2008

the general problem of transcribing multi-timbral polyphonic music as well as a trained human

Ahh, I see what you're getting at. See, I don't see this as even trying to solve that problem. Obviously, it's a side-effect of it (just as the side-effect of autotune is to make it easier to transcribe monophonic material), but the real hook is the hum-drum correction of bum notes, not the whiz-bang recomposing they show off. (Although that's really neat too, I think it has fewer practical uses.)
posted by uncleozzy at 8:18 AM on March 14, 2008

Yep, my *guess* would have been that it does a FFT on the signal -- basically breaking the signal down into its constituent sine waves of various frequencies. But something tells me it's more complicated than that; FFT processors, both software and hardware, have been around for years and this polyphonic nut has only now been cracked.

I'm surprised it's not more expensive. This makes recorded audio nearly as easy to manipulate as MIDI, which is... amazing in its implications. I'm conflicted: the geek in me loves it, but the musical purist in me hates it.
posted by LordSludge at 8:18 AM on March 14, 2008

This looks pretty interesting; I'd be curious to try it out. I'm an engineer, and I don't like overuse of autotune or any other "studio magic". However, it's a means to an end, and like many devices, it can be used sparingly with good results.

I wish everyone that acts so concerned about the purity of music available would stop posturing and apply that passion to making music of their own.
posted by dubold at 8:24 AM on March 14, 2008 [1 favorite]

I wish everyone that acts so concerned about the purity of music available would stop posturing and apply that passion to making music of their own.
posted by dubold at 8:24 AM on March 14 [1 favorite +] [!]

Oh man, I so agree. Also, do you all really listen to things like Cher and Milli Vanilli or the next big American Idol winner? Do you go see them live (or "live")? It's cool if you do and I'm not judging, but it seems the consensus is that it's going to ruin folk music (hint: Bright Eyes and Joan Baez already did that) or classical. There'll be plenty of bum notes/chords left in recordings that you can discuss, even with this tool.

This thing is amazingly cool. Thanks for pointing it out.
posted by sleepy pete at 9:27 AM on March 14, 2008

A lot of the same people who lament pitch-correction as "unnatural" don't seem to have too much trouble with sequenced MIDI controlling samplers and synths that they could never actually play themselves.

I'm a piss-poor keyboard player, but a lot of my music is electronic. I love using MIDI to do things that, sometimes only after the fact, do I discover is not actually playable by a single human being. But damn, that piano on that one track sounds pretty awesome. Does it matter that you'd have to have 4 hands to play all the notes as written? (BTW, for playing that song live, it was simplified to 2 hand capable.)

And as has been said above, Melodyne is a powerful tool that can do everything from fix the occasional flat note, or make an entire take sound mechanical. My favorite use for it in my music is to create harmonies because, no matter how hard I try, I've never gotten the hang of, or the patience to, create harmony takes that are not only note-perfect, but attack and timing perfect. For some songs I can't stand "S" and "T" sounds in harmonies that don't mesh perfectly.

I, for one, watched this video and the word "whoa" kept coming out.
posted by chimaera at 10:27 AM on March 14, 2008

I love using MIDI to do things that, sometimes only after the fact, do I discover is not actually playable by a single human being.

Exactly, when you're *trying* to do that it's great, but when you're aiming for a natural-sounding drum track and you leave the hats under the crashes, well, that's just an invitation for derision from actual drummers, I've noticed. :)
posted by goodnewsfortheinsane at 10:43 AM on March 14, 2008

hats under the crashes

Totally possible in the real world. It just isn't done, because it sounds awful.
posted by Sys Rq at 2:55 PM on March 14, 2008

Also: "The more I pondered it, the more I began to see that what doesn't work in theory may still work in reality."

Reality has always been only for those who lack imagination.
posted by spock at 3:38 PM on March 14, 2008

This comment will be melodyned to sound delicious.
posted by ersatz at 3:50 PM on March 14, 2008

Nope, can't edit it. The new functionality sounds cool though. And practical.
posted by ersatz at 3:51 PM on March 14, 2008

very interesting
posted by entrepreneur35 at 5:25 PM on March 14, 2008

Wow, that's just amazing. I definitely need this.
posted by malocchio at 5:50 PM on March 14, 2008

I just can't wait for the inevitable awesomeness of your favorite pop track (tm) ripped to shreds and assembled into something terifying at the hands of somebody like Aphex Twin.
posted by tehloki at 8:38 AM on March 15, 2008 [2 favorites]

Will it let me rip out guitar tracks for Frets on Fire or hacked Guitar Hero compilations?
posted by obiwanwasabi at 1:01 AM on March 16, 2008

chimaera sez: But damn, that piano on that one track sounds pretty awesome. Does it matter that you'd have to have 4 hands to play all the notes as written? (BTW, for playing that song live, it was simplified to 2 hand capable.)

I was there! And it still sounded awesome live, even though it was dumbed down for the humans.

I'm a shit keyboard player too, and I sequence impossible stuff all the time, and I watched mr. chimaera (gently) melodyne my own vocals recently.. needless to say I am STOKED. I have a tendency to spend a half hour recording guitar and vocals for new song ideas early in the evening and then stay up all night micro-editing the recordings as I work out the arrangement, in order to record a final version of the resulting arrangement later. Once I get that stuck-to-chair-and-monitor inertia, it's hard to get up and record again. This will let me indulge that impulse even further :)
posted by thedaniel at 2:44 AM on March 16, 2008

stuck-to-chair-and-monitor inertia

*So* intensely and sadly true.
posted by goodnewsfortheinsane at 12:53 PM on March 16, 2008 [1 favorite]

I sense another Cher comeback hit using this technology later this summer. Can't f$%^ing wait!
posted by chiefbluefeather at 4:11 PM on March 16, 2008 [1 favorite]

something tells me it's more complicated than that

I made some inquiries about this and received the following from a colleague:

Blind source separation is more-or-less what Independent Component Analysis was constructed to do.

It's a linear decomposition, like PCA. PCA projects onto orhtonormal components, ICA onto components that are statistically independent. Which is to say, to apply this idea in the Celemony context, one note in a chord has no predictive value on another. Of course, on a real dataset, these notes are not statistically independent, so clean separation would be hard to achieve. Moreover, ICA like PCA can generate negative coefficients, which don't really describe notes. <>

So I suspect that these bravura tweezings of the waveform are made possible by a nonlinear dimensionality reduction constrained to positive components. Something like the matrix factorization described in: this (developed for rather different purposes) or the manifold learning in: Unsupervised learning of image manifolds by semidefinite programming [PDF]

"Semidefinite programming" is an optimization that yields among other things strictly positive solutions.
posted by StickyCarpet at 11:15 AM on March 18, 2008 [2 favorites]

Well, if you've got a chord and you want to play another (different) chord, the shortest way is to take your guitar and play another chord, isn't it ?

If you've got a guitar, and I've got a guitar, and my guitar reaches acroooooooss the room... I TUNE YOUR GUITAR CHORD! I TUNE IT UP!
posted by ludwig_van at 9:57 AM on April 3, 2008 [2 favorites]