Classic Mario music but... cHAngED
March 12, 2018 1:34 PM   Subscribe

Super Mario Bros 2 with its music frequency table "reversed.". I'm not sure what that means exactly but it's certainly an, um, unique sound. The same thing done to Mario 3.

If you want to try playing SMB2 like this, the rom is linked in that video description.
posted by JHarris (56 comments total) 22 users marked this as a favorite
I am out somewhere where I can't listen to this immediately and I'm very angry about that.
posted by cortex at 1:38 PM on March 12 [6 favorites]

I listened to it for you cortex and it's awesome and you love it.
posted by odinsdream at 1:40 PM on March 12 [13 favorites]

posted by cortex at 1:40 PM on March 12 [9 favorites]

It's like the uncanny valley of video game music.
posted by borkencode at 2:13 PM on March 12

Just watching that video made me tense up. Time to drag out the old Nintendo and give that tension something to do!
posted by heyho at 2:17 PM on March 12

Can someone with more muscal/NES programming knowledge explain what reversing the frequency table means? It doesn't sound like the music is playing backwards, or even that every note is playing backwards. Some parts sound almost normal, others sound way out there. I'd love to learn what's going on.
posted by sotonohito at 2:18 PM on March 12 [1 favorite]

If like me you're not familiar with these games, here's the original soundtrack for Super Mario Bros 2, so you can compare them. And here's a playlist of the complete OST for SMB 3, somebody else will have to identify the source music used in the "reversed" video.

I thought that "reversing" the frequency table meant that the set of notes is inverted, so for example a scale goes from high to low where it once went low to high. But that doesn't seem to be what's going on either.
posted by ardgedee at 2:20 PM on March 12 [1 favorite]

Whatever this is it hurts my brain and I'm pretty sure we should kill it with fire for being unnatural and an offense to God.
posted by loquacious at 2:26 PM on March 12 [4 favorites]

It looks like they dumped out the ROM and found the HEX table that contained the music notes. Guessing inverting them just meant turning it upside down? Horrifying, regardless.
posted by msbutah at 2:31 PM on March 12

I think they just picked a centerpoint (possibly one defined by the internal representation) and flipped all the notes around it, low notes become high and vice versa.
posted by JHarris at 2:34 PM on March 12 [3 favorites]

This gets under my skin like quartertone Bach.
posted by gngstrMNKY at 2:39 PM on March 12 [6 favorites]

Okay so it turns out this is pretty fuckin' great. And yeah, something along the lines of what JHarris just said: there's definitely something going on along the lines of inverting the movement along music scales while holding everything else in place, so that e.g. a half-step up in a melody line becomes a half-step down, a whole step up becomes a whole step down, etc.

I'm not sure how precisely that's implemented in the carts or the hardware, as far as how much a given instrument voice would move around up or down given the "inverting" process. You have bits where a set of high notes remain a set of high notes post-inversions, and a set of low notes remain low, which suggests that you have in that case an instrument sound that's just high (or low) enough in pitch range that inverting the scale doesn't actually mean inverting e.g. the whole range of a piano keyboard. So stuff stays in the same basic pitch range.

The big tell on this inversion process is how so much of the stuff is characterized by an apparent minor scale feel. Because if you invert the scale so that up is down and down is up, then, well, you invert a major triad so that instead of a major third and then a minor third, you get...a minor third and then a major third. Which is a minor triad! And that bit tends to sound fairly normal, in fact, which makes the overall feel of these feel sort of normal too vs. just totally random garbo noise.

But melodic movement, and harmonic chord structures, in Western composition move with some very specific tendencies that are not symmetrical moving up and down the chromatic scale. So when you invert everything, you get a melody doing things like stepping from the minor second down to the tonic which you just about never hear out side of Dick Dale or metal. That's the stuff that sounds really off kilter, happening over the otherwise odd-but-normalish minor chord stuff.
posted by cortex at 2:42 PM on March 12 [16 favorites]

I think I know what's going on: The order of the notes is played forward, but the order of the pitches is played backwards. I'm suspecting this because the bassline is mostly intact for most of the bars; if the frequencies were inverted the shuffle beat would be in the upper registers.

A very brief demonstration: The first two bars of "Twinkle Twinkle Little Star" is seven notes. The first six are quarter notes (one beat each) and the final is a half note (two beats):
C C G G A A G—
Reversing the note pitches but retaining the note values would look like this, holding the final C for two beats:
G A A G G C C—
posted by ardgedee at 2:56 PM on March 12 [3 favorites]

Oh nice, the .nsf files they provide (2BMS / 3BMS) have all of the music/sounds, not just the parts in the video. Snag those and drop them into a JavaScript-based NSF player, and we're off.
posted by sysinfo at 2:58 PM on March 12 [2 favorites]

For the non-music-theory folks, to expand on the major triad and minor triad thing a little more helpfully:

Imagine a piano keyboard. Imagine you are playing a C major chord, a nice basic meat and potatoes chord. If you can find a keyboard to do this on, virtual or otherwise, go for it. But in any case, a C major is just the note C, then note E, and the note G played together. From a C to an E takes four half-steps, literally four keys (white and black) up the keyboard: C -> C# -> D -> D# -> E. Four half steps is a "major third" in the jargon. Then from E to G is three half-steps: E -> F -> F# -> G. Three half-steps is a "minor third". (Why no E#? That's a whole music theoretical discussion, but that's also why there's no black key between E and F on a keyboard. Bonus weird conversation is the fact that the musical distance between C and E is larger (four steps) than the musical difference between E and G (three steps) and yet the C and E keys are the same distance apart as the E and G keys on the keyboard. Music is *fucked*, y'all.)

So. Take a base note (C in this example), and stack a major third (four steps, C -> E) up. Then from the top end of that stack a minor triad on top of that (three steps, E -> G). Glue those two things together and boom: you've got a major triad, or major chord. C, E, G. Seriously, find a keyboard and pound this out if you get a chance, it's nice to make it feel physical.

So a minor triad is constructed the same way but with the components swapped. Base note, minor third, major third. If you were starting from C again, that'd be C -> D# -> G. Only one note changed but it's all gloomy and shit now, easy peasy.

But! The trick here is they aren't changing the Es to D#s. They're inverting the whole scale. So we shouldn't think about it in terms of the E and the D#. We should think about it in terms of flipping things around some center point.

Let's say that center point is C. (We're just picking this arbitrarily, it's probably not C but it doesn't matter because this would work no matter where you start. The pitch would be different with different central pivot points but the relative change in all the notes would be the same, so it'd have the same feel overall.) Now instead of going up four steps from C to E and then three more from E to G, we move C zero steps (it's at the center), and move four steps down, and then three more steps down. We end up with a C, and then C -> B -> A# -> A -> G#, and then G# -> G -> F# -> F.

So we end up with a chord made up of F, G#, C. A minor triad plus a major triad: it's an F minor chord. That didn't happen because an algorithm went looking for major chords to alter: it happened as a totally happenstance result of turning the musical scale upside down.
posted by cortex at 2:58 PM on March 12 [10 favorites]

Googling but not managing to find and parse a clear explanation (at a level I can grok) of how frequency is managed in the NES's audio processing. I'm really curious about the specifics of the frequency table itself, in part because I don't know if the NES stored those values as (a) twelve distinct semitone values for C -> C# -> ... -> B, and then modified them by octave multipliers, or if it stored a larger multi-octave range of note frequencies.

Why this matters is there's a couple ways you could tell an audio chip to play, say, a high D note. (Imagine a C key around the middle of a piano keyboard. We're calling that "middle C". An octave above that is another C key; we're calling that "high C". The D two half-steps above middle C is middle D; the D above high C is high D. Okay. So.)

1. Maybe we tell the NES, "play the note that has the frequency you have in memory for high D".
2. Or maybe instead we tell the NES, "play the note that has the frequency you have in memory for middle D, but up an octave."

The difference is that in case 1, the NES has to store not just twelve frequency values but however many several dozen cover the range from its lowest supported note to its highest supported note. In case two, it just has to store twelve frequency values, to cover a single octave of notes, and then it does simple math (doubling a given value for each octave up you want to go). Either approach is totally plausible; the first one takes up a little more memory, the second one takes a little more computational effort.

But, so, why it matters, why I'm wondering: because if they flip a big table of frequencies that spans several octaves, from say 65.4 Hz for the C two octaves below middle C to 1046 Hz for the C two octaves above middle C, then notes can move around a LOT on the inverted scale. You'd expect very high notes to now be very low notes, and vice versa. You ask the NES for a note at the bottom of that table of frequencies, but the table's been flipped so it gives you back a note at the top. Roaring mice, squeaking lions.

Whereas if they flip a small table of just an octave's worth of notes, you wouldn't get as much of a jump in the overall pitch of individual notes. The melodies and harmonies get all weird, but high notes stay high and low notes stay low. So if the table runs normally runs C -> C# -> ... A# -> B, the flipped version runs B -> A# -> ... -> C# -> C. So then you ask the NES for a D note in the third octave, and because it flipped the small table the value for D is now actually the value for A, and vice versa, which is definitely a different note, but what you get is an A note in the third octave. Ask for an E in the second octave, get a G in the second octave. Mice still squeak, lions still roar, they just sound a little bit off.

Given what I can hear in the soundtrack, I tend to think it's something like that latter small table arrangement. High notes are still high, low notes are still low, and the melodies jump around in weird ways that might in part be the weirdness of looping around the top or the bottom of a one-octave scale because of the inversion process colliding with the "note + octave" arrangement in a way that wouldn't happen if it really was one big table from low to high notes.
posted by cortex at 3:35 PM on March 12 [5 favorites]

Which means I agree with the diagnostic observation ardgedee made that the general pitch of the notes is staying the same, but I disagree totally with the theory that the order of notes has been altered. I know these SMB arrangements like the back of my hand and they are definitely not reversed in time; every little bit of the melodic and harmonic momentum of the songs are in place, the pitches are just out of whack.

Mario music backwards would, I venture, be much less recognizable still!
posted by cortex at 3:38 PM on March 12 [2 favorites]

Deeply uncomfortable.
posted by boo_radley at 3:47 PM on March 12 [1 favorite]

The person who made these turns out to be @illusionfoxe on Twitter, if you want more things of a mutating NES nature.
posted by JHarris at 3:49 PM on March 12 [1 favorite]

Metafilter: That's the stuff that sounds really off kilter
posted by Old Kentucky Shark at 3:53 PM on March 12 [1 favorite]

Here it is with video and audio reversed.
posted by pmburns222 at 3:57 PM on March 12

This is one of the most beautiful things I've ever heard, and I'd really like to do some theory analysis on it.
posted by curiousgene at 3:59 PM on March 12 [1 favorite]

Pretty sure ardegee is right. Rhythm forward, pitch backwards. Pretty simple.
posted by Sys Rq at 4:04 PM on March 12

If you want to start digging into the what/why of the output, you might want some tools (Windows-based, sorry) - NSFPlay (click the hand/list icon, then view->keyboard shows the current notes and lets you stretch time), NSF Importer to turn it into FamiTracker format (song/track at a time), and FamiTracker which gives you the piano roll view and editor. Adjusting the CPU clock adjusts the pitch, and frequency tables are apparently per-game, not a global NES thing.

I don't have the theory to do anything with it myself.
posted by sysinfo at 4:07 PM on March 12

If they just have an array of unordered notes, which is the sort of thing Nintendo is known to do, and each musical note is a random array lookup, then reversing the array of notes would have nothing much to do with musical theory:

Original: C D E F G A B Bflat Fsharp Csharp

(Notes clearly appended to end by programmers coding in assembler)

New: Csharp Fsharp Bflat B A G F E D C

The answer lies in whether the table of frequencies used is [MetaFilter:] ordered by pitch ascending.
posted by crysflame at 4:38 PM on March 12 [1 favorite]

Oh hey, it turns out you can just export as text once you've got it into FamiTracker and diff against the original. Here's the changes for the first track from SMB3 : pastebin (instruments are colon-separated)
posted by sysinfo at 4:41 PM on March 12 [2 favorites]

C-4 is the only note that never changes, and the rest seem to change by absolute distance from C-4. It must be ordered! (Or, it’s not an array or etc.)
posted by crysflame at 4:43 PM on March 12

Not that I'm especially interested in the theoretical underpinnings of the original track, mind you. I simply would like to look at this as a work in its own right, and explore it from a theory perspective. The Javascript-based player that sysinfo linked to above lets you select individual voices to play, so I may do some transcription later and see what I get.
posted by curiousgene at 4:45 PM on March 12 [1 favorite]

If I still had a working Windows VM and/or Darwine install, I'd load these into Famitracker, but sadly, that is not the case. sysinfo, could I trouble you for SMB2 as text, when you have a moment?
posted by curiousgene at 4:52 PM on March 12

Just the main theme? Sure. I'm just finishing packaging up the first SMB3 track as text/MIDI/GuitarPro5/TuxGuitar, so I'll do that next.
posted by sysinfo at 4:58 PM on March 12

To add insult to injury he didn't get the mushroom on the first little mountain.
posted by St. Peepsburg at 5:26 PM on March 12

Well, that's been a evening. SMB2 & SMB3 main themes (pre/post reversal) in various formats (FamiTracker/text+diff/MIDI/GuitarPro5/TuxGuitar) - Zip file. If you'd like it on a different host, I can do that. No guarantees on the accuracy, this is a pretty ad-hoc workflow. Fittingly, workflow.txt gives links to the tools used - mostly Windows-based - so if you get access to a VM/compatibility layer (free time-limited Win10 VM including Ubuntu-based Linux subsystem from Microsoft) you can pretty easily replicate the process for any other NSF file, or the rest of the 'reversed' tracks in the files that I linked to earlier. Have fun!
posted by sysinfo at 5:30 PM on March 12 [3 favorites]

One benefit to all of this is that I've finally learned what Luigi was doing during SMB1.
posted by sysinfo at 5:39 PM on March 12 [2 favorites]

I played my share of Mario 2 back in the day. I have to say it proves how terrible of an ear I have that, well at least from the character selection screen and on, I can't really say that A) it sounds all that different nor B) that it sounds bad now and double nor C) that it sounded all that capital G Great to begin with.

Neat though. Just above my pay grade it seems.
posted by RolandOfEld at 5:52 PM on March 12 [1 favorite]

Ok, and the Mario 3 sound is the OG soundtrack right? The item sounds are identical and the music sounds not different to me.
posted by RolandOfEld at 5:54 PM on March 12

Original Mario 2 & Mario 3 for comparison. The non-music sounds (jumps, coins, blocks, etc.) appear to be the same though, yeah.
posted by sysinfo at 5:59 PM on March 12

Super cool stuff! Yeah, I'd be interesting in importing a MIDI file (thanks, sysinfo!) into notation software and then analyzing the sheet music it spits out (I've never actually tried to make sheet music from a MIDI file, but I'm assuming it's very, very easy). I thought this music sounded really cool, and I'd be interested to know what the chord progression is like (also, I'd be interested to know what the relationship is between the original chord progression and this inverted progression).
posted by shapes that haunt the dusk at 6:02 PM on March 12

However they made it, my 13 year old Mario obsessed kid with (perhaps) perfect pitch just called out from 2 rooms away "That sounds like Super Mario Bros. 2 but weird" after hearing it from that far away through my crappy Macbook speakers. He had no way of knowing what I was playing. Should I be proud? I'm proud.
posted by mollweide at 6:12 PM on March 12 [6 favorites]

Good ear! Also, good soundtrack, which helps; there's such a specific feel to a lot of early Mario music, it manages to keep its shape even when the harmonization goes kablooey.

To which end, if someone is bored and willing to do the work, I would love to hear a version of these tunes sorta monotonized. Just take the first note played on each channel and make it so that that's the ONLY note played on that channel. I'm guessing it'll *still* feel familiar, just based on the rhythmic aspects of the compositions.
posted by cortex at 6:17 PM on March 12 [1 favorite]

I wonder if it has that weird, slightly microtonal quality at the beginning because the original music detunes one of the voices to get a little bit of a chorus-y, saloon piano effect, and the inverted tones widen the detuned gap? Although listening to the original, man, they really detuned that one voice by kind of a lot. But the inverted version still sounds more dissonant to me, so I'm wondering if that's what people are picking up on when they say this sounds like hell to them.
posted by shapes that haunt the dusk at 6:19 PM on March 12

Music theorist Hugo Riemann had all kinds of late-19th century insight into pitchspace mirrors, and how that would impact harmonic theory. It's referred to as Riemannian Theory, or Dualism.

I'd love to find out how this was done as well; I suspect since it's an NFS hack that they found some single byte to flip that would manipulate SMB2 and SMB3's sound engine. Most of what is here sounds like the pitch classes may be exactly flipped around some axis -- the C major tunes in SMB2 turn out in F minor, so that'd mean C was the axis. But something sounds like the flipping only happens within certain octave bands.
posted by phenylphenol at 6:22 PM on March 12 [2 favorites]

My ear is so bad that maybe I could have said it was like a weird Mario game, but I couldn't have nailed the game. If there's a genetic component to musicianship, he got it from his mother.
posted by mollweide at 6:22 PM on March 12

Important addendum: My kid says he's not Mario obsessed. He is, however, obsessed enough about old video games to be able identify the frequency table reversed Super Mario Bros 2 theme heard through bad speakers from far away. He could probably do it for Sonic, Kirby, and Zelda games, too. As an Atari kid, I have no idea what he's talking about.
posted by mollweide at 6:28 PM on March 12 [1 favorite]

shapes that haunt the dark, both MuseScore and TuxGuitar can import the MIDI (or GuitarPro 5) files in my zip and give you sheet music - I'd suggest deleting the 'noise' staff which was originally various samples, but gets interpreted as the 'noise' drumset to amusing effect.
posted by sysinfo at 6:37 PM on March 12

I like cortex’s theory that the table in question actually stores pitch classes that are then combined with register info somewhere else to produce the final note, because there’s no other way to account for how individual voices seem to stay in the same register after reversal.

You can tell that the notes aren’t under temporal inversion because symmetrical neighboring figures are inverted vertically. Listen to the lower neighbor motives towards the beginning in the figuration of the original music and how they appear at the same time in the reversed but as upper neighbor figures instead.
posted by invitapriore at 6:48 PM on March 12

If you turn your computer upside down, it sounds normal again!
posted by roger ackroyd at 6:52 PM on March 12 [3 favorites]

I mentioned the thread to the person who made it on Twitter (I don't know them, I just found their feed today, from the posts where they linked the videos), and they said that cortex got it exactly right in this comment. Tweet.
posted by JHarris at 7:04 PM on March 12 [4 favorites]

To which end, if someone is bored and willing to do the work, I would love to hear a version of these tunes sorta monotonized. Just take the first note played on each channel and make it so that that's the ONLY note played on that channel.

SMB2/SMB3 but it's all the first note of each channel.
posted by sysinfo at 7:11 PM on March 12 [3 favorites]

There was a post a while back about Jacob Collier, it led me to this other video where he talks about this approach and calls it "negative harmony".
posted by equalpants at 7:28 PM on March 12 [1 favorite]

This is like a musical take on the Rumplestiltskin riddle from the original King's Quest where the troll's name is just "rumplestiltskin" spelled by flipping the alphabet, right?
posted by klausman at 8:18 PM on March 12

I already monotonized them, I'm not dropping another $5 on 'Ifnkovhgroghprm' just to invert the octaves too. Take the magic beans already and don't overthink them all in one place.
posted by sysinfo at 8:27 PM on March 12

Yeah, so like, it's basically just a pitch substitution cipher, right, with fun music theory properties because it's a mirror? So, first row (normal, ascending) mapping to second row (bizarro world, descending):

C4 C#4 D4  D#4 E4  F4 F#4 G4 G#4 A4  A#4 B4  C5
C5 B4  A#4 A4  G#4 G4 F#4 F4 E4  D#4 D4  C#4 C4

Which is mostly consistent with e.g. the text output in sysinfo's zip file for the beginning of the SMB2 main theme:

G5 F5 D5  B4  G4 F4 D4  B3  G3 F#3 F3 D3  B2  G2 G3 G4
... turning into ...
F5 G5 A#5 C#4 F4 G4 A#4 C#3 F3 F#3 G3 A#3 C#2 F2 F3 F4

Except the A#5 in the bizarro version is inexplicably (and sounds like) a B5, maybe a reverso-table typo.

(on preview: I have just spent whole minutes carefully aligning  s; I am not throwing this comment away)
posted by cdefgfeadgagfe at 8:35 PM on March 12 [1 favorite]

One more, and then I'm done. SMB2 overworld music gets the mono treatment (it's not exactly gypsy jazz).
posted by sysinfo at 10:14 PM on March 12 [2 favorites]

This post made me go on a hunt to revisit this old Ask* about understanding exactly what a "key" is. And also I feel better that my "music is logically kind of fucked" is reinforced by cortex's comment above. I'm not stupid, turns out. So thanks!

*Hi, I'm Bob Vila, welcome to This Old Ask. This week Norm's solving relationship problems and answering the age-old question: "Can I eat this?"
posted by middleclasstool at 6:04 AM on March 13 [1 favorite]

And also I feel better that my "music is logically kind of fucked" is reinforced by cortex's comment above. I'm not stupid, turns out. So thanks!

That's kind of the beauty of human systems like this, though. I mean, language is the same, there isn't any argument from first principles I could make that would explain why appending -ass to an adjective is a totally valid-ass way of intensifying that qualifier. I like logical systems too, but they both offer their own joys.
posted by invitapriore at 5:38 PM on March 13 [1 favorite]

I'm a bit late to the party but Jazz wunderkind Jacob Collier recently talked about negative harmony in an interview.
posted by yoHighness at 1:39 PM on March 14

« Older "Using the Power of science. Science! Science!! I...   |   The contrition and shame of inappropriately... Newer »

You are not currently logged in. Log in or create a new account to post comments.