"The concept of being able to do literally anything is hard to grasp"

February 24, 2017 3:37 PM

Arbitrary Code Execution Glitches in video games have allowed creators of Tool-Assisted Speedruns break open a game entirely, using nothing more than the controller inputs that are normally used to guide in-game actions.

Originally used just to skip to the end credits screens, since then ACE has been used to program games to run other games. One recent example by MrWint, begins as a demonstration of running Pokemon Gold from within Pokemon Yellow, and quickly escalates from there.

MrWint had been working for months on his stress test of what emulated GameBoy hardware is capable of, and was inspired to finish when he watched the TASBot block during the Awesome Games Done Quick 2017 fundraiser.

(And of course, ACE glitches are not simply the realm of those using emulation tools, as this previous FPP can attest to.)

posted by radwolf76 (26 comments total)

From the comments: "Has science gone too far?"
posted by Foci for Analysis at 3:38 PM on February 24, 2017 [1 favorite]

I've watched the Super Mario 64 - Watch for Rolling Rocks - 0.5x A Presses parallel universe thing - the entire 24 minutes - and I still am not sure if the people who discover these things are real people or actual wizards.
posted by littlesq at 4:28 PM on February 24, 2017 [19 favorites]

I love how these systems are so complex that people can find and create things that the initial creator had no idea were possible.
posted by grumpybear69 at 4:44 PM on February 24, 2017 [2 favorites]

Turing completeness is a hell of a drug.
posted by glhaynes at 4:57 PM on February 24, 2017 [17 favorites]

Now consider the Universe as a Turing complete computer simulating the rules of physics...
posted by Bringer Tom at 4:58 PM on February 24, 2017 [5 favorites]

The TASBot block from AGDQ is amazing. It's a bunch of great stuff, and then all of a sudden they've built Skype (well, generic video chat) using a couple of old consoles and controller inputs.
posted by zachlipton at 5:06 PM on February 24, 2017

Various explanatory notes from the creator, for reading after you've watched the video.
posted by zachlipton at 5:15 PM on February 24, 2017

I feel like code execution glitches are kind of boring-- the most interesting part is the setup, and after that the fact that you can run (for example) 6502 code on a 6502 processor is not especially compelling. As an analogy, suppose someone showed you an 'amazing hack' where they opened up a computer and replaced its harddrive with one of their own and then ran Solitaire. At best, getting into the case might be impressive (depending on how well it was locked), but after that the ability to swap out the harddrive and run Solitaire isn't adding much to the demonstration.
posted by Pyry at 5:22 PM on February 24, 2017 [4 favorites]

Perhaps, but getting a GameBoy to run 15fps video at 960 colors with decent audio is damn impressive. You're right that the code execution itself is only so exciting, but doing something unexpected with the hardware adds a lot, and this has that in droves.
posted by zachlipton at 5:36 PM on February 24, 2017 [6 favorites]

>I've watched the Super Mario 64 - Watch for Rolling Rocks - 0.5x A Presses yt parallel universe thing -

I got 101 stars in Super Mario 64, and I felt really accomplished... (0_o)
posted by Sing Or Swim at 5:52 PM on February 24, 2017 [2 favorites]

I got 101 stars in Super Mario 64, and I felt really accomplished... (0_o)
posted by Sing Or Swim

Burn the witch!
posted by littlesq at 6:00 PM on February 24, 2017

Where's the truck?
posted by JoeXIII007 at 6:26 PM on February 24, 2017 [2 favorites]

Now consider the Universe as a Turing complete computer simulating the rules of physics...

Sure, but the Universe isn't an old school game where they got the simulation code as perfect as they could manage, burned it to ROM, and never got to touch it again. It's a new-fangled game with hot patches and constant updates, where they got the code just barely ready to ship and then started desperately fixing shit every time someone finds a hack.

Why do you think we have early histories that mix men with monsters, books of spells and miracles that just plain don't work anymore, and entire pantheons full of deities who don't respond to prayers anymore? Somewhere in the crevices of space and time is an tracker full of horrifying issues like "#3461 Chalk Pentagrams Create Mob Spawn Points", and a chat room with logs like "(-1602-07-12 15:21:48) Amun-Ra: Yahweh said he was going to *fix* the water-into-blood bug, not tell players how to exploit it! I quit!"
posted by roystgnr at 7:11 PM on February 24, 2017 [24 favorites]

I feel like code execution glitches are kind of boring-- the most interesting part is the setup, and after that the fact that you can run (for example) 6502 code on a 6502 processor is not especially compelling. As an analogy, suppose someone showed you an 'amazing hack' where they opened up a computer and replaced its harddrive with one of their own and then ran Solitaire. At best, getting into the case might be impressive (depending on how well it was locked), but after that the ability to swap out the harddrive and run Solitaire isn't adding much to the demonstration.


The interesting ones are the ones where you set up the bytes in memory to execute by setting up the game state just so - like those Mario warps. I agree that when they can do it with a straight controller input sequence it's less impressive.
posted by atoxyl at 7:18 PM on February 24, 2017

Yeah, realizing that this was a bootstrap hack loading an animation player, and that it depends on sampling the controller at insane rates (which only works for some emulators) moved this a bit closer to the "most computers work like that" position (to borrow a comment fragment from the previous thread). Ok, cycle-constrained programming takes some effort (*), but the constraints here are at least not completely insane :-)

*) Here's a great talk on this topic, covering constrained hardware, glitches, hackable insects, cycle-precise programming, poetry, and more: Poems for bugs. Suitable for coders and non-coders alike.
posted by effbot at 7:23 PM on February 24, 2017 [1 favorite]

I've watched the Super Mario 64 - Watch for Rolling Rocks - 0.5x A Presses yt parallel universe thing - the entire 24 minutes - and I still am not sure if the people who discover these things are real people or actual wizards.

That was fascinating!
posted by Pope Guilty at 8:23 PM on February 24, 2017 [2 favorites]

The thing I'm not clear on - if I fed the TAS output into real hardware, does it poll the inputs that often, or is that a quirk of the emulator? (i.e. does this only work because the TAS has intimate knowledge of the emulated state.)
posted by Kyol at 8:24 PM on February 24, 2017 [2 favorites]

but after that the ability to swap out the harddrive and run Solitaire isn't adding much to the demonstration.

It's more akin to laparoscopic surgery - sure you could just make a 12 inch incision to get the appendix out but these guys are doing it though a hole the size of a pencil. It takes a lot of skill to do it.

But to your point- why bother even programming on a GBA these days? It's a miserable platform that was driven by low hardware costs that has less capability than an Amazon Dash button. I wouldn't spend my time doing it but I respect these people for seeing how far you can go with a awkward technique on a limited platform just because.
posted by GuyZero at 9:22 PM on February 24, 2017

The thing I'm not clear on - if I fed the TAS output into real hardware, does it poll the inputs that often, or is that a quirk of the emulator? (i.e. does this only work because the TAS has intimate knowledge of the emulated state.)

I can't answer whether it's true for all of these glitches but I recall seeing a video of someone pulling off one of the controller-driven arbitrary code warps live (I mean, with his own hands) on an SNES? I know there are definitely people who have done game-state-driven ones live.
posted by atoxyl at 10:47 PM on February 24, 2017

I suppose technically those may not be fully arbitrary code execution tricks, just unexpected code execution tricks.
posted by atoxyl at 10:54 PM on February 24, 2017

TASBot has been used on real hardware (they call this "console verification" apparently), using special adapters to playback the pre-programmed controller sequences. It's been done for the NES and SNES, as shown in the AGDQ 2017 video, but I'm not seeing anybody who's done it on an actual Game Boy, and that would presumably require modified Game Boy hardware to accept the inputs. But sombody did build Game Boy Player Player, to run TASBot on the Game Boy Player, the accessory that allows one to play Game Boy/Game Boy Advance games on a Game Cube. It would certainly be interesting to see if this particular run could actually be replicated on the real hardware. The creator writes a bit about that here:

Hardware verification at least for this TAS is very unlikely, the precision required to sync the inputs it up is likely to great. In general, I think it's a reasonable question though whether an actual Game Boy could do what this run did, at least in principle (i.e. rule out that emulator inaccuracies were exploited). The best you can do to test that would probably be to create a custom ROM that does these things without any inputs and flash it onto a cartridge that you can play in a Game Boy. That isn't really a fair test, since the cartridge itself is part of the system under test, and you exclude the whole joypad input side, but it would at least determine the plausibility that GB hardware can achieve what is seen here.

Anyway, you really can input that much from the controller if you want to. The "Skype" demo was pushing around 1.15 Mbps into a SNES by polling four multitap inputs 300 times per frame.
posted by zachlipton at 11:31 PM on February 24, 2017 [2 favorites]

Executing arbitrary code like this is exactly like a magic trick. The first time you see it your mind is blown, but like any magic trick the more you see it and know about how it's done the less impressive it becomes.

It helps when you think about how bonkers it would be to explain to your youthful self in the 90s how there was a code in Super Mario World that would take an entire encyclopedia set to write down but would allow you to play a completely different game once input to your SNES.
posted by Mr.Encyclopedia at 12:15 AM on February 25, 2017 [3 favorites]

It helps when you think about how bonkers it would be to explain to your youthful self in the 90s how there was a code in Super Mario World that would take an entire encyclopedia set to write down but would allow you to play a completely different game once input to your SNES.
posted by Mr.Encyclopedia at 3:15 on February 25

Eponysterical.
posted by radwolf76 at 1:01 AM on February 25, 2017 [1 favorite]

The thing I'm not clear on - if I fed the TAS output into real hardware, does it poll the inputs that often, or is that a quirk of the emulator? (i.e. does this only work because the TAS has intimate knowledge of the emulated state.)

The CPU can poll the controller as often as it wants on the real hardware, and in the emulator used here you can control exactly what's being returned for each poll. However, the real physical controller doesn't know when it's polled, it's just a bunch of buttons, so odds are you'll get out of sync pretty quickly. And doing it by hand by actually pressing the buttons is impossible at the data rates we're talking about here.

From a quick skim of the descriptions, the hack consists of the following parts:

• giving your rival a specific weird name (easy)
• saving the game, but resetting at a specific point during the save (maybe doable by hand)
• manipulating the inventory to get the initial bootloader into memory (doable but very tedious, best automated)
• closing the menu (easy)
• using the first bootloader to load a faster version (requires millisecond precision, impossible by hand)
• using the fast loader to load the main program and animation data at high data rates (requires nanosecond precision, most likely, so I'd say impossible without additional sync ability)

posted by effbot at 5:33 AM on February 25, 2017 [1 favorite]

It's worth noting that Pokemon Gold isn't being run in Pokemon Yellow. The Gameboy doesn't have enough RAM to contain it, and Pokemon Yellow doesn't have enough writeable storage to store it, and further the memory access hardware is different. Instead, what they wrote was basically a recorder and playback system, that just played back what Pokemon Gold would have displayed.
posted by JHarris at 9:53 AM on February 25, 2017 [1 favorite]

atoxyl: You might have been thinking of the video of SethBling. He manually injected a Flappy Bird clone into Super Mario World, live on Twitch and on real hardware.
posted by OMGTehAwsome at 7:45 AM on February 27, 2017 [1 favorite]