Everything but the clouds
November 20, 2017 5:42 AM   Subscribe

Cory Arcangel (previously) describes his artwork/Super Mario Bros ROM hack "Super Mario Clouds" as "an old Mario Brothers cartridge which I modified to erase everything but the clouds." Except, as Patrick LeMieux discovers when reverse-engineering the ROM, "Arcangel’s ROM hack does not actually contain Nintendo’s ROM". There was no erasure. This video documents Patrick's analysis of Arcangel's ROM and his own attempt to erase "everything but the clouds".
posted by EndsOfInvention (44 comments total) 30 users marked this as a favorite
 
It's been said that those who can't do, teach.

Yeah, well. Those who can't teach, waffle.
posted by flabdablet at 5:55 AM on November 20, 2017


I don't know whether LeMieux quite gets it and he certainly adds an egg or two too many but it's still interesting and funny.
posted by hawthorne at 6:25 AM on November 20, 2017


And lo, for the navel was gazed upon, and it was full of lint.
posted by grumpybear69 at 6:47 AM on November 20, 2017 [6 favorites]


Mess ye not with the post-modern
With its tropes so limp and sodden
Tarry ye not with pomo
Lest your brains go into slo-mo.

Although I was reminded quite strongly of the PKD short story where a man discovers he's a machine with his perceptions coded onto an internal tape, and sets about editing out... in the end, everything. That has the benefit of an identifiable narrative, unlike this semi-stoned ramble through conceptual woodlands. Hello sky! Hello clouds! Hello hex!

There is a lot of fun and thinking to be had by sly hackery of video games, and I'm confident that we'll get the first genuinely new art form since the cinema when people get to grips with digital spacetime - all the signs are there. That needs forward thinking and imagination, something that this piece can provoke but seems unaware of in its studiously retrospective stance.
posted by Devonian at 7:22 AM on November 20, 2017


I don't know about the art, but I do appreciate the soft g-gif.
posted by uncleozzy at 7:25 AM on November 20, 2017 [1 favorite]


I am unequipped to engage with the critical-theory aspects of the video. Insufficiently trained in art.

But the fact-checking at the basis of this work - Arcangel said he did a thing, when he did not do that thing - seems to be...underrepresented? obscured?

Is someone proving conclusively you didn't do the thing you made your name with not the deal breaker in Art it would be in other cultural spaces?
posted by Fraxas at 7:32 AM on November 20, 2017 [1 favorite]


I don't know about the art, but I do appreciate the soft g-gif.

I didn't realize Nintendo used the Giraffics Interchange Format.
posted by leotrotsky at 7:36 AM on November 20, 2017 [8 favorites]


That's not how acronyms work. It's not the Jay-oint Pee-hotographic Experts Group either, but it is jay-peg.
posted by Dysk at 7:41 AM on November 20, 2017 [10 favorites]


Is someone proving conclusively you didn't do the thing you made your name with not the deal breaker in Art it would be in other cultural spaces?

I'm no artist, but I think the point of the original artwork was not the challenge that he had accomplished (it wasn't "look, I've done this very difficult ROM hack no-one has done before!"), it was the end result ("This is what Super Mario Bros looks like with only the clouds"). Like if someone said "I went to Paris and painted this picture of the Eiffel Tower" but it turned out they'd actually painted it from a photograph while in New York (unsure if this analogy works - but the point is at the end of it they still have a nice painting of the Eiffel Tower). He was presenting it in an art gallery, where people come to examine the final product, rather than presenting it to the ROM hacking community who, I expect, care somewhat more about the process for getting there.

I wonder if Arcangel had had the same issue this guy had (he couldn't remove the final coin icon) or if he'd thought it was easier just to code it himself right from the start.
posted by EndsOfInvention at 7:57 AM on November 20, 2017 [2 favorites]


> It's not the Jay-oint Pee-hotographic Experts Group either, but it is jay-peg.

It is not jay-peg either, it is pronounced j'peg. Inserting vowels into the acronym is obviously changing the meaning of it and creating a revisionist history of modern technology. And it must be pronounced in an italicized way, not slanted or bold to preserve information integrity.
posted by ardgedee at 8:07 AM on November 20, 2017 [7 favorites]


I really enjoyed this. Thanks!

Arcangel said he did a thing, when he did not do that thing - seems to be...underrepresented? obscured?

Is someone proving conclusively you didn't do the thing you made your name with not the deal breaker in Art it would be in other cultural spaces?


Heh. The fun thing here is that Arcangel's claim can be refuted by the instructions Arcangel provides also as part of the "work". So he is both lying and providing evidence for anyone who follows his process/instructions to see that is the case as part of the same piece. How one wants to address those two elements, the explicit claim and the tacit disavowal, is something the video did rather well I thought in connecting Arcangel's piece to that of other artists. Whether the Whitney would agree is an open question.

For me, Arcangel's piece, via LeMieux, provides a fascinating little example of the tension between concept and display values in modern art. Arcangel's instructions, from what I could gather, provide the evidence that he could erase everything but the clouds save for sprite zero, the coin. For purposes of display and visual satisfaction, for which he would be remunerated, he made a "false" copy that better fit the concept of erasure for anyone uninitiated in the processes involved to see. His construct then differs from Super Mario Bros. making it more a replica but one which is essentially invisible unless studied in detail as LeMieux does here.

So, following LeMieux's mention of Warhol, Arcangel created Campbell Soup Cans that appeared to be constructed from the same materials and processes as the real soup cans to the naked eye, but turned out to have been hand crafted using slightly different materials on detailed inspection. The variation is, for those who care about such things I guess, interesting for what it suggests about virtual art versus material works and as simply a new layer to the now old pop art concept. In that sense, it does matter that what Arcangel says doesn't fit what he did, but how it matters is more open to interpretation.
posted by gusottertrout at 8:20 AM on November 20, 2017 [13 favorites]


In other words, the artifice is part of the art. While some in the art community may have taken it at face value, without a statement from the Whitney it's hard to determine whether they were "fooled." It's also maybe beside the point.
posted by fedward at 8:33 AM on November 20, 2017 [1 favorite]


Right, he shows he did, or could do, what he said, save for removing the coin, but then made a "clean" display version to sell and took that coin.

*Bling*
posted by gusottertrout at 8:43 AM on November 20, 2017 [2 favorites]


That was really interesting, thanks. I don't play video games or care all that much about conceptual art (or whatever you call "Super Mario Clouds"), and I don't normally watch twenty-minute videos, but I enjoyed that a lot, because LeMieux makes everything clear even to someone as ignorant as I. It is definitely not a "semi-stoned ramble" (did you watch the whole thing, or just give up after the odd beginning, which irritated me as well?). Arcangel lied about how he made what is apparently a famous artwork, and LeMieux shows exactly how he lied and provides a plausible explanation for why he didn't do what he said he did (that damn coin). And he's very familiar with the history of art and radical thought (I loved the line "Apparently all that is solid does not melt into air"). If you want to settle for rote hot-take snark, well, that's a MeFi tradition, but if you're actually interested in the ideas involved, I suggest watching the link.
posted by languagehat at 8:56 AM on November 20, 2017 [9 favorites]


This was great, thanks
posted by kjh at 9:35 AM on November 20, 2017


So the whole sprite zero thesis - that the game's loop is dependent on collision detection being activated on the flashing coin icon - does not explain why he couldn't make the coin smaller or a different color or otherwise make it non-transparent yet invisible (or less visible). So, as there was artifice and deception involved with the original Super Mario Clouds, the same can be said for this piece. I call BS on the coin being completely immutable and suspect that it remained unchanged more for the purposes of strengthening the author's angle relating to the fundamental corruptive power of money.
posted by grumpybear69 at 9:37 AM on November 20, 2017 [2 favorites]


Tell you what, why don't you try it yourself rather than accuse the author of deception on absolutely no evidence except your vague suspicion?
posted by languagehat at 10:24 AM on November 20, 2017 [3 favorites]


This is really excellent. I believe LeMieux when he says that he couldn't get rid of the coin, but in reality it doesn't even matter if he's telling the truth or not. After all, Arcangel wasn't telling the truth about his construction of the piece either - so if LeMieux is lying about the immutability of the coin, that's part of the meta-artistry here. (I love the fact that the un-erasability of the coin is so obviously weighted with deeper meaning.) Whether the story is true or not doesn't matter. What matters is that this is an artwork with a great fucking story.

It's also striking to me that Arcangel references Brand Nubian in his annotated code, because there's a lot here that parallels sampling in hip-hop. This is akin to someone making a rap track using a well-known sample, only for it to come out after the fact that they had painstakingly recreated the sample from scratch instead of directly sampling it. Great stuff.
posted by Frobenius Twist at 10:46 AM on November 20, 2017 [3 favorites]


It is not jay-peg either, it is pronounced j'peg.

Except that in for example Danish, it is still pronounced jay-peg, even though that is not how the Danish letter J is pronounced.
posted by Dysk at 10:49 AM on November 20, 2017


In Row Dyelun, it's pronounciated "jaaayyyy-pig" - we just think we're pronouncing it jay-peg.
posted by Slap*Happy at 11:05 AM on November 20, 2017 [1 favorite]


Tell you what, why don't you try it yourself rather than accuse the author of deception on absolutely no evidence except your vague suspicion?

Or I can just point you to these pages which clearly indicate what can and cannot be done with Sprite Zero.

If you're don't want to RTFL, here's one of the key scenarios in which Sprite 0 will always be hit:

The palette. The contents of the palette are irrelevant to sprite 0 hits. For example: a black ($0F) sprite pixel can hit a black ($0F) background as long as neither is the transparent color index %00.

To summarize: the coin does not need to be visible, or shaped like a coin, or change colors in order for the scroll event to be triggered. So, yes, deception. But what else would anyone expect from art-as-research? The end result has to be compelling and have zing - truth is not important.
posted by grumpybear69 at 11:31 AM on November 20, 2017


j'peg

K'plah.

gesundheit
posted by Naberius at 11:55 AM on November 20, 2017 [2 favorites]


And here, languagehat, I went and changed the coin into a bunch of zeroes and confirmed that scroll still works.
posted by grumpybear69 at 12:11 PM on November 20, 2017 [4 favorites]


You seem to think that "deception" is a bad thing. It looks like LeMieux isn't telling the truth, but that's irrelevant! In fact, if he's lying, I like what he did even more, because it reinforces the metatheme with the original artwork.

More generally, it seems that you don't like performance art, which is fine! There are a lot of things I don't like also! For example, I hate sports - but I don't go into sports threads and start arguing. If you don't like creative deception then you don't like performance art, and that's totally okay - but there's no need to be all fighty with those of us who do.
posted by Frobenius Twist at 12:31 PM on November 20, 2017


Not trying to fight with anyone. I just got called out specifically for making what lh perceived to be a baseless and mean-spirited accusation. I was told to try it myself, which I did. More to the point, a video like this is presented like technical fact, and particularly because it is debunking the foundation of another piece of technical art it seems on-topic to critique the critique. I understand why LeMieux steered the conclusion where he did, and honestly I wouldn't have given it much thought had he not jumped straight from "I can't get rid of the coin" to "the coin is symbolic of capitalism etc. etc. etc." That made me extremely suspicious. It is a testament to how compelling LeMieux's conclusion is that tempers are flaring around the idea that it may not be based in truth.
posted by grumpybear69 at 12:54 PM on November 20, 2017 [3 favorites]


cloudgate
posted by Sebmojo at 1:10 PM on November 20, 2017 [1 favorite]


> It is a testament to how compelling LeMieux's conclusion is that tempers are flaring around the idea that it may not be based in truth.

Excellent work! Now let's start on the ontology of truth!
posted by ardgedee at 1:22 PM on November 20, 2017


j'peg

alliterates with j'accuse
posted by thecaddy at 2:11 PM on November 20, 2017


Can we leave The Bean out of this?
posted by art.bikes at 2:14 PM on November 20, 2017 [1 favorite]


I'm a programmer and not in any sense an "art" person, but I liked this.

As I see it, the original work claimed to take Super Mario Brothers and erase everything but the clouds. But his isn't what it did, rather it took pieces from the original source material and re-constituted them. Collage rather than erasure, I guess.

This piece does erase everything but the clouds, as far as it can, for a specific meaning of "erase". Both pieces use an unmodified sprite ROM, so changing the content of sprite 0 isn't possible. Otherwise, he could just set every sprite to invisible and be done with it. Instead, by modifying only the game code, he fast forwards to an appropriate cut scene, removes the cut scene's end condition, and removes the instructions to draw everything but the background, clouds, and that damn coin. To my mind that's a far purer expression of "erase everything but the clouds" than the original work is, and not invisibling the coin is also required per the original stated condition of using the unmodified sprite ROM.

+1 would Art again.
posted by russm at 2:41 PM on November 20, 2017 [2 favorites]


> And here, languagehat, I went and changed the coin into a bunch of zeroes and confirmed that scroll still works.

Thanks! Sorry if I sounded, er, grumpy, but you did seem to be shooting from the hip. I respect your evident knowledge and ability.
posted by languagehat at 3:39 PM on November 20, 2017 [3 favorites]


I remember this piece being such a Big Deal at the time -- super super crowded gallery show, and then the later show at the Whitney.
posted by armacy at 7:12 PM on November 20, 2017 [1 favorite]


I am reminded of the time when Frank Zappa wrote a complicated piece for a chamber ensemble but there wasn't time to rehearse, so he got the musicians to mime along to a prerecorded Synclavier version. No one noticed, not even the music reviewers. (Incidentally the piece was called "While You Were Art II")
posted by RobotVoodooPower at 8:20 PM on November 20, 2017 [1 favorite]




That's quite interesting. Point:
Paolo Pedercini
🌹‏Verified account @molleindustria
15h15 hours ago

Cory has been exhibiting it as video, made an ironic "hacker" documentation of it, made a career around his defiance of media art formalism. It's pretty clear that he doesn't see the work as software or cartridge hack but as a visual piece with a good story wrapped around it.
Counterpoint:
Sarah Schoemann‏ @schoemannator
14h14 hours ago

yeah but then like, what was all the documentation and tutorials on how to DIY it about?
I have the same reaction as Sarah, but I guess the answer would be "that's all part of the good story."
posted by languagehat at 5:37 AM on November 21, 2017


Upon further investigation and experimentation, it appears that the "immutable coin" from the video is actually a combination of a background tile ($2E at PPU memory offset 0x00206b) and the now infamous Sprite Zero, which in isolation looks like the bottom sliver of a coin. The tile sits on top of the sprite and triggers the collision event which SMB then uses to drive its game interrupt (that's the main game loop which handles all of the timing, including music, scrolling, etc.). If you change offset 0x00206b to a tile which does not overlap with sprite zero, the game stops. For this reason the one blank tile ($24) - which could in theory obscure sprite zero to no ill effect, were it made of solid background-blue pixels - is of no use, since it is completely empty.

So, as russm noted, without modifying the ROM's graphics set, it is true that you cannot entirely erase the coin, as the small sliver that comprises Sprite Zero needs to be in contact with another pixel to drive the game, and there are no graphics defined that would both trigger a collision and obscure the sprite. The best that can be done is to replace 0x00206b with a black square ($27) or a small glowing dot ($AF).

The use of Sprite Zero's collision detection to drive game timing is a pretty fascinating mechanism, because the X position of Sprite Zero determines when the hit will be triggered - the smaller the X value, the earlier in the draw process the hit will be recorded. So if, as part of the game loop, the position of Sprite Zero oscillated from left to right, the timing of the entire game would oscillate with it.

Thank you, languagehat, for sending me down this path!
posted by grumpybear69 at 6:54 AM on November 21, 2017 [3 favorites]


The use of Sprite Zero's collision detection to drive game timing is a pretty fascinating mechanism, because the X position of Sprite Zero determines when the hit will be triggered - the smaller the X value, the earlier in the draw process the hit will be recorded. So if, as part of the game loop, the position of Sprite Zero oscillated from left to right, the timing of the entire game would oscillate with it.

Which comes back to the digital spacetime concept. If you're processing a list and triggering an event dependent on a list item's status - whether the list scan is tied to video scan timing or some other internal clock - then the distance down the list equates in some more-or-less complex way to that timing. You can go plootering down that rabbit hole quite a long way when you try and draw parallels with Einsteinian causality and frames of reference.

Program branches and jumps sort-of equate to reference frame switches that aren't allowed in real spacetime, although they're still limited by the cycle time of the hardware (in a synchronous digital system), so that's an added traversable dimension, but when an interrupt's triggered by a linear process you get back your distance/time equivalence, at least in some sense.

Perhaps I should revisit this thread next time Dr Hofmann lends me his bicycle.
posted by Devonian at 8:34 AM on November 21, 2017 [1 favorite]


The whole idea of erasure is inapplicable to 6502 machine code.

Every location in that program ROM is going to contain a value, because that's just how ROMs work.

If you apply an actual hardware erase operation to the value of a ROM cell, all that happens is that the cell contents get forced to be 0xFF.

0xFF is not a valid 6502 machine code instruction. Erasing parts of the code ROM is therefore not going to yield workable code.

In order to modify the behaviour implemented by the code ROM, then, it is necessary to change, not erase, some or all of its contents.

All this argument about what can and cannot be done with Sprite Zero assumes that SZ has to stay onscreen because its presence there is what triggers the existing scrolling code to work. But I see no virtue at all in re-using the existing scrolling code if its reliance on Sprite Zero being displayed renders it unfit for the new design's purpose.

If Cory Arcangel made the call that the most straightforward way to make a NES display a field of endlessly scrolling Super Mario Clouds with nothing else onscreen is to write a small bit of code that does just exactly that, put that code in a ROM, and put that ROM in a cartridge where the original program ROM used to be, then

SO FUCKING WHAT?

He left the original graphics ROM in place, and replaced the code ROM with one that makes the NES do what he wanted it to do. He documented his process. The result is that everything but the clouds has been removed (or "erased", if you insist that this is a distinction that makes a difference) from the resulting screen display.

What does the "he didn't really do it by erasing anything" crew want him to have done? Not replaced as much of the existing program ROM contents as he did?

What a tempest in a fucking teapot. And these arty farty types wonder why we engineers find their endless recondite waffling to be such a waste of time. Feh.
posted by flabdablet at 10:29 AM on November 21, 2017


But wait! There's more! I took a look at Doppleganger's disassembled SMB code (the same source used for this video) and lo and behold, the palette data is in there. Because SMB uses a different set of palettes for each level type (overworld, underworld, underwater, castle, etc.) they need to be swapped in and out of PPU RAM. As a result they are stored in the PRG ROM, and are fair game.

So! By modifying the overworld palette which is applied to the coins to make one of the colors the same as the sky color, it becomes possible to place a solid sky-blue block over sprite zero to achieve complete erasure while maintaining normal gameplay. Check it out.
posted by grumpybear69 at 1:40 PM on November 21, 2017 [1 favorite]


I'm prepared to be impressed by a SMC variant that remains totally playable (as evidenced by the sounds it makes in response to control inputs from a seasoned player who remembers how it's supposed to go) but in which nothing at all but clouds is visible.

That would be a cool hack.
posted by flabdablet at 2:31 PM on November 21, 2017


The whole idea of erasure is inapplicable to 6502 machine code.


What do you mean by 'erasure', though? In physics, you can never erase information, only rearrange it - with a possible exception for black holes, although that's very much an ongoing debate.

Erasure in computer terms generally means replacing some data with other data. Erasing a hard disk means writing some pattern to it, erasing a memory buffer means resetting pointers and perhaps overwriting it with a new pattern.

When a processor fetches data from a bus, it has to be some combination of 1s and 0s, and even if the bus is in a non-1 and non-0 state (ie, has a voltage on it that's outside the bus spec), the processor will still read it as a 1 or a 0 due to signal conditioning on its inputs. (Or it could blow up, if you've bunged 240 volts on a 5 volt bus: ask me how I know). Even if you tri-state a bus on read to make it high impedance and incapable of telling the processor anything, something will be read on fetch, whether it's because of pull-up or pull-down resistors on the input lines or just noise pulling a comparator or a hysteresis latch one way or the other. Processors are designed not to let undefined states into the house; they do not exist in their universe.

So one possible conceptual spin of this work is to investigate what we mean when we talk about erasure in computing terms - it is a fuzzy term and you can get into serious trouble if you don't actually know what happens, so that's a valid thing to explore. You might think you've erased that dodgy data from your SSD while the SSD has just remapped the sectors to reduce write-cycle degradation, which the feds might find interesting when they confiscate your gear. So I don't think you can say that 'erasure' is an inapplicable term in this context.

And these arty farty types wonder why we engineers find their endless recondite waffling to be such a waste of time.

Even stuff that pisses you off might leave you with a different way of thinking about things which can pay off in real gonzo engineering. Creativity sparks off the oddest things, but only if you let them in. We are not processors; badly defined states can do us good.
posted by Devonian at 9:29 AM on November 22, 2017 [1 favorite]


you can never erase information, only rearrange it - with a possible exception for black holes, although that's very much an ongoing debate.

Now settled, I hear.
posted by flabdablet at 11:45 AM on November 23, 2017


"Arcangel’s ROM hack does not actually contain Nintendo’s ROM". There was no erasure.

Whatever reasonable meaning one assigns to "erasure", the only way that there could have been no erasure of Nintendo's ROM set would have been for Arcangel's modified ROM set to be identical with the Nintendo original i.e. not modified at all.

Creativity stimuli are all very well, but this particular objection to Arcangel's work is not even wrong.
posted by flabdablet at 11:55 AM on November 23, 2017



Now settled, I hear.


I'd call it fashionable, interesting and seductive, but hardly settled. The enjoyable jousting between Susskind and Hawkin may be over; jaunty discussion is ongoing elsewhere.
posted by Devonian at 3:43 PM on November 23, 2017 [1 favorite]


« Older In the Land of Vendettas That Go On Forever   |   His iron-clad fleet flowed forward Newer »


This thread has been archived and is closed to new comments