Join 3,520 readers in helping fund MetaFilter (Hide)


TRANSSUBSTANTIATIO
July 9, 2012 5:17 PM   Subscribe

1. Save sound file as raw. 2. Open raw in graphics editing program.
posted by brundlefly (69 comments total) 55 users marked this as a favorite

 
Huh, I have an Ikea duvet cover that looks just like that.
posted by These Premises Are Alarmed at 5:20 PM on July 9, 2012 [3 favorites]


Neat.

I used to do something similar by plugging line level audio into the composite video input of a monitor.
posted by werkzeuger at 5:23 PM on July 9, 2012 [2 favorites]


You should really watch Aphex Twin's Windowlicker go by in a spectrogram. He did something similar with Equation.
posted by mhoye at 5:29 PM on July 9, 2012 [6 favorites]


For years I have wanted to use image editing software to edit sound. I wonder what these would sound like if you shooped them and then played them back out as audio? And which effects would have what effects?
posted by Mister Moofoo at 5:30 PM on July 9, 2012 [2 favorites]


Mister Moofoo, let me introduce you to Photosounder.
posted by Lazlo at 5:35 PM on July 9, 2012 [16 favorites]


oh god yes! take the output of any of those and slap a...i dunno....diffuse glow filter on it. playback. what happens? I MUST KNOW
posted by lazaruslong at 5:36 PM on July 9, 2012


So Philip Glass is visually haunting too.
posted by ifandonlyif at 5:36 PM on July 9, 2012


So do I have to cross my eyes or un-cross my eyes in order to see the hidden picture?

Oh wait, I got it! The Philip Glass one is a series of dolphins jumping over power lines!
posted by not_on_display at 5:37 PM on July 9, 2012 [3 favorites]


Art.
posted by Mblue at 5:42 PM on July 9, 2012


This photosounder demo is waaaay cooler than I expected it would be. So great!
posted by Scientist at 5:55 PM on July 9, 2012 [8 favorites]


I mean I guess it basically takes the luminance channel of the image as a spectrogram, calculates the sound that would produce said band on that spectrogram, and then plays the resulting sounds as the bar swipes over the image? It's a really interesting lesson in how spectrograms work, I never really got them before.
posted by Scientist at 5:57 PM on July 9, 2012


I used to have a mac program called Sound Edit 16 and if would open pretty much any file. I remember opening a poem applying some effects then using this sound as a bed for the recording of the poem.
posted by cjorgensen at 6:03 PM on July 9, 2012 [1 favorite]


I wish there was more information on the source files used. I'm guessing a .wav will look very different to a .mp3 of the same song.
posted by TwoWordReview at 6:07 PM on July 9, 2012


what happens? I MUST KNOW

Soundgarden's Rusty Cage + sepia toning = Johnny Cash's Rusty Cage.
posted by jimmythefish at 6:13 PM on July 9, 2012 [6 favorites]


I hearby move that Photoshop's Add Noise command be named the Tom Waits Filter.
posted by JHarris at 6:15 PM on July 9, 2012 [7 favorites]


How long before the RIAA shuts this down?
posted by ZenMasterThis at 6:16 PM on July 9, 2012 [3 favorites]


That Photosounder has some...um...potential.

*mutters to self*

*dread silence*


Err....thanks!
posted by malocchio at 6:18 PM on July 9, 2012


music looks a lot like jupiter
posted by zennie at 6:19 PM on July 9, 2012 [3 favorites]


This is much more interesting than when my mouse slips and I open jpegs in notepad.
posted by pdq at 6:20 PM on July 9, 2012 [1 favorite]


You should really watch Aphex Twin's Windowlicker go by in a spectrogram. He did something similar with Equation.

Those two tracks are the same track. The second track from windowlicker is actually named:
         N
∆Mᵢ⁻¹=−α ∑ Dᵢ[η][ ∑ Fjᵢ[η−1]+Fextᵢ [η⁻¹]]
        η=1     j∈C{ᵢ}

but in id3 tags and other ascii-only media is usually just called "equation" or something.
posted by aubilenon at 6:24 PM on July 9, 2012 [6 favorites]


Yikes, Fleetwood Mac's "Go Your Own Way" is creepy. It looks like a face on a premium cable channel you haven't paid for.

Nice use of that John Cage piece for the website's background wallpaper.
posted by straight at 6:29 PM on July 9, 2012 [3 favorites]


This is a fun way to spend an hour if you have access to Matlab or some equivalent (or if you know C). You'll lose the rest of the afternoon when you realize that you can also open notepad.exe as a .png or turn your favorite painting into a .wav sound file.
posted by tylermoody at 6:30 PM on July 9, 2012


When I was studying electronic music, we learned to do the exact opposite of this.
posted by Relay at 6:31 PM on July 9, 2012


If you like these, I have an old black & white tv that constantly generates images like this, and even produces an ambient soundtrack to go along with it. And I'll let you have it for only $5000 $10,000.
posted by crunchland at 6:32 PM on July 9, 2012 [1 favorite]


Someone should scan some Magic Eye pictures and play them back.
posted by Sys Rq at 6:33 PM on July 9, 2012


In a way, I think this is like looking at something from the side. The most interesting thing for me is when different axes encode entirely separate information, like with those Aphex Twin tracks, or in the shadows of sculptures (previously), or even shapes on an oscilloscope.
posted by lucidium at 6:33 PM on July 9, 2012 [1 favorite]


Steganography! The next big menace for RIAA!
posted by mr.ersatz at 6:38 PM on July 9, 2012


If you like Photosounder then Metasynth is going to make your day. It’s expensive though. It’s been around since the 90’s at least and I think that’s what Aphex Twin used.

You can also edit the pictures like the ones in the link and re-import it as audio. You can even import any kind of file into an audio editor, but mostly you’re going to get a blast of noise so turn down your monitors. Many audio programs will do this, Audacity is free. Someone on KVR had a challenge a few years ago to use part of the actual program code to make tracks.

There is an audio unit plugin called MonaLisa that lets you apply graphics filters to audio, but it was a little buggy and I’m not sure if it’s still around.
posted by bongo_x at 6:41 PM on July 9, 2012


I think i prefer the opposite: images processed in audio editors.
posted by univac at 6:49 PM on July 9, 2012 [1 favorite]


Metasynth is hideously expensive. But if you have a Windows box, Coagula is free.
posted by anigbrowl at 6:50 PM on July 9, 2012 [1 favorite]


1. Save sound file as raw.

How? In what program?

2. Open raw in graphics editing program.

How? In what program?

I WANT TO DO THIS!
posted by Devils Rancher at 6:51 PM on July 9, 2012


How? In what program?

You should be able to save a soundfile as RAW in Audacity and open it in GIMP, I think.
posted by univac at 6:56 PM on July 9, 2012 [1 favorite]


Yep. Pan Sonic looks about as weird as I'd expect. Pity not to see any Sunn O))) on there...
posted by schmod at 7:08 PM on July 9, 2012


I once made a Java applet that did the opposite of this; it generates an image (with a simple fractal) and then turns it into audio. I was hoping for some glorious music-of-nature output, what I got sounded like a robot vomiting.
posted by AndrewStephens at 7:37 PM on July 9, 2012 [3 favorites]


1. Save sound file as raw.

How? In what program?


Most serious audio editors offer this optin - Sound Forge, Audition, Goldwave, Audacity...it's just another format option.

2. Open raw in graphics editing program.

How? In what program?


Any program that can open files from decent digital cameras, which save in RAW format by default. Photoshop, Picasa, whichever.
posted by anigbrowl at 7:37 PM on July 9, 2012


I'm a big fan of taking some grayscale perlin noise (e.g. Photoshop's Filter-Render-Render Clouds) and interpreting that as a sound. You can make it have a defined pitch if you make the width of the image in pixels equal to the number of 44.1kHz samples that a single wavelength should be (so, if you want something close to the note C3, make your image 44100/130.8 = about 337 pixels wide).

The cool thing about this is that each horizontal row in the image is fairly similar, but not identical to, the adjacent horizontal rows. This gives the resulting sound an evolving flavor.
posted by Jpfed at 7:49 PM on July 9, 2012 [1 favorite]


The Bartok came out nice
posted by idiopath at 7:52 PM on July 9, 2012


Took me a minute to find the settings in Audacity, but it worked. This is kinda cool.
posted by Devils Rancher at 7:55 PM on July 9, 2012 [1 favorite]


And which effects would have what effects?

There are a few correspondences that we can figure out right away. I speak Photoshop, so that's what this post will be in.

A single echo could be produced by duplicating the layer, then running an Offset filter on the duplicated layer, then giving the duplicated layer a reduced opacity.

You could give a sound some harsh, tinny distortion with a variety of the Image Adjust menu selections, especially Curves. Auto Levels will make the sound as loud as it can be with minimal clipping (the amount of clipping is an option you can set, defaulting to 5%)

You can do a lowpass filter with a Motion Blur at 0 degrees. Gaussian filters should be similar, BUT since they bleed information from one horizontal row to the next, they should give the audio a more tonal feel, with pitch dependent on the width of the image*. If you want the tonality without the lowpass, Motion Blur at 90 degrees.

*This assumes the pitch equivalent of the image width is within the human ranges of audio perception (about 2000 pixels wide or less). On wider images, I would expect it to sound like shitty reverb early reflections.

You could do a very interesting chorus effect (man, I should do this!) by saving off some rendered clouds (bonus points for rendering separate clouds in the red and green channels) into a separate file, then duplicating your layer, applying a Displacement Map to that (using your rendered clouds as your input file), then cutting the opacity to 50%.

I haven't tried any of this stuff out, but if you want to know why I think these would work as suggested, I can clarify that.
posted by Jpfed at 8:07 PM on July 9, 2012 [3 favorites]


For fun I decided to try going the other way and turning one of the images back into sound. I downloaded the Fleetwood Mac JPEG image and opened it up in GIMP, converted the image mode to greyscale and then saved it as an uncompressed bitmap. Then I renamed the file extension to .raw and imported it into Audacity as raw audio data using the default settings (16 bit signed PCM, stereo, 44kHz). The sample played backwards, because the BMP format stores image data bottom-up, so I flipped the image vertically and then re-imported it. It's only the first 5 and a half seconds, up to Stevie Nicks singing "Loving you...", and there is a bit of noise from saving it as a lossy JPEG and a little glitch at the start from the BMP header, but it worked!
posted by L.P. Hatecraft at 8:20 PM on July 9, 2012 [5 favorites]


jped: those things make sense if and only if the image format uses some multiple of 16 bits per pixel. Otherwise you risk phase differences compared to the audio data size (effectively swapping the high and low bytes, more specifically using the low byte from one sample and the high byte from the next, but in the wrong places).

also most image manipulations would introduce a frequency equivalent to the length of the image scan line
posted by idiopath at 8:22 PM on July 9, 2012


also, hatecraft's comment reminds me, the format sizes are so different that n offset i ter would have to be very wide to give an echo, and normal amounts of offset would just give a bandpass filter.
posted by idiopath at 8:26 PM on July 9, 2012


those things make sense if and only if the image format uses some multiple of 16 bits per pixel.

Nope, you just need a consistent bits/pixel and bits/sample between your image and audio apps. I usually work in 8 bit. You could even do it (with weirder results) if you used 8-bit/pixel and 16-bit/sample as long as the width is an even number of pixels, to avoid what you state next:

Otherwise you risk phase differences compared to the audio data size (effectively swapping the high and low bytes, more specifically using the low byte from one sample and the high byte from the next, but in the wrong places).

also most image manipulations would introduce a frequency equivalent to the length of the image scan line

Right, I alluded to this several times, e.g. Gaussian filters should be similar, BUT since they bleed information from one horizontal row to the next, they should give the audio a more tonal feel, with pitch dependent on the width of the image

also, hatecraft's comment reminds me, the format sizes are so different that n offset i ter would have to be very wide to give an echo, and normal amounts of offset would just give a bandpass filter.

Yes, some offsets will give a bandpass filter. The effect of any given offset (I meant to specify that it should be a vertical offset) depends on the width of the image. I don't know if it makes sense to talk about a "normal" offset; at the scale of the whole image, it's really really easy to get an offset that sounds like an echo. For example, if your image is 100 sample-equivalents wide (in 8 bit, 100 pixels) then an echo of around 250 ms is just 110 pixels down (assuming 44.1kHz).
posted by Jpfed at 8:37 PM on July 9, 2012


This photosounder demo is waaaay cooler than I expected it would be. So great!

Ah, so that's how they got the sound effects for the Hitchhiker's Guide to the Galaxy radio show.
posted by adamdschneider at 9:09 PM on July 9, 2012


I lol'd at the DNA gel, Scientist.
posted by en forme de poire at 9:27 PM on July 9, 2012


I downloaded Procol Harum's A Whiter Shade of Pale and opened the mp3 in Notepad to get
ÿúDm) ˜ 5`
? ¸
#(   œ¢Ù‘hŠ6¥Ö2Á€2 tTNÀêWçP·ñT…ªŒ¢Jª°eQ”IʧYÿÿÿë*ïÿÿ¥ddiõÑ»j†¾Ôÿ¨[ÿî¢ßÉÖ-P2—ÿÿý.F˜+#·ÿÿþ•  Ò)À‰4Ó!œ2“¨4gI‡æŒé0üѝ&š3¤ÃóFt˜~iÚL?4gI‡æŒé1 ÑœÉ‡ÐhÎdƃFxN‹¤Ùl –v~Y‰ýÿÿ»I~y’$›û¨I&³¥å³ wàÛö]Ÿt ª@í=C;áÇ ˜õ


Then I fed that text into at&t's text-to-speech generator and played the resulting sound file on my phone after dialing pizza delivery.

Right now I'm waiting for my food; an anchovy pizza with extra backslashes and a frisbee.
posted by twoleftfeet at 9:57 PM on July 9, 2012 [4 favorites]


twoleftfeet: ert+! Y76p?
posted by en forme de poire at 10:04 PM on July 9, 2012


I really can't express how excited I am to learn about Photosounder. I mean, I guess I should have known about it and other tools that do this, but I didn't and I've been idly thinking about this sort of thing for years and years (my days are spent idly thinking about stuff like this so there's lots of things I never follow-up about). So, yeah, I absolutely can't afford to buy Photosounder, but I think I just might do so tomorrow anyway.
posted by Ivan Fyodorovich at 10:10 PM on July 9, 2012


Metasynth is hideously expensive. But if you have a Windows box, Coagula is free.

Cool! Can't believe I'd never seen this until now.
posted by krinklyfig at 10:22 PM on July 9, 2012


univac - I think i prefer the opposite: images processed in audio editors.

Yes! I've been spending a ridiculous amount of time on the glitch_art subreddit lately, I can't get enough of this process.
posted by jason_steakums at 11:00 PM on July 9, 2012


Not entirely unrelated
posted by hattifattener at 11:13 PM on July 9, 2012 [1 favorite]


this totally makes me want to listen to that song, loud.
posted by 5_13_23_42_69_666 at 1:42 AM on July 10, 2012


"the Eye" was a midlet/s60/probably iOS app that sonifies live video in a similar way, with the aim of providing vision to the blind through sound.

Never really spent long enough with it to test its efficacy, nor was really prepared to blind myself either. Some of these other tools look fun to play with, though - thanks for kicking it off, Brundlefly!
posted by davemee at 2:05 AM on July 10, 2012


For fun I decided to try going the other way and turning one of the images back into sound.

I wonder how long until phone cameras have high enough resolution that you could stick a poster on a wall with one of these images so people could walk past, take a photo of it, and have that photo instantly converted into an mp3? Kind of like a QR code, but useful.

Could be quite cool, and lead to some interesting new graffiti/music piracy crime combinations.
posted by EndsOfInvention at 2:54 AM on July 10, 2012 [1 favorite]


(This reminds me of using Hilbert curves to visualise binaries.)
posted by ver at 3:36 AM on July 10, 2012


These remind me of the images that used to come back from the Soviet planetary probes.
posted by The 10th Regiment of Foot at 4:47 AM on July 10, 2012


For people interested in this kind of thing, a lot of glitch art uses sonification of images; there's a good tutorial on how to do it here. I've made some myself. Also, there was an album put out about ten years ago that's just data played as audio.
posted by rottytooth at 5:01 AM on July 10, 2012 [1 favorite]


This reminds me of the guy who could identify the music on a vinyl record with the label covered, just by looking at the grooves. He was tested by James Randi, who said "yeah he can do just what he says he can do." Here's a challenge for today!


> there was an album put out about ten years ago that's just data played as audio.

It also reminds me of the day when software was distributed on cassette. Stick Mavis Beacon's Typing Tutor in the cassette deck, listen to the voices in the endless static sea.
posted by jfuller at 7:03 AM on July 10, 2012


Also somewhat relevant, check out avante garde composer, Iannis Xenakis' amazing UPIC system, invented in the 1970's. Another apparent Aphex favourite.
posted by boogiefunk at 8:14 AM on July 10, 2012


for ((i=0;i>44100*273;i++)); do echo $'\000' >> 433.raw; done

posted by 7segment at 8:41 AM on July 10, 2012 [2 favorites]


I've since found that open sourced Iannix is UPIC's modern incarnation. Wow, it's capabilities look truly astounding.
posted by boogiefunk at 8:47 AM on July 10, 2012 [1 favorite]


Um, of course I meant < not >. That was not supposed to be so meta.
posted by 7segment at 8:51 AM on July 10, 2012


I've since found that open sourced Iannix is UPIC's modern incarnation. Wow, it's capabilities look truly astounding.

It's excellent, though somewhat fiddly. You need to hook it up with some other software if you want to do anything with MIDI; theoretically it has MIDI built in but it's not friendly. The documentation is atrocious, and it's somewhat unstable. but on the other hand, it's free and it's way more fun than something like Chuck or CSound.
posted by anigbrowl at 10:09 AM on July 10, 2012


IIRC csound is the default backend for iannix, you could just use the csound midi opcodes.

Or better yet, use the OSC opcodes in csound. And if your audio synthesis platform does not do OSC, you need a new synthesis platform.
posted by idiopath at 12:31 PM on July 10, 2012


used to do something similar by plugging line level audio into the composite video input of a monitor.

Did it work? Without the hsync and vsync pulses, I wouldn't think you'd get a picture of any kind drawn on the screen.
posted by Pruitt-Igoe at 1:39 PM on July 10, 2012


"Did it work? Without the hsync and vsync pulses, I wouldn't think you'd get a picture of any kind drawn on the screen."

That was my thought, too. (Well, after I thought, "I'm sure that I would have tried this, or done it accidentally, and so if it worked I'd know all about this because I would have been going nuts with it"...and then I thought, "huh, I don't think that could possibly work, anyway") but passed the comment over because maybe there was a bit more going on that he mentioned.
posted by Ivan Fyodorovich at 2:04 PM on July 10, 2012


This reminds me of the time my [then] boyfriend (now husband) took a picture of our friend's girlfriend, scanned it, and turned the graphic file into an audio wave, and used it in a song. The song was called "Scanhead".
posted by rabbitrabbit at 10:49 AM on July 11, 2012


Regarding composite video, I've done this too, and it does things (strange things). The key is that unlike VGA for example, all the data (h/v/pixel) is decoded from one wire, so what most random signals do may be arbitrary, but you will be able to get some weird stuff, and if you are generating the signal from a synth of some sort so you have low level control of its electrical proerties, you can make shapes happen (with VGA without any pulses sent to hsync/vsync you would just get a single dot on the upper left, at best).
posted by idiopath at 11:34 AM on July 11, 2012


And if your audio synthesis platform does not do OSC, you need a new synthesis platform.

There is not a whole lot of OSC hardware out there, as in any. I made a promise to myself at the beginning of this year that I was not going to originate any sound in the computer for the rest of the year. No plugins, nothing. MIDI isn't going anywhere for a while.
posted by anigbrowl at 12:01 PM on July 11, 2012


« Older The Aikiu's new music video "Pieces of Gold" cleve...  |  Last night at TNA iMPACT Wrest... Newer »


This thread has been archived and is closed to new comments