Greyscales?

December 14, 2004 3:10 PM

The vOICe: Seeing with sound {java} “...vertical positions of points in a visual sound are represented by pitch, while horizontal positions are represented by time-after-click. Brightness is represented by loudness. In this manner, pixels become... voicels!”
posted by Cryptical Envelopment (26 comments total)

it seems like the geek generation loves this sort of "take something mundane, map its characteristics to something unrelatedly mundane, to produce something newly mundane". why would i want to, say, listen to a synthesizer controlled by the random quivering of a dog's anus?
posted by quonsar at 3:18 PM on December 14, 2004

why would i want to, say, listen to a synthesizer controlled by the random quivering of a dog's anus?

Because you're an 'N Sync fan?
posted by mudpuppie at 3:25 PM on December 14, 2004

touche!
posted by quonsar at 3:25 PM on December 14, 2004

i think it will be scary/amazing when software/technology is built that will allow text strings to search out visual or sonic characteristics from underlying data in files posted online (e.g., 'b-flat minor 7; martin d-15 dreadnought; .mp3').

ok - this is my first post so i hope it's not blindingly stupid.
posted by wbm$tr at 3:27 PM on December 14, 2004

no, this is Cryptical Envelopments post.
posted by quonsar at 3:28 PM on December 14, 2004

Funny, I was actually looking for something like this after reading about an experiment in which researchers rewired ferrets' nervous systems so that visual signals got routed to the auditory part of their brains. (The ferrets still responded to visual stimuli, but there's the separate question of whether their brains' auditory areas had adapted to vision, or whether they were 'hearing sound,' as one might do with the applet linked in the FPP).
posted by greatgefilte at 3:43 PM on December 14, 2004

That seems to be what they're after, greatgefilte, but without surgery.
posted by Cryptical Envelopment at 3:50 PM on December 14, 2004

Ahhh. So it's not just for playing with your brain? They want to make blind people see. It could work.
posted by greatgefilte at 3:54 PM on December 14, 2004

...researchers rewired ferrets' nervous systems so that visual signals got routed to the auditory part of their brains

Sensory rerouting is not just for ferrets anymore. (Cache.)
posted by mudpuppie at 4:04 PM on December 14, 2004

Sensory rerouting is not just for ferrets anymore.

Whoa, these are not at all the same things. Bach-y-rita's device uses an array of tactile stimulators on the tongue; Sur and his students actually did surgery on little ferret brains to change the destination of the optic nerve from visual to auditory cortex.
posted by paul! at 4:19 PM on December 14, 2004

Lucky for the subjects in the former's experiment, no?
posted by mudpuppie at 4:24 PM on December 14, 2004

I think this is pretty cool.
posted by interrobang at 4:32 PM on December 14, 2004

Brain surgery - at least to insert implants - has been used on humans for this kind of thing though.
posted by paul! at 4:46 PM on December 14, 2004

So do I. quonsar is a curmudgeon.
posted by neckro23 at 4:50 PM on December 14, 2004

Incidentally, though, Bach-y-rita is the grand old man of research on technologies like this that translate stimulation in one modality (like sight) into another (like hearing), a type of thing that's known in the field as "sensory substitution." He's been designing systems like this with the goal of helping out disabled people for decades now.

The vOICe is also being taken seriously enough by some researchers to be used as a tool to investigate perceptual learning and plasticity. A woman in the lab I'm in just finished a thesis including some experiments using the vOICe (so far unpublished, unfortunately).

Also, a plenary session at the 2002 "Toward a Science of Consciousness" conference focused on sensory substitution, with the vOICe and the Sur ferrets as the two main topics.
posted by paul! at 5:03 PM on December 14, 2004

why would i want to, say, listen to a synthesizer controlled by the random quivering of a dog's anus?
…when you could have, say, MetaFilter with noobies.
posted by weapons-grade pandemonium at 6:16 PM on December 14, 2004

Give quonsar a break. He know's what he doesn't like.
posted by Cryptical Envelopment at 6:42 PM on December 14, 2004

Cool, it's a free version of Metasynth! Too bad it's written in Java, but I'm sure some people will manage to have fun with it anyway.
posted by Mars Saxman at 6:45 PM on December 14, 2004

Oh, I see.
posted by rafter at 6:52 PM on December 14, 2004

Loosely related and oh-so-cool little app.
posted by rafter at 6:59 PM on December 14, 2004

Too bad it's written in Java, but I'm sure some [blind] people will manage to have fun with it anyway.

I sure hope so!
posted by Cryptical Envelopment at 7:18 PM on December 14, 2004

That was pretty cool, but (in my opinion) it can't hold a candle to Amit Pitaru's Sonic Wire Sculptor App. Install the necessary java plugin, strap some decent headphones to your noggin, and give it a try. Highly recommended.
posted by Rattmouth at 9:24 PM on December 14, 2004

it can't hold a candle to

What is a candle and how am I supposed to see it? If I am blind?
posted by Cryptical Envelopment at 9:34 PM on December 14, 2004

quonsar is destructive. :-(
posted by Cryptical Envelopment at 9:56 PM on December 14, 2004

This reminds me of a scary trick Aphex Twin pulled off once...
posted by swordfishtrombones at 11:06 PM on December 14, 2004

Tom Waits is on in your side.
posted by Cryptical Envelopment at 12:12 AM on December 15, 2004