The Doors of Perception

August 14, 2006 7:51 AM

We’ve detected background radiation from the Big Bang. We’ve sent explorers to the bottom of the ocean and the moon above us. We have images of the individual atoms of which our world is made. But we cannot have direct access to the sensory experiences of another human being. Language can help to bridge the gap but it is an imperfect tool. The closest we have come is Brain Fingerprinting and even that only indicates recognition of a scene or object; it does not capture the actual visual memory of the scene or object. This may soon change. Several years ago, researchers at Berkeley wired a cat’s neurons to a computer and were able to obtain videos of what the cat was seeing.
posted by jason's_planet (50 comments total)

That's a very, very big deal.

It is not likely, however, that the cat survived that particular test. Doing this is one thing... doing it non-invasively is quite another.
posted by Malor at 7:57 AM on August 14, 2006

Could this be the first step towards dream recording and playback?
posted by Faint of Butt at 7:59 AM on August 14, 2006

This is awesome. It reminds me of The Final Cut.
posted by Pastabagel at 7:59 AM on August 14, 2006

The first step towards this movie becoming a reality.
posted by jonson at 8:00 AM on August 14, 2006

If only this technology could be extended to the other senses.

Because somewhere, someone is enjoying a delicious pie.
posted by Pastabagel at 8:03 AM on August 14, 2006 [1 favorite has favorites]

"But we cannot have direct access to the sensory experiences of another human being."

That hasn't changed with this old experiment, and it won't, in the future. It's still *You*, looking at the monitor showing the cat's extrapolated visual field. You can't eliminate your self-contributions to your perceptions.
posted by Gyan at 8:06 AM on August 14, 2006 [1 favorite has favorites]

Whoa. [this is awesome]
posted by headspace at 8:06 AM on August 14, 2006

Lets move quickly to record all cat experience to computer.
Onward, science!
posted by shanevsevil at 8:07 AM on August 14, 2006 [1 favorite has favorites]

Can't have a post like this without mentioning Nagel's "What is it like to be a Bat?" (PDF)

I still don't think we'll ever touch the immediacy of experience.

But philosophy aside, great news for science. Bring on the cyborgs!
posted by cowbellemoo at 8:07 AM on August 14, 2006

This is not that surprising. People have been able to do experiments to measure visual neuron output for quite some time, and figure out what exactly triggered those neurons.

So the only thing these people have really accomplished was doing a lot of these probes all at once (and, probably, using some computer software to figure out how the signals corresponded to what was in the cat's visual field)

Impressive from a technological standpoint, but not from a scientific one.

It would be more exciting if they figured out a way to play the video back. Shouldn't be to hard based on the technology.
posted by delmoi at 8:08 AM on August 14, 2006

I'd like to see kind of the opposite. Input a signal from some other source than your eyes into your brain.
posted by joecacti at 8:09 AM on August 14, 2006

I can't help but think of this movie.
posted by ninjew at 8:10 AM on August 14, 2006

On July 14th 2049 we'll probably see a post to metafilter about a site where people take pictures of computers using their cat.
posted by edd at 8:12 AM on August 14, 2006 [5 favorites has favorites]

The article is from Monday, October 11, 1999. I wonder what they've been up to since then.
posted by stopgap at 8:14 AM on August 14, 2006

Researchers are watching you masturbate through ceiling cat's eyes!

(This is indeed cool!)
posted by TwoWordReview at 8:22 AM on August 14, 2006 [1 favorite has favorites]

Gyan++

The image does not represent what a cat sees. It represents what some humans imagine a cat might see: this informal specification is encoded in the measurement apparatus/image processing/image display gestalt.
posted by sonofsamiam at 8:25 AM on August 14, 2006

I can't believe nobody has mentioned Brainstorm. It clearly has more cowbell than the other movies listed.
posted by Mr_Zero at 8:28 AM on August 14, 2006

All our bloody kittens see are our nice new leather chairs as a good place to sharpen their claws ...
posted by dickdotcom at 8:30 AM on August 14, 2006

The seminal film take on this idea was also Natalie Wood's last film before she drowned after falling off Robert Wagner's yacht.

On preview, Mr Zero has beaten me to it.
posted by CynicalKnight at 8:35 AM on August 14, 2006

While I agree that you'll never "see" things exactly like another being "sees" them, it's not implausible to think you'll get a rough estimate. All emotions, feelings, perceptions can be quantified as discrete activity in the brain, whether or not we fully understand that activity at this stage of scientific research. There's no reason to believe that isolating more sensory experience and developing the capacity to replay it won't come about with more research.

That said, I still agree with Gyan and sonofsamiam, in that every brain develops independently and uniquely. I'm not convinced, even capturing all sensory data from a brain and replaying it to another, that it wouldn't still be a different experience through the filtering process of that particular brain.

Oh, and as long as we're mentioning movies, I'm going with Walken.

On preview, damn you CynicalKnight and Mr_Zero! :D
posted by Brak at 8:41 AM on August 14, 2006

I for one don't even know what overlord to welcome as a result of this experiment.
posted by Astro Zombie at 8:43 AM on August 14, 2006 [1 favorite has favorites]

Cold Lazarus

Isn't it then just a short step to actually inserting images and stuff into our heads? (which is what's really terrifying) Won't they be able to record and distinguish exactly which and what kind of impulses are generated and sent to which neurons to form an image or false memory or something?
posted by amberglow at 8:49 AM on August 14, 2006

All our bloody kittens see are our nice new red Colombian leather couches as a good place to sharpen their claws ...
posted by signal at 9:03 AM on August 14, 2006

Who's this "they" you're talking about, AmberGlow? Paranoid much?

Chances are it'd be more like something one might pay for. If you want to have the memory of having been somewhere you can't afford to go, maybe getting it totally imprinted in your head would be the next best thing to having been there. Or maybe you'd want to remove unpleasant memories from your spotless mind. I doubt the gov't would be all that interested in treating everybody's grey matter. That'd be expensive. Better just to do it to the aristocratic elite. For profit.

If one wanted to know how to play golf well, or use a surfboard to impress the ladies, or be an olympic snow skier, they'd just implant the memories of a champion in those fields. Don't wanna go to class next Thursday? Your more studious buddy could go and then copy his experience to a disc that you later download into your brain while you sleep the night before the test. In fact entire collegiate degrees could be uploaded into a person's mind in a manner of hours or days instead of years. This would of course make the minds of those who actually experience these things highly sought after, until after their memories were copied onto a disc, which could then be copied a million times and sold to the masses for whatever the economy will bear. Editing the memories of those with such knowledge so they'd be compatible with those who could afford it, well that'd become a sort of cottage industry.

Scary? No more or less scary than any other technological breakthrough. Granted, there've been some pretty scary technological breakthroughs in the past century. This would just be another one of those.
posted by ZachsMind at 9:39 AM on August 14, 2006

Another fictional take on this science.
posted by NationalKato at 9:45 AM on August 14, 2006

I'd actually be more interested in perceiving what our cats are hearing.

Then I could prove to the fuzzy little bastards that yes, they can hear their own names just as well as they can hear a can opener.
posted by yhbc at 9:46 AM on August 14, 2006

I've been doing much more advanced research in my lab. Check out my picture of what my dog sees. Note that my thermodowhatsit technology overlays his mental reactions to environmental stimuli over objects he perceives.

">
posted by illovich at 9:50 AM on August 14, 2006 [2 favorites has favorites]

Next you'll be able to feel your partner's orgasm and the government will be able to suppress your homosexuality!
posted by Captaintripps at 10:09 AM on August 14, 2006

I have ALWAYS wanted to know what the world looks like to other species. This is fascinating and I'm very excited.

Thanks for this link!
posted by agregoli at 10:33 AM on August 14, 2006

What's the cat equivalent of a homunculus^? A feliculus? Ridiculus?

You do see how silly this is, trapping the nerve activity of a cat, manipulating it to look like a photograph and then imagining a ghostly little cat inside its own brain looking at that resulting picture? It explains exactly nothing. Think what would happen if they did it to your optic nerve. This smudgy, static, flat, black and white picture is what agregoli 'really' sees.
posted by grahamwell at 10:43 AM on August 14, 2006 [1 favorite has favorites]

homunculus^

the carat is just too precious to waste on wikipedia
posted by caddis at 11:30 AM on August 14, 2006

It is not likely, however, that the cat survived that particular test

I am sad to say that I have to agree with you.

Could this be the first step towards dream recording and playback?

Maybe. Me, I wonder what the web would look like if we could upload these images. MyHeadSpace? MyConsciousness?

So the only thing these people have really accomplished was doing a lot of these probes all at once (and, probably, using some computer software to figure out how the signals corresponded to what was in the cat's visual field)

I think that wiring 177 individual neurons, measuring their activity, and creating a video from that output is mighty damned impressive. I’m sorry that you don’t share my enthusiasm.

The article is from Monday, October 11, 1999. I wonder what they've been up to since then.

Garrett Stanley

Yang Dan

Not sure about Fei Lin. Google-Fu failed me.

I agree with the philosophers here; we would probably never have direct access to the sensory experiences of another. We would probably wind up supplementing language with another set of imperfect tools.
posted by jason's_planet at 12:50 PM on August 14, 2006

Huh?
posted by homunculus at 12:56 PM on August 14, 2006 [1 favorite has favorites]

Is it just me, or does the "cat-sampled" version of the man's face look a lot more cat-like than the original?
posted by Ironwolf at 12:58 PM on August 14, 2006

Apparently my cats see the world in the exact same way as Yul Brynner does in Westworld.
posted by fidgets at 1:00 PM on August 14, 2006

The cat's other brain parts could construct some useful concepts from the intercepted signal, which we would not notice. Because a) the signal doesn't enter our brains through the thalamus and b) the cat has a different brain architecture.

It's still very fascinating.
posted by vertriebskonzept at 1:21 PM on August 14, 2006

You do see how silly this is, trapping the nerve activity of a cat, manipulating it to look like a photograph and then imagining a ghostly little cat inside its own brain looking at that resulting picture? It explains exactly nothing. Think what would happen if they did it to your optic nerve. This smudgy, static, flat, black and white picture is what agregoli 'really' sees.


So the fuck what? I'm excited about this research, I think it's fantastic and an awful lot of fun.

Everyone here is so negative I'm amazed you have the will to type at all.
posted by agregoli at 1:32 PM on August 14, 2006

Chill out agregioli. We love this stuff too, that's why we care. There's more to vision than this demonstration would suggest ... bear with me and I'll explain:

Cats, like us, have retinas made up of cones^, rods^, specialist cells to detect motion and edges, together with some cells whose function is much more obscure. In addition, like us, the density of cells is not even, with a high concentration of photoreceptors in a small area, the fovea.

You would therefore expect an complete mapping of neural activity to video pixels to look pretty strange, with motion detectors firing bright points of light, the edges of objects traced out and the whole thing massively distorted to exaggerate the area centred in vision. How would you represent the separate black and white and colour information? The choices are all distortions of one kind or another.

It's hard to reconcile those considerations with the video that these guys have produced. Tempting to think that they just took the nerve signals which 'looked right' from a spy-camera point of view, ie, held up a square to the cat's eye, and picked the nerves which, when artfully arranged in a matrix, produced a square. This is selection bias^ at work and thus the results really don't tell you much - except I suppose that you could use a cat's eye as a really crap camera if you throw away most of the information. This isn't really a surprise but ...

It does seem to be an unhelpful distortion of what we already know about vision.

The hope is to replace the eye with a (distant) camera and restore sight to the blind or see remotely. Well that's the theory but considering the above, you'll see the problem. By the time the signal hits the optic nerve, it is really very sophisticated and multi-channel, there's a lot of processing going on in the eye.

It's this pre-processing that our prophylactic cameras will need to reproduce and this will be hard - reflecting the fact that the distinction between eye and brain is not that clear cut.

Which, in all it's complexity and wonder, is what makes this subject so fascinating.
posted by grahamwell at 1:41 PM on August 14, 2006

I can't wait until they map what a dog sees, because I have always wanetd to spend an entire day looking into a toilet bowl, at my own balls, and at other dog's buthholes.
posted by Sailor Martin at 1:49 PM on August 14, 2006

Chill out agregioli. We love this stuff too, that's why we care. There's more to vision than this demonstration would suggest ... bear with me and I'll explain:

I basically said I liked the link, thought it was exciting, and you used me as an example which to me implied I'm stupid for being excited.

Gee, I wonder why I was annoyed? Did I say anywhere that I thought it was an exact picture of what the cat was seeing?

It's a fantastic link, and I thank jason's_planet again for providing it. I didn't post to debate the merit or science of it. Carry on without me involved, thanks.
posted by agregoli at 1:49 PM on August 14, 2006

Wow.
posted by absalom at 2:01 PM on August 14, 2006

OK agregoli - but I'm puzzled. What did you find exciting about it?

"I have ALWAYS wanted to know what the world looks like to other species."

You're none the wiser I'm afraid.

If you're excited about the lengths we'll go to to torture live animals and then make bogus claims about the result, you do have cause to celebrate.
posted by grahamwell at 2:48 PM on August 14, 2006

People who’ve been blind since birth or childhood and gain sight later on have a heck of a time making sense of what they “see”. To get an idea as to whether the images in the article are what the cat “sees” we could sample images from a recently sighted person. If the images produced by that person’s brain’s cellular firings were recognizable to us (as are the cat’s) we can conjecture that the cat doesn’t see the images it generates as we do since the recently sighted person can’t makes sense of his, yet his brain is still producing an intelligible image to us sighted folks (just like the cat’s brain). There’s a bit of a joke here in that we’re using a human to determine something about a cat, when doing the experiment on the human alone would give us about the same insight.

So, as others have basically said – there would have to be something else in the brain to make sense of/filter the images (neuronal firings). It’d be interesting to see to what degree these types of "images" produced by animal brains are universal, if at all.
posted by Chuckly at 3:33 PM on August 14, 2006

wouldn't this stuff give an upside-down picture? doesn't our eye flip everything or something as it sends the pictures to the brain?

Chuckly, i could see this being useful enough as is --in terms of blind people avoiding obstacles and seeing big structures, or navigating sidewalks, etc. Even if someone's never seen a wall, they've felt one before and can easily make associations and connections that way--"this long tall fuzzy darker thing on the left isn't moving....lemme feel it--yup--a wall...so that's what a wall looks like. These blurry things walking past must be people, and those really fast blurs making noise over there must be cars or something"
posted by amberglow at 3:50 PM on August 14, 2006

Who's this "they" you're talking about, AmberGlow? Paranoid much?
Chances are it'd be more like something one might pay for.

You've obviously never watched Futurama, with its LightSpeed Briefs dreams
posted by amberglow at 3:57 PM on August 14, 2006

but how does the cat feel about what it sees?
posted by quonsar at 7:19 PM on August 14, 2006

Someone FPP the video on YouTube.
posted by Eideteker at 1:12 AM on August 15, 2006

wouldn't this stuff give an upside-down picture? doesn't our eye flip everything or something as it sends the pictures to the brain?

All they are doing is producing a human-readable image from the neuron samples. It can look however they want it to.
posted by sonofsamiam at 9:28 AM on August 15, 2006

All they are doing is producing a human-readable image from the neuron samples. It can look however they want it to.

Assuming shenanigans on the side of the researchers, it'd still be cool to know if image algorithm 'S' (for shenanigans) produced similiar images inter/intra-species.
posted by Chuckly at 10:33 AM on August 15, 2006

Huh?
posted by homunculus at 3:56 PM EST on August 14 [+] [!]

Brak said it much, much better than I did:

While I agree that you'll never "see" things exactly like another being "sees" them, it's not implausible to think you'll get a rough estimate. All emotions, feelings, perceptions can be quantified as discrete activity in the brain, whether or not we fully understand that activ-ity at this stage of scientific research. There's no reason to believe that isolating more sensory experience and developing the capacity to replay it won't come about with more research.

(This is what I meant by: “imperfect tools.”)

Isn't it then just a short step to actually inserting images and stuff into our heads?

I’m not a techie but I suspect it would be several long, drawn-out steps. Probably not something that’ll happen on our lifetimes.

but how does the cat feel about what it sees?

I didn’t see any tunafish in those screenshots, so probably not so enthusiastic.

It's a fantastic link, and I thank jason's_planet again for providing it.

You’re welcome! Thanks to you and everyone else who participated and enjoyed this!
posted by jason's_planet at 12:48 PM on August 15, 2006