I see you!

March 22, 2012 1:57 PM

MIT's new laser camera can see around corners (excellent explanatory video included).

By combining a femtosecond pulse laser and a camera with a time resolution of 2 picoseconds, a computer algorithm can measure the delay in photons bouncing repeatedly off of surfaces not in its direct line of sight. An image is then constructed based on the time delay of the photons returning from the reflected surfaces. MIT's original publication, here.

posted by darkstar (36 comments total)

why not just quickly whip a regular camera

so that you curve the photons

like they did with bullets in wanted

guh mit

you can be so dumb sometimes
posted by Sticherbeast at 2:03 PM on March 22, 2012 [7 favorites]

This mannequin is hidden from this camera by this wall... So how did the camera see this blob of cum?
posted by entropone at 2:07 PM on March 22, 2012 [2 favorites]

"Seeing around corners" is misleading. I think a more intuitive description is that to this camera, any surface can act as a mirror.
posted by Mr.Know-it-some at 2:11 PM on March 22, 2012 [8 favorites]

It can see around corners ...as long as there's another adjacent wall for everything to bounce off, otherwise you're just shooting lasers into empty space.
posted by Sys Rq at 2:14 PM on March 22, 2012 [3 favorites]

Also: Would it work in daylight conditions? Fog/smoke?
posted by Sys Rq at 2:19 PM on March 22, 2012

This would make Deus Ex a lot less fun.
posted by michaelh at 2:19 PM on March 22, 2012

as long as there's another adjacent wall

Ground? Ceiling? Beer can tossed out?
posted by stbalbach at 2:20 PM on March 22, 2012

I saw a talk about this recently and I had almost zero idea how it was working. Pixies, I think.

geddit?
posted by DU at 2:24 PM on March 22, 2012

Wow this is a huge achievement, mode advanced versions can be very useful.
posted by BobS13 at 2:26 PM on March 22, 2012

They presented an earlier version of this idea at ICCV. In order to get a reasonable signal to noise ratio, they had to use a laser whose "average laser power was 420 milliwatts, corresponding to a peak power of greater than 85 kW", and one of the people asking questions after the talk was really concerned with eye safety.
posted by Pyry at 2:28 PM on March 22, 2012

that's what the lean key is for
posted by nathancaswell at 2:36 PM on March 22, 2012 [1 favorite]

So, when you're sneaking around, remember to wear your fuligin, and stay close to the walls.
posted by Mister Moofoo at 2:40 PM on March 22, 2012 [2 favorites]

So it's like a eye-blinding flashbang combined with a dental mirror?
I want to see this in a counter-terrorist FPS.
posted by dragoon at 2:42 PM on March 22, 2012

Hmm. So given the hint about a laser and the camera's temporal resolution it was pretty obvious what the mechanism was. Does that make it unpatentable? ;)
posted by wierdo at 2:45 PM on March 22, 2012

Does this have anything to do with Bootsy Collins?
posted by neuromodulator at 2:48 PM on March 22, 2012

The Marsh brothers are big fans of Hunt for Red October. Just a guess.
posted by tarheelcoxn at 2:56 PM on March 22, 2012

Ground? Ceiling? Beer can tossed out?

In its current form, the math seems to require a planar reflective surface for the diffuser, though I'm sure that could be generalized with more computer horsepower.

Would it work in daylight conditions?

In that conference paper, they say:

We need to sample the incident light with picosecond resolution and be highly sensitive to a low photon arrival rate. Our depth resolution is limited by the response time of the detector and digitizer (250 ps, 7.5cm light travel). The high peak power of our laser was critical for registering SNR above the dark current of our photo sensor. Also, our STIR acquisition times are in nanoseconds, which allows us to take a large number of exposures and time average them to reduce Gaussian noise.

Background light levels are clearly an interference. Might cause some problems, but would also be very amenable to optimization with better optics & gating, I suspect.

Fog/smoke?

Much harder. If the laser can't penetrate the smoke, the technique doesn't work. Dispersion in air is enemy to all lasers.
posted by bonehead at 2:57 PM on March 22, 2012

Reminds me of the "Enhance" stuff where they zoom in on a reflection and then convert it to a full resolution image.

Except, in this case they don't even have a mirror.

Here's how I'm visualizing this to try to understand how it can work

1) Imagine a single point, lets say a tiny sphere smaller then 0.6mm (which is their temporal resolution)

2) a laser bounces off the wall, and creates an expanding 'shell' of photons. Every two femtoseconds, the shell expands 0.6mm.

3) Eventually that shell hits the target object, as in the demo video, that creates a new shell.

4) here's the key part, which they don't try to explain very well in the video: The camera 'sees' where the 'shell' intersects with the paper as a circle

So each 'frame' of video, the circle is slightly larger then before, until it grows two large to be seen on the target.

Once you have that, you just do some basic trigonometry. You have a plane, and you have more then two points on that plane which you know intersect with a a sphere. It should be easy to figure out where the center of the sphere is.

At first I was having trouble because it seemed to me that you could easily have multiple reflectors such that the light would bounce off them at the same time, but if you did that, the circles still wouldn't match exactly, they would overlap but you'd see both of them, and be able to calculate each point.

The real problem comes in when you have not two or three, but, say, thousands. How do you distinguish all the circles from each other? It's a machine learning type problem. I'm guessing the reason they have to do multiple takes from multiple points on the target is due to the difficulty. Probably by looking at minor changes between the reflections generated by different laser positions you can figure out more easily what pattern goes where among the multiple different configurations possible?
posted by delmoi at 2:59 PM on March 22, 2012

I'm going to start selling "walls" as "LaserMirrors"
posted by mrnutty at 2:59 PM on March 22, 2012

2) a laser bounces off the wall, and creates an expanding 'shell' of photons. Every two femtoseconds, the shell expands 0.6mm.

That's exactly where I think their visualization breaks down. This is a problem with that pesky particle/wave duality thing. I see it more as a wave.

I see it as illuminating the hidden object with a point source from the laser hitting the back wall. Then they're reading the reflected light off the back wall, and reconstructing the received wavefront with interferometry. You can see that happening a little, near the end, as the laptop shows layers of wavefronts overlaying and building the object.
posted by charlie don't surf at 3:11 PM on March 22, 2012 [3 favorites]

This is more akin to scanning particle image velocimetry, which is essentially a particle phenomenon. Holographic imaging based on interference is possible, but still rather experimental, as I understand it. This doesn't appear to be a holographic/interference wave measurement, just a particle one.

Still, I'm not a specialist in this area, but we do do some PIV from time to time.
posted by bonehead at 3:59 PM on March 22, 2012 [1 favorite]

This couldn't be done in the 1950's, as Robby the Robot explains @-> 29:07 -- 29:30
posted by 0rison at 4:10 PM on March 22, 2012 [2 favorites]

Corner Cat is watching something masturbate.
posted by not_on_display at 4:26 PM on March 22, 2012 [2 favorites]

What can this do that an intern with a mirror cannot?
posted by jimmythefish at 5:13 PM on March 22, 2012

Is this actually taking photos, or sensing depth? Video is unclear.
posted by smackfu at 5:22 PM on March 22, 2012

Depth sensing. It's using pulse timings to determine distance.
posted by bonehead at 5:41 PM on March 22, 2012

That's exactly where I think their visualization breaks down. This is a problem with that pesky particle/wave duality thing. I see it more as a wave.

I see it as illuminating the hidden object with a point source from the laser hitting the back wall. Then they're reading the reflected light off the back wall, and reconstructing the received wavefront with interferometry. You can see that happening a little, near the end, as the laptop shows layers of wavefronts overlaying and building the object.

I don't think so. These are the same guys who took videos of photons moving through objects. There was no interference because the light was already 'off' before the photons return. The laser's pulse is only 'femtosecond' laser, that's only enough time for light to travel 299 nanometers. They really were using 'balls' of light.

One thing they did do in the previous video was send lots and lots of photons through, and use a technique to record only photons received at time specific times to build up one frame, rather then 'actually' watching a photon packet travel through something in 'real' time.

The waves you see in the video are, I think time/position graphs for a specific scanline. You can see more of those in the thumbnails included with the nature article.
posted by delmoi at 6:32 PM on March 22, 2012 [1 favorite]

To be clear, they are using photons as ping pong balls to bounce off objects (3 times!) and measure distances based on their constant speed. There does not appear to be any interferometry/holography going on at all.
posted by bonehead at 8:59 PM on March 22, 2012

It's like the photo analysis technology from Blade Runner.

Sweet.
posted by scottu at 11:14 PM on March 22, 2012

it's like raytracing in the real world, right?
posted by jrishel at 6:56 AM on March 23, 2012 [2 favorites]

I think a more intuitive description is that to this camera, any surface can act as a mirror.

And, amusingly, an actual mirror acts as a hole in reality.
posted by smackfu at 6:57 AM on March 23, 2012

it's like raytracing in the real world, right?

Yes.
posted by bonehead at 7:35 AM on March 23, 2012

It sure would be nice to see a real example instead of a simulation.

The enemy might be able to see the scanning laser point.

You could do the same thing with SONAR. And the enemy wouldn't see it.
posted by caclwmr4 at 8:54 AM on March 23, 2012

It sure would be nice to see a real example instead of a simulation.

Their paper (PDF) has examples on page 8 and 9.
posted by smackfu at 9:11 AM on March 23, 2012

A major advantage of lidar over sonar is separation of speeds. You could do this out of an moving truck or an airplane with no interference from the vehicle's motion. Sonar also has spacial resolution issues on sub-millimeter scales.
posted by bonehead at 10:39 AM on March 23, 2012

Ridiculous title. Does not do what it says on the tin.
posted by IAmBroom at 8:17 AM on March 25, 2012