Gaze Correction for Home Video Conferencing

September 4, 2013 1:36 PM

"Effective communication using current video conferencing systems is severely hindered by the lack of eye contact ... [Our] system is a face replacement algorithm that synthesizes a novel view of the subject’s face in which the gaze is correct and seamlessly transfers it into the original color image." [auto-playing sound+video]
posted by griphus (39 comments total)

Not sure if genius, creepy, or both.
posted by slater at 1:43 PM on September 4, 2013

Interesting. Kinda strange how videophones sorta snuck up on us. With kinect and skype built into the XBox One, this exact technology is pretty practical for consumers.
posted by Ad hominem at 1:44 PM on September 4, 2013

Not sure if genius, creepy, or both.
I'm gonna go with "creepy", and a side of "do not want".
posted by Thorzdad at 1:45 PM on September 4, 2013

As a bonus, GIS for "gaze correction" results in tons of images like this:

http://meaganmakes.files.wordpress.com/2011/03/gazecorrect4.jpg

apparently, "incorrect" gaze direction is also a problem with some super-niche dolls
posted by parkan at 1:46 PM on September 4, 2013 [1 favorite]

"Hey! My eyes are down here!"
posted by Pope Guilty at 1:50 PM on September 4, 2013 [4 favorites]

O_o
posted by Foosnark at 1:53 PM on September 4, 2013

That has a really strong uncanny valley effect that I can't shake.
posted by potch at 1:54 PM on September 4, 2013 [1 favorite]

In Being Digital, Nicholas Negroponte recounts working on a DOD Telepresence project. It was to be used for negotiations and meetings in the case of nuclear war. They wanted it to convey every nuance of interaction. They ended up with a bunker full of animitrons. Facial features were projected into semi-transparent masks from behind. The faces concealed a video camera for first person views.

The whole thing sounds so surreal. a bunker full of animitrons jerking noisily and shouting at eachother. the US military does some strange shit. Can't make this shit up.
posted by Ad hominem at 1:54 PM on September 4, 2013 [10 favorites]

A photo of the inventor.
posted by zippy at 1:54 PM on September 4, 2013 [4 favorites]

I missed out on the whole videoconferencing thing until this job, and now I can say with conviction that I really, truly hate it. It's unflattering as hell (who looks good slumped at a table in office clothes, eating lunch? NO ONE), it's unnerving to have one of those little cameras zoom at you if you speak/ask a question, and in 99% of the meetings I go to, it's entirely superfluous. Ugh.

Now, me and another camera-shy employee fight each other for the one chair in the meeting room's blind spot.
posted by emjaybee at 2:03 PM on September 4, 2013

I don't even know these people and something looks wrong, so I imagine it'd be even creepier to see this technology subtly distorting faces I know well. Interesting idea, but they've got their work cut out for them if they want to trick peoples' perception of the faces of their close friends and colleagues.
posted by contraption at 2:06 PM on September 4, 2013

Gaze Correction is also the name of my Marcus Bachmann cover band.
posted by MCMikeNamara at 2:07 PM on September 4, 2013 [9 favorites]

I think a better solution is simply to mount the camera near the display, like the one on the bezel of my notebook computer. If the person is looking very near the camera, you don't need gaze correction.
posted by Chocolate Pickle at 2:09 PM on September 4, 2013

in 99% of the meetings I go to, it's entirely superfluous

At least 99% of the meetings I have been in period were superfluous.
posted by Steely-eyed Missile Man at 2:12 PM on September 4, 2013 [5 favorites]

Now can we get it to add some clothing over my pajamas? Because that would be good.
posted by JoeZydeco at 2:14 PM on September 4, 2013 [1 favorite]

You heard it here first. DLC clothes and accessories for skype.

I've never encountered a working videoconferencing setup in a conference room. And I've seen a lot of shit, I've see a working room reservation system tied to outlook where the LCD screens next to the door displayed the correct reservation but you still had to find the one facility guy with the key.
posted by Ad hominem at 2:16 PM on September 4, 2013 [3 favorites]

The facility guy is on the third floor, in the secret room behind the bookcase. To get the key, you have to solve his riddle unless you have CHA or STR >12, in which case you can either charm or intimidate him into giving it to you.
posted by griphus at 2:20 PM on September 4, 2013 [7 favorites]

I learned from an exec that they are weak to Yankees tickets.
posted by Ad hominem at 2:24 PM on September 4, 2013

This isn't actually my story, but I have a friend who would occasionally get sucked into A/V projects for work. Apparently, one of the projects involved interviews with a number of various people. As the team was reviewing the footage, some of which happened to feature a somewhat effeminate young man, their boss had a grumpy, disappointed look on his face.

"I don't know what it is, but there's just a problem with the gays," he muttered.

Pin drop. Everyone was just too mortified for words. Did he really just say that? Friend, who also happens to be gay, just looked over, because damn, his boss really didn't seem like that kind of guy, when, suddenly, a light bulb went on.

I'm told his boss's eyes were huge as he frantically waved his hands to dispel misunderstanding. "GAZE! GAZE! Their eyes! They're not looking at the camera!"
posted by Diagonalize at 2:38 PM on September 4, 2013 [18 favorites]

I'm gonna go with "creepy", and a side of "do not want".

Ah, you'll be wanting our "attitude correction" patch.
posted by yoink at 2:38 PM on September 4, 2013 [2 favorites]

Not to be confused with gays correction, which is illegal in California.
posted by resurrexit at 2:39 PM on September 4, 2013 [1 favorite]

face replacement algorithm

Okay, I'll bite -- what is it replaced with?
posted by ricochet biscuit at 2:42 PM on September 4, 2013

face replacement algorithm

Okay, I'll bite -- what is it replaced with?

I want to be a dragon. Can I be a dragon? With sparkles?

Or just a cute kitten would be fine. Anything but my own pixellated face.
posted by emjaybee at 2:57 PM on September 4, 2013

Now, me and another camera-shy employee fight each other for the one chair in the meeting room's blind spot.

Venues make a huge difference when it comes to videoconferencing. I find the best rooms are often circa 1970s lecture halls like this one with permanently mounted overhead LCD projectors and drop-down rectangular screens because they add the appropriate level of OMG STAR TREK!!!! to any Skype call.
posted by RonButNotStupid at 3:05 PM on September 4, 2013 [1 favorite]

I find that effective use of the video conferencing system is usually hindered by the fact that it takes the first ten minutes of the meeting to set it up.
posted by doctor_negative at 3:14 PM on September 4, 2013 [1 favorite]

> I want to be a dragon. Can I be a dragon? With sparkles?

Second Life is so 2005.

Joking aside -- I do kinda' think that Second Life solves a lot of the problems home conferencing presents, simply by making participants engage through their avatars. There's a lot of real-world expressiveness that gets lost, not to mention having to steer emotions to your avatar through a keyboard and mouse rather than through a camera reading your face.

On the other hand, something that can read your face and manipulate the avatar's face accordingly must be easier than trying to do real-time presentability improvements like this in a live video conference. Cisco Systems' Telepresence Unit and Second Life are more or less trying to solve the same problem, but approaching it from opposite directions and with purposes that are not, I think, as contrary as they seem.
posted by ardgedee at 3:39 PM on September 4, 2013

The proposed solution to what the telecommunications industry's psychological consultants termed Video-Physiognomic Dysphoria (or VPD) was, of course, the advent of High-Definition Masking; and in fact it was those entrepreneurs who gravitated toward the production of high-definition videophonic imaging and then outright masks who got in and out of the short-lived videophonic era with their shirts plus solid additional nets.

Mask-wise, the initial option of High-Definition Photographic Imaging — i.e. taking the most flattering elements of a variety of flattering multi-angle photos of a given phone-consumer and — thanks to existing image-configuration equipment already pioneered by the cosmetics and law-enforcement industries — combining them into a wildly attractive high-def broadcastable composite of a face wearing an earnest, slightly overintense expression of complete attention — was quickly supplanted by the more inexpensive and byte-economical option of (using the exact same cosmetic-and-FBI software) actually casting the enhanced facial image in a form-fitting polybutylene-resin mask

- David Foster Wallace, Infinite Jest, pp 144-151.
posted by The White Hat at 4:03 PM on September 4, 2013 [1 favorite]

Aaaaaaaaa


aaaaa
aaaaaaaaaa






aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
posted by DoctorFedora at 4:57 PM on September 4, 2013

At my company we more or less banned phone conferences in favor of Google Hangouts. The difference is night-and-day better: no first 10 minutes of people calling out their attendance, no people talking over each other (because everyone can use normal visual cues that they're about to start speaking), and in general it's much more humanizing and trivial to start up/shut down. We even use them for almost any one-on-one discussion. I'm in pretty much non-stop meetings for my job and entire weeks go by where I never pick up my office phone.

(That said, do not need weird eye software; putting party hats on coworkers with Google Effects much more effective team-building.)
posted by nev at 5:01 PM on September 4, 2013

The uncanny valley thing isn't that bad, but I bet it's way more noticeable when looking at someone you know well.
posted by es_de_bah at 6:23 PM on September 4, 2013

What's unfortunate is that, as this gets gradually better, those of us with slightly wider uncanny valleys will eventually be the only ones left who mind; perhaps it will even be the same ones of us who feel railroaded into a melange of green, brown and yellow lighting under the guise of "warm and natural" LED lightbulbs. The world will be replete with zombies with their heads reattached 2 degrees off kilter, asking why the hell we're so picky.
posted by chortly at 6:51 PM on September 4, 2013

So if there was a One Direction poster on the wall behind the person you're talking to, would they all be looking at you? Will the old film trope of a painting with eyes that follow you around the room because they're actually spyholes just look like a gaze correction error to futurepeoples?

There should be an effect where you can reach up and pop your head off and hold it in the crook of your arm, Ichabod Crane style.
posted by XMLicious at 7:22 PM on September 4, 2013 [1 favorite]

Being able to look someone in the eye is a very important part of human interaction, but this is a bass-ackward way of approaching the problem.

The correct solution is to place the camera behind the screen about where you expect the eyes to appear.
posted by Tell Me No Lies at 9:23 PM on September 4, 2013 [1 favorite]

The correct solution is to place the camera behind the screen about where you expect the eyes to appear.

Because that way they can't see you, the screen being in the way, and you gain the body language advantage!
posted by Pope Guilty at 9:53 PM on September 4, 2013 [1 favorite]

Because that way they can't see you, the screen being in the way, and you gain the body language advantage!

Exactly! The same tactic the police use when they put you in a room with glass you can't see through. They can't see through from the other side either so you feel all safe and warm and ready to talk.
posted by Tell Me No Lies at 10:12 PM on September 4, 2013 [1 favorite]

At my company we more or less banned phone conferences in favor of Google Hangouts.

Ditto. Webex and go2meeting and whatever that godawful Cisco or Citrix or triscuit whoever I don't remember but I don't have to because they all became obsolete basically overnight when Hangouts launched.

Also you can tell when it's time to end the meeting because people start running the Effects plugin.

(Also they have a beard and glasses in the effects plugin that basically are indistinguishable from my real beard and glasses. So that's nice. Creepy. But nice.)
posted by ook at 11:10 PM on September 4, 2013 [1 favorite]

Is anyone working on combining camera and screen into essentially the same device? So that it can record your real gaze (at the screen) rather than correcting it artificially? Seems to me that would be the ideal solution.
posted by thefool at 3:17 AM on September 5, 2013

I've been imagining a little folding cardboard periscope with a half-silvered Mylar reflector at the bottom that sits over the videochat window and bounces your image up to the camera. Kickstarter?
posted by nicwolff at 6:03 AM on September 5, 2013

Is anyone working on combining camera and screen into essentially the same device? So that it can record your real gaze (at the screen) rather than correcting it artificially?

The documentary filmmaker Errol Morris invented this years ago and named it the Interrotron (diagram here.) A bit bulky for incorporation into a mobile device, unfortunately.
posted by contraption at 10:05 AM on September 5, 2013 [1 favorite]