Nvidia's new face-rendering tech

March 21, 2013 10:28 AM

Nvidia showed their new face-rendering tech, FaceWorks, at the annual GPU Technology Conference this week. It runs on a Titan ($1000) graphics card. More info here. Compare.

The beginnings of videogame characters were more humble: Due to the graphical limitations of arcade hardware at the time [and to] make him appear human onscreen despite his small size, Mario was given distinct features, prominently a large nose and a mustache, which avoided the need to draw a mouth and facial expressions on the small onscreen character.

posted by ersatz (71 comments total)

The future is dewy.
posted by The Whelk at 10:30 AM on March 21, 2013

Those FaceWorks creatures who haunt the Uncanny Valley apparently suffer from alopecia universalis.
posted by komara at 10:34 AM on March 21, 2013 [1 favorite]

Good enough for porn.
posted by hat_eater at 10:34 AM on March 21, 2013

"Could you imagine if Lincoln had this done? And we could literally sit there and talk to him, and listen to him talk, give that speech, in a fidelity like this?"

Really? REALLY?
posted by uncleozzy at 10:35 AM on March 21, 2013 [6 favorites]

The surfaces are quite good (except for the eyes), but the expressions still seem like Lego shifting under the skin. The flow isn't there: they still seem assembled on the fly. They catch.
posted by maudlin at 10:35 AM on March 21, 2013

"Could you imagine if Lincoln had this done? And we could literally sit there and talk to him, and listen to him talk, give that speech, in a fidelity like this?"

Hey guys... hey. Hey, i have a great idea... What if we take a bunch of photos like really, really fast and play them back so it looks smooth... We could call it, like uh, oh I know, a moving picture! Real pictures! Such high quality it we could stop worrying about rendering and just show what was actually there! No more worrying about stupid shadows!

Brilliant!
posted by Quack at 10:43 AM on March 21, 2013 [2 favorites]

Based on the [more inside] I was expecting a horrifying, photo-realistic talking Mario. Instead I get H. Jon Benjamin complaining about yogurt.
posted by theodolite at 10:43 AM on March 21, 2013 [1 favorite]

I can't tell exactly what they're doing -- perhaps an animator can? -- but it looks like they're subtly compensating for the uncanny valley effect somehow. Something about the slight exaggeration of the facial motions, I think.
posted by griphus at 10:44 AM on March 21, 2013

AHHHH! KILL IT WITH FIRE!!!!
posted by schmod at 10:44 AM on March 21, 2013 [4 favorites]

Bah, who cares about faces when you can have TressFX?

(I'm honestly still not sure if this is a parody.)
posted by kmz at 10:46 AM on March 21, 2013

I assume to someone immersed in the face rendering biz this showcases all sorts of exciting, geeky new advances, but to me the dude basically looks like a fancy Sim under dramatic lighting.
posted by threeants at 10:47 AM on March 21, 2013

Still caught in the uncanny valley.
posted by Slap*Happy at 10:47 AM on March 21, 2013 [1 favorite]

TressFX isn't just real, it's also hilariously buggy.
posted by griphus at 10:48 AM on March 21, 2013

Getting out of the valley is going to take more than just triangles and fancy textures. This is the first realtime rendering of a face I have ever seen which nearly gets out of the valley for me.

I am almost fooled, I probably could be fooled by select stills which were tweaked to perfection by consummate artists.

However, the animation is what totally gives it away, the blank stare, the lack of a sparkle behind the eyes. This face looks dead, devoid of life, and its animation seems to be almost a re-animation.

The next step is to make all of this look effortless and graceful, we have the processing power, now we need the art. Props to all of the engineers who worked so hard on TITAN, it is one of the most impressive pieces of tech to come out in years.
posted by jalitt at 10:48 AM on March 21, 2013 [7 favorites]

If you want to imagine the future, it's a picture of an almost-human face stomping on you forever.
posted by DU at 10:50 AM on March 21, 2013 [5 favorites]

the blank stare, the lack of a sparkle behind the eyes. This face looks dead, devoid of life

Best Buy sales staff should probably start looking for new jobs.
posted by uncleozzy at 10:52 AM on March 21, 2013 [21 favorites]

Huh. Not perfect by any means but definitely more lifelike than Mitt Romney.
posted by Justinian at 10:54 AM on March 21, 2013 [6 favorites]

Yeah, this is almost out of the valley for me, at least during the animations. That it's being rendered in real time makes it that much more impressive. The jumps between animations/routines are pretty offputting though — you'd think they could program some interframes between states.
posted by stopgap at 10:58 AM on March 21, 2013

Better and better but still SO far from right.

What's the best speaking CG human? Not even realtime (which is what makes this as impressive as it is) but full on Hollywood effects: what's the most realistic human out there, and does it pass?

Kreacher, humanoid but not human, gets me pretty good but I'm not totally sure he is pure CG, rather than mask/appliance and CG modification. And of course he doesn't really count, because he isn't human and therefore doesn't trip the alarms as readily.
posted by dirtdirt at 10:59 AM on March 21, 2013

If you want to imagine the future, it's a picture of an almost-human face stomping on you forever.

Y'all need to get over this whole google reader thing...
posted by RolandOfEld at 10:59 AM on March 21, 2013 [7 favorites]

These things are never going to look real until someone realizes that since skin is translucent, shadows on faces shade towards red/brown. All these grey faces look dead and plasticy.
posted by 5_13_23_42_69_666 at 11:02 AM on March 21, 2013 [1 favorite]

I find the face to be almost... too fleshy. Like it's bouncing around too much, too loosely. But boy that skin rendering is impressive.
posted by BungaDunga at 11:05 AM on March 21, 2013

He does not know what the word "literally" means.
posted by HuronBob at 11:06 AM on March 21, 2013

I think that I'm put off by the lips and cheeks. When the actors increase their volume something is wrong, but I can't say what.
posted by a robot made out of meat at 11:08 AM on March 21, 2013

"Could you imagine if Lincoln had this done? And we could literally sit there and talk to him, and listen to him talk, give that speech, in a fidelity like this?"

W-e-e-e-l-l-l-l!!
I'm Rappin' A. B. and I'm here to say,
If you want to drink beer,
Duff's the only way!
I said the ooo-nly way!
Break down!
*crushes beer can against forehead*
posted by Atom Eyes at 11:10 AM on March 21, 2013 [11 favorites]

Alien - you can tell by the creepy ears.
posted by R. Mutt at 11:12 AM on March 21, 2013

Now do it with hair.
posted by PeterMcDermott at 11:13 AM on March 21, 2013 [2 favorites]

Quack: "Hey guys... hey. Hey, i have a great idea... What if we take a bunch of photos like really, really fast and play them back so it looks smooth... We could call it, like uh, oh I know, a moving picture! Real pictures! Such high quality it we could stop worrying about rendering and just show what was actually there! No more worrying about stupid shadows!"

Somebody never played Silent Steel with their new Aptiva Computer
posted by symbioid at 11:14 AM on March 21, 2013

Good enough for porn.

Yeah, one of these as the front end to a Fleshlight and we're all set.
posted by PeterMcDermott at 11:15 AM on March 21, 2013 [1 favorite]

It stays in the Valley for me due to the lack of difference in facial expression between discussing breakfast and shouting "Take my monaaay!!"
posted by Greg_Ace at 11:27 AM on March 21, 2013

hanging from a branch on the side of the mountain wall, the edge of the valley almost in sight, but still out of reach...

Hey guys... hey. Hey, i have a great idea... What if we take a bunch of photos like really, really fast and play them back so it looks smooth... We could call it, like uh, oh I know, a moving picture! Real pictures! Such high quality it we could stop worrying about rendering and just show what was actually there! No more worrying about stupid shadows!

I'm looking forward to convincing fake-live-action moving pictures being something a few voice actors and graphics people and a writer/director can put together without a zillion dollar budget. I'd be surprised if we don't see one of these within 20 years.
posted by Zed at 11:28 AM on March 21, 2013

These things are never going to look real until someone realizes that since skin is translucent, shadows on faces shade towards red/brown. All these grey faces look dead and plasticy.

The people working on this kind of stuff have certainly realized that, and they call it subsurface scattering.
posted by ludwig_van at 11:31 AM on March 21, 2013 [3 favorites]

...a few voice actors and graphics people and a writer/director can put together without a zillion dollar budget. I'd be surprised if we don't see one of these within 20 years.

I genuinely look forward to Foodfight! 2: The Wrath of Brand X.
posted by griphus at 11:37 AM on March 21, 2013 [1 favorite]

Ugh... the weird joke at the end... "OK, show me um... 'white guy dancing'... ha ha... only all the Asians laughed... only all the Asians laughed!".

Really, NVIDIA spokes dude?
posted by rh at 11:43 AM on March 21, 2013

What is ironic is that Important people will never use this, while the masses of Unimportant schlubs playing video games will be the main customers.
posted by Blazecock Pileon at 11:43 AM on March 21, 2013

There's a good bit about skin and subsurface scattering in this article about Prometheus' effects here. Say what you like about the film, it looked incredible. Sadly that film just had the effect of making me realize how TOTALLY INCREDIBLE a very high budget modern SF film with a genuinely deep story and compelling characters could be, while not being that film, at all.
posted by Jon Mitchell at 11:46 AM on March 21, 2013

Soooo....It can't do a head of hair?
posted by Thorzdad at 11:47 AM on March 21, 2013

"Lets see what I can make you do..."

I'm glad that we've decided to ignore every future robot/cyborg movie ever. I'm glad that we've made this decision. Just fuck it.
posted by Fizz at 11:55 AM on March 21, 2013 [3 favorites]

The eyes are probably not focusing correctly, or something.

And no, doing hair right without epic lulz would probably require a separate $1000 graphics card.

There's a big difference between an awesome-looking tech demo using all the resources of high-end hardware, and actually doing it in real time along with all the other more important things that have to happen in real time. And there's also a big difference between a (sort of) perfect-looking human face model and actually having the art resources and storage and everything else to devote to it.

It's still cool, but this is the sort of tech that is perpetually 10 years in the future.
posted by Foosnark at 11:56 AM on March 21, 2013

Thorzdad: "Soooo....It can't do a head of hair?"

AMD has the patent on hair.
posted by the_artificer at 11:59 AM on March 21, 2013

I look forward to the day when I can sit down and watch Abraham Lincoln say any old shit in real-time and on-demand.
posted by mazola at 12:00 PM on March 21, 2013 [2 favorites]

Very impressive, but the bit about how awesome video conferencing would be if you had a computer-rendered avatar of a scan of a person instead of, you know, actual video of that person was... unconvincing.
posted by usonian at 12:01 PM on March 21, 2013 [2 favorites]

"This.... is Digital Ira----"

Yaaaaaaaararararggghhhhhhhh!!!! NO NO NO NO NO NO

Sorry, not impressive in stills, and fucking horrible in motion, even in a downsampled youtube window.
posted by seanmpuckett at 12:02 PM on March 21, 2013

'Digital Ira' is going to be my new band.
posted by mazola at 12:04 PM on March 21, 2013

Getting out of the valley is going to take more than just triangles and fancy textures. This is the first realtime rendering of a face I have ever seen which nearly gets out of the valley for me.

Actually, I think with this face it was a little more in the 'uncanny valley' then something like the typical real-time 3D faces you see in video games these days. It wasn't the rendering it was the way the face was moving. Because, actually, it was not a person moving their face but rather a computer program moving from one facial expression to another robotically. It looks really weird because it is really weird. A real person's face would never do that. But this was just intended to show off the quality of the image, not look realistic.

It may be that they actually wanted it to look unreal so that you knew you were looking at a tech demo and not just a video?

I also think the "uncanney valley" effect dissipates for 3D rendering if you see lots and lots of 3D rendered faces, as people who play lots of video games do. OTOH, the effect is still quite strong for those silicone robots people are building that look like human faces.

I mean, the visual fidelity of these robots is clearly pretty high, but they are still quite creepy (and seem to be designed to be maximize the effect)

What's the best speaking CG human? Not even realtime (which is what makes this as impressive as it is) but full on Hollywood effects: what's the most realistic human out there, and does it pass?

The Aliens in Avatar were all 3D rendered. If you're not talking real time on consumer hardware, the quality is so good as to be indistinguishable if you're using a performance capture system, which means recording the movement of someone's face rather then having an animator do it.

I think it's really more about the quality of the animation, whether the face moves the way a human face would.
posted by delmoi at 12:06 PM on March 21, 2013

Did NVIDIA just put a Steve Ballmer on every gaming PC? Is that... a goal?
posted by running order squabble fest at 12:21 PM on March 21, 2013

To be fair, this is really for the developers, developers, developers, right now.
posted by bonehead at 12:32 PM on March 21, 2013 [1 favorite]

At last we meet Adam Selene. Now we're ready to retake our liberty from the Lunar Authority. Free Luna, comrade!
posted by TheCowGod at 12:37 PM on March 21, 2013 [5 favorites]

The eyes.. The eyes are always what I look at first, and nobody's gotten them right yet.
posted by mrbill at 12:50 PM on March 21, 2013

We already have the technology to make a leering bald man. Plus, it doesn't need a $1000 graphics card, it runs on a $1 piece of cardboard.
posted by oulipian at 1:00 PM on March 21, 2013

hat eater: "Good enough for porn."

Seriously- doesn't this tell us in all but certain terms that humans will not need to get naked (let alone perform bukkake) themselves within 10 years at the most?

(bonus points for a thread in which I type ctrl-F-porn at work)
posted by herbplarfegan at 1:07 PM on March 21, 2013

*on camera, that is, obviously. (sorry)
posted by herbplarfegan at 1:07 PM on March 21, 2013

That weird stepping-through-facial-expressions effect is not entirely unrealistic if you put it in the right context: it rather reminded me of that ST:TNG test footage reel that was circulating a couple months ago.
posted by chortly at 1:27 PM on March 21, 2013 [1 favorite]

Do note that this is a rendering demo from the hardware developers to animators. The look of the eyes and the mechanism to adjust the skin in the hands of say the master animators at Pixar would pull all the elements much closer to just seeming real.
posted by sammyo at 1:30 PM on March 21, 2013 [2 favorites]

That's pretty damn impressive. Especially for realtime.

The big question for me? How much of a load on the CPU is something like this? How many of these could we have at once? Could we have entire bodies rendered with this level of sweetness - could we have multiple bodies?

Probably not quite yet, but in 3-4 generations the answer will probably be "yes", and it will be on video hardware that comes standard in a $300 console. Hello, future.
posted by egypturnash at 2:10 PM on March 21, 2013

How much of a load on the CPU is something like this?

The voice-over seemed to suggest it was about half the load of the GPU it was running on. Did I understand that right?
posted by howfar at 2:16 PM on March 21, 2013 [1 favorite]

The realistic CGI faces have always had two pitfalls: eyes and mouth. Textures have been increasingly getting better and better, and this shows skin texture that is pretty much photo realistic. And the mouth movements synced perfectly with the words being said looks quite good--maybe these Nvidia guys are finally getting that down.

Hollywood motion-capture movies, like Beowolf or the like, had great looking films but the mouths were as stiff as ventriloquist's dummies.
posted by zardoz at 2:19 PM on March 21, 2013 [1 favorite]

Shooting these faces in the next Call of Duty is gonna be so satisfying.
posted by turgid dahlia 2 at 2:59 PM on March 21, 2013 [2 favorites]

This proves that technology is not only about mastery and control, it's also about replacing our environment.
posted by KokuRyu at 3:34 PM on March 21, 2013

Loving that the Forbes article had an actual graph illustrating the Uncanny Valley. That's the first time I've seen anyone feel the need to instantiate the metaphor. Always felt like Forbes wrote like a kindly uncle trying to enthuse a seven year old: a style right in my own uncanny valley.
posted by Devonian at 4:34 PM on March 21, 2013

Seriously- doesn't this tell us in all but certain terms that humans will not need to get naked (let alone perform bukkake) themselves within 10 years at the most?

Well, that, and some people will be out of work - how many employees per performer are there in the industry?
In the next 10 years, lousy actors will be out of work too.
Or perhaps a trend to appreciate authenticity will beat the trend to make stuff cheaper, I've heard it's a thing and ordered a hoodie wink wink.
posted by hat_eater at 5:06 PM on March 21, 2013

The thing to realize here is that this is being rendered on a desktop part you can buy today -- admittedly, a ludicrously expensive $1000 card -- but given the way these things have gone in the past decade, that part will cost $300 in about 2 or 3 years time, and the high end will be an order of magnitude faster.

Uncanny valley issues aside, I'm fascinated to see what levels of fidelity (and, more interestingly to me, creation-of-things-that-have-never-existed-but-look-utterly-real) are coming in the next few years and will be within the grasp of the average PC enthusiast.

Fun times.
posted by stavrosthewonderchicken at 5:33 PM on March 21, 2013 [1 favorite]

Humans will still be needed to act out the roles, which will be captured for animation. If you tried to do it without you'd just end up with weird looking stuff, unless you spent far more time doing it then it would take to just hire some actors and have them do it.
posted by delmoi at 6:17 PM on March 21, 2013

Shooting these faces in the next Call of Duty is gonna be so satisfying.


Are we going to create such realistic heads, and then not make the destruction of said heads just as realistic? And at that point what kind of "game" is it?

Is it wrong to think that hyper realistic rendering will make a lot of current games just creepy to play?
posted by OwlBoy at 7:38 PM on March 21, 2013

They've got to get Abe Vigoda into this.
posted by juiceCake at 8:31 PM on March 21, 2013 [3 favorites]

Is it wrong to think that hyper realistic rendering will make a lot of current games just creepy to play?

I don't know but I expect the games on the next Xbox to be able to allow me to shoot brown people made of actual meat.
posted by turgid dahlia 2 at 10:05 PM on March 21, 2013

And the awards for Best Actor, Best Actress, and Best Supporting Actor go to Andy Serkis.
posted by benzenedream at 10:09 PM on March 21, 2013 [1 favorite]

Speaking of rendered faces, there's a rumor that the person in this video (The one with the bandages, not Geoff) is a CG rendering. I'm still on the "real" side of the debate, though.
posted by ymgve at 3:57 AM on March 22, 2013

I think that I'm put off by the lips and cheeks.


It almost looked like they forgot to put a muscle in the bottom lip. Like, I don't think those faces do a voiced bilabial stop, when your lips curl in to pronounce the B in boy.
posted by Nonsteroidal Anti-Inflammatory Drug at 9:26 AM on March 22, 2013

It's funny to read all these comments criticizing the shortcomings and talking about what they think should happen next to make it more convincing. It reminds me of the louis ck bit about internet on the airplane.

This is Fucking. Amazing. It's astonishing how much processing power it takes to make convincing facial expressions.
posted by hellslinger at 12:32 PM on March 22, 2013 [1 favorite]

Activision: This demo is running at 180 fps on current gtx 680 cards.

Note: Yes, this is the same source data from ICT used by Nvidia at GTC, but rendered using a completely different tech.
posted by ersatz at 2:52 PM on March 29, 2013