Is this real or memorex?
July 25, 2006 7:24 AM   Subscribe

[NSFW] Heavy Rain From Quantic Dream Sets a new standard for realism in interactive cinema... I found this to be really creepy. I have a hard time looking into their eyes. Something about it is just...well, see for yourself. Anyway the flash technology is getting better and better. Along with what has become know as augmented reality. Pretty amazing stuff.
posted by Unregistered User (61 comments total) 2 users marked this as a favorite

More on Augmented Reality.
posted by Unregistered User at 7:28 AM on July 25, 2006

Um... so what is this exactly? A demo peice for a video game?
posted by Artw at 7:30 AM on July 25, 2006

seems like its a replacement for shitty actors...or just another flash in the pan special effects company. who knows.
posted by stilgar at 7:35 AM on July 25, 2006

Dunno.... the mouth just doesn't look right to me and strangely, it's far more convincing in close-up than in medium/long "shots".
posted by NeonSurge at 7:35 AM on July 25, 2006

She looked like she had a ton of botox injected into her face. Which is to say she looked almost real, pretty good technology.
posted by oddman at 7:36 AM on July 25, 2006

after watching it i have to say the mouth/word matching is horrible and the arm/body movement is really bad...this is not going to replace any actors.
posted by stilgar at 7:37 AM on July 25, 2006

"the flash technology"? Isn't this just a flash movie?

The CGI's okay, but... her mouth moves like she's got Down's Syndrome. And when does it get interactive?
posted by fungible at 7:37 AM on July 25, 2006

interactive? How so? Play, Pause... fast forward. The future is NOW!
posted by Hanover Phist at 7:39 AM on July 25, 2006 [1 favorite]

Mary Smith is not that chick's real name. What kind of accent is that anyway?

Also how do I earn honored faction with her to get teh cool warlock cloak?
posted by Mister_A at 7:39 AM on July 25, 2006

Welcome to the Uncanny Valley.
posted by GuyZero at 7:40 AM on July 25, 2006

I guess it's good for being in flash...
posted by beerbajay at 7:40 AM on July 25, 2006

Possibly it'sa video and in the real thing you can do all that camera wobbling and looking around yourself... sort of like listening to a character give a long speach in HL2. If you shoot her with the electrified crossbow bolt the screen goes black and a message tells you that you've failed the game due to "Innapropriate use of resources".
posted by Artw at 7:42 AM on July 25, 2006

I thought it was a pretty cool demo piece. CGI keeps moving forward, but getting humans down right will probably be the last hurdle. We're so good at reading body language and facial expression that the slightest flaw destroys the illusion.
posted by justkevin at 7:43 AM on July 25, 2006

I thought some of the character's motions were really impressive, whereas others looked unfinished - the Wikipedia page for Heavy Rain mentions that Quantic didn't originally intend to show the demo yet, and that Sony persuaded them to use it as an example of PS3 graphics. If that's the case, I think the technology's got real potential.

Everyone who's on about this replacing actors: this is a technology test for a videogame. It's being done in real time, and for all its failings it arguably gets closer than any game has to realistic human motion. That's why it's impressive.

That said, Quantic have yet to develop a game that doesn't fall squarely into the 'extremely ambitious but critically flawed' box, so any hope I'm holding out for this leading to amazing things in future games is solely based on the hope that they'll license the technology to someone else.
posted by terpsichoria at 7:44 AM on July 25, 2006

Although having said that, upon rereading the post I've no idea what this has to do with augmented reality or 'flash technology' (actually, the Flash movie player it loads in is horrible).
posted by terpsichoria at 7:47 AM on July 25, 2006

Ooooooh... it's something to do with PS3. I wish the poster had just told us that. It;d be a bit more helpful that 'augmented reality' and 'interactive cinema'.
posted by Artw at 7:51 AM on July 25, 2006

Interesting tech but still young. Things will get interesting when you're given a pair of glasses to wear which carry a camera (to get your perspective) and overlays the added video through a kind of HUD. Something like what Apache pilots use. When that happens we'll have a new level of immersion.

Only problem is that the "set" for the video game would be your living room? That I'm not so sure about. This will probably find a market in arcades where booths/rooms can be setup and dedicated to the platform.

I believe something like this already exists. I forget which plane Boeing was making (777?) but apparently they used a similar system so techs could "see" pipes and wires behind panels to aid in servicing the craft.
posted by ruthsarian at 7:52 AM on July 25, 2006

Does Mary Smith have a MySpace page? Then it's real.
posted by ninjew at 7:56 AM on July 25, 2006

I'm actually more impressed with what Bioware is doing with CG actors in their upcoming game, which won Best of E3, Mass Effect. [Flash, Sound]
posted by NationalKato at 7:59 AM on July 25, 2006

posted by bunglin jones at 8:03 AM on July 25, 2006

NSFW = boobies, damn it! Virtual or otherwise.
Needless to say, I am not impressed.
posted by slimepuppy at 8:04 AM on July 25, 2006

What is NSFW about that? I was waiting for her to strip !
posted by a3matrix at 8:05 AM on July 25, 2006

This is a demo of a realtime game engine, and it does look pretty.

I'm sure I'm not the only one who thinks it a bit of a joke to call the acting 'amazing' when the facial animation looks utterly crippled. She has dolls eyes and the lip sync...doesn't.

It's daft the way companies pour all their money into getting the whole image looking amazing, they spend ages motion capturing actors and cleaning it up, and then they scrimp time and money on getting facial animation right.

Put an eye tracker on someone watching that scene and you'll notice that most of the time they're staring at the face - the eyes in particular - which confoundingly is the weakest part of the whole thing.
posted by 6am at 8:08 AM on July 25, 2006

Yeah I kept skipping forward looking for virtual boobies. No such luck.

She did have man hands.
posted by Justinian at 8:09 AM on July 25, 2006

NSFW= "fucking" "fuck" and "shit"

Ho-hum. Yeah, some of the detailing was nice. BFD. I see in the credits a credit for motion-capture. For me that's the kiss of death to any claim of hot new technology. Call me when you can do this without first dressing a real actor in a motion suit. Until then, it's all just about texture-mapping, lighting, and environmental FX.
posted by Thorzdad at 8:16 AM on July 25, 2006

Interview discussing how the demo was made. Apparently they were experimenting with a new method for lip syncing, which didn't work out too well.

It sounds like they've learned a lot from making Fahrenheit/Indigo Prophecy. (They even admit to not spending enough time fleshing out the second half of the story.) Heavy Rain looks promising.
posted by Sibrax at 8:31 AM on July 25, 2006

"You tell me, you won't do it - you don't have the guts bullets."
posted by zanni at 8:38 AM on July 25, 2006

Apparently I still must not warn Brangelina (TM) to cash out in Penthouse and the viral marketer owes me.
posted by elpapacito at 8:48 AM on July 25, 2006

Meh. I've seen more realistic (and less creepy-looking) humans in Half-Life 2. And that's an action game.
posted by neckro23 at 8:51 AM on July 25, 2006

She's got spit in the corners of her mouth. Ew.

I don't understand why SFX people are always raving about how "lifelike" the hair and fur is on stuff, when the illusion is immediately spoiled by awful lipsynch. In some spots of this piece, it wasn't even close to being in synch. Quit effing around with making the hair bouncy and shiny and make the damned mouth match up already!
posted by macadamiaranch at 8:58 AM on July 25, 2006

You want augmented reality? I've got your augmented reality right here.
posted by public at 9:00 AM on July 25, 2006

I look forward to the day when there are no Tom Cruises.
posted by 517 at 9:05 AM on July 25, 2006

The script was apparently written by a virtual writer, too. Or maybe even a team of virtual bickering hacks.
posted by kozad at 9:10 AM on July 25, 2006

They even admit to not spending enough time fleshing out the second half of the story.

Oh. Then that explains it.
posted by grabbingsand at 9:52 AM on July 25, 2006

It's crazy that for all the fanciness and advances in visual effects, they're nowhere near the point where they'd even attempt a CG voice.
posted by XMLicious at 9:56 AM on July 25, 2006

I really tried to like Indigo Prophecy (I think it was called Farenheit in the UK) from Quantic Dream. The cinematics were well-done, and the character development and storyline were intriguing. But the Simon-sez gameplay and the simple fact that choosing a single conversational response could cause the gamer to unknowingly lop off a third of the game made it more a frustrating exercise in storytelling than a good game (there was an ice-skating sequence? I played through three times and never saw that!).

Still, I'm impressed by the demo. There's something hinky going on with her upper lip, and there's an unnatural abruptness to some of her movements that breaks the illusion, but I look forward to seeing this implemented in next-gen games.

Of course, I still play "Snake" on my cellphone, so what do I know.

More impressive to me is the physics engine on display in Rockstar's Table Tennis game, which they say will be the backbone for the next-gen GTA. Mmm....glistening ragdoll ultraviolence......
posted by BitterOldPunk at 9:58 AM on July 25, 2006

From the time we're born, there's nothing we watch more than other people, especially their facial expressions. Watch a real person talk, and compare the dozens of constantly changing interactions between facial muscles, skin, eye gazes, head tilts, body movements, breathing, clothing, etc. Perhaps we see many of these moves unconsciously, and we know when they're missing.

MIT's Mary 101 project has resulted in some very realistic speech expressions, but they're basically cheating by manipulating small portions of recorded images of real people, then reinserting them into the original videos. The results look very natural, but are still uncanny enough to spoil the effect. For one thing, note how still the female model has to sit: if she moved her head to any large degree, the mouth image would not move with it.

Quantic Dream—or anyone else for that matter—has got a long way to go before they can achieve fully digital facial and body models that look like a normal person.
posted by cenoxo at 10:44 AM on July 25, 2006

I like how she can talk while playing a jews harp.
posted by fire&wings at 10:44 AM on July 25, 2006

pick apart the Quantic Dream piece all you like, but that second link was f**king awesome!
I haven't been that jazzed about computer graphics in a long, long time.
also, this might be the first time Clive was ahead of metafilter! (forgive me, Clive!)
posted by I, Credulous at 11:01 AM on July 25, 2006

posted by euphorb at 11:26 AM on July 25, 2006

Heavy Rain and Quantic Dream were also featured in this month's Edge magazine. The studio's founder, David Cage, was quoted as saying:

"This demo was initially an internal prototype aiming to determine if we could recreate not only the likeness of a real actress but also her acting performance in its emotional dimension. ... The demo had to run in realtime 3D on PlayStation 3D. When we told Sony what we wanted to do, they asked us if they could have the demo at E3 on their booth."

"The most interesting aspect of this demo is that I believe we took the first step out of the uncanny valley. When I present this demo, I watch the audience. As soon as the actress appears , people listen to her. ... They hold their breath when she puts the gun to her head and when the first tear falls, smile when she changes her mind."
posted by runkelfinker at 11:29 AM on July 25, 2006

posted by A189Nut at 11:33 AM on July 25, 2006

Okay, so the writing was horrible, but once in a while the facial expressions were right on. And I think her accent was east Russian? I don't know.

Anyway, this doesn't look like it has anything to do with "interactive cinema", unless you count reviving a genre that died around 1999 due to how horrible the writing always was. They got that part right.

On the other end of the spectrum, yes, Mass Effect is awesome.
posted by blacklite at 11:40 AM on July 25, 2006

Well, some critique having watched this with the sound off.

1: The movement of the arms and legs ranges from slightly overacted to a bit disturbing. They reminded me of a person with some motor defects.

2: I think one of the missing pieces here is the muscles under the face. Jaws don't move on their own, the cheeks deform as the mouth opens and closes. I found this to be a bit errie once I noticed it.

3: Even without the dialog there were parts that were emotionally effective. But then again, you can get that from low-fidelity animation as well.
posted by KirkJobSluder at 12:19 PM on July 25, 2006

that was embarrassingly bad.
posted by jimmy at 1:13 PM on July 25, 2006

I'm not gonna meh this, I think they're really working their asses off to improve the technology -- including the lip-synching. They just aren't there yet.

What's interesting to me is thinking back -- and I'm old enough -- to the 8-bit era and the very beginnings of things like true-color graphics and early flawed attempts at movie CGI. For instance, The Abyss still looks reasonably good, but I'm also quite aware of how Cameron worked around the limitations of the technology with judicious camera placement and careful editing. Here, you have to watch the whole thing, and so everything must work at all times, and it just doesn't. (By the way, the weird immigrant accents are an obvious dodge for the imperfect lip-synching, but maybe they call too much attention to it.) But 20 years ago I would have killed for this level of production capability, and yet here it is, and the uncanny valley remains.

Sure, it's mostly the face, but the body movements could use work, and the way the clothes hang. We spend most of our time as humans looking at people's faces, and our brains have huge areas devoted to processing them, so we have higher expectations there.

* Lip-synch needs work (points for trying)
* Cheeks need more movement, a lot more
* Just putting the background out of focus doesn't mean people won't notice the plastic textures
* They had her looking straight at the camera too much. They could get away with more if she could look away at strategic moments.
posted by dhartung at 1:24 PM on July 25, 2006

Poor acting ruined it.
posted by tkchrist at 1:43 PM on July 25, 2006

And I think her accent was east Russian? I don't know.

To me she sounded an awful lot like that chick from Before Sunrise/Sunset, so...I'm going to guess it's some kind of French dialect. Quantic Dream is also a French development studio, I believe.

Oh, and put me in the "not all that impressed" category...from what I've seen, Metal Gear Solid 4 blows this out of the water in terms of convincing human animation.
posted by Sullenshady at 1:47 PM on July 25, 2006

By the way, the weird immigrant accents are an obvious dodge for the imperfect lip-synching, but maybe they call too much attention to it.

Or it could be that Quantic Dream are a French company.
posted by Orange Goblin at 1:51 PM on July 25, 2006

Um ... you guys check out Poser lately?
posted by RavinDave at 2:08 PM on July 25, 2006

Sorry, but this is so deep into the uncanny valley, it's never coming out.
posted by slatternus at 2:36 PM on July 25, 2006

this is pretty impressive modelling although there's little animation in the trailer.
posted by juv3nal at 3:00 PM on July 25, 2006

I'm just impressed by the fact that its supposed to have been rendered in realtime. Yes, for a movie it sucks, but for a videogame it's better than some studio's "cenematic cut scenes".
posted by sotonohito at 4:23 PM on July 25, 2006

Um. I don't know. Compared with what's been churned out by SqureEnix / Squaresoft ... over the last few years I've been kind of ... maybe a snob about this? I was genuinely impressed with the lifelike graphics of those films along with others that blended computers with real actors.

This just looks bulky and lifeless. A step up from some videogames sure, but it doesn't compare to my previous marveling at the fanatical renderings of a computer animation artist.
posted by eatdonuts at 5:08 PM on July 25, 2006

... but that Dear Annie movie looks cool...
posted by eatdonuts at 5:11 PM on July 25, 2006

Bioware - what is it with them that everything they touch seems to be gold? Mass Effect looks amazing, and if the story is like everything else they've put out, I've no problem throwing them ~$60 for a game. True, the faces didn't match the urgency of the voices in some parts, but I'm sure that they'll eventually conquer that too. I was awed at the small things, like the cheekbones of the charaters and how the shadows moved as their heads turned. god, i'm such a geek, sometimes.
posted by Zack_Replica at 8:58 PM on July 25, 2006

I'm waiting for the technology to take a mediocre acting performance and, in real time, improve the performance so authentically, no one will be able to tell how bad the actor is. Rather than replacing the actor, the technology would need bad actors to enhance. Superb acting wouldn't render properly and would just come out mediocre but due to complicated logarhythms and permutations in the complexities of software exacerbated by grandfathering legacy drivers and old lines of code aquired from other companies because the programmers were lazy, the software would require the unique talents of particularly bad actors, and the end results after rendering would startle and amaze audiences and cause amazing profit margins that'd accentuate film industry output and marketing share.

Then maybe I could live my dream of being a leading male comedic ingenue for motion pictures. They could even airbrush out my sagging beer belly on the fly. I could win an Oscar, but I'd have to share it with the software designers. I could star in The Brad Pitt Story, even though I look more like Kevin Smith. Peter Jackson could direct without ever leaving his posh bungalow in New Zealand. I am optimistic about the future of computing and acting. It wouldn't put all actors out of work. Just the good ones.
posted by ZachsMind at 10:00 PM on July 25, 2006

Yeah I saw this months ago, first off here's a page with a high resolution trailer, much nicer than the choppy flash. There were better examples of motion capture at E3 this year (Andy Serkis in the game Heavenly Sword) but I think the thing most impressive about the Heavy Rain demo is that the developers are thinking outside of what is expected from the video game genre, although we have no idea what the game will be about. MGS4 was technically nicer but there is such a male juvenile vibe to most games I really liked how Heavy Rain looked like it was trying to appeal to adults.

As for the uncanny valley, I feel there have been some really good CG characters. There are several scenes in "A Series of Unfortunate Events" with a CG baby that looked utterly lifelike, I did not notice anything amiss at all during the movie, and I liked the CG Brando in Superman Returns.

Also, here's a webpage with some videos of the actress, Aurelie Bancilhon. It looks like they tried to match the shape of her face.
posted by bobo123 at 11:13 PM on July 25, 2006

The more I watch animation, the more I'm finding "photorealism" of virtual actors to be unexciting from an artistic standpoint. I get more emotional kick from Jan Svankmajer's fornicating cuts of meat (youtube NSFW) than from most of "virtual actor" scenes I've watched.
posted by KirkJobSluder at 9:02 AM on July 26, 2006

I thought the eyes weren't bad, at least the eyebrow action. The gestures weren't bad either but what really bugged me was totally immobile tongue.

It's not the sync with the audio that bothered me either. I don't know if it's a limitation of the 3D model, but her vowels weren't properly formed. Maybe they just need to look at a traditional animator's speech guide.
posted by oraknabo at 9:32 AM on July 26, 2006

One of the most amazing looking games I've ever seen, from an artistic viewpoint, is Okami. I'm sure gamers have already heard of this one but check out the trailers if you haven't. An absolute labor of love from Clover studio.
posted by lucien at 12:09 PM on July 26, 2006

« Older Scientists say they’ve found a code beyond...   |   Clowns Branch Out Newer »

This thread has been archived and is closed to new comments