These are the times we live in!

November 22, 2017 4:01 PM

Meet virtual YouTuber Kizuna A.I.

Watch her challenge google's Quick, Draw!, chat with Siri, furnish her room and buy some friends.
She also answers fan questions, offers advice, and plays PUBG, Splatoon 2, Break-Up Dodge, Superhot VR, and many other games.

(English subtitles)

posted by lucidium (13 comments total)

This is not one of those times when "don't read the comments" means don't scroll down to where people are all but literally drawing drool, bile, urine, feces, or blood out of their own bodies to scrawl hideous and poorly spelled threats on the screen in attempts to be as repellent as possible.

Instead, don't read the comments because you will confront the rising tide of slavering otaku piling on top of each other in their eagerness to praise the ultimate fulfillment of their perfect woman.
posted by ardgedee at 4:33 PM on November 22, 2017 [4 favorites]

I watched about half the self-introduction before finding out a) my sound doesn't work for some reason and b) it was in-character, so there's no indication of how it works. My assumption is that it's a vocaloid-type program rather than it being 'artificial intelligence'.

This is basically an excuse to post one of my favourite YouTube videos, Barbie (as in the Mattel character) cosplaying as GlaDOS. It is bafflingly delightful: someone at Mattel deliberately made this, and someone definitely approved it, and now Barbie has a canonical favourite video game.
posted by Merus at 5:55 PM on November 22, 2017 [4 favorites]

She doesn't sound like a vocaloid to my ears. I'm guessing actress and motion capture of some kind but frankly who knows?
posted by Mr.Encyclopedia at 9:00 PM on November 22, 2017 [1 favorite]

I was thinking similarly. There's face-tracking software for the Kinect, so the cost of entry for hobbyists is pretty low.
posted by ardgedee at 4:26 AM on November 23, 2017

You aren't going to tell me I am creepy if I say the one about buying friends was adorable (barring the whole, erm, bounteous mammaries comment) are you? I liked the idea of the perspective shift of an "A.I." buying digital assets to add to a virtual lifestyle. (The DIY challenge was rather cute too.)
posted by Samizdata at 10:19 AM on November 23, 2017

Also, there is something about the clipped nature of the speaking along with the fact there's nary a stumble, stutter or filler words makes me think it is software talking.
posted by Samizdata at 10:26 AM on November 23, 2017

Yeah, technology aside (my guess is kinect + mmd + a bunch of work), I don't think it would stand up if she wasn't also a genuinely watchable lets player. This screen cap keeps cracking me up.
posted by lucidium at 11:09 AM on November 23, 2017

> there's nary a stumble, stutter or filler words makes me think it is software talking.

There are a lot of trailing words, misspeaking and vocalizing in her PUBG and Resident Evil LP videos.
posted by ardgedee at 1:18 PM on November 23, 2017

Just guessing, but I'd guess is that this is a set of recurrent neural networks generating a bunch of output, but curated by a human.

The "quick, draw" one was thought-provoking... The whole point of that is to make a game of training Google's doodle recognition neural net, to provoke humans into being unpaid teachers. Regardless of what Kizuna AI is or is not doing, the idea of pointing an AI at another AI to get training from humans indirectly is interesting. It probably wasn't intended to be used that way, so it's kind of subversive in a strange way.

Perhaps the next logical progression is for Google (and whoever else) to catch on to this, and try to sell the attention of AI users like they have with their human users.
posted by swr at 3:25 PM on November 23, 2017

Everybody gets that this is a performance, right? This is not an actual AI? No program in the world has both the ability to perfectly parse Japanese from graphical or auditory input, need to carefully sound out English, and also instantly understand that the reason their game character died in a fire is because an unspecified liquid leaked onto the floor and subsequently ignited in Resident Evil 7. That's just not a thing. Certainly not a neural network thing. They are as dumb as bacteria that have been subjected to evolutionary pressure to perform a specific task to a certain level of adequate.

All that said, Kizuna A.I. is a wonderful fiction, with impressive production values and performance capture and I love (and am vaguely terrified by) the idea of an actual childish AI figuring out our world through games and online interaction. Also, she's learned about swearing.

But the closest thing to artificial intelligence in here is the part that takes spoken lines and turns them into mouth positions on the avatar. I'm pretty sure the facial expressions are manually edited in after the performance is recorded.
posted by figurant at 8:22 PM on November 23, 2017 [1 favorite]

So from reading around, it looks like Kizuna A.I. is controlled by a Kinect, probably with facial tracking otherwise her live streams would be pretty hard to take, and animated in Miku Miku Dance. More info and a link to the source model on the MMD wiki.

The reason why I'm guessing facial tracking is that in some of her videos the mouth movements fall apart when Kizuna is leaning over something, such as when playing a game on her phone. And in general it looks like only certain body points from the Kinect tracking data are being used by MMD because there are moments in many videos in which her limbs will clip through her body, I'm guessing (I'm guessing a lot here) due to the differences in proportion between human and anime figures.
posted by ardgedee at 5:31 PM on November 29, 2017

Incidentally, according to Know Your Meme, the first A.I. Channel video was posted one year ago today.
posted by ardgedee at 5:35 PM on November 29, 2017

Aw, if I'd clocked that I would have waited to make the post coincide.
posted by lucidium at 4:39 PM on November 30, 2017