Like Uber, but for automated carjackings...

May 19, 2017 3:22 PM

A neural network is attempting to learn to drive in GTA V [CW: Profanity, and probably some car crashes] [via waxpancake]
posted by schmod (20 comments total)

I'm dying at the start/stop/start/stop. It appears to like driving along the center divide as a visual guide marker/boundary condition.
posted by Existential Dread at 3:28 PM on May 19, 2017

So that's how Google did it.

Also, this neural net drives about as well as I do in GTA. So there's that.
posted by SonInLawOfSam at 3:29 PM on May 19, 2017 [3 favorites]

"At the moment, Charles learns and takes all actions based on single frames at a time, and bases his decisions on just pixel data. Charles only sees exactly what you see.

In time, I intend to give Charles some short-term memory to hopefully improve his driving."

So it doesn't seem that it's actually learning yet -- still very funny regardless!
posted by Snacks at 3:33 PM on May 19, 2017

I love this thing. I watched a bunch of it in distracted bits and pieces today, and it's an interesting mix of just plain bad reckless driving and interestingly weird misunderstanding of the concept OF driving.

My favorite bits:

- When the car drove along the beach for a while before veering into the ocean for no reason.

- When the car got dropped into an aqueduct, and then didn't know what the hell was going on and really wanted to be driving on the ramped sides rather than down in the middle

- When the car drove out onto a raised highway and then suddenly got confused and kept executing three point turns and occasionally stopping to, seemingly, look at the nice windmills
posted by cortex at 3:40 PM on May 19, 2017 [5 favorites]

Currently it thinks it's a skateboard.
posted by Foci for Analysis at 3:40 PM on May 19, 2017 [1 favorite]

Short-term memory: just do frame stacking maybe? Would probably work fine if your only goal is to maximise velocity...
posted by tss at 3:41 PM on May 19, 2017

omg it just killed a bunch of bunnies and in the background you can hear the police execute some dude
posted by Foci for Analysis at 3:42 PM on May 19, 2017 [2 favorites]

it has committed suicide by drowning TWICE. it suffers
posted by Foci for Analysis at 3:44 PM on May 19, 2017 [1 favorite]

There are posts here (I searched "GTA" to find posts about this thing).

It's not 100% clear to me what's going on but I do think it is "learning" in the sense that the NN is being trained....somehow. I'm not 100% sure what the training data is. "not having memory" in this case means the computer is only using as input a single frame of input and making a decision based on that.
posted by quaking fajita at 3:46 PM on May 19, 2017

cool, based on how well I do at driving games neural nets are actually smarter than me

but i can eat tacos so suck on that, neural nets everywhere
posted by palomar at 3:51 PM on May 19, 2017 [5 favorites]

Ah, sorry, the author just confirmed in the chat that it's not currently training.
posted by quaking fajita at 3:52 PM on May 19, 2017

It appears to like driving along the center divide as a visual guide marker/boundary condition.

I mean, to be fair, so do I.
posted by Ray Walston, Luck Dragon at 4:07 PM on May 19, 2017

If this thing ever escapes into BeamNG.drive I'm pretty sure it's only a matter of time before Azathoth consumes us all.
posted by howfar at 4:24 PM on May 19, 2017 [2 favorites]

Okay, so input is a single image, output is the controls. But the big question I have is what does it count as "success?" If you just said don't crash into anything, you'd probably get a car that sits still.

If it's not currently training while we're watching, I'll bet it was trained on footage of humans driving in the game, and it's doing its best imitation of that.
posted by RobotHero at 4:25 PM on May 19, 2017

Maybe it trained on this.
posted by linux at 4:43 PM on May 19, 2017

Can it eat peyote, become a jungle cat and run around mauling golfers?
posted by RobotVoodooPower at 4:53 PM on May 19, 2017 [3 favorites]

Here's a write up about how it works, what it sees, with example python code. Looks like it's just trying to do whatever it takes to keep the road/lane and horizon lines ahead of it and avoid things that break those lines. This'd explain why it sucks so bad at off-roading. I'll be looking forward to when it has some short term memory, and it'd be cool if the dev added analog controls, too!

On another note, don't bother trying to link to images from that page. The entire image is encoded into the URL. I found out the hard way by overloading my SSH session trying to paste it into IRC. Twice.
posted by destructive cactus at 8:46 PM on May 19, 2017 [1 favorite]

Call me when it can turn around on foot and walk through a doorway in GTA... without banging into the doorframe six times and stumbling up and down the stairs trying to line it up first.

It's very pretty and really good at procedural rendering, quickly, but GTA/Red-Dead/LA-Noire/that-engine drives me absolutely batty with the feeling of not being in control of my own character's movements. It's so muddy that it feels like trying to play a game over a remote desktop connection.
posted by rokusan at 2:42 AM on May 20, 2017

Kudos for not calling it an AI like every other site I've seen it on this week.
posted by GallonOfAlan at 7:01 AM on May 20, 2017 [3 favorites]

The car just went out of its way to mow down a dog... The 'he is going to jail for doggo murder', 'RIP pupper' etc comments made me laugh. Then, on the next start, the car backed up a hill, was surrounded by police, waited until a cop got out and shot at it, then ran the policeman over.

This computer Is pretty brutal.
posted by Fig at 12:23 PM on May 21, 2017