That's how I got my call sign

October 12, 2023 1:13 PM

"Women, she figures, caught her vibe. "Every woman who read that was like: Mmm-hmm, you go," Cummings says. But men — friends in Silicon Valley — did not. They thought she had been too mean to Vogt. "He was just trying to help you," they told her." Missy Cummings flew F-18s, has a PhD in systems engineering, and is a professor at Duke. Among other things, she researches self-driving cars.
posted by oneirodynia (29 comments total)

Take Cummings' approach in her new paper. First she had to wrestle with NHTSA's nationwide data for nonfatal crashes by human drivers, to get numbers she could compare to California, the only place where the robot cars run free. Then she had to figure out comparable nonfatal crash numbers and miles traveled for Waymo and Cruise, tracked by divergent sources. Her conclusion: Cruise has eight nonfatal crashes for every human one, and Waymo has four — comparable to the crash rates of the fatigued and overworked drivers at ride-hail services like Uber and Lyft.
posted by oneirodynia at 1:14 PM on October 12, 2023 [15 favorites]

Call sign Shrew? LOVE IT.

"A lot of whinging from Tesla pushed NHTSA to force Cummings to recuse herself from anything involving the company. "

Do not love it.

I want to root for self-driving cars as someone who took forever to drive, BUT.....
posted by jenfullmoon at 1:17 PM on October 12, 2023 [7 favorites]

I think self-driving cars are inevitable and I want to see them succeed on a number of fronts (safety, flexible mass transit, potential to enable the transition off fossil fuels) but I have long wondered just exactly how the crash numbers compare. They're not as bad as I feared and not as good self-driving car people want us to believe.

Move fast and break things might be an okay philosophy when things is a website where people share cat photos but it's much less good when things are cats or people.
posted by jacquilynne at 1:25 PM on October 12, 2023 [20 favorites]

I think we have a right, even a responsibility, to demand that self-driving cars do 10x _better_ than human-driven cars before we allow them to use the roads we've all paid for, before we allow them to displace countless jobs and livlihoods, and before the people making them because they're cool and interesting are allowed to make a bunch of money from being allowed to take these things.
posted by amtho at 1:51 PM on October 12, 2023 [58 favorites]

"Move fast and break things" has mostly referred to financial and safety regulations lately, anyway.
posted by ivan ivanych samovar at 1:54 PM on October 12, 2023 [11 favorites]

Ha -- he wrote Muskovites. A quarter trillionaire bottomless well of need is a scary concept in itself. But a QTBWON with legions of idolizing fanboys to whip up is an even scarier concept. Faster and faster in the ever-tightening circle down the drain go we -- all our attention to his hangar filling ego alone may trigger the Singularity. Which will most likely resemble the Run with the Jewels kitties.
posted by y2karl at 2:29 PM on October 12, 2023 [2 favorites]

Driving a car is just such a huge, uncontrolled, and even hostile environment with lots of edge cases involving human behavior and terrible infrastructure. It seems shortsighted if not arrogant to think that it can be completely "solved" by throwing a bunch of cameras and a sufficiently well-trained neural net at it.

Why are these developers so hell bent on creating a drop-in AI replacement that can handle driving all existing cars on all existing roads? That's literally the hardest, most extensive domain one could possibly imagine. It's absolutely nuts from an engineering perspective. And on top of that, they're expecting a future where tens of thousands of these independent autonomous systems will interact with each other on the road without any central scheduling authority or regulations?

I really wish we could separate self-driving cars from the libertarian, techbro fanfic they've come to represent, but at this point it seems impossible.
posted by RonButNotStupid at 2:32 PM on October 12, 2023 [22 favorites]

I don't have anything to contribute to this discussion apart from saying: Dr. Cummings seems fucking awesome.
posted by beepbeepboopboop at 2:41 PM on October 12, 2023 [23 favorites]

I think we have a right, even a responsibility, to demand that self-driving cars do 10x _better_ than human-driven cars

Mark my words: like it or not, it'll be the actuaries that make the final determination on this.
posted by tclark at 2:42 PM on October 12, 2023 [12 favorites]

A researcher after my own heart. Adding her to the list of head-screwed-on-straight people who deserve more attention than they get, right next to Timnit Gebru.
posted by flabdablet at 2:49 PM on October 12, 2023 [10 favorites]

Hi! I know Missy Cummings personally, and I'm very glad she's been working to hold the AV crowd responsible.
posted by TheKaijuCommuter at 2:52 PM on October 12, 2023 [14 favorites]

I suggest we tie Elon Musk to the front bumper of an Tesla for a week, then set the car on autopilot and put it through some extremely rigorous real-world training.
posted by senor biggles at 3:50 PM on October 12, 2023 [11 favorites]

I think we have a right, even a responsibility, to demand that self-driving cars do 10x _better_ than human-driven cars before we allow them to use the roads we've all paid for,

Under this policy, it will be very interesting to live through the interval between 2x better and 10x better, and watch the fatalities tick upward at a steady 40,000 per year. Can we switch yet? No, they're only 8x better. Soon.
posted by officer_fred at 5:05 PM on October 12, 2023 [5 favorites]

I think we have a right, even a responsibility, to demand that self-driving cars do 10x _better_ than human-driven cars before we allow them to use the roads we've all paid for, before we allow them to displace countless jobs and livlihoods, and before the people making them because they're cool and interesting are allowed to make a bunch of money from being allowed to take these things.

I disagree. Why would we allow - never mind force - human drivers to continue running the risk of killing people when there is a safer alternative?
posted by jacquilynne at 5:12 PM on October 12, 2023 [6 favorites]

The death toll is actually way worse when you look at Tesla- they aren’t running contorted tests in the way Waymo and Cruise are. In my experience Tesla drivers are less attentive than average, beacuse they think the car will help. just Glad Dr Cummings. Is holding everyone accountable
posted by CostcoCultist at 5:15 PM on October 12, 2023 [2 favorites]

From the article:

But other dangers, Cummings says, are more subtle — "sociotechnical," as she puts it. What she calls the "hypermasculine culture in Silicon Valley" intertwines with Big Tech's mission statement to "move fast and break things." Both bro culture and a disruptive mindset, as she sees it, incentivize companies to gloss over safety risks.

The other sociotechnical factor that I think everyone kinda knows but is somewhat impolite to bring up is that we simply can't trust the data that these companies provide. At this point, whether it's Tesla or Google or whoever, no one should be giving tech companies the benefit of the doubt. While any individual employed at these companies might have things like intellectual honesty or scientific integrity or whatever, the organizations that employ them cannot. Their corporate organizations operate like paperclip maximizers, except their paperclips are profits. Anything that deviates from that -- including scientific integrity -- can be, and often is, discarded.
posted by mhum at 5:55 PM on October 12, 2023 [19 favorites]

Self-driving cars are not safe, but they are coming anyway. Not because of the false pretext of safety but because car ownership needs to be relegated to the ownership class, and self-driving vehicles will facilitate near or full privatization of transit options.
posted by They sucked his brains out! at 6:09 PM on October 12, 2023 [6 favorites]

We can still rely on all that stuff Philip Morris and BAT told us about smoking being good for us, right? And I'm sure the lovely folks over at Exxon would be the first to stand up and say so if there was anything to this whole global warming panic.
posted by flabdablet at 6:11 PM on October 12, 2023 [9 favorites]

I disagree. Why would we allow - never mind force - human drivers to continue running the risk of killing people when there is a safer alternative?

This is a fair point that should be explored - but in the context of the entire question being explored.

It's not just about safety, though. First, it's _not that hard_ for the "smart" folks working on this to just do better. These are creative well-educated people who aren't even trying to make their self-driving systems include road-based tech, who aren't focusing on long-distance trips first (the greatest need because there are fewer options for non-drivers), who aren't investing part of every dollar into setting up a system so that people who would otherwise be driving professionally have a choice of more free time instead of desperately looking for yet another job that relies on extracting value from places it should have remained.

In other words, people are seeing dollar signs, and there is a big, big opportunity to turn them into more fairness if there's enough public will.
posted by amtho at 6:29 PM on October 12, 2023 [2 favorites]

I'm glad that this research is being done. I also think.
no know..the data for human driving is utter shit. Between the lack of granular data that can be directly compared with the limited conditions in which everybody but Tesla operates in, the substantial underreporting of non-injury crashes to police/states, and the stubborn refusal of state and local governments to properly code and report the data that they do collect, it's safe to say that the number of non-injury crashes are far greater than the data shows.

Cruise and Waymo, at least, appear to be reporting all of their (contact) incidents in California, as required by law. That puts them at a substantial disadvantage, especially when it comes to low stakes events that cause little more than paint damage. A lot of drivers in incidents like that get out, decide it's not worth the time and effort to bother with, shout "no damage!" and drive off. Or they do it more cooperatively with the other driver. Hell, I've personally been the person saying "don't worry about it" when someone rear ends me at low speed. That's one reason why I drove old beaters with 5MPH bumpers when I still drove. When my SO clipped the corner of my sister's bumper while parallel parking in front of her car did anybody report that? No, I took it to a friendly body shop where the guy took out a heat gun and massaged the plastic back into place and I flipped him a $50 for his trouble.

The point is that we don't have any real idea exactly how much financial damage is being done by cars colliding with each other and roadside objects because so much of it goes unreported. It also happens that the kind of collisions that make up the vast majority of Cruise and Waymo's collision reports are exactly the kind that are least likely to be reported by a pair of human drivers.

All that said, I need to dig in to Cummings' work specifically. If she was able to compare the rates of crashes that caused one or more vehicles to be disabled, that would be much more easily comparable since those are much less likely to go unreported. It still happens, but less so in cities, which happen to be the only place Cruise and Waymo operate.

(Also, the BI reporter is wrong about California being the only place AVs operate. Waymo famously began their operations in Arizona. Cruise is testing in several other states. What is unique shit California relative to AVs is that they are the only state that requires AV operators to file reports in a publicly available database. They're the only ones forcing some level of transparency on them.)
posted by wierdo at 6:39 PM on October 12, 2023 [5 favorites]

I'm pro-self-driving cars and Ms. Cummings sounds like exactly the sort of person I want watching over the companies to make sure they do it right.

I am certain that low-speed collisions with minor damage are under-reported by human drivers, and frankly these aren't the sorts of incidents that should be the basis of regulation. When an event is literally solvable with a small cash payment that is all that is needed to solve the event. The real question is the degree to which this is a proxy for the things that we care about.

I remain optimistic that this technology will ultimately save health and lives, but we need to learn a lot more to say that is the case. At the same time, as someone who watched early self-driving vehicles navigate (closed off) parts of my college campus in the mid 90's, I'm bewildered by those who suggest there is anything sudden about these technological developments. This is the status of an effort over three decades old.
posted by meinvt at 6:52 PM on October 12, 2023 [1 favorite]

Flying is famously much safer than driving. This is because of the regulatory environment around flying which in turn enforces a safety culture to comply. And ONE reason for that is that as passengers, we are at the mercy of the airline and its employees and have zero control over the safety outcome. Whereas our psychology in driving is quite different, as drivers we have considerable control over our own safety. I think the difference in perception is important here. As a society we have demanded more from aviation because it is subjectively scarier than driving.

So I think one reason to hope for aviation-level safety in autonomous vehicles is that it will be scary to be at the mercy of the autopilot as a passenger... but it might take a few passengers being maimed and killed to get there.
posted by i_am_joe's_spleen at 7:03 PM on October 12, 2023 [5 favorites]

I should have noted that I am all for better reporting and stronger oversight of these pilot AV programs. California does better than most states, but it's not really good enough. At the very least, regulators need access to the internal data of these companies and the ability to require them to address specific defects they find in a timely manner.

In the current scheme in California there's no way to know if the AV's behavior is contributing to the crashes by behaving in a manner that is theoretically more safe but causes bad reactions from human drivers. Human drivers should be better, but people have to be met where they are. The complicating factor, of course, is that adjustments to the AVs can't make the roads less safe for people not in cars. At least a few of the incidents I've seen in the California data involved an AV getting rear ended because they actually stopped for a pedestrian as legally required. Should this take going against the AV? I'd argue that risking some property damage is better than risking injury to the pedestrian, but it still catches drivers off guard. Sadly, I don't know how we keep that kind of thing from happening.
posted by wierdo at 8:08 PM on October 12, 2023

As an apropos I wanted to find a post on Twitter/X, but as both that platform and Google has gone to shit I can't seem to find it. So let me sketch it out for you: This guy had some beef with Tesla and/or Musk, and had a post banned for tweeting something along these lines: "If I were to see Elon Musk crossing the road in front of my Tesla, I would turn on the autopilot". Wham, banned for issuing threats.

He had no luck getting Twitter to explain why it was considered a threat, though...
posted by Harald74 at 9:43 PM on October 12, 2023 [11 favorites]

I don’t believe any of the current data are even worth looking at.
posted by Phanx at 1:15 AM on October 13, 2023

IMO this article is kind of bad, but I do agree we need the equivalent of a NTSB for self-driving cars. It's easy to see that the driving software logic is going to converge - and every fatal crash should be investigated and lessons learned provided and software updated to reflect it. And if required, they should be 'grounded' until the investigation and update is done.
posted by The_Vegetables at 8:15 AM on October 13, 2023 [1 favorite]

Just a reminder that it's Dr. Cummings, not Ms. Cummings. She has a PhD. Have some respect.
posted by acridrabbit at 5:29 PM on October 13, 2023 [3 favorites]

Shouldn’t self driving cars fall under the purview of the current NTSB?
posted by The Monster at the End of this Thread at 8:20 AM on October 14, 2023 [1 favorite]

tclark: like it or not, it'll be the actuaries that make the final determination on [the demand that self-driving cars do 10x _better_ than human-driven cars].

The insurance is key: I won't take liability for a vehicle I own but am not in control of, nor for a vehicle whose control programming I can't adapt or improve. Imagine I lease an autonomous vehicle and the insurance is handle but the owners and paid as part of the lease, then the cost of that insurance pool (the class of people sharing the risk) looks to bugs triggered per mile traveled. The cheapest way to share out that cost is more bums on seats in each vehicle -- and here is where autonomous vehicles reinvent mass transit.

The actuaries ought to get together with the Ownership Class to create a second great age of Rail (with autonomous bus links where needed).
posted by k3ninho at 4:35 AM on October 15, 2023 [2 favorites]