"A car that has a JSON parser implemented in Bash 3"
August 24, 2018 11:38 AM   Subscribe

A former Tesla employee, who worked on their IT infrastructure, is posting in a subforum of a subforum, a little-known place for funy computer forgotten by time. His NDA has expired. A single link Twitter thread with screenshots of text.
posted by Artw (135 comments total) 56 users marked this as a favorite
 
Bonus points if you can identify the one that made me physically yell.
posted by Artw at 11:39 AM on August 24, 2018 [6 favorites]


Bonus points if you can tell me how I join this glorious ANSI BBS in 2018.

/es
posted by snuffleupagus at 11:42 AM on August 24, 2018 [15 favorites]


Trick question, all of them are nightmares given form.
posted by Holy Zarquon's Singing Fish at 11:42 AM on August 24, 2018 [2 favorites]


Source link at SomethingAwful. Interesting mostly because you can see the poster's history there, dating back to 2001. I've been wondering if these messages are authentic, maybe digging through the history would help verify them.
posted by Nelson at 11:42 AM on August 24, 2018 [7 favorites]


A single link Twitter thread with screenshots of text
Perhaps the worst method of information transfer the human race has come up with.
I imagine we'll be seeing it a lot.
posted by thatwhichfalls at 11:46 AM on August 24, 2018 [87 favorites]


> thatwhichfalls:
"Perhaps the worst method of information transfer the human race has come up with."

Yeah. I get that people use twitter because they already have an account and followers, etc., but it's never the best medium for, well, anything.
posted by signal at 11:54 AM on August 24, 2018 [8 favorites]


As usual, reading about other people's shitty hacks makes me feel way better about my own shitty hacks.
posted by tobascodagama at 11:56 AM on August 24, 2018 [58 favorites]


Yeah. I get that people use twitter because they already have an account and followers, etc., but it's never the best medium for, well, anything.

It's almost like a metaphor for Tesla's infrastructure.
posted by uncleozzy at 11:56 AM on August 24, 2018 [6 favorites]


Also, I really can't wait to see the n-gate post for this.
posted by tobascodagama at 11:57 AM on August 24, 2018 [7 favorites]


the best part about this being on twitter is that everyone can easily re-tweet it to elon musk and he reads EVERYTHING in his twitter feed
posted by Atom Eyes at 11:59 AM on August 24, 2018 [28 favorites]


Presumably he will take out a full-page ad in the WSJ calling this guy a pedophile.
posted by uncleozzy at 12:02 PM on August 24, 2018 [38 favorites]


A sobering thought: Its reasonable to assume that Tesla is the car company most like this, but they are all rolling pieces of software now, and so all to some extent a bit like this. It's going to be impossible to drive without riding some messy nest of VMs that chatter away with their home server anymore soon.

(Elon Musk is pretty awful in his own way though)
posted by Artw at 12:02 PM on August 24, 2018 [14 favorites]


His NDA has expired.

I predict tech companies pulling some Disney-copyright-level law rewriting to make NDAs last for 70 years after the death of the employees' grandkids.
posted by signal at 12:03 PM on August 24, 2018 [22 favorites]


I'm glad at least one other person thinks Tesla cars are laughably ugly. I smirk whenever I see some late-forties tech doofus (who invariably sports a smartphone belt holster) driving one of these plasticky cars around Victoria BC.

Tesla's design reminds me of the Pontiac Aztek.
posted by JamesBay at 12:04 PM on August 24, 2018 [8 favorites]


I've got a few hundred stories like this

Don't we all brah, don't we all.
posted by gwint at 12:05 PM on August 24, 2018 [10 favorites]


I mean, it's been said many times before but: If you saw all the janky code that runs your life, you'd throw away all your tech and never leave your house.
posted by gwint at 12:07 PM on August 24, 2018 [106 favorites]


Metafilter: I powered my old car with farts too.
posted by Naberius at 12:14 PM on August 24, 2018 [8 favorites]


i would bet that the quality of code, and software development process, are as bad or even worse in any other car company.

in general i think companies whose product is hardware are at best mystified by and at worst actively contemptuous of software, which is why so much IOT stuff is insecure garbage.

where Tesla seems to be uniquely, hilariously bad is how much their cars are networked, and how hacky that setup is. (like, they actually have server problems during rush hour in California, because all the cars are constantly talking to their one database.)

then again, it's possible to hack a jeep over the air and make the brakes lock up, so maybe everything else is similarly fucked.

somebody ought to make a horror movie where all the cars get reprogrammed to hunt and kill people.
posted by vogon_poet at 12:22 PM on August 24, 2018 [22 favorites]


I predict tech companies pulling some Disney-copyright-level law rewriting to make NDAs last for 70 years after the death of the employees' grandkids.

I expect current contracts specify they own your soul for 1000 years, Scientology style.
posted by Artw at 12:24 PM on August 24, 2018 [2 favorites]


Bonus points if you can identify the one that made me physically yell.

Wait, I found it. It's the $100,000 car without a $2* battery-backed clock unit.

* at least that's what I pay for a PCF8563 and 0.1F supercap.
posted by JoeZydeco at 12:25 PM on August 24, 2018 [14 favorites]


The car that runs on Linux time IS a high point, yes.
posted by Artw at 12:27 PM on August 24, 2018 [2 favorites]


Its reasonable to assume that Tesla is the car company most like this, but they are all rolling pieces of software now, and so all to some extent a bit like this

Huh. So, if any given car company goes out of business, will their entire fleet just stop working? I'd honestly never realized there was so much reliance on communication to and from centralized infrastructure.
posted by treepour at 12:30 PM on August 24, 2018 [4 favorites]


(This one did it for me - that’s some hell management)
posted by Artw at 12:30 PM on August 24, 2018 [1 favorite]


That post about the lack of a RTC for the infotainment system....it talks about how everything is bottlenecked by monolithic log ingestion. If that's the same thing as the saturation of the database during rush hour....maybe someone should tell Elon about Splunk.
posted by snuffleupagus at 12:30 PM on August 24, 2018 [1 favorite]


tesla isn't encrypting their firmware and it's really easy to glean information from the vpn with a packet cap because nothing inside the vpn (was) encrypted.
...
for example, at one time you were able to root a model s with a usb stick and a gstreamer exploit.


That sounds... fascinating. And horrifying.
posted by ErisLordFreedom at 12:31 PM on August 24, 2018 [2 favorites]


somebody ought to make a horror movie where all the cars get reprogrammed to hunt and kill people.

Close enough
posted by a snickering nuthatch at 12:31 PM on August 24, 2018 [3 favorites]


"while tesla should be given credit for updating the car over the air to fix issues ...

No. No, they should not. It is the worst idea in the history of the automotive industry.
posted by sfenders at 12:31 PM on August 24, 2018 [25 favorites]


Waitaminnit - so that means they don't even pull time from a GPS receiver? Even the IVI on my cheap little Mazda can do that.
posted by JoeZydeco at 12:33 PM on August 24, 2018 [4 favorites]


I recently did some (absolutely non-critical, I assure you) firmware extraction in a bash script using a mix of dd, od, awk and netpbm (hey, how else would you transpose an 11 byte column-wise vector into an 8×11 bit array on the command line but with pnmflip?). After reading this thread, I feel much more professional.
posted by scruss at 12:37 PM on August 24, 2018 [11 favorites]


OK, what is this shit about algorithmic blacklisting of supercharger "abusers?"

If the battery packs wear out and need to be replaced, tough shit Tesla. Selling a lifetime supercharger entitlement with a car and then surveilling a driver's habits in order to deny them access to that benefit seems fraudulent and actionable to me.
posted by snuffleupagus at 12:41 PM on August 24, 2018 [14 favorites]



while tesla should be given credit for updating the car over the air to fix issues ...

No. No, they should not. It is the worst idea in the history of the automotive industry.


It's a competitive advantage for them right now. And the day it gets hacked could very well be the end of Tesla-- except that everyone else wants to get in on the bandwagon.
posted by gwint at 12:42 PM on August 24, 2018 [3 favorites]


( Fuck you Bosch).

I learned this mantra with my 99 Passat, and here we are 20 years later...
posted by mikelieman at 12:42 PM on August 24, 2018 [4 favorites]


OK, what is this shit about algorithmic blacklisting of supercharger "abusers?"
everyone else wants to get in on the bandwagon

sounds like somebody'll be driving a bannedwagon
posted by uncleozzy at 12:44 PM on August 24, 2018 [34 favorites]


“china has a law in place that mandates all electric cars send real time telemetry to their government servers - model s/x/3, NIO cars and any other electric car if they’re driving already complies with that law to be road certified.”

(from the forum thread) - Ummmm this part stood out to me the most! Google turns up nothing, but that doesn't necessarily rule it out. Does anyone here have more knowledge on this?
posted by aperturescientist at 12:48 PM on August 24, 2018 [6 favorites]


I smirk whenever I see some late-forties tech doofus

Yeah, old people and nerds are the worst. Good thing you’re so cool up in Victoria!
posted by Abehammerb Lincoln at 12:51 PM on August 24, 2018 [24 favorites]


i can just imagine how the json parser in bash grew out of
some edict that python or JS was inherently unsafe and would be stripped from the local Centos install, and some engineer saying "welp, we've got bash in the distro so hold my beer."
posted by zippy at 12:53 PM on August 24, 2018 [21 favorites]


I'm having trouble googling it but is/was Tesla really selling cars without interior manual handles that are difficult or impossible to open without power?

Yes, they are. I wouldn't get in one without a screwdriver.
posted by dilaudid at 12:59 PM on August 24, 2018 [4 favorites]


[Can we get the FPP's link changed to https://twitter.com/atomicthumbs/status/1032939617404645376, since that's the actual top of the thread?]

Sure, have ArtW contact us and say it's cool.
posted by jessamyn at 1:09 PM on August 24, 2018 [2 favorites]


Seriously this is fucking horrifying.
posted by nikaspark at 1:11 PM on August 24, 2018 [1 favorite]


I yelled at "Jenkins" WHY IS THE ENTIRE FLEET CONNECTED TO A CICD PIPELINE.

This is madness!!!
posted by nikaspark at 1:13 PM on August 24, 2018 [6 favorites]


If this is all fake, kudos to the shortsellers.
posted by gwint at 1:16 PM on August 24, 2018 [11 favorites]


Feel like it's only fair to include a follow up post from the person that made the thread after it exploded on Twitter:
man twitter is so dumb this isn't even news. big shock - a large company has some issues with development and some of the issues were lol bad but I guarantee you the technical issues i posted are long gone now

bringing up hardware is always messy and with tight deadlines, shit happens, so to be real for a moment: in my opinion tesla is shipping good (model s/x) products, they are road safe and tested, there are growing pains, quality pains and such but nothing they can't fix. model 3 is another story, I hope they fix it

honestly i started out really liking working there and I feel like my 8 years there had an impact and tesla to this day is still full of good folks doing the right thing. i know for a fact all the crap i worked on is probably gone by now as it should be - they are evolving. you'll notice they haven't had further incidents like the thing I posed earlier with the reboot loop, so at the very least rca's are paid attention.

most of tesla's problems are self-imposed management style issues, they'll be forced to fix them at some point, one way or another, but that's up to the board
posted by foxfirefey at 1:16 PM on August 24, 2018 [22 favorites]


A single link Twitter thread with screenshots of text
Perhaps the worst method of information transfer the human race has come up with.


I see you are unfamiliar with the phenomenon of still videos of screenshots of text, to appeal to algorithms that prioritize the display of videos.
posted by one for the books at 1:21 PM on August 24, 2018 [41 favorites]


Except that (as his posts note} the technical issues are driven by the management ones. It's clear that the shit show he described ultimately stems from the piss poor management at Tesla.
posted by NoxAeternum at 1:22 PM on August 24, 2018 [4 favorites]


> one for the books:"I see you are unfamiliar with the phenomenon of still videos of screenshots of text, to appeal to algorithms that prioritize the display of videos."

Nuke em from orbit.
It's the only way to be sure.
posted by signal at 1:23 PM on August 24, 2018 [6 favorites]


this is why I drive a 1990's toyota and always will
posted by scose at 1:27 PM on August 24, 2018 [8 favorites]


tobascodagama: Also, I really can't wait to see the n-gate post for this.

I'm not sure whether the prime directive applies, but yes.

Reading this is funny, a bit worrisome about heroic measures and burnout and technical debt, and terrifying, because cars can save or kill people.
posted by Pronoiac at 1:32 PM on August 24, 2018 [2 favorites]


this is why I drive a 1990's toyota and always will

Fun fact: Any car made before 2003 can now be classified as a classic car in some areas, and likewise cars made before 1993 can now be certified as antique.
posted by Copronymus at 1:34 PM on August 24, 2018 [6 favorites]


a jenkins pipeline once cause almost the entire fleet to reboot loop for about an hour

it's funny because it's true and also nightmare inducing
posted by numaner at 1:40 PM on August 24, 2018 [5 favorites]


Yeah, my wife has been interested in a Tesla conceptually for years, and when they went from astronomical to "OK maybe you can afford the cheap one" we kinda put it on the long-term maybe list. This stuff makes me think that's gonna be a hard no, though. This is insane. The idea that the car manufacturer is going to continuous release to my fucking car (and that they're running this shit on e.g. VMs and a hope and a prayer, vs any kind of purpose-built embedded software) is terrifying.
posted by tocts at 1:50 PM on August 24, 2018 [3 favorites]


NoxAeternum: I just think that if the source of the thread being discussed has that kind of reaction to the reception of what we're saying, it should be noted.

I certainly still think Tesla is a trashfire. Well, more technically, a carfire. I mean, this is a company that was supposed to produce 5,000 cars in a week recently for a milestone Elon Musk set and it turns out the way they did it (by grinding every employee into dust) meant only 14% of the cars did not require rework, compared to a good auto plant where 80% of cars pass after first inspection.
posted by foxfirefey at 1:52 PM on August 24, 2018 [6 favorites]


I was trying to understand what "Bash 3" meant, for the JSON parser. Is this some brand new variant of Bash? No, no, it's a very old version of Bash. "Bash 4" was released nearly 10 years ago and has a lot of nifty features. Tesla is using something 10 years old or older.

They have good company; MacOS also ships Bash 3 along with all the other ancient crap that makes MacOS such a bad Unix. In Apple's case it's assumed to be a license thing, Apple doesn't want GPLv3 code and has slowly been expunging all copyleft code from its system. (Maybe because of the GPLv3 patent licensing provision.)

Which got me wondering, does Tesla comply with GPLv2 requirements to distribute source for the copyleft binaries they use in the car, like their version of Bash? Apparently not, but they pinky-swear they are working on it. How many years have they been shipping this code while ignoring their legal requirements? They've been "engaging with" open source legal advocates since 2013. That doesn't sound like good faith to me.
posted by Nelson at 1:54 PM on August 24, 2018 [13 favorites]


somebody ought to make a horror movie where all the cars get reprogrammed to hunt and kill people.

THE FATE OF THE FURIOUS has you covered, if you consider Charlize Theron in Matrix II dreads horrific.
posted by Ten Cold Hot Dogs at 1:55 PM on August 24, 2018 [2 favorites]


I mean it was a literal trash fire yesterday.
posted by graventy at 1:58 PM on August 24, 2018 [3 favorites]


Hank Scorpio: The key to motivation is trust. Let me show you what I mean. I want you to close your eyes and fall backwards, and then I'll catch you. That's gonna show you what trust is all about. Ready?

Homer: Gotcha!

[Homer closes his eyes and prepares to fall backwards. Hank holds out his hands, ready to catch Homer]

Hank Scorpio: Three... Two...

[nearby phone rings]

Hank Scorpio: One second.

[Homer falls backwards as Hank picks up the phone. Naturally, Homer falls straight to the ground with a loud thud]

Hank Scorpio: [to the guy on the phone] Oh my god, guy's on the floor!

[to Homer]

Hank Scorpio: That was a phone call! Don't chalk that up to mistrust, now.
posted by Ironmouth at 1:58 PM on August 24, 2018 [13 favorites]


I saw this in the follow-up:
but I guarantee you the technical issues i posted are long gone now [...] i know for a fact all the crap i worked on is probably gone by now
And I mean, even setting aside the fact that the person starts equivocating in the post that's supposed to exculpate Tesla, A) why would a person have any faith that those issues are all gone, when the same problematic culture that created them has never changed, and B) even if the specific issues you mentioned are gone, why would a person have any faith that even more awful technical issues haven't arisen to take their places, when the same problematic culture yadda yadda.

(Also, yeah, this whole thing totally did make me feel much better about many hacky choices I've made in building software and infrastructure.)
posted by protocoach at 1:59 PM on August 24, 2018 [4 favorites]


Until a few weeks ago I had a vague amount of respect for Musk and a vague desire for a Tesla, in a "I want every cool fancy tech thing" kind of way. Then I started following the something awful thread, which is an incredible read. I highly recommend it.
posted by graventy at 2:00 PM on August 24, 2018 [8 favorites]


if you think tesla has it bad, ohhh man. there are other connected cars (jeep) which have been demonstrably remote-rooted in really horrific ways. there's a bunch of shit coming out of china that was made by the Chinese company and Bosch/delphi/whoever and i guarantee you they don't do due diligence like we did. my point in general being that connected cars, privacy, security - these are industry wide issues and should be addressed
Much as I enjoy the lol Tesla fun, this is every damn car now. It's terrifying.
posted by zachlipton at 2:03 PM on August 24, 2018 [17 favorites]


Except that (as his posts note} the technical issues are driven by the management ones. It's clear that the shit show he described ultimately stems from the piss poor management at Tesla.

Also, "every tech company is just as fucked up" is obviously not news to anybody who works in tech, but it's also not, like, a good thing.
posted by tobascodagama at 2:05 PM on August 24, 2018 [15 favorites]


Honestly, I don't trust anybody else who works in tech and manages not to be a complete Luddite.
posted by tobascodagama at 2:05 PM on August 24, 2018 [17 favorites]


Also, "every tech company is just as fucked up" is obviously not news to anybody who works in tech, but it's also not, like, a good thing.

Also also: this is not how it has to be! It's the result of a series of choices and incentives, not just a natural consequence of making things! I have relatives who work at a factory that's been producing chemical products for over sixty years with something like three product contamination issues in that time, all of which were caught quickly and contained before anything made it to a place where it could affect a customer. It's not even a natural consequence of being in tech; NASA builds software that works for decades.
posted by protocoach at 2:10 PM on August 24, 2018 [21 favorites]


Time travel challenge:

Go back 20 years and tell your younger self: I just filled up my hard drive because I wanted to ride a bike.

(Happened to me because I was installing the apps for Ofo, Limebike, and AntCycle.)
posted by ocschwar at 2:15 PM on August 24, 2018 [2 favorites]


> Also also: this is not how it has to be! It's the result of a series of choices and incentives, not just a natural consequence of making things!

It's the result of capitalists discovering that software is the ultimate way to make money, and a lack of government ability to sufficiently punish tech companies for being irresponsible. We as citizens have the power to stop this if we want. "Temporarily embarrassed millionaire" syndrome will make it difficult.
posted by scose at 2:16 PM on August 24, 2018 [3 favorites]


Go back 20 years and tell your younger self: I just filled up my hard drive because I wanted to ride a bike.

20-years-younger-self: "Wow, why the heck would you need 100 MB of bike software??"
posted by tobascodagama at 2:17 PM on August 24, 2018 [28 favorites]


All I'm seeing here is what every stack looks like everywhere I've worked for the past 15 years, in both the startup world and the big corporate ones, though usually for different reasons. Juniper was so bad that I went to a startup, and my god, the crap some people churned out thinking they were clever, or just working around arbitrary constraints put in place by people who couldn't defend their position but would just dig in when challenged, is enough to make me want to throw up my hands and never develop again. These were security products, which is bad enough if / when things go wrong, but cars? I don't want blood on my hands, no thank you. We regulate the shit out of software that aerospace and medical devices use for a reason..
posted by cj_ at 2:20 PM on August 24, 2018 [11 favorites]


THE FATE OF THE FURIOUS has you covered

I remember watching this scene and thinking "yeah every car will be remotely hackable in 10 years or so", but it's actually less probable because of how every brand has their own bullshit network and hacking all of them to the degree in the film using one device like she's doing not only would take years to put together but also by the time you finish standardizing all the separate inputs from each dataset into one control point, they'd have either updated their firmware or moved onto a new one. So really, the wacky incompetence of car firmware management might actually save us from that scenario.

But man that was a fun scene to watch.
posted by numaner at 2:22 PM on August 24, 2018 [6 favorites]


20-years-younger-self: "Wow, why the heck would you need 100 MB of bike software??"
My city 20 years ago is safe enough to even consider riding a bike? I don't think so!
posted by The_Vegetables at 2:27 PM on August 24, 2018 [1 favorite]


Perhaps the worst method of information transfer the human race has come up with.

I put it to you that just about the maximally worst way for me to communicate with you would be to use relativistic planetbuster impacts to encode Morse at you.

Unless I just want to say "E".
posted by GCU Sweet and Full of Grace at 2:28 PM on August 24, 2018 [4 favorites]


I've been watching CLE videos this week, one of them on where to expect changes in civil and criminal liability with automated cars. The lecturer made the point that Volvo has accepted that they will be liable for failures of their automated systems, while Musk "in his habit of saying the stupidest things he can, in public"* stated that anytime Tesla's autopilot contributed to an accident still had to be considered operator error.

So Tesla might be a unique level of bad about this stuff.

*The lecturer was not without his own opinions.
posted by Navelgazer at 2:29 PM on August 24, 2018 [12 favorites]


I'm a little disappointed at folks saying "all tech is like this". All bad tech is like this. Good software companies that take the time to do things right do a lot better. With automated testing, and manual testing, and staging environments, and various devops practices. The kind of shit being described in these posts is serious amateur hour. I'd expect it for a brand new company of 3 fresh college grads. Or someone making something cheap and not critical like firmware for a cheap DVD player. But Tesla is supposed to be a serious tech company. They are selling $80,000 devices with the power to kill people and this is what they mean by software quality assurance?

Do you suppose they have as much shoddy engineering in their drive train production facility? Battery development? I mean we know their assembly line production is a disaster, to the point where they are literally hand-assembling cars under a makeshift tent. That's another story of crappy engineering heroics that belie a serious underlying problem.
posted by Nelson at 2:32 PM on August 24, 2018 [27 favorites]


Honestly, I don't trust anybody else who works in tech and manages not to be a complete Luddite.
Can I just say that I hate the term "Luddite"? It's linguistic reframing of workers fighting for fair treatment as anti-intellectualism and opposing progress, and it's become so ingrained that people don't realize that.
posted by NoxAeternum at 2:34 PM on August 24, 2018 [18 favorites]


The lecturer made the point that Volvo has accepted that they will be liable for failures of their automated systems, while Musk "in his habit of saying the stupidest things he can, in public"* stated that anytime Tesla's autopilot contributed to an accident still had to be considered operator error.

And he really, really meant that, too. The autopilot caused the accident where the car steered into the barrier, to the point where it was even speeding up right before it rammed into the concrete. But, it's the driver's fault, because they weren't paying the exact same amount of attention to the road as they would need to be if they didn't have any autopilot.
posted by foxfirefey at 2:51 PM on August 24, 2018 [3 favorites]


This really doesn't surprise me. Compared to Tesla, the other automakers are paragons of restraint and virtue when it comes to their approach to safety. Mostly because they've learned this the hard way, as it were. This isn't to say they're good; way too much design is being driven by marketing, rather than what's safe or can be made safe, but Tesla is an eyewateringly awful example.

Combine that with engineering approaches that are best described as "this is how an engineer thinks the driver perceives the world" and you get cases like the autopilot system ramming the vehicle into a barrier. Or deciding that a truck was sky. And selling the damn system as "autopilot" when it's more like "can drive in some circumstances, except when it can't, and gets confused easily" is just illustrative of how damned stupid Tesla.

You want pants-shitting terror? Memorial Drive, Cambridge, MA. Early 2016 (read: shortly after Autopilot was released to the public in its initial, unrestricted form). It kinda worked, but didn't really know what curves in the roadway were.

I'd love to see good, safe automation, because the human side of driving is where a lot of errors are. But, and this is critical, we need to understand what the humans can and can't do, and how and why they behave the way they do before this can be a reality.
posted by Making You Bored For Science at 3:07 PM on August 24, 2018 [10 favorites]


This was entertaining and wonderful (and I understood maybe a quarter to a half of it? at most?).

But the Bash thing, I can totally see.

If you have a system that has only bash, and you need to parse JSON but have no other reason to use a full featured scripting language, and you can actually write a legit JSON parser in bash, do it! Use what you've got as effectively as possible rather than adding more things!

Even better, if you have a binary which you wrote in C++ or Go or some shit, which parses JSON beautifully, compile that and ship it.
posted by edheil at 3:18 PM on August 24, 2018 [3 favorites]


Man my first reaction was "is that YOSPOS? Oh my god" but it probably shouldn't have been!
posted by Freeze Peach at 3:28 PM on August 24, 2018 [5 favorites]


Unix command line utilities have been around for decades and are pretty damn venerable and time-tested.
posted by Apocryphon at 3:28 PM on August 24, 2018 [1 favorite]


you can actually write a legit JSON parser in bash

I mean it's a Turing-complete language, so yes you *can*. Whether anyone would, I question. You could take the time to properly handle double-escaped backslashes and quotes, oddly-placed newlines between tokens, unicode handling, and everything else necessary to adhere to the letter of the json spec. Or you could take a couple orders of magnitude less time and effort and come up with something that works with whatever input it's expected to actually be given. The odds of the latter approach eventually causing bugs are... medium, I guess.
posted by sfenders at 3:43 PM on August 24, 2018 [5 favorites]


Also if you really just want JSON parsing glued into a bash script you're much better off using jq. It's a small scripting language, sort of like sed-for-JSON, and a pretty useful tool. Anyone claiming that somehow it's less secure than hacking up some JSON parsing in a shell is, well, they shouldn't be in charge of technology decisions.
posted by Nelson at 3:54 PM on August 24, 2018 [6 favorites]


Metafilter: I powered my old car with farts too.


See, you think you’re joking, but yesterday I filed a proposal with the Minnesota Public Utilities Commission to start selling biogas to customers.
posted by nickmark at 3:56 PM on August 24, 2018 [7 favorites]


I put it to you that just about the maximally worst way for me to communicate with you would be to use relativistic planetbuster impacts to encode Morse at you.
I disagree. While bandwidth would be pitifully low, error correction wouldn't be needed out to, say, the 100 million km mark.
You could also offset the cost by selling video of it on ppv.
posted by thatwhichfalls at 4:00 PM on August 24, 2018 [2 favorites]


You could also offset the cost by selling video of it on ppv.

Hah! Who would be dumb enough to preorder content that plainly can't ever be delivered? To get any value out of this you'd have to be some kind of star citizen.
posted by snuffleupagus at 4:06 PM on August 24, 2018 [10 favorites]


you can actually write a legit JSON parser in bash

You can also, for example, write a large-scale web app in PHP, though if you do so, you'll be forever putting out fires and mitigating vulnerabilities from the language and its libraries' numerous edge cases and flaws. Or you could write one in something more robust, and have an order of magnitude fewer things to worry about.
posted by acb at 4:08 PM on August 24, 2018 [1 favorite]


Who would be dumb enough to preorder content that plainly can't ever be delivered? .

There is a significant difference between the screencaps-on-twitter and the relativistic bombardment methods of information transfer - in the latter those present at the transmission site will be longer be available to complain once the message is delivered. Those on Twitter, however, must soldier on, bearing their scars.
posted by thatwhichfalls at 4:26 PM on August 24, 2018 [1 favorite]


Unix command line utilities have been around for decades and are pretty damn venerable and time-tested.

You would think so, but many of them, in their GNU incarnations, are still being actively developed, so there's always a possibility of new bugs being introduced. And even venerable software may have undiscovered bugs. I personally ran into an apparent threading bug in GNU sort that caused it to deadlock when using the --parallel option that was introduced around 2010, and was made the default at some point. And in 2014 (6 years into Tesla's history) they discovered a root exploit in bash that had gone undiscovered since 1989.
posted by mubba at 4:46 PM on August 24, 2018 [8 favorites]


This is why I am scared to fly after being a wonderful flier for decades. I got over the fact that this huge heavy metal tube could get off the ground. But, the more I hear how automated planes are and how they fly themselves, I break out in sweats. Some coder wrote thousands (?) hundreds of thousands (?) lines of code to react to every possible input all the sensors are getting and I should expect it to work without a bug every single time? Nope. I have no idea about all the technical parts, but I just think about some coder who had a hangover or had a fight with their SO and now they are coding the autopilot? Too much room for error for me. Sure Boeing or Airbus are going to take responsibility or liability, but I be dead and do not give a hoot.

I have my mid-aughts F-150 with 158,000 miles on it with minimal computer reliance and I am happy that my Sirius radio works most of the time. A Tesla? A car made by a company led by a madman? No thanks.
posted by AugustWest at 5:26 PM on August 24, 2018 [3 favorites]


i would bet that the quality of code, and software development process, are as bad or even worse in any other car company.

One need not wager since we have been made privy to the internal workings of several cars now, and in both cases the entire design, if a slapped together house of cards can be said to have a design, is obviously unfit for purpose. Recall that Toyota's ECU mashed everything together in one big long function such that say, the accelerator position sensor would fail to be polled for long periods of time when certain other things happened.

I've been led to believe that much of the compute has been distributed so that such interactions are less likely to impact multiple subsystems at once, but given my status as an outsider I can't say whether that is actually true or not.
posted by wierdo at 5:27 PM on August 24, 2018 [1 favorite]


Also, there exist perfectly good and robust web apps written in PHP, by the way. As with any language, due planning and care being applied prior to writing actual code leads to reasonably reliable software, while slapdash tinkering leads to buggy, unmaintainable shit. That's not to say that other languages don't give you a head start, only that the problem isn't the language or any of the other stuff we have religious wars about, the problem is poor practices.
posted by wierdo at 5:31 PM on August 24, 2018 [4 favorites]


Can I just say that I hate the term "Luddite"? It's linguistic reframing of workers fighting for fair treatment as anti-intellectualism and opposing progress, and it's become so ingrained that people don't realize that.

I assure you I meant it in the original sense. I'm typing on a computer, after all, so I'm demonstrably not anti-technology. But it seems like the only things the tech industry builds that actually function properly are the things that directly enable surveillance capitalism. Smashing them seems like an appropriate response.
posted by tobascodagama at 5:34 PM on August 24, 2018 [12 favorites]


Recall that Toyota's ECU mashed everything together in one big long function such that say, the accelerator position sensor would fail to be polled for long periods of time when certain other things happened.

Wasn't the Toyota code the one with 36,000 global variables or something incomprehensibly perverse like that? How does software like that even get written? Is there a policy banning code modularisation because some ignoramus in a position of authority once looked at some code, disliked the appearance of large numbers of functions, and decreed that separate functions were banned from then on or something?
posted by acb at 5:39 PM on August 24, 2018 [2 favorites]


Can I just say that I hate the term "Luddite"? It's linguistic reframing of workers fighting for fair treatment as anti-intellectualism and opposing progress, and it's become so ingrained that people don't realize that.

Also, the Phillistines were, by all accounts, a far more sophisticated culture than the Israelites of the time. And yet, the term persists.
posted by acb at 5:40 PM on August 24, 2018 [5 favorites]


It would be stupid but very possible to produce a reliable and testable JSON parser in bash by writing a parser generator that outputs bash. The real problem is that someone decided that running bash scripts is safer than using a language more suited to the problem domain.
posted by rdr at 5:46 PM on August 24, 2018 [4 favorites]


...and then putting it in a car.
posted by Artw at 5:51 PM on August 24, 2018 [4 favorites]


But, the more I hear how automated planes are and how they fly themselves, I break out in sweats.

You should hear about the way door latches and rudders used to be designed on planes, then.
posted by ambrosen at 6:08 PM on August 24, 2018 [3 favorites]


¯\_(ツ)_/¯

Depending on what exactly I was trying to get from a slug of JSON and what I intended to do with it, I could easily see myself writing something someone motivated to make me look foolish could truthfully, if somewhat misleadingly, call a JSON parser.

I wrote a couple of lines of Perl to replace a few characters in a ~100ish character long configuration string for a very common hardware/software package. It stayed in production for over a decade. Modifying it required careful usage of a text editor with a ruler function. Certainly one could (correctly!) point and laugh and make dour pronouncements about my clients' technical choices, but it made far more sense than writing a real parser and generator given the actual business requirements and the expected frequency of changes. Turned out to be a good decision given that I only had to spend maybe 4 or 5 hours in total on it in over more than a decade of use.

Point is, I'm loathe to criticize without being privy to more details than can be gleaned from a tweetstorm, especially when there are significant financial interests at stake and the discussion is about systems which are not safety critical.
posted by wierdo at 6:08 PM on August 24, 2018 [7 favorites]


Also, there exist perfectly good and robust web apps written in PHP, by the way. As with any language, due planning and care being applied prior to writing actual code leads to reasonably reliable software, while slapdash tinkering leads to buggy, unmaintainable shit.

Undoubtedly. Though some languages and systems make writing robust, secure code harder than others. PHP's main advantage was that one could do logic in a script that fit in a web page that could be uploaded to a web host without fretting about provisioning, deployment or similar. Then people started writing large-scale apps, with MVC-style architectures, in it, which it was less suited to. On top of that, it tries too hard to be easy for the proverbial beginner writing a guestbook for their homepage to use and helps you along by trying to guess what you want; for example, not only is it dynamically typed, but it tries to never complain about type mismatches, instead guessing what the user meant and working around it; i.e., everything has a truth value, and strings' values depend on whether they're truthy things like "yes" or "T" or not, and adding an integer to a string will silently convert the former type to the latter and such. This leads to a lot of types of unexpected behaviours, some of which can have consequences, such as MD5 hashes starting with "0e" being parsed as floating-point values and considered to be equal.

One can become a top PHP security ninja, keeping on top of all these edge cases, and understanding the internal model well enough to anticipate how things could fail. But that takes away mental bandwidth from actually solving the problem one's writing the code for in the first place. It's the Broken Window Fallacy: just as repairing broken windows does stimulate economic activity, but at the expense of more productive uses of resources, getting good at putting out fires in a system that's badly designed takes resources away from getting results.
posted by acb at 6:12 PM on August 24, 2018 [8 favorites]


systems which are not safety critical.

You have far more faith in their encapsulation than I do.
posted by PMdixon at 6:16 PM on August 24, 2018 [2 favorites]


I'm a little disappointed at folks saying "all tech is like this". All bad tech is like this. Good software companies that take the time to do things right do a lot better.

All tech is not like that, but when people say that what they really mean is that it is shockingly common for companies that you would hope or expect to have their shit together to have the same problems.
posted by atoxyl at 6:33 PM on August 24, 2018 [13 favorites]


As someone who's had to work with PHP I kind of hate it but I feel like you're a little out of date about what's bad about it. I mean, it's stupid that '===' has to exist at all but it does and you don't have to use anything else. There are even sort of declared types - but only for function parameters and return values - though I wouldn't say that was the main thing holding it back in the first place except in combination with the insanely cast-happy comparisons.

The main thing now is it's just a weird hybrid of a shitty version of Java (without the consistency) and a shitty version of [your favorite scripting language] (without the elegance and expressiveness). And it has a bunch of legacy functions that don't work the way you would expect them to or even the same way as each other.

(I don't recommend it, but I don't think it's that hard to write it in a way that won't spontaneously combust if you have to, I guess.)
posted by atoxyl at 6:48 PM on August 24, 2018 [1 favorite]


36,000 global variables or something incomprehensibly perverse like that

Global variables are often bad design when developing for web, desktop, or mobile. However, in embedded and realtime systems, they are actually a best practice. This is due to the different requirements of these systems.

Global variables never go out of scope and they are never garbage collected. You don't need to dynamically allocate memory for them and therefore allocation can never fail, even in an embedded system with kilobytes of RAM. If you don't need dynamic memory allocation, you don't need an MMU, and you can go for a cheaper, more power efficient chip. Without an MMU, you never have page faults or translation lookaside buffer cache misses, so your code has very predictable timing and you can guarantee a critical routine has enough time to do its job.

Of course there are downsides, most notably that you only have one of any given global variable. That might not work in a web app serving thousands of viewers concurrently, or a desktop app with multiple documents open. But presumably the Toyota code was only expected to control a single car.
posted by foobaz at 6:56 PM on August 24, 2018 [26 favorites]


You can write good software in any language. But you have to be willing to bang on it, test the shit out of it, mutation testing, property testing, maybe even formally verify your program. If you're not willing to do that get the fuck out of my vehicle.
posted by dilaudid at 6:57 PM on August 24, 2018 [3 favorites]


Global variables never go out of scope and they are never garbage collected. You don't need to dynamically allocate memory for them and therefore allocation can never fail, even in an embedded system with kilobytes of RAM. If you don't need dynamic memory allocation, you don't need an MMU, and you can go for a cheaper, more power efficient chip.

I suppose, very tentatively, that this could perhaps be a reason to copy-and-paste all of your entire program into one enormous function, because you'll never need to worry about having enough room for a new stack frame.

This still seems completely depraved.
posted by vogon_poet at 7:04 PM on August 24, 2018 [2 favorites]


If you don't need dynamic memory allocation, you don't need an MMU.

Hey now. Dynamic memory allocation doesn't require an MMU; just a chunk of memory and some code to allocate/free blocks from it. An MMU gives you virtual addressing, which is pretty much an orthogonal axis of complexity.

The "make every variable global" thing, when I've seen it, always seemed to be pretty much an anti-pattern that went together with "cram everything into one huge function" by embedded engineers who were irrationally terrified of consuming stack.

In re the Twitter thread: I was rather fond of "the certificates on the cars expired, so we hacked our OpenSSL to ignore expiry."
posted by We had a deal, Kyle at 7:28 PM on August 24, 2018 [10 favorites]


Incidentally: an article on the Toyota ECM code: Toyota's killer firmware: Bad design and its consequences.
posted by We had a deal, Kyle at 7:37 PM on August 24, 2018 [3 favorites]


It would be stupid but very possible to produce a reliable and testable JSON parser in bash by writing a parser generator that outputs bash.

Then you could use it to make a C compiler in bash, and do your JSON parsing in C.
posted by sfenders at 8:17 PM on August 24, 2018 [6 favorites]


You can make all the global variables you want, just put them in things called singletons. Or modules. Wait, then those aren't really global variables anymore...
posted by RobotVoodooPower at 8:20 PM on August 24, 2018 [1 favorite]


> Global variables are often bad design when developing for web, desktop, or mobile. However, in embedded and realtime systems, they are actually a best practice.
I think you're mixing up static variables and global variables. You're completely right on why embedded systems use static variables, but proper scoping (read: not global) is still imperative.
posted by ReadEvalPost at 8:20 PM on August 24, 2018 [4 favorites]


Unix command line utilities have been around for decades and are pretty damn venerable and time-tested.

hahahaha no [PDF]
posted by Harvey Kilobit at 8:24 PM on August 24, 2018 [1 favorite]


Unix command line utilities have been around for decades and are pretty damn venerable and time-tested.

hahahaha no [PDF]


In fairness that paper was published in 1989, and one presumes that there has been at least some modest improvement in the overall reliability of the core command line utilities in the intervening 29 years.

On the other hand it includes this line, which has remained depressingly timeless: "The ability to overflow an input buffer is also a potential security hole, as shown by the recent Internet worm."
posted by jedicus at 8:49 PM on August 24, 2018 [9 favorites]


1995 Re-examination

A bit of jargon.txt style terminological trivia: the 2018 article "Robustness Testing of Autonomy Software" (CMU) cites these papers on the GNU/Unix utilities as the source of the term "fuzz testing."
posted by snuffleupagus at 9:17 PM on August 24, 2018 [2 favorites]


You can make all the global variables you want, just put them in things called singletons.

This is a good joke, but then you kept writing so I can't be one-hundred percent certain it's a joke
posted by atoxyl at 9:32 PM on August 24, 2018 [4 favorites]


I mean I know nothing about Tesla but this jibes with every startup I worked at. Janky tech stack, code patched together with duct tape and wire, environment burning out engineers left and right, possibly insane founder.
posted by Ghostride The Whip at 12:59 AM on August 25, 2018 [5 favorites]


Now go ask your bank, you know, where you keep all your money, about their tech stack.
I'll be waiting here with a cup of hot chocolate and a sympathetic smile for when you get back.
posted by signal at 4:58 AM on August 25, 2018 [10 favorites]


As car manufacturers race to install more computers in cars, I wonder if they realize the IT infrastructure maintenance and upkeep that they're committing to when they do so. If the servers that power my Blu-Ray player's app store (whether the player actually NEEDS an app store is a conversation for another day) stop working, it's a slight inconvenience. But a car can be on the road for decades—what happens when it stops getting updates or can no longer phone home?

Some manufacturers are thinking about how to add technology to their cars in safe ways, but as Jeep and Tesla have demonstrated, others are just jamming things in to make an iPad with four wheels and an engine.

Beginning to think that the designers of the Galactica were right: no networked anything so the Cylons can't take it over.
posted by fifteen schnitzengruben is my limit at 5:03 AM on August 25, 2018 [13 favorites]


But a car can be on the road for decades—what happens when it stops getting updates or can no longer phone home?

🔥🔥🔥🔥¯\_(ツ)_/¯🔥🔥🔥🔥
posted by PMdixon at 5:07 AM on August 25, 2018 [12 favorites]


what happens when it stops getting updates or can no longer phone home?

You miss out on the fun and excitement of wondering whether your car's ride height and handling characteristics are going to be suddenly different due to some distant engineers changing their minds about something? You stop wondering when it will next get stuck in a reboot loop for an hour? You don't have to regularly make time to schedule software updates? You miss out on exciting new features like that thing where the car can drive itself across a parking lot at low speed if you're lucky? The touchscreen display can no longer download new ads to show you? The data from tracking everywhere your car ever goes no longer contributes to Tesla's enormous database? It's a frightening prospect to be sure.
posted by sfenders at 5:58 AM on August 25, 2018 [5 favorites]


I think they’re betting on it having caught fire by then.
posted by Artw at 6:05 AM on August 25, 2018 [2 favorites]


Hm let me see if I can convince my brother to buy a Tesla now. Less than 10 years ago he was ranting about not finding new cars with manual roll up windows like his old one. I mean , what kind of asshole can’t roll up his own fucking window? *

* his words not mine. My dad recently : “eh social media .... shit. What kind of asshole cares whether some person online likes what they say?” I work in tech but I have this gene too ... I try to fight it.
posted by freecellwizard at 6:09 AM on August 25, 2018 [2 favorites]


what happens when it stops getting updates or can no longer phone home?
They probably brick it and make you buy, sorry, license a new one.
posted by Hal Mumkin at 6:51 AM on August 25, 2018 [6 favorites]


If you saw all the janky code that runs your life, you'd throw away all your tech and never leave your house.

I know for a fact--because I've had to work on it myself--that some pretty essential code our main student-facing system depends on is a group of hacked-together and uncommented bash scripts written by my coworker that generate old-school flat text data files and push them into the system every three hours or so.

These scripts, by the way, were written in one single afternoon after he came back from a couple of hours and several pints at the pub and also after my predecessor had apparently spent weeks trying and failing to get the work done in Perl.

Now we're looking at reworking the whole mess to work with PostgreSQL some time before December, and then some nebulous time after that to work in a completely different way that no one has actually managed to do requirements or specs for yet. Fun.
posted by Mr. Bad Example at 7:07 AM on August 25, 2018 [7 favorites]


Your car is not going to explode just because it isn't connected to the Internet. As the former employee says, "tesla did sell cars explicitly with the sim pulled and no network ever." Considering how wealthy many of their customers are, it's no surprise that some would care about privacy and security.
posted by sfenders at 7:08 AM on August 25, 2018 [2 favorites]


> graventy:
"Until a few weeks ago I had a vague amount of respect for Musk and a vague desire for a Tesla, in a "I want every cool fancy tech thing" kind of way. Then I started following the something awful thread, which is an incredible read. I highly recommend it."

BlueGoon sighted!
posted by Samizdata at 7:12 AM on August 25, 2018 [2 favorites]


See, you think you’re joking, but yesterday I filed a proposal with the Minnesota Public Utilities Commission to start selling biogas to customers.

I accidentally read this as bigos, a Polish stew made out of meat, sauerkraut, and fresh cabbage.

Which obviously has the same relevant characteristics.
posted by mikelieman at 7:47 AM on August 25, 2018 [4 favorites]


Now go ask your bank, you know, where you keep all your money, about their tech stack.
I'll be waiting here with a cup of hot chocolate and a sympathetic smile for when you get back.


FWIW, I did a nickle with a national-scale bank's "Government Specialty Banking" in "Application Development and Support".... [ rest of anecdote trimmed, because really, the next paragraph renders the next 500 words pointless, but long story short we scanned and indexed 25,000 returns PER DAY w/ 100% accuracy ].

So fastforward to a couple of years ago, as a NYS election examiner, I learned that the ESS gear runs Linux ( good ) and that their app is in Java ( not so good... ).

(I still believe with proper controls, the "capture scantron after scanning and confirming over/under votes" is solid.)
posted by mikelieman at 8:02 AM on August 25, 2018 [1 favorite]


Feel like it's only fair to include a follow up post from the person that made the thread after it exploded on Twitter:

Regardless of whether my NDA had expired, if I’d written a slightly indiscreet, personally identifying post that later went viral about my former employer, a company run by a notoriously thin-skinned and vengeful billionaire, I’d probably try to hedge a little too.
posted by chappell, ambrose at 8:09 AM on August 25, 2018 [5 favorites]


In my experience banks have really stable software because they are not trying to update it to do anything cool or hiring programmers who want to do cool things.
posted by tofu_crouton at 8:15 AM on August 25, 2018 [3 favorites]


I like to believe my bank runs COBOL on a System/360, maintained by a crack team of 80-year-olds with ponytails, powered by Tang, who are currently in year 26 of their 2nd edition D&D campaign as they wait for something to break.
posted by bigbigdog at 8:46 AM on August 25, 2018 [34 favorites]


Apart from the 80-year-olds bit, that's probably pretty close.
posted by farlukar at 10:55 AM on August 25, 2018 [3 favorites]


vogon_poet: "i would bet that the quality of code, and software development process, are as bad or even worse in any other car company."

Tesla is more bleeding edge with this stuff and they are spreading the R&D over way fewer units So i'd bet they are worse than any other manufacturer you've actually heard of on average.

zachlipton: "Much as I enjoy the lol Tesla fun, this is every damn car now. It's terrifying."

It's a spectrum and Tesla is way over on the bad end.

tobascodagama: "20-years-younger-self: "Wow, why the heck would you need 100 MB of bike software??""

FTFY.
posted by Mitheral at 12:35 PM on August 25, 2018 [3 favorites]


Apple hired scores of ex-Tesla employees this year, and not just for its car project

I guess a hire is a hire. Lets hope they are better managed there.
posted by Artw at 12:44 PM on August 25, 2018 [1 favorite]


Which obviously has the same relevant characteristics.

Close; it's a precursor.
posted by acb at 4:45 PM on August 25, 2018


In my experience banks have really stable software because they are not trying to update it to do anything cool or hiring programmers who want to do cool things.

OTOH, it's a pretty safe bet that, if you go on any Friday afternoon to the pub nearest a bank IT department and ask how the migration is going, you will see pained expressions. It's sort of the IT equivalent of going to an English country pub and asking if The Major is in.
posted by acb at 5:02 PM on August 25, 2018 [8 favorites]


but I guarantee you the technical issues i posted are long gone now [...] i know for a fact all the crap i worked on is probably gone by now

When you want to be decisively wishy-washy.

I suppose a probability could be a fact, but how would you know it without also knowing the answer 100%? I've thought about this sentence for too long now; moving on.
posted by ctmf at 8:19 PM on August 25, 2018 [4 favorites]


Now go ask your bank, you know, where you keep all your money, about their tech stack.

Funny enough, I used to do occasional work for a bank. Their tech stack was a 386 running SCO with a few serial terminals attached and a modem to dial up the Fed to receive the ACH batches. They had some PCs to generate/print loan documents and whatnot, but that was it until they were purchased by a regional bank in the early 2000s. Said regional bank runs all their critical financial stuff on i5/OS and uses PCs running IBM's fine 5250 emulator as terminals. Not all that different from the old way, really, just easier for intruders thanks to everything running over IP.

That said, the AS/400 and its descendants are far easier to secure than Windows or even Linux has ever been. The problem is that most people who work on such systems don't put much thought into security since they feel like the obscurity alone is enough of a deterrent. Ironically, they've been proven correct over and over again given that attackers almost exclusively target ATMs and POS terminals rather than the back end systems.
posted by wierdo at 10:54 PM on August 26, 2018 [4 favorites]


"Huh. So, if any given car company goes out of business, will their entire fleet just stop working? I'd honestly never realized there was so much reliance on communication to and from centralized infrastructure."

Holy hell, I mean, we bailed out car manufacturers a while back for no other reason than "it'd be hard work and stressful to let these horrible companies go out of business like they deserve to." Any car company out there that still has any doubt over the nut-pinch they have the government in can simply say millions of peoples' vehicles will cease functioning if the company rightfully goes out of business, so pay up!
posted by GoblinHoney at 1:36 PM on August 27, 2018 [1 favorite]




« Older sir do you have any idea how unidentifiably you...   |   “I would never run, never. I don’t know where they... Newer »


This thread has been archived and is closed to new comments