UNIX is dead
February 4, 2019 8:25 PM   Subscribe

The Tragedy of systemd A thoughtful analysis of the current state of play from Benno Rice, a longstanding FreeBSD developer, at linux.conf.au 2019 (Christchurch, New Zealand).

Referenced early in Rice's presentation: Aurynn Shaw, Contempt Culture. Folks who simply can't bear to sit through three quarters of an hour of YouTube presentation before offering an opinion on systemd should at least read that piece.
posted by flabdablet (99 comments total) 43 users marked this as a favorite
 
I just finished watching this a couple of hours ago. I found it a refreshing contrast to the usual discussion on the subject.
posted by wierdo at 8:44 PM on February 4, 2019 [1 favorite]


The Contempt Culture link is outstanding, but this one line was a head scratcher for me:

Other self-taught narratives, such as starting with Wordpress-based design backgrounds and moving from more simple themes to more complex themes where PHP knowledge is required, to plugin development is a completely valid narrative, but a path that is predominately for women.

Is this actually true? Is Wordpress development dominated by women? Are PHP users more likely to be women? This came from nowhere, and I would be fascinated to see data on this.
posted by fremen at 9:08 PM on February 4, 2019 [1 favorite]


Folks who simply can't bear to sit through three quarters of an hour of YouTube presentation before offering an opinion on systemd are the overwhelming majority of all people who have ever lived across all history.

That article is worth a read, however. It has implications that extend far beyond programming culture. For those that have ears to hear.
posted by hippybear at 9:14 PM on February 4, 2019 [13 favorites]


> Is this actually true?

Graphic design is 60% female, web development is 62% male. Moving from Wordpress themes (mostly CSS) to Wordpress plugins (mostly PHP) would carry some gender assumptions.

Programming languages that do not have an inroad from female-heavy fields would carry different assumptions.
posted by Phssthpok at 9:17 PM on February 4, 2019 [1 favorite]


The standout points from the presentation for me:

1. systemd as an example of a third conceptual layer, the system layer, that fits between kernel and userspace. The way this was presented kind of hints that the need for such a layer is somewhat new but it seems to me that it's actually quite old, and is to some extent the very thing that the original design of Unix was a reaction against.

Many of Unix's contemporaries and predecessors did have a bunch of more-or-less consistent APIs for stuff both inside and outside the kernel, and did have stuff like logging and message passing and deadlock prevention and device management that the original design of Unix just threw out on the grounds of being all too hard.

The original use case for Unix was as a lightweight OS for machines that were essentially being used as personal computers by Dennis Ritchie and Ken Thompson. It dealt with quite a lot of the issues that begat complexity in contemporary systems by simply ignoring them and forcing application programs to work around them instead.

I see systemd as an attempt to bring the same kind of coherency to actually addressing those issues as some of the larger non-Unix systems already had before worse is better ate all their lunches. In 2019 we rely on software systems way more heavily than could ever have been dreamed of in the 1970s and making them work right is more important than ever; worse is no longer demonstrably better.

But there's a genuine loss involved: that of the Unix ideal of simplicity, and I think that's where a lot of the pushback against systemd originates. There's an instinctive sense that systemd simply tries to do too much; that it's just too ambitious. But in an era where even a fairly small pre-systemd Debian server is forced to launch hundreds of processes on startup in order to get done what needs to be done, it seems to me that the Unix ideal of simplicity is long dead and we need to let it rest in peace.

Modern software is just complicated. It just is. There's no getting around it. It's more complicated because we're using it in more places in more ways on more devices to do more stuff. We actually need something more like Multics and less like Unix to hold it all together; whether systemd in and on top of a Linux kernel is a good way to do that ought properly to be a question to be settled by research and experience, not furious handwaving arguments from ideology.

Other projects well worth keeping an eye on include Redox, a promising attempt to build something rather like a Unix plus a system layer from scratch, but based on a microkernel and implemented in Rust.

2. Change.
People have a complicated relationship with change. I like to say that nerds, especially, have a really complicated relationship with change. We love it to pieces! Change is awesome! ...when we're the ones doing it.

As soon as change is something that's coming from outside of us, it becomes untrustworthy and it threatens what we think of as the familiar.
I thought that was well put.
posted by flabdablet at 9:40 PM on February 4, 2019 [52 favorites]


I didn't have time for the video, so I read the article. Then I had to watch the video to figure out what it had in common with the article!
posted by M-x shell at 9:41 PM on February 4, 2019 [1 favorite]


Great to see Aurynn's work get more visibility.

Contempt culture is rife in tech, but by no means isolated to it. It's one of the shitty-but-common ways group membership is policed. What sucks is that we pretend it's actually reasonable or justifiable.
posted by allium cepa at 10:19 PM on February 4, 2019 [9 favorites]


The video is fantastic and worth watching even if you're not invested in the specific software mentioned because it's the epitome of what I hope and believe is a growing motif in tech talks that amount to "this isn't high school, acting out mean-girls isn't how you win".
posted by fatbird at 10:22 PM on February 4, 2019 [7 favorites]


To paraphrase someone who would likely have been an exceptional software architect if alive today, Benno, a BSD guy no less, explained the common sense of the subject, in terms so plain and firm as to command their assent.

In a less dark timeline, I would have hoped for systemd to originate from the Debian team. What would have been deployed would be more elegantly and considerately crafted, and rolled out/shaken down (as much as possible) in a more incremental manner.

Having observed the concurrent evolution of Yum and Apt, for example, I much prefer Apt. RH and Canonical will always be more biased toward 'kinda works, good enough, ship it now' than Debian's 'let's release the part that is stable, then build it out' model.

Two more things... Am I the only one concerned about Linux (and RHEL specifically) becoming a crypto-proprietary monoculture? And if we must let go of simplicity as Benno noted, may we at least retain a bias toward elegance?
posted by zaixfeep at 10:48 PM on February 4, 2019 [8 favorites]


For a cheap yet balanced laugh, summary responses to Poettering and co.(15s) and Systemd critics (20s).
posted by zaixfeep at 11:26 PM on February 4, 2019


Am I the only one concerned about Linux (and RHEL specifically) becoming a crypto-proprietary monoculture?

No you're not. That concern is exactly the reason why I feel less and less comfortable with operating systems the further removed they are from Debian. It's why I run Debian itself on all my desktop machines, Armbian on all my little utility boxes and OSMC on my media servers.

Speaking of those little utility boxes, I recently found quite a nice feature that's easier to set up in systemd than in any alternative to it that I'm aware of: mount-on-demand filesystems. My little Odroid XU4 server has 20TB worth of drives attached to it in a USB3-connected JBOD enclosure, over which I layer a bunch of RAID5 mdadm arrays, out of which I build a handful of LVM2 volumes, on the largest of which is a huge ext4 filesystem that I only use to hold a BorgBackup repository.

It used to be that the occasional uncontrolled shutdown would cause a very lengthy attempt to check that filesystem on next startup, delaying the launch of useful stuff like the ssh daemon that's my only way into that box for up to half an hour. But by sticking
x-systemd.automount,x-systemd.device-timeout=10,x-systemd.idle-timeout=60
into the mount options for that filesystem in /etc/fstab, systemd handles leaving it completely unmounted for most of the time, which means that a crash-stop can't affect it unless it happens in the middle of an actual backup session.

Of course there are other ways to achieve the same thing and have been for quite some while, but I was impressed by how easy it was to make systemd do it. And the more mature systemd gets, and the more bugs Lennart and Kay shake out of it, the more people I can see finding that it does actually offer them the kind of value that makes overcoming the inherent (and completely sound!) distrust of externally imposed change worthwhile.
posted by flabdablet at 11:58 PM on February 4, 2019 [8 favorites]


Am I the only one concerned about Linux (and RHEL specifically) becoming a crypto-proprietary monoculture?

It's not Linux you should be worrying about, it's AWS.
posted by Leon at 1:54 AM on February 5, 2019 [12 favorites]


Worrying about more than one thing at a time is possible, and sometimes reasonable.
posted by ardgedee at 3:53 AM on February 5, 2019 [7 favorites]


How much crypto-currency do you need to mine off of your AWS setup to pay for the maintenance and storage costs of said cloud? And, do they accept said crypto-currency as payment?
posted by Nanukthedog at 4:43 AM on February 5, 2019


systemd is just bad software. Hardly anyone is saying that something like systemd wasn’t necessary, but it should have been modularized instead of being monolithic, it should have been MUCH better documented, and it could have been implemented in a tiny fraction of the number of lines of code. Something like 75% of systemd only makes sense when running on a desktop PC with a GUI (i.e. Gnome 3). Why do I need all that baggage on my headless server?

Modern software is just complicated. It just is. There's no getting around it.

The solution to complexity is not to add a whole other layer on top of the complexity. It doesn’t solve anything. It just moves the complexity around, it sweeps it under the rug and makes it someone else’s problem.
posted by 1970s Antihero at 5:27 AM on February 5, 2019 [17 favorites]


"stuff like logging [...] that the original design of Unix just threw out on the grounds of being all too hard."

Yes, well I wasn't around in 1972 or whenever that was, but linux has had system-wide logging since forever, in the form of syslog and its various competitors and more-sophisticated replacements. Perhaps the systemd version suits you best, but to me it doesn't seem to have any real advantages other than that it comes with systemd and is therefore widely used. It's as if someone has come up with a really effective cross-branding promotional scheme to get the systemd version of syslog, a DCHP server, a job scheduler, an automount system, a DNS resolver, and the myriad other incarnations of systemd into our systems.

The video includes a link to its author's original justification for the idea of systemd, and it strikes me as largely misguided. On servers and desktop PCs, nobody cares about system start-up time so much any more. We all have SSDs now. On embedded systems, you typically know exactly which services you're going to need and it's not so complex. There was really no need to mess things up so thoroughly just for that. There was certainly some demand for something better than sysv init to deal with "service management" in a more systematized way, but I suppose just solving that problem just wasn't ambitious enough. And then of course came the inexorable expansion of the systemd project into anything vaguely "system" related.

Anyway, I don't dislike systemd enough to bother trying to replace it. It's just a mystery to me how other alternatives didn't win out. Saying that some of its detractors are just being mean doesn't shed any light on that.
posted by sfenders at 6:13 AM on February 5, 2019 [9 favorites]


It doesn’t solve anything. It just moves the complexity around, it sweeps it under the rug and makes it someone else’s problem.

The older I get, the lazier I become. I actually don't mind quite a lot of the fiddly sysadmin stuff I've spent hours and hours doing becoming Lennart's problem and having it dealt with under the rug.

When systemd was new and seemed to break something else every other day the cost/benefit calculations were different, but just like avahi and just like pulseaudio it mostly Just Works these days.

I don't dislike systemd enough to bother trying to replace it.

That's been my longstanding attitude toward it as well, pretty much since Debian first made it the default init system.

It's caused me a few headaches, such as filesystems that mysteriously remain busy and refuse to let themselves be fscked even though they appear to have been fully unmounted and neither lsof nor fuser reveals anything at all using them*. I also still don't really trust its ability to shut a system down in a clean state and have taken to using sync; sync; halt -p as a superstitious tic.

But by and large it seems to work, and I am slowly coming to appreciate the advantages of having one reasonably concise config syntax covering multiple different kinds of service.

*For anybody else afflicted with this particular misery: it happens to filesystems mounted via /etc/fstab, /proc/1/mounts or /proc/1/mountinfo is the place to see the otherwise invisible in-use reference, and systemd-umount is the tool that gets rid of it.
posted by flabdablet at 6:46 AM on February 5, 2019 [6 favorites]


systemd is just bad software. Hardly anyone is saying that something like systemd wasn’t necessary, but it should have been modularized instead of being monolithic, it should have been MUCH better documented, and it could have been implemented in a tiny fraction of the number of lines of code. Something like 75% of systemd only makes sense when running on a desktop PC with a GUI (i.e. Gnome 3). Why do I need all that baggage on my headless server?

Having all the various daemons in one repository does not make systemd monolithic. This particular point was addressed specifically in the video. I don't know about your headless servers, but mine run very few systemd modules. The others could run if they were configured to do so, but they aren't, so they don't.

And it won out because it was there and in many, many ways beats the pants off sysvinit. Nonetheless, there remain many distributions that don't use it, so maybe use one of those if it bothers you so much. Or recruit some like minded folks and roll your own distro.

Of course there are many things I wish systemd did differently, but that's life. I wished Slackware did a few things differently back in 1994, but that didn't stop me from using it since the alternatives available to me at the time weren't any better. Eventually, it became familiar, just like systemd has (to me).
posted by wierdo at 6:47 AM on February 5, 2019 [8 favorites]


I see we've moved on to the part of Metafilter where no one watches the video, or reads the article, and just comments on the keyword systemd.
posted by Nelson at 7:09 AM on February 5, 2019 [32 favorites]


It's just a mystery to me how other alternatives didn't win out.

Seems to me that systemd's ambitious scope has proved, in practice, to be more of an advantage than a disadvantage when it comes to driving adoption. As Benno Rice notes, it's fairly easy to dislike Lennart's vision but the sheer effrontery of it really does merit a certain degree of respect.

The systemd devs all work for Red Hat, the GNOME Foundation's largest corporate supporter. GNOME also remained the preferred desktop environment for a hell of a lot of other distros even after the GNOME devs threw the switch to vaudeville with GNOME 3, and when GNOME made systemd a path-of-least-pain dependency for its login manager it tipped a lot of them over the edge.
posted by flabdablet at 7:09 AM on February 5, 2019 [2 favorites]


Seems to me that systemd's ambitious scope has proved, in practice, to be more of an advantage than a disadvantage when it comes to driving adoption.

An advantage to getting adopted, sure, I suppose it's convenient if you're designing a distro way beyond the point where it's just annoying as a user. It was interesting to see that even its most ardent defender admits towards the end that maybe in hindsight it would've been better to keep the various components in separate projects.

I suppose the GNOME connection explains how D-Bus made it in there.
posted by sfenders at 7:33 AM on February 5, 2019 [2 favorites]


the part of Metafilter where no one watches the video

I rarely watch videos, and I don't comment when I haven't, but this time I have both.

Not yet mentioned (that I can see) is the situation of the graybeards who know init and rc.d and friends, but who mostly run out-of-the-box Linux these days. But on the rare occasion when we need to add some sort of daemon, we go looking for these things and find e.g. systemd instead.

It's not the change that's annoying, it's the "having to figure out a whole new system to do this one simple little thing," which is a thing you rarely have to do, and it's not clear whether systemd will be around in a few months or years when you have to do the one simple little thing again.

If it's not, then whoever decided to switch stole the time spent learning from you twice: once the first time you had to learn systemd, and again when you have to learn whatever replaces systemd.

That said, systemd seems to be adequate software, and I was able to make it do the small daemon thing I needed without too much fumbling, so I won't hate on it too much until I have to deal with its replacement. (Systemd will still deserve some hate at that time for not being good enough to prevent its own replacement.)
posted by spacewrench at 7:38 AM on February 5, 2019 [7 favorites]


Strikes me as well within the realm of possibility that systemd's replacement could easily be just a better implementation of systemd. There's not a lot wrong with the conceptual design as far as I can see, the config syntax involves a pleasing lack of XML, and at this point it looks to me like systemd has about as much momentum as the kernel itself and is therefore almost as unlikely to die quickly as COBOL.

I long ago decided that since search engines are a thing I now have the luxury of being too lazy to remember how to drive any of the tools I use infrequently, so my standard process for doing anything these days starts with googling it, finding some appropriate man pages and going from there. I've generally found dealing with systemd to be negligibly more irritating than rsyslogd or dhcpd or xinetd or whatever.

But you can pry my crontabs out of my cold dead etcetera.
posted by flabdablet at 8:04 AM on February 5, 2019 [8 favorites]


"it should have been MUCH better documented"

What exactly is lacking in the documentation?
posted by floppyroofing at 8:14 AM on February 5, 2019


What exactly is lacking in the documentation?

Like, say, for instance, a broad overview of what systemd is and how it works, and how to use it?

If you were new to systemd and you wanted to add a start/stop script for a daemon you wrote, and you went to the systemd home page for information, how would you figure it out? Why is this sort of fundamental information not immediately accessible? I mean there are plenty of man pages filled with minutiae, but it is not useful to get someone up and running. Where is the systemd equivalent of Kernighan & Pike?
posted by 1970s Antihero at 8:37 AM on February 5, 2019 [4 favorites]


"If you were new to systemd and you wanted to add a start/stop script for a daemon you wrote, and you went to the systemd home page for information, how would you figure it out?"

I'll admit I probably never read a big overview, I just looked at existing unit files and read the man pages when I had questions.

If I were looking for one I think I'd start with this blog series.
posted by floppyroofing at 8:49 AM on February 5, 2019 [3 favorites]


I was preparing to oh no not again this, but on reading the transcript, it's fairly decent. This spoke to me:
… FreeBSD should not be proud of the fact that we don't have a systemd: we should see that as we're lacking something like … if we see if we're mocking systemd, mocking that the fact that Linux has systemd and all those poor Linux users with their terrible software, then it means that we're treating it as a joke and not as a source of ideas

… [T]hat doesn't just apply to FreeBSD it applies to pretty much anyone who sits there going "oh no that systemd's terrible!" … one of the biggest problems that I had with the FreeBSD community on this one was things like this [mocking Linux/systemd] because to me this says "On behalf of my community, come join FreeBSD: we'll never change, we refuse to move forward into the future!"

… [R]eally what I want people to do and they look at … systemd whether they like it or whether they don't — and especially if they don't — is "why did systemd show up?": why is systemd important? It's thinking about that means that you suddenly go "Hang on, so is systemd solving a problem? What problem is it solving? Could I solve it better?" because one of the solutions if you don't like systemd is to go and make your own — at the very least you're going to find out how much fun that is …
I could never get the hang of init scripts — or at least, run them reliably at the right time when all their requirements were satisfied — but systemd works more the way I understand as a good-enough solution to a complex problem. I'm a big fan of user systemd processes: not everything automated should run as root.
posted by scruss at 8:54 AM on February 5, 2019 [5 favorites]


a pleasing lack of XML

OMG THIS. Nothing worse than trying to figure out something by copying something else that you sort of understand (in the face of incomplete or inadequate or wrong-level-of-abstraction documentation) and you find XML config files (or worse, binary). That's one of my pet peeves with MacOS and (especially) Windows. It's inconceivable that you get enough extra startup performance to justify a whole custom binary config-parameter delivery mechanism on top of whatever other shitpile you have.
posted by spacewrench at 8:55 AM on February 5, 2019 [7 favorites]


I have never used systemd. I scrolled down that page to "Manual" and clicked it. Scrolled down to "init" and clicked it. That took me to a page that explained the details, with a link to systemd.service in a section called "concepts." I have no dog in this fight, but I feel like I could figure this out.
posted by Nothing at 8:57 AM on February 5, 2019 [2 favorites]


I think it's pretty ripe that people criticize a piece of software for not having enough documentation.... and then immediately turn around and complain that it's "not Unix-like"! That's like the old ladies at the buffet who complain that the food is terrible... and there's not enough of it!
posted by scolbath at 9:09 AM on February 5, 2019 [8 favorites]


I see that "It's Buggy" was covered in the video, so I'm all good.
posted by clawsoon at 9:34 AM on February 5, 2019


It's a lot of the same sort of griping that surrounds NetworkManager - oh boo hoo it's an API and not the config files we've been spackling more and more functionality onto since the late 90's. And yet, given an environment with thousands of hosts and some obscure bit of network update, would you rather deploy a nmcli one-or-two liner, or wrestle with scripting to handle 2000 ever so slightly different configuration files? (yeah yeah, you'd rather update the git repository and deploy a thousand new machines, retiring the machines with the wrong configuration. I know, I know.)

At the root of it, systemd doesn't bother me all that much, but I spent a lot of time off in a Solaris grotto and occasionally poking at the internals on my Mac, so Solaris' smf and macOS launchd really sort of prepared me for systemd. And systemd truly brings some stuff into the 20th century, things that weren't getting any attention at all previously.

I think the biggest complaint I have is that I swear I've run into a few systems that were an unholy mix of systemd, upstart and traditional inits.
posted by Kyol at 9:34 AM on February 5, 2019 [7 favorites]


Isn't everybody supposed to die in the fifth act of a tragedy? I think that talk was more in the structure of a comedy.
posted by clawsoon at 9:42 AM on February 5, 2019 [2 favorites]


Regarding linux distros and monocultures, I think a shift towards monoculture may be somewhat inevitable due to a partial failure of the distro model itself: no one really wants to value (or do) that type of work anymore. Integration and packaging of software is hard and largely thankless work. Over time, it's fallen behind with the pace at which software is being developed and to make up the difference software projects themselves now feel obliged to manage their own distribution. As a result we have the current explosion of domain specific package managers and alternate packaging schemes (snaps, etc). So, the version of the python library you need in the Debian repo is way out of date? Install it with pip! I'm not passing judgement one way or the other on this evolution, but it seems to leave distros without one of their major selling points. I do think there's some value in a well vetted and integrated repository of software, but it's moot if no one is buying.
posted by wordless reply at 9:43 AM on February 5, 2019 [5 favorites]


I think systemd won for the same reason why linus' kernel beat tanenbaum's and why vhs beat betamax: technology is not a "meritocracy". Other things have influence too, for better or worse.
posted by Poldo at 9:52 AM on February 5, 2019 [4 favorites]


Back when I was a fresh upstart grad student, I used to sit at a SunOS box (not even Solaris, although that came soon) while running Slackware at home and RedHat on my laptop, and I installed Debian on my advisor's laptop - in retrospect, a seriously ill-advised move. ("What's a PCMCIA wireless adapter?") I fought constantly with the ALSA sound daemon, and my ThinkPad wifi adapter broke on every kernel point-release update. Back then, the details of this argument would have mattered to me. For now, I just sighed.

(My Macs seem to be OK at staying functional and none of my compiled stuff breaks as long as I'm conservative on system upgrades. And my workhorse machines are covered by charming Someone-Else's-Problem fields...)

I should say - I read the "Contempt Culture" essay and whole-heartedly agree. I can't believe that I spend most of my time writing in Python these days: a language that assigns semantic meaning to whitespace! Whyyyyyy? So who cares what warts PHP has, or why Julia is so great - whatever gets the job done is fine.

So anyway - can someone please share a link to the talk transcript? Watching a 45 minute video is really not going to work for me...
posted by RedOrGreen at 10:02 AM on February 5, 2019 [1 favorite]


systemd is just bad software. Hardly anyone is saying that something like systemd wasn’t necessary, but it should have been modularized ...

OK. But Poettering stopped talking about it and just did it. Nothing is stopping anyone else from -doing the work- and coming up with something better.

"should ... should ... should" drags on for decades. Like Wayland. Whatever he did, he got it done. Like UNIX got done. Where's the better?
posted by Twang at 10:07 AM on February 5, 2019 [11 favorites]


I think systemd won for the same reason why linus' kernel beat tanenbaum's and why vhs beat betamax: technology is not a "meritocracy". Other things have influence too, for better or worse.

Which is also why it has a ton of desktop-like services in there that don't apply to linux on the server. People are excited about Linux On The Desktop (still, somehow, in 2019), so it gets action that servers don't. Do I necessarily care in the server world? Not particularly. Does it hurt me? Ech. Occasionally I need to wiggle a finger at journalctl because it isn't rotating properly, but no more than I had to wiggle a finger at /var/log because logrotate didn't fire properly.

I will disagree with the notion that boot speed is irrelevant - so long as there are deployers deploying thick instances (versus docker style thin containers), the amount of time and resources spent spinning those buggers up and down is measurable.

And yeah, I grumble a lot about python because oh god it's a turtle with a damn supercharger bolted on to it and it offends like 20 years of perl experience but when I stop trying to apply the wrong knowledge to it, it is a thing of beauty and joy.
posted by Kyol at 10:14 AM on February 5, 2019 [7 favorites]


domain specific package managers and alternate packaging schemes (snaps, etc).

Is snap a safe target towards which to redirect all my surplus contempt for trendy software? I installed it by mistake thinking it was SnappyData, slowly realized what it was supposed to be, and want no further part in it.
posted by sfenders at 10:31 AM on February 5, 2019 [4 favorites]


Whatever he did, he got it done. Like UNIX got done. Where's the better?

Well, as mentioned above, there's Solaris’ SMF, or Darwin’s launchd. Both of which broadly do what systemd does at a fraction of the code size. (Last time I checked, launchd had about one tenth the number of lines of code as systemd). Poettering could have forked either one of those, and changed the config file syntax to not use XML if he hated it that much.
posted by 1970s Antihero at 10:34 AM on February 5, 2019 [1 favorite]


 So anyway - can someone please share a link to the talk transcript? Watching a 45 minute video is really not going to work for me...

Click on the “· · ·” below and to the right of the video, and "Open Transcript" is an option. It's all voice-recognition, so is barely punctuated. The excerpt I put up needed a bunch of editing. I am needlessly amused by the "Leonard puttering with the system D" that it generates, though
posted by scruss at 12:06 PM on February 5, 2019 [1 favorite]


Is this actually true?

In my experience, yes, it's true. Since writing the Contempt Culture post I have had I have lost count women come to me and say thank you for writing it, because it captured their story and their journey and their struggles so well.

If I were rewriting it today I'd focus on frontend JavaScript instead, as that's the majority of where misogyny-based contempt is being deposited nowadays.

And, yes this transcends the tech communities, but I know the tech community and I'm not really qualified to speak about Not The Tech Community.

Glad ya'll liked my piece! 😄
posted by aurynn at 12:19 PM on February 5, 2019 [69 favorites]


flabdablet: "...even after the GNOME devs threw the switch to vaudeville..."

Whoa, that is a sweet turn of phrase! :7)
--
Also, I remember when Sun dropped the Service Management Facility (SMF) on us. We all kind of grunted, read about how it worked, and started using it.

Mind you, I believe that I have personally experienced more unpleasant surprises with systemd than I have with SMF, but that's probably due to my own laziness and not with the tool.
posted by wenestvedt at 12:49 PM on February 5, 2019


That "Contempt Culture" piece is resonating so strongly with me that my ears are ringing! Thanks, aurynn!

When I had a realization like that (and then another one, years later), it reminded me to shine that light all over my life and see where else I should stop running my mouth. I started tempering things I said with "In my experience..." and "For the work I do..." and that helped change my words from categorical PRONOUNCEMENTS to just sharing.

I work in technology, and the tools I use in this environment actually put me in the minority position (despite being, as a white dude, a majority member of the nerd tribe). So I am on the receiving end of this, and I know exactly where it comes from, so it stands out really clearly to me. And every time I wince again at what a loudmouth yout' I was.
posted by wenestvedt at 1:02 PM on February 5, 2019 [6 favorites]


In case anyone else is interested, here's the verison of the same talk that he gave at BSDCan. Slides and audio only, no Q&A section.
posted by vibratory manner of working at 1:38 PM on February 5, 2019


...linus' kernel beat tanenbaum's...

Tanenbaum's kernel (i.e. Minix) was a toy operating system written as the example for a textbook on operating systems. Nobody, including Tanenbaum himself considered it a serious operating system suitable for anything except hobbyists and students. Furthermore, the licensing prevented anyone from easily redistributing improved versions.

Linux was initially (briefly) a drop-in replacement for the Minix kernel that supported multi-process file I/O (under Minix, only one program could use the disk at a time) and 32-bit protected mode, neither of which Minix did.

(Now, if you'd said BSD, you might have had a point. In that case, I'd argue that Linux's early incompleteness ended up being the game changer because the first generation of users had had to become very familiar with its inner workings in a way most BSD users didn't just to get the thing working properly. Once they went out into the workplace, they stuck with Linux because they knew they could make it work.)
posted by suetanvil at 1:48 PM on February 5, 2019 [9 favorites]


Also, wrt. Contempt Culture:

I've always kind of been perplexed at the idea that if language $L sucks, the people who use it must be bad programmers. Surely if $L sucks but you still do great things with it, doesn't that mean that you're a better programmer for having overcome the limitations of $L?
posted by suetanvil at 1:53 PM on February 5, 2019 [8 favorites]


I want to go out on a limb here and say that there are real differences between languages. But first, I have to say that rarely do you get a chance to choose a language: it's usually determined by the platform or other legacy you've inherited.

Nevertheless, there are discriminators that are worthwhile to consider:

- inherent safety (e.g., no scribbling over random memory)
- boilerplate
- available frameworks
- DRY (Don't Repeat Yourself)
- "nimbleness"
- write-only languages
- how much recondite knowledge you need to be effective

There are legitimate differences between languages/frameworks.

Chances are you should not be using a language like C or C++. There are just too many ways for that to go wrong. There's a real need for a low-level, machine-friendly language to replace them and I have high hopes for Rust. Go, too, maybe? But this is a really weak area now and we should all want to do better here.

How much boilerplate do you need to write and will it get in the way of future changes you need to make? The frameworks that are available will have a lot of influence on this. How are your frameworks helping and hindering?

How often do you have to encode the same design decision redundantly? How many files/lines do you need to change to add a new field to a record? One would be ideal.

Can someone else make sense of what you have written? Can you make sense of it in a year? Concision is a benefit for sure, but how legible is your design intent in the code?

Is category theory important? Of course. Should I need to understand it to maintain a business system? Almost certainly no. If someone could come up with a category-based language or framework that worked better because of that and also did not require being in the category priesthood (also note: several rival dogmas) to develop and maintain code, then great. Otherwise, it's just a hurdle for someone trying to get their job done.

There really are some serious issues to contend with here. It should be done with respect and empathy because everyone comes at things from different angles and with different intentions. But at the same time... we need to get better at doing at what we do because it is just not good enough.

"Getting better" means making things more sound, more efficient, and more inclusive.
posted by sjswitzer at 2:25 PM on February 5, 2019 [5 favorites]


1970s Antihero: Conflating launchd or smf with systemd is a bit misleading though, since you'd have to _only_ compare the LoC count of the systemd process itself, not any of the other stuff in the project. When I describe systemd as implementing a "System Layer" for Linux and that that includes a bunch of other stuff I'm not kidding. Yes they could've broken that functionality out (it is actually reasonably modular, as I understand it) but like I also say, being a BSD person I do understand the drive to maintain semi-related code in the one place.

Also: Hi everyone! Glad you all liked my talk. Feel free to hit me with any questions you might have. 🙃
posted by jeamland at 2:28 PM on February 5, 2019 [25 favorites]


From the Contempt Culture piece:

It’s 2015, and I saw a presenter at a Python conference make fun of Java.

LOLOLOLOLOLOL.

It is so insanely juvenile to rag on other languages. I mean, I did give one guy a hard time for insisting on building his website in OCaml, but only because it wasn't really fit for purpose, not because it is a bad language. Yeah, PHP is kind of a mess, but it works, and I built lots of websites with it, and good on anyone who works with it. Java gets hate because 1) Oracle and 2) it is everywhere. All languages, at their core, are doing the same things with different sets of commands. To each their own.
posted by grumpybear69 at 2:34 PM on February 5, 2019 [2 favorites]


Linux Weekly News wrote a summary article that may be tidier than reading the transcript. (LWN is worth a subscription; their technical reporting is very good.)
posted by amk at 2:43 PM on February 5, 2019 [2 favorites]


LOL, another systemd discussion and still at a loss as to the essence of the controversy. I've grouched about the software to myself dealing with some issue or another too insignificant to recall but can say that about hundreds of software tools (emacs user don't get me started there ;-) but there is something about systemd that seems to seriously rankle some folks. I certainly used and hacked the funky little scripts in /etc/init.d/ - no love there, but don't recall anything other than an occasional grumble. There certainly is animalistic level head butting in some parts of the tech communities but that's not quite it either. I enjoy wading into a python vs java (on either side) but that's not quite it either (java is sooo wordy and ugh oracle but come on 2 dot or 3 dot already this decade) but the systemd stuff just seem more spontaneous combustion than real.
posted by sammyo at 3:11 PM on February 5, 2019 [1 favorite]


Tanenbaum's kernel (i.e. Minix) was a toy operating system written as the example for a textbook on operating systems. Nobody, including Tanenbaum himself considered it a serious operating system suitable for anything except hobbyists and students. Furthermore, the licensing prevented anyone from easily redistributing improved versions.

Tanenbaum's "Structured Computer Organization" was groundbreaking for me. A ,what the younglings call, "full stack" treatment from transistors to logic gates, all the way up to o/s. Really tied the whole room together.

The argument was "monolithic vs modular", and honestly I conceded years ago when I stopped building my own kernels with all the needed drivers built in.
posted by mikelieman at 3:19 PM on February 5, 2019 [3 favorites]


It is so insanely juvenile to rag on other languages.

OK but surely we can all agree that ColdFusion is the worst, right? I mean you’d have to be some kinda lunatic to write anything substantial in that
posted by Huffy Puffy at 4:35 PM on February 5, 2019 [18 favorites]


I am reminded of a comedic hacker text file from the 70s which was meant to be a novice questioning an experienced hacker. When asked about which language to choose, the wise hacker says something like:
All languages have the True Tao…but try not to program in COBOL if you can avoid it.
Plus ça change, eh?
posted by rum-soaked space hobo at 4:45 PM on February 5, 2019 [4 favorites]


Conflating launchd or smf with systemd is a bit misleading though, since you'd have to _only_ compare the LoC count of the systemd process itself, not any of the other stuff in the project.

I’m sorry, I should have qualified my statement. When I checked, launchd was one-tenth the LoC of systemd not counting udev and dbus.

I do understand the drive to maintain semi-related code in the one place.

I really don’t give a damn about how the code is laid out. What I do care about is that systemd includes functionality, like dbus, that I have absolutely no use for running a headless server, which I can’t turn off, and increases the likelihood of bugs and security flaws.

Tanenbaum

Interestingly, if you look at the code for Google’s Fuscia, its architecture is exactly like Tanenbaum describes: microkernel, device drivers in userspace processes, etc. It all comes around again.
posted by 1970s Antihero at 6:47 PM on February 5, 2019


It is so insanely juvenile to rag on other languages.

In 20 years of doing this for a living I have noticed 3 constants in high performing, happy, technical teams:

1. Diversity among many axes. Because the total experience of a team is the sum of the *unique* experiences of each member of the team.
2. No time spent complaining about people who use X tool or the people who wrote X code. Who cares.
3. Ruthlessly critical of their tools and code. Always "our" tools and code. Doesn't matter who wrote it or chose it, it is always "ours" and they collaborate on improving it. Specific communication training in how to do this and how to separate your ego from the collaborative creative work.

Why should you care about this if you are a non-programmer? Because we have your personal data and the language we chose has a large effect on the number of security bugs in our code.

Language designers get rightly annoyed when people tell them their entire career and art form is worthless because we're all annoyed at jerks making fun of people stuck writing PHP.

What is juvenile is to tie your ego to your tools. You should examine and improve your workplace physical environment, your workplace social environment - including the language used in it - and your economic environment. You should also examine and improve the mental environment you spend the majority of the workday in, which are the programming languages and software tools you use.

In a world where your corporate bosses want features delivered by certain dates, choosing the right tools can be crucial to your team's mental health. Go ahead and say "PHP is good enough for me", but my team will use something better (definitely something not disowned as "a bad language" by its creator), go home early and suffer less from overtime induced burnout and depression (and the corresponding suicide rate).
posted by Infracanophile at 7:54 PM on February 5, 2019 [11 favorites]


People are excited about Linux On The Desktop (still, somehow, in 2019)

I don't think it will ever be the Year Of Linux On The Desktop.

What I do think might happen is that Linux desktop users will probably keep being Linux desktop users at about the same rate per 100,000 people as we've been for the last few decades, while desktop in general gets its insides hollowed out by personal touch screen devices as Microsoft ever more desperately tries to turn itself into Google or Amazon depending which of its internal insecurities is screaming louder this week.

For the technically unsophisticated, Linux desktop use has involved less frustration and annoyance than Windows desktop use, on balance, since about the release of Windows 8. I'm not only needing to spend less time supporting my Debian-based customers than my Windows-based customers these days (and I have roughly equal numbers of each), I'm spending way less time. The Windows installations keep finding new and innovative ways for updates to break them; the Debian installations typically don't get updated at all until I do it for them, so between my visits they have pretty much 100% functional availability. Several of the Windows users have been impacted by malware. None of the Debian users have.

In 2029 I expect to see endless amounts of speculation about whether or not this is the year of Linux On The Phone.

Glad ya'll liked my piece! 😄

OMG aurynn is MeFi's Own. How did I not know this?

Whoa, that is a sweet turn of phrase! :7)

Not mine; it belongs to our incomparable former Prime Minister.

Contempt culture is toxic to be sure, but Keating wielded contempt with a style nothing short of pure art. Here's Keating's casual dismissal of former Treasurer Peter Costello as "all tip and no iceberg" celebrated in song.

when I stop trying to apply the wrong knowledge to it, it is a thing of beauty and joy

This is an excellent general principle.

I've always kind of been perplexed at the idea that if language $L sucks, the people who use it must be bad programmers. Surely if $L sucks but you still do great things with it, doesn't that mean that you're a better programmer for having overcome the limitations of $L?

That's my own internal justification for persisting with cmd scripts, even though these days my most common use for these is working around limitations in PowerShell (a job which, as it turns out, can also be done with less visual untidiness using jscript).

The more annoying the language, the greater the satisfaction in bending it to one's will (though I am prepared to allow as how this correlation breaks down at Malbolge).

if you look at the code for Google’s Fuscia, its architecture is exactly like Tanenbaum describes: microkernel, device drivers in userspace processes, etc. It all comes around again.

Redox is like this as well, plus all its vitals are both very small and written in Rust. If Redox takes off I will be interested to see how its CVE count compares to Fuschia's, whose kernel is somewhat more complicated and implemented in C++.
posted by flabdablet at 8:10 PM on February 5, 2019 [2 favorites]


In a world where your corporate bosses want features delivered by certain dates, choosing the right tools can be crucial to your team's mental health. Go ahead and say "PHP is good enough for me", but my team will use something better (definitely something not disowned as "a bad language" by its creator), go home early and suffer less from overtime induced burnout and depression (and the corresponding suicide rate).

PHP is so bad it will make you want to kill yourself is 100% contempt culture no matter how much padding and concern trolling you wrap around it.

Anyway I still like PHP even though i also write codeine Haskell and a couple of other more admired languages for the same reason that an author might choose to write in English rather than Esperanto despite English being a clusterfuck of borrowed vocabularies and unintuitive spelling.
posted by Space Coyote at 8:58 PM on February 5, 2019 [9 favorites]


Thanks for the great example of bad faith reading.

PHP being a clusterfuck means work takes longer to complete. Bosses don't care, they want the work done by the deadline. This industry is a garbage fire of mandatory overtime exploitation.

Those are facts, people can connect the dots themselves if you don't want me to do it.

I'll happily take any advice from people on how to better address those issues. So far my work training people in modern languages that get them paid more, are easier to learn, get their work finished faster with less stress and are applicable in more industries than a small subset of webdev has been more fulfilling than any code I've ever put in production. If you can do it without minimizing the mental health issues in my industry all the better.
posted by Infracanophile at 9:17 PM on February 5, 2019 [1 favorite]


I've always kind of been perplexed at the idea that if language $L sucks, the people who use it must be bad programmers.

Well, if they were good programmers, they wouldn't have chosen language $L, would they?
posted by aurynn at 11:37 PM on February 5, 2019 [2 favorites]


I don't follow this debate, but just reading the transcript, it's not a very compelling argument from a research and philosophical standpoint. Like, if you're talking to people who don't like systemd, you'd better first show that you understand on a conceptual level what their criticisms are. If your conclusion is to tell those people to find some module or feature they'll like and start from there... well that is kind of incredibly patronizing and I don't see how that nudging tactic works. I don't know the issue, but it's just this level of persuasive rhetoric that leaps out at me.
posted by polymodus at 12:46 AM on February 6, 2019


Another example, if the complaint was that this is a political move or that the system is "monolithic", then using Eugenia Cheng's talk from last week--you kind of have the obligation to address in what sense are their concerns about politics or about improper engineering? Because if the argument about monolithic is precisely that the whole project is in the sense of hegemonically political, then the fact that people are free to pick and choose modules is a false choice argument. So that would have to be authentically addressed to the audience/reader, etc.

Like, this is all basic analysis that even if you're not in the field, you can sort of pick out these patterns of cross-talking rhetoric and look at the conflict and nuance more closely.
posted by polymodus at 12:50 AM on February 6, 2019


What I hate is that all of this snobbery ends up clouding the actual debates on the relative merits of programming languages. I can read The PHP "clawhammer" essays and identify with a lot of the frustrations in them. I think that as nerds in general we've over-valued performative ranting as a rhetorical style (Thanks, Usenet) and this has led to a horrible Story-of-Mel "REAL PROGRAMMERS CODE IN BINARY" kind of posturing around the whole debate.

We should look to Matthew Garrett's point about not using languages like C. Indeed, one of the real problems with most troublesome languages is the categories of bugs that they let you hide until it's too late. With PHP, it's any number of error conditions that the system grinds on past (because its primary goal was always to spit out HTML, and try not to stop you from doing so by crashing on critical errors), and with C it's the subtle memory allocation problems that won't bite you until the program has been running for a few days (and the original authors of C mostly wrote programs that exited in a matter of seconds).

The real problem here isn't the clawhammers, but the knives without handle guards. We hear about people cutting themselves on the things and say "Oh well that's a theoretical possibility, of course, but nobody I work with has done that in years!" and we look around at our colleagues and ignore the thousands of people who came out of A&E and said "never again" instead of "I must study for years now, to learn to hold that knife properly."

I got involved in that thread after mjg59 posted the OpenLDAP CVEs list. The OpenLDAP maintainer is not only a very skilled and disciplined C programmer, but also a violin virtuoso. He likened engineering languages to actually avoid mistakes to putting frets on a violin. I'm willing to accept that perspective, but also to say that perhaps an ukulele is a better folk instrument than a fiddle, these days.

He also trotted out the old "a poor craftsman (sic) blames his (sic) tools" canard.

"Oh no," I said. "We're not blaming our tools…"
posted by rum-soaked space hobo at 1:35 AM on February 6, 2019 [4 favorites]


Nth-ing the recommendation for aurynn shaw's Contempt Culture
And at the time, as a new developer, I internalised this pretty heavily. The language I was in was blessed, obviously, not because I was using it but because it was better designed than a language like PHP, less wordy and annoying than Java, more flexible than many other options.

It didn’t matter that it was (and remains) difficult to read, it was that we were better for using it.
If you're lucky, with age and experience you can accumulate some semblance of wisdom, and realize that all along it was just personal preference and comfort level.
posted by mikelieman at 1:57 AM on February 6, 2019 [4 favorites]


I am old and set in my ways and so I dislike the sudden whole-cloth "everything is systemd now!" shift in Linux because suddenly, everything I've been doing to keep my system running is no longer working as I expect it. But I think the attitude of systemd folks towards defaulting to using unsupported NTP servers against the wishes of the server operators seems kind of telling.
posted by rmd1023 at 4:39 AM on February 6, 2019 [3 favorites]


So here's the thing:

If the main thrust against systemd was that Lennart Poettering is a tyrannical asshole, I would actually be totally OK with the community refusing to adopt it. I think as a community, software developers (particularly open source software developers) are almost pathologically drawn to what are basically mean father figures (of course, nearly always white). It sucks. It gatekeeps women and minorities. It pushes away people who might want to collaborate more empathetically. It prizes going along with the in-crowd (even if the in-crowd is basically being awful to people) over diversity and acceptance.

I would 100% support the idea that we'd be better off in the long term to refuse to adopt projects with people like Lennart at their head, even if in the short term it hurt.

But: in my experience, most people who bring up Lennart's behavior only do so because they already hate systemd, and it's just one more thing to add to the bonfire. Frequently, the same people who cheer on asshole behavior by other project leads are suddenly "very concerned" with how Lennart interacts with the community.

I'm not saying he's not an asshole. I'm just saying, I'm pretty wary of arguments from people who are totally pro-asshole when it comes to e.g. Linus or Theo or all sorts of other big name project leads, and only seem to care about Lennart after having already decided that systemd ruined their childhood or something.
posted by tocts at 4:55 AM on February 6, 2019 [3 favorites]


i also write codeine Haskell

‘Haskell while High’ … now there's a book waiting to be written.
posted by scruss at 7:30 AM on February 6, 2019 [4 favorites]


To me, the guy always seemed pretty polite for somebody who gets a lot of violent hate mail just for writing software that the people who maintain their distribution of choice like more than they do.

I dunno, maybe there's something to the complaints, but I see a *lot* of people having trouble distinguishing between "that decision didn't get made they way I would have" and "I'm being repressed".
posted by floppyroofing at 9:31 AM on February 6, 2019 [1 favorite]


Well, if they were good programmers, they wouldn't have chosen language $L, would they?

Hey, if someone offered me $800/hour to write database loaders in COBOL, I'd be speedreading COBOL for Dummies so fast it'd make your head spin.
posted by suetanvil at 9:31 AM on February 6, 2019 [7 favorites]


Isn't the saying "familiarity breeds contempt"? I worked with PHP for years... it's terrible, inexpressive, and inconsistent.

I'm sure you can write "good PHP" (satan knows I tried) but the standard of "good PHP" looks a lot like dynamically typed Java.

It's a mistake to conflate a programming language with the programmers who use it. The best knowledge is fairly portable between languages that are in roughly the same family, the rest is details. Any programmer should be able to pick up a second or a third or a fourth language.

To say that PHP is beyond criticism because to criticize PHP would exclude "PHP programmers" is absurd. Even when I worked with PHP I would never have described myself as a "PHP programmer". It wasn't the first language I learned, it was never the only language I knew.. even the first language I learned never subsumed my identity. "PHP programmer" isn't a very person-focused descriptor.


Also, the ones who are saying "I program in Haskell, I write raw machine code, I've tasted the holy grail and I still like PHP" are probably sandbagging. There's no way they're putting in the effort to write high quality software in PHP where every pattern they've learned in higher level languages takes two pages of code and falls over if they look at it funny. They may pick it up for quick and dirty hacks but I highly doubt they're putting any major effort in. "I use PHP when I'm in a hurry and doing something that doesn't matter very much." is a wildly different reality from "I program in PHP to feed my family."

To go further and say that criticism of PHP is rooted in contempt for women is such a stretch. Nothing but PHP taught me to hate PHP. I don't think I ever even dealt with a woman who wasn't a client or a designer when I did my time in the PHP dungeons. I was unaware and will remain skeptical that "design -> wordpress -> php" is a path to programming for large numbers of women programmers. If true, that's horrible--PHP (particularly Wordpress-flavored PHP) seems like a terrible place to start one's programming journey. Would you not talk about fire safety if women were among those setting themselves on fire for money?

I'm not saying that programmers who start with PHP or less than or forever tainted or anything like that.. I just think that PHP and front-end JS aren't great gateways because they introduce a lot of concepts that beginners shouldn't really worry themselves about. (Not gratuitously--since they might both be considered web DSLs there's a lot of fundamentals that might get missed in learning them as a first language in the pursuit of specific work goals.)
posted by yonega at 11:02 AM on February 6, 2019 [4 favorites]


Hey, if someone offered me $800/hour to write database loaders in COBOL, I'd be speedreading COBOL for Dummies so fast it'd make your head spin.

I'm so there with you it's not even funny. 😄

It's a mistake to conflate a programming language with the programmers who use it.

Yes, it is a mistake, which is why I wrote the piece in the first place. Our culture structures our in-group/out-group based on the perceived knowledge we hold, and use of PHP is taken by the culture as lesser or inferior knowledge. It's a reflex learned behaviour, not something we're consciously or actively doing.

To go further and say that criticism of PHP is rooted in contempt for women is such a stretch.

The PHP-based contempt comes from PHP enabling people who weren't part of the "tech culture" being able to make websites and participate in the young Web, contempt that is extremely classist, and classist issues hit women considerably harder. This led into the PHP users getting forced out of spaces by contemptuous behaviour because they were junior programmers, going off and making their own resources (things like Matt's Script Archive come to mind) and creating a cycle where PHP is bad because PHP is bad.

If true, that's horrible--PHP (particularly Wordpress-flavored PHP) seems like a terrible place to start one's programming journey.

And this is the contempt culture I rail against - it's only horrible because we, the rest of us, make it horrible, and you are literally framing that another language would be better to learn, perpetuating the contempt culture. Maybe stop doing that.

People learn on this path, and they're afraid to talk about the things they've done because we are so cruel about language use as the axis of belonging, we use it to divide our communities, we teach programmers in "lesser languages" that they shouldn't advertise that they use a lesser language. I describe myself as a Python programmer but the PHP people I know always, always apologise first. That apology? That's contempt culture.
posted by aurynn at 12:38 PM on February 6, 2019 [14 favorites]


I'm not framing anything: I believe another language would be better to learn first.
posted by yonega at 1:10 PM on February 6, 2019 [2 favorites]


I'm not framing anything: I believe another language would be better to learn first.

Uh, this is literally what a framing is.
posted by aurynn at 1:10 PM on February 6, 2019 [5 favorites]


Have you ever made the majority of your income writing PHP every day, week after week or are you just agitating for PHP from the safety of your "Python programmer" position?

I've eaten it, I've breathed it. It did me harm.

Sure people make cool stuff in PHP. I don't feel anything either way about people who write PHP (unless they were in the codebase before me). I'm not advocating excluding them instead perhaps encourage them to broaden their horizons instead of saying that PHP is where they belong and that's okay.
posted by yonega at 1:19 PM on February 6, 2019 [1 favorite]


Okay now you're framing me as being a member of a different in-group and that I have no right to speak out on issue of the way that groups I participate in marginalise users of languages like PHP.

it did me harm This is contempt culture.

I'm not advocating that PHP is where they belong, I'm advocating for not using "PHP is bad/harmful/whatever" to diminish and 💩 on what they've done. The very act of saying "PHP has caused me harm" is what's done that. It's a direct reference to the Djikstra quote about BASIC causing "irreparable brain damage", it frames PHP as directly harmful to a person as a programmer instead of just being a tool that helps us achieve tasks and make cool things.
posted by aurynn at 1:24 PM on February 6, 2019 [5 favorites]


Again, I'm speaking from my personal experience, which you evidently do not share. Working with PHP was very harmful to me and it is not for you to say that I'm being hyperbolic because I seriously am not.
posted by yonega at 1:32 PM on February 6, 2019 [4 favorites]


I'm not saying you're being hyperbolic. I'm saying you're propagating contempt culture in the way you're discussing your experiences, and that's harmful for the conversation around tech culture and how we manage ingroup and outgroup dynamics.

That it has been a difficult path for you remains true, just, find better words that don't marginalise others.
posted by aurynn at 1:36 PM on February 6, 2019 [5 favorites]


Tools are structures; there's no such thing as "just being a tool that helps". Tools are complicit in harm and to frame it as people having pure agency over their choices is a way to rationalize never improving how things are done. It's programming neoliberalism.

I get that Djikstra's comments were a product of his era. But today, we read Djikstra and generally understand he wasn't making an ad hominem in reference to BASIC and goto statements. People (programmers) can perpetuate a problematic attitude by imitating his communication when there's better ways of communicating and building relationships. On the other hand, it's pretty clear the actual issues Djikstra was concerned about, from an engineering standpoint, putting aside his rhetoric.
posted by polymodus at 1:37 PM on February 6, 2019 [2 favorites]


In any case, glad you all enjoyed my piece, and I'm really glad that it's caused systemic change within programming communities to happen.

Thanks for all your nice comments, but I need to bow out now! 🙂
posted by aurynn at 1:44 PM on February 6, 2019 [6 favorites]


I intended to make fun of a language, the repercussion is that people from minority backgrounds wouldn’t want to talk to me about the things they’d done in that language, they wouldn’t feel safe talking about their achievements and exploits.

As someone whose received some of this tech toxicity, I still see the explanation as centering on the privileged. I would refine it in that, it's not about me not wanting to discuss my work with them, because that's merely a narrative of their interest in a "technological imperialism" in which the privileged have this centered compulsion to know and control all of tech. That is worth acknowledging. The repercussion is the psychological harm on my freedoms and autonomies as a programmer or person working in STEM.

Overall, I was lucky not to be too exposed to tech culture when I learned computer science. For people whose work is to study computer languages and are excited about them, it's curiosity and appreciation that are the motivators, and also necessary is a space that permits appreciation as well as evaluative critique. It's having that intellectual space that's a struggle and an ideal. Meanwhile there's all sorts of examples of dogma and there are times that dogmas get internalized (undergraduate example: "C++ is harmful") without one having had the opportunity to examine it for themselves.
posted by polymodus at 3:18 PM on February 6, 2019


From the talk: "On behalf of my community, come join FreeBSD: we'll never change, we refuse to move forward into the future!"

Mocking BSD for wanting to stick with familiar software instead of enthusiastically switching to "the future" is contempt culture. In general I feel like the talk pulls an interesting rhetorical maneuver where it casts wanting to stick with init as holding systemd in contempt, and thereby makes it the responsibility of the init-favorers to find positive things in systemd rather than the responsibility of systemd-favorers to make systemd welcoming.
posted by Pyry at 3:52 PM on February 6, 2019 [4 favorites]


It seems to me that SystemV, aka init, introduced the 'system' layer a long time ago.

(I'm guessing the euphony with "systemd" is intended.)

From non-developer perspective, real difference is the dependency model rather than the rote priority system, and better handling of configuration changes. With the expected trade offs: less brittle, but more to go wrong.
posted by snuffleupagus at 5:11 PM on February 6, 2019


Is there a translation for any of this for people who are not deeply entrenched in developer culture? I didn't understand any of it.
posted by runcibleshaw at 10:29 PM on February 6, 2019


Again, I'm speaking from my personal experience, which you evidently do not share. Working with PHP was very harmful to me and it is not for you to say that I'm being hyperbolic because I seriously am not.

As someone who has spent years on PHP projects, Python projects, C projects, and even Perl projects at various times, I think you're catastrophizing.

Some languages make doing inadvisable things easy. Some don't. Aside from syntax, which isn't really all that different in most languages in wide use today they are all (at least the ones I've used) perfectly useable. After all, a high school friend of mine used to write DOS TSR utilities in BASIC.

I was going to elaborate on why people end up using the languages they do and how it really doesn't matter in the end, but my browser crashed and I don't feel like rewriting another couple of paragraphs on my phone at 2 in the morning.
posted by wierdo at 11:18 PM on February 6, 2019 [3 favorites]


Is there a translation for any of this for people who are not deeply entrenched in developer culture? I didn't understand any of it.

Programming languages and system architectures are tools that can form serious subcultures around them.

So they are tools to do a job like a hammer or a medicine or a math theorem. Programming as a field is young and our tools suck. They're getting put in everything now. Hating your daily work environment is a common thing. Training takes time and requires access, the combo is an effective gatekeeper. Open discussions and collaborative development, in the open, has been really good at improving things quickly.

So they are subcultures that formed around the tools that take a community effort to build, learn and use effectively. They can become part of your identity and it is definitely part of how your peers perceive you. The public image of a lot of tools have irrational fads that affect your status and your career. The fad driven churn privileges those with better access and time for the training, often for a negative return on investment in the end.

Opinions on how to solve these combined issues, which take priority and how to frame the issue is all over the place.

Also the average culture is toxic and crappy and exploitative and hierarchical and we have a lot of very right wing coworkers (even more than we thought it turns out). So the conversation about it is so often crap and the endless cliche hot takes so overdone that it is a touchy subject. And yet most new knowledge is disseminated on the same media as the crap and involves comparisons between alternative approaches.

In this context a video was posted about an ambitious and controversial new tool that is becoming ubiquitous, which also embodies a very different design philosophy than the one shared by the rest of the environment it is spreading into. It is a really interesting technical design and political issue. The discussion has been ongoing for a long time and fits the usual pattern unfortunately (but the links posted here are very good, imo).
posted by Infracanophile at 2:31 AM on February 7, 2019 [4 favorites]


Well but it does matter. I want to use lambdas in my day-to-day programming, and BASIC doesn't offer that as an abstraction. Languages are defined crucially by their semantics, in terms of what abstractions and what libraries they offer. To say that ultimately it doesn't matter is like the opposite extreme of catastrophizing.
posted by polymodus at 2:34 AM on February 7, 2019 [1 favorite]


I might now also point out that "Systeme D" is a French idiom for "just hack junk together quickly".
posted by rum-soaked space hobo at 3:21 AM on February 7, 2019 [2 favorites]


If I were rewriting it today I'd focus on frontend JavaScript instead, as that's the majority of where misogyny-based contempt is being deposited nowadays.

I thought that JavaScript was codebro-certified these days, but maybe that's mostly Node.js and the like?
posted by thelonius at 4:28 AM on February 7, 2019


The uppermost layers of the Jenga tower of frameworks built from JavaScript generally remain codebro-certified, but contempt for the language itself is close to universal. This strikes me as odd; I've used lots of languages with worse and more numerous semantic and syntactical pitfalls than JS and many that are far less performant.

My favourite JS framework is Vanilla, just because I'm kind of old school and I'm happier when I can maintain some kind of illusion that what I'm typing into the editor is actually what I meant. If I were forced to produce JS in serious quantities at serious speeds that would probably change.

Contemplating trying to be productive in several of the worse-than-JS languages (my personal list of which has COBOL as the earliest member, PHP being among the later arrivals) causes me dread, not contempt. But there are others, like the cmd scripting language from Windows, that are uglier than PHP but working in which I'm completely comfortable even though doing so involves almost continuous swearing; and others still, like the sed stream editing language, that are so bad that attempting to build anything readable in them is akin to solving a crossword puzzle and enjoyable for that reason alone.

The thing to do when programming in any of these manifestly defective languages is to come to grips with their grain, and program with the grain instead of against it. Trying to use techniques that are ill-supported in any given language generally ends badly. Going the long way round in order to stick with what the language can express clearly without endless amounts of spackle and glue is almost always better. Having a solid grasp of just how far you need to keep the nails from the ends of the slats to avoid having the whole thing collapse as soon as somebody other than you sits on it is vital as well.

I can't see myself being motivated to learn enough PHP to get seriously productive in it, but I certainly hold no contempt whatsoever for those who have. My life ended up with a lot of cmd in it. If your life ended up with a lot of PHP in it, and you're as familiar and comfortable with PHP as I am with cmd, I have nothing but respect for the usefulness of your experience.
posted by flabdablet at 6:54 AM on February 7, 2019 [4 favorites]


But: in my experience, most people who bring up Lennart's behavior only do so because they already hate systemd, and it's just one more thing to add to the bonfire. Frequently, the same people who cheer on asshole behavior by other project leads are suddenly "very concerned" with how Lennart interacts with the community.

This is anecdote against anecdote, but this hasn't been my experience. I'm one of those people who mostly finds systemd just fine to use on a regular basis, has had no problem paring down its footprint for headless servers, and (kind of like you've experienced with people griping about Poettering) find the most aggressive screeds against it to be pretty silly and disingenuous (like the one linked above that's all like "LOOK AT THIS ONE BUG! THE WHOLE THING IS SHIT!" or the way that abstractions apparently become a universally bad thing in the context of this one argument). That doesn't mean I don't have real concerns with systemd: I think some of its expansion beyond just being an init system is justified in unifying several different system concerns that are pretty coupled in practice, but its expansion seems less driven by those technical justifications and more by political goals. That's extremely frustrating. And, finally, as far as contempt culture goes, Poettering is among the more potent propagators of it, since his contempt seems to extend to just about anybody who "disagrees" with him on a technical basis (I put "disagree" in quotes because it seems to extend to such instances as "filing a bug report"). That's casting exactly the sort of glib, "PEBKAC"-tattooed, user-hating figure that has done so much to make programming culture uninviting. I think it's more harmful than the rhetoric from the cloud-yellers overall, since for the most part, Poettering has a lot more power over how the system actually works.

It's frustrating, generally, because contempt culture has made it a lot more difficult to talk about the technical downsides of things in an actually beneficial way. The water is polluted by knee-jerk imperiousness, so it becomes hard to tell if someone's objecting to something pragmatically or if it's a proxy argument being made to support their personal holy war. It makes it really unfun to talk about these things with people, and when they're things you enjoy, that really shouldn't be the case.
posted by invitapriore at 2:04 PM on February 7, 2019 [4 favorites]


And, finally, as far as contempt culture goes, Poettering is among the more potent propagators of it, since his contempt seems to extend to just about anybody who "disagrees" with him on a technical basis (I put "disagree" in quotes because it seems to extend to such instances as "filing a bug report").

This. Those who defend Lennart's attitude on the basis that he generally avoids the kind of direct personal abuse that Linus has so often handed out should ponder the fact that brick walls and bulldozers don't indulge in it either.

That Lennart vs Google NTP server admin thread linked above is just teeth-grindingly painful to read.
posted by flabdablet at 2:14 PM on February 7, 2019 [4 favorites]


Antipathy towards JS has everything to do with the history of web browsers, each of which for a very long time had a different implementation of the language which meant, in practice, that web JS was full of IF IE7 THEN X ELSE IF IE9 THEN THIS ELSE IF FIREFOX etc. Serious web shops would have rooms full of computers with different browsers to test all of the various scenarios and it was just generally a huge mess and rarely worked across all browsers. There were also for a long time no real frameworks or structure which meant lots of undecipherable spaghetti code.

Things got a lot better with the advent of jQuery and then Angular and now React and Vue. Also: JS consoles in the browser, standard implementation across the board and, critically, breakpoints and logging. It is far easier to write a coherent, well-structured application in pure JS now that will work for almost everyone than it has ever been in the past.
posted by grumpybear69 at 5:53 PM on February 7, 2019 [1 favorite]


$800/hour to write database loaders in COBOL

As someone who has actually written in COBOL, I'd be saying "come back when you're talking real money".

Didn't watch the video - did see the talk.
posted by HiroProtagonist at 7:19 PM on February 7, 2019 [4 favorites]


Eh, maybe I'm a bit dull, but I've been paid less for programming in RPG. :(
posted by wierdo at 5:25 PM on February 8, 2019 [1 favorite]


$800/hour to write database loaders in COBOL

As someone who has actually written in COBOL, I'd be saying "come back when you're talking real money".


Out of morbid curiosity (as I get ready to send out bills) is that because freelance COBOL programmers actually get paid more than Biglaw partners, or because it's that miserable?
posted by snuffleupagus at 5:42 PM on February 8, 2019 [2 favorites]


There is indeed big money in COBOL programming, but not for people who added the language to their repertoire via COBOL For Dummies.

COBOL and monstrous legacy systems that should have been migrated off twenty years ago, but never were because every year the prospect of doing so looks even more frightening, go pretty much hand in glove.

Legacy COBOL-based systems tend to be brittle, and adapting them to new business requirements risks multiple cascading failures. But they also tend to live at the core of processes responsible for the correct routing of billions of dollars, one way or another.

The folks who earn the big bucks with COBOL are the dwindling band of grizzled old hands who have been maintaining those systems for most of their working lives and whose intuition about their gross and subtle potential failure modes is consequentially keen. It's their mastery of the Dark Arts that allows them to reap the compounded interest on fifty years of technical debt, not mere familiarity with the language that the grimoire was written in.
posted by flabdablet at 6:59 PM on February 8, 2019 [3 favorites]


is that because freelance COBOL programmers actually get paid more than Biglaw partners, or because it's that miserable

The latter.

Also, there's finding that your dog doesn't respect you the next morning...
posted by HiroProtagonist at 9:00 PM on February 10, 2019 [1 favorite]


It's their mastery of the Dark Arts that allows them to reap the compounded interest on fifty years of technical debt, not mere familiarity with the language that the grimoire was written in.

Well, damn, there goes my dream of making the big bucks without a lot of tradeoff since I already spend most of my time working in languages I really don’t like (*cough* Go *cough*). Only half kidding.
posted by invitapriore at 7:10 PM on February 11, 2019


« Older Why are millennials burned out?   |   FamilyTreeDNA is providing their database to the... Newer »


This thread has been archived and is closed to new comments