Primum Non Nocere
October 4, 2019 10:16 AM   Subscribe

In the past few years, we've seen the use of open source software turned to unethical ends, such as we saw recently with Chef and their contracting with ICE. In response, developer Coraline Ada Ehmke has created the Hippocratic License - a license that requires users to "first, do no harm".

The license has generated controversy, as the Open Source Initiative has said that they refuse to recognize it as an open source license on the grounds that such licences "must not discriminate against any person or group of persons" - which has led to arguments about the legitimacy of such positions in the face of harm.
posted by NoxAeternum (61 comments total) 23 users marked this as a favorite
 
Probably not enforceable, but enforcement isn't really the point, is it?

Meanwhile, colour me completely unsurprised that the libertarian techbros of the greater open source community are mad about this.
posted by tobascodagama at 10:22 AM on October 4, 2019 [24 favorites]


It's a great start, now let's do the rest of society.
posted by aspersioncast at 10:23 AM on October 4, 2019 [4 favorites]


Matthew Garrett takes a broader view: Do we need to rethink what free software is?
The definition of free software includes the assertion that it must be possible to use the software for any purpose. But if it is possible to use software in such a way that others lose their freedom to exercise those rights, is this really the standard we should be holding? Again, it's unsurprising that people will attempt to solve this problem through licensing, even if in doing so they no longer meet the current definition of free software.

I don't have solutions for these problems, and I don't know for sure that it's possible to solve them without causing more harm than good in the process. But in the absence of these issues being discussed within the free software community, we risk free software being splintered - on one side, with companies imposing increasingly draconian licensing terms in an attempt to prop up their business models, and on the other side, with people deciding that protecting people's freedom to life, liberty and the pursuit of happiness is more important than protecting their freedom to use software to deny those freedoms to others.
posted by zachlipton at 10:24 AM on October 4, 2019 [11 favorites]


Yeah, without checking I can safely assume there is already a "don't even think about it" email from our lawyers somewhere in my inbox.

Lots of open source software depends on corporate sponsorship, and no corporation that can afford IP lawyers is going to get behind a license that might lead to years of litigation over what exactly "harm" or "discrimination" means.

Probably not enforceable, but enforcement isn't really the point, is it?

Hohohoho. I'm imagining the Scanners-style reaction I would get if I brought this line of reasoning up at my next license audit meeting.
posted by sideshow at 10:26 AM on October 4, 2019 [15 favorites]


Eh, I remember arguments like this back around 2003, when people were surprised to discover how much open source software the military uses. There were a couple of isolated attempts to add license clauses prohibiting military usage to various software projects. All of them, as far as I know, ended up going nowhere and were eventually removed. It turns out that creating a license that's incompatible with existing OSS licenses basically makes your software unusable in the open source context, and neither GPL nor BSD communities (the two largest licenses, which have different philosophies) were open to modifying their licenses (and there's a good question of whether they even could, if they wanted to) to accept software with end-use limitations.

As a way to spur some sort of discussion? Sure, go for it I guess. But as an actual licensing strategy? I don't see it panning out.
posted by Kadin2048 at 10:27 AM on October 4, 2019 [5 favorites]


It turns out that creating a license that's incompatible with existing OSS licenses basically makes your software unusable in the open source context, and neither GPL nor BSD communities (the two largest licenses, which have different philosophies) were open to modifying their licenses (and there's a good question of whether they even could, if they wanted to) to accept software with end-use limitations.

So, was this because of incompatibility, or was it because, as Ehmke pointed out, that:
Open source has long been lorded over by an oligarchy posing as meritocracy, and people have been satisfied with crumbs that fall from their tables.
Because if it's the latter, then the response is not to say "oh well", but to inform those oligarchs that their time is over.
posted by NoxAeternum at 10:36 AM on October 4, 2019 [3 favorites]


Can't wait until religious software developers start demanding their software not be used to aid in abortion or gay marriage or decorating cakes in support thereof. And those who disagree then fork the software.
posted by zaixfeep at 10:39 AM on October 4, 2019 [8 favorites]


Because if it's the latter, then the response is not to say "oh well", but to inform those oligarchs that their time is over.

I mean, no one is going to prevent people from licensing their software (or just not licensing it, at all) however they want. Its just that not nearly enough people are going to use it, support it, fund it, etc, to make it sustainable.

Can't wait until religious software developers start demanding their software not be used to aid in abortion or gay marriage or decorating cakes in support thereof. And those who disagree then fork the software.

There will be no forking since the whole point of the these licenses are to prevent those who "discriminate" from using it, at all.
posted by sideshow at 10:43 AM on October 4, 2019 [2 favorites]


FOSS has always been somewhat of an oligarchy - the term BDFL has been there right from the start, hasn't it?
posted by theorique at 10:43 AM on October 4, 2019


> zaixfeep:
"Can't wait until religious software developers start demanding their software not be used to aid in abortion or gay marriage or decorating cakes in support thereof.

Yes! Came to say that. Thank god republicans can't code, amiright?
posted by kleinsteradikaleminderheit at 10:46 AM on October 4, 2019 [1 favorite]


The JSON license famously includes the sentence: "The Software shall be used for Good, not Evil." I believe the JSMin one had the same, and IBM requested it be removed, and it was amended to include the provision that IBM could do evil with it.

I would accept all computer science education requiring a class on ethics.

I would love for my profession to be filled with people who at least considered having an on-purpose ethos rather than just "move fast and break things" which was for a time a stated goal of Facebook. It's left as an exercise to the reader to determine the extent to which that is no longer their philosophy.
posted by artlung at 10:47 AM on October 4, 2019 [23 favorites]


If it ever were enforced, this would be an interesting way of trying to get US courts on record as to whether US government activities that are otherwise deemed lawful under domestic law violate the UDHR.

I am confused as to how this wouldn't be an "open source" license, since AFAICT it restricts the use of the software rather than access to the source code. Or in the license's terms, while UDHR violators cannot use the software, they would be free to "copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so." Am I missing something, or is "open source" just being used as shorthand for FOSS here?
posted by shenderson at 10:49 AM on October 4, 2019 [2 favorites]


It should be noted that - compatibility aside -using this licence would probably not have helped with the Chef situation. The developer's complaint wasn't that his software was being used for evil (it's not clear that it was ever used by or for ICE at all), it was that his software was being used by Chef, and Chef also provided services for ICE.
posted by Turbo-B at 10:49 AM on October 4, 2019


The definition of free software includes the assertion that it must be possible to use the software for any purpose.

After all, murder is only illegal when you had to pay for your murder weapon first.
posted by Pyrogenesis at 10:55 AM on October 4, 2019 [2 favorites]


it was amended to include the provision that IBM could do evil with it

I don't know if this is true, and the JSON license at JSON.org doesn't have the clause, but dammit, I want to believe.

Frankly, I'm disappointed that IBM wasn't specifically called out as a company that would want to do evil, and was (by implication) powerful enough to force other people to let it have its way.
posted by spacewrench at 10:56 AM on October 4, 2019 [4 favorites]


Can't wait until religious software developers start demanding their software not be used to aid in abortion or gay marriage or decorating cakes in support thereof.

The answer to Brendan Eich (as we saw) is not to turn a blind eye to his bigotry, but to refuse to tolerate it in the first place.

Its just that not nearly enough people are going to use it, support it, fund it, etc, to make it sustainable.

Says who? The response to the Chef debacle showed that there's a constituency for not allowing one's work to be used to commit unethical acts.
posted by NoxAeternum at 11:00 AM on October 4, 2019 [5 favorites]


Re the JSMin license: Apparently it was Google, not IBM? Guess they weren't kidding about getting rid of "Don't Be Evil."

FWIW, the Github version of JSMin has the "Good, not Evil" language, without any exceptions for IBM or others.
posted by shenderson at 11:04 AM on October 4, 2019 [4 favorites]


Says who? The response to the Chef debacle showed that there's a constituency for not allowing one's work to be used to commit unethical acts.

How much of that constituency is signing 6/7 figure support contracts?
posted by sideshow at 11:11 AM on October 4, 2019


A lot of the people asking this question right now, it seems like, are the single individuals and small groups working on dev tooling that gets heavily used by large corporations, evil and otherwise. They want those things to be available to others, but that doesn't necessarily mean all others, no matter what, and they aren't necessarily things that are going to be repackaged in a way where they're actually obligating anybody else to adopt the same licenses. I don't know about this particular license, but I think we're going to see a lot more people reconsidering who gets to profit off of their unpaid labor.

This is like the climate, really. It's not just that ICE is doing evil. It's that ICE is doing evil and that it doesn't have to pay the full cost of all the things it uses to do evil. The important thing is not necessarily the license itself, it's the willingness of people to donate their work and how that might impact the budgets of a lot of organizations doing things we'd rather they didn't. It should be more expensive to do things that hurt society. Not everybody is going to mind if their contributions are much less popular because of more restrictive licenses.
posted by Sequence at 11:13 AM on October 4, 2019 [8 favorites]


I'm unsure what the actual benefit of "approved open source" is for individuals.

Consumers of software care mostly about "free as in cost", and as a maker of software I can get just as much of an unpaid ego boost if I stick all my free work under whatever boutique license I want.

I don't get any benefit from a large company if they uptake and use it internally, and based on my time doing developer tooling I'm certain forking these things internally is much more common than actually contributing back to small tools.

Also, I'd rather pay for important niche software than get it for free. All of my niche free software breaks with time and all of my paid niche software stays updated...
posted by Anonymous Function at 11:18 AM on October 4, 2019 [2 favorites]


there is no eusocial potential available whatsoever in the production of free/open source software, except possibly under licenses that actively forbid all commercial use and that require all derivative software adhere to the same license. but only possibly even under those terms, since there are no practical legal remedies when wealthy organizations violate these licenses.

if you want to develop software for the social good, you must think in terms of achieving competitive advantage over capitalist organizations, rather than in terms of simply adding to the general supply of software.

the old free software guys, who wanted to run software production on quasi-academic lines, had their hearts sort of in the right place, but their naïvety was basically unparalleled. seizing the means of digital production is not an will never be a peaceful friendly open process.
posted by Reclusive Novelist Thomas Pynchon at 11:20 AM on October 4, 2019 [6 favorites]


Who decides what harm is? Is military use always going to be bad? (The internet evolved from a US military project...) Even ICE could transform under the next US president to something not evil.

I see these recurrent arguments as an example of engineer's disease. These licenses didn't just end up the way they are completely through accident - well at least not the GPL. I don't doubt they could be improved, but the principles were evolved and seriously considered in the 80s. RMS for all his flaws is not an idiot.

It's not the first time and it won't be the last time that someone complains about F/OSS being used nefariously. As soon as you start adding extra conditions, especially badly thought out ones like "do no harm" the benefits of open source and the freedoms of free software disappear. The ecosystem shatters with multiple incompatible licenses and then no one benefits.

By all means add your special condition to your own code - but you're severely limiting its reusability by doing so and you better be sure you're compliant with any other license you depend on. I stand by the 4 freedoms (although most of my code has recently ended up being MIT licensed.)
posted by zeripath at 11:24 AM on October 4, 2019 [4 favorites]


RMS for all his flaws is not an idiot.

No, he was just an abusive, misogynistic zealot, which colored his thought and views. It is not coincidence that the author of the Hippocratic License is a transgender woman, nor that this push for ethical accountability is taking place as groups who have historically been ignored are asserting their presence. Perhaps instead of saying that this was "all thought out" by a group of predominantly men who framed their view of "freedom" around their own privileged position, it's time to listen to the people who historically have been ignored.
posted by NoxAeternum at 11:33 AM on October 4, 2019 [22 favorites]


This particular license, even if you were to accept that the definition of open source should be expanded to use restrictions, is a legal nightmare. It basically wraps the UN's Universal Declaration of Human Rights and says "you can't violate any of these."

OK. There are 30 articles including things like "everyone has the right to form and to join trade unions for the protection of his interests" and "motherhood and childhood are entitled to special care and assistance" and "the family is the natural and fundamental group unit of society and is entitled to protection by society and the State."

This isn't merely "don't do evil" but try convincing any lawyer to accept a license that you might be violating if you're not providing "special care and assistance" to motherhood. And who's defining "family" here? Can a government still discriminate against gay marriage if they find "family" is only a man and a woman and their children?

The license is the wrong layer to try to take action on these issues. It's lazy, to boot. "If we just codify it in our licenses, then magic will happen and the world will be ethical." If you don't want ICE, then you have to actively speak up and participate and try to make sure your governemt doesn't do that. Licenses aren't going to solve this problem, but trying to use open source licensing to solve these problems is a really good way to cause endless bikeshedding and make open source toxic to most institutions.
posted by jzb at 11:37 AM on October 4, 2019 [15 favorites]


Who decides what harm is?

I think that might kind of be the point: If you're not sure that you're not doing harm, definitely don't use this software.
posted by axiom at 11:37 AM on October 4, 2019 [4 favorites]


There will be no forking since the whole point of the these licenses are to prevent those who "discriminate" from using it, at all

Fair enough. But if the source is open, it can be downloaded, studied and a FOSS (clean-room if necessary) workalike can be coded. LibreSSL vs. OpenSSL comes to mind, although that was for far different reasons.

If you want to control how your code is used, close your source and/or license it appropriately. But don't be surprised when your work is legally reproduced as unrestricted FOSS and your own work sits unused. I could be wrong, but I believe there are more worthwhile hills to die on.
posted by zaixfeep at 11:38 AM on October 4, 2019


Here is the IBM version.
posted by cowcowgrasstree at 11:40 AM on October 4, 2019 [2 favorites]


Weird, I wrote a piece 6 months ago about how software engineers should adopt Primum Non Nocere. Basically, I think there's no way to prevent malicious use of software, but the important thing for individual developers to understand is that their choices matter. We do not exist in black boxes, and if we know what we are doing contributes to a worse society (and we should find out if this is the case), we should do something else. It's really that simple, and I'm not sure there needs to be an external yardstick for what constitutes a moral good. The big jump is really from an attitude of "I'm a developer of code" to "I'm a developer of real things that have real consequences, so they need to align with what I believe to be good."
posted by swift at 11:53 AM on October 4, 2019 [9 favorites]


So this is a subject I've spent probably far too much time on. I would really like for someone (smarter than me) to come up with a license that both forbids evil use and still encourages the sort of independent collaboration that FOSS licenses encourage.

The thing about FOSS is that you can always tell pretty much immediately whether the license allows the thing you want to do with the software. This is important because software is a huge investment. Even if you're only an end user, you are trusting the data you've encoded with it, the time you've invested in learning it, and the business processes that depend on it to your ability to continue using that software.

And the software benefits from this as well, because you are much more willing to invest in its development if you know that you will be able to keep using it.

The problem with every attempt at an ethical license I've seen is that you lose this. How can I possibly know if something I do with the software is likely to harm someone? Or that someone who doesn't like the work I do with it won't make a convincing case that I am? If I grow cannabis, does that violate the license? Who knows? It's really up to the biases of a judge and frankly, I'm not going to risk my livelihood on that.

The nice thing about FOSS licenses--the thing that makes them work--is that they provides these guarantees.

An ethical FOSS license needs to preserve that property and none that I've seen so far have done that.
posted by suetanvil at 11:55 AM on October 4, 2019 [4 favorites]


I think that might kind of be the point: If you're not sure that you're not doing harm, definitely don't use this software.

In a world where many feel harmed by anything I might say or do and many others feel harmed by my lack of words or actions, I don't see how I can avoid harming somene by my very existence.
posted by zaixfeep at 11:58 AM on October 4, 2019 [5 favorites]


Google followed this literally at its inception. Then they discovered they were leaving a pile of money on the table by ignoring the goldmine of user data they were ingesting. And the second thing they did was capitalize on it.

But first, they did no wrong.
posted by eisenkrote at 11:58 AM on October 4, 2019 [1 favorite]


Also: I'm going to add that FOSS is already more ethical than the commercial software industry. Yes, ICE can use (e.g.) Postgres but ICE can also afford Oracle licenses and Oracle will happily sell them.

But anyone can use Postgress, regardless of race, gender identity, sexual orientation, social status, income level or anything else*. Yes, bad people use it to do bad things, but they already have Oracle available to them. FOSS is also available to good people doing good things.

And because it's FOSS, the people working on it aren't employees of one big commercial entity. The process tends to be much more democratic and likely to prioritize the needs of its users. Yes, it's far, far from perfect but it's better than all of the alternatives.

* Yes, I realize that there are still skill requirements which in turn are a level of privilege. But that's also true of (most) commercial software.
posted by suetanvil at 12:03 PM on October 4, 2019 [4 favorites]


Re the JSMin license: Apparently it was Google, not IBM? Guess they weren't kidding about getting rid of "Don't Be Evil."

If you look at the bottom of the post, you'll see that it was IBM that was previously granted an exception to the general license. That's not added to the license, because it's a separate license, granted to IBM.

Regarding google code, that gets to shenderson's earlier question:

Am I missing something, or is "open source" just being used as shorthand for FOSS here?

In pedantic usage, "Open Source" means "meets the definition of open source as defined by the Open Source Institute". Google Code required you to use an OSI-approved license in order to host your project there, which was why JSMin got booted.

People seem to be using "source available" to refer to non-OSI licenses that might otherwise be considered open source.
posted by vibratory manner of working at 12:07 PM on October 4, 2019 [5 favorites]


I guess I look at Free and RMS differently than some of the characterizations of their work; they moved the Overton window for the sociopolitics of software, because without them there would just be the hegemonic neoliberal discourse around software, IP, and technology.

It's a conflation to suppose that an idea must be broadly adopted in practice to be an important or successful idea. That's kind of the point of idealism. It's also a prejudice to demand perfect answers.
posted by polymodus at 12:16 PM on October 4, 2019 [3 favorites]


If you're not sure that you're not doing harm, definitely don't use this software.

Ok, but if I use an open source library to write GPS software that's used for fleet management by a cleaning company that does the maintenance work at an office that ICE rents in Nebraska, is that software being used to do harm?

What if I write software that's used in difribulators sold to ICE? You might not like the organization but they're used to save lives? What if they're sold to the CIA and it turns out they use them in conjunction with waterboarding as a form of torture?

What is and isn't harm isn't black and white and no one will touch software that could drag them into court over very subjective things.
posted by Candleman at 12:31 PM on October 4, 2019 [10 favorites]


It's a conflation to suppose that an idea must be broadly adopted in practice to be an important or successful idea.

Isn't what sparked this discussion the concept of wanting the OSI to accept use restrictive licenses as open source? It's not merely that folks want to use them, but they want to expand a commonly understood idea of a type of licensing to also include use restrictions. The proponents aren't content to use these licenses on their own, they want validation and acceptance from a wider group. There's where the real friction is at. Wanting these licenses to be broadly adopted or at least seen as adoptable instead of the outliers that they are.
posted by jzb at 12:39 PM on October 4, 2019 [1 favorite]


Google is awesome because Google Code was meant to be a longstanding community resource for hosting code and then they... shut lt down. Who cares if the Crockford license precluded something being on Google Code because Google Code does not function anymore. And thus we see that Software remains a tire fire.

Goal: be better than the current status quo, which is what the license posted about is about. It's nicely aspirational and I dig it even if the practical value is low.

Ultimately licenses are about enforcement, but I'd call a license like this "the court of public (developer) opinion." And if it gets people to think more deeply about licenses - I'll take a small win for Software culture.
posted by artlung at 12:42 PM on October 4, 2019 [2 favorites]


I feel like a blockquote of the actual license provision might be useful here:
The software may not be used by individuals, corporations, governments, or other groups for systems or activities that actively and knowingly endanger, harm, or otherwise threaten the physical, mental, economic, or general well-being of individuals or groups in violation of the United Nations Universal Declaration of Human Rights (https://www.un.org/en/universal-declaration-human-rights/).
There are some grammatical issues here that a cornered attorney could try to exploit. ("Systems and activities cannot know things, Your Honor, therefore this provision has no effect!") But the plain and ordinary meaning of the words is reasonably clear, and the drafter's choice to incorporate the UDHR rather than trying to somehow define wrongful conduct from first principles seems pretty well-thought-out.

The OSI's apparent view that actively and knowingly violating human rights constitutes a distinct "field of endeavor" that it is impermissible to discriminate against is ... interesting. Do they take a similar position against prohibitions on criminal use? (NASA-1.3 seems like it is, at best, skirting the boundaries of discrimination against the important field of endeavor of violating US export law.)
posted by shenderson at 12:49 PM on October 4, 2019 [5 favorites]


> I would accept all computer science education requiring a class on ethics.

I swear to fucking... something. Deep breath. Can we all please take a minute and resist the fashionable brand of bullshit that paints everybody who can code as morally defective? I'm not a fan of Facebook etc etc, but my experience, and yours if you're honest, is that people who write code are nevertheless able to read books, attempt to live a good life, make a positive impact on society. I'm seriously starting to think that people feel OK dehumanizing everybody in a tech-adjacent field because it makes them feel better about wasting $100k on their BA in applied semiotics.

And even if I'm wrong about the last bit, and even if silicon valley is full of hard-core, ayn rand-worshipping, dog-eat-dog, morally bankrupt, ignorant dudebros... We're talking about free, open source software here... the way you get to have an opinion, especially about anybody else's deficient ethics, is to donate a couple thousand hours of your life to create technology to benefit all mankind, without the guarantee of money or fame or even of your pull request being merged. Oh and you'd have to choose that over making low-to-mid six figures creating proprietary technology. Then we'll talk.

I hope everybody feels comfortable airing their opinions on a platform that is, I'm sure but didn't check, powered >90% by open source software. GRRR.
posted by kleinsteradikaleminderheit at 1:10 PM on October 4, 2019 [9 favorites]


Not to set off Yet Another Software Engineering vs. Other Engineering Cage Match, but since we're on the topic, the FE and PE exams contain ethics components, and ABET includes the following among the learning outcomes for accredited engineering programs:

An ability to recognize ethical and professional responsibilities in engineering situations and make informed judgments, which must consider the impact of engineering solutions in global, economic, environmental, and societal contexts.

Obviously none of this actually stops credentialed Professional Engineers from being just as evil as e.g. lawyers (who also have to go through various forms of ethics training), but it's not like engineering ethics is some kind of wild idea that only people with a, quote, "BA in applied semiotics" care about.

(Of course the kind of ethics that the Hippocratic License is concerned with is well beyond the scope of what usually passes for engineering ethics, so I think we may be straying from the point here.)
posted by shenderson at 1:24 PM on October 4, 2019 [5 favorites]


Can we all please take a minute and resist the fashionable brand of bullshit that paints everybody who can code as morally defective?

Many of the people who are making the comments about the lack of ethics in software development are members of the community themselves, who are speaking in response to the behavior that they see themselves in the community. And nobody is "dehumanizing" anyone by pointing out that the tech community has shown repeatedly some deep failings in ethical behavior.

As for trying to assert some sort of inherent "nobility" to open source, the community enabled the behavior of an abusive, misogynistic zealot for three decades, pushing out people (mainly women) from the tech community. Giving away one's labor is not inherently moral, and no group is above either scrutiny or reproach.
posted by NoxAeternum at 1:27 PM on October 4, 2019 [13 favorites]


It would be far more effective to get Microsoft or Apple to adopt this provision, both because of the wide distribution of their products, but even more because they have the resources to actually enforce license terms.
posted by Pyry at 1:30 PM on October 4, 2019 [3 favorites]


ABET includes the following among the learning outcomes for accredited engineering programs

In theory, all accredited engineering schools do require an ethics component somewhere in the curriculum. In practice, the best case scenario is you spend a semester learning about the horrible things that happen to whistleblowers, while the worst case scenario is that they do a seminar where lump you in with the biology students and your whole ethical education amounts to, "ask the IRB before you do anything with animals or humans".
posted by tobascodagama at 1:34 PM on October 4, 2019 [1 favorite]


Heated reactions aside, I don't think anyone here is dehumanizing or calling anyone soulless - just pointing out that educational curricula for fields that can have outsized impacts on the well-being of society and especially marginalized individuals, such as software/engineering/medicine/law/etc, would do well to have a stronger emphasis on ethics, and awareness of the impacts on society that work can have. The Iron Ring tradition, or something similar, being extended to engineers of non-physical systems might be a good start.
posted by NMcCoy at 1:35 PM on October 4, 2019 [4 favorites]


Oh, and that only applies for actual engineering degrees, whereas a lot of universities put their computer science department in a non-engineering college, where there would be no ethics requirement.
posted by tobascodagama at 1:37 PM on October 4, 2019


Certainly ethics education requirements for engineers do not seem to have appreciably reduced the number of prisons and bombs being built.
posted by Pyry at 1:42 PM on October 4, 2019 [4 favorites]


One of my college friends got a Computer Engineering degree, and he was able to be inducted into the Order of the Engineer, the US equivalent of the Iron Ring. So the OoE, at least, has no problem with considering non-physical forms of engineering.

The academic tradition of computer science has often considered itself separate from the field of engineering, however, to say nothing of the wider culture of software development. It was not uncommon to hear "software engineer" as a pejorative when I got my CS degree. (But the worst abuse was, of course, reserved for non-STEM majors, who obviously have nothing worthwhile to contribute to the world, unlike the STEM übermensch who were destined to become overnight millionaires and change the world to our whims.)
posted by tobascodagama at 1:42 PM on October 4, 2019 [3 favorites]


It violates my common sense to believe that forcing additional people to take an ethics course in college is going to have any effect on their behavior later in life. Is there any evidence for this? It sounds to me like kind of a "we have to do something, this is something" sort of solution.

The only related evidence I know of is a couple mediocre studies that seem to show that ethics professors behave very similar to other professors on a handful of morally relevant issues (e.g. they don't donate more to charity.) See https://sci-hub.se/https://doi.org/10.1080/09515089.2019.1587912 (and there is a previous similar one with similar results.)
posted by value of information at 1:58 PM on October 4, 2019 [3 favorites]


FWIW, I think it would be great if professional communities had more thoughtful social norms about not doing harm. I wish I knew a mechanism to accomplish that. I don't think encoding candidates into software licenses makes much sense except as a conversation starter.
posted by value of information at 2:03 PM on October 4, 2019 [5 favorites]


Certainly I believe mere ethics classes alone would do... nothing. At my university, "engineering students" as a group aren't going to pay any more attention to any module on ethics than they currently do to the sexual assault and academic honesty modules they're already made to complete. Those courses exist to cover the university, not to correct misbehaviour.

Social norms about not doing harm, that iron ring thing, it all seems like a start, but yeah, I agree that making it a part of vocational training is too little too late.

On the other side of things, I'm not a legal expert, my googling is showing arguments either way, but the UDHR irritates me. I know "discrimination on the basis of property" can go the other way as well, but to my layperson's eyes it looks like the UDHR enshrines private property in a way other similar declarations are more careful about. So that gives me pause about any ethics class as well - it seems quite likely to me that any ethics class that would get state approval is one I wouldn't want being taught.
posted by Acid Communist at 2:31 PM on October 4, 2019 [3 favorites]


As someone who's probably going to be working in CS their whole life, there's actually two different 'streams' of ethics that related to CS, of which programmers need to be taught both:

[1] Basic engineering ethics: This is the simple one. Making sure that your creation works properly and won't break in ways that are harmful to anyone. This is making sure that the bridge you built doesn't fall down, and it's fairly intuitive to understand why we need it.

Failures of type [1] ethics lead to things like the Boeing 737 Max disasters and the Volkswagen 'defeat device' incidents.

This is the type of ethics that most CS programs already teach.


[2] Social concerns and context: This is more complex. Not so much, "did you build it properly", but "should this have been built at all"? Just because you can make a thing, doesn't mean it should be made.

Failures of type [2] ethics give us things like the Cambridge Analytica scandal, ad tracking, and possibly everything in the Snowden docs.

Most CS ethics programs don't really cover this that well.
posted by Quackles at 3:39 PM on October 4, 2019 [10 favorites]


1: In any capitalist framework bigotry against people with less privilege always involves putting them at a financial disadvantage in order to curtail their ability to improve their circumstances or call those with privilege to account. The fight for social justice along the lines of race, gender identity, orientation or disability/neurotypicality *cannot* exclude socio-economic considerations and remain morally consistent. If your intersectionality doesn’t include class discrimination then your intersectionality is incomplete.

2: The original impetus for the broader open source movement (not just the OSI) was large corporations like IBM, Microsoft, and AT&T beginning to leverage intellectual property law with respect to software as another axis along which to consolidate power and create structural discrimination against those with less money and privilege.

3: Among individuals who objected to this behavior, two primary philosophies emerged: BSD/MIT/Apache which pursued a “this code is fairly basic so here is a clean version free for any to use for the rest of time,” and the GPL which took a more aggressive “this is an explicit viral attack on copyright being weaponized against those with less money/power,” later updated to include the emerging abuse of software patents.

Hopefully we can all agree on at least those three basic points, yes? Because it would be wonderful to see people move beyond debating historical motivations and focus on the “what now?” part.

As to that: just as intersectionality within a capitalist context cannot ignore class and financial discrimination, efforts to combat the serial abusers of the economically disadvantaged cannot ignore other forms of abuse: they cannot waive the remainder of social justice, or dismiss how badly software engineering as a discipline has historically failed those with less privilege on other axes.

What the ideal looks like is an open question. The BSD movement has always appealed to me more because it springs from the same ethos as Universal Basic Income or Universal Healthcare: nobody should have to sweat the obvious, basic stuff*. Society would be more just if “starve to death in a gutter” were removed from the list of possible outcomes, and that threat couldn’t be leveraged against the disadvantaged to coerce them into being exploited. Everybody should have access to the fundamentals, period.

Two things are true: 1) fighting against class/economic discrimination cannot be the whole of one’s commitment to social justice, and 2) insisting that software under a BSD or GPL-style license only help good people or ethical organizations is no less ridiculous than insisting M4A/UBI only applies to people who are not overtly *or* secretly racist. It doesn’t just place a literally impossible burden on developers and users who themselves are often disadvantaged on some axis, it also posits a counterfactual universe in which perfect knowledge, insight, and foresight exist (also any caveats will 100% definitely be abused by the shitheads when they’re in power, guaranteed). Given all that, I’m fairly comfortable saying I think this identifies the problem but I don’t think it is the solution.

Yes, absolutely more needs to be done and the broader open source movement desperately needs a sweeping cultural change. Yes, Stallman should’ve been railroaded out decades ago - it was already clear in the late 90s that he was a yet another dude with one really great idea (copyleft**) abusing the rewards and position that followed. No, this license isn’t the solution, and no, I don’t yet know what the solution is: at the moment I’m just trying to outline the problem as clearly as I can. Trying, as ever, to create a foundation upon which to build.

*Tangent: This is also, for me, the exact same impulse that leads me to posting three fairly basic points and asking if we can just consider them the floor and build from there.

**technically the term was coined by Li-Chen Wang of TinyBASIC, but Stallman was legitimately the first to treat it as more than a joke.

posted by Ryvar at 4:28 PM on October 4, 2019 [7 favorites]


As someone who's probably going to be working in CS their whole life, there's actually two different 'streams' of ethics that related to CS

This comment gets at a lot of what I say every time this stuff comes up. In my experience, one gets with a CS degree a bit of "engineering ethics" along the lines of "don't hurt people through negligence" - though this could still be more robust in CS. I don't really think anybody in engineering gets "don't take a job where you make things that are used to hurt people on purpose" because that has always been a thing engineers are for. I don't know that it would hurt to question this more often in the education of engineers - I had at least one professor who did and I am all for professors making an effort to do this - but I am somewhat cynical about the extent to which it is possible to sincerely make this part of the standard curriculum, given what I said about the history of engineering.
posted by atoxyl at 4:42 PM on October 4, 2019 [2 favorites]


My professor for Human Factors in Design was an old Cold War RAF test pilot who concluded each class with a story about how failures at the topic of that day’s lecture had killed one of his friends, usually horribly.

It was a hell of a class, and motivating as shit on the negligence front. What I want CS students to hear are similarly motivating accounts from women, from minorities and from LGBTQIA people about how bad decision-making wrt the broader ethical decisions in software engineering have impacted them. That’s not an impossible ask, and the people in a position to make it happen (academic heads) are closer to being on-sides than most groups with authority.
posted by Ryvar at 4:52 PM on October 4, 2019 [3 favorites]


OK so computer science, as used in this thread, seems to be entirely a kind of engineering. Which is fine, let's ignore the computer scientists who are actual scientists (like me these days), and also those who are basically mathematicians (like a bunch of my friends). Computer science, like engineering in general, is the discipline of finding, codifying, teaching, and formalizing the way we create and use some brand of technology. Yes?

Ryvar, I like your analysis, but I can't help but notice that many of the largest engineering efforts in history were part of an experiment in totalitarian socialism, not capitalism... and engineering and applied math were A) pretty much the only intellectual endeavor encouraged in the soviet union, and B) more or less exclusively aimed at technologies of war. I'm far from defending capitalism here, but it just doesn't seem that markets are at the core of this. Rather, it seems to me that there is simply no way to create technology that cannot be used for evil.
posted by kleinsteradikaleminderheit at 5:38 PM on October 4, 2019 [4 favorites]


Isn't what sparked this discussion the concept of wanting the OSI to accept use restrictive licenses as open source?

Yeah, so in line with several of the capitalism comments, my response would be that what's missing is a questioning of what the concept of restriction means. Under a capitalist governmentality, we get ideological inversions; what's accepted as free is really restrictive, and what's restrictive can be a path to freedom. Also under this argument, friction is because capitalism is its own collective action that resists efforts toward emancipation.

Now to emphasize I'm not saying I know whether this is true or not. But this is the kind argument I would explore, from a leftist perspective, if tech people are trying to use words like "freedom" and so forth.
posted by polymodus at 11:54 PM on October 4, 2019 [2 favorites]


OK so computer science, as used in this thread, seems to be entirely a kind of engineering. Which is fine, let's ignore the computer scientists who are actual scientists (like me these days), and also those who are basically mathematicians (like a bunch of my friends). Computer science, like engineering in general, is the discipline of finding, codifying, teaching, and formalizing the way we create and use some brand of technology. Yes?

I mean, the theory CS types have the enviable advantage of not having to worry as much about if their work is used for ill.
posted by Quackles at 3:32 AM on October 5, 2019 [1 favorite]


The UDHR says:
"Everyone has the right to a standard of living adequate for the health and well-being of himself and of his family, including food, clothing, housing and medical care and necessary social services"

Does this mean that, for example, health insurance companies that lobby against universal healthcare or medicare for all would be prohibited from using software under this license?
posted by Pyry at 3:43 AM on October 5, 2019 [1 favorite]


> I'm seriously starting to think that people feel OK dehumanizing everybody in a tech-adjacent field because it makes them feel better about wasting $100k on their BA in applied semiotics.

Jeez, what is computer programming if not applied semiotics?
posted by smelendez at 8:16 AM on October 5, 2019 [6 favorites]


> Jeez, what is computer programming if not applied semiotics?

Ha! That's fair.

> theory CS types have the enviable advantage of not having to worry as much about if their work is used for ill.

You'd be surprised... a lot of them end up in operations research, which is basically weaponized logistics.
posted by kleinsteradikaleminderheit at 1:51 PM on October 5, 2019


Following up on jzb's quote of UDHR Art. 23(4) above ("everyone has the right to form and to join trade unions for the protection of his [sic] interests") ... Can the Hippocratic License be usefully regarded as a possible approach to the problem of how to form an effective digital picket line?

Obviously a license with that as its primary purpose would be structured somewhat differently. But given the lack of working solutions to this problem currently, it seems like something like this might be worth further consideration.
posted by shenderson at 4:38 PM on October 5, 2019


« Older Freedom From Toil   |   fluttering unseen through shadows and hiding... Newer »


This thread has been archived and is closed to new comments