.
February 16, 2016 8:46 AM   Subscribe

The NSA’s SKYNET program may be killing thousands of innocent people "In 2014, the former director of both the CIA and NSA proclaimed that "we kill people based on metadata." Now, a new examination of previously published Snowden documents suggests that many of those people may have been innocent."
posted by cjorgensen (49 comments total) 35 users marked this as a favorite
 
Edward Snowden has proved himself an American Hero time and time again with the release of NSA documents. He deserves a complete pardon for anything illegal he did in the process. There is no way we would be hearing about this sort of thing without him. I don't know if the program is as flawed as this reporting claims, but without Snowden to shine a light on it we wouldn't even have had a chance as a public to shine a light on this and demand the government investigates the concerns.
posted by Drinky Die at 8:50 AM on February 16, 2016 [42 favorites]


I get why they use the framing they do, and I see that they comment on the framing late in the article. But I sure wish we lived in a country where folks didn't believe that the only reason not to kill someone out of the blue, without trial and at considerable risk to the people around them was that they were formally innocent. I think it's pretty reasonable to believe that even being a "terrorist" shouldn't rob you of all right to due process.

Here's another thing I hate about America, by the way: We justify what we do on some kind of ideas about foundational ethics - the American way of life is right and good in itself, so when people attack it they are attacking the good guys, the people who believe in jury trials and democracy, etc. But at the same time we feel totally comfortable saying "these great profound ethical truths stop at our borders". Either a thing is universally true or it isn't - this handy-pandy sort of thing is embarrassing at best.

Algorithms increasingly rule our lives. It's a small step from applying SKYNET logic to look for "terrorists" in Pakistan to applying the same logic domestically to look for "drug dealers" or "protesters" or just people who disagree with the state. Killing people "based on metadata," as Hayden said, is easy to ignore when it happens far away in a foreign land. But what happens when SKYNET gets turned on us—assuming it hasn't been already?

Lately I've been thinking about how much of the early USSR has come back in a de-communized form - from ideas that were utopian in the USSR (super mass production, tiny housing) to ideas that were horrible. But now we don't get free healthcare and schools, even. If we're going to have the gulag and the random executions and and constant surveillance so on, I would like sexpol and revolutions in byt and lifetime healthcare and a worldclass research program as well, please.
posted by Frowner at 8:54 AM on February 16, 2016 [49 favorites]


Did we expect anything less from Skynet? The fact that Obama has been complicit in this sickens me, and it's one of the reasons that I won't vote for Hillary.
I couldn't help wondering, as I was reading, how many pakistani pizza delivery persons have been blown up by drones.
posted by OHenryPacey at 9:04 AM on February 16, 2016 [3 favorites]


Base rate fallacy in action.
posted by procrastination at 9:08 AM on February 16, 2016 [7 favorites]


We justify what we do on some kind of ideas about foundational ethics

Yeah, the hypocrisy is worse than the acts. (I mean, no matter what's done in the name of the American people, layering sanctimony on top of it makes it worse.)
posted by spacewrench at 9:09 AM on February 16, 2016 [1 favorite]


what happens when SKYNET gets turned on us—assuming it hasn't been already?

no assumption
posted by j_curiouser at 9:10 AM on February 16, 2016 [3 favorites]


the fact that the NSA named their algorithm for 'justifying' the murder of innocent civilians with barely any oversight SKYNET makes me think that someone there has a sense of humor that's so dark that it borders on sociopathy

and considering this is an implemented program with funding and mandate, I suppose it's actually quite a lot of people there who find profound dehumanization pretty amusing
posted by runt at 9:24 AM on February 16, 2016 [12 favorites]


I think it's pretty reasonable to believe that even being a "terrorist" shouldn't rob you of all right to due process.

Yes. Even if this were done perfectly--that is, we could scan cell phone data and with 100% accuracy determine who is a "terrorist" by some reasonable definition--I'm still not comfortable with imposing a death penalty from half a world away with no trial or due process. That's not the world I want to live in. There has to be some alternative that protects US security without trampling over human rights.
posted by Pater Aletheias at 9:26 AM on February 16, 2016 [27 favorites]


who could have ever predicted a program with a name like skynet might not be the best idea
posted by entropicamericana at 9:28 AM on February 16, 2016 [10 favorites]


There has to be some alternative that protects US security without trampling over human rights

Assuming that this program actually protects US security rather than creating a whole generation of people in that region who really do want to harm the US.
posted by Ickster at 9:29 AM on February 16, 2016 [13 favorites]


ooh or maybe this kind of algorithmic justification for extralegal murder was going to be implemented anyway so whoever was directing it subversively decided to name it after something that nerdo tech journalists would swarm over

that's the slightly less terrible world that I'd be marginally happier with living in
posted by runt at 9:35 AM on February 16, 2016 [3 favorites]


THIS type of shit is the reason why I hate that ongoing drone warfare has reached bipartisan consensus, why it drives me insane to hear Democrats justifying drone strikes by saying things like "it's better than boots on the ground!" or "would you rather send in soldiers and risk their lives instead?"

From the article: "Somewhere between 2,500 and 4,000 people have been killed by drone strikes in Pakistan since 2004"

The problem is that drones are so cheap, both in monetary and (American) human costs, that we're using them to kill way more people than we would have without them. It's not like, if drones hadn't been invented, we'd scrambling a fucking carrier air wing thousands of times to make sure we hit every single one of those people we killed with drones instead. We wouldn't be packing Seal Team Six in a stealth Black Hawk flying unauthorized across the borders of sovereign nations to go take each of those same 2-4,000 people down. We wouldn't be engaged in land wars in Yemen and Somalia and Pakistan and who knows how many other countries on top of everywhere else we're already tangled up.

So no, I don't want risk more American lives, but I also sure as shit don't like the fact that drones cost the military so little money and the nation so few soldiers' lives that we fucking go around executing literally thousands of foreigners (and anybody unfortunate enough to be near them at the time) on the basis of CELL PHONE METADATA fed into a network that was mistrained in a fucking fundamental, Machine Learning 101-level bungle.

I also wish at least one of the Democratic presidential candidates would get on my level, but ah well
posted by cobra_high_tigers at 10:09 AM on February 16, 2016 [31 favorites]


I think it's pretty reasonable to believe that even being a "terrorist" shouldn't rob you of all right to due process...We justify what we do on some kind of ideas about foundational ethics - the American way of life is right and good in itself, so when people attack it they are attacking the good guys, the people who believe in jury trials and democracy, etc. But at the same time we feel totally comfortable saying "these great profound ethical truths stop at our borders". Either a thing is universally true or it isn't - this handy-pandy sort of thing is embarrassing at best.

Well, in a way this goes back to the questions that began after 9/11 with respect to Guantanamo and "enemy combatants" -- is international terrorism a law enforcement problem or a military problem?

If it's a military problem, due process basically boils down to "did we follow our process for mitigating civilian casualties, however opaque or risk-tolerant that process may be?" If it's a law enforcement problem, there are a lot of things that the U.S. would not be able to do that I think, on balance, are probably better done than not, even if something like SKYNET is not the way to go.

And it's not really about the United States. Every country in the world approaches law enforcement threats differently from military ones, because of a basic assumption that some (usually extra-territorial) can't be dissuaded through legal means. The problem is that with current technology it's possible to perpetuate a war basically indefinitely with little apparent cost to your public, so it throws the entire calculus off.
posted by AndrewInDC at 10:12 AM on February 16, 2016 [6 favorites]


I'm more disturbed by the precedent they have established than the sloppiness of the implementation (which is also very disturbing).

It will be very difficult for the United States to object if there are ever any international extra-judicial assassinations on U.S. soil and given that the tech is getting cheaper and cheaper the precedent provides a defacto moral cover for all kinds of inevitable future violations of American sovereignty.
posted by srboisvert at 10:16 AM on February 16, 2016 [12 favorites]


is international terrorism a law enforcement problem or a military problem?

It's a law enforcement problem. It's people engaged in conspiracies to perpetrate violence that is against the law. They aren't comic book villains. And the things that treating it as a military problem allows us to do that we would not be able to do if we treated it as a law enforcement problem (e.g. invading other countries, indiscriminate drone warfare, extraordinary rendition, running extraterritorial gulags where we get to torture people without charges) are just about the best recruiting material possible for creating more of those terrorists rather than actually solving any problem.
posted by cobra_high_tigers at 10:20 AM on February 16, 2016 [7 favorites]


I'm still thinking drones, that wrap rope around people, put them to sleep, and then fly them to a place they can stand trial. Yes? yes? Ok so when I learn how to invent things I will make it for you.
posted by xarnop at 10:24 AM on February 16, 2016 [2 favorites]


Maybe make a giant drone bubble that eats people.
posted by Drinky Die at 10:28 AM on February 16, 2016 [7 favorites]


And the things that treating it as a military problem allows us to do that we would not be able to do if we treated it as a law enforcement problem (e.g. invading other countries, indiscriminate drone warfare, extraordinary rendition, running extraterritorial gulags where we get to torture people without charges) are just about the best recruiting material possible for creating more of those terrorists rather than actually solving any problem.

The problem is that there are far more mundane limits that law enforcement would run into. What about extradition from failed states? Forget ignoring the lack of an extradition treaty, what if there is literally no legal authority to make your extradition pea to?

I don't mean to trivialize the idiocy and self-sabotage that the "War on Terror" has caused. But I think the reality is that there is a continuum of civilian and military force that needs to be applied, and things like SKYNET are an example of where that can run off the rails due to technological advances and the lack of deep thinking in the government about what precedents might be set, as srboisvert noted.
posted by AndrewInDC at 10:33 AM on February 16, 2016 [2 favorites]


I've found that nations that don't militarily involve themselves all over the world to begin with actually run into much fewer of these problems with random people in failed states on the other side of the world wanting to murder them. Instead of debating the best way to stop the ones that exist, we should focus on not making any more of them. The best military solution to terrorism is to be less militaristic in general.
posted by Drinky Die at 10:38 AM on February 16, 2016 [11 favorites]


Well, I'm all for that as a long-term strategy. My point was basically the same as the one cobra_high_tigers' made immediately above my first post, about the destabilizing effect of technologies like drones.
posted by AndrewInDC at 10:46 AM on February 16, 2016 [1 favorite]


The problem is that there are far more mundane limits that law enforcement would run into. What about extradition from failed states? Forget ignoring the lack of an extradition treaty, what if there is literally no legal authority to make your extradition pea to?

I mean, I don't necessarily think that's a problem? Or, at least, I think almost any solution to it other than "meh, let's just throw some missiles their way, just in case" would be better.

I could imagine some kind of UN-brokered treaty that codifies existing extradition agreements and defines some kind of process that can be followed when a signatory wants to "extradite" someone from a place that does not have an extradition framework in place. There are soooooo many problems with that, but less problems than there are with sending drones out to murder everybody within 100 yards of somewhere that somebody we don't like might be.
posted by tobascodagama at 10:55 AM on February 16, 2016 [3 favorites]


It will be very difficult for the United States to object if there are ever any international extra-judicial assassinations on U.S. soil and given that the tech is getting cheaper and cheaper the precedent provides a defacto moral cover for all kinds of inevitable future violations of American sovereignty.

I don't think the moral argument makes a damn bit of difference. No political or military leader on Earth is ever going to say, "Well, you may be killing my people, but we've done the same, so I guess we can't complain." If sovereignty doesn't work as an argument, they'll come up with something else ("existential threat," for example). Either way, the justification will boil down to this: "We're the good guys, so anyone who bombs us is a bad guy, and anyone we bomb is by definition a bad guy too." It may be hypocritical, but when has hypocrisy ever stopped a powerful state from doing whatever it wants to do?
posted by Gerald Bostock at 11:07 AM on February 16, 2016 [5 favorites]


Maybe make a giant drone bubble that eats people.

[INCOHERENT HOWLING]
posted by the man of twists and turns at 11:34 AM on February 16, 2016 [3 favorites]


It may be hypocritical, but when has hypocrisy ever stopped a powerful state from doing whatever it wants to do?

It does if you want to have allies.
posted by srboisvert at 11:43 AM on February 16, 2016


The similarities between the NSA's SKYNET and the plot of Person of Interest (FanFare) are really quite something. Person of Interest also had a plot in the first season concerning an NSA whistleblower which was a clear parody of Edward Snowden... except that the episode aired over a year before Snowden went rogue.
posted by simonw at 11:44 AM on February 16, 2016 [1 favorite]


The leaked NSA slide decks offer strong evidence that thousands of innocent people are being labelled as terrorists; what happens after that, we don't know.

So, it's possible that the NSA actually looks at these people in more detail before they tell the CIA to kill them. It seems unlikely that people are being killed based on SKYNET alone, if only because Ahmad Zaidan hasn't been droned. It's kind of a leap to go from "thousands of people are falsely labelled terrorists" to "thousands of innocent people have been killed after being falsely labelled terrorists."

The whole thing is still hugely problematic- there's nothing quite like bad machine learning to enable baseless decisions...
posted by BungaDunga at 12:15 PM on February 16, 2016 [4 favorites]


One of the things I was thinking when I read this is something I think about whenever the idea of innocents being killed by a drone comes up. If a drone killed one of my family members and I wasn't a terrorist, well, I am now. I would want revenge so terribly I would want those responsible held accountable, but that's not possible. So what recourse do I have available? I'm probably going to dedicate my life to killing as many citizens from that country as I can figure out how to manage, because while those people might not be directly responsible, they are surely culpable. Where is the remorse for innocent lives lost? Where are the reparations? How do you appeal the justification for a drone strike or prove you're not guilty? How can you hold someone accountable for mistake? You can't bomb weddings and funerals and clinics without making a few enemies out of the survivors.

Then I wonder the biographies of recent terrorists and wonder if any are in reaction to a past harm dealt from above.
posted by cjorgensen at 12:26 PM on February 16, 2016 [1 favorite]


Oh my God. Are they seriously using this "Skynet" to find targets? I really hope it's just an experiment that never led to actionable intelligence. But if it did, and people died because they were flagged by a computer program, someone needs to go to jail. Not that they ever will, but they should go to jail. Can you imagine the response if some other nation like Russia or China was surveying US cell phone users, and blowing up the ones who scored above a certain threshold as terrorists? It would certainly be an act of war.
posted by Kevin Street at 12:59 PM on February 16, 2016 [2 favorites]


It's pretty much settled law that suspected terrorists in foreign lands are fair game, even if they are American citizens. They can be blown up like any other military target in the discretion of the chain of command, based on whatever evidence command finds persuasive. Collateral damage is judgment call.

The interesting cases have dealt with captives. The courts have held that once the US captures someone, some form of due process and rights applies, be it POW, unlawful combatant, or criminal suspect. "You stay in a box until the President says otherwise" is in theory not right, although as can be seen in Gitmo, executive discretion is still quite broad.
posted by MattD at 1:01 PM on February 16, 2016 [1 favorite]


Yes, it's "settled law" which relies on secret memos which redefine the word "imminent" to mean "not necessarily the immediate future" and "feasible" to "inconvenient".
posted by RobotVoodooPower at 1:18 PM on February 16, 2016 [7 favorites]


(Oh yeah, and that memo also references an Israeli court ruling for it's justification. I mean why should we spend all this time making law in this country when there's plenty of perfectly good precendents around the world, if you look hard enough?)
posted by RobotVoodooPower at 1:25 PM on February 16, 2016 [2 favorites]


This can't be real. Combining big data with assassination is plausible, but I refuse to believe that anybody is stupid enough to call a learning algorithm SKYNET.
posted by yeolcoatl at 1:44 PM on February 16, 2016


The memo is interpreting international law, so it necessarily looks to foreign law and practice.
posted by jpe at 1:46 PM on February 16, 2016


Ok, but like, is a country that routinely laughs at the UN because their big brother is on the security council really the best precedent?
posted by tobascodagama at 2:06 PM on February 16, 2016


They cited a lot more than the Israeli courts.
posted by jpe at 2:10 PM on February 16, 2016


Settled law doesn't mean "I agree with it." It means that there is bipartisan executive and judicial consensus in support of it -- not likely to be overruled, sooner or later.

The President's prerogative to blow up American terrorists abroad is, for example, much better settled law than Roe vs. Wade, which a Republican President and a 50-50 US Senate could see overturned in the matter of a few years, given Ruth Bader Ginsberg's age.
posted by MattD at 2:34 PM on February 16, 2016


Look at it from the other side for a minute. As far as they are concerned, they are right and we are wrong.
How would we feel if they sent missiles across to pick out General Arseole and took out a School or a hospital as well, because the information was "whoops I'm sorry we didn't know" that was there.
We just might have accepted the General as a valid act of war (though unlikely), but there would be all hell to play about the school and we would be calling for international condemnation and all sorts of reprisals.
posted by Burn_IT at 2:36 PM on February 16, 2016 [2 favorites]


Fuck, we really are living in a dystopian nightmare, aren't we?

Secret government programs to spy on everyone and to blow up thousands of people because a computer algorithm says we should. Apparently we've decided we get to do whatever it takes to prevent even the possibility of another terrorist attack.

Unless, of course, said terrorist attack is a mass shooting by a US citizen. Because, you know, those happen all the time. But whatever, it's our God Given Right to own a gun, so let's shoot first and ask questions later. As long as you're white, of course. If you're a police officer, even better! But if you're black, make sure you don't ever do anything that might in any way make it look like you have a gun or like you might pose a threat to someone who has a gun. Yes, even if you're a 12 year old boy hanging out on a swing set.

Oh, and let's just completely ignore the natural disasters that keep killing people and destroying homes and lives, because, you know, that's just a thing that happens now. Nothing to be done about that!

Just think what could happen if we redirected all this time, money, and manpower that we keep funneling into "The War on Terror" (not to mention "The War on Drugs") and instead used it to try to find away to do something about global warming before it destroys us all.
posted by litera scripta manet at 5:00 PM on February 16, 2016 [5 favorites]


It's kind of a leap to go from "thousands of people are falsely labelled terrorists" to "thousands of innocent people have been killed after being falsely labelled terrorists."

This. I'm all for calling out the NSA's bullshit, but I dislike the headline. I'm having trouble finding anything indicating SKYNET is directly linked to drone strikes.

Mass surveillance is sin enough. No need for clickbait headlines.
posted by Room 101 at 6:19 PM on February 16, 2016


It's not like we can tell which category the victims fall into. The US if it is holding trials at all aren't making the proceedings public even if you trust the US definition of Terrorist.
posted by Mitheral at 6:32 PM on February 16, 2016 [2 favorites]


the 'imminent' bullshit really ushered in the realization of 'newspeak'. fuckers.
posted by j_curiouser at 7:20 PM on February 16, 2016 [3 favorites]


A friend of mine was talking about a new program she'd read about that does data analysis to pre-target potential mass shooters. She admitted it was a little creepy/thoughtcrime-y, but shrugged it off with "but I like not dying in a mass shooting, so it's okay."

When I said you could use the same kind of targeting on black youths or protestors or Muslims, well, then it would be horribly racist and wrong and shouldn't be done.

But that's the real problem. When you put a stick in the hands of authority, they don't stick to using it on people you dislike, which is why I'm so leery of the rush to criminalize everything and track everything just for the sake of feeling safe.
posted by Ghostride The Whip at 2:20 AM on February 17, 2016 [1 favorite]


It will be very difficult for the United States to object if there are ever any international extra-judicial assassinations on U.S. soil and given that the tech is getting cheaper and cheaper the precedent provides a defacto moral cover for all kinds of inevitable future violations of American sovereignty.

The US is currently in exactly that position with respect to cyberwar. It's very plausible to imagine a future scenario where there's no moral high ground or political credibility from which to object, but concerning different security issues.

This can't be real. Combining big data with assassination is plausible, but I refuse to believe that anybody is stupid enough to call a learning algorithm SKYNET.

The NSA has a history of institutional silliness (remember they were at one point the US' biggest employer of mathematicians.) To wit: (search "parking"): this

Also, I'd be highly surprised if there wasn't rigorous further analysis after a target got tagged by the algorithm.
posted by iffthen at 3:26 AM on February 17, 2016


I'm having trouble finding anything indicating SKYNET is directly linked to drone strikes.

Not to ask you to prove it's not, because I won't ask you to prove a negative, but that fact that no one knows the criteria being used is troubling in of itself. No way to prove SKYNET isn't directly linked to drone strikes. The fact that it exists would imply to me that it's being used, but your point is taken.
posted by cjorgensen at 9:13 AM on February 17, 2016 [1 favorite]


. thousands

If this happened in the USA, however improbable that is, the public reaction would be rather.. massive. A foreign extralegal killing campaign guided by programs with dubious algorithms, executed by drones that were reported to have had 10 times (!) more civilian casualty than conventional aircraft warfare, and a track record of killing the non-targets nine times out of ten (!), conducted in the most secretive and -- arguably -- corrupt way possible? It would be unthinkable. Except in this case, the ones that were killed were brown so presumably that's o.k.

Thousands killed; most, according to the article, probably innocent (EKIA or enemy killed in action -- according to the article's source, EKIA was most often used for whomever that were killed alongside the real targets. Often they're children and women).

It's utterly crazy how the people who run this campaign could think that this is in any way, shape or form moral or justifiable. I mean, if you think about it, they killed people based on thoughtcrime. People who haven't been caught doing terrorist actions, haven't got a chance for trial and maybe aren't really actual qaidans. That's not even counting the civilian deaths.

For anyone who want to read the original source, here's the website (created by Glenn Greenwald) that chronicled the program and its (shoddy, murderous) details.
posted by tirta-yana at 10:04 AM on February 17, 2016 [3 favorites]


MeFi comments don't usually include direct quotes, but I think these stellar collection of choice passages from the article merited at least a comment (copied directly from Ars' comment page) :

One.
'“There’s always the off-chance that you could end up targeting somebody, who is totally innocent. Of course, that’s usually not counted. That’s usually confirmed as an enemy killed. And, if the basis for a strike involves human intelligence, the informant is oftentimes lying to get money.”

Two.
“Anyone caught in the vicinity is guilty by association,” the source said. When “a drone strike kills more than one person, there is no guarantee that those persons deserved their fate. … So it’s a phenomenal gamble.”

Three.
'Michael Haas, who was in the 15th Reconnaissance Squadron at Creech Air Force Base between 2005 and 2011, worked as an instructor who trained pilots for drone missions.... Haas also was reprimanded by his superiors for failing a student who insisted individuals were “up to no good” and kept choosing to fire missiles when there was no intelligence to support strikes.'

Four.
“We kill four and create ten [terrorists]. Is that really what we’re trying to achieve? If you kill someone’s father or uncle or family member and they’re not part of the problem, then all of sudden these people want revenge,”

Five.
'The whistleblowers spoke about how their teams dehumanized the people they were killing. Haas mentioned some men would refer to children as “fun-sized terrorists.” They would call individuals “tits,” or terrorists in training. They would refer to what they were doing as “cutting the grass before it grows too tall” or “pulling the weeds before they take over the lawn.”'

Six.
“You can’t buy me. You can’t buy my soul. You can’t buy my conscience. I’m sick of the fact that we excuse our actions, we can’t do that anymore,”

And from the other article,
Seven.
"In the complex world of remote killing in remote locations, labeling the dead as “enemies” until proven otherwise is commonplace, said an intelligence community source with experience working on high-value targeting missions in Afghanistan, who provided the documents on the Haymaker campaign. The process often depends on assumptions or best guesses in provinces like Kunar or Nuristan, the source said, particularly if the dead include “military-age males,” or MAMs, in military parlance. “If there is no evidence that proves a person killed in a strike was either not a MAM, or was a MAM but not an unlawful enemy combatant, then there is no question,” he said. “They label them EKIA.” In the case of airstrikes in a campaign like Haymaker, the source added, missiles could be fired from a variety of aircraft. “But nine times out of 10 it’s a drone strike.”

From other sources, the EKIAs were also used to calculate future target for the killings. So the civilian who got killed for being near the targets, from day one, were also being used to search for more probable targets, i.e what do you think will happen when a killing program counted civilian casualties as terrorists and make future predictions based on that?
posted by tirta-yana at 10:25 AM on February 17, 2016 [4 favorites]








« Older Espionage Techniques of Seventeenth-Century Women   |   Love, Naturally* Newer »


This thread has been archived and is closed to new comments