Surveillance Capit^H^H^H^H Curriculum

June 6, 2019 10:02 PM

[...]A year later, though, Securly also began offering "sentiment analysis" of students' social media posts, looking for signs they might be victims of cyberbullying or self-harm. In 2016, the company expanded that analysis to students' school email accounts, monitoring all messages sent over district networks. It also created an "emotionally intelligent" app that sends parents weekly reports and automated push notifications detailing their children’s internet searches and browsing histories[...]

via Zeynep Tufekci on twitter
posted by postcommunism (33 comments total)

And here I thought Channel One was too much of an intrusion in my school...

We are breaking this cohort of kids, and for what?
posted by fifteen schnitzengruben is my limit at 10:52 PM on June 6, 2019 [9 favorites]

Holy shit, that was depressing. But thanks very much for the post. I had no idea.
posted by It's Raining Florence Henderson at 11:01 PM on June 6, 2019 [2 favorites]

Every time I start to get a little depressed about my plans to become a teacher not working out, I read an article like this and feel a lot better about it. I don't think I could cope in this climate.
posted by The Underpants Monster at 11:16 PM on June 6, 2019 [3 favorites]

Jesus Christ. Are schools actually doing anything to help students in need? If students are reduced to writing swear words in Google Docs to trip up the sensors to get the teachers' attention, probably not.

And sending local police to do wellness checks when there's already plenty of incidents of police brutality against minorities just for existing? I'm surprised there aren't more deaths from overreacting cops.
posted by divabat at 11:28 PM on June 6, 2019 [17 favorites]

Jesus Christ. Are schools actually doing anything to help students in need? If students are reduced to writing swear words in Google Docs to trip up the sensors to get the teachers' attention, probably not.

Not for nothing, but, circa 1985, I literally waited for 2+ hours in a vice principals office, dippping wet in an underheated northern MN school, for the vice principal to tell me that "maybe if I was nicer" to my assailants, and"tried harder to fit in" the swirlie I was there to complain about would not have happened.

The administration is interested in doing the least it can while achieving acceptably minimal results. It has always been thus.
posted by Pogo_Fuzzybutt at 12:32 AM on June 7, 2019 [31 favorites]

Fucking Christ, they are using school shootings to justify this bullshit. Just fucking restrict guns you assholes!!!
posted by emjaybee at 12:57 AM on June 7, 2019 [30 favorites]

Endorsing emjaybee's reaction. This seems to be a characteristic US style. You have a problem, but solving it in a rational, direct and effective way would infringe on a non-negotiable interest of some people who have influence. Rather than ban guns, let's institute a paranoid panopticon bubbling with false positives! Rather than rely on a standard of care to treat disease in the population, let's subordinate delivery of medical services to insurance companies' wish to make money! Rather than help immigrants settle and contribute to the economy here and in their home countries, let's devote endless resources to harassing and abusing them! What waste, what incompetence, and what vast openings for cruelty.
posted by homerica at 1:22 AM on June 7, 2019 [53 favorites]

> Fucking Christ, they are using school shootings to justify this bullshit. Just fucking restrict guns you assholes!!!

That would risk actually fixing the problem, and then they'd lose their pretext for doing these things. Can't have that!
posted by I-Write-Essays at 2:46 AM on June 7, 2019 [6 favorites]

A Martian anthropologist visiting the US would conclude that firearms are sacred in its society, much in the way that cows are in India or elf-inhabited rocks in Iceland*, and that questioning their divinity is one of the culture's greatest taboos. Which is why America bends over backwards to do anything other than encroach on the sacred position of the gun, including PTSD-inducing active-shooter drills in schools and draconian surveillance regimes.

* not actually, though in the popular myth
posted by acb at 3:08 AM on June 7, 2019 [7 favorites]

Also, as mass surveillance and sentiment analysis become more pervasive, how long until someone translates Janusz A. Zajdel's 1984 scifi story Paradyzja from the Polish?
posted by acb at 3:11 AM on June 7, 2019 [1 favorite]

You have a problem, but solving it in a rational, direct and effective way would infringe on a non-negotiable interest of some people who have influence.

Also, nobody would make a profit from that, so why would anyone do it? Our Ferengi American society doesn't do anything that generates no profit.
posted by Kirth Gerson at 3:21 AM on June 7, 2019 [2 favorites]

Is that show Just Shoot Me still on? Better not talk about it ...
posted by Kirth Gerson at 3:22 AM on June 7, 2019

A Martian anthropologist visiting the US would conclude that firearms are sacred in its society

Paging zardoz...

Thanks for this link. It reminds me that I really ought to do some searching on the enterprise system my kid’s school just bought into. Not sure that this kind of surveillance is included, but other kinds presumably are.
posted by eirias at 4:53 AM on June 7, 2019 [3 favorites]

Can’t help but wonder what Grand Rapids schools could maybe do with that $71,000 that could be more productive and less invasive solution than this shit. Like maybe hire a full-time school counselor?

OH WAIT we are talking about SCHOOLS... god forbid we actually spend any money on staffing
posted by caution live frogs at 5:14 AM on June 7, 2019 [4 favorites]

Any harms pale in comparison to the benefits of what is caught.

No. Just no.
posted by chavenet at 5:27 AM on June 7, 2019 [7 favorites]

Can’t help but wonder what Grand Rapids schools could maybe do with that $71,000 that could be more productive and less invasive solution than this shit.

Ugh. I can't help but think that Betsy DeVos, who is from the Grand Rapids area, has something to do with the mentality that would spend $71k on this.
posted by selfmedicating at 6:47 AM on June 7, 2019 [1 favorite]

See, now we don't have to control guns when we can just monitor literally everything people write online! That's SO MUCH better! We did it, everyone! Here's a free gun to commemorate the event, also you've been flagged for making a joke about dying for your cat.
posted by OnTheLastCastle at 6:50 AM on June 7, 2019 [3 favorites]

One of the parts of this that infuriates me is that schools don't take one of the biggest indicators of potential violence seriously.

That indicator being, of course, domestic violence. How many shooters have we seen demonstrate DV behaviors prior to their attacks? Because as I recall, it's a large number.
posted by NoxAeternum at 6:59 AM on June 7, 2019 [8 favorites]

Such is the new reality for America’s schools, which are hastily erecting a massive digital surveillance infrastructure, often with little regard for either its effectiveness or its impact on civil liberties.

Social media monitoring companies track the posts of everyone in the areas surrounding schools, including adults. Other companies scan the private digital content of millions of students using district-issued computers and accounts. Those services are complemented with tip-reporting apps, facial-recognition software, and other new technology systems.

Florida offers a glimpse of where it all may head (EdWeek): Lawmakers there are pushing for a state database that would combine individuals’ educational, criminal justice, and social-service records with their social media data, then share it all with law enforcement.

Across the country, the results of such efforts are already far-reaching.

The new technologies have yielded just a few anecdotal reports of thwarted school violence, the details of which are often difficult to pin down. But they’ve also shone a huge new spotlight on the problems of suicide and self-harm among the nation’s children. And they’ve created a vast new legal and ethical gray area, which harried school administrators are mostly left to navigate on their own.

“It’s similar to post-9/11,” said Rachel Levinson-Waldman, a lawyer with the liberty and national security program at the Brennan Center for Justice at the New York University law school. "There is an understandable instinct to do whatever you can to stop the next horrible thing from happening. But the solution doesn’t solve the problem, and it creates new issues of its own.”
...
“If you’re responsible for the safety and security of a school, you have to pay attention to the places where harm is being foreshadowed,” said Gary Margolis, the CEO of Social Sentinel, which claims “thousands” of K-12 schools in 30 states are using its service.

Margolis said it’s unfair to focus on the false positives that may slip through a company’s monitoring system. Any harms pale in comparison to the benefits of what is caught. He pointed to a recent incident in which Social Sentinel flagged a college student who threatened on Twitter to shoot his professor for scheduling an early morning exam. (The student, who said he intended no harm, was arrested.)

This feels like self-checkout lanes at grocery stores: instead of investing in more staff, it's cheaper to install new systems and manage those. Except who's managing this, and how are they being trained? Are they being trained? Are schools hiring new "security monitors" to manage this flow of information, or is it being heaped on the existing administration, who need to manage "traditional" student issues and performance, teacher performance, school budgets, while keeping an eye on test scores, which are tied to all of the above.

Also, "where it all may head" is starting to sound like China's surveillance state and "citizen scores". Which makes me think of new phrases that get used to bypass internet censors, and the fact that kids who do want to cause harm at school can learn from active shooter drills to know how to get around obstacles. Relying on technology is a game of cat-and-mouse, and kids are resourceful. Increasing investments in staffing, for the number of staff, their salaries, and their training, would be better than these tech "investments."

Maybe, and I'm just spitballing here, divert some of that para-military police budget to schools, pitched as "preventative community security." That means changing how funding is distributed, not only at the local level, but also at the federal level -- certain funding is earmarked for "tech purchases" and training (Tech.Ed.gov), which cannot be shifted to staffing budgets. While the EdWeek article doesn't address how these purchases are being funded, I have the feeling (from hearing about my wife's school's tech budget), this is part of the problem. Money is being thrown at "more technology for the classroom" when test scores are low, or the student population is poor and minority, instead of increased staffing and staff training.

In short: fuck all this shit.
posted by filthy light thief at 7:29 AM on June 7, 2019 [5 favorites]

- Is it possible to make a deployable tech tool, with a well-considered oversight system, that might actually get at the real heart(s) of the problems people have? I think so, it's just not a obvious-enough solution.

- To push back against something like this, the harms it could cause need to be made clear and immediate. We know that being watched is a problem, but we need to be able to communicate _why_ in specific, irrefutable terms. How do we do that?
posted by amtho at 7:38 AM on June 7, 2019

I in fact just now emailed the president of my local school board to enquire as to the details of our district monitoring policy. This article is particularly timely, given that my child will be getting his school-issued Chromebook this fall. (I already receive emails from Securly regarding his internet use at school, but since he's fairly young, there's not much in them now. Starting next year he'll have a significant increase in online ... stuff.)

Others might ought to do the same. I feel like this is one of those things where a few more people asking questions might help shape policies.
posted by telepanda at 7:50 AM on June 7, 2019

- Is it possible to make a deployable tech tool, with a well-considered oversight system, that might actually get at the real heart(s) of the problems people have? I think so, it's just not a obvious-enough solution.

Is it possible? Sure. Should we? Hell no. Besides, the reason that school administrators are caught flat-footed is because they don't engage in basic harm reduction techniques like building rapport and taking domestic violence seriously.

- To push back against something like this, the harms it could cause need to be made clear and immediate. We know that being watched is a problem, but we need to be able to communicate _why_ in specific, irrefutable terms. How do we do that?

We've done this repeatedly. The problem is not a lack of communication, it's school administrators looking for a magic bullet because they don't want to do the hard work.
posted by NoxAeternum at 7:51 AM on June 7, 2019 [6 favorites]

Sure, some bold dissidents are unafraid to express their disapproval of this now, but give it twenty years and it will seem just as normal as any other networked surveillance camera system in schools, on the public street, in doorbells, park benches, hotel lobbies, taxis, grocery stores, and every other public and private place.
posted by sfenders at 7:55 AM on June 7, 2019 [2 favorites]

We know that being watched is a problem, but we need to be able to communicate _why_ in specific, irrefutable terms. How do we do that?

I drearily guaran-damn-tee you that inside five years there will be at least one instance where an administrator has used this surveillance information to target specific victims for sex, possibly also using social-media information to blackmail them into it, and possibly using whatever automated psychological profiling is in the system. And several more instances of adminstrators caught with directories full of not necessarily directly pornographic images of kids culled from their social media that he uses to masturbate with and/or trades with other pedophiles.
posted by GCU Sweet and Full of Grace at 8:01 AM on June 7, 2019 [13 favorites]

(if these systems become widespread, that is)
posted by GCU Sweet and Full of Grace at 8:04 AM on June 7, 2019

They're training students to be 100% monitored and thought-policed so that they'll be compliant with this as adults.
posted by grumpybear69 at 8:32 AM on June 7, 2019 [2 favorites]

How does that old quote go? America can usually be counted on to do the right thing, but only after they've tried everything else...
posted by Cris E at 10:35 AM on June 7, 2019 [1 favorite]

I'd almost take that monstrous database of everything as long as it contained all the gun ownership data too so we could finally do some correlation studies. Of course once the first of those is published the gun folk will sue and the whole thing will be set afire by the courts, but it would be good to get some of that into the open for once.
posted by Cris E at 10:38 AM on June 7, 2019

We've done this repeatedly. The problem is not a lack of communication, it's school administrators looking for a magic bullet because they don't want to do the hard work.

To be fair, many educators and administrators work very hard and want to do the right thing, but often the problems the education system is being asked to solve are well beyond their means. If we want schools to feed, clothe, counsel and ultimately raise our children then perhaps we should spend more than the tutoring money we've budgeted. It's not just hard work, but a lot of it, and in some communities magic is the best chance of meeting all need.

But I'm just quibbling. In places of plenty where there are resources for Orwellian contraptions like this your point is true. This is bonkers.
posted by Cris E at 10:52 AM on June 7, 2019

They're training students to be 100% monitored and thought-policed so that they'll be compliant with this as adults.

Though some will take this as a challenge, and find ways to communicate in the open about deeds that are forbidden by the school, if not fully illegal, while the software flags basketball players who talk about an upcoming "shooting clinic" and a teacher who is happy that their credit score is "shooting up."

An example of students bypassing software limits is the fact that L.A. U.S.D high school students found a way to bypass software blocks on the devices that limit what websites the students can use within a week (NPR).
posted by filthy light thief at 10:57 AM on June 7, 2019 [1 favorite]

I am not framing this as "hurf durf just don't use Facebook", but I sincerely want to know: if students did not ever use their social media accounts on school-owned devices, would this tracking be possible? If yes, how are they gathering that information?
posted by nakedmolerats at 11:52 AM on June 7, 2019

Many schools have assignments that require social media accounts; they think nothing of assigning students, "join a facebook group about this literary genre and have a discussion about this book" or "design a website for #TrendyTopic and post it on Wix or Wordpress; advertise it on Twitter."

Parents may be able to push back, pointing out that it's unethical and possibly illegal to require kids' parents to agree to third-party TOU contracts in order to get their grades - but if the parents don't directly object, the school will just blithely hand out assignments that say, "each team must create a Google Doc to work on this project together..."
posted by ErisLordFreedom at 2:28 PM on June 7, 2019 [3 favorites]

Aggression Detectors: The Unproven, Invasive Surveillance Technology Schools Are Using to Monitor Students (Jack Gillum and Jeff Kao, June 25, 2019 for ProPublica, co-produced by Wired)

In the wake of the shooting at a Parkland, Florida, high school and other massacres, U.S. schools are increasingly receptive to [aggressive sound monitors]. Congress approved more than $25 million for school security improvements last year, and one analyst expects new technology could augment the $2.7 billion market for education security products.

The ProPublica link includes clips you can play that set off technology. None sound "aggressive," including a student coughing and comedian Gilbert Gottfried saying “Is it hot in here or am I crazy?” -- and

Some experts also dispute the underlying premise that verbal aggression precedes school violence, since they say mass shooters like Nikolas Cruz at Marjory Stoneman Douglas High School in Parkland, Florida, tend to be quiet beforehand. “I can’t imagine when it would be useful, honestly,” said Jillian Peterson, an assistant professor of criminology and criminal justice at Hamline University in St. Paul, Minnesota.

This article also clarifies that part of the problem is that Congress has earmarked money to improve school security, instead of providing more flexible funding to schools to let them set out their own priorities.
posted by filthy light thief at 9:05 AM on June 25, 2019 [1 favorite]