No Database is Neutral

January 16, 2023 1:08 AM

Over time, the mere existence of such databases becomes a continued justification for their use, so entrenched are they in everyday governance, in policy and decision-making. They aren’t merely representative of everything that the state already knows about an individual, but what’s possible for the state to know, if and when it becomes ostensibly necessary. from Database States by Sanjana Varghese

The Temptation to Abuse Government Databases [Civil Rights, Big Data, and Our Algorithmic Future]
Across US, police officers abuse confidential databases [Associated Press]
Yes, Police Are Snooping Through Criminal Databases For Personal Reasons All The Time [TechDirt]

posted by chavenet (11 comments total)

See also James C. Scott, who writes about what he calls High Modernism-- the idea that a small number of people can know what's going on and, with their view from above, regularize and organize the rest of society. See _Two Cheers for Anarchism_ (short and manageable), _Seeing Like a State_ (general theory and history), and _The Art of Not Being Governed_ (mostly the highlands of southeast Asia, but hard-to-govern regions in general).
posted by Nancy Lebovitz at 2:42 AM on January 16, 2023 [13 favorites]

So, in the late 90's, early aughts, there were two brand spanking new things: Friendster, and personals sites.

Both of them were new, and so both of them were full of people who were still intuiting what was okay and not okay to do on those sites. Among them, me. I very quickly grew frustrated with the personals sites because as a man starting conversations with women, I had to find the proper sweet spot between too-formal, which smacked of desperation, and too informal, which smacked of entitlement. But I also knew that these women had to make an intuitive judgement call on where I stood on the scale from prospective soul mate to possible axe murderer, on the basis of my profile and messages, and if my intuition was getting me nowhere, why should theirs be any better?

But there was also Friendster. Profiles on that site only disclosed your first name (and it seemed almost everyone was using their real name, or a very publicly known nickname), and a good amount of details about your personal interests. And your friends list. The network was readable to all users. I was quickly finding profiles that were 5 hops away from me, and learning which of my friends and local socialites were hubs on this network. And that everyone, just about everyone who had profiles on some of these personals sites, also had Friendster profiles. With a little geographic querying, I found dozens of personals profiles matching their Friendster profiles, which meant I knew exactly how to make myself a known entity to them before the first message. It made life so much easier if I could mention exactly which people and social cliques were known to the both of us. This, mind you, was without digging into any details that I was not supposed to know, and with me ready to reveal exactly what kind of digging my date could do to find the same information about me that I was finding.

So in short, these databases are ticking time bombs, and we should be thankful that cops who abuse them are meatheads.
posted by ocschwar at 7:17 AM on January 16, 2023 [8 favorites]

I mentioned it in passing once, but decades ago I designed a system for a bid to allow cross-querying of the databases of the various social services departments of the D.C. government. I was careful in my design to ensure that improper disclosure between departments was impossible and to protect the people from the police using it against them. Our bid was not selected.
posted by ob1quixote at 8:13 AM on January 16, 2023 [12 favorites]

Police Are Snooping Through Criminal Databases For Personal Reasons All The Time

I work in a hospital, and everything we do in the electronic medical records system is logged, and somebody does random audits. If we look at the data on a co-worker we have to fill out an extra screen explaining why we need to look at this information before we're let in, and somebody does random audits. If we look at the data for a celebrity, somebody does random audits. There's a mechanism to report a co-worker (or if you're a patient, a provider) and because everything is logged they can go back and see if you acted improperly. People get fired for looking at stuff they aren't supposed to look at.

This is a solved problem, you just have to care about misuse of data.
posted by joannemerriam at 11:40 AM on January 16, 2023 [33 favorites]

I am reminded of the classic paper "The Dark Side of Numbers" about how data storage systems created the necessary preconditions for the genocides of the 20th century. All data should be treated as toxic waste, don't keep more than necessary, don't accumulate too much in one place and have a clear plan for disposal.
posted by crossswords at 12:47 PM on January 16, 2023 [10 favorites]

Thank you. Zotero'd for the paper I am writing to shame my own profession, which taught me to value privacy, into actually observing those values.
posted by humbug at 3:15 PM on January 16, 2023 [1 favorite]

When my father worked at the IRS in the 80s and 90s, he once got written up for looking up himself. Agree with Joanne Merriam, the caring about it is the problem.
posted by toodleydoodley at 7:13 PM on January 16, 2023 [3 favorites]

A COMPUTER
CAN NEVER BE HELD ACCOUNTABLE

THEREFORE A COMPUTER MUST NEVER
MAKE A MANAGEMENT DECISION

-an IBM Presentation from 1979, as widely reposted
posted by snuffleupagus at 7:21 PM on January 16, 2023 [9 favorites]

But as theorist Liam Young argues in his book List Cultures: Knowledge and Poetics from Mesopotamia to BuzzFeed, statistics and information have emerged as the “lifeblood” of the modern state.

This sounds like it might be right up my alley. Over the past couple of years I've been idly thinking about the power of keeping track of people.

Reading about union head-buster Hal Banks may have kicked it off - a major source of his power was that he kept files on every one of thousands of workers in his industry, so that if someone crossed him he knew exactly which hotel in exactly which seaport to have his goons show up at.

Or the court records that Sumption drew on for his history of the Hundred Years' War: The English and French had larval bureaucratic states at best at the time, but the fact that those larval states were the ones who kept the records, preserved them over decades and then centuries, became one of the cores of their power.

And that power, developed over centuries, jumped out at me in the fate of the mutineers on the Bounty: After Captain Bligh made it back to England, the bureaucratic state whose job it is to keep track of everyone hunted them down on the other side of the earth - because what else could it do? no-one can be forgotten!

It seems strange that just writing down a few facts about everybody could be a source of power.
posted by clawsoon at 9:46 PM on January 16, 2023 [4 favorites]

A concept at least as old as taxes, so probably at least as old as writing itself.
posted by snuffleupagus at 9:55 PM on January 16, 2023 [1 favorite]

Probably older than writing, in that the human organizations (whether you call them tribes or bands or family units, whatever) that existed before writing probably prospered better if they had more individuals who lived to old age and had more accumulated experience to draw on. E.g. something like "ah yes, there was a drought like this when I was young, the wildebeest have probably gone [wherever]" might be life-or-death intel.

Writing was presumably a game-changer because it allowed human organizations to compile and store information outside of the minds of individuals, and making it more difficult to edit on-the-fly based on the whims of the recollector.

Computerized databases pale in comparison to that change, which was fundamental; I think electronic records are more an evolutionary step. There's little you can do with digital, electronic records that you couldn't do with a sufficiently-sized punchcard system (e.g. a Pentagon-sized Memex). I'm not even necessarily convinced, given the amount of money that governments spend on information systems, that it's markedly cheaper than storing stuff on rolls of microfilm. But the querying is admittedly faster.

We should always consider, before letting a government or other large organization compile a database, what the intent is of compiling the data and what can be enabled through it. Database designs are not value-neutral; their construction implies preferences about what type of queries and questions they aid in answering.

Interestingly, the one place I have seen this type of foresight—and to be honest, perhaps the only place—is in the statutory prohibition on constructing databases of firearms and owners. Perhaps just a broken-clock-right-twice-a-day thing, but folks concerned about hypothetical gun confiscation back in the 90s got a bunch of laws and regulations passed that prohibit the Federal government from constructing any database that might be de facto a gun registry, where you could easily type in someone's name and come up with a list of firearms they own. Instead, the records are kept in a distributed fashion across a number of participants, such that you can trace the ownership of a firearm (reliably to a first owner, at least) if you have it in your possession, but you can't easily go the other way.

Regardless of your feelings on the merits of that particular use case, it does show that it's possible to arrange data in such a way that it allows for certain types of queries while preventing others, if there's a strong enough desire to do so.

We could do similar things with other types of sensitive data, if we wanted: the trick, as usual, is overcoming the political headwinds created by the people who benefit from the status quo, and are able to leverage that benefit to subvert what would be in the best interests of the average person.
posted by Kadin2048 at 8:56 AM on January 17, 2023 [7 favorites]