If you feel you are being watched, you change your behavior
June 16, 2017 2:34 PM   Subscribe

Social Cooling describes the long-term negative side effects of living in a big-data-driven reputation economy. Data brokers derive thousands of scores from personal data; these form a “digital reputation”, and have the potential to affect their subjects' lives and opportunities. The long-term effects of this are a culture of conformity, self-censorship, risk aversion and social rigidity.
posted by acb (54 comments total) 49 users marked this as a favorite
 
Yup, Buttle Tuttle and away to the camp we go!
posted by Pembquist at 2:48 PM on June 16 [2 favorites]


So the Panopticon II: Electric Boogaloo?
posted by Homo neanderthalensis at 2:53 PM on June 16 [8 favorites]


I still retain hope that the advantages of the hivemind will offset and ultimately overcome these negative side effects.
posted by fairmettle at 2:53 PM on June 16 [2 favorites]


I'm thankful that organisations like the Electronic Frontier Foundation are fighting the good fight.
posted by cynical pinnacle at 3:08 PM on June 16 [11 favorites]




Metafilter: The long-term effects of this are a culture of conformity, self-censorship, risk aversion and social rigidity.
posted by resurrexit at 3:22 PM on June 16 [30 favorites]


You might find The Sudden Appearance of Hope by Claire North interesting.
posted by GenjiandProust at 3:45 PM on June 16 [1 favorite]


Is there a version of this written in a way that is readable?
posted by runcibleshaw at 3:46 PM on June 16 [4 favorites]


See also: the Black Mirror episode "Nosedive" (Netflix link).
posted by limeonaire at 3:54 PM on June 16 [20 favorites]


I can't read this at my desk because its so bright and colorful, everyone will notice I am reading it. This may affect my real life reputation for time-wasting at the office.
posted by Joh at 3:58 PM on June 16 [11 favorites]


-Delete facebook & minimize your presence on social websites. Nothing online is private. Assume people who hate you will be reading all of it. Everything will come back to haunt you.
-Don't put pictures of yourself on public websites. You know they're totally working on facial recognition, right? Don't help them.
-Don't even post vacation pics or mention stuff like going on vacation or traveling. It tips off burglars.
-Be conscious of the apps you install on your phone. Apps like to go through your contacts and keep track of your location and other information. They also are working on listening with the microphone.
-Don't use your real name for stuff.
-Don't take stuff on the internet seriously or try to impress people on it or compete with them. The internet is full of lies, trolls, and a lot of people on it who have nothing better to do in their lives.

I'm an introvert so this has been easy for me.

It is our human nature to want to be social and important and have people's attention and to be a star. Social media tickles this urge but it is not real. It's only simulated and hollow attention.

Websites like facebook are free. How could they possibly afford all the servers and 20k employees in California? Have you seen that episode of the Twilight Zone where aliens show up and make a big deal about how they only seek to serve man and do good things and then they start offering people trips to their home planet? (Spoiler alert: it's a cookbook)

There have been discussions in forums like this about how malls in essence privatized and enclosed what had always been free and public spaces like main streets. The point of social media is to capture, privatize, and monetize your social interactions. Not only that, it substitutes genuine real interaction and relationship-building with "liking" what other people have like and superficial little notes or one-up-manship to get more likes. Additionally social media encourages people to simply be seen doing something or to take a picture of an experience rather than to be doing something for the sake of doing it because you want to or you enjoy it.
posted by Blue Tsunami at 4:05 PM on June 16 [64 favorites]


If the only reason you are being "good" is because you are being watched, then you aren't being good—you are trying to be popular or acceptable. Being acceptable is perfectly appropriate for issues like politeness and modesty, as they aren't ethical concerns, merely social conventions. Incentivizing doing what is right with anything other than it being the right thing is both a short-sighted and dangerous policy as well as immoral.
posted by koavf at 4:09 PM on June 16 [5 favorites]


But it is so simple. Your parents had money. You have money. You have a dick. You have more money than debt. We want your money that is left over. The world really is OUR oyster, but we will sell you an oyster to use briefly. Don't have a dick, but do have money? We will send someone you might like, over to get your money. Are you middle aged, with a good retirement income? We will replace you with one of our retiring agents, we match face, gender, height; our plastic surgeon will take care of vague similarity. You will ignore your neighbors and move. Your kid is at home alone. Does this kid have access to your money, or any money? What can your kid do for us? Will your kid let us in the house when you are away? Does the kid know where you stash your valuables? Are you vaguely talented in an area where we are hiring? What is your Meyers Briggs Score? We have that, do you have enough money to get to work, to dress for work, to be entertaining for your coworkers, or our management? Can you think outside the box, but behave inside our box? Do you know when to STFU, when to beg, when to go blank and not hear? Are you suicidal, do you know how to drive a truck?

I have made it my forte to be useless to anyone, to be poor, to have nothing anyone wants, I will never grovel for work again, nor do I seek social interaction. I watch the water flow under the bridge, it just keeps on going.
posted by Oyéah at 4:17 PM on June 16 [39 favorites]


"We are becoming too transparent. This is breeding a society where self-censorship and risk-aversion are the new normal."

but, twitter
posted by scantee at 4:19 PM on June 16 [2 favorites]


Actually I do have a pertinent anecdote. My mom has early stage alzheimer's and was going to a dentist that offered a free lift to their office, great. Well....she breaks off a tooth and it turns out she has some problems so he has worked up a treatment plan for an implant and a referral to an endodontist for a root canal gone bad, she doesn't quite understand and I call them to find out what is what. Then....I don't know, I'm cynical?, I have a crippling abhorrence to feeling like someone is getting over on me?...I google the dentist and the endodontist and turns out dentist was sanctioned by dental board for screwing up some woman's mouth with an oversized implant not responding to complaint properly not doing a ct before...whatever. The endodontist.....well he was jacking off at work, got 'caught' by more than one woman at the office, was drummed out by the two other endodontists that he shared the space with, he sued for and won 500k from them.

Perhaps I am not being fair but... screw those cowboy's. Maybe it would be better for society if I didn't know about their pecadilloes and I recognize and find the algorithms of mass destruction thing very disturbing but on the other hand....it is my mother's mouth. Suffice it to say she is seeing another dentist.
posted by Pembquist at 4:19 PM on June 16 [14 favorites]


but, twitter

The problem is that the people who don't self-censor are precisely the ones who don't give a shit that the crudely self-tattooed swastika on their forehead freaks people out.
posted by acb at 4:22 PM on June 16 [4 favorites]


it's like no one ever grew up in a small town or went to a small high school
posted by pyramid termite at 5:24 PM on June 16 [34 favorites]


let me hasten to add - it was absolutely awful
posted by pyramid termite at 5:24 PM on June 16 [24 favorites]


Peppet has influenced my thinking on this to the extent that this piece seems almost like a footnote to "Unraveling Privacy".
posted by dmh at 5:37 PM on June 16


Can I just say once and for all, as someone whose (noncustodial, thank God) dad has a form of brain damage that left him with poorly regulated impulse control that once led him to pop me in the eye because he got annoyed I repeated a joke one too many times as a kid, and as someone who's studied and worked professionally as an editor that I am sick to death of the bullshit framing of impulse control and the use of editorial judgment and taking responsibility for what you're trying to say and having something to say in the first place as "self-censorship"?

Censorship is by definition when someone with more power than you stops you from speaking freely. That's not what using your own judgment about what you should and shouldn't do (in terms of speech acts) is, not by a mile. Self-censorship is a terrible incoherent concept. If *you* aren't even allowed to be the final authority on what you do or don't choose to say, there's no freedom involved. If the argument is that it sucks when people make you feel bad about something you said, okay, fine, but in no meaningful sense is any of that "censorship"-- not even metaphorically.
posted by saulgoodman at 6:41 PM on June 16 [27 favorites]


If only it were true that people self-censor. Online comments and the bizarre things people post on Twitter contradict that daily, I suspect because people belong to affinity groups and therefore think their comments aren't controversial.
posted by Peach at 7:44 PM on June 16 [1 favorite]


> Censorship is by definition when someone with more power than you stops you from speaking freely.

I don't accept this definition, and it is not the definition in dictionaries or wikipedia. Censorship is coercing someone not to express something, or otherwise quashing its communication. If you have the power to do that, you have the power to censor, even if you are less powerful in some broader sense. North Korea censors American journalism, even though North Korea is broadly less powerful that US—it doesn't make it "not censorship." They have the power to censor inside their territory, and censor they do.

> Self-censorship is a terrible incoherent concept.

It is coherent if it is understood to mean enforcing someone else's censorship in anticipation or fear of what they will do to you. That kind of fear can get inside your head, shape what you talk about. No one is saying "people shouldn't consider what they should or should not express, from a moral perspective." The FPP is saying something like "it is bad that people seem to be operating out of fear large, unaccountable, judgmental, and largely abstract structures penalizing and categorizing them for expressing something the speaker, in their *own* judgment, would otherwise think is fine."

The article is not anti-conscience, as you seem to be suggesting. In fact, it seems to think that this sort of panopticon climate is bad *because* it replaces conscience with a hollow, fear-based lookalike. I, for one, agree.
posted by andrewpcone at 7:46 PM on June 16 [32 favorites]


Here's a good piece on how an effective climate of threat can produce self-censorship: you don't, as an authoritarian regime, stand behind the shoulder of every person in society and monitor every speech act, instead you create a climate of risk around speech that makes it likely they will do the policing for you. Self-censorship isn't really about the judgments of the self in question about what speech is ethically demanded and what speech isn't. It's about a prudential fear-based calculation: is this the speech that will land me in prison/lose me my job?
posted by Aravis76 at 7:59 PM on June 16 [15 favorites]


The internet is full of lies, trolls, and a lot of people on it who have nothing better to do in their lives.

Um, hello, nice to meet you, too...
posted by The Toad at 9:03 PM on June 16 [5 favorites]


I am reminded of Temporary Autonomous Zones.
posted by gryftir at 9:13 PM on June 16 [1 favorite]


Is this why people have told me that they feel jealous that I live "fearlessly"?

Funny, this week I was told people can lose their career standing / reputation being seen doing stuff with me online.

(I'm risque but it's all legal consenting adults having fun on my social media)
posted by RichAndCreamy at 1:08 AM on June 17 [1 favorite]


Maciej Cegłowski has many related previouslies on this subject but is my favourite writer on this subject. (Sorry for the mad links, copy pasting from another post I made on another site, I'm being intentionally vague to fool future algorithms trying to dox my mefi profile. Even if it won't work)

http://idlewords.com/talks/build_a_better_monster.htm
http://idlewords.com/2015/11/the_advertising_bubble.htm
http://idlewords.com/talks/what_happens_next_will_amaze_you.htm

Also I've installed ScriptSafe in Chromium and am manually vetting every single domain a site tries to access when it loads. One can see the same few tracking companies' domains everywhere. Yours truly is mostly chilled.
posted by yoHighness at 3:10 AM on June 17 [6 favorites]


More like Social Cooled here.

Though, commenting here, favoriting this post, and favoriting a comment describing how to cool your social, contradict that very cooling.

Pure isolation would hurt.

Cooled, not frozen.
posted by filtergik at 3:41 AM on June 17


So glad this made it onto Metafilter.
Tijmen Schep (who started the site) is my good friend. In the past year(s) he's been giving more and more presentations to get this to the attention of the bigger public, and onto the agenda of organisations, policy makers.
I hope the Metafilter post helps it further a bit ;)
posted by Thisandthat at 4:19 AM on June 17 [1 favorite]


Someone I know works at the Human Project, and though they're going to be working more off metadata, they sit at that crossroads between private datamining and the shaping of public policy where a lot of these issues will play out, seems to me. (I've asked what their take is on the norming effect of their initiative.)

Also, as Schep puts it on his other site/thing Mathwashing, "code is law", and the legislative void around all this (that Ceglowski adresses in a recent talk of his, Notes from an Emergency) is starting to feel more than a little worrying.
posted by progosk at 4:54 AM on June 17 [2 favorites]


I'm thankful that organisations like the Electronic Frontier Foundation are fighting the good fight.

But they aren't. As I've pointed out in prior threads, their "Who's Got Your Back" report solely deals with websites interacting with the government, and not about the sites gathering the information in the first place. And this is mainly a function of not biting the hand that feeds them - they receive a good amount of funding from companies like Facebook and Google who are the ones engaged in this data collection.
posted by NoxAeternum at 5:24 AM on June 17 [8 favorites]


I am reminded of Temporary Autonomous Zones.

Peter Lamborn “Hakim Bey” Wilson's pirate-anarchist-utopia ideal? That's problematic for a number of reasons, not least of all it being a fig leaf for his own rampant pederasty.
posted by acb at 5:36 AM on June 17 [3 favorites]


I'm reading The Circle by Dave Eggers right now. It's basically a story about the ramifications of the next Facebook, a social media company so powerful it unseats Facebook.
posted by COD at 7:51 AM on June 17


-Delete facebook & minimize your presence on social websites. Nothing online is private. Assume people who hate you will be reading all of it. Everything will come back to haunt you.

One of my neighbors has been letting LinkedIn scrape her contacts or something over the past couple months, so periodically, I get emails from them asking me to join LinkedIn and be part of her professional network. I am not entirely even clear what her profession is, and I'm pretty sure that's mutual, so it's not even relevant. I suspect it's actually the city unemployment office that had her sign up for that, too. It's become pretty common for your use of public services to be contingent on private companies, which is really fucked up.

That's just a recent example where I'm personally being targeted by the company, and not all companies that gather your personally identifiable information from others inform you of it. They just mine it and sell it to other companies.

-Don't put pictures of yourself on public websites. You know they're totally working on facial recognition, right? Don't help them.
- Don't even post vacation pics or mention stuff like going on vacation or traveling. It tips off burglars.


These are both things I've had people post about me without my consent. One of my friends once posted a helpful legend translating the internet code names I use online to my and my family's real legal names. It wasn't even malicious or anything. Just thoughtless.

-Be conscious of the apps you install on your phone. Apps like to go through your contacts and keep track of your location and other information. They also are working on listening with the microphone.


Also make sure to yell "OK Google (or Siri), who's wearing a wire?" whenever you go anywhere, and whenever someone comes to visit.

There are things you can and should do to sort of protect your privacy and anonymity, but the weakest link is always the person you know with the loosest standards. If you don't hand your own information over willingly, you probably know plenty of people in real life who'll do it for you.

Your contact information is being exposed right now by everyone who doesn't read the privacy policies on the shitty little apps they install,* from sketchy, unaccountable companies, to everyone who has some personal justification for letting social network sites scrape their contacts, and to everyone who thinks it's fun to have an always on listening device on them at all times.

It's not something you can do on your own. The social norms are being established based on whatever the average naive, impulsive user will accept, and the only way to effectively opt out of those standards would be to cut off your social interactions with, well, probably most of the people you know. You can try explaining to them what the problem is, but in my experience, people get really defensive and will argue with you that it's somehow their right to effectively publish everything they know about you anywhere they want.

I don't really think most people believe that, but by the time they realize what they've been doing, they also realize they're already culpable, so they normalize it.

But even if you did somehow manage to get all of your social circles cleaned up, your ISP knows pretty much everything you're doing online. You can, of course, use a VPN, which slows you down and costs money, but it doesn't take much information at all to turn supposedly anonymized data into personally identifiable information.

There are a lot of horses already out of the barn, a lot of bells rung, and toothpaste that's out of the tube. None of which is an excuse to let it continue, but there is very little an individual can do to take back their privacy to any meaningful degree. Everyone needs to be better informed and start taking these things seriously, if anything is going to change, but that's clearly not about to happen in the US anytime soon.

Of course, I feel like this is where I'm supposed to suggest some kind of solution, but I don't have one. Uh, municipal broadband, widespread social shaming, and mass corporate disruption somehow. That's all I've got.

* That's not a carefully selected 'worst of' example. That's a randomly selected privacy policy for a crappy little hidden object game in the Google Play store that has no legitimate use for all that data.
posted by ernielundquist at 8:44 AM on June 17 [10 favorites]


I spent the last few days looking at Google Timeline. Exact maps of where I've been, every day in the last 2 years or so (since I got a phone with always-on location services). Google at least is kind enough to let me see the data they have on me - Apple, Facebook, the cell phone company and the NSA have the same data and won't share it.

It's interesting but really creepy. I've only been to 191 unique places in that whole time. Every work meeting, lunch out, or shopping trip is in there. I have a script now that can read the data and tell me how long on average I've spent at work and at home.

I'm much more afraid of that kind of thing than I am of silly social media posts.
posted by miyabo at 8:46 AM on June 17 [3 favorites]


Cynical Pinnacle, what led you to think that EFF was fighting this fight?

Is anyone seeing such scoring being gamed?
posted by Baeria at 10:52 AM on June 17


-Don't use your real name for stuff.

And yet:

(b) “Personal identifying information" means any of the following information:
1. An individual's name.
(2) Whoever, for any of the following purposes, intentionally uses, attempts to use, or possesses with intent to use any personal identifying information or personal identification document of an individual, including a deceased individual, without the authorization or consent of the individual and by representing that he or she is the individual, that he or she is acting with the authorization or consent of the individual, or that the information or document belongs to him or her is guilty of a Class H felony:
(a) To obtain credit, money, goods, services, employment, or any other thing of value or benefit.

Is a fake name or non-de-plume to gain a login "a benefit"? If so, do you want to be the caselaw?
posted by rough ashlar at 11:14 AM on June 17




RichandCreamy, you know I immediately went to your profile to find your various outlets. Sad to see that I can't endanger my job just now.
posted by janey47 at 2:14 PM on June 17 [1 favorite]


> I'm much more afraid of that kind of thing than I am of silly social media posts.

I'm not. I'm more scared about people who can't express their views, identities, or experiences without fear or losing their jobs/housing/etc. This is not some dystopian future in which some Google/NSA thing makes 1984. This is the case right now for many people, and it seems to be getting worse, with no obvious solution in sight.

I'm less worried about a shadowy corporate elite warehousing my data than I am about popular support for ferreting out, publicly shaming, and penalizing people for their shenanigans. Corporate and government oppression is far scarier when it enjoys popular support.
posted by andrewpcone at 3:40 PM on June 17 [6 favorites]


Is a fake name or non-de-plume to gain a login "a benefit"? If so, do you want to be the caselaw?


What? No. That's not going to happen. Using a fake name is not the same thing as fraudulently using someone's identity, and this is not something people need to be worried about.
posted by ernielundquist at 4:03 PM on June 17


Yikes, sounds pretty horrifying. One thing that you can do if you have a rooted phone is lock down much, much more what information apps and even Google have access to (on Android; or whatever the Apple equivalent), but even with that you can't block nearly everything. I turn off location services first thing because the amount of benefit I would get from it isn't even close to worth it for the privacy loss. On my computer I have so many browser privacy plugins installed they probably defeat each other. Then on the other hand, I sign dozens of online petitions with my real name, email address, and city location publicly displayed (with the idea that is more effective - does anyone know if it is?). That isn't even mentioning vehicles, which all of the newer ones have tracking built in. And on and on.
posted by blue shadows at 4:22 PM on June 17


A fake login isn't illegal - yet...
posted by blue shadows at 4:24 PM on June 17


The Obs: Google, not GCHQ, is the truly chilling spy network .

False dichotomy, anyone?

Related: Surveillance-media-state complex as feedback-control mechanism, [pdf, my title, not theirs].
posted by runcifex at 11:46 PM on June 17


Closing the loop
... Google sells the co-mingling of offline and online data as "Closing the Loop".

For example, if you bought a TV offline, Google would match your credit card history to your ad profile (containing your GPS record and your browsing data) to "prove" to the merchant that you did, or didn't, see one of its advertisements, the Washington Post reports.

...

Last year too, Google created "super profiles" of its users, breaking an earlier promise never to mingle data from your search history, YouTube viewing history or GPS location (constantly tracked by Android) with DoubleClick cookie information unless you explicitly opted in. Super profiles have prompted an antitrust complain from Oracle, arguing that the combined data hoard creates an insurmountable barrier to entry for any ad competitor to Google.

...

Having your location data – and thanks to mobile Google knows where you've been – allows Google to dictate terms to advertisers. Google knows where a merchant's customers are before the merchant itself does. Location data also allows Google to insert itself into every transaction. In the litigation with Skyhook, a rival supplier of location data, it emerged that Google insisted on a monopoly on personal location data collected on Android devices. The company was prepared to halt Motorola's production lines to ensure that only Google was collecting this data.
posted by sebastienbailard at 12:57 AM on June 18 [1 favorite]


So I am the only one seeing that this kind of scoring appears to be being gamed?
Who would know, if it was happening more widely?

EFF doesn't seem to be on the ball, on this.
posted by Baeria at 8:17 AM on June 18


One trouble with algorithms is that they are terrible, even when people aren't gaming them. That's why Netflix recommendations are frequently laughably bad. They also get details wrong. Google has figured out that I like soccer, but it seems indiscriminate about whether I am a Seattle Sounders fan or a Portland Timbers fan, and that's clearly the most important detail. Show me the right jersey, I might buy something, show me the wrong one, and I'm likely to close my computer in a rage and go out to enjoy the sunshine.

Re "Fake name": isn't that a contradiction in terms? I get that it's a term of art as far as deceptive names go, but I figured that people use it because it's short and contains a clever contradiction. If you give something a designation, that's a bona-fide name, by definition. How could it be fake?
posted by surlyben at 10:03 AM on June 18 [2 favorites]


One trouble with algorithms is that they are terrible, even when people aren't gaming them. That's why Netflix recommendations are frequently laughably bad. They also get details wrong.

And of course this may lead to even worse results than an all-knowing Big Brother. A pushy Big Brother who's half-aware of what is going on at all times and inclined to react to his imperfect intelligence is as likely to show up at your door with torches and pitchforks as mine.
posted by kittens for breakfast at 11:30 AM on June 18 [3 favorites]


I'm interested to see how "the kids" deal with this. I'm of an age where I don't do much embarrassing stuff. After the last work-party, folk were going about saying "Oh my god! It was so funny watching John dance while drunk. Do you want to see the video of it I put on Facebook?!" This is surely going to result in social cooling- those who are being shamed months later for getting their groove on at Christmas party, will likely look to avoid letting their hair down too much at the next party.

I wonder if there is already a nascent "no social media" rule amongst teenagers when they are messing around/partying? My generation need to get a grip with unthinking oversharing. (Not to dismiss the wider political and commercial changes that need to be made to make the future fit for humans to be humans.)
posted by Gratishades at 5:03 AM on June 19


Gratishades - I'm interested to see how "the kids" deal with this

I have no idea, it must be awful for them!

In possibly related news, this weekend I attended the first nightclub event I have seen with a strict 'no photography' rule. Some young people were in attendance.
posted by asok at 7:34 AM on June 19 [1 favorite]


advantage baby boomers. our core bonafides established pre digital. hardly any social rep needed. mostly we play and care little what others think of us.
posted by judson at 7:46 AM on June 19


mostly we play and care little what others think of us.

Are you comfortable looking up various symptoms you experience? Like, if your legs hurt or you start forgetting things more often, you're OK with letting Google and your ISP and anyone else add that information to your personal dossier?

It's not just "legitimate" business you need to worry about, either. Elder fraud is a huge industry that preys on people who are vulnerable, lonely, desperate, and naive.

A long, long time ago, I used to deliver the sketchiest Meals on Wheels route in my city, to elderly and/or disabled people living in urban trailer parks and residential motels in a very high crime area. They got mugged and robbed a lot. And many of them ended up there because they'd been defrauded and some dirtbags had cleaned out their retirement accounts.

This was a long long time ago, like I said, and the dirtbags have more information and better technologies to work with now. They can just buy a list of people with dementia and target them specifically.
posted by ernielundquist at 8:37 AM on June 19 [1 favorite]


advantage baby boomers. our core bonafides established pre digital. hardly any social rep needed. mostly we play and care little what others think of us.

The boomers I know are the absolute worst about posting everything online with full legal names of people without permission precisely because they have no understanding of everything being connected.
posted by winna at 12:15 PM on June 19 [2 favorites]


One of the reasons the Netherlands had one of the highest rate of Jewish deaths during the Holocaust (3/4s of the Jewish population) was because the Dutch had kept such good census records for so many years. As much as we want to be terrified by modern technology, Big Data has always been used nefariously.

Activist Flavia Dzodan has written extensively about this.
posted by Brittanie at 1:28 PM on June 20 [2 favorites]


« Older Peregrine Falcon Live Cam from PG&E Building...   |   “Morrowind feels like it begs you to come home,” Newer »


This thread has been archived and is closed to new comments