10,000-foot-tall, carrot-chomping, gay cocaine addict
December 14, 2018 8:56 AM   Subscribe

"What we learn from that finding is that 'mischievous responders' think some responses are funny but not others. For example, a 12-year-old saying that he has used heroin over 40 times has a certain panache to it compared to, say, disclosing suicidal ideation. But that could have the net effect of making it seem like gay and bisexual boys are abusing substances at much higher rates than they actually do." These Teenage Trolls Are Pretending to Be LGBT, and Screwing Up Scientific Studies [SLTheDailyBeast]
posted by Grandysaur (39 comments total) 12 users marked this as a favorite
 
Study shows teens are awful, although slightly less awful than adults. IRBs get increased requests to administer electric shocks “just ‘cause.”
posted by GenjiandProust at 9:15 AM on December 14, 2018 [9 favorites]


I wonder if we could use a similar methodology to parse the raw data on support for Donald Trump, both as a candidate in 2016 and subsequently as President, given what we know about the Pepe avatars who have been gaming the polls.
posted by Strange Interlude at 9:17 AM on December 14, 2018 [1 favorite]


We were given a survey like that when I was in high school, back in '86. I don't remember sexual orientation being on it, but I do remember thinking it was bullshit and nobody's business so I lied all over it about how many serious drugs I was supposedly doing (was not actually doing any drugs).

I didn't see any upside to being honest and I didn't care about the mission of the people who designed the survey. It wasn't to be funny so much as coming from my lifelong setting as a cynic and a skeptic. Not your cog.
posted by Squeak Attack at 9:18 AM on December 14, 2018 [18 favorites]


Several years ago, for example, colleagues of Cimpian’s made the mistake of having write-in options for height and weight, which had an obvious result: “Some kids were saying that they were 10,000 feet tall or that they weighed 666 pounds.”
I'm sure that this is something that lots of people have thought of, but these clearly false (and the stuff about carrots etc) responses feel like a clue to one way of weeding out the little shit ahem, 'mischievous' responders. If you worked in some questions with answers that are simultaneously impossible (or exceptionally improbable) and irresistible, it seems to me like they would provide very valuable data for the process of sifting out Beavis and Butthead's answers.
posted by howfar at 9:20 AM on December 14, 2018 [13 favorites]


Several years ago, for example, colleagues of Cimpian’s made the mistake of having write-in options for height and weight, which had an obvious result: “Some kids were saying that they were 10,000 feet tall or that they weighed 666 pounds.”
In a way, an answer is kind of staring us in the face: provide free-form fields that are honey traps for the mischievous!

If a kid says they're 10,000 feet tall that seems like an excellent sign that you should disregard the rest of their survey.

on preview: howfar knows what I'm talking about ;).
posted by pmv at 9:23 AM on December 14, 2018 [9 favorites]


Stuff like the Youth Risk Behavioral Survey is really important for the US and states to understand when and where there are problems, allocate resources to efforts to improve people's health, and figure out whether those are successful.

We all benefit enormously from this kind of public health work - though the benefit is often invisible. Answering an anonymous survey honestly is a pretty small price to pay for these benefits - it's a pretty small ask from people who are working on your behalf.

It's sort of like not cutting in line, or like not sneezing directly on doorknobs - a small way of cooperating with the world around you for a shared benefit.

I remember taking this survey and being confused at the questions - and wondering if they would trigger some sort of guidance counselor intervention. I probably would have been more honest if I understood what was happening with the information.
posted by entropone at 9:24 AM on December 14, 2018 [17 favorites]


Answering an anonymous survey honestly is a pretty small price to pay for these benefits

That's a super nice sentiment that I understand as a grown-up person, but did not give any shits about as an angry 17 year-old girl. And being told something like "it's a pretty small ask from people who are working on your behalf" probably would've triggered even worse behavior from me at that age. I was a jerk, but so are a lot of teens.
posted by Squeak Attack at 9:29 AM on December 14, 2018 [20 favorites]


hey kid, wanna try some derbisol?
posted by BungaDunga at 9:31 AM on December 14, 2018 [1 favorite]


I had always assumed that researchers were already correcting for this kind of thing.

I remember being a teenager decades ago and filling out utter nonsense on the surveys we were given.
Why yes, I do smoke marijuana 5 times a day while simultaneously having sex with 3 or more partners and maintaining a 3.5-4.0 GPA in my household with an income of between $80,000-100,000.

I like the idea of the bait questions. I wonder how they slide a question about carrots into a survey, though.
posted by madajb at 9:31 AM on December 14, 2018 [2 favorites]


Answering an anonymous survey honestly is a pretty small price to pay for these benefits

That's also a real hard thing to get teenagers to get on board with, in my experience. I was a good kid, overall, but I sure was a mischievous responder.
posted by Grandysaur at 9:33 AM on December 14, 2018


Squeak Attack, I interpreted your comment ("not your cog") as meaning that you still felt that way. Cause yeah I totally get teens who don't understand - I was one too, of course.

But unfortunately I run into a lot of adults who still do feel that way - our society has a pretty out-of-wack calibration around individual good and collective good.
posted by entropone at 9:45 AM on December 14, 2018 [10 favorites]


When this survey was given to me I don't recall it being clarified at all that it was anonymous, or universal, or even actually important. I lied flagrantly, presenting myself as a perfectly functional very respectable teenager with no intrusive thoughts or suicidal ideations who did not resent my peers and whose only indulgence was a trip to the malt shop fridays after school

Some of that was knowing full well how my school handled "problems." The rest of it was a lifelong distrust of authority figures asking about my inner life.
posted by a power-tie-wearing she-capitalist at 9:51 AM on December 14, 2018 [11 favorites]


IRBs get increased requests to administer electric shocks “just ‘cause.”

WHAT????
posted by Sophie1 at 10:10 AM on December 14, 2018 [1 favorite]


I probably would have been more honest if I understood what was happening with the information.

Personally, I would not have believed them if they'd explained in all honesty what the survey was about. I remember this survey and I remember lying my ass off on the grounds that there was no way filling it out honestly could possibly help me, whereas it might well subject me to the unwanted attentions of the school guidance counselor or worse.

If they had told me that the survey was only for collection of general statistics and that it would not be used against me in any way, I would have just assumed they were liars because by that point in my life I had realized that adult authorities lie to kids all the fucking time so they would have had absolutely zero credibility with me.
posted by Anticipation Of A New Lover's Arrival, The at 10:27 AM on December 14, 2018 [30 favorites]


Why yes, I am still mad about high school.
posted by Anticipation Of A New Lover's Arrival, The at 10:30 AM on December 14, 2018 [10 favorites]


Yeah, I still feel that way.

Teenagers don’t owe us anything. They’re a demographic of fully self-aware human beings to whom we collectively and uncontroversially deny fundamental rights. We demand accountability without offering them agency. We treat them like children, we lie to their faces, and then we punish them for expressions of distrust or disrespect. We lock them up in unregulated Skinner boxes every day for years, and then we mock their endemic mental illnesses as being hormonal.

If you sincerely just want to help them, good for you I guess. But expecting them to understand that and trust you is a little bit like being a Christian missionary who expects indigenous people to understand and trust that you sincerely just want to save their eternal souls. There’s no reason any rational person would.
posted by emmalemma at 11:06 AM on December 14, 2018 [36 favorites]


Almost everyone in a position of authority, left or right, seems to under-plan for contrariness. It’s like once you’re vested with power, you forget the general mistrust the powerless feel.
posted by bendybendy at 11:16 AM on December 14, 2018 [9 favorites]


Emmalemma, I'm going to print our your comment and hang it in my office.
posted by Grandysaur at 11:19 AM on December 14, 2018


So, basically, kids who present as “good” are really bad, and kids who present as “bad” are really good, if a bit shitty with an untraceable Scantron sheet.

So just invert the results for perfectly flawless results. Duh. You’re welcome, science!
posted by Sys Rq at 11:34 AM on December 14, 2018


Have them write a freeform one paragraph response, that way we can tell which high schoolers are actually bots.
posted by Query at 11:44 AM on December 14, 2018


/draws line on calendar at the point YouTube was invented.

“Anyone born after here is lost.”
posted by Artw at 11:51 AM on December 14, 2018 [2 favorites]


Man, now I am realizing all over again just how much high school fucked me up. Here I am sixteen years after graduation, and my blood still boils when I think of the years of kafkaesque misery I was forced to endure. A lot of the lessons I learned in high school have been pretty maladaptive later in life, and a lot of the emotional and psychiatric issues that I now have stem directly from that time. And I was a "smart" kid in a "good" school with a loving and supportive family that doesn't need scare quotes.

We are permanently traumatizing a huge percentage of our children by forcing them through this fucked-up system.
posted by Anticipation Of A New Lover's Arrival, The at 12:29 PM on December 14, 2018 [4 favorites]


Teenagers don’t owe us anything.

Oh, I agree that our society by and large fucks teenagers over - in how we treat them now and how we've treated the world they'll inherit. And I'll bet we agree that changing that involves asking a lot more of ourselves - changing the structure we build for children and teens, and the society that we make them and raise them and give to them.

But doesn't part of changing it all also ask that we let and welcome teenagers to be more or less full members of society? With a healthy balance of rights and obligations? Accepting some responsibility for how something works, that affects more than just yourself, is like... a huge and valuable and important developmental moment. I remember those moments in my life.

So I think that there's an opportunity somewhere to build a form of civic and social trust. It's mostly on us - but it will involve asking more of teenagers, too. Maybe that's not quite the same as them owing us anything - I don't know. But every now and then I'm an optimist and think that our society isn't irreparable.
posted by entropone at 12:58 PM on December 14, 2018 [1 favorite]


How about rights first, then responsibilities. If we want to repair this relationship, we should be prepared to put in most of the work up front because we are the abusers who are trying to reform. The victims do not bear an equal burden of responsibility.
posted by Anticipation Of A New Lover's Arrival, The at 1:23 PM on December 14, 2018 [5 favorites]


Yeah, "this is anonymous" meant "we are going to relay this information to your parents immediately" pretty much 100% of the time when I was a teenager. Only a sucker would fill these things out honestly.
posted by enn at 1:53 PM on December 14, 2018 [7 favorites]


As to high school, I was lucky that they showed The Prisoner on TV at that time. Seriously, it shaped my attitude to authority - all the adult lies. And that attitude has stayed with me almost fifty years later. As the last episode said, youth rebels because it’s expected of them. Later someone who I should remember said youth must rebel, but they also crave conformity. So they rebel against adults and conform to each other. The Prisoner said that we have to strive towards authentic rebellion not these inauthentic forms. Can we try to teach this?
posted by njohnson23 at 1:57 PM on December 14, 2018 [2 favorites]


I think this is going to be one of those threads where it is fairly obvious who has not been in a room filled with teenagers since their own teenage years.
posted by betweenthebars at 2:17 PM on December 14, 2018 [4 favorites]


I mean, they search schools with drug dogs; you'd have to be stupid to admit in writing to using drugs. And that's if you have any respect whatsoever for the weird bullshit going on all the rest of the time, like getting "abstinence pledges" to sign.
posted by starfishprime at 3:15 PM on December 14, 2018 [5 favorites]


What Anticipation Of A New Lover's Arrival, The said. Reconciliation and rebuilding trust is a conversation you can have after the abuse has stopped. Trying to have that conversation while the abuse is ongoing gives something of the impression that you don’t really intend for the abuse to stop.
posted by emmalemma at 3:25 PM on December 14, 2018 [3 favorites]


I find it frankly insulting that someone would say I can't identify abuse when it's happened to me, just because I am not a parent myself. Like, if I only had a teenager, of course I would see that they don't deserve civil rights.

OK, forget rights. How about we just stop systematically gaslighting our children? That would be a start.
posted by Anticipation Of A New Lover's Arrival, The at 3:38 PM on December 14, 2018 [1 favorite]


I remember taking this survey and being confused at the questions - and wondering if they would trigger some sort of guidance counselor intervention. I probably would have been more honest if I understood what was happening with the information.

Ideally, the survey researchers should have had the opportunity to tell you the student that your individual information cannot be traced to your answer. That's just standard good practice for the protection of human subjects when you're doing surveys, even when the respondents are teenagers. Heck, especially when the respondents are teenagers.

The problem in practice is that, if you're doing surveys of children in school, you not only have to deal with the school acting in loco parentis, but you might also have to deal with the kids' parents themselves. School officials and parents aren't necessarily cool with the idea that kids can tell anything about sex or drug use on a survey & not have some responsible adult get back to the kid. So, researchers pretty much have to deal with people trying to circumvent their efforts to maintain confidentiality.

Then you often have open-ended text responses in these surveys. Some kids will use the opportunity for text comments to complain about cafeteria food or make dick jokes, but some kids will actually use the comments as a way of making a cry for help, even to the point of admitting suicidal thoughts or admitting that they've been abused. Then you get into the territory where you're dealing with duty-to-warn statutes that override confidentiality predictions. And, well, let's just say it's complicated & when I used to do surveys for the government, I had to take tests every six months to show that I could handle these issues.
posted by jonp72 at 4:04 PM on December 14, 2018 [4 favorites]


hey kid, wanna try some derbisol?

The drug survey I took in junior high school didn't mention derbisol. Instead, I think the fake drug that survey researchers put in to find the "mischievous responders" was called awarna. I seem to remember the hessian kids trying to figure out what awarna was & wondering where they could get their hands on some.
posted by jonp72 at 4:07 PM on December 14, 2018 [1 favorite]


We study minds as if they were in outer space.
posted by ethansr at 4:58 PM on December 14, 2018


I worked on the Youth Risk Behavior Survey for a while and I felt that many of the issues with the results were related to the way the survey is administered. In some areas teachers administer the survey so the accuracy and completeness of your results will have a lot to do with student's relationship to that teacher and their level of trust. In other areas public health agencies administer the survey, which was what I did. I would expect that when surveys are being administered by someone the teens do not know then they are more likely to be honest. After collecting surveys and leaving the classroom we would do a quick review for accuracy (write ins, checking for patterns, checking for consistency in the skip pattern). A lot of the trolls would get bored within the first 30 questions (it is a very long survey) and they would put in a pattern and those surveys would be thrown out. I'm sure plenty of trolls still get through in spite of this.

There are additional issues in the data because not all question subsets are administered in all areas. Some states ask very few questions about sexual activity and sexual health while others ask all of the sexual health questions as well as about LGBTQ issues. You can only look at the data at state level, not school district or county level, though you can look at some large MSAs. It's not the best possible tool and it is error prone but it's the best tool we currently have for reviewing teen health.

Also, the survey covers a whole host of issues including nutrition. For some reason, asking teens to estimate how many times a week they ate potatoes that weren't french fries is funnier then asking them how often they use condoms. I could always tell when they hit the nutrition section.
posted by arachnidette at 5:12 PM on December 14, 2018 [9 favorites]


I find it frankly insulting that someone would say I can't identify abuse when it's happened to me, just because I am not a parent myself. Like, if I only had a teenager, of course I would see that they don't deserve civil rights.

Unless something has been deleted, no one here is saying that.
posted by betweenthebars at 9:03 PM on December 14, 2018 [6 favorites]


If you're a teenager in America, nearly every adult you interact with is either a cop or closely cop-adjacent. You learn pretty quick to not tell the truth to anyone, for any reason.
posted by The Horse You Rode In On at 8:52 AM on December 15, 2018 [2 favorites]


Ideally, the survey researchers should have had the opportunity to tell you the student that your individual information cannot be traced to your answer.

My kid wouldn't believe that, and if he asked me, I would have told him not to believe it and told him to lie freely on the survey.

The fact is that schools (and Utah schools in particular) have a history of questionable ethics, i.e. asking questions "in confidence" and not keeping them confidential, and as someone mentioned above, a history of letting religious groups and other groups with agendas pose as legitimate school activities. And even when they try to do things that are confidential, they let underpaid teachers and even student volunteers pass them out, collect them, and so on.

My kid has had a survey about "the evils of pornography", and a "What would you do on a date" quiz where the girls get different questions than the boys, and the "abstinence pledge" someone mentioned above, and at least two assemblies that turned out to be religious presentations.

In my school in the 80s, they had "random locker checks" where they opened certain students' lockers, or sometimes all of the lockers, looking for drugs. There's no way I would have honestly answered a survey in that environment.

And I've personally had a schoolteacher and a counselor share private information about my child with someone that did not have the right to that information.

Schools have to be trustworthy before they can expect teenagers to trust them with private information.
posted by mmoncur at 7:20 PM on December 15, 2018 [3 favorites]


Ideally, the survey researchers should have had the opportunity to tell you the student that your individual information cannot be traced to your answer.

Unless the CDC sends people out to oversee the administration of every classroom's survey, tracing a student's answers back to them is as simple as remembering which form they handed you.
posted by GCU Sweet and Full of Grace at 8:36 PM on December 15, 2018 [1 favorite]


I was never administered this survey, or any like it, as a teenager, although I probably would have been honest because I was that kind of kid.

I administered surveys like this over the phone for a while, including the adult equivalent, the Behavioral Risk Factor Surveillance Survey. Good move on dropping "surveillance" from the name of the version for teenagers.

I did administer a survey where I talked to teenagers. For that one, the goal was to study teenagers who lived in areas where a major tornado had recently hit, with follow-ups certain intervals. I'd ask them a set of questions off a standard depression and anxiety screener, which was paired with asking their parents about how much the tornado had affected their family.

I never had any of the teenagers obviously lying to me, but the dynamics of talking on the phone to a stranger, unaffiliated with anything in your life versus filling out a form in your school are pretty significantly different. If any were lying, it was giving me the "normal" answers to get through it as painlessly as possible, I imagine. More of a problem for me was the control group families who lived on the other side of town from the direct impact or whatever, and the kids (and sometimes parents) who would feel like none of this applies to them, why are they being asked these things.
posted by vibratory manner of working at 10:01 AM on December 20, 2018 [1 favorite]


« Older Minimalist City Maps with a Twist   |   The internet spawned Uber and Amazon, not the... Newer »


This thread has been archived and is closed to new comments