You are neither too skeptical nor too gullible
August 13, 2023 9:28 AM Subscribe
misinformation susceptibility In this study, you will be asked to rate 20 news headlines as real or fake and answer a few optional questions about your background. Results given in four categories, including:
- Veracity Discernment: 100% (ability to accurately distinguish real from fake news)
- Distrust/Naïvité: 0 (ranges from -10 to +10, overly skeptical to overly gullible)
A YouGov survey of 1,516 U.S. adult citizens tested Americans' susceptibility to falsehoods in news, and found that on average, they failed to correctly classify one-third of headlines as either real or fake. And certain segments of the population are even more likely to be fooled by misinformation. In particular, the poll finds that there are certain subgroups who are more likely than others to fail to distinguish fake headlines from real ones: Younger adults perform worse than older adults, Republicans perform worse than Democrats, and people who get their news from social media perform worse than people who do not.
- Veracity Discernment: 100% (ability to accurately distinguish real from fake news)
- Distrust/Naïvité: 0 (ranges from -10 to +10, overly skeptical to overly gullible)
A YouGov survey of 1,516 U.S. adult citizens tested Americans' susceptibility to falsehoods in news, and found that on average, they failed to correctly classify one-third of headlines as either real or fake. And certain segments of the population are even more likely to be fooled by misinformation. In particular, the poll finds that there are certain subgroups who are more likely than others to fail to distinguish fake headlines from real ones: Younger adults perform worse than older adults, Republicans perform worse than Democrats, and people who get their news from social media perform worse than people who do not.
Meant to include this disclaimer:posted by spamandkimchi at 9:29 AM on August 13, 2023 [6 favorites]
Safari is not letting me open the site due to inability to establish a secure connection
posted by midmarch snowman at 9:31 AM on August 13, 2023 [3 favorites]
posted by midmarch snowman at 9:31 AM on August 13, 2023 [3 favorites]
I got 19/20, misjudging one real story as fake on a coin-flip, and I would have had to step outside the bounds of "real or fake news headline" and into "meta analysis" and "test design" in order to have gotten it right intentionally.
posted by seanmpuckett at 9:35 AM on August 13, 2023 [16 favorites]
posted by seanmpuckett at 9:35 AM on August 13, 2023 [16 favorites]
20/20, if you choose to believe me.
posted by Mei's lost sandal at 9:38 AM on August 13, 2023 [30 favorites]
posted by Mei's lost sandal at 9:38 AM on August 13, 2023 [30 favorites]
I got 20/20, and I’m suspicious because I would have expected to miss one or two.
posted by matildaben at 9:39 AM on August 13, 2023 [9 favorites]
posted by matildaben at 9:39 AM on August 13, 2023 [9 favorites]
I got 19/20 also. Wonder which real headline I judged to be fake.
posted by larrybob at 9:40 AM on August 13, 2023 [5 favorites]
posted by larrybob at 9:40 AM on August 13, 2023 [5 favorites]
19/20. I didn't know if fake meant "factually untrue" vs "not actually published by a news organization". Because some of those headlines could have been actually published even when false
posted by airmail at 9:40 AM on August 13, 2023 [76 favorites]
posted by airmail at 9:40 AM on August 13, 2023 [76 favorites]
19/20 on MIST-20, 16/16 on MIST-16
extremely liberals, lets gooooooooooooo skew these research results lol
posted by paimapi at 9:41 AM on August 13, 2023 [13 favorites]
extremely liberals, lets gooooooooooooo skew these research results lol
posted by paimapi at 9:41 AM on August 13, 2023 [13 favorites]
I mean, I absolutely would believe you could have a study that found "Left-Wingers Are More Likely to Lie to Get a Higher Salary"- I wouldn't actually believe it was a real effect, but it wouldn't be hard to produce that result in a crappy (but real) study to get that headline. It's not obviously a purely made-up story. I marked it down as "fake" (correctly), but still.
posted by BungaDunga at 9:42 AM on August 13, 2023 [31 favorites]
posted by BungaDunga at 9:42 AM on August 13, 2023 [31 favorites]
Pleased to say I got 20/20. Shared with my family, will be curious to see how they do!
posted by jzb at 9:44 AM on August 13, 2023 [2 favorites]
posted by jzb at 9:44 AM on August 13, 2023 [2 favorites]
18/20 for me. Great! It's harder to evaluate some of the headlines without knowing a source, interesting.
posted by tiny frying pan at 9:44 AM on August 13, 2023 [5 favorites]
posted by tiny frying pan at 9:44 AM on August 13, 2023 [5 favorites]
Shit. I got 16/20. I need to rethink my life.
posted by brundlefly at 9:45 AM on August 13, 2023 [6 favorites]
posted by brundlefly at 9:45 AM on August 13, 2023 [6 favorites]
13/16 I surprised myself.
posted by manageyourexpectations at 9:46 AM on August 13, 2023 [2 favorites]
posted by manageyourexpectations at 9:46 AM on August 13, 2023 [2 favorites]
well, that was easy. And I swore off breaking news over a year ago. The obviously fake ones really are obviously fake.
The only thing I got wrong was I rated my ability to recognize real and fake news as good. Apparently, it's GREAT. So on that note, let me just weigh in on the S-Files UFO stuff and say ... I don't trust anybody on this stuff.
posted by philip-random at 9:46 AM on August 13, 2023 [1 favorite]
The only thing I got wrong was I rated my ability to recognize real and fake news as good. Apparently, it's GREAT. So on that note, let me just weigh in on the S-Files UFO stuff and say ... I don't trust anybody on this stuff.
posted by philip-random at 9:46 AM on August 13, 2023 [1 favorite]
19/20, I would really like to know which one I got wrong though. And I was a little confused at first because I could see all of these being actual headlines somewhere, even if I think they are obviously fake.
posted by maggiemaggie at 9:47 AM on August 13, 2023 [15 favorites]
posted by maggiemaggie at 9:47 AM on August 13, 2023 [15 favorites]
as a sidenote and because it's the skeptical thing to do, the way this study was constructed is very sus and clickbaity as hell. their first link explaining this study just gave so many red flags like
1) The first survey to use the new 20-point test, called ‘MIST’ by researchers and developed using an early version of ChatGPT (LLMs are biased based on their sources, we already know this)
2) “To understand where and how best to fight misinformation, we need a unified way of measuring susceptibility to fake news. That is what our test provides,” said van der Linden, author of the new book Foolproof. (so weird how a study that got a ton of press coincidentally has a lead author who just released a new book on the exact same subject)
3) To create false but confusingly credible headlines – similar to misinformation encountered “in the wild” – in an unbiased way, researchers used artificial intelligence: ChatGPT version 2. (anybody that puts the word unbiased next to ChatGPT lol)
and then reading the actual study itself
1) three different sampling platforms—Respondi, CloudResearch, and Prolific (already selects for people who gravitate towards online questionnaires)
2) the fact that it's behavior psych at all just give the ick lol
people much smarter and data-bright than me can dig into the study design and methods but, again, single study, not-an-RCT smacks of fun weekend quiz and not 'let's take this info to heart'
posted by paimapi at 9:51 AM on August 13, 2023 [33 favorites]
1) The first survey to use the new 20-point test, called ‘MIST’ by researchers and developed using an early version of ChatGPT (LLMs are biased based on their sources, we already know this)
2) “To understand where and how best to fight misinformation, we need a unified way of measuring susceptibility to fake news. That is what our test provides,” said van der Linden, author of the new book Foolproof. (so weird how a study that got a ton of press coincidentally has a lead author who just released a new book on the exact same subject)
3) To create false but confusingly credible headlines – similar to misinformation encountered “in the wild” – in an unbiased way, researchers used artificial intelligence: ChatGPT version 2. (anybody that puts the word unbiased next to ChatGPT lol)
and then reading the actual study itself
1) three different sampling platforms—Respondi, CloudResearch, and Prolific (already selects for people who gravitate towards online questionnaires)
2) the fact that it's behavior psych at all just give the ick lol
people much smarter and data-bright than me can dig into the study design and methods but, again, single study, not-an-RCT smacks of fun weekend quiz and not 'let's take this info to heart'
posted by paimapi at 9:51 AM on August 13, 2023 [33 favorites]
Yeah, I take this about as seriously as "Which My Little Pony are you?"
I also was "too skeptical," getting three wrong by calling them fake. I went with "probably not true even if it could be factually reported as true", because headlines are generally lies anyway even for good reporting.
posted by Scattercat at 9:58 AM on August 13, 2023 [16 favorites]
I also was "too skeptical," getting three wrong by calling them fake. I went with "probably not true even if it could be factually reported as true", because headlines are generally lies anyway even for good reporting.
posted by Scattercat at 9:58 AM on August 13, 2023 [16 favorites]
17/20 for me. I correctly spotted all the fakes, and I marked 3 real news items as fake as well.
posted by EmpressCallipygos at 10:06 AM on August 13, 2023 [3 favorites]
posted by EmpressCallipygos at 10:06 AM on August 13, 2023 [3 favorites]
The easiest way to get someone to believe your fake news is to write it such that it panders to their inner sense of identity.
I didn't get the feeling that any of those headlines did that for the typical MeFier, which made it easy for us.
Or maybe the typical MeFier has a sense of identity that is soundly rooted in reality.
Ironically, that sounds pretty ingratiating to MeFi which makes me want to dismiss the study itself as fake news.
posted by droro at 10:12 AM on August 13, 2023 [9 favorites]
I didn't get the feeling that any of those headlines did that for the typical MeFier, which made it easy for us.
Or maybe the typical MeFier has a sense of identity that is soundly rooted in reality.
Ironically, that sounds pretty ingratiating to MeFi which makes me want to dismiss the study itself as fake news.
posted by droro at 10:12 AM on August 13, 2023 [9 favorites]
20/20, based mainly on whether or not the phrasing is inflammatory.
posted by InfidelZombie at 10:12 AM on August 13, 2023 [34 favorites]
posted by InfidelZombie at 10:12 AM on August 13, 2023 [34 favorites]
20/20, based mainly on whether or not the phrasing is inflammatory.
Same.
posted by Zumbador at 10:14 AM on August 13, 2023 [4 favorites]
19/20, slightly on the skeptical side. But I'm not sure that this really tells us anything useful. By my rough estimate, about half of these are "can you spot highly emotive language" (illegally manipulated the weather to cause devastating storms!) and half of them are "can you spot which views are far outside the mainstream consensus" (Ebola Virus 'Caused by US Nuclear Weapons Testing', New Study Says)
A good understanding of what the mainstream consensus is and why it is that way is part of having a good first-pass bullshit detector, but too great a trust of the mainstream consensus is also a mistake, and not every view far outside of the mainstream consensus is wrong. If you are the kind of person who believes it's plausible that it could be true that the Ebola virus is caused by US nuclear weapons testing, then you're going to actually have to read, and evaluate the study - which most of us are not very good at, even if you're lucky enough to find the study not behind a paywall.
The real test of whether you're susceptible to misinformation, I think, is how you proceed after you read the headline.
posted by Jeanne at 10:15 AM on August 13, 2023 [25 favorites]
A good understanding of what the mainstream consensus is and why it is that way is part of having a good first-pass bullshit detector, but too great a trust of the mainstream consensus is also a mistake, and not every view far outside of the mainstream consensus is wrong. If you are the kind of person who believes it's plausible that it could be true that the Ebola virus is caused by US nuclear weapons testing, then you're going to actually have to read, and evaluate the study - which most of us are not very good at, even if you're lucky enough to find the study not behind a paywall.
The real test of whether you're susceptible to misinformation, I think, is how you proceed after you read the headline.
posted by Jeanne at 10:15 AM on August 13, 2023 [25 favorites]
A bunch of the questions seemed too inane to be even ascertainable. "Most Americans think it's ok for sports figures to speak out on politics" - how/why is this even a headline? I recognise that I'm failing on some cultural level, that everyone knows sports figures should/should not speak out about political issues but I don't begin to know which it 'should' be or why how I would know. That is, I don't care if a sport figure speaks out on politics but I recognise as they are public figures who operate not only in their sport of choice but also as cultural totems and there are strong feelings when they espouse alignment 'X' but I haven't the faintest idea if 'Most Americans' think this is good or bad or...
So, in conclusion, this was not a test I could complete in any kind of worthwhile way.
posted by From Bklyn at 10:16 AM on August 13, 2023 [6 favorites]
So, in conclusion, this was not a test I could complete in any kind of worthwhile way.
posted by From Bklyn at 10:16 AM on August 13, 2023 [6 favorites]
14/16, once I realized the assignment was to avoid WSJ op-eds
posted by credulous at 10:21 AM on August 13, 2023 [14 favorites]
posted by credulous at 10:21 AM on August 13, 2023 [14 favorites]
Headlines like that come from the following:
1. Some famous sports person says something that conservatives can be performatively outraged about on Fox News
2. They say "shut up, nobody cares about your dumb opinions, get back to the game"
3. Some news organization or pollster puts a question into their next poll: "Is it okay for famous sports people to express opinions"
4. Most respondents do not give a shit and say "sure, it's fine."
5. The poll result is written up as a news article.
You know these headlines aren't fake because nobody would bother writing a headline like that if it weren't true.
posted by BungaDunga at 10:22 AM on August 13, 2023 [7 favorites]
1. Some famous sports person says something that conservatives can be performatively outraged about on Fox News
2. They say "shut up, nobody cares about your dumb opinions, get back to the game"
3. Some news organization or pollster puts a question into their next poll: "Is it okay for famous sports people to express opinions"
4. Most respondents do not give a shit and say "sure, it's fine."
5. The poll result is written up as a news article.
You know these headlines aren't fake because nobody would bother writing a headline like that if it weren't true.
posted by BungaDunga at 10:22 AM on August 13, 2023 [7 favorites]
I don't understand the design of the study - is it just bad? I didn't take the test because I didn't really want to. Whether someone can accurately guess if a headline is real or not is a bad proxy for how susceptible to misinformation they are because people don't judge news stories based on headlines alone, and headlines are not what people who are good at judging the reliability of information rely on.
I mean, to give a very simple example, "30 dead in factory fire" is a totally plausible event, but you're going to put a lot more stock in it if it's reported on CNN, rather than if it's reported on a content farm that is just paraphrasing a viral tweet about a recent disaster.
Conversely, an implausible or inflammatory-sounding headline can be written about real events; we know the bad incentives at play there. For example, "local right-wing activist arrested for child kidnapping" could be true, but then when you read further you find out that it was a custody dispute. Also, there have definitely been cases where I've thought "no way" about a headline, only to read further and discover I was missing some information that made it plausible.
People who are good at avoiding misinformation practice things like lateral reading and critically eyeing the text itself.
Like, this study doesn't seem like it's telling us much about the participants' media literacy and susceptibility to misinformation. It seems to be addressing a related but different question, which is how much our background influences our judgement of an out of context headline.
posted by Kutsuwamushi at 10:24 AM on August 13, 2023 [23 favorites]
I mean, to give a very simple example, "30 dead in factory fire" is a totally plausible event, but you're going to put a lot more stock in it if it's reported on CNN, rather than if it's reported on a content farm that is just paraphrasing a viral tweet about a recent disaster.
Conversely, an implausible or inflammatory-sounding headline can be written about real events; we know the bad incentives at play there. For example, "local right-wing activist arrested for child kidnapping" could be true, but then when you read further you find out that it was a custody dispute. Also, there have definitely been cases where I've thought "no way" about a headline, only to read further and discover I was missing some information that made it plausible.
People who are good at avoiding misinformation practice things like lateral reading and critically eyeing the text itself.
Like, this study doesn't seem like it's telling us much about the participants' media literacy and susceptibility to misinformation. It seems to be addressing a related but different question, which is how much our background influences our judgement of an out of context headline.
posted by Kutsuwamushi at 10:24 AM on August 13, 2023 [23 favorites]
20/20, based mainly on whether or not the phrasing is inflammatory.
Holy cow, that works! I got only 16/20 right originally (a little too skeptical apparently?)
I tried doing the 16 question one without thinking about the actual headlines, just judging on whether it seemed inflammatory, and got them all correct.
I'm not sure what my takeaway from this is, honestly. Mostly I try to RTFA before coming to a conclusion so this was sort of an artificial situation. And honestly I adore salt so pretty much everything I consume has some sprinkled on. The biggest surprise was about Hyatt, I totally got that wrong the first time.
posted by Baethan at 10:32 AM on August 13, 2023 [5 favorites]
Holy cow, that works! I got only 16/20 right originally (a little too skeptical apparently?)
I tried doing the 16 question one without thinking about the actual headlines, just judging on whether it seemed inflammatory, and got them all correct.
I'm not sure what my takeaway from this is, honestly. Mostly I try to RTFA before coming to a conclusion so this was sort of an artificial situation. And honestly I adore salt so pretty much everything I consume has some sprinkled on. The biggest surprise was about Hyatt, I totally got that wrong the first time.
posted by Baethan at 10:32 AM on August 13, 2023 [5 favorites]
20/20. The big tip-offs to me were the ones that referred to "The Government."
posted by Sophocles at 10:33 AM on August 13, 2023 [11 favorites]
posted by Sophocles at 10:33 AM on August 13, 2023 [11 favorites]
20/20. I felt strange because a lot of those sounded like true headlines of fake news.
posted by dances_with_sneetches at 10:35 AM on August 13, 2023 [4 favorites]
posted by dances_with_sneetches at 10:35 AM on August 13, 2023 [4 favorites]
The real test of whether you’re susceptible to misinformation, I think, is how you proceed after you read the headline.
Implicit in the rest of your comment is an assumption that people do proceed in some manner past reading the headline. That would be an optimistic assumption in a large proportion of cases, I think.
posted by eviemath at 10:40 AM on August 13, 2023 [6 favorites]
Implicit in the rest of your comment is an assumption that people do proceed in some manner past reading the headline. That would be an optimistic assumption in a large proportion of cases, I think.
posted by eviemath at 10:40 AM on August 13, 2023 [6 favorites]
20/20, can't really explain my method other than to say that the fake ones just sounded stupid (or implausible, or wildly overstated, or conspiratorial...).
posted by box at 10:44 AM on August 13, 2023 [5 favorites]
posted by box at 10:44 AM on August 13, 2023 [5 favorites]
a low score implies that favorites here are more like "possible favorites but could be co-opted political / socio-political manipulation or asymmetric cultural cyberwarfare type of favorites"
posted by MonsieurPEB at 10:47 AM on August 13, 2023
posted by MonsieurPEB at 10:47 AM on August 13, 2023
I mean, in response to a headline I wasn't sure about, I did research. There was a bit about left-wing lies for certain purposes. It wouldn't be implausible that you could produce research that could lead to that headline. So I looked up, and found actual research tended to lean towards a different category of lies.
Was that against the rules? Because that is how you tell if something is true or not.
posted by NotAYakk at 10:47 AM on August 13, 2023 [7 favorites]
Was that against the rules? Because that is how you tell if something is true or not.
posted by NotAYakk at 10:47 AM on August 13, 2023 [7 favorites]
What this really tests is familiarity with the style of respectable English-language headline writing. I got 20/20 by thinking "does this sound like a newspaper headline in a 'respectable' paper". I mean, most though not quite all of the false ones I got were obviously false (eye color is not ever going to be strongly linked to intelligence; that just isn't going to happen) but fundamentally it was about style. This seems like it's not really about gullibility versus non-gullibility, since of course the NYT writes respectable but fundamentally untrue headlines all the time.
It is in the interests of power to assert that the gullible are those other stupid people off over there who are just, like, stupid or bad, when really it's a question of class and language. Not that it's not a bad thing to believe racialized garbage about eye color, of course; that's a nazi dog whistle and you don't need to be very smart not to go in for nazi dog whistles, you just need to not be a nazi.
posted by Frowner at 10:52 AM on August 13, 2023 [24 favorites]
It is in the interests of power to assert that the gullible are those other stupid people off over there who are just, like, stupid or bad, when really it's a question of class and language. Not that it's not a bad thing to believe racialized garbage about eye color, of course; that's a nazi dog whistle and you don't need to be very smart not to go in for nazi dog whistles, you just need to not be a nazi.
posted by Frowner at 10:52 AM on August 13, 2023 [24 favorites]
I am a nanobot created in a federal government lab and I got 20 out of 20 correct. One half of the previous sentense is a lie and the other half is a fact. If you knew me in real life, you would have to think a very long to time to discern which is which.
posted by NoMich at 11:00 AM on August 13, 2023 [6 favorites]
posted by NoMich at 11:00 AM on August 13, 2023 [6 favorites]
It would be easy to craft a set of headlines that would be a coin toss for everyone.
Even worse, I suspect doing so would invoke some sort of rule 34 for headlines and they would all shortly become real.
posted by Tell Me No Lies at 11:06 AM on August 13, 2023 [3 favorites]
Even worse, I suspect doing so would invoke some sort of rule 34 for headlines and they would all shortly become real.
posted by Tell Me No Lies at 11:06 AM on August 13, 2023 [3 favorites]
I take all the criticisms on board (particularly paimapi's on the quality of the design, although I think there may be answers to that.)
I see criticism in the comments that this is just a test for how inflammatory the phrasing is, or that real headlines and real journalism generally are still full of exaggerations and misinformation.
I think we need to distinguish between "facts", "real news", and "fake news". The test is how well you can discern fake news. Whether when you see a headline floating around on Facebook or Twitter, you are able to discern if it is the product of real journalism, or has been made up for clicks and to entrench extreme views.
That does not mean that "real news" is equivalent to "real facts". Real journalism can still be total bullshit. You shouldn't turn off your brain just because the headline is in a newspaper. But misleading or low quality journalism is a different problem and the test is not assessing your ability to perceive that. Or to put it another way, "Saddam Hussein played a role in 9/11" and "A cabal of Democrats are abusing children under a Washington DC pizza shop" are both bullshit but one became real news because the government provided misinformation to credulous and biased journalists who should have been doing their jobs better. [Stealth edit: I may be confusing "fake news" and "conspiracy theories" here, but you get my point.]
So of course the "fake" headlines are very inflammatory and probably only a minority of people would consistently believe them, while the real ones are bland and reflect a different sort of dangerous brain deadness (economic liberalism, centrism, cultural conformity, etc). But still, the latter is a different problem.
In short, I find it credible that the stupid headlines really are what this particular problem looks like.
posted by Probabilitics at 11:09 AM on August 13, 2023 [9 favorites]
I see criticism in the comments that this is just a test for how inflammatory the phrasing is, or that real headlines and real journalism generally are still full of exaggerations and misinformation.
I think we need to distinguish between "facts", "real news", and "fake news". The test is how well you can discern fake news. Whether when you see a headline floating around on Facebook or Twitter, you are able to discern if it is the product of real journalism, or has been made up for clicks and to entrench extreme views.
That does not mean that "real news" is equivalent to "real facts". Real journalism can still be total bullshit. You shouldn't turn off your brain just because the headline is in a newspaper. But misleading or low quality journalism is a different problem and the test is not assessing your ability to perceive that. Or to put it another way, "Saddam Hussein played a role in 9/11" and "A cabal of Democrats are abusing children under a Washington DC pizza shop" are both bullshit but one became real news because the government provided misinformation to credulous and biased journalists who should have been doing their jobs better. [Stealth edit: I may be confusing "fake news" and "conspiracy theories" here, but you get my point.]
So of course the "fake" headlines are very inflammatory and probably only a minority of people would consistently believe them, while the real ones are bland and reflect a different sort of dangerous brain deadness (economic liberalism, centrism, cultural conformity, etc). But still, the latter is a different problem.
In short, I find it credible that the stupid headlines really are what this particular problem looks like.
posted by Probabilitics at 11:09 AM on August 13, 2023 [9 favorites]
people who get their news from social media perform worse than people who do not.
For some years legacy media has been losing traction. In Canada, our federal government has decided that part of the problem is links to news stories on social media and so put in place a fee for social media platforms to pay whenever they link to a news story; in response, social media has announced that there will no longer be links to news stories.
I’m not sure the electorate will be better informed when their grasp of an issue is formed not by a Globe and Mail piece but by what their Uncle Randy heard from a buddy.
posted by ricochet biscuit at 11:13 AM on August 13, 2023 [3 favorites]
For some years legacy media has been losing traction. In Canada, our federal government has decided that part of the problem is links to news stories on social media and so put in place a fee for social media platforms to pay whenever they link to a news story; in response, social media has announced that there will no longer be links to news stories.
I’m not sure the electorate will be better informed when their grasp of an issue is formed not by a Globe and Mail piece but by what their Uncle Randy heard from a buddy.
posted by ricochet biscuit at 11:13 AM on August 13, 2023 [3 favorites]
Looking forward to thanksgiving when crazy uncle bob tells us that he has both a 20/20 on the fake news quiz and has a shipping container of ivermectin ready for the end-times which are coming as soon as drumpf gets convicted.
posted by kaibutsu at 11:16 AM on August 13, 2023
posted by kaibutsu at 11:16 AM on August 13, 2023
I felt strange because a lot of those sounded like true headlines of fake news.
Which I think is considered a fake headline for the purposes of this study. If you're calling it fake news, and it has a headline, then that is also a fake headline. I think they muddy the water by mixing "real headlines" with the chatgpt stuff. If they'd used actual fake news headlines instead, the distinction would be clearer and less confusing.
posted by Dysk at 11:25 AM on August 13, 2023 [4 favorites]
Which I think is considered a fake headline for the purposes of this study. If you're calling it fake news, and it has a headline, then that is also a fake headline. I think they muddy the water by mixing "real headlines" with the chatgpt stuff. If they'd used actual fake news headlines instead, the distinction would be clearer and less confusing.
posted by Dysk at 11:25 AM on August 13, 2023 [4 favorites]
I was shocked to score 100%. There were a could that could have gone either way. But I'm posting a gold star for myself anyway because I never do well with online quizzes.
posted by cccorlew at 11:32 AM on August 13, 2023 [1 favorite]
posted by cccorlew at 11:32 AM on August 13, 2023 [1 favorite]
20/20 and agree that a good tip off is whether it's click-baity.
posted by CheeseDigestsAll at 11:34 AM on August 13, 2023
posted by CheeseDigestsAll at 11:34 AM on August 13, 2023
15/16 and 20/20. Mostly I refused headlines which looked conspiratorial.
posted by doctornemo at 11:39 AM on August 13, 2023
posted by doctornemo at 11:39 AM on August 13, 2023
Got 18/20 basically just by distinguishing between what I judged to be or not be "headline length."
posted by kensington314 at 11:41 AM on August 13, 2023 [1 favorite]
posted by kensington314 at 11:41 AM on August 13, 2023 [1 favorite]
19/20
But yeah, this seems like a silly way to do this study. I feel like a better metric would have been to give people more options: a) definitely real, b) could be real, or could be fake c) definitely fake. I don't think you're susceptible to fake news if your response is at least "could be fake - better look into whatever their sources are"
posted by coffeecat at 11:44 AM on August 13, 2023 [2 favorites]
But yeah, this seems like a silly way to do this study. I feel like a better metric would have been to give people more options: a) definitely real, b) could be real, or could be fake c) definitely fake. I don't think you're susceptible to fake news if your response is at least "could be fake - better look into whatever their sources are"
posted by coffeecat at 11:44 AM on August 13, 2023 [2 favorites]
19/20; one I rated as fake was actually real. I’m now thinking I could see this kind of method becoming “use internet ‘quiz’ to train next generation LLM to generate more convincing fake news.”
posted by eirias at 11:47 AM on August 13, 2023 [11 favorites]
posted by eirias at 11:47 AM on August 13, 2023 [11 favorites]
The big tip-offs to me were the ones that referred to "The Government."
Counterpoint:
“Feed Your Mind is our new education initiative to help consumers better understand genetically engineered foods, commonly called GMOs or genetically modified organisms.”
I’m guessing this was one of the two I got ‘wrong’…
posted by soy bean at 11:47 AM on August 13, 2023
Counterpoint:
“Feed Your Mind is our new education initiative to help consumers better understand genetically engineered foods, commonly called GMOs or genetically modified organisms.”
I’m guessing this was one of the two I got ‘wrong’…
posted by soy bean at 11:47 AM on August 13, 2023
20/20. I surprised myself.
Yeah - not counting the ones that were preposterous, I too was cued by how much emotional, inflammatory, controlling spin there was in the headline.
I resisted the urge to Google some of the ones I was unsure of. I hope that's evidence of a useful skepticism in media consumption: the willingness to question and dig into a statement before passing judgement.
posted by Artful Codger at 11:49 AM on August 13, 2023
Yeah - not counting the ones that were preposterous, I too was cued by how much emotional, inflammatory, controlling spin there was in the headline.
I resisted the urge to Google some of the ones I was unsure of. I hope that's evidence of a useful skepticism in media consumption: the willingness to question and dig into a statement before passing judgement.
posted by Artful Codger at 11:49 AM on August 13, 2023
This test is bogus.
Which, as far as I'm concerned, is the real test.
posted by AlSweigart at 11:52 AM on August 13, 2023 [8 favorites]
Which, as far as I'm concerned, is the real test.
posted by AlSweigart at 11:52 AM on August 13, 2023 [8 favorites]
imho they should have thrown in some falsities that have a more "liberal" valence, like "Flint still doesn't have clean water" (it does actually) or "GMOs proven to destroy gut health" or whatever
posted by BungaDunga at 11:54 AM on August 13, 2023 [10 favorites]
posted by BungaDunga at 11:54 AM on August 13, 2023 [10 favorites]
18/20, having missed one of each. They were both things that, had I come across them and been interested in reading them, I would likely have looked for additional sources.
posted by The Great Big Mulp at 11:55 AM on August 13, 2023 [2 favorites]
posted by The Great Big Mulp at 11:55 AM on August 13, 2023 [2 favorites]
(to follow up, there should be more liberal ones in there not because there's just as much lefty fake news, but because you're trying to diagnose whether people can notice fake news, and if people are just more skeptical of fake news when it is against their own ideology, you should have a mix of both, otherwise you could have a bunch of liberal-but-gullible people score 100% just because all the fake news headlines are "the liberal media are infecting the water supply")
posted by BungaDunga at 11:59 AM on August 13, 2023 [5 favorites]
posted by BungaDunga at 11:59 AM on August 13, 2023 [5 favorites]
18/20, and I'm pretty sure I know which one I missed; which I took as slanted as a headline but not 'fake news.'
Veracity Discernment: 80% (ability to accurately distinguish real news from fake news)
Real News Detection: 90% (ability to correctly identify real news)
Fake News Detection: 90% (ability to correctly identify fake news)
Distrust/Naïvité: 0 (ranges from -10 to +10, overly skeptical to overly gullible)
posted by snuffleupagus at 12:03 PM on August 13, 2023
Veracity Discernment: 80% (ability to accurately distinguish real news from fake news)
Real News Detection: 90% (ability to correctly identify real news)
Fake News Detection: 90% (ability to correctly identify fake news)
Distrust/Naïvité: 0 (ranges from -10 to +10, overly skeptical to overly gullible)
posted by snuffleupagus at 12:03 PM on August 13, 2023
I was afraid to take it, but no one would know how I did, so…. 20/20…. I just sort of whipped through it, with discernment based on similarity to certain kinds of thought. I guess that worked.
posted by njohnson23 at 12:15 PM on August 13, 2023
posted by njohnson23 at 12:15 PM on August 13, 2023
19/20, my rule of thumb was "could this be a plausible, though possibly misleading, headline to a factually true story". The one which bit me was "Government Officials Have Manipulated Stock Prices to Hide Scandals", which I thought was unlikely, but might be a technically true story somewhere for slippery enough definitions of "Government Officials", "Manipulated Stock Prices", and "Hide Scandals", though I should have gone with my gut.
For those who are wondering which they got wrong the test will let you scroll back up and change your answers and resubmit to see your new score.
posted by Reverend John at 12:19 PM on August 13, 2023 [8 favorites]
For those who are wondering which they got wrong the test will let you scroll back up and change your answers and resubmit to see your new score.
posted by Reverend John at 12:19 PM on August 13, 2023 [8 favorites]
For the 16 question quiz I took, I had the same thoughts about needing more leftist "fake news" but when I went back and looked there were at least three and maybe more:
One was a conclusive statement that the US government was responsible for 9/11, second one was about the government manipulating stock prices, and third was that "they" are poisoning our air and water. I spotted all the "fake news" from a lefty perspective but on first glance it felt value-neutral to me.
posted by muddgirl at 12:22 PM on August 13, 2023 [2 favorites]
One was a conclusive statement that the US government was responsible for 9/11, second one was about the government manipulating stock prices, and third was that "they" are poisoning our air and water. I spotted all the "fake news" from a lefty perspective but on first glance it felt value-neutral to me.
posted by muddgirl at 12:22 PM on August 13, 2023 [2 favorites]
The people who got 20/20: was the "left wingers lie to get a better salary" real or fake?
The framing and wording seemed reactionary so I said fake, but I would believe leftists think lying to the boss is cooler than the bootlickers do.
One was a conclusive statement that the US government was responsible for 9/11
I would honestly need to see how they defined "responsible" before passing judgment on the headline.
posted by Reyturner at 12:38 PM on August 13, 2023
The framing and wording seemed reactionary so I said fake, but I would believe leftists think lying to the boss is cooler than the bootlickers do.
One was a conclusive statement that the US government was responsible for 9/11
I would honestly need to see how they defined "responsible" before passing judgment on the headline.
posted by Reyturner at 12:38 PM on August 13, 2023
The people who got 20/20: was the "left wingers lie to get a better salary" real or fake?
Fake.
The one I was dubious about was 'Reflecting a Demographic Shift, 109 US Counties Have Become Majority Nonwhite Since 2000'. Called it correctly as real in the end though.
posted by MattWPBS at 12:45 PM on August 13, 2023 [2 favorites]
Fake.
The one I was dubious about was 'Reflecting a Demographic Shift, 109 US Counties Have Become Majority Nonwhite Since 2000'. Called it correctly as real in the end though.
posted by MattWPBS at 12:45 PM on August 13, 2023 [2 favorites]
109 counties didn't seem like too many. So yeah, real.
posted by philip-random at 12:47 PM on August 13, 2023 [3 favorites]
posted by philip-random at 12:47 PM on August 13, 2023 [3 favorites]
I vaguely clicked on things and got 15 out of 16, to my surprise. But then I mostly don't believe headlines because they are written by headline writers and don't actually reflect what is reported in the article or (as anyone knows who has ever been interviewed for a profile or who knows anything about the topic of an article) actual reality.
posted by Peach at 12:53 PM on August 13, 2023
posted by Peach at 12:53 PM on August 13, 2023
18/20, and I really flipflopped over the two I missed.
Given the crap headlines and outright lies in major media*, sometimes I'm not sure exactly how to rate the 'truthiness' of a headline any more.
*looking at you NYT
posted by BlueHorse at 1:02 PM on August 13, 2023
Given the crap headlines and outright lies in major media*, sometimes I'm not sure exactly how to rate the 'truthiness' of a headline any more.
*looking at you NYT
posted by BlueHorse at 1:02 PM on August 13, 2023
The easiest way to get someone to believe your fake news is to write it such that it panders to their inner sense of identity.
I didn't get the feeling that any of those headlines did that for the typical MeFier, which made it easy for us.
Exactly right.
posted by praemunire at 1:10 PM on August 13, 2023 [6 favorites]
I didn't get the feeling that any of those headlines did that for the typical MeFier, which made it easy for us.
Exactly right.
posted by praemunire at 1:10 PM on August 13, 2023 [6 favorites]
19/20 - I missed one fake news. Not bad, all things considered, given that I like to *read* the story rather than just go off the headline, so the 20th one likely would have tipped its hand before long. I agree that about 10-12 were very clear (either dispassionate facts or charged language that sounds like the stuff my father reads daily), which made a good chunk of the task much easier.
posted by mystyk at 1:16 PM on August 13, 2023
posted by mystyk at 1:16 PM on August 13, 2023
20/20 - 62 year old extreme liberal.
posted by Grumpy old geek at 1:26 PM on August 13, 2023
posted by Grumpy old geek at 1:26 PM on August 13, 2023
I Would honestly need to see how they defined "responsible" before passing judgment on the headline.
That's pretty much what makes it a fake news headline.
posted by muddgirl at 1:45 PM on August 13, 2023 [3 favorites]
That's pretty much what makes it a fake news headline.
posted by muddgirl at 1:45 PM on August 13, 2023 [3 favorites]
News: whistleblower reveals ties between Bush administration and Saudi royal implicated in funding 9/11 attacks
Fake News: Bush caused 9/11.
posted by muddgirl at 2:01 PM on August 13, 2023 [1 favorite]
Fake News: Bush caused 9/11.
posted by muddgirl at 2:01 PM on August 13, 2023 [1 favorite]
I got a good score but some of that is true, you're just not supposed to say it.
posted by kingdead at 2:39 PM on August 13, 2023
posted by kingdead at 2:39 PM on August 13, 2023
19/20.
The one I got "wrong" had to be the one about mainstream media companies being controlled or manipulated by oil companies and corporate interests. The rest were easy.
Strongly disagree with that one. Main stream news has definitely been manipulated by major corporations and governments, especially by fossil fuel and other extractive industries and companies. This isn't some random conspiracy theory, this has been repeatedly investigated and buried and re-buried multiple times about how much corporations and "sponsors" can influence the news.
And I knew the test was going to ding me for it and I still answered it "wrong" anyway because: fuck you!
posted by loquacious at 2:44 PM on August 13, 2023 [5 favorites]
The one I got "wrong" had to be the one about mainstream media companies being controlled or manipulated by oil companies and corporate interests. The rest were easy.
Strongly disagree with that one. Main stream news has definitely been manipulated by major corporations and governments, especially by fossil fuel and other extractive industries and companies. This isn't some random conspiracy theory, this has been repeatedly investigated and buried and re-buried multiple times about how much corporations and "sponsors" can influence the news.
And I knew the test was going to ding me for it and I still answered it "wrong" anyway because: fuck you!
posted by loquacious at 2:44 PM on August 13, 2023 [5 favorites]
Not sure I can get behind "skeptical" and "gullible" as antonyms; I think it's quite easy to be too skeptical and too gullible.
posted by aws17576 at 2:52 PM on August 13, 2023 [5 favorites]
posted by aws17576 at 2:52 PM on August 13, 2023 [5 favorites]
My Bullshit detector is at 100%. I'm a little skeptical about some real headlines, 80% (I reserve the right to read the rest of the article, maybe do some checking, before I make up my mind).
posted by mule98J at 2:52 PM on August 13, 2023
posted by mule98J at 2:52 PM on August 13, 2023
I got 20/20 because I googled each headline to see whether it was real or not, which is the legit way to catch fake news, not by assuming that I'm too smart to get fooled.
posted by straight at 2:57 PM on August 13, 2023 [5 favorites]
posted by straight at 2:57 PM on August 13, 2023 [5 favorites]
Looking at the YouGov split it's really obvious in retrospect; it was really between categories of "plausible news" and and "inflammatory headlines to fake stories."
The one I missed was "One-in-Three Worldwide Lack Confidence in Non-Governmental Organizations," which I found perfectly plausible but treated as a factual question and guessed wildly that it was wrong. I mean, I knew some number of people lacked confidence in NGOs, I just didn't care much what that was and so just guessed it was either lower, higher, or never reported on. There were three or for in this category, I thought, that seemed like factual questions rather than plausibility questions.
The one I got "wrong" had to be the one about mainstream media companies being controlled or manipulated by oil companies and corporate interests. The rest were easy.
Strongly disagree with that one. Main stream news has definitely been manipulated by major corporations and government . . .
The actual fake headline was "The Corporate Media Is Controlled by the Military-Industrial Complex: The Major Oil Companies Own the Media and Control Their Agenda." It wasn't about manipulation or influence (which is a different standard.)
Going to say there can't be a factually correct news piece that says this. There can be an Op-Ed with this headline that you might agree with, there've been leftist arguments (with a lot of validity!) like this since well before Chomsky.
But in news terms for this to be a headline for a true story it would be like Exxon and its competitors had majority ownership of the NYT, Washington Post, CNN, Fox, and so on; that it had been exercising editorial control; and the documents and testimony to this effect were now uncovered.
posted by mark k at 2:59 PM on August 13, 2023 [5 favorites]
The one I missed was "One-in-Three Worldwide Lack Confidence in Non-Governmental Organizations," which I found perfectly plausible but treated as a factual question and guessed wildly that it was wrong. I mean, I knew some number of people lacked confidence in NGOs, I just didn't care much what that was and so just guessed it was either lower, higher, or never reported on. There were three or for in this category, I thought, that seemed like factual questions rather than plausibility questions.
The one I got "wrong" had to be the one about mainstream media companies being controlled or manipulated by oil companies and corporate interests. The rest were easy.
Strongly disagree with that one. Main stream news has definitely been manipulated by major corporations and government . . .
The actual fake headline was "The Corporate Media Is Controlled by the Military-Industrial Complex: The Major Oil Companies Own the Media and Control Their Agenda." It wasn't about manipulation or influence (which is a different standard.)
Going to say there can't be a factually correct news piece that says this. There can be an Op-Ed with this headline that you might agree with, there've been leftist arguments (with a lot of validity!) like this since well before Chomsky.
But in news terms for this to be a headline for a true story it would be like Exxon and its competitors had majority ownership of the NYT, Washington Post, CNN, Fox, and so on; that it had been exercising editorial control; and the documents and testimony to this effect were now uncovered.
posted by mark k at 2:59 PM on August 13, 2023 [5 favorites]
The Morocco one got me. Knowing very little about the country, I reckoned if they're that concerned with inequality why do they have a king who is actually involved in governance to the extent of appointing committees?
posted by juv3nal at 3:09 PM on August 13, 2023 [1 favorite]
posted by juv3nal at 3:09 PM on August 13, 2023 [1 favorite]
20/20, but I used to be the one writing the headlines at our paper.
posted by ricochet biscuit at 3:11 PM on August 13, 2023 [1 favorite]
posted by ricochet biscuit at 3:11 PM on August 13, 2023 [1 favorite]
The headlines and their veracity are at :
https://osf.io/pqguh
posted by llc at 3:13 PM on August 13, 2023 [2 favorites]
https://osf.io/pqguh
posted by llc at 3:13 PM on August 13, 2023 [2 favorites]
20/20. But then again, I am a MeFite.
posted by Faint of Butt at 3:49 PM on August 13, 2023
posted by Faint of Butt at 3:49 PM on August 13, 2023
This is a BuzzFeed quiz with academic flair. Asking a question like this about terms with highly contested meanings without a thorough definition of terms, you can structure your questions to get any answers you want. I got 19/20 asking myself “What would a centrist think?” Also, if the most left position in your survey is “extremely liberal,” without an “other” option, you have completely sacrificed your credibility in terms of accuracy.
posted by vim876 at 5:26 PM on August 13, 2023 [5 favorites]
posted by vim876 at 5:26 PM on August 13, 2023 [5 favorites]
Well, damn! (Spoiler alert):
Hyatt really is getting rid of little plastic bottles!
posted by TedW at 6:04 PM on August 13, 2023 [6 favorites]
Hyatt really is getting rid of little plastic bottles!
posted by TedW at 6:04 PM on August 13, 2023 [6 favorites]
"The Government Is Conducting a Massive Cover-Up of Their Involvement in 9/11"
After nearly 20 years, however, the FBI has changed its story (ProPublica)
posted by credulous at 6:12 PM on August 13, 2023
After nearly 20 years, however, the FBI has changed its story (ProPublica)
posted by credulous at 6:12 PM on August 13, 2023
📈 Your MIST-20 results: 17/20
Veracity Discernment: 70% (ability to accurately distinguish real news from fake news)
Real News Detection: 70% (ability to correctly identify real news)
Fake News Detection: 100% (ability to correctly identify fake news)
Maybe I missed the intent, but I rated anything I would fact check as "fake". There were a few (I guess three) I felt were real headlines, but I would fact check if I had read them as a matter of course during my day.
posted by The Power Nap at 6:41 PM on August 13, 2023 [2 favorites]
Veracity Discernment: 70% (ability to accurately distinguish real news from fake news)
Real News Detection: 70% (ability to correctly identify real news)
Fake News Detection: 100% (ability to correctly identify fake news)
Maybe I missed the intent, but I rated anything I would fact check as "fake". There were a few (I guess three) I felt were real headlines, but I would fact check if I had read them as a matter of course during my day.
posted by The Power Nap at 6:41 PM on August 13, 2023 [2 favorites]
I was 16/16, but I am in fact rather skeptical about the quality of this survey. A lot of them seemed like they could go other ways -- Hyatt could be removing small bottles (presumably in favor of large dispensers to have less waste), but then again they could be not doing that. There's nothing inherent about that headline that clues you into whether it is fake news or not.
posted by tavella at 8:59 PM on August 13, 2023 [1 favorite]
posted by tavella at 8:59 PM on August 13, 2023 [1 favorite]
I got 20/20, despite the fact that I agree with the military industrial complex headline. Haha
The only one that was hard was the one about 1/3 not having confidence in NGOs because its obviously related to covid and the WHO etc so it could have gone either way based on the framing.
posted by dry white toast at 9:16 PM on August 13, 2023
The only one that was hard was the one about 1/3 not having confidence in NGOs because its obviously related to covid and the WHO etc so it could have gone either way based on the framing.
posted by dry white toast at 9:16 PM on August 13, 2023
I think the clue on the Hyatt one was its matter-of-factness. The headline as given is just a statement about a thing that's ostensibly happening, and doesn't make any sense as a thing to lie about. The fake news version would be "Hyatt is removing small bottles from rooms due to woke" or something else designed to provoke a reaction.
posted by rifflesby at 9:21 PM on August 13, 2023 [2 favorites]
posted by rifflesby at 9:21 PM on August 13, 2023 [2 favorites]
People who read MeFi are not the ones who should be taking this test. Those who would benefit from testing their ability to recognize bogus information are very unlikely to be exposed to this resource.
posted by Metacircular at 9:37 PM on August 13, 2023
posted by Metacircular at 9:37 PM on August 13, 2023
I scored 100% but I don't think this quiz was presented well. I may be resilient to fake headlines but I don't think that's the hard part. Simply being able to spot clickbait is only part of the problem. Training AI to write more convincing headlines isn't really doing the work of inoculating readers from consuming bad analysis or badly sourced info. I feel like this quiz was flattering to skeptical news consumers like me while not really addressing the issue of bias, or even bad journalism, and i'm just a random idiot
posted by biddeford at 11:20 PM on August 13, 2023
posted by biddeford at 11:20 PM on August 13, 2023
The easiest way to get someone to believe your fake news is to write it such that it panders to their inner sense of identity.
This has LONG been my internal bullshit detecting algorithm. Does it confirm my worst thoughts about the people I disagree with? Do I want to scream "of course they would do that?" Then I have to stop myself and ask some questions. 9/10, its fake (or misrepresented enough it might as well be.)
LONG before fake news was Fake News, I'd call out facebook friends for their fake news, even if we were politically aligned. Well actually, more when we were politically aligned.
A friend posted some story skewering some right wing politician for something related to abusing a dog. And it stunk of "not only is he a republican, but he tortures dogs too!" I don't remember why it struck me as fake, but I went, did some digging, and sure enough, what happened was awful but not his fault.
So I presented the information to my friend. I'll never forget his response. "Well it might as well be true, he would do something so awful."
And that day I learned that it doesn't matter that its not real, the truthiness of it will win for those that want to believe. We can argue until we're blue in the face about the best way to help people discriminate real from fake information and its not the problem. People are just looking for information that confirms their biases.
(17/20 and 16/16 - the first time I was TOO cynical, marking 3 real headlines as fake.)
posted by [insert clever name here] at 11:31 PM on August 13, 2023 [5 favorites]
This has LONG been my internal bullshit detecting algorithm. Does it confirm my worst thoughts about the people I disagree with? Do I want to scream "of course they would do that?" Then I have to stop myself and ask some questions. 9/10, its fake (or misrepresented enough it might as well be.)
LONG before fake news was Fake News, I'd call out facebook friends for their fake news, even if we were politically aligned. Well actually, more when we were politically aligned.
A friend posted some story skewering some right wing politician for something related to abusing a dog. And it stunk of "not only is he a republican, but he tortures dogs too!" I don't remember why it struck me as fake, but I went, did some digging, and sure enough, what happened was awful but not his fault.
So I presented the information to my friend. I'll never forget his response. "Well it might as well be true, he would do something so awful."
And that day I learned that it doesn't matter that its not real, the truthiness of it will win for those that want to believe. We can argue until we're blue in the face about the best way to help people discriminate real from fake information and its not the problem. People are just looking for information that confirms their biases.
(17/20 and 16/16 - the first time I was TOO cynical, marking 3 real headlines as fake.)
posted by [insert clever name here] at 11:31 PM on August 13, 2023 [5 favorites]
19/20. I got the Hyatt™ one wrong because I assumed that they were talking about the liquor bottles in the damn minbar. My reaction was that no one is going to pay $400 for a quart of rum, and how freaking big would that fridge need to be to offer any kind of selection?
"Why yes, my liver is the size of a rugby ball, why do you ask?"
posted by PareidoliaticBoy at 12:12 AM on August 14, 2023 [1 favorite]
"Why yes, my liver is the size of a rugby ball, why do you ask?"
posted by PareidoliaticBoy at 12:12 AM on August 14, 2023 [1 favorite]
19/20, immediately knew which one I'd got wrong (Government Officials Have Manipulated Stock Prices to Hide Scandals), went back and corrected that and got my 100%.
The only reason I initially rated that one as factual is that I know for sure that there are government officials who have engaged in insider trading, and would not find it super surprising if hiding scandal was the motivation for some of that. So instead of applying the simple test that works for all the other questions (i.e. if this headline is something I've heard a conservative say, it's fake) I thought I'd try to counteract my own clear bias and give it the benefit of the doubt.
"Reality has a left wing bias": real headline or fake news? Shoulda trusted my gut.
After getting that score, went back and nailed the 16 question test, which was made easier by a lot of questions already having appeared on the 20 question version.
posted by flabdablet at 12:15 AM on August 14, 2023 [1 favorite]
The only reason I initially rated that one as factual is that I know for sure that there are government officials who have engaged in insider trading, and would not find it super surprising if hiding scandal was the motivation for some of that. So instead of applying the simple test that works for all the other questions (i.e. if this headline is something I've heard a conservative say, it's fake) I thought I'd try to counteract my own clear bias and give it the benefit of the doubt.
"Reality has a left wing bias": real headline or fake news? Shoulda trusted my gut.
After getting that score, went back and nailed the 16 question test, which was made easier by a lot of questions already having appeared on the 20 question version.
posted by flabdablet at 12:15 AM on August 14, 2023 [1 favorite]
People are just looking for information that confirms their biases.
That is so, but it also remains the case that having a left bias rather than a right bias puts you closer to the truth more of the time.
Given that most media organizations are run by billionaires for billionaires, I would also expect a left bias to outperform an absence of bias for the purpose of genuine truth-seeking. But perhaps that's just me showing my left bias.
posted by flabdablet at 12:23 AM on August 14, 2023
That is so, but it also remains the case that having a left bias rather than a right bias puts you closer to the truth more of the time.
Given that most media organizations are run by billionaires for billionaires, I would also expect a left bias to outperform an absence of bias for the purpose of genuine truth-seeking. But perhaps that's just me showing my left bias.
posted by flabdablet at 12:23 AM on August 14, 2023
Also, if the most left position in your survey is “extremely liberal,” without an “other” option, you have completely sacrificed your credibility in terms of accuracy.
Yup.
posted by busted_crayons at 12:41 AM on August 14, 2023 [3 favorites]
Yup.
posted by busted_crayons at 12:41 AM on August 14, 2023 [3 favorites]
Where's the test where we read twenty articles and perform a similar discrimination
posted by DeepSeaHaggis at 1:04 AM on August 14, 2023 [2 favorites]
posted by DeepSeaHaggis at 1:04 AM on August 14, 2023 [2 favorites]
Is it good to know that I'm more resilient to misinformation than 95% of the UK population, or is it bad because I still have to live with the 95% of people who are less resilient to it?
posted by terretu at 2:20 AM on August 14, 2023 [2 favorites]
posted by terretu at 2:20 AM on August 14, 2023 [2 favorites]
I did the 16 question one (because lazy), and got 15/16 💅🏻, but the results didn't tell me which one I got wrong. My question is do we know where the pool of "1516 U.S. Adult citizens" in the original poll / study came from? I didn't see that in the YouGov survey, and it seems like volunteers for this would already be somewhat set apart from the total population by their interest in the premise. (I mean, I've seen some "man-on-the-street" interviews where the awareness of current (or past!) events is beyond abysmal, and there's definitely a chunk of people who aren't believing or disbelieving news stories because they aren't consuming any non-pop culture news, as far as I can tell.)
posted by taz at 2:26 AM on August 14, 2023
posted by taz at 2:26 AM on August 14, 2023
The test won't load for me. But I still learnt something:
BungaDunga: imho they should have thrown in some falsities that have a more "liberal" valence, like "Flint still doesn't have clean water" (it does actually) or "GMOs proven to destroy gut health" or whatever
Huh. Living in Australia, I hadn't actually heard the Flint water crisis was resolved. But apparently Flint has had clean water since February 2019, although many of the residents didn't trust the water, despite it being tested safe.
posted by davidwitteveen at 2:49 AM on August 14, 2023 [1 favorite]
BungaDunga: imho they should have thrown in some falsities that have a more "liberal" valence, like "Flint still doesn't have clean water" (it does actually) or "GMOs proven to destroy gut health" or whatever
Huh. Living in Australia, I hadn't actually heard the Flint water crisis was resolved. But apparently Flint has had clean water since February 2019, although many of the residents didn't trust the water, despite it being tested safe.
posted by davidwitteveen at 2:49 AM on August 14, 2023 [1 favorite]
I am finding it interesting that a lot of the errors made by Mefites appear to have been caused by overthinking the premise.
posted by kyrademon at 4:38 AM on August 14, 2023 [6 favorites]
posted by kyrademon at 4:38 AM on August 14, 2023 [6 favorites]
I have many many many family members and FB friends I need to do this... ASAP...
posted by rozcakj at 6:33 AM on August 14, 2023
posted by rozcakj at 6:33 AM on August 14, 2023
Left-Wingers Can Be Paralyzed By Bean-Plating, Reports UN Committee
posted by fantabulous timewaster at 8:45 AM on August 14, 2023 [6 favorites]
posted by fantabulous timewaster at 8:45 AM on August 14, 2023 [6 favorites]
I was a teaching assistant for a history professor; every year he gave a 100 question text (that I would get to grade) on European history. Every question (all 100!) were false, but students would consistently get only 10-20% correct - much worse that just guessing or randomization. Why? I've thought about this for years, and I think the answer is that it is the internalization of test taking experiences and that most T/F tests have some of each answer. For this media survey - aren't we inclined to think most media stories are at least somewhat trueful, not totally BS? But in an age when ET is the subject of a congressional hearing, what is not at least within the realm of the possible absurd reality in which we are living?
posted by mfoight at 9:18 AM on August 14, 2023
posted by mfoight at 9:18 AM on August 14, 2023
19/20, my rule of thumb was "could this be a plausible, though possibly misleading, headline to a factually true story". The one which bit me was "Government Officials Have Manipulated Stock Prices to Hide Scandals", which I thought was unlikely, but might be a technically true story somewhere for slippery enough definitions of "Government Officials", "Manipulated Stock Prices", and "Hide Scandals", though I should have gone with my gut.
I bet if it had "alleged to" after "Government Officials" they'd put it on the true side. Frowner's point about this being more about recognizing respectability shibboleths than actually gauging truth is well taken.
posted by Lentrohamsanin at 10:48 AM on August 14, 2023 [2 favorites]
I bet if it had "alleged to" after "Government Officials" they'd put it on the true side. Frowner's point about this being more about recognizing respectability shibboleths than actually gauging truth is well taken.
posted by Lentrohamsanin at 10:48 AM on August 14, 2023 [2 favorites]
apparently Flint has had clean water since February 2019
One thing that's always bothered me about the whole Flint debacle is how the fuck lead pipe ever found its way into any reticulated water supply constructed since the fall of the Roman Empire.
Yes, the Latin name for lead is "plumbum" and yes, that's the root of the word "plumbing" but ancient Romans using lead to build their fabled water supplies has been held up as the canonical example of a tragic public infrastructure mistake since my grandparents were tiny children in primary school, and my grandmother's school years pre-dated cars, pretty much.
So I went and looked it up, and it turns out that the answer is the same as why the tobacco and fossil fuel extraction industries are still workable moneymakers: it's the commercial public relations ghouls. Again.
A Very Brief History of Lead in Water Supplies
PR as a practice is all about the systematic cultivation of gullibility; PR is to gullibility as agriculture is to food. I object to having my gullibility cultivated, which is why I will generally treat any Things Are All Fixed Now press coverage as motivated horseshit until it's demonstrably, reliably, definitively shown to be otherwise.
If Flint's water supply is really OK now, then great! But I'm not about to argue the case with any Flint resident who acts as if it isn't.
posted by flabdablet at 12:17 PM on August 14, 2023 [3 favorites]
One thing that's always bothered me about the whole Flint debacle is how the fuck lead pipe ever found its way into any reticulated water supply constructed since the fall of the Roman Empire.
Yes, the Latin name for lead is "plumbum" and yes, that's the root of the word "plumbing" but ancient Romans using lead to build their fabled water supplies has been held up as the canonical example of a tragic public infrastructure mistake since my grandparents were tiny children in primary school, and my grandmother's school years pre-dated cars, pretty much.
So I went and looked it up, and it turns out that the answer is the same as why the tobacco and fossil fuel extraction industries are still workable moneymakers: it's the commercial public relations ghouls. Again.
A Very Brief History of Lead in Water Supplies
PR as a practice is all about the systematic cultivation of gullibility; PR is to gullibility as agriculture is to food. I object to having my gullibility cultivated, which is why I will generally treat any Things Are All Fixed Now press coverage as motivated horseshit until it's demonstrably, reliably, definitively shown to be otherwise.
If Flint's water supply is really OK now, then great! But I'm not about to argue the case with any Flint resident who acts as if it isn't.
posted by flabdablet at 12:17 PM on August 14, 2023 [3 favorites]
Actually, per my snarky response above and in light that some of these were GPT-generated headlines, where *is* the test with full articles to perform this task?
posted by DeepSeaHaggis at 2:08 PM on August 14, 2023 [1 favorite]
posted by DeepSeaHaggis at 2:08 PM on August 14, 2023 [1 favorite]
There are no full articles. This was the test. That's the whole thing.
Practically speaking, I imagine it would apply to a news aggregator like the default page Edge shows me if I open it up. Lots of headlines from different sources, all trying to get me to click. If I just scan things which headlines should I immediately discount? The ones that are like those in the "fake headline" examples.
It is a weird test, and my opinion of it has gone downhill since my first comment here. But I think it gets at something. Lots of people skim headlines and things get half-remembered. What you want is to give this shit zero credence at all, right away. Even if it's part real, it will get published in a place that does an honest headline.
posted by mark k at 4:45 PM on August 14, 2023 [1 favorite]
Practically speaking, I imagine it would apply to a news aggregator like the default page Edge shows me if I open it up. Lots of headlines from different sources, all trying to get me to click. If I just scan things which headlines should I immediately discount? The ones that are like those in the "fake headline" examples.
It is a weird test, and my opinion of it has gone downhill since my first comment here. But I think it gets at something. Lots of people skim headlines and things get half-remembered. What you want is to give this shit zero credence at all, right away. Even if it's part real, it will get published in a place that does an honest headline.
posted by mark k at 4:45 PM on August 14, 2023 [1 favorite]
It is a weird test, and my opinion of it has gone downhill since my first comment here.
It is a weird test, and I'm starting to suspect that by immediately leaping at the chance to prove how clever and worldly I am, I have in fact just failed a disinformation susceptibility test.
Opportunities for self-congratulation make tasty baits; I would not be astonished to find them laid at the mouths of many rabbit holes dug by rabbits who ain't like no rabbits I ever seen.
posted by flabdablet at 3:00 AM on August 15, 2023
It is a weird test, and I'm starting to suspect that by immediately leaping at the chance to prove how clever and worldly I am, I have in fact just failed a disinformation susceptibility test.
Opportunities for self-congratulation make tasty baits; I would not be astonished to find them laid at the mouths of many rabbit holes dug by rabbits who ain't like no rabbits I ever seen.
Here's one I prepared earlier.
Question 1: Don't think about Kate Bush.posted by flabdablet at 3:00 AM on August 15, 2023
I got 17/20, recognized all the real ones but also called three fake ones real.
I dunno, it's a cute game but I really don't feel like I'm applying the same skills here I do when I am actually critical with my media consumption. Whether a story has merit or not is not something you get from the headline. This is just kind of a really abstract fact-checking exercise.
posted by mellow seas at 7:19 AM on August 15, 2023
I dunno, it's a cute game but I really don't feel like I'm applying the same skills here I do when I am actually critical with my media consumption. Whether a story has merit or not is not something you get from the headline. This is just kind of a really abstract fact-checking exercise.
posted by mellow seas at 7:19 AM on August 15, 2023
> Whether a story has merit or not is not something you get from the headline.
I think the opposite. I felt that this little test has illustrated how the headlines often do tease at the slant of the story. It seems deliberate; the hyped headlines that screamed "fake!" to most of us are intended to draw in that very type of reader that is most inclined to put stock in such nonsense.
(see also: clickbait)
posted by Artful Codger at 7:03 PM on August 16, 2023 [4 favorites]
I think the opposite. I felt that this little test has illustrated how the headlines often do tease at the slant of the story. It seems deliberate; the hyped headlines that screamed "fake!" to most of us are intended to draw in that very type of reader that is most inclined to put stock in such nonsense.
(see also: clickbait)
posted by Artful Codger at 7:03 PM on August 16, 2023 [4 favorites]
« Older Remember how it improved society somewhat | The World Is Not Ending Newer »
This thread has been archived and is closed to new comments