Join 3,434 readers in helping fund MetaFilter (Hide)


Comments Are Important?
January 14, 2013 1:53 AM   Subscribe

Online comments hurt science understanding, study finds

Science, New Media, and the Public, from Online science news needs careful study.

The Science of Why Comment Trolls Suck
The researchers were trying to find out what effect exposure to such rudeness had on public perceptions of nanotech risks. They found that it wasn't a good one. Rather, it polarized the audience: Those who already thought nanorisks were low tended to become more sure of themselves when exposed to name-calling, while those who thought nanorisks are high were more likely to move in their own favored direction. In other words, it appeared that pushing people's emotional buttons, through derogatory comments, made them double down on their preexisting beliefs.
The study, which was in pre-publishing form, appears to have been taken offline.

Don’t Panic: Challenges Regarding Science, News and Comments Online
I don’t want anyone to forget these questions, so I’m asking them again. Please think about them, and weigh in here. What do you make of these?

•What can be done to get around the Google bottleneck – that self-reinforcing search-engine feedback loop mentioned above?
•What can we do to consistently draw attention to good science reporting?
•Do the benefits of online commenting outweigh the costs?
John Stuart Mill, Internet Trolls, And The Principle Of Charity, Part I
As Mill highlighted a person “is capable of rectifying his mistakes, by discussion and experience. Not by experience alone. There must be discussion to show that experience is to be interpreted… Very few facts are able to tell their own story, without comments to bring out their meaning.” For Mill this allowed us to step ever closer to the truth of a situation or gain ever more clarity on ideas. This was the most important goal of all rational debate and, for Mill, thoughtful discussion one of the most powerful ways to acquire it.
Part II
Reading charitably does not even mean reading realistically or knowing the reader’s “true” intention, since that is largely impossible especially on a platform like the Internet. It means reading a comment, an idea, or a question in the best possible light, warranted only by two properties: (1) we are fallible and cannot know everything, no matter how certain we or our group might be; (2) many people are bad at communicating and sometimes have never encountered the ideas being presented, thus their asking questions should be viewed as if from a Martian rather than a murderer.
Charity, Accuracy, And Being 'Nice' In Online Debates

Xark: Why I Shut Down Comments

Climate Trolls: An Illustrated Bestiary
posted by the man of twists and turns (46 comments total) 65 users marked this as a favorite

 
Well, that's my break-reading material sorted for the day. Thank you!
posted by likeso at 2:02 AM on January 14, 2013 [1 favorite]


All of this stuff is just implausable claptrap.
Hamburger
posted by jaduncan at 2:09 AM on January 14, 2013 [2 favorites]


The researchers were trying to find out what effect exposure to such rudeness had on public perceptions of nanotech risks. They found that it wasn't a good one. Rather, it polarized the audience: Those who already thought nanorisks were low tended to become more sure of themselves when exposed to name-calling, while those who thought nanorisks are high were more likely to move in their own favored direction. In other words, it appeared that pushing people's emotional buttons, through derogatory comments, made them double down on their preexisting beliefs.

Whoever says ad hominen attacks don't work is an idiot.
posted by three blind mice at 2:17 AM on January 14, 2013 [27 favorites]


This settles it once and for all: it's time to shut down comments on Metafilter.
posted by item at 2:27 AM on January 14, 2013 [10 favorites]


Those who already thought nanorisks were low tended to become more sure of themselves when exposed to name-calling, while those who thought nanorisks are high were more likely to move in their own favored direction. In other words, it appeared that pushing people's emotional buttons, through derogatory comments, made them double down on their preexisting beliefs.

This also suggests that objectively unreasonable beliefs are self-reinforcing; the very act of having them mocked/disagreed with makes them more intense. See social backlashes and religious fundamentalism, I guess.
posted by jaduncan at 2:34 AM on January 14, 2013 [5 favorites]


Online comments hurt science understanding, study finds

Exhibit A: the comments section of this very article.
posted by spoobnooble II: electric bugaboo at 2:41 AM on January 14, 2013 [3 favorites]


All online comments suck, especially this one.
posted by LarryC at 2:41 AM on January 14, 2013 [4 favorites]


Jokes aside; I think this illustrates the importance of a civil discourse, in online discussions as in real life.

I have seen het up commenters dismiss such calls as "tone arguments" - here and elsewhere, especially on blogs like Hate Required, and anywhere else where there is a top-down approach sanctioning insults and the like as simply the cut-and-thrust of debate, if-you-can't-handle-it-you-shouldn't-be-commenting, etc etc.

But - as if we didn't already know (and we bloody should, given how reluctant most people are to demean others in person) - tone matters.

It matters firstly, if you want to change someone's mind; secondly, if you want to make sure your own opinion isn't becoming increasingly deranged; and thirdly (my own reason) because it hurts empathy, mutual respect and understanding, and makes it easy to dismiss other people and their feelings as incidental characters to your own internal narrative.

I am perpetually grateful for the *relatively* aggressive moderation here, that tolerates dissent - and oft-times assent - but only if it is (relatively) respectful and substantive. I think it's one of the things that make this site unique and contributes vastly to its appeal. The alternatives seem to be moderation-free cesspits of bullying, or moderation-heavy cesspits of groupthink, where bullying is tolerated so long as it's directed at outsiders of one form another (like Making Light, I would argue).
posted by smoke at 2:41 AM on January 14, 2013 [21 favorites]


Y'all wanna see some shiny things?
posted by mannequito at 2:41 AM on January 14, 2013


This is very interesting, and I have many thoughts (and also doubts/questions) about the implications... but of course I can't help but note the irony in the original study being currently (?) unavailable to readers and commenters. I think asking how science communicators can adjust to new information paradigms is a great issue to address, and that making the research available to all at the time of the most intense online scrutiny would be a peachy idea.

If scientists feel dismay that some of their points are being lost or even subverted in the reporting and reception by readers in popular venues, it might help to publish the actual study. It doesn't resolve all the problems, obviously, but at least careful readers have a way of correcting misunderstandings in comments based on source material instead of relying on guesses or dumbed down "translations" published as click-bait STUDY REVEALS X, where X is not at all what the study reveals.

In the case of this study, maybe I missed it, but does anyone know if the rude/trollish comments that were used were equally pro and anti? In other words, do rude comments always introduce more doubt and suspicion about a science topic regardless of whether they support or challenge it? Or did researchers include only antagonistic comments (against the science being presented)?
posted by taz at 2:53 AM on January 14, 2013 [4 favorites]


And, lo! The prophecy comes true!

The flood of wan jokes in this thread is making me bed down my opinion of people that don't RTFA or make an effort.
posted by smoke at 2:54 AM on January 14, 2013


We should nevertheless head towards even more comments internet wide, such as browser addons that give one access to search results the site cannot control. Feel cheated by a product? Dislike Carmen Ortiz? Tag appropriate urls in your scathing blog entry. Voila your "comment" appears when people using an appropriate browser extension find the page.

How do you combat incorrect comments? Just provide a better curated source. It'll still create a net positive since the serious anti-science positions like climate change denial have enough money to do whatever they want until people really witness the truth anyways.
posted by jeffburdges at 2:56 AM on January 14, 2013


*paralyzed into silence, rtfa now*
posted by infini at 3:01 AM on January 14, 2013 [1 favorite]


This post is worth it just for "Climate Trolls: An Illustrated Bestiary."
posted by MuffinMan at 3:03 AM on January 14, 2013 [6 favorites]


This also suggests that objectively unreasonable beliefs are self-reinforcing; the very act of having them mocked/disagreed with makes them more intense.

Not self-reinforcing. Comments are just another form of external validation. When the idiot who disagrees with you resorts to name-calling, well, then you must be right. The interesting comparison would be if positive validation had the same galvanizing effect.

*I favorited your comment and just wanted to say that this font looks very nice on you.*
posted by three blind mice at 4:02 AM on January 14, 2013


When I was working for the Very Large Pharmaceutical Company of America (where we were curing cancer one meeting at a time) I used to try and solve all my problems with small discussions in the hallway and never bring them up before the entire group for exactly this reason. The person with the most easily visualized explanation always won the debate, even if their hypothesis required your molecule of interest to have properties that would make Larry Niven blush.
posted by Kid Charlemagne at 4:47 AM on January 14, 2013 [6 favorites]


Metafilter: hurting science since 2000
posted by quonsar II: smock fishpants and the temple of foon at 5:28 AM on January 14, 2013 [1 favorite]


This also suggests that objectively unreasonable beliefs are self-reinforcing; the very act of having them mocked/disagreed with makes them more intense. See social backlashes and religious fundamentalism, I guess.

Mocking is not the same as disagreeing with, and as far as I can tell the study was looking at the effects of the former, not the later. This is what I personally find tiresome about a lot of the discourse around atheism on the web; many participants seems to start from the premise "it is obvious there is no god and anyone who cannot perceive this must be stupid; therefore, they and their beliefs are proper objects of contempt."

Some of this is doubtless because people often begin to think about the nature of religious belief when they are teenagers, which is a particularly naive and assholish stage in life. But it often seems apparent to me in professional skeptics of all stripes as well; it's the flipside of their ever-burning outrage, inseparable from it. The outrage at least has its uses.
posted by Diablevert at 5:35 AM on January 14, 2013 [2 favorites]


Exhibit A: the comments section of this very article.

"I think you can take all these studies by pointy headed scientists, 99% of whom are socialists and communists, and stick them where the sun don't shine. Just listen to Rush and Hannity, and you will learn why you shouldn't trust "science." It is all designed to let the government control every aspect of our lives."

Haha, no, this is obviously a parody, right? right?
posted by en forme de poire at 5:42 AM on January 14, 2013 [8 favorites]


No comment.
posted by Splunge at 6:18 AM on January 14, 2013 [1 favorite]


*Reads wrong article, eats comment and burps*
posted by infini at 6:32 AM on January 14, 2013


Am I alone in that Mefi is the only site on which I have even the slightest desire to read the comments? Like, I've heard tell there's a plugin to block YouTube comments from being visible, and I get that YT comments are universally terrible, but why would you even be compelled to look?
posted by Uther Bentrazor at 6:36 AM on January 14, 2013 [1 favorite]


I'd like to see something like a cross-site online ID service that would let people use something more than just anonymous throwaway IDs but not quite real names and addresses. Has anything like that been tried?

To sign up, you might show who you are, maybe with a credit card number or physical address, and pay a fee, mainly to discourage people from making lots of sock puppets. Then any cooperating site could decide whether to let your ID participate on that site based on your ID's current reputation, which could be based on moderator comments, peer ratings, a quick skim of your activity elsewhere, real-life meetup behavior, etc. Make an ass of yourself on one site and you might find your ID rejected by or dropped from other sites. You wouldn't have to reveal your real name and address to sites or their readers, but you'd have a general online ID with a reputation to maintain.
posted by pracowity at 6:50 AM on January 14, 2013 [2 favorites]


pracowity: Disqus
posted by the man of twists and turns at 7:07 AM on January 14, 2013


Sadly as my hometown newspaper's comment section demonstrates, even tagging people's comments with their actual real life name and Facebook photo does not seem to prevent anyone from being a total dickass.
posted by en forme de poire at 7:16 AM on January 14, 2013 [4 favorites]


I see the following problems with online discourse..

1. First post phenomenon - Since comments are usually chronological, the first poster does so under time pressure to get something out first, or get ignored. This is a quality issue right off the bat. My first response to anything is likely emotional, and not well thought out.

2. 15 Minutes of fame. - Any post gets attention, and then gets pushed under by the oncoming store of other stories in 15 minutes, maybe a bit more if you're evil like Carmen Ortiz... but only a bit more. This means that any effort you do put into something isn't going to pay off much.

3. The inherent messy non-machine-parsed nature of comments. If you agree with someone, it's not color coded or hyper-linked or indexed to be sort or search able. This means it's not easy to seek out an opposing view, or to find supporting views in a cloud of comments.

4. Popularity contests are 1 dimensional - There are lots of reasons someone might want to flag a post (which isn't quite granular enough for me, but you have to start somewhere)... agreeing with a post as "having value" is the standard here in the blue, but elsewhere it's a direct measure of the groupthink agreement. (/. for example)

Wouldn't it be better to have multiple dimensions of ratings? Factual/wrong, Conservative/Liberal, Cheap/Expensive, True/Lie, etc.? They wouldn't have to be the same set of things either, but it would be easier to code for lets say Funny, True, Insightful as separate (orthogonal) dimensions.

Facebook doesn't allow the inclusion of negative numbers, so it's actually only 1/2 of a dimension.

5. Anonymity allows people to say things they'd not say in person. Remember the Id in Forbidden Planet?
posted by MikeWarot at 8:05 AM on January 14, 2013 [1 favorite]


"Am I alone in that Mefi is the only site on which I have even the slightest desire to read the comments?"

Nope . .. well, except for the occasional swing by phys.org to catch the latest vehement defense of aether wave theory.
posted by Phyllis Harmonic at 8:21 AM on January 14, 2013


What can be done to get around the Google bottleneck – that self-reinforcing search-engine feedback loop mentioned above?

Always look for the expressions of the contrary of the position you hold.

What can we do to consistently draw attention to good science reporting?

Avoid triumphalism: "Latest research confirms my point of view!!!1111!!" invites a smack-down.

Do the benefits of online commenting outweigh the costs?

Scientific work, like everything else, is now subject to intense scrutiny, criticism and ridicule. Scientists and their supporters may have to eat a little humble pie, and learn some rhetorical skills. Any delusion of an unquestioned scientific aristocracy has to be discarded.
posted by No Robots at 8:33 AM on January 14, 2013 [1 favorite]


has anyone else noticed that on political and environmental threads here on the blue, quite often the majority of comments are by those who are never, or rarely, here? climate change stories, in particular, often have more 'debate'...
posted by sexyrobot at 8:50 AM on January 14, 2013


among this audience and on this site, where comments get read, they would stand out like a very clever simile inserted here at this point
posted by infini at 8:57 AM on January 14, 2013


quite often the majority of comments are by those who are never, or rarely, here?

Hard to tell if people are lurking or not, but there's a noticeable group of people who will apparently spend or expense $5 merely to shout at the ether about their holy cause for one thread.
posted by jaduncan at 9:01 AM on January 14, 2013 [1 favorite]


They don't mention their methodology, particularly how long after reading the participants were polled.

I'm in a crappy mood after reading a bunch of snippy comments too. Give me a couple of days to think about the subject matter and you'll get a much more reasoned answer.
posted by Tell Me No Lies at 9:50 AM on January 14, 2013 [1 favorite]


Wouldn't it be better to have multiple dimensions of ratings? Factual/wrong, Conservative/Liberal, Cheap/Expensive, True/Lie, etc.? They wouldn't have to be the same set of things either, but it would be easier to code for lets say Funny, True, Insightful as separate (orthogonal) dimensions.

I don't think this is a problem coding can solve. In a metaphorical way, it feels like applying a digital solution to an analogue problem --- how I feel when I read something is a synthesis of interpretation, experience, nuance which produces the holistic experience of pissedoffedness and/or amusement. You can't apply a discrete number to that an expect me to notch my reaction to a comment up or down like it was a volume button.
posted by Diablevert at 10:19 AM on January 14, 2013 [1 favorite]


Here is a similar article, on a different topic (media bias), that sheds some light on some of the cognitive processes going on here: That's Not the Way It Is: How User-Generated Comments on the News Affect Perceived Media Bias.
posted by DiscourseMarker at 10:29 AM on January 14, 2013 [2 favorites]


Any delusion of an unquestioned scientific aristocracy has to be discarded.

This is sort of a surprising statement to me. Can you give an example of an extant unquestioned scientific aristocracy?
posted by en forme de poire at 11:59 AM on January 14, 2013


Can you give an example of an extant unquestioned scientific aristocracy?

You can start perhaps by looking at The Tyranny of Science by Paul Feyerabend.
posted by No Robots at 12:10 PM on January 14, 2013 [1 favorite]


And, lo! The prophecy comes true!

The flood of wan jokes in this thread is making me bed down my opinion of people that don't RTFA or make an effort.


I don't see anything in the linked articles about "wan jokes."

Re the study itself: I wonder if they aren't potentially throwing the baby out with the bathwater here. The implication that scientific literacy is obviously being "damaged" if it is influenced by the tone of the comments is not really established by the study they seem to have undertaken. If I read a report on some scientific or technological "breakthrough" (say, for example, on Metafilter) and it's not in a field I have any expertise in, I would say that my "scientific literacy" is generally enhanced by reading the comments; if there are lots of people saying "this is a load of nonsense"--even if they haven't explained why they think it's a load of nonsense--in most cases that will usefully reflect to me the claims being made in the linked article are, at least, controversial and that I should hunt for further material about the topic before accepting them.

This only really breaks down in those relatively few cases where scientific/technical issues have been badly distorted by political controversy (why, hello Climate Change!). Obviously, in those cases the fact of a chorus of negative comments in response to new scientific findings has zero information value. But those will always be a pretty small minority of cases.
posted by yoink at 12:53 PM on January 14, 2013


Look at the new post on the Dead Sea Scrolls: here we have profound discourtesy within the scholarly community. Physician, heal thyself.
posted by No Robots at 2:41 PM on January 14, 2013


I think that people hold beliefs less because they're true, and more because those beliefs are socially valuable. I think that makes perfect sense. The validity of some particular religion, for example, is of awfully small factual import (unless, you know, heaven and hell), but the way that one is perceived by one's neighbors is huge.

Ten years ago, everybody that surrounded me complained about Starbucks. After a while, their complaints stopped making sense to me. If Starbucks was pushing out independent coffee shops, well, it was also paying a livable wage, with health benefits, to its employees, where those independent coffee shops weren't; and it responded to pressures for things like free trade coffee.

It's not important whether I was correct. What's important is that, even though I believed that Starbucks didn't deserve the vitriol launched its way, I still avoided Starbucks. The consensus of my social circle affected my actions, even though I disagreed with that consensus. (Being as I'm something of a twit, I'd still argue with people given the chance, so I didn't even get to reap the benefits of my conformity. Here's to youthful self-destruction.)

I worry very deeply about that effect. I think Metafilter's comments are of relatively high quality. But there's a search box up there: I dare you to type in "then you're an idiot".

Social weight has incredible power to form opinion. "Then you're an idiot" might not be the most valid argument, but it's among the most effective. So it's no wonder that we use social weight to convince people of certain things: there's no time to wait for an informed consensus on something like climate change, so maybe an uninformed consensus is what we need to shoot for. (Look at how the bestiary characterizes its antagonists; look at even the use of "troll" as casual invective in many of the links.)

But I really wish that we were, instead, teaching people good ways to reach the truth. Given a choice between widespread scientific literacy and widespread knowledge of scientific fact, the former is vastly preferable, and the latter is what we've achieved.

(PS: Hey, "then you're an idiot" isn't nearly as ubiquitous as I thought it'd be.)
posted by nathan v at 2:57 PM on January 14, 2013 [1 favorite]


> "Can you give an example of an extant unquestioned scientific aristocracy?

You can start perhaps by looking at The Tyranny of Science by Paul Feyerabend.
"

That seems to take an absurdist stance that some concepts are ineffable or unknowable. Since the above piece of research brings meaningful understanding on something as irrational as the effect of tone on opinion formation, I'm not seeing such a roadblock on rational enquiry.

Nevertheless, if you've read the book, does it propose any alternative?
Does the alternative involve embracing one's biases and prejudiced opinions?
posted by Tobu at 3:37 PM on January 14, 2013



It involves acquiring some philosophical insight:
The younger generation of physicists, the Feynmans, the Schwingers, etc., may be very bright; they may be more intelligent than their predecessors, than Bohr, Einstein, Schrödinger, Boltzmann, Mach and so on. But they are uncivilized savages, they lack in philosophical depth.--For and against method
Science is part of the political realm, and the political realm is ever more democratic, which means that scientists and those who write about science must come to terms with the demos.
posted by No Robots at 6:39 PM on January 14, 2013


Philosophers are slippery… I've tried to understand the thesis of one book, and you're giving me another book to read instead. I'll take it your last sentence is the point you were trying to make.
posted by Tobu at 8:23 PM on January 14, 2013


The problem I have with a lot of comments and also a lot of stories is in the language that is used. Science has all of this very precise language to tell different things and most journalists just dismiss it as jargon that needs to be simplified. They don't realise that their simplification doesn't mean the same thing as the language they were trying to simplify.
posted by koolkat at 9:55 AM on January 15, 2013 [1 favorite]


Science has all of this very precise language to tell different things and most journalists just dismiss it as jargon that needs to be simplified. They don't realise that their simplification doesn't mean the same thing as the language they were trying to simplify.

The point of writing a newspaper or magazine article about any scientific discovery or controversy is to explain something interesting and/or important to a general reader who cannot be expected to have relevant background knowledge beyond what they learned in high school. That's who the article is for. If the journalist restricts him or herself to the language of the field they're writing about, they fail in their purpose and the public is no better enlightened. If the subject is so obscure or abstruse that its relevance cannot be explained without the use of technical jargon (or without doubling the length of the piece explaining technical jargon) then it's not an appropriate subject for the general interest reader.
posted by Diablevert at 11:25 AM on January 15, 2013 [1 favorite]


The problem is that they get things wrong that they should have learned in high school. From this article:

So far scientists haven't been able to prove that the process generates more energy than it requires.

The quote is impossible. I am fairly certain that the scientists would have couched it in different language such as: "We don't know if the process with be energetically economical"

In layman language using the quote it should be: "So far scientists haven't been able to prove that the process is economical on large scale" or something like that. If a process makes more energy that it requires you would violate thermodynamic laws.

That example is just the most recent one I have read. There was another one about someone printing specific chemical reactors using a 3d printer and the article was talking about printing personalised drugs. The scientists were all talking about how the reactors were designed and optimised and said that the reactors can then be used to run specific reactions, but the journalist took the leap to essentially a star trek replicator. The language used is important it would be like talking about designing a better spring for a pogostick and then a journalist talking about traveling to the moon by jumping. Not all precise language is technical jargon, and I wouldn't expect a detailed synthetic breakdown in any talk about a new medicine, or even a detailed pathway of what proteins it hopes to target, because that would be above a layman explanation, but it would be nice for once to read a newspaper article about the discovery of a new pathway to target a specific cancer line didn't just turn into an article about how cancer will be cured in 10 years.
posted by koolkat at 1:38 AM on January 16, 2013 [1 favorite]


Freedom To Comment Isn't Free
The default setting on most blogs and online forums is for anyone to be free to post comments unless they repeatedly violate rules against swearing or personal abuse. In the past I have used this policy on my personal blog and Facebook profile and only blocked a handful of people over many years, usually for pretty outrageous remarks. This policy ensures that all comments, even those judged negatively by the original author, can be found somewhere in the resulting thread. But it has some major downsides, and I now wonder if it was a mistake.
posted by the man of twists and turns at 8:25 PM on February 11, 2013 [1 favorite]


« Older Today is Coming of Age Day in Japan, when everyone...  |  Dean Kamen, inventor of the Se... Newer »


This thread has been archived and is closed to new comments