Bad, badder, baddest?
July 5, 2011 3:28 AM   Subscribe

"It seems that the majority of health claims made, in a large representative sample of UK national newspapers, are supported only by the weakest possible forms of evidence." So states the Guardian's Bad Science columnist and blogger Ben Goldacre in an article describing a study he performed with several colleagues investigating the quality of health advice given in British newspapers. The study can be found here (only the abstract is free for those who don't subscribe, unfortunately). The Guardian's science editor, James Randerson's critique of the article. Goldacre replies in the comments.
posted by jonnyseveral (34 comments total) 10 users marked this as a favorite
 
Well they are newspapers. Shouldn't this be "the majority of claims made about X, in a large representative sample of UK national newspapers, are supported only by the weakest possible forms of evidence"? Diet science is notoriously unreliable, partly because of the difficulty of carrying out controilled trials. I'm actually kind of surprised that 27% of advice was 'probable' or 'convincing.'
posted by carter at 3:48 AM on July 5, 2011 [1 favorite]


People still take newspapers seriously?
posted by chavenet at 3:55 AM on July 5, 2011 [3 favorites]


James Randerson's critique may be the first example of a mainstream journalist interrogating the science behind a press release that I've ever seen. No surprise that it was spurred by self-interest. I really don't buy his argument though, which seems to be that responsible and balanced reporting of science is boring. Journalists seem to operate under the notion that all science is eureka moments and big bangs; if it hasn't been shown to cure or cause cancer, it's not news. At a time when science is increasingly under fire from idealogues and ignoramuses, we need accurate and intelligent science reporting more than ever. The alternative is a world of vaccine scares and intelligent designers.
posted by londonmark at 4:01 AM on July 5, 2011 [9 favorites]


Another problem is not so much the sample of Goldacre's study, but that most readers will not have access (either because they don't know how to, or because of paywalls, or because of lack of disciplinary background) to the papers and the experiments and the science that are actually quoted in the newspapers. Newspapers pretty much don't provide citations to articles, although they may provide a link to a university web site puffing the research in question (and this was probably written by a journalism major).
posted by carter at 4:06 AM on July 5, 2011


People still take newspapers seriously?

hell no! I get all my health advice from blogs.
posted by russm at 4:20 AM on July 5, 2011 [9 favorites]


System Justification is always a kick to watch in action.
posted by srboisvert at 4:21 AM on July 5, 2011 [1 favorite]


londonmark: exactly! I'm not trained in science but so often I see articles that contain dubious claims, but when I try and read the original sources I get stuck behind 100+ $/year academic paywalls.

I wouldn't mind dropping a dollar or two to read a scientific paper, but that's not an option. If you are not allied with a university or research laboratory paying for a hefty subscription you have pretty much no way of getting to the original paper.
posted by sixohsix at 4:42 AM on July 5, 2011


The idea of regular medical "news" for lay people distorts the picture. You don't need daily health updates. Medical news unfolds relatively slowly. As a lay person, you'll learn all you need to know by reading the Mayo Clinic HealthLetter, or Cleveland Clinic Be Well, or Johns Hopkins Health Alerts. If you're in the business, you know where to go for more granular information. The daily science and health news cycle is based on the belief in breakthroughs, and the public's eager hope that someday they'll wake up and read the headline: "Pill discovered that allows you to eat all you want and not gain weight, screw complete strangers without danger of disease, and sit on your butt for 24 hours a day and retain muscle tone. So go ahead, keep on doing what you're doing!" Since that headline will not soon be forthcoming, you can relax and not worry that something important will happen in the health field and you'll miss it.
posted by Faze at 4:50 AM on July 5, 2011 [7 favorites]


A lighter take on the science news cycle.
posted by MonkeyToes at 4:53 AM on July 5, 2011 [2 favorites]


Apparently British newspapers are like shit, except that shit can be used as fertilizer.
posted by aeschenkarnos at 4:56 AM on July 5, 2011


The daily science and health news cycle is based on the belief in breakthroughs, and the public's eager hope that someday they'll wake up and read the headline: "Pill discovered that allows you to eat all you want and not gain weight, screw complete strangers without danger of disease, and sit on your butt for 24 hours a day and retain muscle tone. So go ahead, keep on doing what you're doing!"

This is already happeneing. You only have to follow the Daily Mail's ongoing mission to classify all inanimate objects as either cancer causes or cancer cures.

I would challenge the view that this is driven by public demand, though. There is no public demand for misrepresentation of science, for misleading claims, for scare-mongering or snake-oil (but I'd accept that the public is not completely blameless in their willingness to swallow the lies and their inability to scrutinise reporting with any degree of distance). Papers pass off this rubbish as truth out of lazy journalism and a cynical desire to make a fast buck. You can't underestimate the importance of advertisers and PR to the existence of newspapers.
posted by londonmark at 6:06 AM on July 5, 2011 [2 favorites]


I am a practicing scientist and (sometimes) a science writer, though I don't write about medicine. I think I have some sense of how this works. Editors -- not all editors, but many editors -- want you to give their readers an answer. If there's a controversy, they want to know who's right and who's wrong. Of course, most scientific papers don't give definitive answers to such questions; they say something more like "this study lends some evidence to side A of the controversy, and is interesting because it suggests a new experimental mechanism through which we can gather evidence about the controversy, and makes it more likely that in the years to come the scientific community will slowly come to some kind of consensus about the controversy."

But I can tell you from experience that most editors don't respond well to a pitch like that. And why should they? I think they're almost certainly right that most readers don't want to read that kind of coverage.

So how do science writers deal with this? Well, we write things like

"A recent Italian study linked the combination of Italian food and dark chocolate with lower levels of a protein in the blood related to inflammation – C-reactive protein (CRP)."

Note that the writer has carefully avoided making any causal claim -- one can hardly object to using "linked" to describe what I presume was a correlation found in the study. The word "related to" is also carefully chosen to avoid claiming too much.

But the writer also knows perfectly well that this sentence will be read as "eat chocolate and spaghetti, it's good for your health."
posted by escabeche at 6:22 AM on July 5, 2011 [3 favorites]


The Mail's carcinogen crusade is far more nuanced than that. It's best thought of as a giant Venn diagram in which everything either causes cancer, does not cause cancer, cures cancer, or increasingly is simultaneously capable of causing/not causing/curing cancer, depending when it was you last read the Mail.
posted by permafrost at 6:22 AM on July 5, 2011 [1 favorite]


The lowly question mark has probably done more for bogus scientific / medical journalism than any other single piece of...whatever it is scientific journalism is made of.
posted by ShutterBun at 6:29 AM on July 5, 2011


"A recent Italian study linked the combination of Italian food and dark chocolate with lower levels of a protein in the blood related to inflammation – C-reactive protein (CRP)."

The sub comes along and slaps a headline on this that reads "chocolate pasta makes you live longer" and the ad sales manager sells an adjacent quarter page to Cadbury's.
posted by londonmark at 6:46 AM on July 5, 2011 [2 favorites]


Is this one of those things where you have to know that correlation doesn't equal causation to understand?
posted by d1rge at 7:12 AM on July 5, 2011


I'm still unclear about which specific brand of chocolate sauce I'm supposed to be pouring over my manicotti.
posted by Wolfdog at 7:23 AM on July 5, 2011


"A recent Italian study linked the combination of Italian food and dark chocolate with lower levels of a protein in the blood related to inflammation – C-reactive protein (CRP)."

Why would anyone outside of the specific field of nutritional biology actually need to know something like this? This sort of research is just one step in a process that we hope will lead to a better understanding of human health, but in and of itself it only suggests a direction to look. You don't see automotive journalism running stories on the first CAD sketch of one small gear that may eventually get used in the 2015 model of car. Writing stories about a specific, rather narrow study is equally silly.

Heck I think that's half the problem with science journalism right there: too much focus on the specifics without ever giving anyone a broad understanding of just what the heck is going on. I mean its hard because if there's ongoing research there may not be a broad understanding of the topic just yet, but someone with time who understands the field can probably figure out a way to convey a general idea of where an avenue of inquiry seems to be headed. But then again that would require a whole stable of writers, not just one designated science journalist, so who knows how that'll ever get fixed.
posted by Zalzidrax at 7:41 AM on July 5, 2011


Why do newspapers hire journalists to fill column inches with such tripe? Surely they could just buy the syndicated Dr. Oz column and call it a day, no? It's still tripe, but it's got to be cheaper.
posted by Sys Rq at 7:41 AM on July 5, 2011


People still take newspapers seriously?

Mainly for the placebo effect.
posted by StickyCarpet at 7:43 AM on July 5, 2011 [5 favorites]


See also, the fantastic kill-or-cure which profiles every article in which the Daily Mail (probably the worst british newspaper) alleges causes or prevents cancer.
posted by Another Fine Product From The Nonsense Factory at 7:59 AM on July 5, 2011


Correlation causes cancer.
posted by blue_beetle at 8:09 AM on July 5, 2011 [1 favorite]


In other news, water has been found to be associated with the condition known as wetness, and researchers report finding a link between night and an increased risk of darkness.
posted by kyrademon at 8:15 AM on July 5, 2011


Is there a Kutcher in the house?
posted by Beardman at 9:08 AM on July 5, 2011


Even if you present a physician with a NIH or National British study that says that old line drugs cause less problems than newer ones, and she says "that has not been my experience", where do you go then?
posted by Bitter soylent at 9:21 AM on July 5, 2011


This post is missing this, a genuinely best of the net site: NHS Choices: Behind the Headlines.

NHS Choices takes UK press stories and explains them, including a review of the quality of the original research and how good the stories are at representing it (protip: whoever writes it almost certainly agrees with Ben).
posted by jaduncan at 9:58 AM on July 5, 2011 [2 favorites]


Apparently British newspapers are like shit, except that shit can be used as fertilizer.

Extra! Extra!
posted by The 10th Regiment of Foot at 10:09 AM on July 5, 2011


Joe Jackson was right. Everything gives you cancer.
posted by dglynn at 10:46 AM on July 5, 2011


People still take newspapers seriously?

Well, as much as people say they don't take newspapers seriously, a large chunk of what gets passed around the blogosphere is recycled from a handful of mass-media news sources.
posted by KirkJobSluder at 11:32 AM on July 5, 2011


This is divisive, and everyone's choosing sides, but actually, I found both men's positions kind of underwhelming.

Goldacre accidentally wound up with a terrible week for his study. Numerically, most of the claims that made it into the study ended up being from papers with, let's say, "a certain ideological spin", if not outright a reputation for unreliability. Wouldn't it be more useful to point out which papers had how many problems, so that the public can change its habits and reward the better papers proportionately? What's more, he completely ignored the question of whether an article's claims about the level of evidence for its claim matched the level of evidence, which I think is a very important question here.

However, even if the articles do print all the right qualifications and caveats, it's still a little questionable to even inform the public about things that are not yet well-established. Does the public need to see the process as science tries to figure something out? I don't think so. If they want to, they should subscribe to a journal (or read a specialty news source with more credibility in this area). I think most people would just be confused. And frankly, most of the other counter-arguments to Goldacre's paper were kind of a reach.

I would say that Goldacre's claims are a little more sensational than his evidence supports, though that's not to say that science reporting in newspapers is just fine. The media has a strong sensationlist bias, and this spoils its science reporting along with everything else.
posted by Xezlec at 12:24 PM on July 5, 2011 [1 favorite]


Even if you present a physician with a NIH or National British study that says that old line drugs cause less problems than newer ones, and she says "that has not been my experience", where do you go then?

Presumably to a different physician.
posted by BigHeartedGuy at 1:16 PM on July 5, 2011


Xezlec: Numerically, most of the claims that made it into the study ended up being from papers with, let's say, "a certain ideological spin", if not outright a reputation for unreliability. Wouldn't it be more useful to point out which papers had how many problems, so that the public can change its habits and reward the better papers proportionately?

They might have a reputation for unreliability, but remember that he looked at the 10 bestselling papers in the UK. Unreliable or not, a lot of people are reading these papers and are being misinformed by the health advice they contain (the top ten papers in the UK have a circulation of about 10 million). With so many misleading health claims reaching millions of people, I think it's fair to say that any sensationalism on Goldacre's behalf is the direct result of the sensationally bad journalism on the part of UK newspapers.

That said, I agree that it would be useful to know which papers had the most problems.
posted by joedan at 1:53 PM on July 5, 2011


Casey Anthony, Strauss-Kahn, Madeleine McCann, medical reports... See the pattern yet?
posted by Ardiril at 2:49 PM on July 5, 2011


They might have a reputation for unreliability, but remember that he looked at the 10 bestselling papers in the UK.

You don't understand. He (randomly) happened to pick the week that President Obama was elected to do the study. So, although they read the top 10 papers, only a handful of them made up the majority of the dietary news stories he looked at. The more serious papers covered the bigger news in the world to the exclusion of minutiae such as health, so he ended up getting a sample set composed primarily of stories from only a subset of those 10 papers.
posted by Xezlec at 3:20 PM on July 5, 2011


« Older There's no place like home   |   Thrills! Chills! Stills! Newer »


This thread has been archived and is closed to new comments