"There is no cure for our addiction to medical hype"

March 23, 2015 5:17 PM

Every day, news sources report on medical studies that describe promising new treatments. Most of them don't pan out in the end. Julia Belluz reports on "why you shouldn’t believe that exciting new medical study." Her article includes a figure showing which foods are reported to cause cancer and which ones are reported to prevent it. (Spoiler: they're the same foods.)
posted by grouse (25 comments total)

Lab coats cause cancer in rats.
posted by eriko at 5:23 PM on March 23, 2015 [7 favorites]

After considering the phenomena discussed here, it's even more alarming that research funders are redirecting money from understanding how our bodies and cells work to "translational" research designed to bring "breakthroughs" to patient care faster. So many of those "breakthroughs" aren't going to go anywhere, and translational research is much more expensive.

Meanwhile, we'll slow our accumulation of new biomedical knowledge, especially the kind of reliable knowledge described in Belluz's article that comes from having many studies in the same area. People have called this shift in research priorities eating our own seed corn, or a "a going out of business sale". No matter what you call it, it doesn't bode well for future discovery in medical research.
posted by grouse at 5:29 PM on March 23, 2015 [8 favorites]

Just want reassurance that good red wine and chocolate prevent just a bit more than cause. Can live without the rest. Well, for a while.
posted by sammyo at 5:39 PM on March 23, 2015 [1 favorite]

Most of those breathless preliminary reports of medical breakthroughs are the same as breathless preliminary reports of high tech breakthroughs, especially in the realm of "alternate energy". The primary purpose is to scare up investors.
posted by Chocolate Pickle at 5:51 PM on March 23, 2015 [1 favorite]

It's that one weird old trick for creating pageclicks that "they" don't want you to know about.
posted by Halloween Jack at 5:52 PM on March 23, 2015 [3 favorites]

After considering the phenomena discussed here, it's even more alarming that research funders are redirecting money from understanding how our bodies and cells work to "translational" research designed to bring "breakthroughs" to patient care faster. So many of those "breakthroughs" aren't going to go anywhere, and translational research is much more expensive.

As someone who no longer does biomedical research, because I only worked on basic and not translational research and my funding dried up, I totally agree!
posted by gaspode at 5:53 PM on March 23, 2015 [7 favorites]

This means early medical research will mostly be wrong until maybe eventually, if we're lucky, it's right. More tangibly, only a tiny fraction of new science will lead to anything that’s useful to humans.

I mean, yeah, this is just called normal science. Getting anywhere requires a shit ton of failed research and a shit ton of redundant research. That's how you slowly hone in on consensus and progress and eventually breakthrough and paradigm shift. Just because something 'fails' doesn't mean it's put science back or whatever. An experiment that doesn't work tells you as much as one that does.

That's not to say that all studies are equally well-constructed, but the complaint that most potential medical treatments don't come to market is not a very well-founded one. Science by definition is a process of trial and error, usually with an emphasis on error, because that's what happens when the evidence bar is set high (as it is with treatments that end up in the clinic, and as you'd hope it would be).

Media places don't understand how to actually read the damn papers, so all that gets reported and sent around on facebook and whatever is either the PI's public summary, which will often be inflated, or some other over-hyped nonsense someone took from just reading the paper's conclusion. Most people don't have either the time or knowledge or both to really look past the sciencebait headline and see what the actual data is, and the media does a big disservice by usually completely misrepresenting the results and the gravity of those results from any given study.
posted by Lutoslawski at 5:53 PM on March 23, 2015 [7 favorites]

Hm, Vox's graphic captioned "Everything we eat both causes and prevents cancer" seems to be a selection from this figure in the AJCN article it's based on, which doesn't seem to say that exactly, but maybe that salt and pork were associated with risk in every study and olives never were? Perhaps I've misunderstood--certainly, the general message that I don't know how to evaluate these studies or their representations in popular media isn't lost on me.
posted by Monsieur Caution at 6:09 PM on March 23, 2015

I have a blog on autism and the signal to noise ratio on supposed cures is immense in the 20-30 research articles I read per week.
posted by ITravelMontana at 6:11 PM on March 23, 2015

So I have several thoughts about this, after thinking about it since this morning when I read the article.

1. This problem is just a function of a changing public perception of the abilities and meaning of 'science' that started around the 30's and 40's and carried on through the late 70's and 80's. For a period of time, the governments of the US and many European countries were both bolstered by WWII influenced technological advancement and a cold war desire to maintain that advancement. While primarily military, this general funding and public support push for 'serious' science led to tremendous advancements in space exploration, medical treatment, QOL improvements, fuel and energy extraction and use, and so much more. That progress slowed in terms of breadth and depth because of eventual public funding issues in the 90's, in my mind simply because the countries that preformed such research really weren't as scared of the real or imagined enemies they had then. The cold war was ending. The west* lacked an enemy. The same rapid development we saw of space tech, communications tech, medical tech and industrial tech is now only preformed when there is a possible commercial market for that research. We've come to expect such great things from "SCIENCE", but are completely lacking in the political will to fund them adequately.

2. There is still plenty, PLENTY of completely ignored or not broadly considered serious research happening in more places than ever before at a higher level than ever before. We are not actually hurting for incredibly in-depth, well-designed and relevant research, it's just that very few people actually want/have the ability to read and understand it. That's not exactly the fault of the media, they aren't the scientists for a reason. I do not know a single serious scientist who is capable of actually explaining their work in understandable terms for someone who doesn't work in that field, and worse, many actually consider 'condescending' to the public level to be an insult to their field. From what I can tell, we don't lack for 'real' science, we lack people who do that science who can and want to talk to the public.

3. Nations that could conceivably preform 'real' science are extraordinarily comfortable ATM, as are the people who could eventually become scientists in the current system. There is basically no reason besides an ethical dedication to "SCIENCE" to preform research in America that doesn't both lead to a 'possible', no matter how unlikely, stupendous increase in the QOL of western people. We're all blinkered by cold war advancements and the rapid development of consumer applications of tech.

4. Knowing some cancer researchers, many developments in individual cancer treatments are actually coming up fairly soon. We are, in our most serious science, coming up on personal and specific approaches to many illnesses that plague mankind. It's more than a decade or so out, but I think that the reality of medical research and the woo in medical research are not so far apart in what they promise people publicly. It's simply that one falsely claims to be here now (And as the article points out, are always, always crushed by the scientific community as soon as they can, and it's the community of scientists that decides what is a realistic goal to achieve.) and one claims to be here later. Is it really the fault of non-scientists that one gets more focus? Of course people want their cancer cured now, and no amount of "Well that study has to few subjects, makes big assumptions, uses faulty methodology, etc." is going to stop people from wanting the best thing as soon as possible. Or course they do, and most don't or can't understand why it's not necessarily possible.


In summation I guess, we're in a historical rut of funding and support for long-term (which changes quickly, it's idiotic to suggest that what research looks like to us now is the way it'll always be, or even that it'll continue for more than 10 years.) serious scientific work, there's less reason for any individual nation that CAN fund more serious research to do so, given the relative comfort they enjoy, there's more capitalistic reasons to fund faulty science, the area between fake and real research is dwindling in terms of what is considered 'possible' given the actual advancement of serious science while the woo shit's been at full force, giving people more reasons to believe the woo shit, and there all all kinds of actual, serious science being preformed all the time.

* And let's be serious here, we're talking about English language or, in some cases broader European studies. This is not a conversation about science, this is a conversation about Western science. There are still thousands of decade-old untranslated papers from everywhere else, and more are made every day. Talking about this as a world issue is incredibly insulting to tens of thousands of non-english speaking scientists around the globe, and even part of the issue, ignoring 'boring' science from other places.
posted by neonrev at 7:08 PM on March 23, 2015 [8 favorites]

Because that wasn't long enough, here's the angry part:

Everyone I've seen whingeing and complaining about this "SUPER SERIOUS PROBLEM" had damn well better have been actually following real scientific efforts or they need to shut the fuck up. In my experience, few do. You do not get to bitch about all the shitty science in the world and also ignore all the good science in the world because it's a tiny bit of effort to actually seek it out. If you can bitch about how bad it is, then you had also damn well better be able to identify good science. Otherwise you're just an asshole. As shitty as the current situation is, there are still a lot, a LOT of very well done and important studies that no one pays attention to.

If your only connection to science is through excited FB posts, then you deserve exactly what you get. As it stands now, science is heavily capitalistic for funding reasons, the media is similarly limited, and you do not get to complain about how clickbaity science headlines are without actually trying to engage with the real thing. Similarly, scientists do not get to complain about no one understanding their work if they aren't willing to preform (boring, sure) outreach. It's a two-way street, and from what I can tell, it's the scientific community falling down on the job, not the media or regular people. Only one of them has the burden of proving their worth to the other, and that ain't the hoi-polloi.
posted by neonrev at 7:09 PM on March 23, 2015 [4 favorites]

The new and old Daily Mail ontological ontology projects seem apropos here.

"An ongoing quest to track the Daily Mail's classification of inanimate objects into two types: those that cause cancer, and those that cure it. "
posted by lalochezia at 7:37 PM on March 23, 2015

More and more, I feel like I should really just check the news at all once every year. My stocks would do better.
posted by cacofonie at 8:53 PM on March 23, 2015 [2 favorites]

neonrev: Similarly, scientists do not get to complain about no one understanding their work if they aren't willing to preform (boring, sure) outreach.

Two problems:

1. A lot of science is just not interesting to anyone outside of the field, no matter how you dress it up. For my master's thesis, I did a multigene phylogeny (evolutionary analysis) of a somewhat obscure group of microscopic fungi. It's pretty hard to dress that up as interesting to a large audience. Most mycologists didn't even care about it. My friend did a study of a biochemical pathway involved in liver regeneration involving a hypothesis that turned out to be negative. How do you dress up "this biochemical marker doesn't seem to be involved in liver regeneration, despite being upregulated, according to our study in genetic knockout mice" for a broader audience? And a lot of studies are even more indecipherable than that! How do you get the public interested in a methods paper, for instance? Most of the people who cite it will barely understand it!

2. We need funding for this. Scientists are, generally speaking, only subject experts. They're not good at outreach and don't have the time for it. We need dedicated people to help us, but we don't have money for them. So outreach always suffers, unless you get someone who is naturally charismatic and has the other skills required (webmastering, social media, etc).
posted by Mitrovarr at 9:25 PM on March 23, 2015 [4 favorites]

Most supplement and dietary studies are just plain terrible and should not have been done in the first place, let alone trumpeted as reasons to change your lifestyle. I would much rather see a prospective study with low N and very good controls (e.g. controlled diets rather than self reporting) than some big retrospective study which publishes a bunch of weak correlations. Some of the blame for this should be laid at the doorstep of the personal fitness press, which needs new nonsense every issue, even though a sensible eating and exercise guide could be 2 pages long. Michael Pollan's dietary advice is three sentences long and better than most fitness magazines.

Meanwhile, studies like the one below get no press, even though it demonstrated that thousands of published claims about turmeric, red wine, soy, and green tea were actually artifacts from cell-based assays:

Phytochemicals perturb membranes and promiscuously alter protein function.
A wide variety of phytochemicals are consumed for their perceived health benefits. Many of these phytochemicals have been found to alter numerous cell functions, but the mechanisms underlying their biological activity tend to be poorly understood. Phenolic phytochemicals are particularly promiscuous modifiers of membrane protein function, suggesting that some of their actions may be due to a common, membrane bilayer-mediated mechanism. To test whether bilayer perturbation may underlie this diversity of actions, we examined five bioactive phenols reported to have medicinal value: capsaicin from chili peppers, curcumin from turmeric, EGCG from green tea, genistein from soybeans, and resveratrol from grapes. We find that each of these widely consumed phytochemicals alters lipid bilayer properties and the function of diverse membrane proteins. Molecular dynamics simulations show that these phytochemicals modify bilayer properties by localizing to the bilayer/solution interface. Bilayer-modifying propensity was verified using a gramicidin-based assay, and indiscriminate modulation of membrane protein function was demonstrated using four proteins: membrane-anchored metalloproteases, mechanosensitive ion channels, and voltage-dependent potassium and sodium channels. Each protein exhibited similar responses to multiple phytochemicals, consistent with a common, bilayer-mediated mechanism. Our results suggest that many effects of amphiphilic phytochemicals are due to cell membrane perturbations, rather than specific protein binding.

posted by benzenedream at 10:01 PM on March 23, 2015 [4 favorites]

And whatever they're telling you this week will be flipped to the opposite 20 years from now.

Ignore it all and eat what makes you happy; if you're happy, your body will be too.

You can't escape carcinogens and toxic chemicals, no matter how hard you try. You can't live to be 120 and in perfect health, no matter how hard you try. If you're very careful to eat only purely organic, "healthy" food, you'll breathe some auto exhaust while at the farmer's market and get cancer from that.

I worked in a hospital for a number of years doing patient care. I can think of three examples of people who just could NOT accept their diagnosis of terminal cancer: One was a Physical Education teacher who was a fitness fanatic, ate exactly what she was supposed to, kept up on the latest medical information and advice, took no medications, had no illnesses, never married, no children, focused on health - she came in with terminal colon cancer in her early 50s and flat refused to believe it. Second patient was an older lady who had lived on her own land and raised her own food including goats for milk and chickens and an incredible vegetable and herb garden; again, never married, no children, knew every herb in the book and how to use them. She had a major stroke and was so angry about it I think that's why she had her second one some months later - she just couldn't accept that all her perfectly-laid plans failed her, and it was SO hard on her - it was very sad. Third patient was a man from Alaska who lived the life of a REAL MAN - he hunted bear and lived off the grid, etc. He came in with advanced colon cancer and needed a colostomy, which he absolutely was not going to have - wasn't going to have his shit coming out his belly - like that. The charge nurse got someone - another big macho-type man - from the local colon cancer support group to come see him and he did relent and have the surgery, but the cancer had spread way too far to be curable.

On the other side of the coin were the people who were given the diagnosis of cancer and responded by saying, "It figures." These were the people who smoked, who ate red meat and meat fat, who ate eggs and sugar and butter, who were overweight, who drank way too much alcohol - they weren't surprised.

Obviously, this is only a tiny story about a tiny number of people who have been given that horrible diagnosis we all fear - and there's no way anyone can predict how we'll manage it when that day comes if it ever does, but my only point is not to expect all the determination and commitment in the world to a healthy life to keep you healthy - there are just too many variables.

I have to add that even within medical science that isn't considered pop science - the "real science" world - there are studies whose results clash and crash all the time; none of it's reliable in itself, but most of it has a grain of truth.
posted by aryma at 10:53 PM on March 23, 2015 [1 favorite]

And whatever they're telling you this week will be flipped to the opposite 20 years from now.

I don't think swapping one worn out nostrum for another is very helpful, you know? This "split the difference" approach is leveraged by people like tobacco companies, for example, to obscure and create uncertainty where none should be. Some things are carcinogenic, and history is littered with many examples of organisations working to obfuscate that reality.

It also promotes a discourse where science, indeed truth itself, is seen as inaccessible to the layperson, encouraging people to create their own realities, as it were. If the boat is leaking, let us build a better boat rather than give up seafaring altogether.

On the other side of the coin were the people who were given the diagnosis of cancer and responded by saying, "It figures." These were the people who smoked, who ate red meat and meat fat, who ate eggs and sugar and butter, who were overweight, who drank way too much alcohol - they weren't surprised.

Interestingly, I've read quite a bit of research - like this study - demonstrating how many people still minimise the most carcinogenic aspects of their lifestyles, and maximise ones where there is no clear link, or a very minor one.
posted by smoke at 2:20 AM on March 24, 2015 [3 favorites]

I work at Johns Hopkins Office of Research Administration. There are two parts to my office, one that deals with the government, non profit funding, and non human clinical research, and one that deals with for profit entities and human trial clinical research. I work with the latter that deals with for profit entities and can only speak from that perspective.

There is nothing that comes through my office that isn't "breakthrough" oriented, because profit. I work with oncology mostly and the sheer number of studies that are being done at Hopkins alone is mind boggling. It is also often insanely expensive to do these studies and with federal money being tighter than it used to be, our side of the ORA is busier than ever. For example, a Phase III trial that lasts several years can reach a million dollars in costs just at Hopkins, and whatever entity is funding that is likely doing studies at Mass General etc... as well.

But, this is part of how doctors make their money: clinician and PI.
posted by josher71 at 4:57 AM on March 24, 2015 [1 favorite]

Everyone I've seen whingeing and complaining about this "SUPER SERIOUS PROBLEM" had damn well better have been actually following real scientific efforts or they need to shut the fuck up. In my experience, few do.

In my experience, they all do, because the only people I've heard talk about this are career scientists. This is not a cyclical trend - it's a long term one way trend of forcing universities and institutes to be more 'business-like' by focusing on 'return on investment' and 'commercial crossover' etc.

Here is an open letter addressed to the Irish government last week from 800 Irish scientists, many very prominent internationally, including many friends and former colleagues of mine, complaining about the short-signtedness of Irish research policy.

I heard similar complaints from PIs about NIH funding etc. when I worked in UC Berkeley.
posted by kersplunk at 6:54 AM on March 24, 2015 [1 favorite]

I work at Johns Hopkins Office of Research Administration.

When our next Wilmer bill comes for something that was supposed to be covered by the study - which happens every visit like clockwork - I will now yell "curse you, josher71!" Which is going to seriously confuse everyone else but it's nice to have a specific person to (mis)blame.
posted by phearlez at 7:38 AM on March 24, 2015 [1 favorite]

I'm a university research administrator as well, for a university that is just starting to dip its toes into the clinical research world. We are already so good at media hype about the tech research we do (which is really good and really cool but maybe not quite as cool as reporters tend to claim), that I suspect the hype levels are going to be mind-boggling once we have clinical results to report.

We're definitely seeing a shift here in where research funds come from, government to for-profit entities, and it's both an interesting job challenge and job security for me as an employee, and sort of alarming to me as a person who would like my government to be funding science properly.
posted by Stacey at 9:03 AM on March 24, 2015

I will now yell "curse you, josher71!

I also work for ophthalmology! *hangs head*
posted by josher71 at 9:30 AM on March 24, 2015

I'm looking for a job at OHSU, though... (cough cough)
posted by josher71 at 9:31 AM on March 24, 2015

We're actually 99% sure this is someone screwing up coding in the actual Wilmer treatment office but those are people who sometimes make decisions about putting needles in my wife's eyes. So we're going to just blame faceless nicknames before we sigh and hand over the mistaken bills to the study coordinator. Thanks for taking one for the team.
posted by phearlez at 9:50 AM on March 24, 2015

I also work for ophthalmology! *hangs head*

You can always curse "Damn your eyes, josher71!"
posted by benzenedream at 12:08 PM on March 24, 2015 [1 favorite]