The seven biggest problems facing science, according to scientists
July 14, 2016 6:02 PM   Subscribe

In the past several years, many scientists have become afflicted with a serious case of doubt — doubt in the very institution of science... So we sent scientists a survey asking this simple question: If you could change one thing about how science works today, what would it be and why? We heard back from 270 scientists all over the world, including graduate students, senior professors, laboratory heads, and Fields Medalists. They told us that, in a variety of ways, their careers are being hijacked by perverse incentives. The result is bad science.
posted by forza (39 comments total) 29 users marked this as a favorite
 
The deep, deep irony here is that this article fell prey to the need to make headlines: BAD SCIENCE!!!

There are many endemic and structural problems in academic science, for sure. Ask me on a different day and I'd be enumerating them. But the huge, huge leap from those problems to BAD SCIENCE!!! that this article makes is, honestly, deeply offensive to me.

I get up every day, face these problems, and still do the best science I can, thankyouverymuch.

BAD SCIENCE my ass.
posted by Dashy at 6:42 PM on July 14, 2016 [16 favorites]


Sure, but you can do great science on the wrong question, or great science but not enough of it, or great science but you can't finish the project because funding disappeared. We can't use science well as a society if we don't talk about the constraints on its practice which are social in origin, and the outcomes can definitely be, in a word, bad.
posted by Miko at 6:54 PM on July 14, 2016 [10 favorites]


We do have problems using science well as a society. But the underlying problem there isn't that the science is bad.

The outcomes of these constraints on science practice? Yes, limiting. But again, to broadly brush us all as producing bad science? Disgusting.
posted by Dashy at 7:03 PM on July 14, 2016 [2 favorites]


I'd also have to criticize their sampling of what is defined very broadly in this article as 'scientists'. As far as I can tell, they have not included any 'industry' scientists in the pool. Do we not count as scientists anymore?
posted by Tandem Affinity at 7:09 PM on July 14, 2016 [6 favorites]


Framing aside, the points raised by the article are all valid ones. There's a lot of good science out there, but there's also a lot of bad science mixed in with it. And mediocre science, and everything in between. If you're in a popular field, it can be overwhelming to keep up with all the relevant papers being published every month, let alone sift the good from the bad. And notably, even "good science" does not necessarily mean "correct conclusions", for various reasons.

I chose to do a PhD because I found biology fascinating, believed strongly in the medical research process, and wanted to contribute my tiny part. I'm trying to find a non-lab job now instead of a postdoc, in part because I've become very jaded about the research process (the other part is the soul-crushing amount of negative data/failed experiments/paper rejections). I did my very best to produce "good science", gaining a reputation for being very nitpicky, but I'm still not really convinced that my contribution to science was worth all the resources invested in me*. I would love to find a job where I could improve the efficiency of scientific research somehow but changing a huge system like that is such a daunting idea.

(*disclaimer: might still be the burnout talking, though it's been a few months now since my defense)
posted by randomnity at 7:32 PM on July 14, 2016 [9 favorites]


"In many cases the expectations were and often still are that faculty should cover at least 75 percent of the salary on grants,"

Unless, you coach football.
posted by Thorzdad at 7:39 PM on July 14, 2016 [7 favorites]


Honestly, I think there is a relatively easy* way to improve the efficiency of science: Have fewer graduate students, and more long-term, paid research positions. It's absurd to expect graduate students to be as efficient or insightful as long-term researchers in the field; it's the the title, we're students. We're still learning.

* I mean, it's easy when compared to drastically overhauling the system, but right now is not easy at all.
posted by Kutsuwamushi at 10:09 PM on July 14, 2016 [8 favorites]


Kinda hoped that "supervillains" would be in there somewhere, but I guess we're ignoring Lex Luthor and Elon Musk for the moment.
posted by klangklangston at 10:35 PM on July 14, 2016 [3 favorites]


Lots of tough problems to solve. Computational biology allows greater reproducibility, because software is almost always open source and available to test locally. Software that is closed-source is almost always laughed at, because it can't really be tested. That part is "solved". Wet-lab work makes things harder, but the computational part helps. Once the data sharing policies are worked out that protect the wet-lab people who put in a yeoman's effort to generate good data, it will be easier to share both algorithms and data, to see if the same answers come out from different labs. The real cancer in science is the money. You see people leave the public sector to work for or otherwise lobby for or defend the socially and scientifically poisonous efforts of the Monsantos of the world, with predictable results. Those who don't leave, you have a very small handful who fake their data and make their field look bad. Corrupted, moneyed science is a tumor that makes the rest of science look just as cancerous to outsiders.
posted by a lungful of dragon at 11:46 PM on July 14, 2016 [1 favorite]


Skimmed it, but looks like there's nothing wrong with science. There's a lot wrong with the incentives (or, to summarize, with money) but that by itself is not the fault of science.
posted by DreamerFi at 2:28 AM on July 15, 2016 [2 favorites]


Funding is slightly different in the UK

I work in admin in a research-focused, postgraduate institute within a large Russell Group university and to be an academic (Lecturer, Senior Lecturer, Reader, Professor), you will be on Government (HEFCE) funding. These people make up the large bulk of senior staff here. There are those on the research-only track and they will be funded from grant money (post-doc positions) or from Fellowships, i.e. Wellcome Trust, NIHR etc, but at the senior level they are in the minority.

With respect to publishing, peer-reviewed journals are a bloody cartel, no doubt, but that should change soon, again in the UK. In order to get funding from the government UK HEIs have to go through the rigmarole of a performance review (REF) every few years. As part of that, published papers are reviewed to see how "good" the outputs of research are. For the next round, only papers published in open access journals are submissible.
posted by fatfrank at 3:21 AM on July 15, 2016 [4 favorites]


there's nothing wrong with science

coming from a background in physics and mathematics, the "science" in biology looks like a very well funded attempt at pure Humean empiricism. between cheap grad student labor and technical advances it's become economic to produce a lot of data. "good" science, in this context, is an experiment that produces data on all of the plausible causal factors ie. even more data. in some sense this is inevitable since biologists are trying to reverse-engineer machines designed through a weighted iterative process ie. completely inscrutable, but one can't help but wonder whether more time could spent thinking about what is or isn't plausible and less time spent cranking out more data. and the thing is that data production in biology is entirely driven by economic factors: cost of lab labor, cost of new special mice, new special reagents, etc. what gets done is very much driven by what is economically viable, and that goes double for "scientific" medicine.

you start off with: "science is the scientific method: hypothesis, experiment, synthesis" and end up with "science is what experiments we can afford to do." maybe part of the problem is the model for what is considered science. maybe society can afford to pay people to sit around and think about things, rather than endless churning out data.
posted by ennui.bz at 3:34 AM on July 15, 2016 [2 favorites]


maybe part of the problem is the model for what is considered science. maybe society can afford to pay people to sit around and think about things, rather than endless churning out data.

They can, since this isn't the model for a lot of other disciplines. I'm not sure if that model is better, however.
posted by MisantropicPainforest at 3:50 AM on July 15, 2016


maybe society can afford to pay people to sit around and think about things, rather than endless churning out data.

I think you're looking for the philosophy department, just down the hall.
posted by IjonTichy at 3:55 AM on July 15, 2016 [3 favorites]


They can, since this isn't the model for a lot of other disciplines. I'm not sure if that model is better, however.

it's funny how saying it that way gets a reaction, but that's exactly what we do in math and theoretical physics.
posted by ennui.bz at 4:36 AM on July 15, 2016 [1 favorite]


Yes, different fields use different methodological tools. The tools which theoretical physicists use are not the appropriate tools to use to answer certain scientific questions, and it would be silly to conclude from this that those questions are not worth investigating.
posted by IjonTichy at 4:43 AM on July 15, 2016 [4 favorites]


Yes, different fields use different methodological tools. The tools which theoretical physicists use are not the appropriate tools to use to answer certain scientific questions

so biological science is a different sort of science from physical science? that's a novel philosophical viewpoint.
posted by ennui.bz at 4:52 AM on July 15, 2016


I do not agree that this statement is a reasonable summary of what I just said.
posted by IjonTichy at 4:59 AM on July 15, 2016 [4 favorites]


I do not agree that this statement is a reasonable summary of what I just said.

well, the methodology you were objecting to for biology was "sitting around and thinking" which is pretty generic. but that's kind of the problem with the "scientific method." in principle it's a procedure for artificial intelligence, you shouldn't need any thinkers to sit around if all you do is iterate experiments and generate statistical correlations.

the problem is that every experiment is essentially asking: Does X cause Y? Now, first you have to establish that Y actually exists, but then you also have to rule out every other causal factor other than X. This is clearly impossible because anything can be a causal factor. So, most experimental scientists set out to test all (or at least some) of the most *plausible* causal factors, which requires sitting around and thinking about what's plausible (this often doesn't work because the causal factor is implausible.)
posted by ennui.bz at 5:13 AM on July 15, 2016


coming from a background in physics and mathematics, the "science" in biology ...

Roger Penrose, is that you? I love the way you put everybody else's discipline in scare quotes... it's really just what happens when people get confused about the physics, amiright?
posted by kleinsteradikaleminderheit at 5:44 AM on July 15, 2016 [8 favorites]


Well, the thing about biological field flocking to use the latest and greatest (techniques, tools, strains, etc)--yes, absolutely, there is an opportunistic aspect to it. This is not a bug, it's a feature. If you're coming from a computational and mathematical background, you may not fully appreciate what a huge effort and investment is involved in generating something novel enough and useful enough to be an entirely new experimental platform, if you will. It's not like writing code from scratch.

I recently heard someone describe the state of knowledge in molecular biology and genetics, particularly in relation human health and disease, with this analogy. We--science, society--want the Google maps-level understanding of how organisms work, on a molecular and cellular level; to understand how it all works, and to develop safe and efficacious drugs, we need the biological equivalent to driving directions and traffic predictions. The problem is that the map isn't even remotely fully described: we have some really well-elaborated islands, but a lot of blank space between. So the challenge bioscience has now is to both do basic exploration and sophisticated analytics at the same time.

Riffing on that analogy--when someone makes a new (technique, reagent, strain), it's a really valuable way to expand the coastline of the islands we already understand.
posted by Sublimity at 5:51 AM on July 15, 2016 [3 favorites]


The section on perverse economics also missed a big point: grad students and post-docs are expected to live on poverty wages for five to ten years of their lives. (Extra special bonus: even if you're going to school somewhere with an absurd cost of living, you still get to live on the same tiny stipend, so placed like UCSF can't attract grad students) A friend of mine just finished her Ph.D in neuroscience, and after eight years of earning a pittance, she was given the choice between looking for a post-doc paying $30K a year or leaving the field entirely. Guess which one she chose.
posted by Mayor West at 5:54 AM on July 15, 2016 [6 favorites]


1. Republicans
2. Republicans
3. Republicans
4. Republicans
5. Republicans
6. Republicans
7. Lack of funding

The article focuses on the problem scientists face getting science done. That is the least of its worries. The biggest problem science has is that when science does get done, there is a significant segment of the nation's population that simply doesn't believe it. The more empirical evidence put in front of them, they ignore it in favor of whatever pundit was on FOX that morning or their televangelist.
posted by prepmonkey at 6:08 AM on July 15, 2016 [1 favorite]


the problem is that every experiment is essentially asking: Does X cause Y? Now, first you have to establish that Y actually exists, but then you also have to rule out every other causal factor other than X.

This doesn't make sense to me at all. If we are using a Rubin causal model then if X is the treatment and the treatment and control groups have covariate balance and random assignment, and there is a difference in Y for each group, then X causes Y. No real myster on that front.
posted by MisantropicPainforest at 6:43 AM on July 15, 2016 [1 favorite]


The biggest problem science has is that when science does get done, there is a significant segment of the nation's population that simply doesn't believe it. The more empirical evidence put in front of them, they ignore it in favor of whatever pundit was on FOX that morning or their televangelist.

The vast majority of scientific findings are ignored by the general public. Also those on the left are excellent at steadfastly resisting scientific findings if it makes them uncomfortable, look at , uh, any post on mefi where such a finding is reported.
posted by MisantropicPainforest at 6:54 AM on July 15, 2016 [3 favorites]


The section on perverse economics also missed a big point: grad students and post-docs are expected to live on poverty wages for five to ten years of their lives.

The low pay, long hours and long training period for young scientists is covered in the last section. Given that the first section is complaining that the competition among scientists for grants is too intense vast improvement option seem somewhat limited, unless you're going to be more aggressive about weeding out chunks of students in other ways.

It wasn't a bad article but bear in mind as discussing that a lot of the problems with science as identified by scientists are that they should get more pay for less work, along with more respect and less pressure to produce measurable advances.
posted by mark k at 6:59 AM on July 15, 2016 [1 favorite]


It wasn't a bad article but bear in mind as discussing that a lot of the problems with science as identified by scientists are that they should get more pay for less work, along with more respect and less pressure to produce measurable advances.

Thats...a misreading fo sho. Its the grad students who are saying they should get more pay. Considering many of them make less than 30k a year for 5+ years, thats a reasonable thing to want. I haven't heard of anyone complain about a lack of respect. from whom? students? no idea.

And its not 'measureable advances' I dont...know how one could read it that way. Unless you think that positive results are measurable advances and negative results are not. Which is a completely unjustified position.
posted by MisantropicPainforest at 7:13 AM on July 15, 2016 [2 favorites]


Computational biology allows greater reproducibility, because software is almost always open source and available to test locally. Software that is closed-source is almost always laughed at, because it can't really be tested. That part is "solved". Wet-lab work makes things harder, but the computational part helps

Look at what's actually going on in practice and the types of work not being reproduced. In practice the computational part also empowers researchers to walk into a mess of data and find something that is statistically significant and thus justify their grants or make a more 'interesting' claim. Anecdotal, but I've had informatics folks say they get called in at the end to do something obviously besides the point, and they say its because the PI said in their grant they'd do this thing so someone's going to do it.

Also you also can't use open source code and reanalysis to determine if a publishing lab is being coy about how many hypotheses they tested or what they originally expected their endpoints to be. The clean test-a-single-hypothesis experiment can be much more cut and dry.

So the computational part in the context of this article is quite capable of contributing to (1), (2), (3) and (6). I'm not saying its worse all the time but it is not a universal blessing.
posted by mark k at 7:14 AM on July 15, 2016


prepmonkey: "Republicans ... the nation..."

Remind me again... which nation would that be in which all the world's scientists live and work?
posted by langtonsant at 7:32 AM on July 15, 2016 [2 favorites]


1. Republicans
2. Republicans
3. Republicans
4. Republicans
5. Republicans
6. Republicans
7. Lack of funding


We have a distinct lack of Republicans here in Canada (although god knows Harper did a number on research), and all 7 problems exist here as well. There are some differences in the research process between countries, but these problems are seen everywhere to some extent. That's part of why they're so hard to solve - it's a global system, with many collaborations between researchers in different countries.
posted by randomnity at 8:47 AM on July 15, 2016 [4 favorites]


> and there is a difference in Y for each group, then X causes Y. No real myster on that front.

Biological systems are complex. There's also A, B, C, D, E, F, G, H.... &c&c that also affects Y, directly or indirectly.

Also, I, J, K, L, M., &c&c can also affect how X affects Y.

Also, C, D, E, F, &c&c can also affect X.

Also, I, J, K, &c&c can also affect how C, D, E affects X.

Also, B, E, J, K, &c&c are incompletely understood.

Also, p, q, r, s, t, &c&c haven't even been discovered yet, much less well understood.
posted by porpoise at 11:45 AM on July 15, 2016 [3 favorites]


Roche's Biochemical Pathways.
posted by porpoise at 11:48 AM on July 15, 2016


There are some differences in the research process between countries, but these problems are seen everywhere to some extent.

Dr. Poliakoff (of Periodic Table of Videos fame) talks here in some depth about the damage that Brexit will do to science funding in the UK, as well as longer-term damage to the British scientific community.
posted by a lungful of dragon at 12:52 PM on July 15, 2016


Pure research and science should NEVER be constrained by the requirement to justify it in ANY way; cost or results.
posted by Burn_IT at 1:32 PM on July 15, 2016 [2 favorites]


I think it's great that many have doubt in the very *institution* of science. Institutions are indeed the problem for science, much as they are the problem for spirituality, and for art, and in fact for most of humanity.

Science is an activity of free individuals and/or teams that are unfettered by constraints like practicality and return-on-investment. I prefer to reserve the term 'technology' for what the author's writing about.

Rather than go on at length, I'll again recommend reading "Against Method" by Feyerabend. A lovely person who had his finger on the pulse of the victim decades ago.
posted by Twang at 3:21 PM on July 15, 2016 [2 favorites]


Pure research and science should NEVER be constrained by the requirement to justify it in ANY way; cost or results.

Can't tell if joking.

If not please send me money. Don't ask any questions about why.
posted by mark k at 5:32 PM on July 15, 2016


Is one of the things 'science journalism'?

No? Fancy that.
posted by obiwanwasabi at 5:47 PM on July 15, 2016


I am required to publish two papers a year. I like publishing good papers, and two good papers a year is hard in my field. Plus, due to budget cuts, I don't have a technician, so I am doing the work of two people and forced to publish two papers a year. Plus, I have to find funding, so in addition to the two papers, I have to write two or three grant proposals. Plus, I spent my youth in poorly paying post-docs, so even though I now have a permanent job, I can never retire, but even as an elderly scientist I will have to be writing those two papers a year. Right now I throw up every morning from stress.

I agree with most of the points in the article.
posted by acrasis at 6:51 PM on July 15, 2016 [5 favorites]


Is one of the things 'science journalism'?

No? Fancy that.


What do you mean? Item 6:
Science journalism is often full of exaggerated, conflicting, or outright misleading claims. . . . [B]ad stories are peddled by university press shops . . . . Opinions differed on how to improve this sorry state of affairs — some pointed to the media, some to press offices, others to scientists themselves. . . . .Plenty of our respondents wished that more science journalists would move away from hyping single studies. Instead, they said, reporters ought to put new research findings in context, and pay more attention to the rigor of a study's methodology than to the splashiness of the end results.
I really dislike 99% of science reporting so I wish they'd done more, like singled out "sites like Vox" as part of the problem, but they do describe the issues with it.
posted by mark k at 7:04 PM on July 15, 2016 [1 favorite]


« Older Impressive singing contest breaks out on...   |   "Yelling at clouds" is my new euphemism for sex Newer »


This thread has been archived and is closed to new comments