Good science is boring science
August 6, 2015 8:39 AM Subscribe
A study published Wednesday in the journal PLOS ONE suggests that measures currently afoot in nearly every area of science to increase the transparency requirements for research will mean we can expect to see more of these seemingly dull results in the future -- and that's a good thing. Far from boring, those trials that find a drug doesn't do what we hoped can be equally as important -- or even more so -- than the ones that do.
More truth is good.
Hopefully publishing in the various journals of negative results will also become more prestigious.
posted by clawsoon at 9:15 AM on August 6, 2015
Hopefully publishing in the various journals of negative results will also become more prestigious.
posted by clawsoon at 9:15 AM on August 6, 2015
"Failure is an option."
posted by grefo at 9:40 AM on August 6, 2015 [3 favorites]
posted by grefo at 9:40 AM on August 6, 2015 [3 favorites]
This is immensely useful: I was diagnosed with with an obscure but deadly eye cancer in 2012, and because I'm in the genetic group most likely to get metastatic disease, I've been watching as new drug trials appear, register participants, launch to often great fanfare and then vanish -- without results, without closure, without any information at all.
For those of us who have a diagnosis of something presently untreatable and are following current research, it's immensely frustrating. Yes, "nothing happened" or "everyone died" or "all the participants got violently ill and dropped out" is helpful for the larger community, even if it isn't for the company that's hoping to recoup its money.
posted by jrochest at 9:56 AM on August 6, 2015 [4 favorites]
For those of us who have a diagnosis of something presently untreatable and are following current research, it's immensely frustrating. Yes, "nothing happened" or "everyone died" or "all the participants got violently ill and dropped out" is helpful for the larger community, even if it isn't for the company that's hoping to recoup its money.
posted by jrochest at 9:56 AM on August 6, 2015 [4 favorites]
Ark B Scientists?
posted by blue_beetle at 9:59 AM on August 6, 2015
posted by blue_beetle at 9:59 AM on August 6, 2015
So many good things are boring when done properly. Good engineering work is often boring. Good technology work is often boring. good science is definitely often (but not always) boring. Good government should absolutely, definitely be boring more often than it is. Not everything has to be a drastic change or a critical breakthrough.
The problem is that boring isn't rewarded. This is something I see in the technology field all of the time -- The good devs and systems guys are often invisible. This is especially true at the systems level - If you are doing your job right, nobody even really knows you are doing it at all unless you are incredibly transparent. Typically, it's the guys who come in to rescue the day who get the attention - despite the fact that they are rescuing people from a problem that was created on their own. I think many people in government are trying to create a similar narrative... The problem is - it's easy to select for the sucess stories, and it generally works, unless you TRULY have transparency across the board.
On the science/research side, I don't think you really have the same self-created crisis - but you have the same imperative to produce successful results. If your outcome doesn't live up to your hypothesis, then its widely seen as a failure - despite the fact that these failures are absolutely critical to how science works. If our hypothesis was always correct, we wouldn't even need the method itself.
Make no mistake, more transparency is a great thing. The only problem I see is cultural - The culture of the USA in particular is entirely too critical and punishing of failure across the board, which encourages people to fabricate or hide results. You can see this everywhere you look, and it's a major contributor to corruption. While I'm a big fan of increased transparency across the board, it raises many opportunities for "Why are we funding this, it's a failure" as an argument - often a valid discussion if a corporate division is not living up to expectations.
The game changes a bit when you are discussing public services or research. Too often, elements of failure will be selected for political purposes - elements that, when viewed through the context of the program or institution itself, are not only unfair to the overall picture, but are often an expected part of it. The problem is that we tend to focus on the fact that there is any failure whatsoever - We believe that these organizations should run with a 0% failure rate, and any failure means that the organization is failing overall. And until that changes, it will continue to be used as a political weapon, and it will be seen as a liability.
One particularly strong argument against privatizing many public services is that it becomes entirely too easy to paint a false picture of success when everything that lead up to that can be concealed. Not that there isn't concealment in public services as well -- but there tends to be much more oversight and transparency, as a rule, although we need to keep fighting for more. The problem is that viewed under these cultural eyes, the public program will always be "burdened with more failure" - because they are the ones with the obligation to share that data.
posted by MysticMCJ at 10:33 AM on August 6, 2015 [4 favorites]
The problem is that boring isn't rewarded. This is something I see in the technology field all of the time -- The good devs and systems guys are often invisible. This is especially true at the systems level - If you are doing your job right, nobody even really knows you are doing it at all unless you are incredibly transparent. Typically, it's the guys who come in to rescue the day who get the attention - despite the fact that they are rescuing people from a problem that was created on their own. I think many people in government are trying to create a similar narrative... The problem is - it's easy to select for the sucess stories, and it generally works, unless you TRULY have transparency across the board.
On the science/research side, I don't think you really have the same self-created crisis - but you have the same imperative to produce successful results. If your outcome doesn't live up to your hypothesis, then its widely seen as a failure - despite the fact that these failures are absolutely critical to how science works. If our hypothesis was always correct, we wouldn't even need the method itself.
Make no mistake, more transparency is a great thing. The only problem I see is cultural - The culture of the USA in particular is entirely too critical and punishing of failure across the board, which encourages people to fabricate or hide results. You can see this everywhere you look, and it's a major contributor to corruption. While I'm a big fan of increased transparency across the board, it raises many opportunities for "Why are we funding this, it's a failure" as an argument - often a valid discussion if a corporate division is not living up to expectations.
The game changes a bit when you are discussing public services or research. Too often, elements of failure will be selected for political purposes - elements that, when viewed through the context of the program or institution itself, are not only unfair to the overall picture, but are often an expected part of it. The problem is that we tend to focus on the fact that there is any failure whatsoever - We believe that these organizations should run with a 0% failure rate, and any failure means that the organization is failing overall. And until that changes, it will continue to be used as a political weapon, and it will be seen as a liability.
One particularly strong argument against privatizing many public services is that it becomes entirely too easy to paint a false picture of success when everything that lead up to that can be concealed. Not that there isn't concealment in public services as well -- but there tends to be much more oversight and transparency, as a rule, although we need to keep fighting for more. The problem is that viewed under these cultural eyes, the public program will always be "burdened with more failure" - because they are the ones with the obligation to share that data.
posted by MysticMCJ at 10:33 AM on August 6, 2015 [4 favorites]
As a derail, the second biggest thing against privatization is that it stimulates a profit motive, so that even if the lowest pricepoint is reached through competition (and that unseen hand itself is often a myth), it will be more expensive, per definition, because a profit must be made.
The biggest issue? That profit will be arrived at by any means, even if that means that the service/good will be overpriced (maybe even to the point of out-pricing the poor) and/or diluted/bad quality, as long as that profit is made.
Transparency would solve that, though, by showing how it is achieved.
But in science transparency will also lead to less waste; things which failed will not have to be repeated. The failures by the wayside could literally point to the path to be taken. And the way things failed might show other paths.
There is one piece missing, however, and that is a good way of sorting and visualising all that data. A list of publications which all need to be read to form a mental map just isn't good enough. I long for the day when universities (and any research organisation) have a central map of work to which these things are contributed to, in some form which allows for a clearer insight.
posted by MacD at 11:29 AM on August 6, 2015 [1 favorite]
The biggest issue? That profit will be arrived at by any means, even if that means that the service/good will be overpriced (maybe even to the point of out-pricing the poor) and/or diluted/bad quality, as long as that profit is made.
Transparency would solve that, though, by showing how it is achieved.
But in science transparency will also lead to less waste; things which failed will not have to be repeated. The failures by the wayside could literally point to the path to be taken. And the way things failed might show other paths.
There is one piece missing, however, and that is a good way of sorting and visualising all that data. A list of publications which all need to be read to form a mental map just isn't good enough. I long for the day when universities (and any research organisation) have a central map of work to which these things are contributed to, in some form which allows for a clearer insight.
posted by MacD at 11:29 AM on August 6, 2015 [1 favorite]
The issue of whether or not something is boring is complex, and "boringness" depends on the perspective from which the research is being evaluated, and how closely it is evaluated. Research in psychology that may seem very interesting to the layperson may be boring to someone with more training. For instance, research making claims about the supposed brilliance of infants (e.g., claiming that they have proto-concepts of mathematics, intentionality, physics, and so forth) is much less interesting when one examines the findings closely, which (IMO) reveals how open to other (much less compelling) interpretations they are.
posted by patrickdbyers at 12:30 PM on August 6, 2015
posted by patrickdbyers at 12:30 PM on August 6, 2015
The Journal of Negative Results in Zoology, for example, is just called “not seeing animals.” And the Journal of Negative Results in Homeopathy is the entire field of homeopathy.
posted by clawsoon at 12:31 PM on August 6, 2015 [1 favorite]
posted by clawsoon at 12:31 PM on August 6, 2015 [1 favorite]
« Older #iLookLikeAnEngineer | 100 Years of ... Newer »
This thread has been archived and is closed to new comments
posted by pipeski at 8:54 AM on August 6, 2015