350 Years of All-Science, All the Time

May 14, 2015 3:36 PM

The Guardian celebrates 350 years of the publications of Philosophical Transactions of the Royal Society by asking are journals the best way to communicate science? They also propose another way.
posted by GenjiandProust (20 comments total)

The another way link method already raises my hackles just reading it. Especially in fields that are highly competitive, advertising your intent is not at all a great idea.
posted by dhruva at 4:33 PM on May 14, 2015 [3 favorites]

The protocol described sound that it would be fine for something like a clinical trial where it is important to establish the protocol early and stick to it. But so much of science is exploratory. Sure you have reasonably well defined goals at the start such "as we will characterise the subcellular trafficking of this set of proteins." But the interesting stuff is when you notice that one of the proteins is miss-trafficked in the cell when it has a mutation, and what do you know that mutation is associated with Parkinson's disease, to take an example from the research of some of the people I am lucky enough to work with. Results like that can't be predicted, though you try and establish a research environment and plan that will enable such things to be found. If you know what is going to happen, the system the article describes will work, but for most of science it would be putting it in a very constrained box.
posted by drnick at 4:39 PM on May 14, 2015 [4 favorites]

You can't afford to collaborate because all 300 of the postdocs doing DNA repair in Xenopus will be applying to the same faculty jobs as you, and there are only so many Nature papers to go around. Selfishness is rewarded and cross-lab collaboration is punished, except in rare cases (cross discipline, etc.).
posted by benzenedream at 4:45 PM on May 14, 2015 [1 favorite]

benzenedream: You can't afford to collaborate because all 300 of the postdocs doing DNA repair in Xenopus will be applying to the same faculty jobs as you, and there are only so many Nature papers to go around. Selfishness is rewarded and cross-lab collaboration is punished, except in rare cases (cross discipline, etc.).

I think this depends greatly on your field. My field was a little obscure and there was nobody else doing exactly what I was doing, so collaboration was only positive (since it could do much more with a team than alone).
posted by Mitrovarr at 4:53 PM on May 14, 2015

The proposal is really fascinating to me because it highlights a tension between an idealized picture of science as a process of inquiry and the reality of science as a career.

The proposal to have a refereeing process before the research has actually been performed seems great, since it will lead to less overall wasted time and energy on experiments with flawed designs. A good change, then, from the perspective of science as an idealized process of inquiry.

But at the same time it leads to a serious danger of having your project/idea scooped. (I think this is what dhruva is suggesting above, and it comes out in the comments of the Guardian essay as well.) So this would likely make things more tricky for junior scientists, since their success in the profession requires that they keep ownership of their ideas. A bad change, then, from the perspective of science as a career.

It could still be a good proposal on the whole, but what this suggests to me is that changes to the present model of scientific research are unlikely without some accompanying changes in how scientific careers are evaluated.
posted by voltairemodern at 4:59 PM on May 14, 2015 [4 favorites]

oh my god, in the "another way" thing people will obviously all just wait to submit the vague "preliminary" briefing until they've basically finished the research and know it's going to work, i.e., exactly the way that grant funding works now
posted by en forme de poire at 5:09 PM on May 14, 2015 [11 favorites]

In some ways, the current funding system already works like the proposal described in the "another way" link. To get competitive research grants from most biomedical agencies in the US, you have to specify the aims of your study and then demonstrate that you can achieve those aims by including "preliminary data."

There are two big problems with this approach. First, you need money to get preliminary data, which excludes smaller laboratories from competing in a lot of situations. In practice, the experimental plans proposed in most grant applications are already mostly complete because the early-stage projects aren't fundable yet.

Second, innovative ideas have often have a high likelihood of failure, and this funding process biases the field towards funding more conservative projects that everyone can agree on. I think this is probably the larger problem that needs to be addressed.
posted by thechameleon at 5:10 PM on May 14, 2015 [2 favorites]

haha, jinx
posted by en forme de poire at 5:12 PM on May 14, 2015

In my field it is:
1. Read everything in your area of research.
2. Perform simulation or experiment for something that has not been done before.
3. Perform experiment or simulation to match results from step 3.
4. Publish.

So since I am up to speed in my field I know it has not been done before and if I told my peers what I was about to do ( and it was good) then I would get scooped. I like the present system.
posted by sety at 5:13 PM on May 14, 2015

Much of the problem has to do with signal to noise ratio. For my PhD, my adviser required me to publish 4 peer-reviewed papers before I could graduate. I had essentially two major contributions in my research, and so rather than publishing two really strong papers, I published four papers that were more incremental. The result of all this is that the vast majority of what gets put out there only exists for grad students to satisfy their requirements, rather than being truly novel and interesting contributions to the community.

My experience is in the engineering field, are things similar in the sciences?
posted by jpdoane at 6:02 PM on May 14, 2015 [1 favorite]

That one time Newton was the secretary of the Royal Society and he engineered the official report on the calculus priority dispute and then there was an anonymous summary in Phil. Transactions which Newton totally wrote awwwww yeaaah
posted by grobstein at 6:03 PM on May 14, 2015 [3 favorites]

I mean, I'm not trying to be cynical or anything but isn't the purpose of publication about career advancement, not communication? There are already lots of alternative ways of communicating results, but career advancement in the sciences is what really needs to be reformed.

"Publish or perish" isn't some natural law...
posted by ennui.bz at 6:45 PM on May 14, 2015 [1 favorite]

I think the pressure around publishing is a much more pernicious aspect of science than the papers themselves, which is why I don't understand at all what the point of these articles is.

It's possible that the conference itself made a more convincing case, but to me, honestly, the ideas floated in the article seem pretty half baked, and that's where I can even tell what they're arguing. For example, the suggestion about having figure-only papers seems totally misguided. The however-many months people are spending on paper revisions isn't down to polishing prose, it's from running additional experiments -- either ones that reviewers suggested (because no paper as first submitted is ever "good enough"), or ones to better support the main line of argumentation -- and making the figures more convincing. Stripping the prose out is actually a pretty minimal savings of effort compared to the total amount of work that goes into one paper, and it would end up requiring more effort on the part of the people reading the publications (communication after all being a two-way street, ideally).
posted by en forme de poire at 9:20 PM on May 14, 2015 [1 favorite]

I've actually heard a lot of grumbling about how there are "too many" papers now, and I don't buy it. I don't think publishing 4 somewhat incremental papers vs. 2 more impressive papers presents any kind of actual problem, especially in the era of search engines and RSS feeds and .pdfs. I cited a bunch of small-scale "we did a few assays on a knockout strain" papers in modest-impact journals when I was writing my thesis, and they were no less valuable to my argument than e.g. CNS papers. There's obviously a point where salami shaving becomes extreme and ridiculous, but my sense is that, at that point, it doesn't actually help anyone's career... unless it's because of really specific messed up policy incentives like people getting bonuses for publishing, or getting promoted purely based on the length of their CV. In that case, though, those policies are the problem, not that there are "too many" papers in some kind of cosmic sense.
posted by en forme de poire at 9:32 PM on May 14, 2015 [1 favorite]

A look at Philisophical Transactions Volume 1.
posted by PenDevil at 10:44 PM on May 14, 2015 [2 favorites]

For possibly helpful context, the author of the article in the 'another way' link is Oxford Professor of Developmental Neuropsychology Dorothy Bishop who has a blog. I think she's the bee's knees, and I like her comments on other academic irritations (the REF) - and I like that she's proposing alternatives rather than just moaning.
posted by you must supply a verb at 3:45 AM on May 15, 2015 [1 favorite]

PenDevil's link has them showing Hooke reporting a moving spot on Jupiter in No. 1. I wonder what they would have thought about cats in scanners.
posted by bukvich at 7:49 AM on May 15, 2015

In my field the bigger problem is the the fields flagship journal , the APSR, is run by an editorial team that has questionable judgment. For example, they published two articles on Xenophon in one journal. This is in political science.

Also, half the stuff I read are working papers or seminar talks or early drafts. Then eventually become journal articles, often in notable places, but some papers can be well cited and circulated before they're even published.
posted by MisantropicPainforest at 9:42 AM on May 15, 2015 [1 favorite]

Did the articles on Xenophon make spurious claims to statistical significance? Did they report small effects that had been mined out of unreported multiple comparisons? It sounds like your problem is that some in your field disagree with you about what the important research topics are, which I sympathize with but which is probably unavoidable.

The point about working papers (etc.) is a good one, though. It seems like in many fields actual scholarly communication happens by fast, informal channels, and journal publication serves more of a post-hoc certification role. (This is corroborated by how many journal articles apparently go unread.)

It's good to have faster channels than journal publication, but the loss of formality is a real cost. It seems to mean (to communicate your work effectively) you have to learn a whole system of unwritten codes and be an ace socializer. It would be nice to have fast-but-formal communication channels to diminish the centrality of informal networking.
posted by grobstein at 10:06 AM on May 15, 2015 [1 favorite]

At least in my substratum of biology, it's becoming somewhat more common for people to submit pre-prints when they submit a paper draft for publication. I think that's a good move in the right direction. However, some publishers (though not nearly as many as I thought) can be really cagey/jerky about whether a pre-print constitutes "prior publication."

There is also the approach taken in various small conferences (Keystone, Gordon, etc.) where people submit unpublished work with the understanding that the work is in progress and the conference is a "safe space" with regard to scooping, etc. But these conferences do require you to secure access to the conference in the first place and that's obviously not something that's uniformly available (being in the right prestigious circles, being able to afford travel, etc.). Still, it might be possible to create broader formal channels with a similar kind of ethos, where there are strict, enforceable guidelines for how that information may be properly used.
posted by en forme de poire at 1:10 PM on May 15, 2015