In Nature, the biggest study on gender citation gaps EVER!
December 13, 2013 8:34 AM   Subscribe

We analysed 5,483,841 research papers and review articles with 27,329,915 authorships. We find that in the most productive countries, all articles with women in dominant author positions receive fewer citations than those with men in the same positions. And this citation disadvantage is accentuated by the fact that women's publication portfolios are more domestic than their male colleagues — they profit less from the extra citations that international collaborations accrue. Given that citations now play a central part in the evaluation of researchers, this situation can only worsen gender disparities. The data are also used to make a really cool interactive map.
posted by MisantropicPainforest (53 comments total) 18 users marked this as a favorite
 
Until citation metrics can distinguish between "This result confirms the hypothesis of Bloggs (2013)" and "This result disproves the hypothesis of Bloggs (2013)" the entire industry can, as one of my students succinctly put it, "get to fuck". It's a literal echo chamber and will be looked back on as a black period in civilisation.
posted by cromagnon at 8:40 AM on December 13, 2013 [5 favorites]


Perhaps "quantify everything" should be reconsidered. Googles link counting may value a highly linked example of something wrong or bad, and the academic citation counts are perhaps giving value to prestige or an idea that in fashion rather than some intrinsic information of value?
posted by sammyo at 8:43 AM on December 13, 2013 [2 favorites]


“Quantification is a powerful agency of standardization because it imposes order on hazy thinking, but this depends on the license it provides to ignore or reconfigure much of what is difficult or obscure” - Theodore Porter Trust in Numbers
posted by Dr. Send at 8:50 AM on December 13, 2013 [3 favorites]


Perhaps "quantify everything" should be reconsidered. Googles link counting may value a highly linked example of something wrong or bad, and the academic citation counts are perhaps giving value to prestige or an idea that in fashion rather than some intrinsic information of value?

Or -- OR -- maybe there's actually a gender gap in how scientists are viewed and valued, and that shows up in some easily quantifiable places such as citation counts as well as places where the pernicious influence of institutionalized sexism is harder to identify?

It's a little tiresome how these discussions always go "well, that's just anecdata, surely if there were actually massive institutionalized gender bias we would be able to generate some numbers on it" and then when numbers get generated, it's all "Perhaps 'quantify everything' should be reconsidered."
posted by KathrynT at 8:54 AM on December 13, 2013 [42 favorites]


I have literally never seen a citation in an academic work that cites something because it is "bad".

Until citation metrics can distinguish between "This result confirms the hypothesis of Bloggs (2013)" and "This result disproves the hypothesis of Bloggs (2013)"

Why would that distinction matter?
posted by MisantropicPainforest at 9:03 AM on December 13, 2013 [1 favorite]


Let's see. So they admit that "There is no consensus on the reasons for these gender differences in research output and collaboration — whether it is down to bias, childbearing and rearing, or other variables" and yet this doesn't stop them from issuing a nebulous call for "action" to remove unspecified "barriers" to women in science.

What if the "barrier" is that it takes obsessive, inordinate, monomaniacal time and devotion at the expense of family and everything else to publish quality, highly-cited research, and that maybe women are less prone to wanting to live this sort of admittedly unbalanced life?
posted by shivohum at 9:04 AM on December 13, 2013 [2 favorites]


Or -- OR -- maybe there's actually a gender gap in how scientists are viewed and valued, and that shows up in some easily quantifiable places such as citation counts as well as places where the pernicious influence of institutionalized sexism is harder to identify?

Especially when this fits with previous studies, like the one where scientists, both male and female, scored female lab applicants lower and offered lower starting salaries.
posted by Dip Flash at 9:05 AM on December 13, 2013 [12 favorites]


Yes, clearly the issue is that women just don't want it as bad as men do.
posted by MisantropicPainforest at 9:09 AM on December 13, 2013 [7 favorites]


What if the "barrier" is that it takes obsessive, inordinate, monomaniacal time and devotion at the expense of family and everything else to publish quality, highly-cited research, and that maybe women are less prone to wanting to live this sort of admittedly unbalanced life?

Then maybe we should look at why women are less prone to wanting to live that kind of unbalanced life, and fix THAT. Or maybe we should try reforming standards of the industry such that obsessive, inordinate, monomaniacal time and devotion at the expense of family and everything else isn't required in order to work in the field.
posted by KathrynT at 9:09 AM on December 13, 2013 [21 favorites]


Why would that distinction matter?

Well, I'd love this kind of h-index if I were on the job market.
posted by cromagnon at 9:39 AM on December 13, 2013 [1 favorite]


To fully connect the dots here: People, university profs, but also government and industry scientists, are hired and fired based on the numbers given in the Nature article. Authorships and citations are as important in science right now as RBI and on-base percentages are to pro baseball players. If women are denied the opportunites to even "step up to the plate", to get onto big progects, to be part of the research consortia that are increasingly important to science, they can't get the same numbers their male colleagues do. So fewer women getting opportunities leads to lower career sucess, which leads to fewer opportunities. It's a classic viscious cycle.

This study has been long needed. It's disheartenting to see how far we still need to go, but these numbers are pretty indisputable, in my view.
posted by bonehead at 9:49 AM on December 13, 2013 [12 favorites]


One of the joys of science is reading articles and being color-blinded and often gender-blinded. Many journals (most?) publish initials instead of first names.

The Michaelis-Menten equation? Maud Menten.
posted by dances_with_sneetches at 9:57 AM on December 13, 2013 [3 favorites]


There's a hashtag on Twitter for people who calculate their own personal gender gap (#MyGenderGap) following a blog post by The Lab & Field on how to calculate it.

My gender gap is better than I expected due to a single paper published as an undergrad. Otherwise, I am 0% advised (PI and committee), taught, or published with females, except my current supervisor. Which, as a feminist scientist, is absolutely horrifying.

Yes, part of the issue is that women don't want it badly enough are doing other things. But one of the common things they are doing is raising children for their scientist husbands, at least for my friends.

But to either of the above issues - I don't know what I should do personally. I might explicitly seek out other women to publish with. And encourage my friends' efforts to rejoin the academic/publishing world.

One of the joys of science is reading articles and being color-blinded and often gender-blinded. I actually published under my initials because I have heard women are judged more harshly and cited less. It's not a great sign of equality in science; it's my way of hiding my identity so that I can get a fair crack at things.
posted by hydrobatidae at 10:27 AM on December 13, 2013 [1 favorite]


I actually published under my initials because I have heard women are judged more harshly and cited less. It's not a great sign of equality in science; it's my way of hiding my identity so that I can get a fair crack at things.

By the point in your career where you're getting NSF grants, publishing in high-profile journals, and collaborating internationally with top researchers, your gender identity will be well known. My institution doesn't pay for Nature, so unfortunately I can't read this. But what is the mechanism they're proposing - is it that female-led papers are being cited less because people read them and discount them because they're written by females (the "resume" effect)? Or is it because well-known researchers are better-cited, leading to higher profile publications, leading to more recognition and more citations, etc.? It's easier to get citations at all levels if you're a well-known researcher than not. The problem is that females are given fewer opportunities to promote their research.
posted by one_bean at 10:36 AM on December 13, 2013


On perception of the gender gap: I had a fantastic sociology teacher who related the story of an experiment she did in one of her classes once. Throughout the semester, she kept track of the number of times she had called on males and females and made sure she called on them equally. She had multiple students in that class tell her that she called on the females more than the males, that she was not being equal, etc. Ironically.
posted by aniola at 10:40 AM on December 13, 2013 [11 favorites]


The authors are not proposing any solutions (other than careful consideration of the data), but that's not their job either. This is an important piece of the conversation about gender equality in science, but it's not an attempt to solve all of those problems.
posted by bonehead at 10:43 AM on December 13, 2013


It's a little tiresome how these discussions always go "well, that's just anecdata, surely if there were actually massive institutionalized gender bias we would be able to generate some numbers on it" and then when numbers get generated, it's all "Perhaps 'quantify everything' should be reconsidered."

A core issue that the article leaves out is that so many women just keep blundering ahead with their failed strategy of being a woman. Of all those scientists, how many have even tried not being a woman?
posted by ROU_Xenophobe at 10:54 AM on December 13, 2013 [18 favorites]


Here is a great previous FPP about women in science, and it links to the FPP about the study of gender in science that I mentioned.

A core issue that the article leaves out is that so many women just keep blundering ahead with their failed strategy of being a woman. Of all those scientists, how many have even tried not being a woman?

I haven't reread the links in the previous FPPs to see if it is in there, but I can recall in the last year or so a fascinating section of an article describing changed perceptions after a scientist had transitioned genders. In other words, that experiment has been tried, over and above the small things like using initials as described above.
posted by Dip Flash at 11:08 AM on December 13, 2013 [1 favorite]


one_bean, they haven't proposed a mechanism. But the highly cited PI and the my-first-paper undergrad are equally weighted in the analysis so you should get a combination of both methods. Although if PI are selecting fewer female co-authors, you get a multiplicative effect.
posted by hydrobatidae at 11:09 AM on December 13, 2013


My internet is too dreadful at the moment to even try to download that article, but I assume there's a break-down by discipline in there? That will be interesting to see.

By a quick browse through my pubs list, it looks like over 50% of mine have female co-authors. On the other hand, 100% have male co-authours...
posted by Jimbob at 12:01 PM on December 13, 2013 [1 favorite]


So I'm finding it funny that the first N comments here were all about how CITATION METRICS SUCK.

Because yeah, okay, they do suck for most purposes — but this sort of study has got to be the one thing they're actually good for.

We all know they don't measure quality. We all know they measure things like "how much mainstream attention have you gotten?" and "how much prestige do you have?" But okay, here someone's testing the hypothesis that women get disproportionately mainstream little attention and prestige — So hey! Perfect! It works! What better measure could you ask for in this situation?
posted by Now there are two. There are two _______. at 12:30 PM on December 13, 2013 [2 favorites]


As a female academic, I think most of my co-authored papers are published together with other women. But that's because I tend to co-author stuff with my friends. We'll be sitting around and have an idea for a joint paper and write it together. Most of my male colleagues have more co-authored papers published with their mentors and supervisors (who tend to be male, because every single tenured academic in our department is male). None of these senior men have ever offered to co-author anything with me, and as such a junior person, I'd feel weird approaching them about it. On the third hand, feeling weird approaching a senior person to ask them to spend some time on something important to you is an experience most of my female colleagues share, and my male colleagues do not seem to.
posted by lollusc at 1:18 PM on December 13, 2013 [4 favorites]


One of the joys of science is reading articles and being color-blind and often gender-blinded. Many journals (most?) publish initials instead of first names.

Once you're writing and publishing in a field, however, you must choose who to cite; and those are people whose gender you know well by then.
posted by Dashy at 2:12 PM on December 13, 2013


I have literally never seen a citation in an academic work that cites something because it is "bad".

I have cited papers for the sole purpose of mentioning that they are incorrect.
posted by erniepan at 5:57 PM on December 13, 2013 [1 favorite]


Then maybe we should look at why women are less prone to wanting to live that kind of unbalanced life, and fix THAT. Or maybe we should try reforming standards of the industry such that obsessive, inordinate, monomaniacal time and devotion at the expense of family and everything else isn't required in order to work in the field.

How can you fix people's desires? People will do what makes them happy and if pouring themselves into their research at the cost of their family is not what they want (or vice versa), who are you to suggest otherwise?

Your second suggestion that "industry standards" of good research should be fixed so that it takes less devotion is ridiculous. Good research is incontrovertible — the definition can't be changed. If it could be done quickly, someone else would have usually done it.

Or -- OR -- maybe there's actually a gender gap in how scientists are viewed and valued, and that shows up in some easily quantifiable places such as citation counts as well as places where the pernicious influence of institutionalized sexism is harder to identify?

If they want to establish a causal relationship, they need to account for confounders by making more measurements. Arguing from a correlation is not convincing.
posted by esprit de l'escalier at 11:02 PM on December 13, 2013


How can you fix people's desires? People will do what makes them happy and if pouring themselves into their research at the cost of their family is not what they want (or vice versa), who are you to suggest otherwise?

My suggestion is that there are sexist reasons why it is so much harder for women to balance science careers and a family than it is for men. We HAVE to figure this out -- we can't afford to lose half our brilliant scientists to an impossible work-life balance.
posted by KathrynT at 11:22 PM on December 13, 2013 [4 favorites]


Yes, you're right about that. It seems hard to fix that because it comes down to individual choices.
posted by esprit de l'escalier at 11:27 PM on December 13, 2013


Yes, you're right about that. It seems hard to fix that because it comes down to individual choices complex structural factors that are hard to even see because they are perceived as individual choices.

Seriously, this is gender-in-academia 101. There is increasingly good research (including the study in this FPP) quantifying the way this is not about women being intrinsically less motivated or smart or whatever.

Good research is incontrovertible — the definition can't be changed. If it could be done quickly, someone else would have usually done it.

No one in academia is suggesting lowering standards for good research. But people are noting that the quantity of research required often assumes you are not a primary caregiver, are willing to make serious household sacrifices, and perhaps even have a supportive spouse who will type your papers like in the good old days. Refusing to see the embeddedness of gender in the way academia is constructed is not a helpful starting point.
posted by Dip Flash at 6:09 AM on December 14, 2013 [6 favorites]


A core issue that the article leaves out is that so many women just keep blundering ahead with their failed strategy of being a woman. Of all those scientists, how many have even tried not being a woman?

At least one.
posted by bile and syntax at 7:02 AM on December 14, 2013 [1 favorite]


Refusing to see the embeddedness of gender in the way academia is constructed is not a helpful starting point.

What do you imagine that academia can do for someone who is "a primary caregiver, [and is not] willing to make serious household sacrifices, and [has no] supportive spouse"?

Yes, you're right about that. It seems hard to fix that because it comes down to individual choices complex structural factors that are hard to even see because they are perceived as individual choices.

There may exist structural factors, but that's not an argument against the fact that individuals make choices within their couples. I don't see how the government can impose upon couples a different value system and I don't think it should. You might remunerate academics more, which a couple may divert towards childcare, but that money would also attract people who had left academia for the private sector.
posted by esprit de l'escalier at 7:47 AM on December 14, 2013


What do you imagine that academia can do for someone who is "a primary caregiver, [and is not] willing to make serious household sacrifices, and [has no] supportive spouse"?

This is an issue people have been grappling with for 30+ years. What is new is the increasingly unequivocal research showing that it is not just individual choices. What isn't new is the continued denial of this structural element, both within academia and in discussions like this.

Most places by now have procedures in place for the first order problems, like flexible tenure clocks, etc.

More forward thinking places are working on the second order problems, like how to account for, acknowledge, and protect against unequal service expectations for women and diverse faculty.

Research like this points to a set of third order problems -- even if women are successful in researching, writing, and publishing, what about gendered distribution of citations and coauthorships? There has been similar research into student evaluations of female and diverse faculty, too. Remember that these things are used for hiring, promotion, and salary decisions, so pretending they are value-neutral meritocratic measures masks serious biases and structural barriers.
posted by Dip Flash at 8:50 AM on December 14, 2013 [2 favorites]


This is an issue people have been grappling with for 30+ years. What is new is the increasingly unequivocal research showing that it is not just individual choices.

No, it is impossible to show such a thing unequivocally. How can you anyone know where individual influence ends and societal influence begins? Society is made of individuals and so ultimately the social structures that lead us into conflict are the responsibility of individuals.
posted by esprit de l'escalier at 9:09 AM on December 14, 2013


Reading over the article again, the bold conclusion is ridiculous: ”Our study lends solid quantitative support to what is intuitively known…” First of all, it is revealing the authors' bias, which only discredits them, and second, correlation is not very good "quantitative support" after acknowledging a major confounder: ”age indisputably has a role — perhaps even the major role — in explaining gender differences in scientific output”!

They should do their study again estimating age using the date of the author's first paper, and removing unproven intuitions that weaken their credibility. It would be better still to have more than 2 explanatory variables.
posted by esprit de l'escalier at 9:24 AM on December 14, 2013 [1 favorite]


What do you imagine that academia can do for someone who is "a primary caregiver,

Quality work-based childcare would help a lot of primary carer and single parent academics/researchers continue their professional lives.
posted by Kerasia at 8:14 PM on December 14, 2013 [3 favorites]


I have literally never seen a citation in an academic work that cites something because it is "bad".

I have cited papers for the sole purpose of mentioning that they are incorrect.

Incorrect doesn't mean 'bad'. You don't cite papers by undergrads because they are shitty, do you?
posted by MisantropicPainforest at 7:24 AM on December 16, 2013 [1 favorite]


Reading over the article again, the bold conclusion is ridiculous: ”Our study lends solid quantitative support to what is intuitively known…” First of all, it is revealing the authors' bias, which only discredits them, and second, correlation is not very good "quantitative support" after acknowledging a major confounder: ”age indisputably has a role — perhaps even the major role — in explaining gender differences in scientific output”!

They should do their study again estimating age using the date of the author's first paper, and removing unproven intuitions that weaken their credibility. It would be better still to have more than 2 explanatory variables.


Your incessent harping on 'correlation does not equal causation', 'biases' in a scientist because they have intuition, and 'proof' tells me that your relationship with social science is, at best, a passing familiarity. This study is simply masterfully conducted.
posted by MisantropicPainforest at 7:27 AM on December 16, 2013 [2 favorites]


Your incessent harping on 'correlation does not equal causation', 'biases' in a scientist because they have intuition, and 'proof' tells me that your relationship with social science is, at best, a passing familiarity. This study is simply masterfully conducted.

The reason that revealing your intuition discredits attempts to prove causation is well understood in statistics (and now e.g. biology, computer science, and epidemiology) and I will explain it below. It is unfortunate if social scientists don't understand what makes a good experiment because not knowing invalidates their conclusions.

The reason that revealing "intuition about the conclusion you expect to draw" is discrediting is that there is a universe of possible models that can be chosen. He might choose to do a linear regression between independent variables X1 (gender) and X2 (age) against dependent variable Y (citation probability). He may also add an interaction term of the product of X1 and X2. He may choose to consider some nonlinear terms f(X1). He may do binning on the ages (old, young). And so on.

Of this infinite universe of models, some may show a positive relationship X1 between X1 and Y, some may show a negative relationship. Since he has decided what he would like to show in advance, he can simply choose the model that shows what he wants to prove and no one can verify that he has not done this.

This is why it is critical to do some kind of validation, which does not at all show up in his study.

The reason that positive correlation with few explanatory variables is unconvincing is because of Simpson's paradox. It is easy for a correlation to be reversed when confounders are taken into account. See a study on a very similar topic:
Examination of aggregate data on graduate admissions to the University of California, Berkeley, for fall 1973 shows a clear but misleading pattern of bias against female applicants. Examination of the disaggregated data reveals few decision-making units that show statistically significant departures from expected frequencies of female admissions, and about as many units appear to favor women as to favor men. If the data are properly pooled, taking into account the autonomy of departmental decision making, thus correcting for the tendency of women to apply to graduate departments that are more difficult for applicants of either sex to enter, there is a small but statistically significant bias in favor of women. The graduate departments that are easier to enter tend to be those that require more mathematics in the undergraduate preparatory curriculum. The bias in the aggregated data stems not from any pattern of discrimination on the part of admissions committees, which seem quite fair on the whole, but apparently from prior screening at earlier levels of the educational system. Women are shunted by their socialization and education toward fields of graduate study that are generally more crowded, less productive of completed degrees, and less well funded, and that frequently offer poorer professional employment prospects.

If you think this study is "masterfully conducted", I humbly suggest that you take a statistics course. (Honestly, do you know what cross validation is? Do you know about Simpson's paradox? Or are you simply dazzled by graphs?)
posted by esprit de l'escalier at 9:25 AM on December 16, 2013


I see that you just started your phd in social science? Why don't you take the opportunity to talk to your phd supervisor about repairing methodological errors in this study? Get the data from the study authors. If you don't know how to do good statistical modeling, find someone who can teach you. You could publish your first paper right out of the gates.
posted by esprit de l'escalier at 10:01 AM on December 16, 2013


The reason that revealing your intuition discredits attempts to prove causation is well understood in statistics (and now e.g. biology, computer science, and epidemiology) and I will explain it below.

This is just asinine.

The idea that revealing your intuition discredits anything is laughable on its face, for two reasons. First, any research product needs a firm set of intuition; this is what we call theory, or at least what theory is based on, and empirical research in the absence of theory is just data mining. Second, given that a researcher should have a theory, it's absolutely incumbent on the researcher to clearly and distinctly state what their theory is. Or, in your terms, what their intuition is.

Of this infinite universe of models, some may show a positive relationship X1 between X1 and Y, some may show a negative relationship. Since he has decided what he would like to show in advance, he can simply choose the model that shows what he wants to prove and no one can verify that he has not done this.

Most obviously, this would still be the case if the researcher didn't reveal what their intuition was, except that it would be even harder to verify that the researcher hadn't done that.

But this is really an empty criticism. If it is the case that their model is the only model specification that ``works,'' this will become obvious when sixty zillion graduate students who also want a publication in Nature submit their hit pieces. Frankly, it would have been obvious in the review stage, when reviewers would have been virtually certain to demand some manner of robustness checks if the models looked a bit hinky. If you want to disprove the piece, do so your own damn self. Scientific research is an interactive process between researchers, not a ``once and done'' affair.
posted by ROU_Xenophobe at 11:12 AM on December 16, 2013


But it's still not masterful. Like so many studies, it goes straight after what I think of as the Big Dumb Prediction, ignoring any other aspects of whatever they think the causal process is.* A better study would have also looked for empirical corroboration of other aspects of their causal story, especially bits that seemed, on the surface, to be not directly connected to the treatment of women in the sciences.

*You can't really do this in a course, but I always think of the sex education scene from Meaning of Life: You don't have to leap straight for the clitoris like a bull at the gate! What's wrong with a kiss, boy?
posted by ROU_Xenophobe at 11:19 AM on December 16, 2013


The idea that revealing your intuition discredits anything is laughable on its face, for two reasons. First, any research product needs a firm set of intuition; this is what we call theory, or at least what theory is based on, and empirical research in the absence of theory is just data mining. Second, given that a researcher should have a theory, it's absolutely incumbent on the researcher to clearly and distinctly state what their theory is. Or, in your terms, what their intuition is.


That's fine provided you validate your result to remove the bias introduced by possibly rejecting models that don't fit your theory. This is really common with selective reporting of results by pharmaceutical companies.

it would have been obvious in the review stage

By their own admission, that they are neglecting a confounder that could easily reverse their result. Did you read the study I cited whereby "a clear but misleading pattern of bias" was uncovered? Does that sound familiar?

One doesn't need a counter-model to be unmoved by such poor modeling.
posted by esprit de l'escalier at 11:23 AM on December 16, 2013


Yeah, one actually does. If you want to disprove it, disprove it.
posted by ROU_Xenophobe at 11:40 AM on December 16, 2013


To be more specific, it's the science version of "What a wuss. I could totally beat him up." Well, then, do it.
posted by ROU_Xenophobe at 11:54 AM on December 16, 2013


No, that's not how it works. It's his job to convince us, not ours to disprove his hypotheses. It is not "accepted until disproved", but "rejected before a compelling argument".
posted by esprit de l'escalier at 12:09 PM on December 16, 2013


And he offered a peer-reviewed argument. Got one better? Subject it to peer review.
posted by ROU_Xenophobe at 12:26 PM on December 16, 2013 [2 favorites]


I've already explained in detail the methodological problems with the study. Simply being peer reviewed doesn't mean that the conclusions of the study are now accepted by the world as fact. It is pretty understandable that many of their peers who did the review were not statisticians and do not have the background to understand what makes a convincing argument. Did you understand my argument? This "appeal to (anonymous) authority" that you keep insisting on is inferior to argumentation from statistical principles.

Let's work through it together.

We can both agree, that this study asserts a mere correlation between two variables taking no confounders into account and that the study admits that at least one significant confounder exists. I think it's safe to conclude that that is a serious blow to the study's implications on the strength of the relationship it tries to prove.
posted by esprit de l'escalier at 12:43 PM on December 16, 2013


Simply being peer reviewed doesn't mean that the conclusions of the study are now accepted by the world as fact.

Indeed not. And people who don't accept it should disprove it, rather than simply note its methodological shortcomings and drop the microphone. Especially since "you forgot to include X" or "your functional form is wrong" can be said of any social-scientific study ever.

It's certainly possible that including additional variables, or making other changes to the model, might reveal that it's limited or just outright wrong. We'd know whether that's the case or not by actually examining those things, not merely asserting what their possible effects might be.
posted by ROU_Xenophobe at 12:51 PM on December 16, 2013


That's great if you have time to constructively build counter-models. The point is that this model is clearly simplistic by the authors' own admission. I think you will find the idea that one has to constructively disprove every bad argument to be very time consuming the day you start reading papers. One learns to distinguish bad arguments from good ones.
posted by esprit de l'escalier at 12:56 PM on December 16, 2013


If it's been published and accepted, it has passed review. While that does not imply correctness, it does mean, in the conversation around this topic that their argument has some significance. They've shown their work and their model. By publishing in one of the best journals around, they've passed a significant sniff test.

If you want to refute or disagree at this stage, the (only) way to do it is to publish a counter argument. In simplest form this could be in Nature's letters section. These are typically only a page or two, not as high a bar as a full paper. But yes, you would have to have a substatial argument to pass the editors' review to get published.

In my view, that's how academic conversations should, must work; they need to be public and on the record, not just grousing in private.
posted by bonehead at 7:32 AM on December 17, 2013


Well, getting a counterargument accepted is the only way to disagree in a way that anyone will take seriously. The alternative being pointless kvetching because you don't like the conclusions.
posted by ROU_Xenophobe at 8:34 AM on December 17, 2013


It's not about whether you like or don't like someone's conclusions. It's unfortunate that the social scientists doing the review didn't have a statistics background to recognize that the claim was very poorly supported.

And it's just unfortunate that sociology maybe doesn't attract the best statisticians the way statistics, mathematics, finance, computer science and biology do.

It has nothing to do at all with disliking someone's conclusions. Did you read any of my comments?
posted by esprit de l'escalier at 12:35 PM on December 17, 2013


Yep. And basically ignored them as noise mixed with insults, because if you were really so het up about this terrible piece that doesn't pay nearly enough attention to the idea that women as individuals just make dumb choices and there's no structural or systematic issues, you'd demonstrate it. Not just kvetch.

I don't even disagree that there are ways to improve the study. But if you want to improve it, improve it. To reply in the insulting and patronizing ways you've replied in:

Most of us learn pretty early in grad school that just grousing about how awful this or that study is doesn't cut it, and that if you actually give a shit you have to do the work and, ideally, publish the hit piece.
posted by ROU_Xenophobe at 12:50 PM on December 17, 2013 [1 favorite]


I explained the methodological flaws that I think make the study a bad study. I thought we were having a conversation about the merits of this study, not writing papers here, so I don't understand this constant refrain of "do your own study". If you want to repeat your blind deference to the reviewers, then you don't really have a contribution to the discussion, do you? If you want to discuss what makes a good study, then that's what I'm doing and you're welcome to explain why you think their conclusion is supported.

If you thought there were any insults in my comments, I think you should look again. I thought it was clear that I was criticising the study — not the authors, who have started out an interesting project, and maybe intend to do a better job in a future paper.
posted by esprit de l'escalier at 1:01 PM on December 17, 2013


« Older Vidi mammam osculari Sanctum Nicolaum   |   "de-CAP"-itation Newer »


This thread has been archived and is closed to new comments