Github Gender Gap
February 12, 2016 9:59 PM   Subscribe

 
It'd be vastly more accurate to say that users on GitHub approved code written by women at a higher rate than code written by men. GitHub the corporate entity does not have any say over which pull requests get approved in users' repositories; it is up to the user who created the repository and/or those who that user has delegated authority to. (disclaimer: I am a GitHub employee)
posted by jordemort at 10:05 PM on February 12, 2016 [30 favorites]


GitHub the corporate entity has much more sexist shit to answer for than rate of acceptance of patches.
posted by Space Coyote at 10:07 PM on February 12, 2016 [10 favorites]


There's a post on this at The Slate Star codex blog, here.
posted by HoraceH at 10:08 PM on February 12, 2016 [4 favorites]


Why are we reporting on non-peer reviewed papers that try to draw conclusions from a single day's worth of data?
posted by I-baLL at 10:22 PM on February 12, 2016 [7 favorites]


That it's a single day's worth of data is irrelevant, only the size of the sample matters. And it looks large.

The real objection is that while it's true that women's code is approved at a higher rate when their profiles are gender neutral men's code is also approved at a higher rate when profiles are gender neutral. So yeah.
posted by Justinian at 10:35 PM on February 12, 2016 [9 favorites]


There's a post on this at The Slate Star codex blog, here.

Slate Star Codex on gender issues, based on a preliminary analysis by Hacker News? I don't even...
posted by effbot at 10:35 PM on February 12, 2016 [5 favorites]


"That it's a single day's worth of data is irrelevant, only the size of the sample matters. And it looks large."

How does it not matter? How can you tell what patterns hold up over time if you only look at a single day of data? If you want to find out if more women than men ride public transportation would you look at a single day of data and draw your conclusions from that?
posted by I-baLL at 10:38 PM on February 12, 2016 [1 favorite]


Why are we reporting on non-peer reviewed papers that try to draw conclusions from a single day's worth of data?

Because science journalism is broken because journalists are incentivized to get clicks rather than improve anyones understanding of the issue? Because researchers are incentivized to make their work as inflammatory as possible so they have a slightly better position in an abysmal job market?
posted by hermanubis at 10:41 PM on February 12, 2016 [13 favorites]


If you want to find out if more women than men ride public transportation would you look at a single day of data and draw your conclusions from that?

Yes, you absolutely would. And people do, all the time. If an event occurred that caused all women or men to stay home from work one day, it would be in the newspaper, otherwise commuting patterns from one day to the next will be statistically similar.
posted by Space Coyote at 10:41 PM on February 12, 2016 [6 favorites]


I'm steering clear of the public transportation comparison, there's a briar patch! Can we just consider how big a percentage one day is? You know, like per capita.

Anyway, for you readers of comments, I just want to second that Slate Star Codex article, linked above. I don't know much about them, but this one piece at least will tell you more than the comments here. So far.
posted by Rich Smorgasbord at 11:06 PM on February 12, 2016


This is such a weird fucking thing, since being a woman in tech means 1) not being taken seriously half the time and 2) being patronized as a special unicorn very much the other half. I can't take this study seriously at this point, but I also cannot stand the "should we not be suspicious of institutionalized social justice??" bs. In conclusion, tech is a land of contrasts, and I'm glad it's mostly men arguing with other men over this so I can take a nap.
posted by stoneandstar at 12:19 AM on February 13, 2016 [22 favorites]


Weird how defensive people are IMMEDIATELY over something like this, which all of other data gathering in this wonderful society of ours lends credence to.
posted by maxwelton at 12:22 AM on February 13, 2016 [7 favorites]


I don't know much about them

That doesn't necessarily mean that those of us that do know about them are wrong when we decide to not give a flying fuck about how they will spin this story.
posted by effbot at 12:24 AM on February 13, 2016 [2 favorites]


Sexism is fractal: it is present at all levels. In Github, at the aggregate level you see women's contributions accepted less as soon as they reveal their gender. At the personal level you see people kicked out of projects for suggesting variable names like iGiveHead aren't professional.
posted by sukeban at 1:09 AM on February 13, 2016 [27 favorites]


The real objection is that while it's true that women's code is approved at a higher rate when their profiles are gender neutral men's code is also approved at a higher rate when profiles are gender neutral. So yeah.

This is an interesting observation, but not actually an objection to the paper's results. From Figure 5, women with gender neutral profiles actually outperform men with gender neutral profiles, but the opposite is true for clearly gendered profiles.

It only seems to be true for "outsiders" rather than "insiders" though: "insiders" have high accept rates in general, and visibly female "insiders" appear to have a small benefit. And the fact that exposing gender seems to be so disadvantageous for both men and women does give me pause, like maybe there's another variable that visible gender is associated with? Finally IHNI where those error bars are coming from and there's no p-value so I'm not certain the difference is significant...
posted by en forme de poire at 1:19 AM on February 13, 2016 [7 favorites]


To be clear they have good p-values for the overall effect that women in general having higher accept rates (and make bigger edits!) than men... check out Figure 4. It's when they start breaking it down by visible gender in the profiles that things seem to get a little fuzzier. The bimodal distribution for women but not men in Figure 2 is also intriguing and I'd be curious to see these researchers explore that more.
posted by en forme de poire at 1:28 AM on February 13, 2016 [2 favorites]


effbot: Can you explain what is factually incorrect about the SSC blog posting?

Another interesting thing turned up in the authors' discussion: "Our analysis (not in this paper -- we've cut a lot out to keep it crisp) shows that women are harder on other women than they are on men. Men are harder on other men than they are on women."
posted by pw201 at 1:28 AM on February 13, 2016 [2 favorites]


People just love to hate on SSC round here.

Anyway, isn’t it weird that men have a reduced pull request acceptance rate when their gender is known too? the effect is weaker than for women, but not by much. I wonder if there’s some effect where the more you know about the individual proposing a pull request, the more you’re aware that they’re a potentially fallible human being rather than a code producing being from some higher plane.
posted by pharm at 1:32 AM on February 13, 2016 [3 favorites]


There’s also a strong insider / outsider difference going on: Quite a lot to unpick in this paper.
posted by pharm at 1:34 AM on February 13, 2016


NB - annoyingly the paper doesn’t give figures for the drop for men, but eyeballing the data suggests that it might be within a standard deviation of the figure for women. I really, really hope the reason the researchers didn’t give those numbers is because the error bars overlapped with the numbers for women, because that’s simply dishonest - it would undermine the headline conclusion if so.
posted by pharm at 1:37 AM on February 13, 2016 [4 favorites]


The Slate Star Codex piece (for anyone who's interested, and he does tend to go on a bit), is summed up with this paragraph:
A non-peer-reviewed paper shows that women get more requests accepted than men. In one subgroup, unblinding gender gives women a bigger advantage; in another subgroup, unblinding gender gives men a bigger advantage. When gender is unblinded, both men and women do worse; it’s unclear if there are statistically significant differences in this regard. Only one of the study’s subgroups showed lower acceptance for women than men, and the size of the difference was 63% vs. 64%, which may or may not be statistically significant. This may or may not be related to the fact, demonstrated in the study, that women propose bigger and less useful changes on average; no attempt was made to control for this. This tiny amount of discrimination against women seems to be mostly from other women, not from men.

The media uses this to conclude that “a vile male hive mind is running an assault mission against women in tech.”
So there's that.
posted by Grangousier at 2:28 AM on February 13, 2016 [1 favorite]


I would recommend reading the paper to anyone who wants to know what is actually being said, as the second-hand discussion of this appears rather noisy. I can't imagine why.

I would note that the Slate Star Codex's interpretation of the study's comments about the nature of women's contributions ("women’s changes are larger and less likely to serve project needs") is either a misinterpretation or just plain disingenuous, in terms of the 'serve project needs'. The text that I think is being referred to is for 'serve project needs' is:

"Contrary to the hypothesis, women are slightly less likely to submit a pull request that mentions an issue, suggesting that women’s pull requests are less likely to fulfill an immediate need. Note that this doesn’t imply women’s pull requests are less valuable, but instead that the need they fulfill appears less likely to be recognized and documented before the pull request was created."

Which is not the same thing at all.
posted by nfalkner at 2:56 AM on February 13, 2016 [25 favorites]


Contrary to the hypothesis, women are slightly less likely to submit a pull request that mentions an issue, suggesting that women’s pull requests are less likely to fulfill an immediate need.

So more likely to be "scratching a developer's personal itch.", basically? Also known as lesson #1 of good open source.
posted by effbot at 3:37 AM on February 13, 2016 [2 favorites]


And the fact that exposing gender seems to be so disadvantageous for both men and women does give me pause, like maybe there's another variable that visible gender is associated with

...maybe more experienced developers are less likely to disclose their genders.
posted by amtho at 4:01 AM on February 13, 2016 [2 favorites]


Extraordinary that this is being framed as anything but "women are better at coding than men"
posted by iotic at 5:04 AM on February 13, 2016 [11 favorites]


Extraordinary that this is being framed as anything but "women are better at coding than men"

That was the biggest take away for me. women were the first programmers, perhaps they're the best. It would be great to know more about why. I'd love to see some analysis of the accepted and rejected pull requests. Do women do a better job of tailoring their code to the contribution guidelines of the project? Do they tackle harder problems or problems whose resolution is more desirable by the project as a whole? Is the code simply of a higher quality, full of fewer obvious bugs and more algorithmically sound?
posted by dis_integration at 5:59 AM on February 13, 2016 [2 favorites]


I'm somewhat surprised that female developers don't stand out more as better developers because the ones that are still submitting pull requests on the githubs are the ones who have, and continue to, run the sexist gauntlet.
posted by srboisvert at 6:14 AM on February 13, 2016 [5 favorites]


If an event occurred that caused all women or men to stay home from work one day, it would be in the newspaper, otherwise commuting patterns from one day to the next will be statistically similar.

Do they tell us which day it was?
posted by StickyCarpet at 6:18 AM on February 13, 2016


The whole idea that your Github commits and pulls has anything to do with whether you are a good programmer is an increasingly frequent delusion among HR, who are so eager for something to use for screening that doesn't require them to actually understand anything. Lots of pull requests accepted doesn't mean you are a good programmer. The open source community accepts crappy code almost constantly. A lot of pull requests are also not even code at all – despite being a coder, most of my public pull requests are documentation.

Oh and then there is a whole barrel of my stuff no one can even see because it's part of private corporate accounts. So my public pull requests are really just a measure of the tiny fraction of time I code in my spare time.
posted by melissam at 6:20 AM on February 13, 2016 [21 favorites]


Extraordinary that this is being framed as anything but "women are better at coding than men"

Do you think there are some inate differences that make that so?
posted by huguini at 6:27 AM on February 13, 2016


Extraordinary that this is being framed as anything but "women are better at coding than men"

As another woman in IT, it's not extra-ordinary at all, it is in fact depressingly par for the course, but yes I do get your gist and am unsurprisingly-surprised at the same thing.

I am pretty sure that part of the reason their code is better (for whatever variables of "better" than means) is because women who succeed in IT have to run the gauntlet to even get in the door in the first place. I cannot even begin to describe the bullshit I've lived in the IT world. It beggars belief. And I say that as the person who has myself lived it. When people ask me for examples, depending on what kind of day I've had, I'm unable to say specifics, not because I don't have them but because it's so fucking depressing that my mind just blocks them and I would really rather think of my cats and getting work done rather than donating yet more mind space to men who laugh at me and roll their eyes while saying "omg how on earth can you possibly believe the sky is blue, who ever made you believe that" (because you can't think on your own, you know, it's always something someone else taught you, never mind that the words coming out of their mouths are also words that someone else taught them at some time in the past).
posted by fraula at 6:29 AM on February 13, 2016 [24 favorites]


dis_integration, they discuss that a bit in the PeerJ article itself; they note that women's contributions tended to be longer and were also less likely to be responding to an issue that had already been raised, which I thought was interesting.
posted by en forme de poire at 6:45 AM on February 13, 2016 [2 favorites]


"women are better at coding than men"

Women Dominate the Film Editing World
posted by StickyCarpet at 6:48 AM on February 13, 2016 [3 favorites]


Extraordinary that this is being framed as anything but "women are better at coding than men"

Do you think there are some inate differences that make that so?


It could be a reflection of what level of skill is needed for success which ends up selecting for better then average skills. In order for a woman to have what in comparison to male counterparts is an average amount of success they have to have to push for a higher level of skill and be better then average.

There's less room for mediocre to average to get anywhere at the same rates an average male counterpart could so they self-select out.
posted by Jalliah at 7:01 AM on February 13, 2016 [2 favorites]


Right. To be clear, "women are better at coding than men" is an entirely different proposition than "women who successfully enter the world of coding are better at coding then men who enter the world of coding." The latter seems reasonable and likely. The former seems like it's part of an "innate differences" trail that's not worth following. And of course whatever destination is reached would include trans women because trans women are women.
posted by xigxag at 7:14 AM on February 13, 2016 [6 favorites]


When a group of computer science students decided to study the way that gender bias plays out in software development communities

By default I trust computer science students doing social statistics projects at about the same level I trust sociology students with complex C code projects.
posted by BinaryApe at 7:23 AM on February 13, 2016 [10 favorites]


I also wonder if women only submit code that they are confident is really good, because men are conditioned to think that if they submit bad code, people will think that they submitted some bad code, while women are conditioned to think that if they submit bad code, people will conclude that they are trash coders and that's evidence that all women are trash coders. The stakes are higher for women, so they learn only to put stuff out there when they know it's good. And having internalized that expectation, they continue to do it even when their gender isn't evident to their audience.
posted by ArbitraryAndCapricious at 7:29 AM on February 13, 2016 [26 favorites]


Anyone has comments on the study methodology? Or on the fact that it wasn't peer reviewed? Or on the fact that it was designed by Computer Science researchers with no one from Psychology or Sociology (who are a lot more familiar with research methodology around these subjects) on the research team?
Or are we just very accepting today because the results fit our worldview?
posted by huguini at 9:05 AM on February 13, 2016 [6 favorites]


BinaryApe, you've hit the nail in the head. Any psychology graduate could easily pick apart the methodology and all the confounds it brings to the data collection process.
Just one example: they used G+ profiles to get data about the gender of the GitHub users, that assumes that men and women, with the same skill level, will make their gender visible (or inferable) at a similar rate on G+. Maybe, maybe not.
posted by huguini at 9:12 AM on February 13, 2016 [1 favorite]


What are the stats about what proportion of women coders are working in the kinds of fields where submitting to GitHub is even a thing? As a scientist, I spend 75% of my day writing code, and I work with more women than men who do similar. But not many of us, men or women, use GitHub frequently if at all. I wonder if all those amazing female coders/scientists in my field that I go to constantly for advice are completely ignored by this kind of analysis because they're not "real programmers".
posted by Jimbob at 9:25 AM on February 13, 2016


Maybe this is more about the coding quality of people who fill out their G+ profiles and connect them to their Github ;)

I have a obviously "girly" picture as my avatar but I don't think my account is connected to G+.
posted by melissam at 1:33 PM on February 13, 2016


Right. To be clear, "women are better at coding than men" is an entirely different proposition than "women who successfully enter the world of coding are better at coding then men who enter the world of coding."

That's the statistic though - women are outperforming men, by a certain measure, in an apparently representative sample of professionals. Whatever the reason we don't know yet - assuming validity, the study doesn't go that far.

Also yes, "extraordinary" was somewhat tongue in cheek. "Should be extraordinary" would be more accurate. To me that's the lede - we know about the sexism, or we should do. Clearly people do need reminding of that and seeing how it plays out here, but the result that women are doing the job better than men shouldn't be missed.
posted by iotic at 1:52 PM on February 13, 2016


I also wonder if women only submit code that they are confident is really good, because men are conditioned to think that if they submit bad code, people will think that they submitted some bad code, while women are conditioned to think that if they submit bad code, people will conclude that they are trash coders and that's evidence that all women are trash coders. The stakes are higher for women, so they learn only to put stuff out there when they know it's good. And having internalized that expectation, they continue to do it even when their gender isn't evident to their audience.

Yes, this effect is known as stereotype threat.

The type of effect described in the article has been commented upon before, as well.
posted by eviemath at 6:07 PM on February 13, 2016 [2 favorites]


Slate Star Codex: men get their requests approved 64% of the time, and women 63% of the time.

A one percentage point difference is interpreted in the mass media as evidence of massive systemic sexism in the open-source software world. More proof that journalists are effectively useless, and "technology journalists" even more so.
posted by theorique at 6:22 PM on February 13, 2016 [2 favorites]


Status 451 emphasizes that peer review is important and preprints are not a reliable source for news articles:
This is something I can shed at least a little light on, because coincidentally, two years ago I coauthored a paper, “More ties than we thought,” that went viral not once but twice — first the preprint on arXiv, then the peer-reviewed version in, also coincidentally, PeerJ.

Just to make things even more interesting, in the preprint version, our result was wrong.

[...]

The thing is, though, nobody makes serious decisions that affect other people based on what they know about tie knots. People do make serious decisions that affect other people based on what they know about gender bias — or, more importantly, what they think they know. How Vice, CNN, and other venues that are reporting on this preprint as if it’s already scientific consensus report will affect some of those decisions, and after looking at the data on my own experience of viral science reporting, “are they going to follow up when the paper is actually complete?” is a question readers deserve an answer to from reporters who are treating this paper like a genie that’s been let out of its bottle.
posted by Rangi at 5:29 PM on February 14, 2016 [2 favorites]


« Older HERCULES MULLIGAN! A TAILOR SPYING ON THE BRITISH...   |   Apprentice & Mentor: A First Deer Season Newer »


This thread has been archived and is closed to new comments