Baseball and Gender Bias: “Number of Women in Physics Departments: A Simulation Analysis “

I’ve spent a bunch of time recently blogging about baseball statistics, which you might be inclined to write off as some quirk of a sports-obsessed scientist. I was very amused, therefore, to see Inside Higher Ed and ZapperZ writing about a new AIP report on women in physics (PDF) that uses essentially the same sort of rudimentary statistical analysis to address an important question.

I say “amused” because of the coincidence in methods, not because of the content. And, in fact, the content is… not likely to make them friends in a certain quarter of the blogosphere. I actually flinched when I read the sentence “This is further evidence that there is no systematic bias against hiring women.” That phrasing is incredibly unfortunate in that it is likely to be interpreted in ways that will really upset some people.

It is, however, a true statement given their analysis and the relatively narrow question they’re trying to answer. And it’s an important enough point that it’s worth writing up in a little more detail than IHE or ZapperZ did, to make clear what they are and are not saying. I’m not going to do this is funny Q&A format, though– writing about this at all is somewhat fraught, and trying to crack wise while writing about this study is basically guaranteed to blow up in my face, so I will keep this as matter-of-fact as I can.

The new study is a statistical analysis of the distribution of women in physics, attempting to address the question of all-male departments. The statistics are pretty striking, and summarized in this table that I cribbed from Inside Higher Ed (hopefully the formatting won’t go all wacky due to differences in CSS):

Physics Departments, Faculty Size and Gender

Those data seem pretty damning at the bachelor’s-only level: nearly half of all physics departments have no women at all. Surely, this is evidence of bias, right? Those all-male departments must be a result of old-boy networks of sexists who won’t hire women.

What the study shows, however, is that this can’t actually be taken as evidence of bias, because many of these departments are very small– the median size of a department at a bachelor’s-only institution is four professors (meaning, for the record, that Union, where I work, is way above average– we have eight tenure lines and two (soon to be three) permanent but non-tenured lecturer positions). Given that, there’s actually a pretty decent chance of ending up with an all-male department just from basic statistics.

They demonstrate this by basically the same method I used in the first of the baseball posts linked above: they set up a simulation where they take an imaginary population of faculty with the same gender ratio as in the real sample, and assign them to departments of the same sizes found in the real sample completely at random. This is a little more complicated than the simple toy model I used for the baseball thing, because the probability of getting each gender changes as they assign faculty to departments. Then they look at what fraction of the imaginary departments ended up being all male. They repeated this 500 times, and got the graph showing how many of their 500 simulations gave a particular percentage of all-male departments that’s the featured image up top, which I’ll repeat here:

What you see is something that looks pretty much like a classic “bell curve” distribution, showing that there’s an average fraction of departments with no women, and some uncertainty about that average. The yellow bar indicates what they see in the actual sample, which you can see is slightly below the peak of the distribution (the most likely value is 49%, they see 47%), though within one standard deviation of it. The same graph for Ph.D.-granting institutions looks like this:

Again, you see an average with some uncertainty, but the yellow bar is a little harder to see, because it’s way off in the left wing. The observed number of all-male departments is much smaller than you would expect from a random distribution– the most likely value is 12%, and the real sample is 8%. In fact, 90% of their simulations gave higher all-male fractions (so the real value is a bit less than two standard deviations below the mean). The overall numbers are all much lower, reflecting the larger average size of departments at Ph.D. granting institutions– given a larger number of faculty, the odds of a random draw ending up all-male are much lower.

This suggests that if there’s any systematic preference happening in hiring, it goes in the opposite direction of the most basic sort of sexism– women are, in fact, somewhat more broadly distributed among departments than you would expect from simple chance. The fact that a sizable fraction of physics departments do not have any women is not by itself an indicator of bias against women, given the fraction of the faculty pool who are women. If you wanted to insist on putting a negative spin on this, you could try to argue that it’s indicative of some sort of tokenism– a willingness to hire one woman so the department isn’t entirely male, but not more than one. But that doesn’t seem all that likely, and if anything goes a bit against the conventional wisdom about women on hiring committees and so on.

Now, there’s a big and important caveat to this, which is the emphasized clause in the previous paragraph. They have assumed a particular gender distribution among the imaginary faculty pool in their simulation, which matches the gender distribution of the real sample– 16% of the bachelor’s-only faculty are female and 11% of the Ph.D.-granting pool. That’s descriptive, not prescriptive– they would undoubtedly prefer a more equal split (and in fact re-run their simulations for higher fractions of women), but they’re looking at what’s actually out there for the purposes of this analysis. And it’s important to remember that those low percentages are over all ranks of faculty– from newly hired assistant professors to the moldiest of why-won’t-he-retire-dammit full professors– and thus include the effects of decades of past hiring decisions.

(They do, for what it’s worth, give one quick indication of the present state, in their Figure 5, which shows that the percentage of women increases as you move to younger cohorts. The Assistant Professor (that is, pre-tenure) ranks are over 20% female, a percentage that’s slightly higher than the fraction of women receiving Ph.D.’s in recent years. This is where you’ll find the sentence quoted above that made me flinch– “This is further evidence that there is no systematic bias against hiring women.” Which is, as I said, a somewhat unfortunate phrasing, but not an inaccurate summation of their data: women are hired into tenure-track faculty positions in the same proportion that they graduate with Ph.D.’s, so looking at the system as a whole from the 30,000-foot-altitude kind of level, there’s no clear indication of bias– if anything, there’s a very slight preference for hiring women. Though they note that even with a higher fraction of women in the pool, you would still expect some number of single-sex departments– they ran simulations with numbers matching the assistant professor distribution (where, again, the proportion of women is slightly higher than among recent Ph.D. graduates), and those would give 37% single-sex departments at the bachelor’s level and 3% at the Ph.D. level. Even at a 50-50 split among faculty, you’d end up with around 10% of bachelor’s-only departments having no women (though in that case you’d also get 10% with no men…).)

What are the limitations of this? Well, it’s a very global, 30,000-foot-altitude kind of study. All they can really say is that, on the basis of statistics, it is unlikely that there is a global, systematic bias against women that leads to the large number of all-male departments. The fact that a department has no women, particularly at a smaller school, does not necessarily indicate any bias in hiring beyond whatever may be indicated by the gender distribution of the available faculty pool.

This does not mean that there is absolutely no sexism anywhere, and that’s not what they claim. All they can and do say is that the distribution we see in reality is no worse than you would expect from a purely random distribution. This does not rule out the possibility of bias in any individual department, or even some large number of biased departments, provided they are balanced by some number of unbiased or oppositely biased departments.

This also doesn’t say anything about what kinds of jobs people have, or what they’re paid, or any of a host of other kinds of potential bias. You could undoubtedly construct a pathological sort of system whereby the global appearance of no bias was produced by systematically excluding women from a small number of highly prestigious positions while distributing them more evenly among a larger number of low-status jobs. This kind of analysis will not allow you to detect that sort of problem (though there are other ways to get at those kinds of questions, and I have no doubt that AIP and other organizations are doing those tests).

This is basically analogous to the situation in the second of those baseball posts linked at the beginning. The batting average data I was playing around with are broadly consistent with what you would expect for a single, constant “innate” average– that doesn’t mean that you can’t have an innate average that changes with time, just that you can’t point to anything in the statistics that unambiguously shows those sorts of changes. Similarly, these data are broadly consistent with an unbiased (in a statistical sense) random distribution of women among faculty jobs, and that doesn’t mean bias doesn’t exist, just that there’s nothing in the statistics that unambiguously indicates the presence of bias.

(Now, there are other arguments you could raise about this, like whether we ought to expect or want the hiring of candidates for faculty positions to resemble a random distribution. You could also argue that there ought to be a much stronger preference given to women in order to equalize the overall distribution, but that’s a flamewar of a different color. And, obviously, a much more equal distribution in the candidate pool (more that 20% women) would be a wonderful thing, though that’s an issue with an earlier part of the pipeline than they’re considering here. Again, this study is descriptive, not prescriptive.)