Reflections of a sellout; how diversity would strengthen social science

December 16, 2015 6:01 PM

José L. Duarte is one author of an upcoming paper in Behavioral and Brain Sciences, "Political Diversity Will Improve Social Psychological Science." The authors review how academic psychology has lost its former political diversity, and explore the negative consequences of this on the field's search for true and valid results. Duarte has blogged about his own experience of bias when he was denied admission to a Ph.D program, possibly for for his perceived political views in another blog post.

One typical example of academic political bias (he lists many others on his blog):

Weve got people measuring a purported fundamental personality trait of Openness to Experience by asking participants "I see myself as someone who..."

... is ingenious, a deep thinker.
... values artistic, esthetic experiences.
... is inventive.
... is sophisticated in art, music, and literature.
... likes to reflect, play with ideas.

You've got to be kidding. These items are obviously grounded in – and biased in favor of – academia. This core personality trait of "openness" is measuring intellectualism and urban sophistication. These items are invalid on their face, and should not have lasted this long.

How are people in rural communities going to show up on this scale? How about people in developing countries? How would they express their openness to experience? Where do we give them a voice?

[...]

Note also that rural communities are likely to be conservative, and urban sophisticates are likely to be liberal or libertarian, so we've rigged a systematic political bias against conservatives showing up on our "openness" measure. The fact that conservatives score lower on openness is widely reported and savored by politically biased and incurious science writers and sloppy scientists.

Duarte notes his dissatisfaction with their paper's focus on political diversity:

We need more diverse voices in social science. We need them for functional scientific reasons. Ideally, politics should have no place, and if there's anything I would change about the paper, it's that we focused on politics as the only level of analysis. We spoke of political diversity. I would have preferred to focus on intellectual diversity – to also get at deeper levels of analysis where there the field is homogeneous in some of its assumptions about human nature, the inferences that can be drawn from certain inferential statistics, and the particular brand of empiricism we operate with.

MetaFilter has previously discussed the lack of replication of some psychological studies. Duarte notes in an article for Medium that political bias is part of this problem: a study linking "free market systems" and "social inequality" was originally done nine times, and only the five significant results were reported.

posted by Rangi (24 comments total)

Interesting piece. I think Duarte touches on several things, some of which a more strongly connected than others.

1) Poor methodology in social science and psychology in particular. (I know this is a thing; I don't know how bad the "regular" sciences are, I think sometimes that gets elided).

2) Issues with replication and the larger academic impetus to publish novel results (definitely a thing!)

3) Bias/prejudice in social science (I think we're on shaker ground, here. Some of his examples are better than others)

4) Lack of diversity and ethnocentrism in social science (I think Duarte is obviously right here, but I wonder if he narrows his scope too much, is it just social sciences? His anecdote is powerful, but doesn't necessarily mean there's a systemic problem,nor that what he thinks they thought is necessarily true).

I think there are definitely some things to be said about diversity in higher education. I'd love to see some posts/articles from academics other than Duarte talking about it, so I could get a better sense of response/awareness etc about this issue.
posted by smoke at 6:19 PM on December 16, 2015 [4 favorites]

Poor methodology in social science and psychology in particular.

While (especially social) psychology has a problem where the results don't replicate, sociology has a problem where replication can't even be attempted.
posted by save alive nothing that breatheth at 6:36 PM on December 16, 2015

I'd love to see some posts/articles from academics other than Duarte talking about it, so I could get a better sense of response/awareness etc about this issue.

The paper's other authors are Jarret Crawford, Charlotta Stern, Jonathan Haidt, Lee Jussim, and Philip Tetlock. They, along with other professors, researchers, and graduate students, are all concerned enough about the issue of intellectual uniformity in science that they've formed a collective site, Heterodox Academy. Their publications page focuses on social psychology, but is not limited to that area.

Jonathan Haidt has collected links to other papers and commentary on their efforts, "some supportive, some critical." Notably, Paul Krugman and Jonathan Chait disagree that a problematic bias exists.
posted by Rangi at 6:39 PM on December 16, 2015 [3 favorites]

Their theoretical premises are flawed, I think. They seem to posit a taxonomic model of political opinion in which the two main branches are "liberal" and "conservative," based (at least in part) upon how people self-identify. At the same time, they purport to account for the complexity of variation by noting that the "conservative" branch may include other ostensibly whole groups like libertarian and moderate (moderate conservative, I guess?) and that "...self-identified conservatives may be more diverse in their political beliefs than are liberals."

In other words, they assume too much that is unproven (and even unstated) about the sufficiency of the language they use to describe people accurately and validly. I can't see any reason to assume the correctness of their basic theoretical foundation; I simply don't think that liberal and conservative are meaningful if they're supposed to characterize a comprehensive and regular political tendency, except insofar as they give insight into how people see themselves, which they've already argued is not a reliable indicator of their actual values anyway. People called "liberal" these days want to conserve an economic standard of living that goes back to the 50's, while people called "conservative" want to liberalize regulations on big business and finance. So why use them at all? It seems foolish to base so much on such slippery and imprecise labels.

Finally, their actual clams are, with due respect, bullshit. See here:

The lack of diversity causes problems for the scientific process primarily in areas related to the political concerns of the left—areas such as race, gender, stereotyping, environmentalism, power, and inequality—as well as in areas where conservatives themselves are studied, such as in moral and political psychology

It is bottomlessly disingenuous to say that "the left" is concerned about power, race, and gender while the right is apparently not. That's like saying that the woman whose house is on fire is really fixated on the subject of fire while the arsonist is disinterested in the matter. Jesus fucking Christ. Right-wing types are obsessively interested in all of those things, they just don't want to see them openly discussed or problematized, especially not by people who are members of historically-marginalized groups.

Maybe there's something useful in here, but it stinks to high heaven of baseless special pleading.
posted by clockzero at 7:02 PM on December 16, 2015 [34 favorites]

They aren't saying that conservatives aren't interested in issues like power, race, and gender. That would be absurd. They are saying that conservatives have a fundamentally different perspective on these issues, and that difference is useful in creating a diverse perspective regarding e.g. what it might mean for people to psychologically healthy, for family systems to work well, or for societies to ethically and effectively support those who live in them. In my experience this is a reasonable presupposition.
posted by zipadee at 7:12 PM on December 16, 2015 [2 favorites]

I kinda don't buy that this is how the Big Five works at all. Openness to experience isn't some kind of objective good, just a bunch of traits that cluster together. I suppose the fact that conservatives tend to score high on conscientiousness indicates that liberal psychologists secretly despise being conscientious, or, uh, something.
posted by thetortoise at 7:34 PM on December 16, 2015 [4 favorites]

That this is an argument based in a very contemporary and specifically American political framework reminds me of a previous post:

And that's a bad idea. Much of standard group behavior data in Sociology/Economics/Psychology is based on Americans. Which don't seem (contrary to universal assumptions) to be shared by a lot of the World.

So I'd say that there's something to this, but that the way they're framing political and intellectual diversity doesn't go anywhere near far enough.
posted by spindle at 7:35 PM on December 16, 2015 [5 favorites]

That used to be called the WEIRD demographic in psychology.

Also recently making the media rounds: Feeling like you're an expert can make you closed-minded.

The point those authors make is that "openness" is a totally double-edged sword.
posted by polymodus at 7:40 PM on December 16, 2015

They aren't saying that conservatives aren't interested in issues like power, race, and gender. That would be absurd. They are saying that conservatives have a fundamentally different perspective on these issues

First they strongly imply a more-or-less binary political spectrum, then they list some basic social categories and identify them only as "areas of interest to the left." Unless they address the issue again and say something else, they are very clearly suggesting that right-wing folks don't see a whole lot of problems in the areas of race, gender, or power, and they're correct -- right-wingers often say that if only leftist types would stop talking about these things, there would literally be no problem; or else, they deny that any such problems are really social problems at all.
posted by clockzero at 8:25 PM on December 16, 2015 [3 favorites]

Wow. That's some badly-done factor analysis being refuted by some other badly done factor analysis, in the hope that it encourages more people from one end of one of those imaginary factors to do more factor analysis.
posted by cromagnon at 8:55 PM on December 16, 2015 [2 favorites]

This is really too rich.

Historically, the APA has spent quite a bit of effort to protect "conservative" view points. That those view points have become increasingly untenable in both a social and scientific sense might encourage introspection.

The authors of the article almost have a point when they argue that there are useful outcomes driven by political diversity, but fail to offer anything interesting to say about political diversity. Their attempt illustrates the obvious; a better analysis than the terms Liberal and Conservative is needed to perform good research.

For example, the first author might reject torture as being against conservative (Economic | Secular | Libertarian) values, but in a completely different sense than as Paul Ryan claims that Trumpism is not conservatism (P90X).

Really, the clearest statement of the article's thesis is buried in a footnote of the blog post [some white space added):

It's also worth noting that one's intellectual or philosophical framework need not give rise to a political identity. I'm not comfortable with politics being used as the primary sorting variable in the intellectual sphere.

People don't have to care about politics, and separately, they don't need to have an easily labeled political identity that neatly fits the political terrain of our era. Many scholars and scientists do not fit into such schemas.

(Politics as a sphere is central to the modern left in part because their ideology includes explicit and specific claims about the force of politics and power structures – life is largely about politics in that frame. Most of the forces in people's lives are attributed to politics, privilege, discrimination, and so forth, and there isn't a whole lot going on in human affairs that isn't political, at least not that they talk about.)

posted by ethansr at 9:14 PM on December 16, 2015 [3 favorites]

I might be wrong, but, just thinking (very casually) about what-all I've read over the course of my life, it seems to me there sort of is a roughly binary split between some of the foundational assumptions of social science (causation, materialism, and at least soft determinism; descriptivist aims) and "things you'd hear from somewhere on the right" (emphasis on prescriptive morality and free will, or something like it. I don't actually know how a social scientist can believe the latter and do their work. And it is, for that reason, just weird to me to imagine a libertarian psychologist, e.g., Haidt, who's a co-author on that paper).
posted by cotton dress sock at 9:21 PM on December 16, 2015 [1 favorite]

Essay: ... A final preparatory comment we must make is that the lack of
political diversity is not a threat to the validity of specific
studies in many and perhaps most areas of research in social
psychology. The lack of diversity causes problems for the
scientific process primarily in areas related to the political
concerns of the left — areas such as race, gender, stereotyping,
environmentalism, power, and inequality — as well as in areas
where conservatives themselves are studied, such as in moral and
political psychology.


First they strongly imply a more-or-less binary political spectrum, then they list some basic social categories and identify them only as "areas of interest to the left." Unless they address the issue again and say something else, they are very clearly suggesting that right-wing folks don't see a whole lot of problems in the areas of race, gender, or power, and they're correct -- right-wingers often say that if only leftist types would stop talking about these things, there would literally be no problem; or else, they deny that any such problems are really social problems at all.

They've made an intensely self-contradictory statement there: 'Conservatives are not politically concerned in these areas. (Elsewhere:) It is a problem for science that conservatives are not entering academia to work on these areas.'

I think they are suggesting there should be environmental researchers who are interested in researching the environment but who would fight or dismiss efforts to protect the environment.

This is kind of bullshit.
posted by sebastienbailard at 9:21 PM on December 16, 2015 [8 favorites]

There are such environmental researchers, who are interested in extracting as much as possible from the environment.
posted by clew at 10:39 PM on December 16, 2015 [2 favorites]

Weve got people measuring a purported fundamental personality trait of Openness to Experience by asking participants "I see myself as someone who..."

... is ingenious, a deep thinker.
... values artistic, esthetic experiences.
... is inventive.
... is sophisticated in art, music, and literature.
... likes to reflect, play with ideas.

Hm... the very page he links to has 10 measures of openness. I wonder which questions he happens to have left out...?

... Is original, comes up with new ideas
... Is curious about many different things
... Has an active imagination
... Has few artistic interests
... Prefers work that is routine

I don't know what sort of caricatures of "rural communities" and "developing countries" he has in mind, but I suspect that even in these conservative redoubts, there may be people who are not "embarrassed" to say they are curious, inventive, original, like ideas, and dislike routine work. The nice thing about aggregate measures is that, even if not every measure is "speaking their language" (to use his own insulting terminology), enough of them may be comprehensible to these rubes that the measure can distinguish the playful from the less playful.

Anyway, this blog post, the paper it links to, and Haidt's work more generally, are largely exercises in cherry-picking, much like the "fair and balanced" strategy Fox pursued years ago (before it no longer needed to). And like Fox, their main argument often seems to "since there are few conservatives in field X, therefore there should be more conservatives in field X" -- because, "diversity." This Duarte guy was not hired because of his politics, therefore all psychologists are doctrinaire liberals; a few questions in a couple papers by Jost (Haidt's foil at NYU) look reductive when extracted from the context of their papers, therefore all of political psychology presumes what it seeks to prove; some personality questions include political attitudes, therefore all psychology measures are politicized; one claim he retracted (which presumably implies no generalization); and these questions from the Openness measure, of which he silently dropped 50% before generalizing.

Standards are pretty low these days, so one wouldn't say they are truly like Fox -- their claims are merely wrong, rather than being self-contradictory nonsense. Haidt and his compatriots have been arguing for years that conservatives are misunderstood by political psychologists, and to a small degree he probably has a point -- I rather like Duarte's questions about boats, lakes, and stars, caricatures though they may be, and Crawford and Haidt's work have definitely captured overlooked symmetries or asymmetries between the two sides. But their claims about the magnitude of the problem are poorly served by these meagre examples, and especially by the misappropriation of terms like "diversity," "bias," "minorities," "in the closet," etc, to political positions. Sure -- if you presume that political views are purely identity categories with no right, no wrong, and no connection to facts or to the scientific process, then "bias" against one of these identities may be unfair. But without that presumption, the whole thing crumbles.

I sympathize with them -- over the last few decades, a large branch political psychology has moved from merely enumerating the differences between conservatives and liberals, to demonstrating through numerous experiments that conservatism in particular is linked (in both causal directions) to a variety of cognitive traits that are almost universally considered normatively undesirable (see p 30 of their paper, eg). But cherry-picking these examples and backing them up with caricatures about rural life in boats staring up at the stars, or a few minor studies where conservative thinking was a bit more consistent than the researcher imagined, do not constitute the sort of systemic evidence these guys seem to think. It's tough -- mainstream American conservatism these days is clearly dissociated from facts, reasons, tolerance, diversity, and many other things the authors themselves value. (Though by their own lights, their own paper is pretty biased, since in it they tout the fact that none of them actually identify as conservative.) And clearly there's a lot of failure on the left to understand just what the other side is thinking. But hijacking the language of the left to amplify this problem into a major call for thousands of more conservative academics is really counterproductive to their basic research goals. It just makes the research look like a stalking-horse for Fox-style "fair-and-balanced" bullshit.
posted by chortly at 11:26 PM on December 16, 2015 [7 favorites]

There are such environmental researchers, who are interested in extracting as much as possible from the environment.

Sure, fisheries researchers, Ducks Unlimited, and so on. Not so much "Dudez, I love++ deforestation = teh greatest! Only thing better is 💚phosphate detergent + high sulphur coal smoothies with PCB and mercury-rich blubber side-dish💚."

Cato Institute scholars are pretty thin on the ground for some reason, unless I'm experiencing a sampling bias.
posted by sebastienbailard at 11:36 PM on December 16, 2015

Duarte raises some good points, 5 in total. Funny that he already had to retract one of those points because his description of a paper was completely wrong.

Also, anyone who believes that they were denied by a Ph.D. Program because of something they said about jimmy carter is just a complete moron it's so mind bogglingly stupid and I don't even know how he would know that.

But he does have a point. Academia would be well served if they had more diverse viewpoints (and not just diverse backgrounds). I study with a lot of conservative military folk and their perspective if nice to have around.
posted by MisantropicPainforest at 4:16 AM on December 17, 2015 [1 favorite]

There are such environmental researchers, who are interested in extracting as much as possible from the environment

I assumed shared beliefs with an aquaintance who said he planned to study environmental law. Turns out his plan was to work in corporate defense. There are economists who study why welfare is bad, I don't see why there shouldn't be psychologists who study why authoritarian thinking is good. I would disagree with both, but that doesn't make the work illegitimate.
posted by Octaviuz at 5:05 AM on December 17, 2015 [1 favorite]

It's also worth noting that one's intellectual or philosophical framework need not give rise to a political identity.

He's right, I guess, but it often does.

It's been a while since I've read anything by Haidt (and I'm not up for a TED video atm), but based on this random scraping (in which admiration for Hayek is expressed, or at least for his "epistemological modesty"), I gather that the rough gist of his program is a kind of nudgey virtue ethics, like:

muddled, irrational humans --> free markets ("complexity") --> smarter, better humans

Which may or may not work out, but isn't not a conservative idea.

(Also, I think moaning about a lot of research on "reverse discrimination" not being published might have to do with some people missing a major bit of cultural context surrounding what's ordinarily understood by "discrimination".)
posted by cotton dress sock at 5:55 AM on December 17, 2015 [2 favorites]

Haidt's tweet about "It's time to start darwinizing economics. Evolution, not equilibrium" is complete nonsense from a biological perspective, and probably from an economic one as well. Duarte's statements above seem to be inviting some kind of Lysenkoism, and Haidt suggests the motivation. Reality has a Liberal bias.
posted by sneebler at 6:18 AM on December 17, 2015

You've got to be kidding. These items are obviously grounded in – and biased in favor of – academia. This core personality trait of "openness" is measuring intellectualism and urban sophistication. These items are invalid on their face, and should not have lasted this long.

The following study may be of interest:

The five-factor model (FFM) of personality variation has been replicated across a range of human societies, suggesting the FFM is a human universal. However, most studies of the FFM have been restricted to literate, urban populations, which are uncharacteristic of the majority of human evolutionary history. We present the first test of the FFM in a largely illiterate, indigenous society. Tsimane forager– horticulturalist men and women of Bolivia (n
632) completed a translation of the 44-item Big Five Inventory (Benet-Martínez & John, 1998), a widely used metric of the FFM. We failed to find robust support for the FFM, based on tests of (a) internal consistency of items expected to segregate into the Big Five factors, (b) response stability of the Big Five, (c) external validity of the Big Five with respect to observed behavior, (d) factor structure according to exploratory and confirmatory factor analysis, and (e) similarity with a U.S. target structure based on Procrustes rotation analysis. Replication of the FFM was not improved in a separate sample of Tsimane adults (n
430), who evaluated their spouses on the Big Five Inventory. Removal of reverse-scored items that may have elicited response biases produced factors suggestive of Extraversion, Agreeableness, and Conscientiousness, but fit to the FFM remained poor. Response styles may covary with exposure to education, but we found no better fit to the FFM among Tsimane who speak Spanish or have attended school. We argue that Tsimane personality variation displays 2 principal factors that may reflect socioecological characteristics common to small-scale societies. We offer evolutionary perspectives on why the structure of personality variation may not be invariant across human societies.
posted by belarius at 6:57 AM on December 17, 2015 [1 favorite]

Bleh. I grew up in Farmville, and from my personal perspective, people in rural communities are usually more narrow minded than people in urban communities. (Rural cultures are more tribal in nature; People tend to be more honest, generous, and supportive, but also more conformist, resistant to innovation, and distrusting of outsiders).

Duarte sounds like someone who goes into job interviews saying "IF-You-Hire-ME-Then-I'M-Going-To-BE-Annoying-And-Make-Problems!"

Haidt started off with good intentions about trying to find solutions to problems with the cultural logjam, but he ended up sucking up to bigots.
posted by ovvl at 9:05 AM on December 17, 2015

So, wow.

Tl;dr: university campuses have become hotbeds of a kind of victimhood Olympics that are a) unjustifiable, in the context of the major inequalities already having been dealt with, as far as Haidt's concerned, and b) actively damaging to people's sense of competence (or something). People who complain about microaggressions (at least on university campuses) should take it on the chin and deal with it themselves.

they are very clearly suggesting that right-wing folks don't see a whole lot of problems in the areas of race, gender, or power, and they're correct -- right-wingers often say that if only leftist types would stop talking about these things, there would literally be no problem; or else, they deny that any such problems are really social problems at all.

Seems so.
posted by cotton dress sock at 5:25 PM on December 17, 2015

The Coddling of the American Mind is totally the title I would have these guys author if I were writing a satire, how can they not have noticed this
posted by thetortoise at 7:13 PM on December 17, 2015 [1 favorite]