Do Democrats and Republicans Grade Differently?

June 10, 2011 8:18 AM

Democratic and Republican professors grade differently according to a recent study. Here is a link to a PDF of the published study, minus the critical data tables and figures.
posted by Seymour Zamboni (43 comments total)

Woops.....the PDF does have the tables and figures at the very end of the paper, just not embedded. Sorry about that.
posted by Seymour Zamboni at 8:24 AM on June 10, 2011

Using SAT scores as a proxy for the preparedness of students, the researchers were able to rule out patterns in which Republican or Democratic professors had better students.

No, they weren't. SAT scores do no such thing.
posted by valkyryn at 8:29 AM on June 10, 2011 [10 favorites]

What about Catholics and Protestants? Hindus and Muslims? Red Sox fans and Yankees fans? Mac users and PC users? People who like pickles and people who don't? There is literally no end to the number of ways data like this could be collected and interpreted to suit the needs of the observer.
posted by The Card Cheat at 8:35 AM on June 10, 2011 [2 favorites]

Someone had to fund this research. Think about that.
posted by fourcheesemac at 8:37 AM on June 10, 2011 [1 favorite]

Did they control for differences in what kind of classes the professors taught? It's not like Republican and Democratic professors are evenly distributed throughout the disciplines. Maybe professors in Business and Econ are just more likely to give high and low grades, no matter what their political inclinations.
posted by craichead at 8:39 AM on June 10, 2011 [7 favorites]

Did they control for differences in what kind of classes the professors taught?

That was my thought too. All of the overtly conservative professors that I had taught either economics or computer science.
posted by octothorpe at 8:48 AM on June 10, 2011

They do roughly break it down into Humanities, Social Sciences, and Natural Sciences but there is a problem that most of the "gateway courses (courses which students have to pass to get into various majors -- Chemistry 1XX for Nursing (and most Science Majors), for example) fall into the Natural Sciences, and that is going to skew things -- you will have a deformation around the "breakoff point" for progressing in your major as students retake courses until they meet that grade. Depending on the school, there may be no record of repeated attempts in the regular transcripts.

On another tangent, I really hate the idea that "Professors give grades." I teach a heavily skills-based course, and students earn a grade, I don't give it to them (yeah, I know my personal outlook has an impact on how I evaluate; I am human, but students who do poorly usually do so for failure to show mastery of the material -- not turning in assignments is a pretty good metric for failure).
posted by GenjiandProust at 8:52 AM on June 10, 2011

It's not like Republican and Democratic professors are evenly distributed throughout the disciplines.

They aren't. Republicans made up 3.4% of the professors in the humanities and 10.8% of the professors in the natural sciences. And they only constituted 4.9% of the professors in the entire sample.

I guess you can get away with calling this "science," but you'd have to be an economist.
posted by valkyryn at 8:53 AM on June 10, 2011 [13 favorites]

Silly me. I thought all professors were socialists and commies.
posted by Benny Andajetz at 8:57 AM on June 10, 2011

Benny Andajetz: "Silly me. I thought all professors were socialists and commies."

Clearly, you've never taken an economics course.
posted by schmod at 8:57 AM on June 10, 2011 [1 favorite]

Tongue-in-cheek. I actually have a marketing degree and took a lot of econ courses. And my professors were, by and large, very conservative. (I always claimed I was learning by looking at the other team's playbook.)
posted by Benny Andajetz at 9:05 AM on June 10, 2011

Benny Andajetz: "Silly me. I thought all professors were socialists and commies."

Clearly, you've never taken an economics course.
posted by schmod at 10:57 AM on June 10


My state representative - the former econ professor
posted by jillithd at 9:09 AM on June 10, 2011

right, a republican giving an A.
posted by clavdivs at 9:14 AM on June 10, 2011

Okay so we've established that we are dealing with two different clades of humanity, distinguished primarily by cognitive differences, vis perception of and means for dealing with society and the world but also by some minor variations to brain structure, that is really clear. I find this really fascinating and wonder to what extent this is driven by genetics and what is driven by environmental factors?

Certainly there are evolutionary advantages to both patterns.

In good times, a rational approach to exploring one's world will be the best result as marginal niches can be exploited very quickly without genetic shift; a authoritarian approach will not work as well because there is too much inertia.

In bad times, such as war or famine, the authoritarian approach will provide the best result because the gene line itself is at risk so society members do-what-they-are-told-and-don't-ask-questions because asking questions wastes time when the elders have already made the decisions and simply need grunts to carry them out. The rational approach won't work as well because, well, they're already dead because they were doing research on resource development when overrun by an army who will just fucking take stuff.

Okay maybe this is a generalization but come on the anecdotes and research studies just totally stack up this way.

I really think there is a programmed-in push-pull of these cladistic traits in our genetic make-up and it has brought us really far on this planet but OH MAN I get tired of this tug of war thing we have to do just to be survivors!
posted by seanmpuckett at 9:15 AM on June 10, 2011 [1 favorite]

Where I went to school, we always joked that if the history department and political science department had softball teams, they could re-enact the Eastern Front.
posted by FelliniBlank at 9:16 AM on June 10, 2011 [2 favorites]

Someone had to fund this research. Think about that.

There is no acknowledgment of external funding source in the paper. One of the authors is a tenure-track faculty member at Cornell, who is supported by teaching and university service; the other is a lecturer at Hebrew University, who lists a few small grants on their CV. So unless they are being duplicitous, there doesn't seem to be a nefarious hand at work here.

Using SAT scores as a proxy for the preparedness of students, the researchers were able to rule out patterns in which Republican or Democratic professors had better students.

No, they weren't. SAT scores do no such thing.

The InsideHigherEd gloss does a disservice to the authors here. The authors cite several studies showing correlation between SAT scores and college academic performance, and acknowledge that SAT scores may only explain part of the variation in scores. So they are well aware that SAT scores are an imprecise proxy for academic ability.

Did they control for differences in what kind of classes the professors taught?

In fact, they do try to control for these differences by including department and discipline in their model. Again, its an imprecise way of controlling for this, but they do give it a shot.
posted by googly at 9:19 AM on June 10, 2011 [2 favorites]

schmod: "Clearly, you've never taken an economics course."

Macro-econ professor who assigned us Free to Choose and spent a lot of time telling us how the newly elected president Clinton would ruin the US economy.
posted by octothorpe at 9:20 AM on June 10, 2011

The Card Cheat: What about Catholics and Protestants? ... People who like pickles and people who don't?

I'm relieved to now know why I got that B in Numerical Analysis. It had nothing to do with my poorly constructed matrix conditioning proof on the midterm. The professor just didn't like pickles! :-)
posted by gruchall at 9:25 AM on June 10, 2011

Table 1 (from PDF) shows they had roughly 27 republican professors and 520+ democrat/others professors. They're drawing a conclusion based on this? Why not draw a conclusion on the professors have breakfast vs. those that didn't?
posted by kproto at 9:28 AM on June 10, 2011 [2 favorites]

If all the professors taught the same class,
And all the professors used the same syllabus,
And all of the students had taken the SAT only once,
And SAT scores were actually an accurate predictor of class preparedness,
And sll the students were the same gender,
And all the professors were the same gender,
And all the students had been born in the same hospital, delivered by the same doctor, on the same day, to the same parents (nice trick, that) or at least been raised in strikingly similar circumstances,
And all the professors were likewise exactly the same age and from the same geographical area,
And all of the participants had the same religious affiliation,
And the study itself was conducted in a closed environment,
There would still be no scientific basis for the difference in grading to be attributed to the political affiliation of the professors.
posted by misha at 9:28 AM on June 10, 2011 [3 favorites]

I find this really fascinating and wonder to what extent this is driven by genetics and what is driven by environmental factors?

I think we already covered this in the 80s.
posted by RobotVoodooPower at 9:34 AM on June 10, 2011

those who note grades etc by discipline are right on target. In liberal arts, arts, social sciences you get mostly liberals. In Business, P.E., Education, mostly conservative. In science, mixed.
posted by Postroad at 9:37 AM on June 10, 2011

Wow, think of the implications! If a post secondary institution were to enforce certain grading standards, then they could effectively control the party allegiances of their professors and thus potentially influence the next election!

I knew those logic classes would pay off.
posted by ODiV at 9:38 AM on June 10, 2011

There would still be no scientific basis for the difference in grading to be attributed to the political affiliation of the professors.

I agree, in general. But the question is an interesting one. It's definitely an assumption on my part, but I would think that self-identified political identification by an academic would carry some weight as a "short-cut" to their general personality and attitudes.

There are obviously differences between people who stake out positions as Democrats and those who position themselves as Republicans. That's no excuse to do bad science, of course - but it is an area that could (and should) be explored.
posted by Benny Andajetz at 9:40 AM on June 10, 2011

This might be a good place to leave this, which I found earlier this week but can't figure out how to frame a post about...Since we're talking about apparent differences

Liberals and Conservatives RElay on Different Sets of Moral Foundations(pdf)
posted by never used baby shoes at 9:41 AM on June 10, 2011

I get an F in typing, no matter who is grading - "rely", not "RElay"
posted by never used baby shoes at 9:42 AM on June 10, 2011

Did they control for differences in what kind of classes the professors taught?

Yeah, they either included dummies for broad field (soc. sci vs humanities vs sciences), or they included "fixed effects for department," which boils down to "a fuckload of dummies that uniquely identify every department in their sample."

Two relatively serious problems are

(1) As kproto notes, there are only 27 Republicans in their sample, so those data points are working very hard. I would worry that the results are dependent on a bare few of those people happening to be harsh graders for whatever reason. Maybe they talk about influential outliers; I dunno.

(2) They're making lots of inferences about cross-level interactions. IE, in table 1, that "Republican professor X SAT score" means that they're using the political party of the *professor* to predict the *individual* effect that SAT scores have on grades. This makes my skin crawl a bit. If they weren't doing so many cross-level interactions*, then their approach of just running OLS with clustered standard errors might be okay. Even then, running it as a proper, fully paid-up hierarchical model is trivial, so they ought to. But with their big mess of cross-level interactions, I'd really have wanted to see them using a proper multilevel model instead of just clustering their SEs.

If you REALLY wanted to do it right, it would probably be as a cross-classified model with student and professor identifiers, but I'll admit that I've never run one of those. Or students cross-classified with courses nested in professors, if you can do that.

(3) Grades are probably better modeled as an ordered logit or probit than with a linear model, but this is unlikely to affect sign-and-significance of the coefficients.

*Or, if you'd rather, coefficient-as-outcome models
posted by ROU_Xenophobe at 9:59 AM on June 10, 2011 [1 favorite]

I'm not going to comment on the validity of the study. But people who are saying that there's no way there could possibly be a difference between the grades given by democrats and republicans clearly have never given (earned, assigned, whatever) grades. There are a lot of small decisions that go both into creating the system by which one assigns grades (writing the assignments, deciding how much of the grade they're worth, and so on) and into actually coming up with the numbers that will eventually be smushed together into a letter. To think that those decisions couldn't possibly be correlated with some aspect of the personality of the instructor - especially since there aren't some "best practices" for grading that everyone in every discipline and at every institution follows - is a
bit silly.

In other words, I have made the sausage, and I know that a little bit of my finger is chopped up in there.
posted by madcaptenor at 10:06 AM on June 10, 2011 [2 favorites]

But people who are saying that there's no way there could possibly be a difference between the grades given by democrats and republicans clearly have never given (earned, assigned, whatever) grades.

I don't think anyone is saying that. We're suggesting that the study doesn't prove that contention, not that the contention couldn't possibly be true. And you're not the only person here who has graded college classes.
posted by craichead at 10:13 AM on June 10, 2011

In other words, I have made the sausage, and I know that a little bit of my finger is chopped up in there.

Oh, absolutely. However, it's the professor's job to be clear about that in the syllabus and then apply the measures described in the syllabus as fairly as humanly possible. Your finger may be in the sausage, but I hope you have labeled it appropriately.
posted by GenjiandProust at 10:17 AM on June 10, 2011

valkyryn, I'll bite. I absolutely agree with you, that SAT scores have not been shown to have a strong correllation with preparedness and performance of a student. With that stated, one could assume that if you took a range of scores, say 1000-1400 (I'm not sure what that translates to today - 1600 was max when I took it), that you would therfore have at least a good distribution of scores. While those students might not actually be prepared for anything other than the test, one could assume that students who scored between those bounds were, population-wise, equally likely to contain outliers.

As to adress craichead, if this is really in an applied economics journal, my guess would be that the crossectional data would have addressed the types of curriculum. While those results may not have had a strong showing in any model built (I'd at least examine the fixed effects to make sure that there wasn't some baseline skew by course being taught or the major of the student) - but that's just me... and I'm an unpublished engineer with a BS that works in economics. I'd sure as hell have a couple of econometricians look it over to make sure that I wasn't commiting some major sins before I put out even a tepid statement as to the grade skewing by political affiliation. This is a bit more micro than I generally do, so I'm not even convinced that this kind of thing would hold the test of time.

And to address valkyryn again: an economist doesn't call this science, an economist attempts to quantify the contribution or affect of a given factor on an end result. Is it right? No. Any economist that says that their model can be a reliable predictive tool without imposing a ridiculously long list of constraints and exceptions on their model is probably a fool. An economist is steering a ship at a continent via dead reckoning - and is probably content with knowing whether they hit florida or newfoundland. Is there rigor behind it? Yes. Does the rigor make their case right? No - its just a CYA effort to make sure that people know that the course plotted was with as much information as possible.

You don't have to call it science - it isn't... but it sure as hell isn't making an emotionally charged business guess.
posted by Nanukthedog at 10:19 AM on June 10, 2011 [1 favorite]

"What about Catholics and Protestants?"

Catholic professors somewhat more likely to strand you in academic purgatory with an incomplete for failure to meet all course standards properly; Protestant profs just flunk you for that.
posted by Eyebrows McGee at 12:00 PM on June 10, 2011 [1 favorite]

Who the hell gives letter grades in college? You come home from junior year and shout, "Mommy, I got an A+ in advanced nuclear physics!"? WTF. Never understood this, use a point scale you dopes.

Students in my class were graded based on performance. Get a lot right, get a high score. Get a lot wrong, get a low score. Never looked at names when grading. Preferred scantron-type exams in larger classes because it helps take my bias out of the equation.

If I changed grades because I liked someone, it would make me an unfair asshole to everyone else. I never gave a point to a student unless he or she could demonstrate that he/she had earned it and I had taken it away unfairly. If I had an unfair question on an exam, I gave everyone a point on that question to make up for it. I explained it to sad students very simply: If I give you half a point because I feel bad and you are on the verge of a better grade, it isn't fair unless I also give that half point to everyone else. Then another student will be on the cusp of a better grade, another half point, and in the end everyone has a 4.0 they didn't earn, and I get fired. Nobody wins.

Politics shouldn't matter in education. You do it right and you do it fairly, or you get the hell out of the industry.
posted by caution live frogs at 12:52 PM on June 10, 2011 [1 favorite]

Who the hell gives letter grades in college?

Americans.
posted by madcaptenor at 1:00 PM on June 10, 2011

Who the hell gives letter grades in college?

Every instructor at almost every college and university in the United States. Even the few exceptions are more places that just don't grade than places that record that your final course grade as a 78.

Where have you been associated with that recorded score-based grades?
posted by ROU_Xenophobe at 1:09 PM on June 10, 2011

There are many places where students don't just get O or X as their result. There is a lot of leeway for professors opinions to influence grades in any exam or project that isn't is simple recitation of fact. That student's outline for research into subject A? Maybe its methodology is perfect, but the prof thinks it is too simple or that subject A is worthless.

Of course in something like econ your paper explaining why socialism/free trade is awesome could be graded quite differently based on the political outlook of the professor.
posted by Winnemac at 1:17 PM on June 10, 2011

Assuming by "O or X" you mean the usual system of letter grades, not actually "O" and "X" ... where?

There are a few schools that just don't have grades and record narratives instead. Where in the US are you thinking of that assigns course grades, but with a system finer than an A to F scale with pluses and minuses?
posted by ROU_Xenophobe at 1:28 PM on June 10, 2011

I'm guessing that by "O or X", Winnemac means "right or wrong". (Or "wrong or right", I'm not sure of the order.)
posted by madcaptenor at 2:56 PM on June 10, 2011

"Republicans are more likely than Democrats to award very high grades and very low grades."

Goes along with the Republicans see the world as more black and white.
posted by e40 at 4:07 PM on June 10, 2011 [1 favorite]

"Republicans see the world as more black and white"

While Metafilter's discussion of Republicans is infinitely more balanced and nuanced.
posted by joannemullen at 2:24 AM on June 11, 2011 [1 favorite]

Silly me. I thought all professors were socialists and commies.

I just finished a graduate math class where the professor actually handed out a pamphlet on Ayn Rand and Objectivism on the last day of class. His sylabus had the Ayn Rand quote: "Wealth is the product of man's capacity to think" on the top of it.

Great math teacher though.
posted by jpdoane at 5:59 PM on June 11, 2011

Who the hell gives letter grades in college?

I think it's quite common for US universities to have a numerical scale (0-4) which officially corresponds to a letter scale (F-A). Mine did at any rate, and it frequently referred to various grades and so on by letter ("you must maintain a C grade average", that kind of thing).
posted by hattifattener at 9:25 PM on June 11, 2011

ROU_Xenophobe: "Where have you been associated with that recorded score-based grades?"

In the 12 combined years I spent as an undergrad, grad student and instructor at Michigan State U, I never saw anything except point scale grades. And from what I understand letter grades from high school are regularly translated into point scale grades for comparison purposes by admissions boards. Maybe it's more common outside of the Big 10 schools? it may be more common than I realized - but "Every instructor at almost every college in the US" seems a bit overreaching, doesn't it?
posted by caution live frogs at 10:57 AM on June 22, 2011