Data Visualization: Gendered Language in Teaching Reviews
February 9, 2015 3:30 PM Subscribe
"Gendered Language in Teacher Reviews: This interactive chart lets you explore the words used to describe male and female teachers in about 14 million reviews from RateMyProfessor.com." Created by Ben Schmidt, a professor at Northeastern University, the chart lets you enter specific words to see how they correspond with the professor's gender and teaching discipline. Schmidt provides more background on his blog. Feministing compares the results for "genius" and "bossy."
Organized
Prepared
Helpful
Available
Yeah it seems like students perceive male professors as being more genius-y and female professors as better at ... doing the actual job of being a professor?
posted by rustcrumb at 3:48 PM on February 9, 2015 [5 favorites]
Prepared
Helpful
Available
Yeah it seems like students perceive male professors as being more genius-y and female professors as better at ... doing the actual job of being a professor?
posted by rustcrumb at 3:48 PM on February 9, 2015 [5 favorites]
Another intriguing result: "awesome" skews male. "Amazing" skews female.
posted by yoink at 3:48 PM on February 9, 2015 [4 favorites]
posted by yoink at 3:48 PM on February 9, 2015 [4 favorites]
"Boring" skews male; "tedious" skews female.
posted by Pseudonymous Cognomen at 3:56 PM on February 9, 2015 [2 favorites]
posted by Pseudonymous Cognomen at 3:56 PM on February 9, 2015 [2 favorites]
rustcrumb, it's interesting that "organized" appears in more reviews of female profs, but "disorganized" does too. It seems as though female profs are the ones being scrutinized for organization (or lack of it), whereas it's less on students' radar for male profs. I tried out "caring" as well, and the word does come up much more often for women. It's interesting because this is something women notice, that we are criticized for not being more nurturing, whereas it's not even something expected of male colleagues most of the time. I'm not saying whether this is good or bad, but women are often being judged on quite different criteria. What exactly should the role of a professor be?
yoink, it's true that there is a problem inherent with only looking at data from RMP, and my perception of the site is that it's less popular than it used to be (but still definitely used) and that the students who post there are *often* either the ones who loved their prof or hated their prof. There's not a whole lot of in-between there.
posted by hurdy gurdy girl at 3:58 PM on February 9, 2015 [3 favorites]
yoink, it's true that there is a problem inherent with only looking at data from RMP, and my perception of the site is that it's less popular than it used to be (but still definitely used) and that the students who post there are *often* either the ones who loved their prof or hated their prof. There's not a whole lot of in-between there.
posted by hurdy gurdy girl at 3:58 PM on February 9, 2015 [3 favorites]
"Encourages" and "encouraging" also skew female. As for "caring" it's interesting that it only skews radically female in positive ratings. In the negative ratings, neither "caring" nor "uncaring" have distinct gender profiles. That is, women seem to be singled out for praise for being "caring" but are not singled out for criticism for being "uncaring" or "not caring."
And, yeah, RMP is an odd site. My experience has been that students at my university have kinda drifted away from it (we make the numerical part of our evals available to students now, so that renders RMP fairly redundant). I once went through the RMP reviews of my colleagues--whose actual teaching evaluations I know perfectly well. It was amazing how little correlation there was. There are people who routinely get excellent evals in class who came across as whackjob tyrants on RMP and vice versa. It was impossible to see any consistent pattern in the way RMP would skew, either.
posted by yoink at 4:13 PM on February 9, 2015 [2 favorites]
And, yeah, RMP is an odd site. My experience has been that students at my university have kinda drifted away from it (we make the numerical part of our evals available to students now, so that renders RMP fairly redundant). I once went through the RMP reviews of my colleagues--whose actual teaching evaluations I know perfectly well. It was amazing how little correlation there was. There are people who routinely get excellent evals in class who came across as whackjob tyrants on RMP and vice versa. It was impossible to see any consistent pattern in the way RMP would skew, either.
posted by yoink at 4:13 PM on February 9, 2015 [2 favorites]
The difference between "best professor" (skews male) and "best teacher" (gender neutral) is interesting.
posted by yoink at 4:28 PM on February 9, 2015
posted by yoink at 4:28 PM on February 9, 2015
In addition to being the eight layers of stupid that it is, RMP also gets fucked with in ways that real evals don't. Like, I know a guy whose friend logged onto RMP and rated him as "the kindest, bravest, warmest, most wonderful human being I've ever known in my life." And both are too young to have served in Korea.
posted by ROU_Xenophobe at 4:29 PM on February 9, 2015 [2 favorites]
posted by ROU_Xenophobe at 4:29 PM on February 9, 2015 [2 favorites]
I saw this earlier today and thought it was interesting. I'm still living and dying by RMP -- next year I'll be pretty much locked into my major's core classes and won't have choice in who teaches what, but for now, it's still invaluable. Does this instructor have practice exams? Weird attendance requirements? Are they a total jerk? As far as the broad strokes go, I haven't known it to be wrong, although I generally discount whether a class is reported as "hard" or not. ("Easy" can be counted on, though)
posted by curious nu at 4:53 PM on February 9, 2015
posted by curious nu at 4:53 PM on February 9, 2015
Yeah it seems like students perceive male professors as being more genius-y and female professors as better at ... doing the actual job of being a professor?
Disorganized
Unprepared
Unhelpful
Unavailable
So much for optimistic gender essentialism.
posted by Sys Rq at 5:04 PM on February 9, 2015 [2 favorites]
Disorganized
Unprepared
Unhelpful
Unavailable
So much for optimistic gender essentialism.
posted by Sys Rq at 5:04 PM on February 9, 2015 [2 favorites]
Look at the gender split on the term "female" between anthropology and health science. What does it mean? So confusing.
posted by Schmucko at 5:11 PM on February 9, 2015
posted by Schmucko at 5:11 PM on February 9, 2015
"Not qualified" or "unqualified", engineering vs. everything else. Ouch.
posted by darksasami at 5:43 PM on February 9, 2015 [1 favorite]
posted by darksasami at 5:43 PM on February 9, 2015 [1 favorite]
One of the biggest confounders here would be age/career stage. Women profs will skew much more heavily to younger and early-career/mid-career positions than male profs. A lot of the "organized" and "available" type differences probably relate significantly to that.
posted by yoink at 5:52 PM on February 9, 2015 [2 favorites]
posted by yoink at 5:52 PM on February 9, 2015 [2 favorites]
Huh. Surprised computer science reverses the trend for "unqualified". Not true for "not qualified," though.
"Abrasive" isn't that common, but I'm not seeing the same extreme discrepancy I read about. Maybe it's just not a common word on campuses?
posted by halifix at 5:53 PM on February 9, 2015
"Abrasive" isn't that common, but I'm not seeing the same extreme discrepancy I read about. Maybe it's just not a common word on campuses?
posted by halifix at 5:53 PM on February 9, 2015
I'm not seeing this in the FAQs: does it just measure whether the word is used in a review, or does it actually tell whether the word is used about a professor? So if I type in "lazy," is it going to count reviews that say "don't take this class if you're lazy," as well as ones that say "Professor Jones is stupid and lazy"?
Anyway, I'm particularly interested in ones where some fields are major outliers. So in general, the word "agenda" shows up more in ratings of women than of men, but there are some exceptions: men are more likely to have that word used about them in education, economics, health science, and music. Education, economics and music are also among the not-very-many disciplines where women aren't much more likely to have the word "biased" used in their reviews. Women are much more likely to get "unqualified" in most disciplines, but men are more likely to get it in computer science, health science, economics and physics. I wonder why the discrepancies.
Mostly, though, this makes me want to slit my throat a little bit. That did make me look up "messy," which at least doesn't seem to have a clear gender bias.
posted by ArbitraryAndCapricious at 5:53 PM on February 9, 2015
Anyway, I'm particularly interested in ones where some fields are major outliers. So in general, the word "agenda" shows up more in ratings of women than of men, but there are some exceptions: men are more likely to have that word used about them in education, economics, health science, and music. Education, economics and music are also among the not-very-many disciplines where women aren't much more likely to have the word "biased" used in their reviews. Women are much more likely to get "unqualified" in most disciplines, but men are more likely to get it in computer science, health science, economics and physics. I wonder why the discrepancies.
Mostly, though, this makes me want to slit my throat a little bit. That did make me look up "messy," which at least doesn't seem to have a clear gender bias.
posted by ArbitraryAndCapricious at 5:53 PM on February 9, 2015
It's difficult for me to see any really broad patterns just from entering random words into the interactive chart, but I guess the intention was to look at how specific words are gendered rather than whether male or female professors are seen as more positive/negative overall. It is interesting to look at the really specific extremes, like "bossy" and "gross" overwhelmingly being used for female computer science professors over all other fields.
posted by picklenickle at 6:03 PM on February 9, 2015
posted by picklenickle at 6:03 PM on February 9, 2015
Surprisingly, "sexy" skews strongly male.
"Hot" goes one way or another depending on the field - perhaps not surprisingly I think it kinda tracks with which tend to have a preponderance of men versus women. Though the skew toward female for engineering profs is huge, even more than I would expect.
posted by atoxyl at 6:19 PM on February 9, 2015
"Hot" goes one way or another depending on the field - perhaps not surprisingly I think it kinda tracks with which tend to have a preponderance of men versus women. Though the skew toward female for engineering profs is huge, even more than I would expect.
posted by atoxyl at 6:19 PM on February 9, 2015
Oh, hey; I was literally just having a conversation about the types of gendered comments that TAs I know get in our evals earlier today. It started out as being about whether the person who told my labmate she was cute in her evals was more inappropriate than the one who wrote that she was "easy on the eyes," but it eventually devolved into the way that students treat us differently based on gender and gender presentation in the class room.
Interesting to note that stuff like "rude" and "unhelpful" skew female. I'm not actually that surprised by that--students seem way more solicitous of my time than they do of my male colleagues. Even when I compare myself to male friends of mine who teach roughly the same way I do, I get more students at my office hours and emailing me with occasionally bizarre requests than they do. That's true for other women I know in my department, too. Students seem to expect me to welcome their contact in a way that I don't see men dealing with.
By the way, if the results actually reflected a bias in seniority along gender lines, man, my experience is that senior male faculty are way more likely to be absent or not available to students than younger women. I think it's more about student expectations of women than seniority. Especially given that I think a lot of students, particularly freshmen, tend to slot female teachers in the bit of their head that says "high school teacher"--judging by the way that I know lots of women who get addressed as "Ms. Lastname" but never ever men.
posted by sciatrix at 6:22 PM on February 9, 2015 [4 favorites]
Interesting to note that stuff like "rude" and "unhelpful" skew female. I'm not actually that surprised by that--students seem way more solicitous of my time than they do of my male colleagues. Even when I compare myself to male friends of mine who teach roughly the same way I do, I get more students at my office hours and emailing me with occasionally bizarre requests than they do. That's true for other women I know in my department, too. Students seem to expect me to welcome their contact in a way that I don't see men dealing with.
By the way, if the results actually reflected a bias in seniority along gender lines, man, my experience is that senior male faculty are way more likely to be absent or not available to students than younger women. I think it's more about student expectations of women than seniority. Especially given that I think a lot of students, particularly freshmen, tend to slot female teachers in the bit of their head that says "high school teacher"--judging by the way that I know lots of women who get addressed as "Ms. Lastname" but never ever men.
posted by sciatrix at 6:22 PM on February 9, 2015 [4 favorites]
The word "smelly" has a very, very interesting breakdown.
As you might expect, it is rarely used across the board, with a small bias towards males
EXCEPT in engineering, in which the female dot is wayyyyyy on the far right of the graph, heavily contrasted with the male dot, which is at the extreme left.
What does this mean? Are engineers impervious to the scent of their male instructors, while hypersensitive to feminine odours? Is it blatant sexism and bullying of the female staff?
Or could it be a statistical error, owing to outlier Prof. Stenches Georg, who lives in a cave eating lutefisk adn should not have been counted.
posted by mrjohnmuller at 6:23 PM on February 9, 2015 [4 favorites]
As you might expect, it is rarely used across the board, with a small bias towards males
EXCEPT in engineering, in which the female dot is wayyyyyy on the far right of the graph, heavily contrasted with the male dot, which is at the extreme left.
What does this mean? Are engineers impervious to the scent of their male instructors, while hypersensitive to feminine odours? Is it blatant sexism and bullying of the female staff?
Or could it be a statistical error, owing to outlier Prof. Stenches Georg, who lives in a cave eating lutefisk adn should not have been counted.
posted by mrjohnmuller at 6:23 PM on February 9, 2015 [4 favorites]
It's seriously a huge disparity - male engineering professors get the second least "hot" and female the second most.
Also, Criminal Justice wins for both!
posted by atoxyl at 6:24 PM on February 9, 2015
Also, Criminal Justice wins for both!
posted by atoxyl at 6:24 PM on February 9, 2015
I said gender and gender presentation for a reason, by the way--I'm fairly butch, but none of my female coworkers are, and I'm pretty sure that has an impact on my teaching evals and the way my students respond to me. Not necessarily always positive or always negative, but if anything I get less of the skeevy comments than most of the women I know do.
posted by sciatrix at 6:24 PM on February 9, 2015
posted by sciatrix at 6:24 PM on February 9, 2015
I think I'm enjoying this for the wrong reasons - I've stopped looking at the gender splits and started searching for all the most insulting terms I can think of looking for funny stereotypes of different fields.
posted by atoxyl at 6:34 PM on February 9, 2015 [2 favorites]
posted by atoxyl at 6:34 PM on February 9, 2015 [2 favorites]
I was also pleasantly surprised to see that practically no-one included the word "tits" in their reviews,
with the exception of English (0.03ppm), History (0.05ppm) and, tellingly, Philosophy (0.1ppm)
(all of which occurred in male reviews, interestingly enough)
posted by mrjohnmuller at 6:42 PM on February 9, 2015
with the exception of English (0.03ppm), History (0.05ppm) and, tellingly, Philosophy (0.1ppm)
(all of which occurred in male reviews, interestingly enough)
posted by mrjohnmuller at 6:42 PM on February 9, 2015
I'm not seeing this in the FAQs: does it just measure whether the word is used in a review, or does it actually tell whether the word is used about a professor? So if I type in "lazy," is it going to count reviews that say "don't take this class if you're lazy," as well as ones that say "Professor Jones is stupid and lazy"?
I may have missed it somewhere, but from what I can tell he's just doing straight up word counts and not any sort of sentiment or contextual analysis (beyond identifying professor gender and field). It doesn't even look like he's filtering out simple, common words such as "and", "it", or "I". So yes, I think I think if you type in "lazy", both types of reviews you suggest would be counted.
posted by noneuclidean at 7:05 PM on February 9, 2015 [1 favorite]
I may have missed it somewhere, but from what I can tell he's just doing straight up word counts and not any sort of sentiment or contextual analysis (beyond identifying professor gender and field). It doesn't even look like he's filtering out simple, common words such as "and", "it", or "I". So yes, I think I think if you type in "lazy", both types of reviews you suggest would be counted.
posted by noneuclidean at 7:05 PM on February 9, 2015 [1 favorite]
Yeah, I'd be surpised if he did anything different. Raw word counts is the norm in text analysis.
posted by MisantropicPainforest at 7:11 PM on February 9, 2015
posted by MisantropicPainforest at 7:11 PM on February 9, 2015
Oh jeez this was addictive. Fart skews heavily male (old fart?) except for philosophy and humanities. Enthusiastic is close but more female overall. Liar seems to be a healthy even mix. Cute is mostly male, except for physics and engineering. Booger is confusing. Smarmy is rather male. Sleazy is male, other than a strong female philosophy showing. Slimy is dominated by male profs, though. Douchebag is topped by female criminal justice. Fat is close, but slightly more male. Ineffective looks rather even. Abusive seems to be evenly mixed. Bad breath seems to be a trait of male fine arts profs. Fondled is used once: male business prof, probably the same one who is also a jerkass. Fun homework seems to be assigned evenly, except that women engineering profs are experts at it. Everyone gives confusing lectures or is a confusing lecturer, but men tend to give tedious lectures. Female profs seem to be more often accused of stabbing (stabbed me) or shooting shot me) their students. Both groups of profs are inspiring (inspired me) and convincing (convinced me) their students, but women are helping them (helped me) more. Female profs are way ahead of men in being great people, kind people, understanding people, awesome people, amazing people, cool people, and although it's closer, even funny people and brilliant people. Overall more male profs need fired, but it's hard to tell the pattern of who should be fired. More men seem to get calls for a raise and to be investigated. Men seem to edge out women barely in being insensitive while women seem to edge out men for being cruel. Female profs are rated as unkind, and yet also as very kind, super nice, and super helpful. Men, meanwhile, get the ninja tag (except in anthropo and science!) while women get points for being stylish. Even mix on cleavage, perhaps due to occuring in "looking at ..." phrases? Every one is sloppy. With the exception of profs in science and communication, men are the party animals. Pretty even mix of easy graders and easy grades and courses where the TA sucks. Something about the female profs' courses is leading to calls to stay away, but it's a more even mix of calls to avoid this, and apart from physics and engineering students, students are saying male profs suck/professors suck. And I'm not sure what's going on, but only male professors seem to have anything to do with Donkey Kong.
posted by barnacles at 8:56 PM on February 9, 2015 [5 favorites]
posted by barnacles at 8:56 PM on February 9, 2015 [5 favorites]
I feel like there's nothing more to be said after Barnacles' comment, but I found it interesting that most of the looks-related terms I could come up with skewed male. I had always thought women get more looks-related comments on evaluations, but "hair", "fat", "hair dye" and "tall" all skew male. (I figured "short" was more likely to refer to quizzes or essays, and make-up could be a make-up exam). "Ugly" skews female, though.
I also noticed that "a" and "the" skew slightly male, however, so god knows what's going on.
posted by lollusc at 12:55 AM on February 10, 2015 [1 favorite]
I also noticed that "a" and "the" skew slightly male, however, so god knows what's going on.
posted by lollusc at 12:55 AM on February 10, 2015 [1 favorite]
Also, as a comparison of disciplines, I like the "bullshit" graph.
posted by lollusc at 1:01 AM on February 10, 2015 [1 favorite]
posted by lollusc at 1:01 AM on February 10, 2015 [1 favorite]
Shrill and moist were my favourites. Hopefully for (different) obvious reasons.
posted by taff at 4:54 AM on February 10, 2015
posted by taff at 4:54 AM on February 10, 2015
Wow. "Annoying" is used to describe female professors about 20% more often than it is to describe male professors.
Results for "feminist" are also terribly interesting. I expected to see it infrequently used to describe male profs in the hard sciences, but would have predicted more frequent usage for those in, say, Psychology or the Humanities. Alas.
posted by nicodine at 6:58 AM on February 10, 2015
Results for "feminist" are also terribly interesting. I expected to see it infrequently used to describe male profs in the hard sciences, but would have predicted more frequent usage for those in, say, Psychology or the Humanities. Alas.
posted by nicodine at 6:58 AM on February 10, 2015
By the way, if the results actually reflected a bias in seniority along gender lines, man, my experience is that senior male faculty are way more likely to be absent or not available to students than younger women.
That's my experience too--which is why I was suggesting that this might help explain why "helpful" and "available" skew so strongly female. I was thinking that there must be a lot more men in the senior-prof-who-hardly-ever-talks-to-undergraduates category.
It is, by the way, interesting to compare the negative and positive reviews with the overall results--there are some slightly puzzling effects there. Take "unhelpful" for example. That skews female in the overall results, but if you look at just the negative reviews, it's pretty much gender-neutral. It's also closer to gender-neutral in the positive reviews. So there must be a very strong female skew in the middle-of-the-road reviews. But what the hell is that about?
posted by yoink at 7:26 AM on February 10, 2015
That's my experience too--which is why I was suggesting that this might help explain why "helpful" and "available" skew so strongly female. I was thinking that there must be a lot more men in the senior-prof-who-hardly-ever-talks-to-undergraduates category.
It is, by the way, interesting to compare the negative and positive reviews with the overall results--there are some slightly puzzling effects there. Take "unhelpful" for example. That skews female in the overall results, but if you look at just the negative reviews, it's pretty much gender-neutral. It's also closer to gender-neutral in the positive reviews. So there must be a very strong female skew in the middle-of-the-road reviews. But what the hell is that about?
posted by yoink at 7:26 AM on February 10, 2015
Try "psycho"....yowch.
posted by SinAesthetic at 11:59 AM on February 10, 2015
posted by SinAesthetic at 11:59 AM on February 10, 2015
This article points to a study "in which students could register their evaluations without knowing the actual sex of the instructor, using an on-line course in which the same teacher presented as a male and as a female". The results might surprise you (but probably won't if you've been paying attention).
Students in the two groups that perceived their assistant instructor to be male rated their instructor significantly higher than did the students in the two groups that perceived their assistant instructor to be female, regardless of the actual gender of the assistant instructor….For example, when the actual male and female instructors posted grades after two days as a male, this was considered by students to be a 4.35 out of 5 level of promptness, but when the same two instructors posted grades at the same time as a female, it was considered to be a 3.55 out of 5 level of promptness.posted by mhum at 5:39 PM on February 10, 2015 [3 favorites]
Courtney Gibbons, an assistant professor of mathematics at Hamilton College, writes at mathbabe:
Bad interpretations of data don’t advance the cause.posted by daveliepmann at 4:55 AM on February 20, 2015
...If you look at the ratings for “genius” and then break them down further to look at positive and negative reviews separately, it occurs predominantly in negative reviews. I found a few specific reviews, and they read, “you have to be a genius to pass” or along those lines.
...So yes, the data shows that students are using the word “genius” in more evaluations of men than women. But there’s not a lot to conclude from this; we can’t tell from the data if the student is praising the professor or damning him. All we can see that it’s a word that occurs in negative reviews more often than positive ones. From the data, we don’t even know if it refers to the professor or not.
« Older But it was a beginning | An Oral History of Laurel Canyon in the 60s and... Newer »
This thread has been archived and is closed to new comments
That said, it's interesting that the typical news write up on this is to race to look for positive terms that skew male and negative terms that skew female. It's also interesting to look at the opposite, though. Tough luck, for example, if you're a male prof who wants to be thought of as "caring," "helpful" or--to a lesser extent--"thoughtful." Surprisingly, "sexy" skews strongly male. Women may get "bossy" but then men get "jerk" and "bully" and, massively, "arrogant."
posted by yoink at 3:45 PM on February 9, 2015 [15 favorites]