Student Evaluations Get It Wrong When It Comes To Professor Quality
June 12, 2010 8:58 AM Subscribe
"Does Professor Quality Matter? Evidence from Random Assignment of Students to Professors" by Scott Carrell and James West
posted by scunning (44 comments total)
29 users marked this as a favorite
is the title of an interesting new study in this month's Journal of Political Economy
, a leading journal in economics. (For a summary of the paper, see this review
. An ungated version
, too). The authors are interested in determining the role of "professor quality" in student learning. They do this by exploiting an unusual institutional feature of the Air Force Academy whereby all undergraduates are randomly assigned their professors, and all professors use the same syllabus. The authors also have the professor's student evaluations, as well each student's subsequent performance in the follow-up classes. To keep it simple, they focus only on Calculus I and the follow-up courses in Calculus (which are mandatory), though they note that an earlier study that looked at Chemistry and Physics found similar things.
They find a few things that are worth consideration. First, they find that professors with higher student evaluations have students who do better on "contemporaneous examinations". Nothing new there. But interestingly, they also find students whose professor had higher student evaluations typically did worse
in subsequent courses. They attribute this to the "teaching to the test" that they think may go on in classes where professors have high student evaluations.
Secondly, they find that students who took Calculus I from professors with lower student evaluations did better
on subsequent courses. They attribute this effect to "deep learning" that is the focus of the professors with the lower student evaluation scores.
Thirdly, the factors that predict student learning in the longrun were not the student evaluations, but rather, were teacher qualities like years of experience, higher academic rank and teaching experience.