Get an A by exploiting a loophole in the grading curve
February 18, 2013 3:49 PM Subscribe

In several computer science courses at Johns Hopkins University, the grading curve was set by giving the highest score on the final an A, and then adjusting all lower scores accordingly. The students determined that if they collectively boycotted, then the highest score would be a zero, and so everyone would get an A.
posted by Foci for Analysis (160 comments total) 23 users marked this as a favorite

I am pretty sure this is how government functions.
posted by TwelveTwo at 3:53 PM on February 18, 2013 [29 favorites]

Ha. We used to talk about doing this all the time but could never organize. That and prisoner's dilemma. This is great.
posted by Lutoslawski at 3:53 PM on February 18, 2013 [8 favorites]

OTOH, if ONE student breaks the boycott and gets ONE answer correct, everyone else is screwed.
posted by Repack Rider at 3:54 PM on February 18, 2013 [3 favorites]

People say that the incentive to defect is too high, but I actually think the social stigma of being responsible for a bunch of your comrades failing a class would be enough of a threat that you'd have to cooperate.
posted by vogon_poet at 3:56 PM on February 18, 2013 [5 favorites]

That is a shitty way to grade.
posted by ChuraChura at 3:57 PM on February 18, 2013 [33 favorites]

I think the focus should be less on "students gaming the system" and more on:

"Computer Science knowledge can be objectively measured. Why on earth is there such an arbitrary grading curve?"
posted by pmv at 3:57 PM on February 18, 2013 [64 favorites]

The students determined that if they collectively boycotted, then the highest score would be a zero, and so everyone would get an A.

Yeah, the impressive is pulling it off, not realising this. Because, really, everyone realises this in the first class you have that's curved, so at the age of 12 or so.
posted by hoyland at 3:57 PM on February 18, 2013 [4 favorites]

Why is a curve adjustment to your grade part of the grading system? Why are the grades not based on the work done by each individual?
posted by radiosilents at 3:58 PM on February 18, 2013 [3 favorites]

Zero is a special case, and this is compsci ffs. It's like whoever came up with the curve was daring someone to boycott.
posted by lordaych at 3:58 PM on February 18, 2013 [10 favorites]

The thing is, there is no disincentive to not taking the exam. You walk in and take the exam, then all who follow you will do the same, and you are not guaranteed the highest grade.

On the other hand, you do nothing, and you are guaranteed the highest grade.

The problem isn't that the students exploited the loophole, its that the system was designed poorly. It's a perfect example of how code, rigid by nature, can have unintended consequences.

I say these kids deserve the A grade they got - quite literally.
posted by Xoebe at 3:58 PM on February 18, 2013 [28 favorites]

Most of my classes had an extra clause stating that you needed to take the final in order to pass the course, or something to that effect.
posted by crasiman at 3:59 PM on February 18, 2013 [1 favorite]

And it's so obvious as not even to need stating, apparently, that all the students and the professor believe the exam serves no pedagogical purpose, only an evaluative one? Where's the Nash equilibrium for "what are any of these people doing taking, or teaching, a class like this?"
posted by RogerB at 3:59 PM on February 18, 2013 [5 favorites]

That and prisoner's dilemma.

There's no prisoner's dilemma here. You get an A whether you defect or not, so there's no reason to defect.
posted by Tell Me No Lies at 4:03 PM on February 18, 2013 [7 favorites]

So another option is let them pass and then remove the curve and grade on objective goals based on what they need to know to be sufficiently founded in the subject, vs watering down education to the smartest guy in the room when rooms aren't that big.
posted by lordaych at 4:04 PM on February 18, 2013 [1 favorite]

Well, there is sort of a prisoner's dilemma here. It's actually more like, I dunno, the Bastard's Dilemma?

1. You defect, you get the only A and avoid the inevitable reprisals that come with any sort of defiance of the system.

2. You don't defect, get an A, and leave yourself open to the same.
posted by absalom at 4:05 PM on February 18, 2013 [1 favorite]

The Scab's Dilemma.
posted by absalom at 4:06 PM on February 18, 2013 [38 favorites]

I vaguely know the prof, and his philosophies lie along the lines of my username; wouldn't be surprised if he had let the idea slip in class once or twice.
posted by anarch at 4:06 PM on February 18, 2013 [11 favorites]

The instructor, Peter Fröhlich, lists as his latest publication* "Game Design: Tricking Students into Learning More." Kinda flips the whole thing back into "deliciously ironic" territory for me.

PowerPoint slides count as "publications" in CS? I knew I picked the wrong field
posted by RogerB at 4:06 PM on February 18, 2013 [26 favorites]

And it's so obvious as not even to need stating, apparently, that all the students and the professor believe the exam serves no pedagogical purpose, only an evaluative one? Where's the Nash equilibrium for "what are any of these people doing taking, or teaching, a class like this?"

It's not uncommon for the sole grading criterion in graduate courses to be "Is this person enrolled in the course?" If so, you get an A. You could be dead and, as long as the university hasn't noticed and cancelled your registration, you get an A. (So it's always funny to me when someone goes on about their grades in graduate courses on AskMe. Their grades might be meaningful, but I assume they aren't.) Universities like grades, so people are stuck assigning them. In an undergrad computer science course, it's a pretty good bet that some people didn't do any work. Or that whatever work they did hand in was half-assed. On some level, having a final exam is a good way of ensuring these people don't scrape a passing grade.
posted by hoyland at 4:07 PM on February 18, 2013

Interesting definition of grading on a curve. I would think that what would happen is that the normal distribution would become infinitely flat and everyone would get a C, or whatever the median was set to be. Unless the median was set to be an A? Surely grade inflation isn't that bad at JHU.
posted by jedicus at 4:08 PM on February 18, 2013 [7 favorites]

Kinda flips the whole thing back into "deliciously ironic" territory for me.

(This is not what ironic means.)
posted by incessant at 4:08 PM on February 18, 2013

1) This guy taught my compilers class at a different university. I remember him being a guy who would be ok with something like this.

2) Setting the scores like this does serve a purpose. Sometimes the tests are far too easy so everyone gets an A. Sometimes they're unfairly difficult, maybe some students had better TAs than others, maybe the professor kind of flaked, maybe questions weren't clear, there's a whole number of reasons that someone could bomb a test not related to the subject matter. By setting the curve like this there's a buffer in the event that the test is too hard. I had some math classes where the top score was somewhere around 60% because the teacher was a crazy Russian genius that barely spoke English. Without a curve everyone would have died. Even with the top score at a 60%, a significant number failed.
posted by mikesch at 4:10 PM on February 18, 2013 [4 favorites]

I lol'd. I had computer science professors who graded this way generally but they also demonstrated that they wouldn't hesitate to flunk the entire class on the exam if they weren't satisfied with their performance. One term a full 80% of the class failed not just the test, but the whole class.
posted by mullingitover at 4:10 PM on February 18, 2013 [1 favorite]

OTOH, if ONE student breaks the boycott and gets ONE answer correct, everyone else is screwed.

Solidarity is important.
posted by The Whelk at 4:13 PM on February 18, 2013 [6 favorites]

The whole thing feels gross to me. Yeah, I guess it's "clever," but in the most intellectually bankrupt kind of way possible, like Tim Ferriss becoming a "world-class" Sanshou expert by mastering a very specific type of DQ.
posted by en forme de poire at 4:15 PM on February 18, 2013

If your exams are hard enough that everyone bombs, then you are teaching poorly. Whether you are a crazy genius or not.
posted by ChuraChura at 4:15 PM on February 18, 2013 [7 favorites]

Just out of curiosity, was the bell curve rule that the students exploited a set-in-stone kind of deal? I was always under the impression that, within reason, the grade came at your professor's discretion. Some professors could grade on a curve, others linearly. It was never a hard rule that was dictated to the professors.

If it were up to me, as a professor I'd give everyone the zero score they earned and flunk them out of the class (assuming this was a make-or-break midterm). However, I'd temper the punishment with a smiley face on the exams I graded, and buy them all a beer for saving me the trouble of grading for the rest of the term.
posted by mullingitover at 4:18 PM on February 18, 2013 [4 favorites]

"Joke's on you! I instituted the curve to see if you could figure out the bug in the system and exploit it!"
posted by Marisa Stole the Precious Thing at 4:19 PM on February 18, 2013 [4 favorites]

I once took a 3rd year Combinatorics class - everyone tanked the midterm, which consisted of 5 questions each worth 20%. Bell curve method was to just score your top 3 - I did equally poorly on all 5, so my mark hardly changed. My pal only answered 3 (but correctly), so went from 60% to 100%. Bell curving sucks - you're teaching and/or setting tests poorly.
posted by parki at 4:22 PM on February 18, 2013

Ha. We used to talk about doing this all the time but could never organize.

I tried to convince the summer Differential Equations class I was teaching to do this. Unsurprisingly, perhaps, they didn't believe me. Maybe I should have failed all of them for failing to heed my instructions.
posted by ennui.bz at 4:26 PM on February 18, 2013 [4 favorites]

(This is not what ironic means.)

What, are we curving the Understanding Literary Terms exam now too? A guy who thinks he can "trick" students into learning more becomes the victim of a trick his students figure out that allows them to learn less — that reversal of expectation there is the fucking textbook definition of irony.
posted by RogerB at 4:26 PM on February 18, 2013 [6 favorites]

There's no prisoner's dilemma here. You get an A whether you defect or not, so there's no reason to defect.

Your A is more valuable if you are the only one who gets it as that will increase your relative ranking in the class (this is a mark for only one test in the class, not the entire class), so you have incentive to defect and hope everyone else co-operates. Classic prisoner's dilemma.
posted by kithrater at 4:27 PM on February 18, 2013 [1 favorite]

I want to make it clear that I'm totally the guy who would show up and take the exam just to screw you all over, you slackers.
posted by The Tensor at 4:27 PM on February 18, 2013 [15 favorites]

Bell curve method was to just score your top 3 - I did equally poorly on all 5, so my mark hardly changed. My pal only answered 3 (but correctly), so went from 60% to 100%. Bell curving sucks - you're teaching and/or setting tests poorly.

This isn't a bell curve. In a really strict sense, you make the mean a C, go half a standard deviation up for a C+, half a standard deviation down for a C-, then another one in each direction for Bs and Ds, then As and Fs with the rest. Approximating this is to say 10% of students get an A, 15% a B and whatever on down. It's entirely possible the professor took everyone's best three problems and then graded on a strict curve, but that's not what you've described.
posted by hoyland at 4:29 PM on February 18, 2013 [1 favorite]

I want to make it clear that I'm totally the guy who would show up and take the exam just to screw you all over, you slackers.

Lemme guess - this was also you:

"Hey, didn't you tell us we were taking a quiz today?"

Friday bell rings, and "Whoah, hang on, no homework this weekend?"

"Why don't we try to organize the pledge drive? The whole class!"
posted by Marisa Stole the Precious Thing at 4:29 PM on February 18, 2013 [16 favorites]

Isn't the bell curve supposed to reflect the distribution of IQ in the whole population? Why then is it used to assign grades to people who are all in the top 10 or 15 percent, as students are in highly selective colleges? It's never made any sense to me.
posted by mareli at 4:30 PM on February 18, 2013

Shhh, boycott the article! If you read it he may use your comments to update it, thereby making your participation here part of what you read there! It's a complex loop!
posted by cjorgensen at 4:30 PM on February 18, 2013

I also totally agree with a couple of the comments on the NYT (for once): he should have just given them all incompletes.
posted by en forme de poire at 4:30 PM on February 18, 2013 [1 favorite]

(In conclusion, undergrads need to get off my lawn and I need to eat something before I get even crankier)
posted by en forme de poire at 4:32 PM on February 18, 2013

I had some math classes where the top score was somewhere around 60% because the teacher was a crazy Russian genius that barely spoke English.

I know this is my pet peeve, but the top score on a math exam being 60% is not outlandish of in classes taught by native English speakers in English-language universities. In my experience, that means the exam wasn't really successful, but everyone writes lousy exams occasionally.
posted by hoyland at 4:35 PM on February 18, 2013 [1 favorite]

Think of this from the point of view of the test-writer: you are not giving the same test as any other year, and you can't really test the test without ruining the "surprise value" of the specific questions (not that there aren't at least a few college courses where you're told in advance that the exam will consists of M questions taken from a list of N questions, M≤N).

So you may have at least somewhat misjudged the difficulty of the questions. You could misjudge in one of two ways: so that it is very easy to get a high score, or so that it is very easy to get a low score. Since it's at the top end that it is important to measure, you have to write difficult questions, maybe ones that you judge an "A" student would get 90% on. Once the percentile or point results are in, you select an appropriate "curve".

However, if you're going to publish a specific curve rule, and you don't want your students pulling this s— on you, pick one that will not give everyone an A on the final exam if nobody takes the test. For instance, you might curve by picking a scaling factor K such that K*(1+max(scores)) = 100% except K=1 if max(scores)=100%.
posted by jepler at 4:36 PM on February 18, 2013

This famously happened once (once!) in one of the engineering disciplines at Waterloo. Except that four students rebelled, causing 85 of their classmates to fail the term and repeat the semester.

The four students went on to finish the program as a class of four: holding lectures in professor's living rooms, having their pick of internships and grad placement openings, and their graduation class photo was a set of four portraits.

The subsequently overcrowded cohort in the year behind them was pissed.
posted by ceribus peribus at 4:38 PM on February 18, 2013 [40 favorites]

Isn't the bell curve supposed to reflect the distribution of IQ in the whole population?

With good reason, the Bell curve is also called the Normal Distribution. It can be used to model a very wide range of phenomena.

That said, Bell curving grades is crazy. I find my grades often come out bimodal (two peaks), with one cluster of students who did the homework and one cluster who did not.
posted by justsomebodythatyouusedtoknow at 4:42 PM on February 18, 2013 [15 favorites]

This whole thing could have been avoided if they just graded each exam on its merits. Sure the students might be slackers - so is the faculty.
posted by Kurichina at 4:44 PM on February 18, 2013

Ahahaha. I would have worked SO HARD for that test just to fuck over everyone else in the class. When I was in school, if I had a class that has easy exams, I would compete with the other top students to be first to finish. I would be absolutely delighted to troll every student that intentionally failed just to fuck up their GPA.
posted by ryanrs at 4:45 PM on February 18, 2013

We get it, ryanrs...you're a tremendous douche.
posted by kjs3 at 4:49 PM on February 18, 2013 [50 favorites]

Pretty sweet how this thread is bringing out the people bragging about what assholes they are.
posted by adamdschneider at 4:50 PM on February 18, 2013 [11 favorites]

Pretty sweet how this thread is bringing out the people bragging about what assholes they are.

Yes, nourish me with your warm, delicious tears!
posted by The Tensor at 4:56 PM on February 18, 2013 [6 favorites]

I LOATHED curved grades when I was at University. There was a class I took, "Science of Sleep" for my psych major, which was notorious for having both difficult material and a difficult instructor. Fine, no problem, I've spent my entire college career at a highly-selective, competitive university, no big.

Until that first evening, when he unveiled the syllabus that proclaimed that the grades would be on a curve in ALL CAPS. You could hear the collective moan of eight students gunning for graduate school from thirty blocks away.

That semester (my last of Senior year, BTW) was extraordinarily difficult. Usually small upper-level classes mean classmates study together more and are a little more supportive. This one? A fucking blood bath. One of my classmates was a research partner of mine and we didn't speak to each other while we were in that damn room.

I knew it was ridiculous.

I still was very very pissed when I was one of the two students to be granted a B, thus dashing any chance I had of graduating cum laude on the stones. Fuck him and the high horse he rode on.

Curved grades do nothing good when your students are insanely competitive bastards.
posted by lineofsight at 4:56 PM on February 18, 2013 [1 favorite]

The subsequently overcrowded cohort in the year behind them was pissed.

That's an absolutely horrible situation, and I'm both surprised and slightly disgusted that the school allowed it to happen whether or not the students who conspired to throw the course were in the wrong. As was mentioned earlier in the thread, everyone should have just received an incomplete.

This is a college course, not the Kobayashi Maru.
posted by RonButNotStupid at 4:57 PM on February 18, 2013 [10 favorites]

Hey, it's not my fault you're a workshy schemer that didn't bother to learn the material. Why not skip the whole program and save some money?
posted by ryanrs at 4:57 PM on February 18, 2013 [5 favorites]

The real world tends to just ream you out when you feel clever and try to pull this shit off.

This was a computer science class. The "real world" these students are preparing to work in is an abstract zone of formal rules and rigid mechanisms. Exploiting edge cases in those mechanisms is a big deal, in both positive and negative ways. I suspect that the professor understands this.
posted by Mars Saxman at 4:57 PM on February 18, 2013 [2 favorites]

That's an absolutely horrible situation, and I'm both surprised and slightly disgusted that the school allowed it to happen

It was a learning experience.
posted by ryanrs at 4:58 PM on February 18, 2013 [1 favorite]

jepler: You (and the other people defending the curve, but you were most recent) seem to be assuming that any class which is taught and scored correctly should actually follow the normal distribution. This assumes that there are a certain set of students in every class who deserve to fail, or to get a poor grade because they mastered the material poorly.

This seems to me like a bad assumption. I think that a well-taught class should educate anyone in the room who is trying to learn it on all of the material covered. Sure, sometimes people aren't willing to put in the effort and/or have more trouble, but it doesn't seem like that should be mathematically enforced onto the class. Either students get the material or they don't, and the professor should be able to tell, and should grade them accordingly.
posted by contrarian at 5:00 PM on February 18, 2013 [11 favorites]

It was a learning experience.

Only if you want people to become rules lawyers.
posted by RonButNotStupid at 5:05 PM on February 18, 2013 [2 favorites]

I would imagine that actually pulling it off is nothing about rules lawyering and everything about the kind of skills that are useful everywhere.
posted by anonymisc at 5:08 PM on February 18, 2013 [7 favorites]

Pretty sweet how this thread is bringing out the people bragging about what assholes they are.

Not only assholes, but overrating what they think they could do (or just not reading the article). Excerpt:

The students refused to come into the room and take the exam, so we sat there for a while: me on the inside, they on the outside,” [Peter Fröhlich, the professor,] said. “After about 20-30 minutes I would give up…. Then we all left.” The students waited outside the rooms to make sure that others honored the boycott, and were poised to go in if someone had. No one did, though.

So, even assuming you are willing to show your face to your classmates in order to take the test, it's not like everyone else would have gotten a zero--they would have simply walked in at the same time you did and taken the test. At best you would have studied and they didn't, which is some advantage I guess, though cramming is probably overrated in general if you've actually worked through the semester.
posted by dsfan at 5:08 PM on February 18, 2013 [1 favorite]

Only if you want people to become rules lawyers

Who do you think were the ones trying to skate by on the rules rather than earnest effort?
posted by ryanrs at 5:09 PM on February 18, 2013 [1 favorite]

I tried not to call for making the test's outcome fit any particular preconceived notion of what the distribution of outcomes should be, but I do believe that a good exam is going to show whether students have learned the subject matter; if some of them have not, then the test should show that. (unless your goal is simply to have the most cumbersome attendance-taking system possible)

And I absolutely don't think that a "curve" should be designed so that some fraction receive a failing grade no matter how well they did in raw terms.
posted by jepler at 5:09 PM on February 18, 2013

Oh wow ... this thread is slowly metamorphosing into a hypothetical classroom!
posted by Marisa Stole the Precious Thing at 5:11 PM on February 18, 2013 [1 favorite]

I'd say make them all take it over again with no curve. The real world tends to just ream you out when you feel clever and try to pull this shit off. Unless you're gloriously rich or it's cut and dried legal stuff. But at work? I find loopholes all of the time but they get crushed if you draw attention and there's no recourse when the loophole is removed and you're told to do it the hard way.

It's summer, I am 18, and going through basic training in the Canadian militia (long story).

At the end of our morning five km run and exercises, we are doing windsprints across the vehicle pool parking lot. Each round the winner gets to go back to barracks to shower and change.

I am not by any stretch athletic, certainly in the lower half of my platoon in physical strength and endurance. But I figure out that if I take it really easy for a couple of rounds I can then turn it on and win. So, technically and according to the rules as laid out, I easily won the third or fourth race and waited to hear my name called for the showers. The sergeant has seen what I am up to, and he isn't having it.

"Jefferson to the showers! Meatbomb what do you think you are doing?"

"Strategy, sergeant."

"You leave strategy to the officers! When I say run you fucking RUN!"

I ended up last, or almost last anyways.
posted by Meatbomb at 5:12 PM on February 18, 2013 [37 favorites]

This reminds me of the first season finale of the old TV series The Paper Chase. Oh hay, look everything is on the internet now! Episode 22, "Scavenger Hunt:"

Kingsfield's annual scavenger hunt is much tougher than usual in recognition of the superior quality of this year's batch of students. They have only three days to answer at least 75 of 100 fiendishly difficult legal questions in order to pass. It soon becomes clear that this is an impossible task, as competing groups sabotage each other's efforts, hiding or keeping needed reference texts. The dean and the rest of the faculty try to get Kingsfield to cancel the hunt, as it is disrupting the entire university. Hart figures out Kingsfield's hidden agenda and how to pass the test.

Since they're studying contract law, the solution for the students to survive the test is to share the materials they're hoarding -- which requires them to write contracts that detail the agreements they are making.
posted by localroger at 5:12 PM on February 18, 2013 [3 favorites]

Only if you want people to become rules lawyers.

Programmers are rules lawyers. We make an living out of understanding, manipulating, and building hyper-literal rules systems that have no unspoken exceptions and where all possible flaws will be exploited at some point by somebody, intentionally or otherwise.

As a professional software developer I see nothing at all wrong with what they did. If you don't want a given outcome, don't build a ruleset that allows that outcome and then let a bunch of snotty young hacker types attack it.
posted by Tomorrowful at 5:16 PM on February 18, 2013 [28 favorites]

If your exams are hard enough that everyone bombs, then you are teaching poorly. Whether you are a crazy genius or not.

Not necessarily. At one point I started a new job and was teaching the same intro course I've taught since time immemorial. Being busy in the new job, I admit that I figured "Hey, these guys probably don't know anyone in Raleigh. Why not just recycle the old exam?" So I did.

Average at the old place: 70-something. Same old me teaching the same old material, evaluated with the same old exam, but different students: average of about 35.
posted by ROU_Xenophobe at 5:16 PM on February 18, 2013 [3 favorites]

Low marks for sportsmanship.
posted by StickyCarpet at 5:19 PM on February 18, 2013

When I went to art school, this is how they graded. Yup.
posted by St. Alia of the Bunnies at 5:19 PM on February 18, 2013

If you don't want a given outcome, don't build a ruleset that allows that outcome and then let a bunch of snotty young hacker types attack it.

None of you will earn any points for this exam. Congratulations, you have just learned that syllabi are not rulesets, contracts, or programming languages.
posted by ROU_Xenophobe at 5:21 PM on February 18, 2013 [7 favorites]

"My homebrew grade-recording software seems to be coercing 0/0 to 0, rather than NaN. I wonder why that is?" Problem solved.

Anyway, I hate this sort of thing. It's not so much being smart as being a smartass. I'd have taken the test, just to fuck with the people who hadn't studied because they thought they could pull this trick off. Not because I'd be confident of setting the curve- I kinda suck at CS tests- but because my parents paid for me to actually learn something, and if it's a halfway competently taught class, the testing is part of that. Why take a class if you're not willing to be graded on it?

This sort of "trick" is kind of along the lines of writing a program that may technically solve the problem as stated, but completely miss the mark due to smartassery (doesn't behave as it's supposed to, is completely indecipherable to anyone else, screws up existing code, etc).
posted by BungaDunga at 5:25 PM on February 18, 2013 [1 favorite]

From the article:
First, exploiting loopholes ensures increasing rules, laws, and language (to close previous loopholes), which lead to creating more complexity. More complexity, in turn, leads to more loopholes (among other things). … you see where this is going.

Second, ‘gaming the system’ is a form of game theory. What’s best for you, the individual (or in this case, a small group), may not be best for society.

Today’s college kids are tomorrow’s bankers and CEO’s. Just because you can do something doesn’t mean you should.

First of all, that opening assumption is sort of asinine, but whatever.

Second of all, I can't speculate to the motives of the professor (to be honest it seems like a stunt that says something about game theory instead of an honest attempt to structure a fair grading curve), but these students seemed to do what was 'right for society'. They used collective bargaining power to protest an unfair curve.

Third, (maybe less-so in programming) but exams at a graduate level strike me as a really poor way to evaluate anyway. At that point you should be preparing people to do difficult work in teams, and having them cram for a one-time regurgitation of material isn't an effective way to do that.

None of you will earn any points for this exam. Congratulations, you have just learned that syllabi are not rulesets, contracts, or programming languages.

Enjoy 60 or so angry emails to the dean, a solid year's worth of bad performance reviews, and other massive risks to achieving tenure at some point.
posted by codacorolla at 5:26 PM on February 18, 2013 [1 favorite]

Boston Rob would be proud.
posted by ZeusHumms at 5:27 PM on February 18, 2013 [3 favorites]

Where the rules are made up, and the points don't mean anything...

Two decades on in my professional career, I've found the correlation between GPA and real world ability is vanishingly small.
posted by underflow at 5:31 PM on February 18, 2013 [3 favorites]

While one can argue that things like the OP action are a bit too creative, it seems to me almost as if the prof was thinking of that Paper Chase episode and hoping for something like this to happen. Besides which, these were in fact computer science courses, so the metasolution is not out of line.

Besides which, there is nothing quite as destructive to my own sense of motivation as having the rules changed in the middle. I don't care how crazy the rules are, but if you set a target and I make it, and THEN you decide there's some reason I don't get the reward -- fuck you, I will never be able to work with you again. This has happened IRL and I mean it.

If I had been in Meatbomb's situation with the drill sergeant I would have simply walked the rest of the way and gotten in a nice rested and relaxed last, and I'd have explained in clear terms why, and if I was then told to do more laps I would have sat down. But then, that's why I knew better than to ever enter the military. The people I work for know that certain things often done to other people of my own position skill set are not permissible.

I will hack your embedded controller and make it do things the manufacturer doesn't think are possible, but I am not in the shit business.
posted by localroger at 5:34 PM on February 18, 2013 [8 favorites]

Second of all, I can't speculate to the motives of the professor (to be honest it seems like a stunt that says something about game theory instead of an honest attempt to structure a fair grading curve)

"Mark the exam out of the largest total score" is a fairly common system. Certainly it was a regular feature of my high school experience. I don't think I ever saw it explicitly used in college, probably because you get too wide a distribution of grades in a class that's large enough to curve. Certainly in a large calculus class you can assume a perfect or near perfect exam and a near blank exam (or on one memorable occasion, a mostly completed exam receiving 1 point), with the mean somewhere in the 50s or 60s.
posted by hoyland at 5:39 PM on February 18, 2013

> Yes, but with very few exceptions a programmer who makes use of an undocumented feature to achieve a desired result is a terrible programmer.

This feature was documented, explicit and widely understood. It was just awaiting testing.
posted by Space Coyote at 5:47 PM on February 18, 2013 [8 favorites]

Average at the old place: 70-something. Same old me teaching the same old material, evaluated with the same old exam, but different students: average of about 35.

I think that part of being a good teacher is figuring out where your students are at and fairly assessing their knowledge. If you don't have a grasp of your students' level of understanding, and give them an exam that is too hard for them to pass, then you probably are not teaching particularly effectively.
posted by ChuraChura at 5:47 PM on February 18, 2013 [4 favorites]

Professional software developers should be intelligent enough to recognize the quality of the ruleset they're working within, discern the intent of those rules, and work accordingly.

That's a great philosophy if you're writing enterprise software for a bank. If you're competing against me in the embedded world it will allow me to drink your milkshake.
posted by localroger at 5:50 PM on February 18, 2013 [8 favorites]

This is a joke, right? I mean to say, what locked the prof into this arrangement? What prevents him from saying at the entrance to the classroom, don't take the test, automatic fail?

(By the way, as an employer I want to know if you got your grades by real accomplishment or by gaming the system. Call me old fashioned, but I'd rather not have the latter sort. Sometimes judges and courts are not as easily amused or cowed by smart-assery.)
posted by IndigoJones at 5:54 PM on February 18, 2013

What prevents him from saying at the entrance to the classroom, don't take the test, automatic fail?

His word.
posted by localroger at 5:57 PM on February 18, 2013 [6 favorites]

Tell Me No Lies: "There's no prisoner's dilemma here. You get an A whether you defect or not, so there's no reason to defect."

Well, besides the epic lolz, presumably there's some value to an undergraduate degree from a prestigous university, and defecting would make your scarce degree/GPA even more scarce.
posted by pwnguin at 6:02 PM on February 18, 2013

If you don't have a grasp of your students' level of understanding, and give them an exam that is too hard for them to pass, then you probably are not teaching particularly effectively.

What?!?? Exams are not set based on how well the students are thought to have learned the material, they're based on the syllabus. At many universities, in fact, the teaching professor has no say in the examination process whatsoever.
posted by Pre-Taped Call In Show at 6:03 PM on February 18, 2013

There's no prisoner's dilemma here. You get an A whether you defect or not, so there's no reason to defect.

Let's assume everyone else is cooperating. Then you get something good whether you defect or not. But if someone else defects, then if you cooperate, you're royally screwed, but if you defect too, you're neither particularly well off nor particularly badly off.

I just described both the curved test and the prisoner's dilemma. They are almost exactly the same game, with the exception being that, in the test, the payoff for (you defect, others cooperate) is the same as (you cooperate, others cooperate).

The reason to defect is that if anyone else were to defect, you'd be protected. If I were in this situation, I would have no faith that a coalition of people explicitly formed around the notion of gaming the system would somehow be free of suitably paranoid/cynical people. I would defect to save my skin.
posted by a snickering nuthatch at 6:15 PM on February 18, 2013

> The reason to defect is that if anyone else were to defect, you'd be protected. If

The twist in this case was that everyone showed up to the exam location, and would all have gone in to write the test if someone had defected. Prisoner's dilemma relies on not knowing what your counterpart's choice is going to be.
posted by Space Coyote at 6:17 PM on February 18, 2013 [9 favorites]

The point of exams, as far as I'm concerned, is to give my students the opportunity to show me that they've understood the material. I see no benefit at all to giving students an exam where they'll get a 35% average. I am lucky to have the freedom to design my course so that students who do the work will pass the class. The idea that exams being so difficult that everyone fails is some mark of special brilliance of the professor is stupid and damaging to student learning. I want my students to learn what I am teaching, and exams are one way they have to show me that they've learned stuff. If they all fail, I've done a poor job. And if I didn't notice that they were going to all fail and do something to stop that before everyone gets, on average, 1/3 of the questions right, then I have done something wrong as a teacher.
posted by ChuraChura at 6:18 PM on February 18, 2013 [4 favorites]

Frankly I'd consider someone's experience in being part of the class that pulled this off far more valuable than whatever they were supposed to be learning in the class. It's not like you can get your entire degree with stunts like this, but pulling it off even once is an exercise in real life organization, planning, and dedication that few college students can bring to the table.
posted by localroger at 6:21 PM on February 18, 2013 [3 favorites]

The amazing thing isn't that they figured it out. The amazing thing is that someone didn't break the boycott and cause everyone else to fail "for the lulz."

Most people would look at the scenario and think, "You get an A either way, so why break the boycott and screw it up for everyone?"

But there's always someone who would look at it and think, "You get an A either way, so why NOT break the boycott and screw it up for everyone?
posted by ErikaB at 6:22 PM on February 18, 2013

Add a little Bayes in, and it's not surprising this happened once out of all the times courses were ever graded on a curve.
posted by Apropos of Something at 6:23 PM on February 18, 2013

I'm particularly curious about what level of course this kind of curve is done on. It's reasonable enough, in a large group of introductory students, to think that there is likely to be an approximately normal curve. (It's also reasonable enough not to force it if there are a lot of premeds or other very competitive types.) But in a small class it's just too variable, and in an advanced class where there has been pre-selection it's also a non-ideal concept.

If the exam is too easy, well, the students were lucky. If the exam is too hard, there's nothing wrong with grading it out of 85 or 70 or whatever makes sense. Looking at how the grades fall out and deciding what bands are A and B and C etc also works. But claiming in advance that 15% will get an A and 15% will fail -- I don't see how that fosters any kind of learning environment.
posted by jeather at 6:23 PM on February 18, 2013

I'm amazed this worked, even leaving aside the question of a close reading of the syllabus and university regulations looking for a counter-loophole the professor could exploit.

I'd expect a significant number of students would make a point of undermining the group, in three different categories: True believers, who would claim that academic integrity requires they meet the obvious intention of the class syllabus rather than its details; assholes, who would go out of their way to avoid devaluing their grade by inflating the grades of others; and scardycats, who would fear the professor's wrath and take pains to avoid it no matter what their peers thought of them. And, it seems to me all three camps would have a pretty compelling point in this case, since there really is nothing good at stake here. (I supported walkouts in support of several causes when I was a student, but "the right to scammy legalistic nitpickery" isn't one of them.)

Finding loopholes in a syllabus in order to get out of doing the things you signed up for is a pretty cheesy move. Perhaps you believe that the class and or the test is genuinely unfair and it's thus justified. . . but, how likely is it that everyone in the class would make that claim? What's more, it seems like there's a very real possibility they'd all wind up with a rescheduled exam or even a need to retake the class, which could seriously screw up people's schedules.

In defense of the asshole position, while it's true that joining the boycott doesn't hurt you in the context of the grade for one particular class, it's also true that in a broader context it genuinely will hurt people, if only very slightly. There are plenty of opportunities and awards that depend not only on your grades, but on your grades compared to your peers. One or two classes aren't going to have a major impact on anything, but it could make a real difference at the margin. I'd expect at least a couple of hotheads, panicked about grad school admissions, to claim that this is their very best class and they deserve to get a better score than others. While they'd be overreacting, they wouldn't be wrong.

Frankly, it's hard to imagine that nobody in Introduction to Programming forced the issue all the way to the stage of "unless you physically assault me, I'm going into the room." Maybe Johns Hopkins students are nicer than the people I went to school with?
posted by eotvos at 6:23 PM on February 18, 2013 [1 favorite]

But there's always someone who would look at it and think, "You get an A either way, so why NOT break the boycott and screw it up for everyone?

Because all the students were standing outside the door. If one person went in, all the students would, meaning, presumably, that their scores would have been distributed as if there never had been a boycott. So, the only way to guarantee that A is to not go in.
posted by elephantday at 6:25 PM on February 18, 2013

None of you will earn any points for this exam. Congratulations, you have just learned that syllabi are not rulesets, contracts, or programming languages.

They are certainly rulesets, arguably contracts, and it is a computer science course. You want programmers to think like this.

I agree with whomever above said that it was an earned A.
posted by ChurchHatesTucker at 6:29 PM on February 18, 2013 [2 favorites]

That's an absolutely horrible situation, and I'm both surprised and slightly disgusted that the school allowed it to happen...

Well, not to brag, but Waterloo's engineering program already had a reputation for both academic excellence and for being a real meat grinder; typical freshman classes per discipline at the time were 200-250 and graduating classes were 30-40. In second and third year, it was typical for your class to shrink by thirty five students net from semester to semester, with twenty new faces who had failed out of the year ahead.

The administration was definitely Sending A Message with that class: the answer to "they can't possibly fail all of us, can they?" was unquestionably "Oh yes, we can, and will". So the already competitive class coming up behind them became pretty vicious when it turned into a double cohort.

FWIW the situation was a little different from the one at JHU. The Eng Department didn't do any grade curving, because it was considered unfair for the top student to be belled from 98% to 98.25% while someone "surfing the curve" goes from 35% to a C. The only grade adjustment condoned at the time was "sliding": adding the same amount to everyone's grade, usually in the form of giving everyone bonus marks to compensate for a problem with a test question. Professors were also granted the option of holding a make up exam at their discretion. Those students weren't responding to an established curve policy; they were openly challenging the department by walking out of their final exam (absence was already an automatic incomplete), and their gamble didn't pay off.

It's not like they didn't have their share of wacky grading schemes, though (since I'm sharing undergrad stories):

A calculus professor disclosed that grades on late assignments would be modified by (0.9)^N where N was the number of days late. In upper years he got more creative by having your midterm mark affect the decay curve. "How'd you do on that math assignment?" "Man, I can't even figure out the equation for my score."
A stats prof disliked grading his course as 50% midterm (M), 50% final (F). Instead he preferred using sqrt(M*F), making a grade of less than 36% on either test equal a course fail.
Electronic submission for software projects had brutal deadlines. Students basically had to check in their source code to the grading filesystem repository, which would lock up and refuse late submissions even if they were one second late. Check-in early and check-in often, because a failure to submit anything on time means a zero!

These were all disclosed in the syllabus and mentioned on the first day of class, of course.
posted by ceribus peribus at 6:32 PM on February 18, 2013 [8 favorites]

Frankly, it's hard to imagine that nobody in Introduction to Programming forced the issue all the way to the stage of "unless you physically assault me, I'm going into the room."

It's a good thing they taught all those programmers to give away personal advantages when confronted with peer pressure.
posted by kithrater at 6:42 PM on February 18, 2013

it's hard to imagine that nobody in Introduction to Programming forced the issue all the way to the stage of "unless you physically assault me, I'm going into the room."

Assault was unnecessary, only the undestanding that if anyone went in the room everyone else would too, the deal was dissolved, and it was back to the curve.
posted by localroger at 6:48 PM on February 18, 2013

It's a good thing they taught all those programmers to give away personal advantages when confronted with peer pressure.

Plays Well With Others

K101
posted by ChurchHatesTucker at 6:48 PM on February 18, 2013 [1 favorite]

Not that I don't appreciate all the insight provided by the comments in this thread, but it would have been great if we had all stuck to the agreement and refused to comment here.
posted by dogwalker at 6:57 PM on February 18, 2013 [4 favorites]

As lordaych and Mars Saxman point out, this is a perfect example of thinking through edge cases. If it was an interview problem, they'd definitely get points for pointing out the unintended result.

That, and overcoming the prisoner's dilemma inherent in the situation would give them an A if it was my class. Bravo JHU CS students, you're great engineers and have refreshed my faith in the possibility and power of collective bargaining.
posted by formless at 6:58 PM on February 18, 2013

This famously happened once (once!) in one of the engineering disciplines at Waterloo. Except that four students rebelled, causing 85 of their classmates to fail the term and repeat the semester.

This kind of sounds like an urban myth, sorry. Do you have any other info on this? I tried searching and can't find anything.
posted by jacalata at 7:01 PM on February 18, 2013

Yeah, to everybody saying they'd be the one guy who'd go in and take this, I find that kind of hard to believe. Because you'd have to walk past your classmates, who are standing outside to make sure that if one person goes in, they all go in. And so now you're not guaranteed an A at all. I mean, maybe you'll get the highest grade in the class, but maybe a bunch of other students hedged their bets and studied just in case this situation arises.

So now you're looking at a lower grade than you would have received had you just stayed outside the room. But what you have also gained with certainty is the ire of a large number of your classmates. People whom you will be spending some time with; people who will remember you for all of their days. People whom you might be paired with on future assignments in future classes, or people who might some day be in a position to give a job reference.

Anyone who breaks that line is short-sighted, I should think. If they all came to me looking for jobs, I'd rather interview the kid(s) who came up with the idea to boycott, as they're clearly thinking about edge cases, and also have the charisma and diplomacy to convince a number of people to do something that, while technically right, still seems to be very risky from an individual student's POV.
posted by nushustu at 7:04 PM on February 18, 2013 [4 favorites]

Don't underestimate the contempt I had for the large majority of my classmates.
posted by ryanrs at 7:05 PM on February 18, 2013 [1 favorite]

The "real world" these students are preparing to work in is an abstract zone of formal rules and rigid mechanisms. Exploiting edge cases in those mechanisms is a big deal, in both positive and negative ways. I suspect that the professor understands this.

The real world also expects students to have learned basic CS concepts, which apparently they were never tested on. Maybe if this had been "Intro to Game Theory" it would have made sense; as it is, it's an asinine stunt that shows this professor is not taking his own course seriously.
posted by qxntpqbbbqxl at 7:08 PM on February 18, 2013 [1 favorite]

If you understand a set of rules well enough to understand the usefulness of this stunt, you don't need a course in CS. You wouldn't go along unless you know more than the course is meant to teach you.
posted by localroger at 7:11 PM on February 18, 2013

I don't care how crazy the rules are, but if you set a target and I make it, and THEN you decide there's some reason I don't get the reward -- fuck you, I will never be able to work with you again.

So, like, if I run a restaurant, and I've got a sign that says "No shoes, no service," you're gonna be the guy who comes in barefoot holding a pair of shoes, insisting that you're following my instructions so where's your damn dinner?
posted by escabeche at 7:13 PM on February 18, 2013 [1 favorite]

hoyland: ""Mark the exam out of the largest total score" is a fairly common system."

Hard curves like described in the article are idiotic. Either you are getting a significant number of people achieving near a 100% every year in which the curve is meaningless or you aren't and you risk fucking a whole class when you get an exceptional student who shifts the curve by how much they outperform the norm. It sucks to be a B+/A- student stuck in a hard curve graded class with the course's equivalent of Micheal Jordan or Wayne Gretzky.
posted by Mitheral at 7:14 PM on February 18, 2013 [1 favorite]

This kind of sounds like an urban myth, sorry.

Yeah, it does sound that way; I was bracing myself for lots of "we had that myth at my school too" responses. I swear some friends of mine ended up in the cohort class, though. It was over twenty years ago so it's difficult to find anything.
posted by ceribus peribus at 7:18 PM on February 18, 2013

That said, Bell curving grades is crazy. I find my grades often come out bimodal (two peaks), with one cluster of students who did the homework and one cluster who did not.

I remember a Jeff Atwood post from a few years ago about the bi-modal distribution of introductory CS grades: separating programming sheep from programming goats. Which goes a bit further and supposes that some students just can't learn programming, at least in the average undergrad time-frame.

One of the theories proposed is that you have to come to terms with the concept of "meaningless". I don't know if I agree with it, and I definitely don't agree with the basic idea that some students just "can't" learn CS.

But the bi-modal distribution (in incoming students) is real, and applying a normal curve/fit to it just seems like bad stats.
posted by formless at 7:20 PM on February 18, 2013 [2 favorites]

A stats prof disliked grading his course as 50% midterm (M), 50% final (F). Instead he preferred using sqrt(M*F), making a grade of less than 36% on either test equal a course fail.

That's kind of genius and kind of pointless. Most people save would save themselves the effort and require you have to pass all components of the course to pass the course.

(Where I am now, there's a single math course that counts as a 'writing intensive' course. To pass the course you have to have passed the writing assignment portion. There's a panic among students that occurs like clockwork every semester when people finally read the syllabus and realise maybe they had better hand in some work.)

Hard curves like described in the article are idiotic.

Note that the rest of my comment observed I hadn't seen it done in college, likely because it doesn't work in large intro courses where you're all but guaranteed one 100% paper and one 0% paper.
posted by hoyland at 7:21 PM on February 18, 2013

If you don't have a grasp of your students' level of understanding, and give them an exam that is too hard for them to pass, then you probably are not teaching particularly effectively.

What?!?? Exams are not set based on how well the students are thought to have learned the material, they're based on the syllabus. At many universities, in fact, the teaching professor has no say in the examination process whatsoever.

But both positions are true to a certain extent, aren't they? I teach intro to lit analysis, and the first exam is pretty technical (lots of literary terms, some scansion). Students usually ace it. But the one semester where everybody botched the exam? Results that consistently bad told me that I had messed up somewhere, so I chucked the whole thing, we did some more work, and then I tested them again.
posted by thomas j wise at 7:24 PM on February 18, 2013 [1 favorite]

So, like, if I run a restaurant, and I've got a sign that says "No shoes, no service," you're gonna be the guy who comes in barefoot holding a pair of shoes, insisting that you're following my instructions so where's your damn dinner?

The grading methodology was specified a bit more exactly than "no shoes."
posted by localroger at 7:25 PM on February 18, 2013 [2 favorites]

The four students went on to finish the program as a class of four: holding lectures in professor's living rooms, having their pick of internships and grad placement openings, and their graduation class photo was a set of four portraits.

The subsequently overcrowded cohort in the year behind them was pissed.

I'll be honest, I'd be afraid not to take the test because I'd be scared of a vindictive professor, but I'd be far, far more afraid of having pissed off my peers to that extent.
posted by winna at 7:38 PM on February 18, 2013

I can think of one particular Comp Sci course that was clearly the "weed out" class for the department. After sweating bullets through 3/4ths of the semester, the prof was like, "Oh, Bee-Tee-Dubs, this class is graded on a curve and all of the drop-outs will be included in the curve according to their last score before dropping out. Translation: Most of you who are left are actually getting Bs and As."

What that Comp Sci jerk didn't account for was that I was a double major and I was getting a D in another unrelated course because I was putting all my energy into the weed out class. Luckily there was time to pull that one up to a C before the end of the semester, but, yeesh.

Now that the social web is a thing, I hope profs can't pull that stuff off as easily as they used to.
posted by Skwirl at 7:39 PM on February 18, 2013 [1 favorite]

Frankly, it's hard to imagine that nobody in Introduction to Programming forced the issue all the way to the stage of "unless you physically assault me, I'm going into the room." Maybe Johns Hopkins students are nicer than the people I went to school with?

I got the distinct impression that the professor encouraged it.
posted by empath at 7:46 PM on February 18, 2013 [1 favorite]

argh, grading on a curve sucks, this cutesy stunt sucks, everything sucks, where is my vermouth

also you guys at my high school there was a history teacher who made everyone write an in-class essay on the meaning of courage and this one kid wrote "this is" and handed in his paper and he got an A and everyone else got an F
posted by en forme de poire at 7:47 PM on February 18, 2013 [4 favorites]

Hopkins kids are generally pretty nice to each-other because they get exploited by the University hard core. So they stick together. I'm sorry a lot of yall seem to have turned your bad experiences into various kinds of acrimony towards your fellow students.
posted by Potomac Avenue at 7:50 PM on February 18, 2013 [3 favorites]

Yeah, I would have seen this stunt as simply another aspect of the culture of cheating that seemed to permeate all my courses. Maybe that isn't the case at Johns Hopkins.
posted by ryanrs at 7:55 PM on February 18, 2013

"Joke's on you! I instituted the curve to see if you could figure out the bug in the system and exploit it!"

"You passed the test!"
posted by Behemoth at 8:02 PM on February 18, 2013

I remember a Jeff Atwood post from a few years ago about the bi-modal distribution of introductory CS grades: separating programming sheep from programming goats. Which goes a bit further and supposes that some students just can't learn programming, at least in the average undergrad time-frame.

That study just will never die, will it? It got mindshare too quickly. As a programmer, it's appealing to think I've got some special sauce that the plebs don't. But before we programmers get too self-congratulatory, note this subsequent paper by those same authors:

When we began this work we had high hopes that we had found a test that could be used as an admissions filter to reduce the regrettable waste of human effort and enthusiasm caused by high failure rates in universities' first programming courses. We can see from the experiments reported above that our test doesn't work if the intake is already experienced, and in experiment 3 didn't work at all. We cannot claim to be separating the programming goats from the nonprogramming sheep: experiment 3 demolishes the notion that consistent subjects will for the most part learn well, and others for the most part won't. And even in the most encouraging of our results, we find a 50% success rate in those who don't score C0 or CM2.

None the less, some of our results indicate that there may be something going on with consistency. There is a case for continuing our investigations. In the meantime we present this work as a case study in good science. Having some preliminary results that appeared extremely promising, we and colleagues have refined the test instrument, conducted further experiments, and applied the appropriate analysis. It is unfortunate that the outcome does not live up to the initial promise, but it has not quite closed the door on our explorations.

posted by a snickering nuthatch at 8:07 PM on February 18, 2013 [16 favorites]

Jpefed, thanks for posting that update. I was wondering about any further results. I'm sure that an easy admission test would be a dream for any program

In the meantime we present this work as a case study in good science.

Science done right!
posted by Librarygeek at 8:18 PM on February 18, 2013

We had a great professor for Physical Chemistry who understood that many of us simply were not in the game when it came to Calc 2 and quantum theory.

Instead of curving the class, each test would curved (but not homework or surprise quizzes) to the highest grade. Additionally, everyone ran on a sort of individual curve. The final was the first three tests (slightly different questions) and one cumulative BIG question. You could add to your old test score an improvement on the relevant section of the final. So... it was weird and a little confusing and somehow all of the molecular biologists were kept from outright failing.
posted by Slackermagee at 8:25 PM on February 18, 2013

If you don't have a grasp of your students' level of understanding, and give them an exam that is too hard for them to pass, then you probably are not teaching particularly effectively.

You may be teaching them just fine, but not testing effectively.

On my first exam in my freshman year calculus class, the professor set 55 as failing before the test. The class average was a 56.

Now, the professor apologized to us after the test. That was a test writing failure. The test was too long and too difficult. That did not happen with subsequent exams. But they didn't go back and reset the failing criteria either.
posted by maryr at 8:25 PM on February 18, 2013

I'm surprised at all the hate for tests with low average scores. Tests where the mean is low (say around 50) offer you plenty of opportunities to distinguish yourself, allow you to recoup lost ground on subsequent tests (since the ceiling is so much higher), and are often way easier to grade because you're not running up against the boundaries. They are only discouraging if you don't tell people what the mean and stdev was -- I don't know anyone who would be disappointed by getting a 75 on a test where the mean was a 40. In contrast, IME, tests where the mean is high (say around 90) encourage graders to find any old excuse to mark people down just so they can get some kind of spread in the scores.
posted by en forme de poire at 8:41 PM on February 18, 2013 [2 favorites]

Really, the fact that people would actually pull this just pisses me right off. I work hard and go to a prestigious university, and I work my ass off for my A's. And here these assholes are getting A's because they figured out how to get away with not doing work. Bullshit. If the professor thinks the final isn't a good evaluation of their abilities, don't give them a goddam final. Have them do a final project or presentation. At my school, a good half of the classes don't give final exams for that very reason, but instead give papers and projects.
posted by DoubleLune at 8:43 PM on February 18, 2013 [1 favorite]

Tests where the mean is low (say around 50) offer you plenty of opportunities to distinguish yourself

Ha. Haaaa, no. All this says is a) the professor wasn't teaching well and/or b) the exam was badly written.
posted by DoubleLune at 8:45 PM on February 18, 2013

Law students joke about this in hated classes all the time. Everyone writes the same thing or checks the same boxes and everyone gets the forced mean. In theory. Because we're law students, though, the objections are always the same: everyone would expect enough people would double-cross the group to undermine the gambit, and so would feel pressure to do the same (the prisoner's dilemma, which to my eyes does exist here--I could get the A instead of the target mean by answering just one or two more questions correctly), and moreover the administration would be expected in invalidate the results (because despite being able to force a mean they have no way to force a curve) and maybe even reprimand everyone enrolled.
posted by snuffleupagus at 8:54 PM on February 18, 2013

(The difference being that in law school the top paper or exam doesn't set the curve in the same way––the curve is set arbitrarily, via a forced mean and the professor being able to make only slight adjustments within a narrow range of allowable standard deviation.)
posted by snuffleupagus at 9:00 PM on February 18, 2013

DoubleLune, you'd hate my algorithms class.

Somehow I think I'll live.
posted by erniepan at 9:04 PM on February 18, 2013

I dunno, I think it's fair to give problems that are all within the scope of the material that was covered but so hard that only the people who really know what they're doing are likely to do a near-perfect job during the time allotted, as long as you warn people what they're getting into. I mean, general exams in my department are basically a 3-hour cross-examination - nobody comes out feeling like they aced it, even when they pass. I don't think it's a perfect system by any means, but I think it's probably correct that you really have to get right up against the limits of what people know in order to accurately evaluate how they're doing.
posted by en forme de poire at 9:05 PM on February 18, 2013 [1 favorite]

Some excellent professors design exams deliberately to be really hard, so the medium and hardest questions test knowledge at the very edge of the students' abilities. It does not mean they're bad teachers or badly written exams.

Some terrible professors do the exact same thing. I've been on both ends of this, and the difference is that the excellent professors who do this really make an effort to get students to understand the material and are generally clear lecturers so that the majority of the class gets the general material. The bad professors sound like they're lecturing in ancient Nepalese (and are the ones whose slides made people more confused) and the students mainly learn through the textbook. It's not difficult exams I have a problem with, or exams that push you -- it's exams that test beyond what students are being taught, and it's students not learning.
posted by DoubleLune at 9:14 PM on February 18, 2013

I have to say, I did resent assignments and tests where I was asked to re-invent the wheel or figure the coefficient of gravity or some such. I am not Newton nor was meant to be.
I am an attendant clod, one that will do
to fill a seat, raise a hand or two.
Derive with hints, no doubt a queasy tool.
Differentials seem to me obtuse
Hieroglyphic, noxious, and meritless;
Full of high sentence, but of less use;
At times, indeed, almost ridiculous –
Almost, at times, just school.

...I don't know what happened there.
posted by maryr at 9:41 PM on February 18, 2013 [5 favorites]

I don't want to go all "Old man yells at cloud" here, but I have a feeling if this was done in my profs' classes they would have gone full R. Lee Ermey in "Full Metal Jacket" on us. i.e. not one of us would have passed the class.

I gotta give points for the solidarity, though.
posted by InsertNiftyNameHere at 9:58 PM on February 18, 2013 [1 favorite]

I studied first at a university in Ireland first that at the time examined mostly using the tripos system, where a pass or D was 40%, an A equivalent was >= 70%, and everything else was smooshed in between in a pseudo bell-curve. It was generally mostly essay and free-form based exams, usually a single exam per subject at the end of an academic year. So studying a lot of the same material again in US colleges that used grading curves, grade-point averages and regular, periodic multiple-choice ranking and selection exams was interesting. In the US schools, I didn't notice a lot of the beggar-your-neighbour competition people mention here, and given that so much of the grade had accumulated by the final couple of exams of a sequence, there seemed to be less end-period anxiety in general. A lot of the gaming of the tripos system comes from hoarding notes, or useful answers. I did notice a lot more use of stimulants throughout the academic year in US schools, compared with basically end-of-term binges under tripos. I think overall, having experienced both, that from my experience the US system with grade curves and regular grade point accumulation is actually easier and less stressful on students, and tends to create a less competitive class experience with less demoralisation and exam strife overall.
posted by meehawl at 10:40 PM on February 18, 2013

I only had one teacher grade on a curve. It was high school, math for dummies. The teacher handed out a practice exam to work on the week before the final. On the day of, I noticed that out of four pages, only the first one was different--the other three were the same practice exam. I actually went up to ask the teacher if he had somehow screwed up, and he just went "Shhh" at me. Yes, I was the only one to notice.

I got the highest grade in the class...but he said he had to give me an A- because everyone else was so stupid that he had to grade on a curve. Whaaaat?

Back on topic, I am amazed that this guy let them get away with it.
posted by jenfullmoon at 10:50 PM on February 18, 2013

jenfullmoon: "Back on topic, I am amazed that this guy let them get away with it."

It's Intro to CS. The punishment is teaching the class next term.
posted by pwnguin at 10:58 PM on February 18, 2013 [2 favorites]

I gotta give points for the solidarity, though.

I'd give an A to the guy that thought of it, and fail everyone else.
posted by empath at 11:09 PM on February 18, 2013

Maybe Johns Hopkins students are nicer than the people I went to school with?

lol.
posted by charmcityblues at 11:11 PM on February 18, 2013 [1 favorite]

Programmers are rules lawyers. We make an living out of understanding, manipulating, and building hyper-literal rules systems that have no unspoken exceptions and where all possible flaws will be exploited at some point by somebody, intentionally or otherwise.

As a professional software developer I see nothing at all wrong with what they did. If you don't want a given outcome, don't build a ruleset that allows that outcome and then let a bunch of snotty young hacker types attack it.

Most programmers have to work with other people. In fact, many programmers have to work with other people who (gasp!) aren't programmers. I think one of the keys to being a productive and happy programmer is knowing when you're gaming the system innocuously because a stupid rule was made, vs. actually taking advantage of your non-programmer coworkers and employer who didn't think through all the possible loopholes when they wrote the employee handbook. This is especially true working in a business where programming is a support function rather than the main point of the business (e.g. bank or e-commerce website, vs. google or microsoft). Programmers who become contemptuous of their non-programming coworkers get cynical and jaded and mean, and they end up really unhappy even though they find all kinds of ways to take advantage of the system. Programmers who value getting along with their coworkers over taking every possible advantage seem to be a lot happier.

To me, whether these students fall into the first category or the second depends a lot on whether the professor was hoping/expecting/encouraging the class to make such a deal. If the professor truly didn't think through the ramifications of the grading system, then the students are kind of on the jerkface end of the spectrum. But it sounds like this student behavior was an anticipated possible outcome, so the students were only taking advantage of something that was meant for them to take advantage of.

I've got no problem with encouraging that kind of outside-the-box thinking... as long as the students also get tested on what they've learned in some other way. This approach doesn't test whether everyone in the class learned enough to skip the test; it really only takes one student smart enough to notice this loophole and charismatic enough to convince everyone else to play along. I'm all for tests as learning experiences, but at some point you also have to assess how much the individual students learned.
posted by vytae at 11:15 PM on February 18, 2013 [2 favorites]

Some excellent professors design exams deliberately to be really hard, so the medium and hardest questions test knowledge at the very edge of the students' abilities. It does not mean they're bad teachers or badly written exams.

This is great for allowing the top few students to show off their deep grasp of the subject. But having a handful of students who really get the material shouldn't mean that other students who have fulfilled the learning requirements of the course get C's. If you understand the material that was identified in the syllabus as the course objectives, you should get a good grade. If the super-deep level of understanding is required to get an A, the course objectives should be updated to reflect the level of learning that is really expected of the students.

Then again, maybe I'm writing as someone who grew up with grade inflation. I still shudder in horror when someone identifies a C as an "average" grade.
posted by vytae at 11:25 PM on February 18, 2013 [1 favorite]

And the moral of the story is not to grade on a curve, as yet another example of the obsession the supposedly individualistic and libertarian US has with instituting the sort of collectivistic rules that makes even the most hardened stalinist blanch.
posted by MartinWisse at 12:53 AM on February 19, 2013

So we can assume that with this level of publicity that the professor got their way and the school will no longer be forcing the intro to cs course to be graded on a curve?
posted by ambrosen at 1:03 AM on February 19, 2013

Many people here are assuming the students didn't study for the exam and put all their eggs on the longshot of pulling off the boycott. I think that's unrealistic (and uncharitable). And if I were in that class, I would study like crazy, and make it known that if anyone walks into that room, I am extremely prepared and there is every chance I will smoke them in the curve and destroy their grade. I doubt I would be the only one taking that approach either. I think people both studied hard AND organized. They aced that exam and earned the A.
posted by anonymisc at 1:35 AM on February 19, 2013 [3 favorites]

I would have thought though, that we all had to take an exam paper and put our names to it, such that we got a zero instead of an absence. That would complicate the enforcement/trust system, but would still be doable.
posted by anonymisc at 1:42 AM on February 19, 2013

Hard curves like described in the article are idiotic. Either you are getting a significant number of people achieving near a 100% every year in which the curve is meaningless or you aren't and you risk fucking a whole class when you get an exceptional student who shifts the curve by how much they outperform the norm. It sucks to be a B+/A- student stuck in a hard curve graded class with the course's equivalent of Micheal Jordan or Wayne Gretzky.

It sucks even more to be that exceptional student who is stuck in the position of having every single other class member hate them for something that they ought to be able to be proud of. The exceptional student is not the one at fault for the effects of the grading curve on everyone else, but they're the ones who get punished for it.
posted by talitha_kumi at 3:09 AM on February 19, 2013 [6 favorites]

Honestly, whether this sort of stunt is legit to me depends on what's being tested? If that had been a medical exam, flunk the lot of them. But for Computer Science, Psych, or Game Theory they all deserve As for carrying through this highly risky scheme.
posted by Francis at 3:59 AM on February 19, 2013

I've always been not a particularly big fan of summative assessment in teaching. It's an institution, but that doesn't always make sense in CS and certainly not in the software engineering aspects of the discipline (which leaves other problems: what prevents students from cheating in their work via stack overflow or even outsourcing homework). Formative or portfolio assessment is probably better in most cases.

A few memories from my CS program:
1. While taking a final exam in Discrete Structures (a class wherein we treated typical programming language constructs as mathematical elements subject to proof of correctness), I was presented with a question to prove that a REPEAT ... UNTIL expr statement was correct. The exam was not proctored, so I set aside my work and walked to the professor's office and said, "look - I know why you're asking this question but I can show that this is equivalent to a WHILE expr DO ... loop, which we proved in class and reduce to a problem already solved. Can I do that?" "Sure, but you need to also supply the WHILE proof. Nice try."

He knew that if I was working on that question this early that I'd already slam-dunked the exam, but he wanted me to jump through the hoop. Oh well.

In my CS theory class, the professor selected one of us as a grader (she was a double major, CS and Math and the CS theory class was easy for her), so he grader her work then she graded our work. She was not graded on our curve (which was another 8 students or so). At the final exam, he walked in and gave us each individual brainteaser bar puzzles to solve. After about 5 minutes he announced that the venue was not right and wouldn't we rather take our exam over a few drinks at the local watering hole?

That, my friends, is how you give an exam.
posted by plinth at 4:52 AM on February 19, 2013 [1 favorite]

If you are studying CS at Johns Hopkins you are not the sort of person who wouldn't have done the studying.
posted by Pope Guilty at 6:27 AM on February 19, 2013

But for Computer Science, Psych, or Game Theory they all deserve As for carrying through this highly risky scheme.

As someone who is patiently waiting for the next code deployment and hoping hoping hoping that huge bug is fixed so all the work we did this weekend wasn't for naught, I don't like the idea of computer scientists being 'risky' as a job requirement.
posted by sweetkid at 6:43 AM on February 19, 2013 [1 favorite]

People who are openly fantasizing in this thread how much they'd enjoy taking the test and fucking up everyone else's grade sound like bitter children.

"You guys think you're so smart?! Well, I'd be even smarter than you! I'd show you for making me feel dumb!!"
posted by Uther Bentrazor at 6:47 AM on February 19, 2013 [1 favorite]

People who are openly fantasizing in this thread how much they'd enjoy taking the test and fucking up everyone else's grade...

Should have to take a Hopkins CS exam.
posted by ChurchHatesTucker at 6:55 AM on February 19, 2013

As a college teacher, if my students showed as much creativity on their design assignments as they do trying to weasel a higher grade for less work, they'd all get an A anyway.
posted by Mcable at 6:58 AM on February 19, 2013 [5 favorites]

This is a joke, right? I mean to say, what locked the prof into this arrangement? What prevents him from saying at the entrance to the classroom, don't take the test, automatic fail?

Exactly. And it would be an automatic fail, because there's a difference between missing every question completely on a final, earning a 0, and not submitting a final at all, which, in my book, rates an incomplete at best, and a F for abandoning the class after the last day of free drop period. And I'll bet in university regulations, there's something that will let the professor do exactly that, and to be honest, I think that's exactly what should be done.

Translate this into the real world. Your declares that your raises will be based on lines-of-code that pass QA, graded on a curve. You all decide to submit 0 LOC. And, on the day when you're all expecting the max raise, since you all did identical work, you are all fired since you haven't done any work.

If they had all entered the exam room, and turned in an answer book with their name on it and nothing else, then I'd say that they all took the exam and got 0, so, by the curve, they'd all get an A. They would have documented proof that they took the exam. They all sucked -- equally, so they all get the max grade. And if some did that, and others simply didn't show up and turn in an exam, those who turned in an answer book would get an A, and the others, who failed to take the final, would be given an F for not completing required work (namely, actually taking the final.)

But what they all did was skip the exam. Fail. Literally, Fail the course. Unless they have an acceptable excuse like documented illness, in which case, Incomplete for that person, and Fail for everyone else.

See, if you're going to play rules lawyer, step one is knowing *all* the rules.
posted by eriko at 7:19 AM on February 19, 2013 [5 favorites]

See, if you're going to play rules lawyer, step one is knowing *all* the rules.

Since they got the A's they obviously *did* know the rules. It seems very likely that the instructor deliberately set up the possiblity to see if a class could pull it off, and he will probably be cackling madly and whooping about it for the rest of his life.
posted by localroger at 8:19 AM on February 19, 2013

People lauding these students for their rules exploitation to get the desired outcome should take note of the fact that they did not achieve the desired outcome. As a Computer Science graduate, the desired outcome is not "I got the same high grade as everyone else," which tells future employers "I am indistinguishable from any of my classmates." The desired outcome is "I got a higher grade than all my classmates, thus you must select me."

They may have gotten an A on that test, but they failed to help themselves (in multiple ways, I would argue).
posted by jermsplan at 8:21 AM on February 19, 2013

But having a handful of students who really get the material shouldn't mean that other students who have fulfilled the learning requirements of the course get C's.

Well, you don't have to make the mean a C - with modern grade compression "fulfilled the learning requirements" without any special distinction is probably a B, so that's what you would curve around instead.
posted by en forme de poire at 8:32 AM on February 19, 2013

It makes absolutely no sense to curve around the highest grade though. It's like you're trying to maximize the effects of outliers.
posted by en forme de poire at 8:51 AM on February 19, 2013 [1 favorite]

Well, not to brag, but Waterloo's engineering program already had a reputation for both academic excellence and for being a real meat grinder; typical freshman classes per discipline at the time were 200-250 and graduating classes were 30-40.

Unless the workload in the UW undergrad engineering program accurately reflects the workload that would be required in the real world - in which case, those who couldn't hack the program wouldn't be able to handle an engineering job - this strikes me as an enormous waste of human potential.

I seem to recall that the Waterloo engineering program only accepts people with exceptional grades and abilities - why force these people to bash their heads against a brick wall? Engineering school isn't Marine boot camp.
posted by tallmiddleagedgeek at 8:56 AM on February 19, 2013

It's not uncommon for the sole grading criterion in graduate courses to be "Is this person enrolled in the course?" If so, you get an A. You could be dead and, as long as the university hasn't noticed and cancelled your registration, you get an A.

I don't know what discipline you're in or which school you might be referencing, but that's most definitely not true of mine.

I've used the grading scheme described (incorrectly labeled a "curve"). I use it on the midterm, with no guarantee I'll use it on the final. Since my small non-random sample of the student body is unlikely to represent a normal distribution, and it's equally unlikely that nobody in the class is the kind of student likely to get an A, I assume that whatever the high grade is was the most that could be learned. Also I don't tell them ahead of time, precisely to avoid the outcome of the article. To date, I've never had to scale more than a few percentage points, and it affects nobody's final grade (since their grade is based on more than just two tests), but it does let them get a feel for my teaching and exam style without feeling punished. Well, too much, anyway. They seem to be capable of working themselves into a frenzy over assignments worth all of 5% of their final grade, so there's no accounting.
posted by CoureurDubois at 8:59 AM on February 19, 2013

If you people hadn't wrecked it, I would have infinity favorites by now.
posted by Zed at 4:49 PM on February 19, 2013

You've got none. Same thing :P
posted by ChurchHatesTucker at 5:38 PM on February 19, 2013

« Older Scarfolk: One visit is not enough | Artistic Portraiture Newer »

This thread has been archived and is closed to new comments

MetaFilter

Get an A by exploiting a loophole in the grading curve
February 18, 2013 3:49 PM Subscribe

Tags

Share

Get an A by exploiting a loophole in the grading curve February 18, 2013 3:49 PM Subscribe

Tags

Share

Get an A by exploiting a loophole in the grading curve
February 18, 2013 3:49 PM Subscribe