Join 3,494 readers in helping fund MetaFilter (Hide)


Michelle Rhee's "Reign of Error"
April 14, 2013 1:49 PM   Subscribe

DCPS Chancellor Michelle Rhee oversaw radical reforms to Washington, DC's failing public schools. Amongst the results were widespread irregularities on standardized tests that suggest they were tampered with by adults.

During her unpopular 2007-2010 tenure as chancellor of D.C. Public Schools, Michelle Rhee (previously) enacted a series of controversial and radical reforms that tied teacher pay and job security to students' standardized test scores. Her policies both awarded high-achieving schools and punished poor ones; a single day in 2010 saw the dismissal of 165 DCPS teachers on the basis of test scores. Rhee resigned later that year after Adrian Fenty -- the mayor who had appointed her in 2007 despite her lack of experience -- lost reelection in what was widely construed as a referendum on DC education reform.

Test-score based policies have been linked to cheating by teachers and administrators in other communities (just two weeks ago, a grand jury indicted 35 school administrators and teachers in Atlanta for alleged standardized test cheating), and DC is no exception. Not long after Rhee's reforms were implemented, DC standardized test scores improved... and with the improvement came a rash of suspicious "wrong to right" (WTR) erasures on the standardized tests. In 2011, scandal broke when USA Today investigated a pattern of abnormally high DCPS WTR erasures between 2008--2010. The cheating continues: on Friday, the Washington Post reported 11 DCPS schools showing evidence of test-tampering in 2012.

Although Rhee denied awareness of the cheating at the time, a memo acquired and reported last week by veteran PBS/NPR education reporter John Merrow reveals that Rhee was aware of the issue as early as 2009. Merrow's documentary The Education of Michelle Rhee was broadcast on Frontline in January and is watchable on-line.
posted by Westringia F. (72 comments total) 30 users marked this as a favorite

 
Regardless of whether these kind of testing-based policies are effective or not, I find it really strange that every account of teacher cheating I've read seems to blame the whole thing on an unfair system. Would we say that companies with GPA cutoffs for applicants are "linked" to student cheating on exams? Or would we just say that students who cheat lack morals, and even that they're representative of a spoiled/entitled/whatever generation?
posted by Ralston McTodd at 2:03 PM on April 14, 2013 [6 favorites]


High stakes evaluations for teachers with little support are one of the reasons I don't teach anymore; DCPS burned me out. I'm not at all surprised this happened and, while I absolutely don't think it's okay, I also recognize that it's easy to underestimate how people, even well-meaning people, will react in unbelievably stressful situations, especially when the issue is not only your own job but the well-being of the actual school and funding for the students.

Unless you've been there, it's hard to understand how much pressure there is on teachers in DCPS and how hard it is to go into work every day in those circumstances, even when you work really, REALLY hard.
posted by Mrs. Pterodactyl at 2:03 PM on April 14, 2013 [48 favorites]


Would we say that companies with GPA cutoffs for applicants are "linked" to student cheating on exams? Or would we just say that students who cheat lack morals

I hope we would, instead of making that choice, keep firmly in mind that most things have multiple causal arrows pointing their way.
posted by escabeche at 2:07 PM on April 14, 2013 [12 favorites]


I'm the son of two teachers and the brother of another teacher, and I've heard the horror stories about standardized testing, teacher accountability standards gone haywire, and public school-killing "reform" measures. There is absolutely nothing that would make me happier than to see Michelle Rhee finally called out as the charlatan that she is. Thanks Westringia!
posted by Strange Interlude at 2:09 PM on April 14, 2013 [6 favorites]


I am VERY surprised Rhee was involved in this. She's the last person I would expect, which sort of makes me like the crazy homeschoolers more.
posted by roomthreeseventeen at 2:11 PM on April 14, 2013 [1 favorite]


So far as I've read, only 18 classrooms have been implicated in the cheating scandal: that's 18 teachers out of 4000-some. Is that really widespread?
posted by anotherpanacea at 2:12 PM on April 14, 2013 [1 favorite]


Considering how we treat corporations doing similar things (ie "fudging" on public reports), I would say that market is working as intended.
posted by GenjiandProust at 2:12 PM on April 14, 2013 [6 favorites]


I think this previously on Metafilter should be related.
posted by roomthreeseventeen at 2:14 PM on April 14, 2013 [1 favorite]


I am VERY surprised Rhee was involved in this.

I'm really not; in general, many of my issues with Michelle Rhee were not with her ideas but in her belief that she was right and everyone else should get the hell out of her way. She never felt a need to consult other stakeholders like, for example, parents/guardians, teachers, or students, and she has demonstrated that she is willing to justify her decisions in retrospect. In addition, she made it very clear during the Fenty/Gray campaign that she believed that she was the savior of the city's schools and that putting anyone else in charge was basically immoral. Considering all this, I think it's very plausible that she knew and decided that any damage to her DCPS legacy and her belief that testing/evaluation is the answer to everything was unconscionable and that by covering up irregularities she was nobly protecting a system that was designed to improve the schools at any cost.
posted by Mrs. Pterodactyl at 2:17 PM on April 14, 2013 [28 favorites]


OK, that's perhaps a bit flip. On the other hand, it's pretty clear that high-stakes testing is not intended to improve education. It seems pretty much designed to get rid of teachers, especially in public schools that don't have the luxury of refusing or washing out underperforming students. If the system was actually designed to help students, the tests would be used, not towards graduation, but to identify students who needed specific help in specific areas and give them the kind of support they need to improve. The entire system, though, is based on punishing schools, not students (although they get plenty of indirect punishment).

I'm really not; in general, many of my issues with Michelle Rhee were not with her ideas but in her belief that she was right and everyone else should get the hell out of her way.

This seems to be a bit of an occupational hazard for the heads of school systems.
posted by GenjiandProust at 2:20 PM on April 14, 2013 [8 favorites]


What Mrs. Pterodactyl said. If anything, Rhee (and other zealots of her ilk) should be the FIRST person we should suspect when it comes to things like this.
posted by Strange Interlude at 2:20 PM on April 14, 2013 [5 favorites]


We are a seriously considering homeschooling, something we definitely did not want to do, because of what we hear from other parents....some of them teachers...about how crazy testing gets starting about third grade in our state.

I just want my kid to learn, and I know testing can help when it's used right, but classes just seem more and more like nothing but test prep. It feels like a defeat, though. Homeschooling is just a stop gap that only works for a few; I still want a better system. I am baffled as to how to get there, and even reading the heated education threads on Metafilter only adds to my confusion.
posted by emjaybee at 2:22 PM on April 14, 2013 [2 favorites]


Starting in 3rd grade in just about every state.
But don't worry. That will change.
There is currently a push to assess K-2 also.
posted by Seamus at 2:26 PM on April 14, 2013 [1 favorite]


Yeah sorry this is my last comment for a while but last year I taught second grade and my kids were tested. The problem is, the actual act of taking a standardized test is cognitively more difficult for second graders than what they are actually being asked to do on the test itself. If you're going to go on about best practices and research and science and data that's all great but you actually have to pay attention to what it says, you can't just keep testing more kids because you've decided that research says testing is good. Also, testing happens around now but you're tested on material for the whole year; I followed the DCPS scope and sequence for second grade (what you're supposed to teach and when) and then it turns out there was stuff on the test I hadn't covered because I was following a published DCPS document that told me to teach that stuff later. I didn't prepare my students for that material which is unfair to them because it's stressful and scary to take a test on something you haven't seen before, especially when you're seven, and unfair to me because I was totally willing to teach that stuff, I was just doing what I thought I was supposed to based on what I'd been given to guide me by the District itself.

Not only are the results for second graders basically useless but it was super stressful for them, a huge pain for me, resulted in way less teaching time, affected the younger grades as well because they couldn't have recess during testing, and it also means that after testing it's really easy to slow down on teaching. After testing ended I took my class outside for math every afternoon. This actually worked really well and they loved it (and so did I) but it would have been just as easy to decide we were having playtime since you can just sort of assume that anything they learn after testing isn't going to count.
posted by Mrs. Pterodactyl at 2:35 PM on April 14, 2013 [30 favorites]


Mrs. Pterodactyl: "She never felt a need to consult other stakeholders like, for example, parents/guardians, teachers, or students,"

It's understandable why she decided to circumvent that process. The "debate" over education reform in DC is practically a textbook example of concern trolling.

Furthermore, the whole idea of "respect" in DC politics, where all of the established political players feel the need to be consulted on every...single...issue is completely strangling our political process.
posted by schmod at 2:36 PM on April 14, 2013 [1 favorite]


Homeschooling is just a stop gap that only works for a few; I still want a better system

There are probably more choices out there w/r/t homeschooling than you'd expect: tutorials, hybrid schools, co-ops and the like. You can get off-the-shelf curricula covering an entire year of a subject, or you can roll your own completely. We've been homeschooling in various ways for about nine years now and have made use of all of them.

One of our kids just flat didn't do well with us as teachers and parents. She goes to a Catholic grade school now and thoroughly enjoys it. As the other kids get into the middle- and high-school grades, we will move them into tutorials/hybrid schools (classes meet once or twice per week, homework and assignments are done on the other days).

My takeaway thus far: there is no "best way" or method. Kids are different and learn in different ways.
posted by jquinby at 2:43 PM on April 14, 2013 [6 favorites]


It sent clear she was involved in the actual cheating. It does appear that she didn't act with her usual direct and agressive style when provided with evidence that someone was fucking up. I'm going to guess that she didn't want anything to interfere with her personal myth.
posted by humanfont at 2:50 PM on April 14, 2013 [1 favorite]


Or would we just say that students who cheat lack morals, and even that they're representative of a spoiled/entitled/whatever generation?

No. I blame school principals who cheat on tests by systematically erasing and correcting student answers to raise overall scores for their schools, the "wrong-to-right" practice described in several of the linked articles. Students are innocent victims here.

I'm even more likely to blame administrators like Rhee who have set up this counterproductive system. Bad incentives lead to perverse results. This is what you get for LARPing the fourth season of The Wire.
posted by justsomebodythatyouusedtoknow at 2:51 PM on April 14, 2013 [5 favorites]


She did get a lot of milage out of misleading a lot of people about her test scores when she was a teacher.
posted by peeedro at 2:53 PM on April 14, 2013 [1 favorite]


You better cheat cheat, no reason to play fair
Cheat, cheat or don't get anywhere
Cheat, cheat if you can't win...

posted by BitterOldPunk at 3:03 PM on April 14, 2013 [2 favorites]


Here's one relatively result of the push for high-stakes testing:
New York is requiring most testing to go online next year and the following year. Districts ,which are trying to budget ahead, still do not know what programs, software or computers to buy. While next year is a long way away for most of us, it's just around the corner for school districts.I don't know what provisions will be made for kids who aren't as proficient, or how stable the servers will be when a gazillion third graders log on at the same time from around the state. The test has become more important than the learning, if parents are to be believed.
posted by etaoin at 3:13 PM on April 14, 2013 [1 favorite]


And in the NY Times today:
Tests So New They Outpace the Lesson Plan
posted by etaoin at 3:15 PM on April 14, 2013 [1 favorite]


On the other hand, it's pretty clear that high-stakes testing is not intended to improve education. It seems pretty much designed to get rid of teachers, especially in public schools that don't have the luxury of refusing or washing out underperforming students. If the system was actually designed to help students, the tests would be used, not towards graduation, but to identify students who needed specific help in specific areas and give them the kind of support they need to improve.

I teach remedial math exclusively to students whose standardized test scores and grades are a cause for concern, giving them the sort of help you advocate. I don't share the cynicism about standardized testing, though I initially did before taking the job, before I started studying the ~200 page technical report that accompanies my state's test results, and before I committed myself to delivering good instruction based on higher order questions and frequent formative assessment as opposed to "item teaching" even though my position is nominally one of "test prep." My conclusion is that good instruction gets good results on those tests, and that the tests are indeed used to help students.

Unfortunately, they're also used to punish teachers who don't deserve to be punished, but the new methods being used to analyze student data (i.e. growth models using student growth percentiles) as the common core standards and PARCC assessments are being implemented do a much better job of showing what's going on, while the new evaluation system that's being implemented in my state (NJ) encourages teachers to learn from the best of their colleagues in ways that we weren't really encouraged to do before.

I think that as education gets better at using data, teachers who a few years ago would have found themselves under the gun for low test scores as measured in terms of strict pass/fail numbers are going to find themselves vindicated for the good work they are doing provided the data shows their students are progressing.

As for Michelle Rhee, the only background I have on her is reading The Bee Eater, which was not exactly the hardest hitting account of what happened in DC. But she did strike me as committed to focusing the conversation about what happens in education on what's best for kids, and I agree with her that the profession has been too adult-centered.

I find the argument that testing incentivizes cheating a lot less compelling than the case that the lack of it incentivizes stagnation. There may well be a way to provide accountability without standardized testing, but there's also a way to get decent results on tests without resorting to cheating. And the system that's going to be in place within the coming years is a vast improvement over what was in place under NCLB.
posted by alphanerd at 3:30 PM on April 14, 2013 [11 favorites]


Campbell's Law: "The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor."

David Berliner on student testing and Campbell's Law

Goodhart's Law "Once a social or economic indicator or other surrogate measure is made a target for the purpose of conducting social or economic policy, then it will lose the information content that would qualify it to play such a role."

"When a measure becomes a target, it ceases to be a good measure." - Marilyn Strathern

What with Compstat and now the testing scandals, you would think that we would have understood this by now.
posted by longdaysjourney at 3:33 PM on April 14, 2013 [29 favorites]


As a sergeant I worked with once said: "Teaching to the test only teaches the test."
posted by Etrigan at 3:36 PM on April 14, 2013 [2 favorites]


And in the NY Times today: Tests So New They Outpace the Lesson Plan

From that article: Statistically speaking, city officials said, people should not worry too much about falling marks because everyone is taking the same new tests. Schools, students and teachers will be judged against one another.

I'm not really sure I believe that. For a start, it's not just going to drop everyone's score by 20 points or whatever--the top students will spread out. But, is anyone going to be surprised when kids from populations that do disproportionately poorly on standardised tests take a disproportionate hit on this one? Yes, theoretically, in a perfect world of unbiased tests, everyone but the top students will take the same hit, but I don't know that we should expect that to be the case.
posted by hoyland at 3:36 PM on April 14, 2013 [1 favorite]


I'm no expert on this but something about the way this is framed makes me a little uneasy.

Its the assumption that teachers can do no wrong.

So we have:

1) Michelle Rhee imposes a high-stakes testing regime
OK, I can see how that pretty hardcore. But some teachers are crappy, right? Surely everyone agrees that its possible for a crappy teacher to exist? Its more a question of what proportion of them are crappy and whether a drastic programme like this is the right approach to reduce any crappyness?
Well, OK, on this point I could go with the arguments that the whole approach is counterproductive. But you've got to bear in mind that any group of people will always be resistant to their performance being measured.

2) Turns out someone employed in the schools - teachers or admins - have been adjusting the tests to increase scores.
Well, OK the gist of the article is that this could have been anticipated and Rhee could have investigated this harder or sooner or in more depth. Yes, I guess there's a case to answer there.
But are the teachers still beyond reproach in this scenario? Or are we taking a behaviourist position here and saying 'well that was bound to happen, shouldn't have set tests to begin with?'

In summary: I'm a little uncomfortable when an initiative to measure a professions performance is being shouted down so hard that even apparent outright cheating by the professionals is still blamed on someone else. Thats how the John Merrow article reads to me.
posted by memebake at 3:41 PM on April 14, 2013 [4 favorites]



In summary: I'm a little uncomfortable when an initiative to measure a professions performance is being shouted down so hard that even apparent outright cheating by the professionals is still blamed on someone else. Thats how the John Merrow article reads to me.


You see this sort of thing in the business world all the time: "executive demands rapid improvement in quarterly metric (profit, sales, etc.) and promises to fire anyone who is underperforming and big bonuses for those who overperform." At the end of the quarter: a miracle improvement! executive is promoted, praised to the heavens (even the president mentions her glowingly) but then, some time later, the accountants start digging into the numbers and discover, surprise, the numbers have been fudged by individuals under the supervision of the executive. Now, was it just a few bad apples or did the executive cultivate an environment where this was going to happen. Then, we discover a memo where the executive to informed in no uncertain terms about what's going on...

The question is whether you think Ken Lay is a martyr for corporate governance or not.
posted by ennui.bz at 4:06 PM on April 14, 2013 [13 favorites]


This says way more than I ever could about the validity of standardized testing.
posted by philip-random at 4:10 PM on April 14, 2013 [6 favorites]


#EraseToTheTop
posted by localroger at 4:11 PM on April 14, 2013 [6 favorites]


Michelle Rhee needs to watch Glengarry Glen Ross. Or maybe she did, but thought Alec Baldwin was the hero.
posted by benzenedream at 4:27 PM on April 14, 2013 [7 favorites]


I find the argument that testing incentivizes cheating a lot less compelling than the case that the lack of it incentivizes stagnation. There may well be a way to provide accountability without standardized testing, but there's also a way to get decent results on tests without resorting to cheating. And the system that's going to be in place within the coming years is a vast improvement over what was in place under NCLB.

I'm actually not defending cheating by teachers nor do I think that they should be let off the hook. On the other hand, pretending that high-stakes testing won't drive some people toward cheating as a predictable outcome of the system is... well, not looking at the whole system.

And I am glad you are providing remedial math. That's great, and there should be more of it. Do you work at a public school, though? In a poor area? Because, unless every child has access to "booster courses" for the areas that they are weak, the system isn't really working.
posted by GenjiandProust at 4:29 PM on April 14, 2013 [1 favorite]


Or would we just say that students who cheat lack morals?

You know that student cheating has nothing to do with this issue right?
posted by freebird at 4:29 PM on April 14, 2013 [11 favorites]


Pessimism: you just aren't going to get much performance out of schools, in a country where most people fundamentally do not care about learning. To improve the prison, we need a better class of inmates.
posted by thelonius at 4:30 PM on April 14, 2013 [1 favorite]


philip-random: This says way more than I ever could about the validity of standardized testing.

But with the right statistical approach, that test could be made to work. I'm not joking. Instead of measuring the absolute result, you measure the improvement over time, and possibly cross reference by number of legs or weight or whatever factors are considered important. With enough animals, enough teachers, and enough tests, you'd then get to find out who was good at teaching animals to climb trees and who was crappy at it.

Then, by focussing on improving the skills of the teachers, over time you'd be able to help more and more animals climb trees better. (NB: Hopefully this is all happening in a universe where tree climbing will be useful for the animals, i.e. the test bears some resemblance to something beneficial in their later lives).

Please dont read this comment without looking at that cartoon philip linked or you'll all think I'm bonkers
posted by memebake at 4:31 PM on April 14, 2013 [5 favorites]


But some teachers are crappy, right? Surely everyone agrees that its possible for a crappy teacher to exist?

This is actually a point that I think it's hard for people outside education to understand. Most teachers enter the profession with less than one year of classroom experience. That is, they are a full on-your-own teacher with usually a year of classes and part of a year of student-teaching. Many professions have a graduated increase in responsibility. When you become a teacher, well, you're a teacher. Good luck. Everyone is a crappy teacher the first year they are a teacher. Most are crappy for several years.

That does not mean we should kick everyone out. It may mean that we need to rethink teacher preparation (plumbers usually have 4-5 years of apprenticeship - is teaching really 4-5 times easier to master than plumbing?), but it doesn't mean that most teachers deserve to be shamed, starved, or kicked out. It takes years or decades, to become a truly good teacher, and years or decades of professional development (that's training).

Sure, there are probably people out there who are teaching who are inherently bad teachers, but anecdotally, I'd say most people who really should not be teaching drop out of teaching - teaching isn't any easier when you're a bad teacher, in fact it's way harder.

We know a fair amount about what master teachers do and do not do. It usually makes more sense to evaluate teachers based on what we know good teaching looks like (or does not look like), and couple that evaluation with substantive support and development - unfortunately that kind of support is way outside the budget of school districts. The district I work in spends a comparatively large amount of money on education, and is supposed to provide a coach for 2.5 hrs/week for new teachers, but in practice many new teachers get about 1 hr/week because there aren't enough coaches, and in-house support for teachers is often sporadic or nonexistent.

Standardized testing has many unintended negative consequences for students and teachers, and in most cases, bad teachers need administrative support and professional development so they can become better teachers, not a pay-cut or a pink slip.
posted by Salvor Hardin at 4:41 PM on April 14, 2013 [31 favorites]


If the "stakeholders" knew what was wrong with the system, wouldn't they already be doing whatever was necessary to correct it? I saw lots of angry parents when Rhee was appointed, but the thing they seemed to be most angry about was that they weren't being listened to. I heard a lot of anger about everything, but I heard very very little about what positive steps should be taken to improve things. And my general response to that, irrespective of what Rhee was doing (which I didn't agree with anyway), was, "This isn't about you. Get over yourselves."
posted by 1adam12 at 4:45 PM on April 14, 2013 [1 favorite]


Testing is not the problem. Standardized testing isn't even the problem.

Standardized testing that is not very specifically designed to work within a curriculum and that isn't used to focus on student-based ability is the problem.

You want to evaluate a student? Look at all evaluative material from their history, current scores, and coursework.

Want to evaluate a teacher? Look at their credentials, observe their classroom, and have them put together a portfolio.

Want to evaluate a school? Observe a principal, survey the staff, parents, and student body, and talk about program implementation.

The thing I can never figure out is why would use something to evaluate one group with the metrics and tools made for another group.
posted by Lord Chancellor at 4:47 PM on April 14, 2013 [6 favorites]


And of course, teachers should not cheat on tests. But if you are a bad teacher (remember, one of the ones who need support and development to become good teachers), and your next rent payment depends on your students' standardized test result, and you work 10 hours every day in a kafkaesque netherworld where you are expected to corral a crowded room of unhappy, angry, disengaged students through learning goals that are out of reach for many socially-promoted students who do not have the skills they need to achieve those goals, with little or no administrative support for discipline* or teacher development, and you cheat on behalf of your students on a standardized test...

...well, I'll shake my finger at you, but they're going to have to string up that cheating teacher without my help, I don't have the heart.


*discipline is another interesting story of a measure becoming a target and therefore no longer being a useful measure - schools are often evaluated based on number of referrals/suspensions/expulsions, so administrators go out of their way to avoid disciplining poorly-behaved students. Which means teachers have little realistic way of maintaining order in their classrooms. But hey, suspensions were down 10% this year. Yay teaching.
posted by Salvor Hardin at 4:49 PM on April 14, 2013 [13 favorites]


We have a weird perspective on teachers. On the one hand, some have this indignant reaction to the firing of bad teachers as memebake indicates. At the same time, others look at teachers and say "your job is to be a miracle worker and I expect nothing less". These extreme positions kind of displace the role and responsibility of students in their own learning, and the parents for creating an external environment where learning can take place, and the broader community for creating structures that support the parents. It becomes solely about the teachers who are looked upon as miracle workers operating in a cultural vacuum. In the Frontline piece, Rhee actually makes this point. She doesn't think any of the external factors matter (because she was just so gosh darn brilliant when she was teaching in the inner city). And within this vacuum we develop simple minded rubrics to assess the very messy business that is teaching. We are giving teachers a huge amount of responsibility, taking away all of their authority and autonomy, and paying them (new teachers) like crap. I don't know much about management. But I know that model will not work.
posted by Seymour Zamboni at 4:53 PM on April 14, 2013 [6 favorites]


Do you work at a public school, though? In a poor area? Because, unless every child has access to "booster courses" for the areas that they are weak, the system isn't really working.

Admittedly I work in a public school in an affluent town and teachers working in an urban setting have it much harder than I do. But I think my position is one that is widespread, and in fact may be mandated by NCLB. At the very least, it's mandated that schools take steps to bring students up to grade level besides firing teachers.

You see this sort of thing in the business world all the time: "executive demands rapid improvement in quarterly metric (profit, sales, etc.) and promises to fire anyone who is underperforming and big bonuses for those who overperform."

In business, that rapid improvement figure may be something that the executive pulled out of his ass, but when it comes to testing, the process that determines what constitutes proficiency is actually pretty sensible and democratic. This was surprising to me before looking into how testing works, because I'd really only been hearing what was "conversationally true" among teachers before reading my state test's technical report, but you can read about it on page 59 of the 2009 edition of my state's report (subsequent editions don't describe the procedure and point back to this one) if you are interested. Also, the "panelists" they mention are public school teachers. (One of them is a colleague of mine.)

As far as the NCLB mandate that 100% of students REACH proficiency by 2014, however, well... I think that figure has more than a faint aroma of fecal matter. Under the new system of student growth percentiles, the targets for growth should be achievable for most educators and that fact should be provable or disprovable using data, and a lot of teachers would do well to adopt a more constructive mindset toward this information.

Also, memebake gives a very good description of what is actually going to be done with data in the coming years.

Also, to further party-poop that cartoon, nobody expects that penguin to be able to climb a tree, and that penguin will still be in awesome shape if he can't. But assessments, especially the new generation of PARCC ones, are designed to predict readiness for college and careers. So if the people who designed it and its accompanying curriculum are correct, there aren't a whole lot of other clear, attainable routes to independent adult life for kids who can't "climb the tree."
posted by alphanerd at 5:29 PM on April 14, 2013


Michelle Rhee is committed to focussing the conversation on Michelle Rhee. Nothing more.
posted by jindc at 5:37 PM on April 14, 2013 [4 favorites]


Wait, USA Today does reporting?
posted by maryr at 5:42 PM on April 14, 2013


Regardless of the original well intentioned idea of standardized testing, the reality is that it is now just used by politicians as a mechanism for busting teacher unions and funneling money into the private sector via charter schools.
posted by Grumpy old geek at 5:43 PM on April 14, 2013 [6 favorites]


So Michelle Rhee is like Blake in Glengarry Glen Ross: third place and you're fired. Seriously, if you base someone's career and livelihood on student test scores, something for which they are only partly responsible, behind parents, the entire education history of the student, community support, school administration... fuck yeah, I support cheating for any teacher who conscientiously does a good job. The babysitter teachers are another matter.

Long ago, I discovered a rule of bureaucracy: if you shake the tree hard to get rid of the rotten apples, the rotten apples hang on - that's what they do well, that's all they do well. You will get rid of the good apples, though.
posted by dances_with_sneetches at 5:52 PM on April 14, 2013 [17 favorites]


This kind of thing is exactly why I'm really suspicious of metric driven approaches to evaluating anything... These days everyone games the metrics and plays the refs.

Another example: A few years back, people started noticing our state's 4th grade reading scores were slipping a lot. So the Republican governor at the time announced a sweeping new reading initiative, with all sorts of facets and new money, etc. But none of it really mattered. The only real motive behind the proposal was to score cheap political points to help smooth the way for his future political ambitions. Why do I think that? Because one crucial thing the new legislation did was this: it established a new reading pretest in the third grade and then required that any third graders not scoring well enough to pass before they got to 4th grade would be retained, so no students who couldn't already pass the 4th grade test were allowed to advance to 4th grade.

Predictably, the 4th grade reading scores ticked up dramatically, and the improvement was later touted as evidence of how much reading improvement there had been under this particular governor's administration. The problem is not with the kids or even the teachers. The problem is the reality-distortion fields around our leaders and administrators.
posted by saulgoodman at 5:59 PM on April 14, 2013 [9 favorites]


This woman was on the Daily Show less than three months ago.
posted by JHarris at 6:07 PM on April 14, 2013


Anecdotally, I think it's true that it takes time to become a great teacher. Most of the great teachers I ever had were older people, the youngest ones were probably in their mid fourties, up to ones nearing retirement.

But there was also a pretty big difference between the young teachers. Some who had just a year or two of experience were already pretty good, you could tell they were going to be great.

And there were teachers in their fourties or fifties who were just plain horrible, and had plenty of experience. They also had a reputation for being horrible, older kids who had gone to the same school told tales of them, and they just plain didn't improve.

This was not in the US, but even in social democratic Norway, there were very little students and their parents could do when they got stuck with these bad teachers. I never had the bad luck to get any of them as a main teacher (the one that stays with you throughout several years), so it was mostly a matter of toughing it out a year, but it was still frustrating. There were people who had near-perfect grades in every class except the ones taught by these specific teachers.

So yeah, I think some people should probably not be teachers. How to figure that out, though, seems to be a tricky question.
posted by Joakim Ziegler at 6:43 PM on April 14, 2013 [2 favorites]


"At the same time, others look at teachers and say "your job is to be a miracle worker and I expect nothing less"."

…for an entry salary of $30k.

Good teaching costs real money, but folks get all bent out of shape about "unions" and want to get miracles for free.
posted by klangklangston at 6:53 PM on April 14, 2013 [13 favorites]


If a student cheated on his final exam, I wouldn't listen to his argument that the test itself is unfair and he shouldn't have to suffer through any evaluation of his work. I'd kick him out of class and report him to the authorities regardless of the merit of his argument. And I'd expect anyone else to do the same.

But when a teacher cheats on a test, all the other teachers leap to his or her defense.

If teachers want to start being treated as professionals, they have to stop defending the scumbags in their ranks. It doesn't matter if the test is poorly designed, you have a legal and moral responsibility to either carry it out in an honest manner or resign your position -- exactly like any other professional employee would when given direct instructions to do something.
posted by miyabo at 7:15 PM on April 14, 2013


My wife is a former DCPS teacher but also a research scientist who deeply understands how to construct experiments and analyze results. She liked the idea of testing & performance pay in the abstract – you need something for planning – but there are many flaws in DC's approach to teacher assessment, ranging from small sample sizes and bad stats (imagine if you knew your chance at an annual bonus of half your salary basically depended on the students you were randomly assigned), poor handling of common problems – e.g. grading teachers on students with high absence rates or recent immigrants with poor English fluency & limited past education being placed in grades based on age – and some structural problems with the tests, the most obvious being that DC's standardized tests have no meaning for the students — not only are there no meaningful downsides to the student for failing but they *know* they can hurt an unpopular teacher by deliberately bombing the test!
posted by adamsc at 7:41 PM on April 14, 2013 [3 favorites]


> So far as I've read, only 18 classrooms have been implicated in the cheating scandal: that's 18 teachers out of 4000-some. Is that really widespread?

That's just the 2012 investigation. In 2008, 96 schools were flagged by McGraw-Hill for high WTR erasure rates (including 8 of the 10 schools that had received "TEAM" awards for improvement that year); in 2010, 110 classrooms in 41 schools were flagged. That 2009 memo from a consultant to Rhee's chief of data and accountability (uncovered by Merrow in the first FPP link; see also WaPo) raises suspicions about nearly 200 classrooms in 2008:
I’ve been working furiously on the Erasure Study. It is common knowledge that in the high stakes testing community that one of the easiest ways for teachers to artificially inflate student test scores is to erase student wrong responses to multiple choice questions and recode them as correct .... There are 191 representing 70 schools that are implicated in possible testing infractions by the study....
The memo goes on to emphasize that the teachers are only "possibly culpable at some level" and stops short of concluding that cheating necessarily took place, but begs for further investigation and action in its recommendations.
posted by Westringia F. at 7:45 PM on April 14, 2013 [2 favorites]


It's not that hard to come up with reasonable systems for keeping or rejecting teachers - the issue is that no one's interesting in doing it because everyone has huge vested interests. Like so many other things, it won't get fixed until everything collapses and people realize they have to work together or perish.

The system I'd propose for a later, happier environment is something like a more formalized one of the way that people deal with teachers in places like Norway.

There would be a number of "black areas" - bad things you could do that would get you fired, much like the essential things that will get you fired today - violence, undue anger, racist slurs and the like. Yes, if you had to have standardized tests, if all your kids failed repeatedly you could get fired.

Aside from that, you'd have to "show strengths" - there would be a large number of evaluation areas, and you'd have to actually show positive results in some number of them. Test performance would be just one of these areas - others would be student evaluations, parent evaluations, peer evaluations, extra-curricular activities, curriculum creation, problem resolution...

In other words, if you're not really terrible at anything, and show some flair somewhere, you're in. And face it, isn't that most of us? We bumble along doing a tolerable job at most things, but a few things we're pretty good at?

But for this to work there would have to be all these bonds of trust and respect that simply don't exist in American at all. Parents don't respect teachers; teachers don't respect administrators; administrators don't respect anyone; and no one respects kids at all.

My mother worked in early childhood education and her last job was in a cooperative daycare in Canada. Everyone was paid the same, from the coordinator to the cook (who had a PhD in economics and a successful career beside him but had decided that cooking healthy food for kids was more important).

Their coordinator left and they got a new one. A few months later they had to fire her. She just wasn't keeping it together - so all the workers had a meeting where everyone cried and they gave her notice.

There are few places in the US where this could happen. The consequences of losing a job are not life-threatening in other first-world countries. Many people have religious beliefs that are equivalent to right-wing political beliefs and impose them on others. Non-hierarchical systems are considered akin to Satanism and homosexuality. The idea of a system without quantitative evaluations where a group could sadly decide to fire one of their members would be considered alien and dangerous in most parts of the country.
posted by lupus_yonderboy at 8:12 PM on April 14, 2013 [6 favorites]


If teachers want to start being treated as professionals, they have to stop defending the scumbags in their ranks. It doesn't matter if the test is poorly designed, you have a legal and moral responsibility to either carry it out in an honest manner or resign your position -- exactly like any other professional employee would when given direct instructions to do something.

Why, it's as if you are viewing this situation in the narrowest possible manner, ignoring the systemic structural problems with high-stakes testing. Calling teachers "scumbags" doesn't help, and makes you look like a person wholly ignorant of the context of these problems.
posted by zardoz at 8:41 PM on April 14, 2013 [8 favorites]


Standardized tests are a big business that makes a few people a shit ton of money. They fuck the student, the teachers, and the low level administrators. The main people they profit are politicians, owners of standardized test related companies and so-called gurus like Michelle Rhee. As with all things, follow the money and you know what the motives are.

There are a plenty of ways to make things student centered and improve education that don't involve standardized testings. There are also ways to fuck public education to drive money to private schools, including religious schools.

Michelle Rhee has mastered the art of claiming to make things student centered while simultaneously doing her billionaire corporate backer's bidding.
posted by Joey Michaels at 8:59 PM on April 14, 2013 [8 favorites]


Two school superintendents I write about on Long Island have been warning for months that parents should be for big dropoffs in scores for these state tests because they go beyond the curriculum.
My hometown district in Ohio was taken over by the state last week because it had failed to meet standards four years in a row. Of the 26 standards, it made one last year (I don't know what it was). It makes me ill to read about this.
I grew up with a very strong sense of the importance of public education in our society. Whatever the reasons for failure, we have to fix this now.
posted by etaoin at 9:14 PM on April 14, 2013


memebake, responding to philip-random's pithy comic about standardized testing, showing a guy testing a bunch of animals (including an elephant, a fish and monkey among others) on their ability to climb trees:
But with the right statistical approach, that test could be made to work. I'm not joking. Instead of measuring the absolute result, you measure the improvement over time, and possibly cross reference by number of legs or weight or whatever factors are considered important.

1. Perhaps. But the less suited the testers are to the material on the test, the larger the sample size you need to learn anything, and the less certain you are of whatever it is you're testing for. (Tree climbing ability? Skill? General "goodness?") And, it's another step, another thing to go wrong: now you also have to get the statistics right, and you aren't so confident when they are right, you end up just having to assume the numbers are valid. It's better just to use a fair test to begin with.

2. The key to the comic is: the guy who is giving the test is inherently biased in his criteria, and doesn't realize it. He thinks the test is a fair and accurate measure by itself, and why wouldn't he, as a higher-order primate he's not necessarily that bad at climbing trees. In order to have a chance at fixing a broken test, you have to see that it's broken to begin with, and right now the criteria being used are not those of educators (who as a whole have been surprisingly demonized lately) but politicians determined to make a difference, whether they really understand the situation they're fixing or not.
posted by JHarris at 9:17 PM on April 14, 2013 [2 favorites]


If you work in private industry, certainly you are suspicious of how metrics are invented as THE MOST IMPORTANT STAT EVER (we actually get memos like that), and then rigged to show how successful the corporate plan is. When that number doesn't work out, then a new way of measurement is found or invented and then everyone goes racing off to try to capture that number. There's a house of cards sensibility that prevails. Shift over to education, compound that by all the bureaucrats on the federal, state and local level, all trying to protect themselves as the landscape shifts, and you're asking for trouble.
posted by etaoin at 9:25 PM on April 14, 2013 [4 favorites]


[R]ight now the criteria being used are not those of educators (who as a whole have been surprisingly demonized lately) but politicians determined to make a difference, whether they really understand the situation they're fixing or not.

My problem with this argument is that it takes a very different skill set to come up with a curriculum geared to college and career preparation and to design a good assessment of that curriculum than it does to teach that curriculum. There's often a real lack of humility on this point from teachers criticizing education reform. As an educator myself, I trust people who have gone to grad school and put the time in to learn how to communicate with people in industry, higher education, and public education, and who've learned to translate the results of these conversations into curriculum and assessment.

Teachers shouldn't be expected to do all that, and they shouldn't be telling people they're capable of it or of producing something of quality while ignoring it. So, while you may see an assessment that nobody in a position of leadership realizes is broken, I see one that a lot of educators aren't giving enough credit to.
posted by alphanerd at 10:07 PM on April 14, 2013


Michelle Rhee needs to watch Glengarry Glen Ross. Or maybe she did, but thought Alec Baldwin was the hero.

Considering David Mamet's political leanings, what makes you think he wasn't?
posted by MartinWisse at 11:20 PM on April 14, 2013 [1 favorite]


In summary: I'm a little uncomfortable when an initiative to measure a professions performance is being shouted down so hard that even apparent outright cheating by the professionals is still blamed on someone else. Thats how the John Merrow article reads to me.

Michelle Rhee never cared about kids learning more stuff. She cared about setting arbitrary 'standards' for test scores and then exceeding them. Anybody who cares about actually teaching kids knows that standardized testing is horseshit.
posted by empath at 11:42 PM on April 14, 2013


If you work in private industry, certainly you are suspicious of how metrics are invented as THE MOST IMPORTANT STAT EVER (we actually get memos like that), and then rigged to show how successful the corporate plan is. When that number doesn't work out, then a new way of measurement is found or invented and then everyone goes racing off to try to capture that number.

I've spent a lot of time in help desks, and I feel you there. I once had a boss tell me to stop working on fixing customer problems and close all tickets within 48 hours no matter what. When I said, but won't they just call back? She said, when they do, open a new ticket. We're getting measured on how many tickets we close, and we're going to close them.
posted by empath at 11:47 PM on April 14, 2013 [5 favorites]


When I investigated police misconduct in NYC, we had a couple of interesting statistics that I thought were helpful for thinking about the scale of the problem: there were @40,000 police officers at the time. We received @4,000 complaints each year. And we substantiated @400 of them. In terms of the size of the problem of police misconduct, I thought that 10% complaint rate and 1% substantiation rate was helpful: the vast majority of police officers were doing their jobs honorably and well. Even with margins for unreported misconduct and cases where there was wrong-doing but not enough evidence to prove it, that's impressive.

The memo goes on to emphasize that the teachers are only "possibly culpable at some level" and stops short of concluding that cheating necessarily took place, but begs for further investigation and action in its recommendations.

Right: and further investigation found evidence of actual cheating at 11 schools in 18 classrooms. So they started with suspicions in 191 and 70, and winnowed it down to the 11 and 18.

That's all out of 135 schools and 4,017 teachers.

So look here: 18 ÷ 4071 = 0.45% That's half a percent of DCPS teachers cheating, after 4.75% started out under suspicion.

I'm a teacher, and my plagiarism rate is higher than that... approximately 5%-10% in any given class (one or two in a class of 25) will try to submit someone else's writing as their own. (Which is why I don't fail them, but rather require them to admit their error and rewrite the papers.)

This just doesn't look like a "widespread" problem to me. And meanwhile, this is giving us information about the quality of instruction among the rest of the faculty.

The problem with folks in caring professions like teaching is that we burn out. Some of us become masters, and some of us become exhausted and start going through the motions, comforting ourselves that we're serving unteachable populations and that no one else could possibly understand how hard that is. But the whole point is that the caregiver is not as important as the person who needs that care. In this case, those people are the students.

If you could fire me tomorrow and replace me with someone who did a better a job teaching urban youth, I'd want you to do it. I'm an upper-middle class white dude with privilege out the wazoo, and my students are black, underprepared, and historically underserved. But I'm not going to quit, because I've got the mortgage and a newborn who goes through diapers like crazy.

So yes! let's make a test, so that I have something that I can look at and say: wow, my colleagues are way better at this than me, I need to bow out. Or, alternatively: I'm doing a pretty good job and it's my colleagues who are slacking off. And in either case, it looks like the odds that I would cheat on such an assessment are low enough to ignore.
posted by anotherpanacea at 4:37 AM on April 15, 2013 [3 favorites]


"even that they're representative of a spoiled/entitled/whatever generation?"

This sort of broad brush rhetoric has gotten extremely old. Can we stop doing this? It's not really helping anyone.
posted by Phalene at 5:13 AM on April 15, 2013


So yes! let's make a test, so that I have something that I can look at and say: wow, my colleagues are way better at this than me, I need to bow out. Or, alternatively: I'm doing a pretty good job and it's my colleagues who are slacking off. And in either case, it looks like the odds that I would cheat on such an assessment are low enough to ignore.

Standardized tests only give you an accurate snapshot of student progress as long as there are no incentives for high scores.

As soon as you reward high scores and punish low scores, the test is useless. Everyone from the students and parents to the school board has incentives to game the system and absolutely no one has incentive to get it right, aside from whatever morals they have. If your job is on the line, or your kid's future is on the line, you are not going to let a stupid multiple choice test ruin you.

Standardized testing should only be used to figure out how much and what kind of help that kids need. That's it. Any other use is at best meaningless, and at worst, corrupting.

If you want metrics that matter? College acceptance rates, drop out rates, teen pregnancy rates, violence, truancy, quality of school lunches, surveys of student happiness. I can think of a million things you can measure that are relevant to student success that don't depend on meaningless tests. They're just a lot harder to achieve than drilling rote facts into kids heads.
posted by empath at 5:25 AM on April 15, 2013 [12 favorites]


> Right: and further investigation found evidence of actual cheating at 11 schools in 18 classrooms. So they started with suspicions in 191 and 70, and winnowed it down to the 11 and 18.

anotherpanacea, please read the links. The 191 classrooms / 70 schools were for the 2007-08 academic year; 18 classrooms / 11 schools were for the 2011-2012 academic year. These are separate sets of data showing that the irregularities, while declining, continue to be detectable over time, not a "winnowing down" of suspicions.

You may still conclude that DCPS doesn't have a widespread problem, of course, but please do so without misconstruing the facts.
posted by Westringia F. at 5:36 AM on April 15, 2013


Well, the comparison is between 191 classrooms in 2008 and 41 classrooms in 2012: that's the "suspected" number. They only substantiated 18, so about half. That means that they might only have substantiated 90 of the 191 suspected classrooms in 2008.

And again: 90 teachers out of 4017 isn't really that many, and wouldn't count as widespread.
posted by anotherpanacea at 6:26 AM on April 15, 2013 [1 favorite]


empath: As soon as you reward high scores and punish low scores, the test is useless.

So setting kids a test for which high scores are rewarded is useless? Isn't that the whole basis of how we award academic qualifications across the developed world?

I'd love to move to a system where schools were built to output people with more happiness or better citizenship or better people skills or something. But practically, most stakeholders of the education system right now are going to judge its performance by the qualifications gained at the end of the pipeline.
posted by memebake at 10:35 AM on April 15, 2013


So setting kids a test for which high scores are rewarded is useless? Isn't that the whole basis of how we award academic qualifications across the developed world?

We're not talking about awarding academic qualifications. The whole idea is that you can use standardised tests to identify and reward good schools and identify and punish bad schools, not students.

Perhaps you are too old to have been in school for this brave new world of standarised testing. I went to a school that was inevitably going to do acceptably well on the test. (Not because it was necessarily the best school ever, but because it was a rich school. Funny how areas with money do well.) We were still exhorted to care about the exam and to take it seriously. Not because our scores mattered one bit to our academic careers, but because it mattered to the school. (Even if we'd all failed intentionally, we'd have left the school by the time any consequences hit. It genuinely did not matter to our futures one bit.)
posted by hoyland at 11:23 AM on April 15, 2013


another grifter in an age of grifters. In many respects she's a 'Clintonian'
posted by SteveLaudig at 4:10 PM on April 15, 2013


Folks, let's get something straight: the standardized tests, in and of themselves, are not the problem. Ask most experienced teachers and they will tell you that annual standardized tests provide important and useful information.

The problem is they have become the whole enchilada, the whole deal, everything and the kitchen sink on top. Assessments are key to education, and standardized tests are one kind of assessment. They are a legitimate tool in the toolbox.

The problem is they have become the entire goddamn toolbox. There needs to be more assessment, especially informal assessment. Problem is that's not as easy to monetize for the education industry.
posted by zardoz at 7:22 PM on April 16, 2013 [1 favorite]


Valedictatorian
The central proposition of so-called education reform is that it endeavors to make schooling more entrepreneurial. Now this is bogus on its face. The most salient fact about entrepreneurialism is that most ventures fail. Is that the proper model for the delivery of a universal service? Consider the question irrespective of your thoughts about the larger questions surrounding the provision of universal education. Ostensible reformers say they want to mimic the dynamism and innovation of the private sector. The first question is: to what end, exactly? The second is: do you know how dynamism and innovation work?

Like most pro-market types, these people are ignorant of the actual workings of capitalism. They see Apple’s glittering headquarters, Google’s quarterly revenue numbers, and they think, Damn! I wish schools could be more like that! Strewn across the historic landscape behind all this success are hundreds of thousands of failed attempts, many of which don’t make it out of their first year. And you want school to look like this? Well, uh, no; we only want school to imitate successful ventures! Well, I want better arms and a bigger dick, but editing every other eighth of an inch out of the measuring tape will not make it so.
posted by tonycpsu at 9:23 PM on April 25, 2013 [1 favorite]


« Older David Sedaris buys an owl....  |  150 U.S. city flags, ranked fr... Newer »


This thread has been archived and is closed to new comments