A Level in Competence
August 13, 2020 5:49 AM Subscribe
As students across England, Wales and Northern Ireland receive their adjusted exam results, private schools see a rise in A* and A grades, while 40% of teacher assessments in England have been downgraded.
Education secretary Gavin Williamson remains defiant after vowing to end grade inflation, and Minister for School Standards Nick Gibb refuses to follow Scotland's lead in allowing students to use grades from mock exams instead of their calculated results. The Office of Qualifications warns that "because of the role of the rank order in grading this year", any appeals would "affect other students in your cohort".
Education secretary Gavin Williamson remains defiant after vowing to end grade inflation, and Minister for School Standards Nick Gibb refuses to follow Scotland's lead in allowing students to use grades from mock exams instead of their calculated results. The Office of Qualifications warns that "because of the role of the rank order in grading this year", any appeals would "affect other students in your cohort".
It's a really difficult issue. In Scotland, grades were awarded based on teachers' estimates. Those estimates resulted in pass rates shooting up by 14 percentage points compared with last year, the highest annual increase ever rewarded. Is it believable that, in lockdown and with closed schools, students' ability shot up by an unprecedented amount? So the grades were moderated downward to make them consistent with past years. That moderation was so unpopular that it's had to be reversed, and estimated grades reinstated.
Although we can all agree it's a great shambles, it's not completely clear what the education authorities were supposed to do. Just accept teachers' estimates, no matter how implausible and inflated? That's not great for confidence in the exam system, and for past and future years' students.
It's a good reminder that politics is always about individuals not aggregates, and that winners from a particular policy always stay silent while losers get front pages.
posted by Klipspringer at 6:27 AM on August 13, 2020 [1 favorite]
Although we can all agree it's a great shambles, it's not completely clear what the education authorities were supposed to do. Just accept teachers' estimates, no matter how implausible and inflated? That's not great for confidence in the exam system, and for past and future years' students.
It's a good reminder that politics is always about individuals not aggregates, and that winners from a particular policy always stay silent while losers get front pages.
posted by Klipspringer at 6:27 AM on August 13, 2020 [1 favorite]
One option would be to just not issue any poorly modeled “official” grades and instead just publish the mock results and teacher estimates as-is, with appropriate caveats and asterisks on whatever certificate gets issued.
Then let the employer or university decide how to interpret the result. If half the cohort write “I got BBB but was estimated AAA” on their cover letter then institutions are just going to have to dig deeper than the official results any way. Nobody cares about specific A Level or GCSE exam results after tertiary education or a first job.
posted by rh at 6:43 AM on August 13, 2020 [4 favorites]
Then let the employer or university decide how to interpret the result. If half the cohort write “I got BBB but was estimated AAA” on their cover letter then institutions are just going to have to dig deeper than the official results any way. Nobody cares about specific A Level or GCSE exam results after tertiary education or a first job.
posted by rh at 6:43 AM on August 13, 2020 [4 favorites]
In Scotland when they capitulated they made the point that they had focused too much on the wider issue of grade inflation at the expense of what students' personal experiences of this would be like. The U-turn was allegedly based on the realisation that perhaps the latter was more important than the former.
We're in England and my son gets his GCSE results next week. He's in a state school, he's not academic and our best hope was a scrape into his first choice for his next step, but I won't be at all surprised if the government doesn't U-turn in the meantime and that he gets nonsensically low grades because this government fundamentally doesn't seem to like or give a damn about people who are not like them, and we are not like them.
posted by dowcrag at 6:43 AM on August 13, 2020 [9 favorites]
We're in England and my son gets his GCSE results next week. He's in a state school, he's not academic and our best hope was a scrape into his first choice for his next step, but I won't be at all surprised if the government doesn't U-turn in the meantime and that he gets nonsensically low grades because this government fundamentally doesn't seem to like or give a damn about people who are not like them, and we are not like them.
posted by dowcrag at 6:43 AM on August 13, 2020 [9 favorites]
This all sounds so difficult and stressful for the students especially.
posted by Dip Flash at 6:50 AM on August 13, 2020 [3 favorites]
posted by Dip Flash at 6:50 AM on August 13, 2020 [3 favorites]
The sheer gall of Williamson saying "Increasing the A Level grades will mean a whole generation could end up promoted beyond their abilities" when he's a former fireplace salesman with a Bsc in social sciences who ended up as Minister for Education after previously being sacked as Defence Minister for leaking information from the National Security Council. Plus the amount of his cabinet colleagues who have previously been fired for incompetence or lying (or both).
Hell, this is the government who tried to appoint Chris "failing" Grayling as intelligence committee chair, whose crowning achievement was arguably signing a multimillion ferry contract with a company with no ferries and fast food restaurant terms & conditions on their website - in total, he managed to rack up £3.5 billion of wasted money for previous screwups as a minister. Lets not even mention the ongoing covid carehomes/PPE/staffing or brexit omnishambles.
While the argument that A-level results would have increased a bit too much in comparison to previous rises is not totally without merit, any statistical system that results in multiple schools and colleges seeing 60%+ of their grades downgraded to be worse than any previous set of results is a failure on a systemic level - and the students paying the highest cost are those who have been exceptional throughout, and were predicted AAA or better yet end up getting Bs and Cs at best because they happen to live in the wrong postcode. I've seen reports of at least 2 such students having now lost their oxbridge place as a result.
"Although we can all agree it's a great shambles, it's not completely clear what the education authorities were supposed to do."
Talking to schools at any point in the process would have a start. There have been multiple stories now of schools who did all they were told to, including using an external agency to process their teacher-assigned grades so they were overall equivalent to previous years results before submission, yet still see 60 or 70% of their grades lowered. Schools which produced wide-ranging results drastically out of line with previous results (i.e. the apocryphal school that gave all their C students As) should have had to provide additional evidence (up OR down) such as mocks and GCSEs and individual assessments; small numbers of individually strong pupils should not be penalised for previous school cohorts performance.
Even on the face of it, if the centre-assigned grades (not just teacher assessed, they had to be signed off by their head and his team at the very least) were 12% overall too high, that shouldn't result in 49% of all grades being lowered, that's just nonsensical.
They've had weeks of signs that this was going to be a problem - rushing out a quick fix appeal system based upon mock results (which are definitely NOT standardised and done all sorts of ways) the day before is weak sauce.
Students have suffered enough this year. Even if this year's A level results might end up a bit too high by relying solely on teachers' professional judgement because Ofqual couldn't statistic their way out of a paper bag, so what. They're always going to have an asterix by them as 'covid results' anyway.
posted by Absolutely No You-Know-What at 6:52 AM on August 13, 2020 [40 favorites]
Hell, this is the government who tried to appoint Chris "failing" Grayling as intelligence committee chair, whose crowning achievement was arguably signing a multimillion ferry contract with a company with no ferries and fast food restaurant terms & conditions on their website - in total, he managed to rack up £3.5 billion of wasted money for previous screwups as a minister. Lets not even mention the ongoing covid carehomes/PPE/staffing or brexit omnishambles.
While the argument that A-level results would have increased a bit too much in comparison to previous rises is not totally without merit, any statistical system that results in multiple schools and colleges seeing 60%+ of their grades downgraded to be worse than any previous set of results is a failure on a systemic level - and the students paying the highest cost are those who have been exceptional throughout, and were predicted AAA or better yet end up getting Bs and Cs at best because they happen to live in the wrong postcode. I've seen reports of at least 2 such students having now lost their oxbridge place as a result.
"Although we can all agree it's a great shambles, it's not completely clear what the education authorities were supposed to do."
Talking to schools at any point in the process would have a start. There have been multiple stories now of schools who did all they were told to, including using an external agency to process their teacher-assigned grades so they were overall equivalent to previous years results before submission, yet still see 60 or 70% of their grades lowered. Schools which produced wide-ranging results drastically out of line with previous results (i.e. the apocryphal school that gave all their C students As) should have had to provide additional evidence (up OR down) such as mocks and GCSEs and individual assessments; small numbers of individually strong pupils should not be penalised for previous school cohorts performance.
Even on the face of it, if the centre-assigned grades (not just teacher assessed, they had to be signed off by their head and his team at the very least) were 12% overall too high, that shouldn't result in 49% of all grades being lowered, that's just nonsensical.
They've had weeks of signs that this was going to be a problem - rushing out a quick fix appeal system based upon mock results (which are definitely NOT standardised and done all sorts of ways) the day before is weak sauce.
Students have suffered enough this year. Even if this year's A level results might end up a bit too high by relying solely on teachers' professional judgement because Ofqual couldn't statistic their way out of a paper bag, so what. They're always going to have an asterix by them as 'covid results' anyway.
posted by Absolutely No You-Know-What at 6:52 AM on August 13, 2020 [40 favorites]
rh: the universities are likely to be pretty happy since there are more people with acceptable grades and they have been allowed to pick up an extra 5% above what was initially stated. That means they'll stay solvent. The Russell Group anyway, which is who the government listen to.
The problem is a social one. Worked your arse off in a comp in a poor area? The exam boards will downgrade your mock exam grade and estimated grade so overall your school got about what it got last year. Tough shit on you. Done the same amount of work at the place your daddy got you into? Have an A*.
If someone is doing 3 A levels and needs AAA to get on their chosen programme there is literally no way to get it if their grades go down by 4 grades overall.
posted by biffa at 6:56 AM on August 13, 2020 [15 favorites]
The problem is a social one. Worked your arse off in a comp in a poor area? The exam boards will downgrade your mock exam grade and estimated grade so overall your school got about what it got last year. Tough shit on you. Done the same amount of work at the place your daddy got you into? Have an A*.
If someone is doing 3 A levels and needs AAA to get on their chosen programme there is literally no way to get it if their grades go down by 4 grades overall.
posted by biffa at 6:56 AM on August 13, 2020 [15 favorites]
Talking to schools at any point in the process would have a start.
One of the things that bothers me is how has it taken the same amount of time to NOT have to mark any exams as it takes to mark them? Why didn't the government work with the exam boards on addressing this over the last month? Instead they just all seemed to pretend it was just like a normal year, apart from they were making up the numbers. The result is most uni programmes start in the next 4-5 weeks, thousands of students will have to roll the dice on appealing for grades and on their appeal being addressed in the next 2-3 weeks and hope they can get into the programme they applied for; many will miss out on either getting their programme this year, or on getting a good alternative this year. And since the number of 18 year olds rises next year then the ask for some programmes will also go up, so some people will miss out on their programmes entirely.
posted by biffa at 7:04 AM on August 13, 2020 [3 favorites]
One of the things that bothers me is how has it taken the same amount of time to NOT have to mark any exams as it takes to mark them? Why didn't the government work with the exam boards on addressing this over the last month? Instead they just all seemed to pretend it was just like a normal year, apart from they were making up the numbers. The result is most uni programmes start in the next 4-5 weeks, thousands of students will have to roll the dice on appealing for grades and on their appeal being addressed in the next 2-3 weeks and hope they can get into the programme they applied for; many will miss out on either getting their programme this year, or on getting a good alternative this year. And since the number of 18 year olds rises next year then the ask for some programmes will also go up, so some people will miss out on their programmes entirely.
posted by biffa at 7:04 AM on August 13, 2020 [3 favorites]
Thanks for this roundup, lucidium! The HEPI article has a lot of detail and is well worth reading. The money-quote:
For the first time ever, students are competing within the school with their schoolmates for the limited numbers of grades available at each level. These are not the grades this year’s students deserve, they are the grades which the Ofqual model says the school deserves on the basis of past students’ achievements.posted by adrianhon at 7:09 AM on August 13, 2020 [12 favorites]
The only way to restore individual fairness is to restore individual appeals which look at the student’s actual achievements, not the past record of the school. Every previous student has had, every future student will have, access to an individual appeal process. Not in 2020. So much for the Secretary of State’s pledge that this year’s students should not face ‘a systematic disadvantage as a consequence of these extraordinary circumstances’.
Is it believable that, in lockdown and with closed schools, students' ability shot up by an unprecedented amount?
I wouldn’t actually dismiss this out of hand. Online school, even implemented poorly, is vastly better for many disabled kids, bullied kids, anxious kids, etc. It’s not actually impossible that a significant portion of students did much better in lockdown. Just anecdotally, in a clinical setting, kids are so much more patient and engaged from the comfort of their own home than in a clinic setting. I wouldn’t be surprised if that’s a factor in school as well. Is it enough to combat the other problems? I don’t know, but I don’t think we should take at face value that some portion of students couldn’t actually have done better in lockdown.
posted by brook horse at 7:10 AM on August 13, 2020 [19 favorites]
I wouldn’t actually dismiss this out of hand. Online school, even implemented poorly, is vastly better for many disabled kids, bullied kids, anxious kids, etc. It’s not actually impossible that a significant portion of students did much better in lockdown. Just anecdotally, in a clinical setting, kids are so much more patient and engaged from the comfort of their own home than in a clinic setting. I wouldn’t be surprised if that’s a factor in school as well. Is it enough to combat the other problems? I don’t know, but I don’t think we should take at face value that some portion of students couldn’t actually have done better in lockdown.
posted by brook horse at 7:10 AM on August 13, 2020 [19 favorites]
"Online school" basically didn't exist in the state sector. "In April, just 3% of state-funded primary and 6% of state-funded secondary schools managed to provide “live” online lessons for students with their teachers" (Guardian).
posted by Klipspringer at 7:13 AM on August 13, 2020 [2 favorites]
posted by Klipspringer at 7:13 AM on August 13, 2020 [2 favorites]
Is there some reason that universities couldn't relax acceptance requirements for this year?
posted by jeather at 7:15 AM on August 13, 2020 [2 favorites]
posted by jeather at 7:15 AM on August 13, 2020 [2 favorites]
Is it believable that, in lockdown and with closed schools, students' ability shot up by an unprecedented amount?
Well, maybe. As a university lecturer in Scotland, I've been really impressed with the quality of work that our students in all years have produced since lockdown started - not just good given the circumstances but good overall this year. So I don't think it's entirely unreasonable to suggest that students a couple of years younger than mine have also put in a lot of work in a really difficult situation, and could achieve higher grades than usual as a result.
posted by offog at 7:29 AM on August 13, 2020 [6 favorites]
Well, maybe. As a university lecturer in Scotland, I've been really impressed with the quality of work that our students in all years have produced since lockdown started - not just good given the circumstances but good overall this year. So I don't think it's entirely unreasonable to suggest that students a couple of years younger than mine have also put in a lot of work in a really difficult situation, and could achieve higher grades than usual as a result.
posted by offog at 7:29 AM on August 13, 2020 [6 favorites]
Even on the face of it, if the centre-assigned grades (not just teacher assessed, they had to be signed off by their head and his team at the very least) were 12% overall too high, that shouldn't result in 49% 39% of all grades being lowered, that's just nonsensical.
Apologies for the typo. Perhaps I should apply for a job at Ofqual.
posted by Absolutely No You-Know-What at 7:30 AM on August 13, 2020 [2 favorites]
Apologies for the typo. Perhaps I should apply for a job at Ofqual.
posted by Absolutely No You-Know-What at 7:30 AM on August 13, 2020 [2 favorites]
Hell, this is the government who tried to appoint Chris "failing" Grayling as intelligence committee chair...
It's a meta-level of irony that Grayling failed to become chair of the Intelligence Committee because he didn't spot a coup against him.
posted by Wordshore at 7:33 AM on August 13, 2020 [11 favorites]
It's a meta-level of irony that Grayling failed to become chair of the Intelligence Committee because he didn't spot a coup against him.
posted by Wordshore at 7:33 AM on August 13, 2020 [11 favorites]
Although we can all agree it's a great shambles, it's not completely clear what the education authorities were supposed to do. Just accept teachers' estimates, no matter how implausible and inflated? That's not great for confidence in the exam system, and for past and future years' students.
But we know this was a weird year, and any solution is going to be imperfect. Using a broad statistical measure to weigh down students grades will obviously be unfair, and it will be statistically unfair. That is, you are applying a mean measure to individuals. You are basically baking in inequality.
For a lot of people, this has been an incredibly bad year, but for students who have been cut off from their friends for their final year it must have been particularly stressful. The decisions made here have just made that so much worse. It's just appalling.
posted by Cannon Fodder at 8:30 AM on August 13, 2020 [2 favorites]
But we know this was a weird year, and any solution is going to be imperfect. Using a broad statistical measure to weigh down students grades will obviously be unfair, and it will be statistically unfair. That is, you are applying a mean measure to individuals. You are basically baking in inequality.
For a lot of people, this has been an incredibly bad year, but for students who have been cut off from their friends for their final year it must have been particularly stressful. The decisions made here have just made that so much worse. It's just appalling.
posted by Cannon Fodder at 8:30 AM on August 13, 2020 [2 favorites]
Is there some reason that universities couldn't relax acceptance requirements for this year?
Government (re)introduced a quota with an upper limit. If universities recruit over it the fines are eye-watering.
posted by biffa at 8:35 AM on August 13, 2020
Government (re)introduced a quota with an upper limit. If universities recruit over it the fines are eye-watering.
posted by biffa at 8:35 AM on August 13, 2020
Can someone from the UK explicate the background here? What is a mock exam? Is this like the pre-SAT and SAT, where kids took a set of tests twice?
posted by tavella at 8:37 AM on August 13, 2020
posted by tavella at 8:37 AM on August 13, 2020
Yes, mocks (England and Wales; prelims in Scotland) would be like the pre-SAT: pretty much the same level of exam, just sat a few months earlier to gauge areas of competence and exam readiness. The prelims I remember from Scotland in the 1980s were held in a very slightly more relaxed situation than the actual national exams. They were rigorous enough, however, to be used in place of the exam, if for some reason the student was prevented from sitting the final exam. A friend was in hospital over their Higher exams, so their prelim results were considered by the exam board for their final results.
School systems are very different in different parts of the UK, and very different from the North American model.
posted by scruss at 8:48 AM on August 13, 2020 [2 favorites]
School systems are very different in different parts of the UK, and very different from the North American model.
posted by scruss at 8:48 AM on August 13, 2020 [2 favorites]
What is a mock exam? Is this like the pre-SAT and SAT, where kids took a set of tests twice?
Here's an article from the BBC explaining why headteachers aren't in favour of using mocks results, which also gives an idea of what they can be like.
posted by scorbet at 8:56 AM on August 13, 2020
Here's an article from the BBC explaining why headteachers aren't in favour of using mocks results, which also gives an idea of what they can be like.
posted by scorbet at 8:56 AM on August 13, 2020
@jessicaelgot: Today’s English A-Level students are getting results decided by a computer based on whether they are rich or poor and if they are lucky will get into up to £55k of debt to watch online lectures in their bedrooms. It’s a betrayal of this generation.
Oh my god, it's my sci-fi book.
Sorry if this is self-promo on the blue but people are tagging me on Twitter because this is like half the scam setting up my sci-fi novel only instead of a pandemic and online schools mine ends in space pirates.
Hate to say it but the racket inevitably escalates from this point if it isn't crushed. I probably don't have a case for copyright infringement but god I wish I could sue to make them stop.
posted by scaryblackdeath at 8:57 AM on August 13, 2020 [9 favorites]
Oh my god, it's my sci-fi book.
Sorry if this is self-promo on the blue but people are tagging me on Twitter because this is like half the scam setting up my sci-fi novel only instead of a pandemic and online schools mine ends in space pirates.
Hate to say it but the racket inevitably escalates from this point if it isn't crushed. I probably don't have a case for copyright infringement but god I wish I could sue to make them stop.
posted by scaryblackdeath at 8:57 AM on August 13, 2020 [9 favorites]
Can someone from the UK explicate the background here? What is a mock exam? Is this like the pre-SAT and SAT, where kids took a set of tests twice?
Students usually take 'mock' A-level exams in Dec/January, i.e. papers that look like real exam papers but aren't. This allows teachers and students to see how they're handling the curriculum, and target help at those who need it most before they sit the real exams in May/June. Some schools specifically use past exam papers' questions on material that isn't covered yet, so students get an idea of what they're actually going to face, while they've still time to catch up, but they're not held to anywhere like the same standard as a real exam.
Since our university application system is fucked up, these mock results are used as the basis to apply for university places in or so; usually students will apply for a place at a shortlist of unis in order of preference, and unis will give them a provisional place (or not) based upon their predicted results, usually with a required minimum result to get the place.
If students get the real grades necessary for one of their offers in August, i.e. today, their provisional place is confirmed (if they have multiple offers they meet the requirements for, they can choose which one to take). If they're close, some universities will let them keep the place anyway (but usually not the top-tier unis). If they fail to get the necessary grades for any of their offers, of have changed their mind about where they want to go, they go into "clearing" - i.e. a system which tries to match up students with available places that meet their grades - and you may have very little time to decide if you fancy going on a different course or uni in a totally different part of the country than before, when some unis will retain at least some distance learning, and/or on-campus housing may not be available.
The real exams didn't happen due to covid, so teachers instead submitted what grades they thought students would have got, based upon mock results and work ethic and in numbered order of strongest to weakest per subject, ofqual mangled them with statistics upon the school's past performance (not the student's), and those are the what students have received today - with 39% of grades lowered by ofqual to worse results than predicted, in some cases much worse, and students in effect competed against their fellow students to get what grades were available for a school based upon past results, rather than the actual exam or any personal talent.
Some unis are being more lenient than others in letting lower than expected grades result in a place anyway; others are saying 'bad luck chum'.
A much higher number are also entering clearing because the uni half the country away with no digs available now is suddenly a lot less appealing than it was in January, and they want to stay closer to home.
Students who would normally now enter clearing, are now left with the unenviable position of deciding if they want to
1) wait for their school to appeal the entire set of results for that school, which may even result in large scale regrading.
2) They can also try and take an actual exam in october to get a better grade.
3) go through some as yet undefined appeal so they get at least their mock grade result, announced last night.
But uni starts in september, and places will not remain open, as 'clearing' fills them up. So the other option is
4) to accept the lower grades and try to make the best of it in a hurry, and end up in any uni that'll take them with their poorer results. Even if their grades are subsequently raised on appeal, they're fairly locked into the uni they've now accepted, unless the nope out and try again next year - when unemployment is heading towards record levels, and a 'gap year' job is going to be hard to find, while bumming round europe (another traditional choice) is not on the cards either.
Basically, if you went to a school that's not an exam factory top-flight (i.e. high end private school), you're probably screwed, and what you do about it has to be decided in short order.
posted by Absolutely No You-Know-What at 9:05 AM on August 13, 2020 [25 favorites]
Students usually take 'mock' A-level exams in Dec/January, i.e. papers that look like real exam papers but aren't. This allows teachers and students to see how they're handling the curriculum, and target help at those who need it most before they sit the real exams in May/June. Some schools specifically use past exam papers' questions on material that isn't covered yet, so students get an idea of what they're actually going to face, while they've still time to catch up, but they're not held to anywhere like the same standard as a real exam.
Since our university application system is fucked up, these mock results are used as the basis to apply for university places in or so; usually students will apply for a place at a shortlist of unis in order of preference, and unis will give them a provisional place (or not) based upon their predicted results, usually with a required minimum result to get the place.
If students get the real grades necessary for one of their offers in August, i.e. today, their provisional place is confirmed (if they have multiple offers they meet the requirements for, they can choose which one to take). If they're close, some universities will let them keep the place anyway (but usually not the top-tier unis). If they fail to get the necessary grades for any of their offers, of have changed their mind about where they want to go, they go into "clearing" - i.e. a system which tries to match up students with available places that meet their grades - and you may have very little time to decide if you fancy going on a different course or uni in a totally different part of the country than before, when some unis will retain at least some distance learning, and/or on-campus housing may not be available.
The real exams didn't happen due to covid, so teachers instead submitted what grades they thought students would have got, based upon mock results and work ethic and in numbered order of strongest to weakest per subject, ofqual mangled them with statistics upon the school's past performance (not the student's), and those are the what students have received today - with 39% of grades lowered by ofqual to worse results than predicted, in some cases much worse, and students in effect competed against their fellow students to get what grades were available for a school based upon past results, rather than the actual exam or any personal talent.
Some unis are being more lenient than others in letting lower than expected grades result in a place anyway; others are saying 'bad luck chum'.
A much higher number are also entering clearing because the uni half the country away with no digs available now is suddenly a lot less appealing than it was in January, and they want to stay closer to home.
Students who would normally now enter clearing, are now left with the unenviable position of deciding if they want to
1) wait for their school to appeal the entire set of results for that school, which may even result in large scale regrading.
2) They can also try and take an actual exam in october to get a better grade.
3) go through some as yet undefined appeal so they get at least their mock grade result, announced last night.
But uni starts in september, and places will not remain open, as 'clearing' fills them up. So the other option is
4) to accept the lower grades and try to make the best of it in a hurry, and end up in any uni that'll take them with their poorer results. Even if their grades are subsequently raised on appeal, they're fairly locked into the uni they've now accepted, unless the nope out and try again next year - when unemployment is heading towards record levels, and a 'gap year' job is going to be hard to find, while bumming round europe (another traditional choice) is not on the cards either.
Basically, if you went to a school that's not an exam factory top-flight (i.e. high end private school), you're probably screwed, and what you do about it has to be decided in short order.
posted by Absolutely No You-Know-What at 9:05 AM on August 13, 2020 [25 favorites]
thanks very much for that hepi article in particular, lucidium - highly recommend reading the comments on it, which cover a lot of other points that aren't directly addressed in the article.
i don't have a problem with applying a degree of standardisation (or with the thousands of downgrades that might result, given the inflated cag figures) or with doing so via an algorithm (because how else are you going to do it), and i recognise that any standardisation model would result in some unfair individual outcomes, because the model will never be perfect. but given how much of an impact a-level results have on students' lives, you've got to be sure that your model is as good as it can possibly be for both aggregate and individual outcomes, and have a decent appeals process with enough time and resources for that process to conclude before university acceptance deadlines. this was a dismal failure on both counts, and it sounds like the cag process was a core part of that.
a few comments from the hepi article that i found interesting, about how predicted grades (not cags) could have been used to achieve fairer individual outcomes without blowing up aggregate integrity, despite the fact that predicted grades are often wildly out of sync with actual exam results (which is what the cag process was intended to address, i believe):
"UCAS predicted grades look awful. The mean and the distribution look nothing like achieved. Its easy to find stats that make them look ridiculous. But, predicted grades are misunderstood. They are estimates of the likely upper bound of attainment, roughly top quintile. Which is what you need if you are a university making conditional offers on potential. Also, be wary of asumming that a difference between predicted and exam awarded means an error in predicted. Exam awarded grades have high inherent randomness themselves. Error if you prefer.
You can control this upper-quintile property out. Further university admissions are about individuals not exams. The UCAS *set* of grades contains much more information than a single predicted grade. And the course that someone holds an offer with is very important too. Put these together in proper statistical models, which we have done for some, and you get a very accurate estimate of the grades someone would have got in exams. And because the predicted grades were collected as normal pre-pandemic than are unbiased and you can test their equality properties against previous years. We say more about this approach here – https://wonkhe.com/blogs/we-can-make-admissions-work-without-a-levels/"
"I’m pointing to the difference between definitive accuracy of predicted grades (a point to point mapping) & them doing their job of predicted what range of grades a student has the potential to achieve. On the latter they are actually very good, and can be modelled to allow for fluctuations to a very high resolution level at every uni in the land.
UCAS holds years of data on this, and could have acted on its duty to fair admissions by releasing this data and equipping the sector to u/stand how predicted grades work against multi dimensions and what normal achievement looks like. It’s really simple to do. They have not released this data (and yes I think that is wrong), nor have they even discussed the power of them with DfE/SQA/OfS/Ofqual. So modelling predicted to achieved grades is very accurate way of understanding what an individual would likely have achieved in normal exam circumstances, much more than the CAG."
"The huge scope of the SQA/Ofqual job – everyone, in every exam – did make it more or less impossible. Perhaps they should have put more weight on deviations from what would have been expected from individual-level (rather than aggregate) prior attainment. But that wouldn’t have worked in all cases. Highlighting that exam-awarded grades have high randomness even in good circumstances would have been helpful and pointed towards different solutions. But overall – for what they have been asked to do in difficult circumstances – they have come up with a solution that has some terrible problems, but perhaps not more so than other things that could have been done.
Where things could have been much better is in university entry. The awarding bodies tend to overlook this not-incidental use of the grades but it is the urgent and large-scale impact on life chances. Here careful processing of the pre-pandemic UCAS predicted grades were the perfect answer for the large majority of university applicants. We could have placed everyone fairly by now. The Government decision not to allow early confirmations was unhelpful, and the lack of a central large-scale effort to provide universities and schools with estimates of normal attainment based on the UCAS predicted grades has been a serious omission.
UCAS’ decision not to help students by releasing the the predicted x achieved matrices, essentially a tool to know if their calculated grades have gone wrong, is as mystifying as it is damaging. The claim that this can’t be done without disclosing personal information for large numbers of students is unsubstantiated and simply not very credible.
The part of the equation which isn’t being given enough attention is the hastily introduced number controls for English universities in particular. These are choking off the ability of universities to use their judgement to redress the wrongs introduced by not having exams. You could award everyone A*/D* but that doesn’t solve the problem of not enough places. This is the key problem and one the Government still has time to solve by relaxing number controls."
posted by inire at 9:06 AM on August 13, 2020 [2 favorites]
i don't have a problem with applying a degree of standardisation (or with the thousands of downgrades that might result, given the inflated cag figures) or with doing so via an algorithm (because how else are you going to do it), and i recognise that any standardisation model would result in some unfair individual outcomes, because the model will never be perfect. but given how much of an impact a-level results have on students' lives, you've got to be sure that your model is as good as it can possibly be for both aggregate and individual outcomes, and have a decent appeals process with enough time and resources for that process to conclude before university acceptance deadlines. this was a dismal failure on both counts, and it sounds like the cag process was a core part of that.
a few comments from the hepi article that i found interesting, about how predicted grades (not cags) could have been used to achieve fairer individual outcomes without blowing up aggregate integrity, despite the fact that predicted grades are often wildly out of sync with actual exam results (which is what the cag process was intended to address, i believe):
"UCAS predicted grades look awful. The mean and the distribution look nothing like achieved. Its easy to find stats that make them look ridiculous. But, predicted grades are misunderstood. They are estimates of the likely upper bound of attainment, roughly top quintile. Which is what you need if you are a university making conditional offers on potential. Also, be wary of asumming that a difference between predicted and exam awarded means an error in predicted. Exam awarded grades have high inherent randomness themselves. Error if you prefer.
You can control this upper-quintile property out. Further university admissions are about individuals not exams. The UCAS *set* of grades contains much more information than a single predicted grade. And the course that someone holds an offer with is very important too. Put these together in proper statistical models, which we have done for some, and you get a very accurate estimate of the grades someone would have got in exams. And because the predicted grades were collected as normal pre-pandemic than are unbiased and you can test their equality properties against previous years. We say more about this approach here – https://wonkhe.com/blogs/we-can-make-admissions-work-without-a-levels/"
"I’m pointing to the difference between definitive accuracy of predicted grades (a point to point mapping) & them doing their job of predicted what range of grades a student has the potential to achieve. On the latter they are actually very good, and can be modelled to allow for fluctuations to a very high resolution level at every uni in the land.
UCAS holds years of data on this, and could have acted on its duty to fair admissions by releasing this data and equipping the sector to u/stand how predicted grades work against multi dimensions and what normal achievement looks like. It’s really simple to do. They have not released this data (and yes I think that is wrong), nor have they even discussed the power of them with DfE/SQA/OfS/Ofqual. So modelling predicted to achieved grades is very accurate way of understanding what an individual would likely have achieved in normal exam circumstances, much more than the CAG."
"The huge scope of the SQA/Ofqual job – everyone, in every exam – did make it more or less impossible. Perhaps they should have put more weight on deviations from what would have been expected from individual-level (rather than aggregate) prior attainment. But that wouldn’t have worked in all cases. Highlighting that exam-awarded grades have high randomness even in good circumstances would have been helpful and pointed towards different solutions. But overall – for what they have been asked to do in difficult circumstances – they have come up with a solution that has some terrible problems, but perhaps not more so than other things that could have been done.
Where things could have been much better is in university entry. The awarding bodies tend to overlook this not-incidental use of the grades but it is the urgent and large-scale impact on life chances. Here careful processing of the pre-pandemic UCAS predicted grades were the perfect answer for the large majority of university applicants. We could have placed everyone fairly by now. The Government decision not to allow early confirmations was unhelpful, and the lack of a central large-scale effort to provide universities and schools with estimates of normal attainment based on the UCAS predicted grades has been a serious omission.
UCAS’ decision not to help students by releasing the the predicted x achieved matrices, essentially a tool to know if their calculated grades have gone wrong, is as mystifying as it is damaging. The claim that this can’t be done without disclosing personal information for large numbers of students is unsubstantiated and simply not very credible.
The part of the equation which isn’t being given enough attention is the hastily introduced number controls for English universities in particular. These are choking off the ability of universities to use their judgement to redress the wrongs introduced by not having exams. You could award everyone A*/D* but that doesn’t solve the problem of not enough places. This is the key problem and one the Government still has time to solve by relaxing number controls."
posted by inire at 9:06 AM on August 13, 2020 [2 favorites]
also i note this is yet another instance of stack ranking fucking things up
posted by inire at 9:16 AM on August 13, 2020 [8 favorites]
posted by inire at 9:16 AM on August 13, 2020 [8 favorites]
last comment, but this other hepi article gives a good explanation of the ways in which the cags are likely to have ended up being inflated (tl;dr it's more likely to be tragedy of the commons and lack of guidance for teachers, rather than over-optimistic teachers, systemic bias against disadvantaged pupils, or schools seeking to game the process).
"If teachers had been instructed how to do the rounding, if teachers had been instructed just how close they had to be to the average and if teachers had been given the same calculation tool that looked after all this techy stuff consistently and ‘behind the scenes’, then they might have submitted CAGs-that-the-algorithm-first-thought-of, these being the ‘right answers’. And even better if they had also been allowed to submit well-evidenced outliers.
But in the absence of these rules, teachers were aiming at moving goalposts in the dark. No wonder there have been so many misses.
My thesis is that ‘plausible overbids’ are not the fault of the teachers. To me, the blame lies at the door of the SQA and Ofqual for not making the rules clear. (Chancers and game players are another matter, of course.)"
posted by inire at 9:30 AM on August 13, 2020 [1 favorite]
"If teachers had been instructed how to do the rounding, if teachers had been instructed just how close they had to be to the average and if teachers had been given the same calculation tool that looked after all this techy stuff consistently and ‘behind the scenes’, then they might have submitted CAGs-that-the-algorithm-first-thought-of, these being the ‘right answers’. And even better if they had also been allowed to submit well-evidenced outliers.
But in the absence of these rules, teachers were aiming at moving goalposts in the dark. No wonder there have been so many misses.
My thesis is that ‘plausible overbids’ are not the fault of the teachers. To me, the blame lies at the door of the SQA and Ofqual for not making the rules clear. (Chancers and game players are another matter, of course.)"
posted by inire at 9:30 AM on August 13, 2020 [1 favorite]
Fundamentally, you cannot assign grades rigorously in a system that relies on exams when you don't do the exams. There is no way of squaring that circle. Here is how a few other European countries have done this:
-Germany: were able to do their exams, some students protested against this but the exams went ahead.
-Italy: oral exams happened, written exams cancelled
-France: cancelled bacc exams (apparently for the first time ever) and based on coursework and tests. Since French universities have to take students with passing marks, they've had to create 10,000 extra places for September.
-Netherlands: cancelled central exams. Noticed substantially higher pass rate. Almost 5x as many schools as usual had everyone pass.
-Spain: carried out the selectividad but under modified conditions
So countries have done one of two things:
-Done the exams
-Not done the exams and used a different method of grading. In basically all cases this has led to grade inflation.
What the UK (Scotland and England both did a variation of this) did is the following:
-Asked teachers to provide expected marks
-Asked teachers to strictly rank order their pupils by expected results
For subjects with lots of people taking them like English and Maths, they fitted the rank ordered grades to that schools average results for the last few years in that subject. So if the top 10% usually get As, the top 10% ranked will get those As this year.
If it was a subject with a small number, they used the predicted grades.
Since private schools typically get better exam results, this has led to less downwards adjustment there. However this was not inevitable. It is easy enough to say that a system that revises down 40% of all marks is obviously wrong, and my first response was to wonder what on earth the teachers were doing with their grade forecasting. Did they really think that based on the available evidence, their pupils this year were performing substantially better than in the previous few years? If not, why have they given them expected grades that reflect that?
On reflection though and reading some of the comments at that hepi link, it makes more sense. Basically predicted grades are always systematically an upper bound, that's true in every year and in fact, the average downgrade is about 40% so it looks like in the aggregate the expected pattern has manifested itself here.
In Scotland, they've essentially backed down and just given everyone the grades their teachers forecast for them after all. I'm undecided whether that was the right thing to do or just cowardice. It's not like they didn't know this was coming.
In England they've given them the possibility of relying on mock exam results.
Can someone from the UK explicate the background here? What is a mock exam? Is this like the pre-SAT and SAT, where kids took a set of tests twice?
It's an opportunity to do an exam a few months before the actual one, using actual papers and under actual conditions. The problem with letting them use those grades is that:
-Basically no-one does better on mocks than the actual exam because it's the same exam but taken without the months of revision leading up to it.
-Not all schools do them under proper exam conditions, some don't test material not taught yet, some don't even keep the grades.
If you're an average to good student at a "normal" school, this process has probably correctly predicted the grades you were going to get. That being said, there are a few categories of exam takers who will be disadvantaged:
-Anyone who got ranked lower by their teacher than they should have been, maybe they were quietly mediocre and looking at a likely B based on hard work but the teacher has put them in a place where they'll get a C
-Anyone at a school that has been improving rapidly
-Anyone who would have put a last burst of work in to revise in the weeks before exams
-Anyone in a class with an unusually strong cohort. So if you class usually has 10 As but all of a sudden this year it has 20 who would have gotten As, there will be 10 people done out of a grade. It won't happen too often but there must be individual cases where it does.
Basically, if you went to a school that's not an exam factory top-flight (i.e. high end private school), you're probably screwed, and what you do about it has to be decided in short order.
The most obviously bright pupils at mediocre schools will actually do ok most of the time out of this as their teachers will rank them highly, the ones who will suffer are the ones a tier or so down who haven't stood out but could have got good enough results to get into some kind of tertiary education. In a school that gets a lot of Cs and Ds, it can be easy to end up somewhere in the undistinguished middle and get assigned a C that really should have been a B.
Of course, universities will be desperate for intake and missing their foreign students so they will accept grades that they ordinarily might not. Arguably this is an argument for just giving people their inflated predicted grades but putting an asterisk next to them - given that A level grades are really only used once and that's for university entry that year or the next no-one will really care. In other words - make the universities deal with it.
also i note this is yet another instance of stack ranking fucking things up
Is it? Stack ranking is usually considered unfair because there's implicitly no distribution that the ranking is fitted to. It's assumed that if you're ranked in the bottom X% you are bad at your job and have to go which means that team of consistent high performers is under pressure to fire it's bottom decile regardless of whether they are any good.
That's not the case here since the distribution is based on previous results.
posted by atrazine at 9:31 AM on August 13, 2020 [3 favorites]
-Germany: were able to do their exams, some students protested against this but the exams went ahead.
-Italy: oral exams happened, written exams cancelled
-France: cancelled bacc exams (apparently for the first time ever) and based on coursework and tests. Since French universities have to take students with passing marks, they've had to create 10,000 extra places for September.
-Netherlands: cancelled central exams. Noticed substantially higher pass rate. Almost 5x as many schools as usual had everyone pass.
-Spain: carried out the selectividad but under modified conditions
So countries have done one of two things:
-Done the exams
-Not done the exams and used a different method of grading. In basically all cases this has led to grade inflation.
What the UK (Scotland and England both did a variation of this) did is the following:
-Asked teachers to provide expected marks
-Asked teachers to strictly rank order their pupils by expected results
For subjects with lots of people taking them like English and Maths, they fitted the rank ordered grades to that schools average results for the last few years in that subject. So if the top 10% usually get As, the top 10% ranked will get those As this year.
If it was a subject with a small number, they used the predicted grades.
Since private schools typically get better exam results, this has led to less downwards adjustment there. However this was not inevitable. It is easy enough to say that a system that revises down 40% of all marks is obviously wrong, and my first response was to wonder what on earth the teachers were doing with their grade forecasting. Did they really think that based on the available evidence, their pupils this year were performing substantially better than in the previous few years? If not, why have they given them expected grades that reflect that?
On reflection though and reading some of the comments at that hepi link, it makes more sense. Basically predicted grades are always systematically an upper bound, that's true in every year and in fact, the average downgrade is about 40% so it looks like in the aggregate the expected pattern has manifested itself here.
In Scotland, they've essentially backed down and just given everyone the grades their teachers forecast for them after all. I'm undecided whether that was the right thing to do or just cowardice. It's not like they didn't know this was coming.
In England they've given them the possibility of relying on mock exam results.
Can someone from the UK explicate the background here? What is a mock exam? Is this like the pre-SAT and SAT, where kids took a set of tests twice?
It's an opportunity to do an exam a few months before the actual one, using actual papers and under actual conditions. The problem with letting them use those grades is that:
-Basically no-one does better on mocks than the actual exam because it's the same exam but taken without the months of revision leading up to it.
-Not all schools do them under proper exam conditions, some don't test material not taught yet, some don't even keep the grades.
If you're an average to good student at a "normal" school, this process has probably correctly predicted the grades you were going to get. That being said, there are a few categories of exam takers who will be disadvantaged:
-Anyone who got ranked lower by their teacher than they should have been, maybe they were quietly mediocre and looking at a likely B based on hard work but the teacher has put them in a place where they'll get a C
-Anyone at a school that has been improving rapidly
-Anyone who would have put a last burst of work in to revise in the weeks before exams
-Anyone in a class with an unusually strong cohort. So if you class usually has 10 As but all of a sudden this year it has 20 who would have gotten As, there will be 10 people done out of a grade. It won't happen too often but there must be individual cases where it does.
Basically, if you went to a school that's not an exam factory top-flight (i.e. high end private school), you're probably screwed, and what you do about it has to be decided in short order.
The most obviously bright pupils at mediocre schools will actually do ok most of the time out of this as their teachers will rank them highly, the ones who will suffer are the ones a tier or so down who haven't stood out but could have got good enough results to get into some kind of tertiary education. In a school that gets a lot of Cs and Ds, it can be easy to end up somewhere in the undistinguished middle and get assigned a C that really should have been a B.
Of course, universities will be desperate for intake and missing their foreign students so they will accept grades that they ordinarily might not. Arguably this is an argument for just giving people their inflated predicted grades but putting an asterisk next to them - given that A level grades are really only used once and that's for university entry that year or the next no-one will really care. In other words - make the universities deal with it.
also i note this is yet another instance of stack ranking fucking things up
Is it? Stack ranking is usually considered unfair because there's implicitly no distribution that the ranking is fitted to. It's assumed that if you're ranked in the bottom X% you are bad at your job and have to go which means that team of consistent high performers is under pressure to fire it's bottom decile regardless of whether they are any good.
That's not the case here since the distribution is based on previous results.
posted by atrazine at 9:31 AM on August 13, 2020 [3 favorites]
But we know this was a weird year, and any solution is going to be imperfect. Using a broad statistical measure to weigh down students grades will obviously be unfair, and it will be statistically unfair. That is, you are applying a mean measure to individuals. You are basically baking in inequality.
But this is the English notion of fairness in education as I understand it. Draft an arbitrary, blanket rule, then apply ruthlessly with an excess of bureaucracy. It's basically applying that Anatole France quote about sleeping under bridges as a guiding principle.
If students get the real grades necessary for one of their offers in August, i.e. today, their provisional place is confirmed. If they're close, some universities will let them keep the place anyway (but usually not the top-tier unis). If they fail to get the necessary grades for any of their offers, of have changed their mind about where they want to go, they go into "clearing" - i.e. a system which tries to match up students with available places that meet their grades - and you may have very little time to decide if you fancy going on a different course or uni in a totally different part of the country than before, when some unis will retain at least some distance learning, and/or on-campus housing may not be available.
And if all that's not bad enough, forcing desperate students to call around begging for admission, the ones fielding those calls aren't American-style admissions officers, or actually anyone working in administration. They're lecturers on the course (major) students are trying to apply to. Already overworked, underpaid lecturers who aren't trained as crisis counselors but nevertheless get put in that position.
posted by HumuloneRanger at 9:40 AM on August 13, 2020 [7 favorites]
But this is the English notion of fairness in education as I understand it. Draft an arbitrary, blanket rule, then apply ruthlessly with an excess of bureaucracy. It's basically applying that Anatole France quote about sleeping under bridges as a guiding principle.
If students get the real grades necessary for one of their offers in August, i.e. today, their provisional place is confirmed. If they're close, some universities will let them keep the place anyway (but usually not the top-tier unis). If they fail to get the necessary grades for any of their offers, of have changed their mind about where they want to go, they go into "clearing" - i.e. a system which tries to match up students with available places that meet their grades - and you may have very little time to decide if you fancy going on a different course or uni in a totally different part of the country than before, when some unis will retain at least some distance learning, and/or on-campus housing may not be available.
And if all that's not bad enough, forcing desperate students to call around begging for admission, the ones fielding those calls aren't American-style admissions officers, or actually anyone working in administration. They're lecturers on the course (major) students are trying to apply to. Already overworked, underpaid lecturers who aren't trained as crisis counselors but nevertheless get put in that position.
posted by HumuloneRanger at 9:40 AM on August 13, 2020 [7 favorites]
Is it? Stack ranking is usually considered unfair because there's implicitly no distribution that the ranking is fitted to. It's assumed that if you're ranked in the bottom X% you are bad at your job and have to go which means that team of consistent high performers is under pressure to fire it's bottom decile regardless of whether they are any good.
That's not the case here since the distribution is based on previous results.
hand on heart, my unquenchable thirst for favourites and knowledge of mefi's prejudices drove me to post a glib zinger. it won't happen again (this week).
but also, my understanding was that the use of a ranking system (any ranking system) was a big part of the issue here. unlike predicting a pupil's expected grade, teachers are completely unfamiliar with ranking, and ranking an entire cohort for ability (in order, with no equal rankings permitted) is an absurdly unreliable method to achieve fair individual results (as in, reflective of individual ability) in any sizeable cohort, even if you fit it to a distribution based on previous results.
posted by inire at 9:52 AM on August 13, 2020 [11 favorites]
That's not the case here since the distribution is based on previous results.
hand on heart, my unquenchable thirst for favourites and knowledge of mefi's prejudices drove me to post a glib zinger. it won't happen again (this week).
but also, my understanding was that the use of a ranking system (any ranking system) was a big part of the issue here. unlike predicting a pupil's expected grade, teachers are completely unfamiliar with ranking, and ranking an entire cohort for ability (in order, with no equal rankings permitted) is an absurdly unreliable method to achieve fair individual results (as in, reflective of individual ability) in any sizeable cohort, even if you fit it to a distribution based on previous results.
posted by inire at 9:52 AM on August 13, 2020 [11 favorites]
Dr Michelle Meadows, executive director for strategy, risk and research at Ofqual, said on Thursday that there is "no evidence of systematic bias" in the moderation system.So there's no evidence of systematic bias. You can tell by looking at this evidence of systematic bias.
This being promoted above one's abilities thing is spreading like corona.
posted by fullerine at 10:24 AM on August 13, 2020 [2 favorites]
If you're an average to good student at a "normal" school, this process has probably correctly predicted the grades you were going to get. That being said, there are a few categories of exam takers who will be disadvantaged:
You missed a category. Bright students in 'bog standard' or weak schools. If your school say had the best result be Cs in your subject in the last couple of years, then if you're the best in your class in a popularish subject, you get a C. You might have been predicted an A*, but that grade wasn't available for you cos your parents couldn't afford a more expensive house in the cachement area of a better school. That AAB offer you worked damn hard for? Better hope you like that CCD course in a different uni instead, or take a chance on the mocks appeals system and hope that AAB place still exists in a few weeks.
If you take that CCD place - welp, the student that would have had it gets to go somewhere else.
The fundamental flaw of ranking students within a subject is that the school itself was also ranked. We're seeing growing reports of schools getting downgraded to well below their statistical mean, because the higher grades were already taken by 'better' schools and we can't let that country-wide average go up!
I cannot think of a better way to reinforce the class system than this, and it is *unfair*.
I was very lucky when I was young - my parents could afford a house next to a good school, I got good grades and got my 1st place offer. I saw several friends go through clearing, and it was *brutal* even back then. I then dropped out in my final year (based upon later experience, it was probably undiagnosed depression), didn't graduate, and that's blighted my career for ooh, 20 years. But at least I still had 4 good A levels, 2 years at a good uni, and I have a loving family with a decent job at an ok-ish salary. At a private school, ironically.
More than 50% of students don't go on to university. More will drop out, and they will be going into the working world, many with lower A-level grades than they deserve and they're going into the biggest recession in possibly ever, when job losses are soaring, and 100s of people are competing for every shit job out there. Those results will also be needed to decide if they qualify for apprenticeships or further education later in life. (I had to dig out my GCSE certificates some months ago to prove I could get on a management course, which I haven't even seen for *mumble* years)
On reflection, I'm not surprised - this Trump-like government does everything it can to keep the boot on the neck of the poor and disadvantaged, while feeding them bullshit all day about how it's all the fault of the foreigners taking their opportunities away from them.
We could have assessed students on coursework, or modules (AS levels) but they were pretty much all scrapped in England a little way back because girls did too well and Mr Gove decided the best way to improve results for boys was to make it all about make-or-break final exams where boys historically do better. Except, of course, we decided to cancel them.
But for me, those wittering on about 'the integrity of the exams system'* when 10s of thousands students have just been substantially disadvantaged for years in the middle of a once-in-a-century pandemic because they didn't go to a private school (whose results of course improved) butters very few fucking parsnips.
* this is not directed at anyone here, to be clear
posted by Absolutely No You-Know-What at 10:45 AM on August 13, 2020 [23 favorites]
You missed a category. Bright students in 'bog standard' or weak schools. If your school say had the best result be Cs in your subject in the last couple of years, then if you're the best in your class in a popularish subject, you get a C. You might have been predicted an A*, but that grade wasn't available for you cos your parents couldn't afford a more expensive house in the cachement area of a better school. That AAB offer you worked damn hard for? Better hope you like that CCD course in a different uni instead, or take a chance on the mocks appeals system and hope that AAB place still exists in a few weeks.
If you take that CCD place - welp, the student that would have had it gets to go somewhere else.
The fundamental flaw of ranking students within a subject is that the school itself was also ranked. We're seeing growing reports of schools getting downgraded to well below their statistical mean, because the higher grades were already taken by 'better' schools and we can't let that country-wide average go up!
I cannot think of a better way to reinforce the class system than this, and it is *unfair*.
I was very lucky when I was young - my parents could afford a house next to a good school, I got good grades and got my 1st place offer. I saw several friends go through clearing, and it was *brutal* even back then. I then dropped out in my final year (based upon later experience, it was probably undiagnosed depression), didn't graduate, and that's blighted my career for ooh, 20 years. But at least I still had 4 good A levels, 2 years at a good uni, and I have a loving family with a decent job at an ok-ish salary. At a private school, ironically.
More than 50% of students don't go on to university. More will drop out, and they will be going into the working world, many with lower A-level grades than they deserve and they're going into the biggest recession in possibly ever, when job losses are soaring, and 100s of people are competing for every shit job out there. Those results will also be needed to decide if they qualify for apprenticeships or further education later in life. (I had to dig out my GCSE certificates some months ago to prove I could get on a management course, which I haven't even seen for *mumble* years)
On reflection, I'm not surprised - this Trump-like government does everything it can to keep the boot on the neck of the poor and disadvantaged, while feeding them bullshit all day about how it's all the fault of the foreigners taking their opportunities away from them.
We could have assessed students on coursework, or modules (AS levels) but they were pretty much all scrapped in England a little way back because girls did too well and Mr Gove decided the best way to improve results for boys was to make it all about make-or-break final exams where boys historically do better. Except, of course, we decided to cancel them.
But for me, those wittering on about 'the integrity of the exams system'* when 10s of thousands students have just been substantially disadvantaged for years in the middle of a once-in-a-century pandemic because they didn't go to a private school (whose results of course improved) butters very few fucking parsnips.
* this is not directed at anyone here, to be clear
posted by Absolutely No You-Know-What at 10:45 AM on August 13, 2020 [23 favorites]
Some great explanations and commentary here. As an Australian in the UK with a kid only 2 years away from Uni applications (and he is home educated to boot) the ‘expected grades’/provisional offer system of allocating university places seems rather ridiculous.
This was how it was done in Canada when I went to University back in the eighties. The universities made their decisions before the high school year is up so they used mid-term grades from your final semester (the now defunct grade 13 in my case) in their decision. It would have taken an epic screwup for your grade to change much because the teachers knew the importance and this was well before the runaway grade inflation of today but it could still happen.
Without this system students wouldn't get an acceptance until after they had finished high school and their application had been processed. There are just two months between the end of the high school semester and the start of university (less if you have an orientation week for uni) for that to happen it and it would leave no time for students to sort out off campus housing if that was how they wanted to go.
posted by srboisvert at 10:49 AM on August 13, 2020
This was how it was done in Canada when I went to University back in the eighties. The universities made their decisions before the high school year is up so they used mid-term grades from your final semester (the now defunct grade 13 in my case) in their decision. It would have taken an epic screwup for your grade to change much because the teachers knew the importance and this was well before the runaway grade inflation of today but it could still happen.
Without this system students wouldn't get an acceptance until after they had finished high school and their application had been processed. There are just two months between the end of the high school semester and the start of university (less if you have an orientation week for uni) for that to happen it and it would leave no time for students to sort out off campus housing if that was how they wanted to go.
posted by srboisvert at 10:49 AM on August 13, 2020
Guardian has data suggesting private schools had above average gains in top grades.
posted by biffa at 10:58 AM on August 13, 2020 [5 favorites]
posted by biffa at 10:58 AM on August 13, 2020 [5 favorites]
Maybe something that a UK person could clarify; are the grades relatively big categories (A*, A, B, C) each represented a substantial fraction, or are numeric values given? Is A* a super-star rating like top 0.1%?
If they are fairly broad categories, the in-class rank to letter grade is likely to be pretty stable. The year to year variability in within-school within-course grades (including cohort variability) is an observable that can be used to generate approximate uncertainty. It is unfortunate but true that relatively stable socioeconomic facts have a large influence on likely educational outcomes. Rapid change in overall school performance is also likely able to be bounded from prior year-year changes and other data (like changes in mocks).
That teacher assessments are measuring something different than exams is unfortunate but hard to get around. Teacher assessments tend to be pretty meaningful though, and in many contexts do better than test scores in predicting future performance.
posted by a robot made out of meat at 11:05 AM on August 13, 2020
If they are fairly broad categories, the in-class rank to letter grade is likely to be pretty stable. The year to year variability in within-school within-course grades (including cohort variability) is an observable that can be used to generate approximate uncertainty. It is unfortunate but true that relatively stable socioeconomic facts have a large influence on likely educational outcomes. Rapid change in overall school performance is also likely able to be bounded from prior year-year changes and other data (like changes in mocks).
That teacher assessments are measuring something different than exams is unfortunate but hard to get around. Teacher assessments tend to be pretty meaningful though, and in many contexts do better than test scores in predicting future performance.
posted by a robot made out of meat at 11:05 AM on August 13, 2020
Absolutely No You-Know-What highlights a huge part of the problem.
As I understand it, if you are a very bright pupil in a very poorly performing school, and your true ability is A in a subject and you are predicted as an A in that subject, but no one in your school has got an A in your subject in the last three years, Ofqual's algorithm will likely downgrade you. Possibly by more than one grade.
Nothing to do with your academic ability. It's like losing out in a job interview to a ghost.
And yes, private schools have done better, because as I understand it the algorithm assumes that very small number-of-pupil A levels cannot have the same statistical rules applied about previous performance, and so places more weight on teacher prediction.
So, Ofqual would tell you, it's not advantaging public schools, it's advantaging small niche A level subjects in schools.
Which of course state schools usually can't afford to run. But private schools can. So...
posted by reynir at 11:26 AM on August 13, 2020 [14 favorites]
As I understand it, if you are a very bright pupil in a very poorly performing school, and your true ability is A in a subject and you are predicted as an A in that subject, but no one in your school has got an A in your subject in the last three years, Ofqual's algorithm will likely downgrade you. Possibly by more than one grade.
Nothing to do with your academic ability. It's like losing out in a job interview to a ghost.
And yes, private schools have done better, because as I understand it the algorithm assumes that very small number-of-pupil A levels cannot have the same statistical rules applied about previous performance, and so places more weight on teacher prediction.
So, Ofqual would tell you, it's not advantaging public schools, it's advantaging small niche A level subjects in schools.
Which of course state schools usually can't afford to run. But private schools can. So...
posted by reynir at 11:26 AM on August 13, 2020 [14 favorites]
Maybe something that a UK person could clarify; are the grades relatively big categories (A*, A, B, C) each represented a substantial fraction, or are numeric values given? Is A* a super-star rating like top 0.1%?
Broad-ish. as a nominal level, if I remember correctly, A* is 90%+, A is 80% plus, B is 70%+, C is 60%+ etc
However, you're going to love the next bit. In normal years, England & Wales do not set exams at a national level. (Scotland does its own thing, no idea how that works). Instead, we have private orgs that write exam papers based nominally upon the National Curriculum (the set of things you're expected to study, and how). The school chooses the exam board they wish to use for a given subject and teaches the material that board expects for 2 years; the school gets sent the physical papers, the students do them, and they get sent off for marking based upon the board's marking criteria. That gives you your nominal grades.
Ofqual, the overseeing org, then looks at all the grades assigned by all the boards, and (simplifying massively) goes "hmm. AQA board results in French were better this year than everyone else. Gonna mark them all down 5 points cos it must have been too easy. Edexcel were too low, we're going to mark them up 2 points."
Then they look at the overall results. Did too many students overall get A* in French this year compared to last year? Well, then we move the grade boundary for A* to say 92% so less students qualify. Did too few get Ds? Let's lower the boundary a bit, bumping previously E grade students to a D.
With this adjustment process, they usually allow results to improve year on year by about 2%, and no more, regardless of how many students would or wouldn't get the 'raw' result. Results keep improving naturally (students getting better or exams getting easier, you pick) but ofqual ensures the system is "fair".
This year, they pretty much went "Too many schools predicted students got an A* in French than last year. We have no idea what percentage of students best deserve one, so we'll pick the best schools and give them slightly more than the amount of A* results they got last year, allocated to the top individually ranked French students in that school, and so on until we run out of A*s we're gonna allow. We do the same for the Bs, Cs and so on."
The English & Welsh A level exams system is already fucking odd. This year is much worse.
posted by Absolutely No You-Know-What at 11:34 AM on August 13, 2020 [6 favorites]
Broad-ish. as a nominal level, if I remember correctly, A* is 90%+, A is 80% plus, B is 70%+, C is 60%+ etc
However, you're going to love the next bit. In normal years, England & Wales do not set exams at a national level. (Scotland does its own thing, no idea how that works). Instead, we have private orgs that write exam papers based nominally upon the National Curriculum (the set of things you're expected to study, and how). The school chooses the exam board they wish to use for a given subject and teaches the material that board expects for 2 years; the school gets sent the physical papers, the students do them, and they get sent off for marking based upon the board's marking criteria. That gives you your nominal grades.
Ofqual, the overseeing org, then looks at all the grades assigned by all the boards, and (simplifying massively) goes "hmm. AQA board results in French were better this year than everyone else. Gonna mark them all down 5 points cos it must have been too easy. Edexcel were too low, we're going to mark them up 2 points."
Then they look at the overall results. Did too many students overall get A* in French this year compared to last year? Well, then we move the grade boundary for A* to say 92% so less students qualify. Did too few get Ds? Let's lower the boundary a bit, bumping previously E grade students to a D.
With this adjustment process, they usually allow results to improve year on year by about 2%, and no more, regardless of how many students would or wouldn't get the 'raw' result. Results keep improving naturally (students getting better or exams getting easier, you pick) but ofqual ensures the system is "fair".
This year, they pretty much went "Too many schools predicted students got an A* in French than last year. We have no idea what percentage of students best deserve one, so we'll pick the best schools and give them slightly more than the amount of A* results they got last year, allocated to the top individually ranked French students in that school, and so on until we run out of A*s we're gonna allow. We do the same for the Bs, Cs and so on."
The English & Welsh A level exams system is already fucking odd. This year is much worse.
posted by Absolutely No You-Know-What at 11:34 AM on August 13, 2020 [6 favorites]
I actually benefited from something similar like this because I went to a shit school back home and our tertiary entrance scores were both ranked and adjusted as per our school's potential. I made it into university successfully and got my degree.
The obvious answer band-aid solution is that there needs to be two grades, the raw score and the adjusted score. Raw scores for straight qualification into whatever courses, adjusted scores for plotting that raw score vs the socio-economic and performance of the school in general and then allocating government funded spots from those adjusted scores. You want to get in from an elite school? You gotta be the best of the best, not just able to afford the tutors.
It's not surprising that the Tories want to end adjusted grading. If they could they'd just skip the whole rigmarole of testing and let the economic elite buy grades wholesale anyway. Competency is just an illusion to these fuckers anyway.
posted by Your Childhood Pet Rock at 11:40 AM on August 13, 2020 [2 favorites]
The obvious answer band-aid solution is that there needs to be two grades, the raw score and the adjusted score. Raw scores for straight qualification into whatever courses, adjusted scores for plotting that raw score vs the socio-economic and performance of the school in general and then allocating government funded spots from those adjusted scores. You want to get in from an elite school? You gotta be the best of the best, not just able to afford the tutors.
It's not surprising that the Tories want to end adjusted grading. If they could they'd just skip the whole rigmarole of testing and let the economic elite buy grades wholesale anyway. Competency is just an illusion to these fuckers anyway.
posted by Your Childhood Pet Rock at 11:40 AM on August 13, 2020 [2 favorites]
The English & Welsh A level exams system is already fucking odd. This year is much worse.
i find the use of multiple private exam boards completely inexplicable (and would be interested if anyone else has an explanation that isn't 'capitalism / tories'), but the normal ofqual standardisation process seems to me to make sense, at least in principle.
given the need for different questions every year, and the impossibility (i presume) of having anything like a consistent difficulty level between questions year after year (especially for non-quantitative subjects?), the 'raw' results without standardisation would presumably be really quite swingy year-on-year.
exam grades are a bit like money - particular grades have value because people believe in the integrity of the system that produced them. large swings each year detract from that, hence the focus on steady outcomes. i don't see how you achieve that without some kind of standardisation process.
posted by inire at 11:44 AM on August 13, 2020
i find the use of multiple private exam boards completely inexplicable (and would be interested if anyone else has an explanation that isn't 'capitalism / tories'), but the normal ofqual standardisation process seems to me to make sense, at least in principle.
given the need for different questions every year, and the impossibility (i presume) of having anything like a consistent difficulty level between questions year after year (especially for non-quantitative subjects?), the 'raw' results without standardisation would presumably be really quite swingy year-on-year.
exam grades are a bit like money - particular grades have value because people believe in the integrity of the system that produced them. large swings each year detract from that, hence the focus on steady outcomes. i don't see how you achieve that without some kind of standardisation process.
posted by inire at 11:44 AM on August 13, 2020
What I feel like is missing in the data is whether or not more students have missed their university places than normal. The by subject results have been moderated down to fit the curve, but have they looked at how that affects each students overall results across their typical 3 subjects, for example by calculated UCAS tariff score.
Clearing, where people who missed the grade try to find an open place at a university after results day, is always a big challenge/nightmare. There are always lots of stories about people scrambling round to find places. Does anyone know if more students are affected this year? Ideally, with covid problems you'd want to fix it so that fewer students ended up in clearing through missing grades.
Maybe something that a UK person could clarify; are the grades relatively big categories (A*, A, B, C) each represented a substantial fraction, or are numeric values given? Is A* a super-star rating like top 0.1%?
It depends on the subject. You can see historic percentiles for different grades. Generally 40+% of Maths A-Levels are an A or A* but only about 17% in Psychology. Those are two of the most popular A-Level subjects in recent years.
posted by plonkee at 11:49 AM on August 13, 2020
Clearing, where people who missed the grade try to find an open place at a university after results day, is always a big challenge/nightmare. There are always lots of stories about people scrambling round to find places. Does anyone know if more students are affected this year? Ideally, with covid problems you'd want to fix it so that fewer students ended up in clearing through missing grades.
Maybe something that a UK person could clarify; are the grades relatively big categories (A*, A, B, C) each represented a substantial fraction, or are numeric values given? Is A* a super-star rating like top 0.1%?
It depends on the subject. You can see historic percentiles for different grades. Generally 40+% of Maths A-Levels are an A or A* but only about 17% in Psychology. Those are two of the most popular A-Level subjects in recent years.
posted by plonkee at 11:49 AM on August 13, 2020
For clarification, it's the competiting exam boards bit that is fucking odd IMO, standardisation is obviously a necessary function given the variability of a given paper for a given subject. There have been examples where a particular question couldn't actually be answered with the information given, so that percentage of the grade was ignored and everybody renormalised as if it had never existed.
Edexcel is owned by Pearson, a profit making company, the rest I believe are non-profit and most are the result of mergers of what used to be regional exam boards. But schools can pick any board for any subject, and usually have a variety. The same exists for GCSE, and schools don't even have to pick the same board for the same subject in GCSE as they do A-level. Nor do boards even have to cover the same material as other boards, or expect it to be tought the same way.
My french wife thinks this is nuts, because the exams are set by the government, and everybody does the same one.
posted by Absolutely No You-Know-What at 11:52 AM on August 13, 2020 [4 favorites]
Edexcel is owned by Pearson, a profit making company, the rest I believe are non-profit and most are the result of mergers of what used to be regional exam boards. But schools can pick any board for any subject, and usually have a variety. The same exists for GCSE, and schools don't even have to pick the same board for the same subject in GCSE as they do A-level. Nor do boards even have to cover the same material as other boards, or expect it to be tought the same way.
My french wife thinks this is nuts, because the exams are set by the government, and everybody does the same one.
posted by Absolutely No You-Know-What at 11:52 AM on August 13, 2020 [4 favorites]
i find the use of multiple private exam boards completely inexplicable
It's for historical reasons. Originally they were the boards that set individual university entrance exams, when there were relatively few universities. There have been a lot of mergers since then but its basically the same system. In the 60s and 70s I believe schools mainly took exams from their local board, but exam shopping by syllabus has been a thing for years.
posted by plonkee at 11:53 AM on August 13, 2020 [1 favorite]
It's for historical reasons. Originally they were the boards that set individual university entrance exams, when there were relatively few universities. There have been a lot of mergers since then but its basically the same system. In the 60s and 70s I believe schools mainly took exams from their local board, but exam shopping by syllabus has been a thing for years.
posted by plonkee at 11:53 AM on August 13, 2020 [1 favorite]
As an A Level teacher I can tell you that the marking of exam papers is hasty and shoddy work. I have in the past paid to see the marked papers because the results are so different from my understanding of my students' abilities. In some cases students had given literally identical answers (I teach Computer Science so in some cases there is a clear correct answer, such as the name of an algorithm) and some were given full marks, and others marked wrong, or given half marks. Also if you pay (£40) to have your paper re-marked they give it to one of their more experienced markers and the grade goes up, in some cases by several tiers.
I attribute all of this to market forces which in my opinion have no place in a fair education system.
posted by communicator at 11:57 AM on August 13, 2020 [26 favorites]
I attribute all of this to market forces which in my opinion have no place in a fair education system.
posted by communicator at 11:57 AM on August 13, 2020 [26 favorites]
And the marking is done poorly because markers are paid pennies per question and given hundreds of papers to mark in a very short time, so even the best subect experts must work hastily, and in general teachers with other options do not choose to be markers. I do have some colleagues who mark papers from a sense of loyalty to their subject and to limit the damage. The influence of money has been highly destructive in my opinion.
posted by communicator at 12:01 PM on August 13, 2020 [19 favorites]
posted by communicator at 12:01 PM on August 13, 2020 [19 favorites]
You missed a category. Bright students in 'bog standard' or weak schools. If your school say had the best result be Cs in your subject in the last couple of years, then if you're the best in your class in a popularish subject, you get a C. You might have been predicted an A*, but that grade wasn't available for you cos your parents couldn't afford a more expensive house in the cachement area of a better school. That AAB offer you worked damn hard for? Better hope you like that CCD course in a different uni instead, or take a chance on the mocks appeals system and hope that AAB place still exists in a few weeks.
Yeah. I'd been thinking of a school where a small number of pupils every year got A*s, in those schools you would be ok because you would be highly ranked and snag one of the few top grades on offer.. Obviously if a school has literally no results at that level that would be impossible. I wonder if they adjusted for that (so that it was never totally impossible for anyone to get an A*).
Guardian has data suggesting private schools had above average gains in top grades.
Interesting that there were any gains in top grades given the model used and that gains were actually the second smallest at selective state secondaries and relatively high at state comprehensives.
What I feel like is missing in the data is whether or not more students have missed their university places than normal. The by subject results have been moderated down to fit the curve, but have they looked at how that affects each students overall results across their typical 3 subjects, for example by calculated UCAS tariff score.
That will ultimately be what matters. I do think it would have saved a lot of aggravation if they'd just accepted that grades would be inflated in this one year because it doesn't matter if they all are and A-level grades are not in practice ever compared across more than a year or two since they are only really used for university entry in the first place.
posted by atrazine at 12:07 PM on August 13, 2020 [2 favorites]
Yeah. I'd been thinking of a school where a small number of pupils every year got A*s, in those schools you would be ok because you would be highly ranked and snag one of the few top grades on offer.. Obviously if a school has literally no results at that level that would be impossible. I wonder if they adjusted for that (so that it was never totally impossible for anyone to get an A*).
Guardian has data suggesting private schools had above average gains in top grades.
Interesting that there were any gains in top grades given the model used and that gains were actually the second smallest at selective state secondaries and relatively high at state comprehensives.
What I feel like is missing in the data is whether or not more students have missed their university places than normal. The by subject results have been moderated down to fit the curve, but have they looked at how that affects each students overall results across their typical 3 subjects, for example by calculated UCAS tariff score.
That will ultimately be what matters. I do think it would have saved a lot of aggravation if they'd just accepted that grades would be inflated in this one year because it doesn't matter if they all are and A-level grades are not in practice ever compared across more than a year or two since they are only really used for university entry in the first place.
posted by atrazine at 12:07 PM on August 13, 2020 [2 favorites]
How the fuck can teachers accurately rank all their pupils by skill, no ties? Nobody can accurately rank, say, all the players on the top tennis circuit, and unlike students there's a) tons of objective data about them, b) there is a single clear objective they need to achieve, and c) they face off head to head on a frequent basis. Yeah, the #1 rank player is almost certainly better than the #28th ranked player, but #28 is not necessarily better than #29.
I went to a high school that happened to be a small school that attracted a small cohort of good math students because we had an excellent math teacher. The year I graduated, we had a larger cohort of good students than most years, so whoever was on the bottom would be downgraded in this system. I was definitely in the top 10 or so students my year out of 100, but it would be arbitrary where in the top 10 I wound up; honestly, I'd probably be somewhere between 4th and 8th, and I was a bit of a trouble maker, so I'd probably be ranked 6th to 10th. I won the city-wide math championship that year.
The problem with the adjustment procedure is given away when they described using the previous several years of school grade distributions. Why several years? It's more work, so that's not the reason. It brings in older, more out-of-date data, so that's not the reason. The obvious reason is that school grade distributions vary substantially from year to year. Which means that using the average will by definition not produce accurate results.
A more fair solution, taking everything else in the system as a given would be to allow teachers to rank students with ties, and then produce 20 random rankings for each class, breaking the ties randomly. Each of the 20 random rankings is then adjusted not by the average of the past several years for that school, but by each of the past years individually. Say the past 5 years are used, now there are 100 grades for each student. If every time the same grade comes up, then great, that's the grade. If multiple different grades come up, then the student gets all of them.
So instead of a school having 10 A* students, 20 A students, 30 B students and so on, there might be 3 A* students, 13 A*/A students, 6 A students, 18 A/B students, 10 B students, 22 B/C students, and so on. It would acknowledge that there's actual uncertainty in the real world (honestly more than the existing system). It would make things a little more complicated for universities, but welcome to 2020, shit's complicated everywhere.
posted by Superilla at 12:15 PM on August 13, 2020 [3 favorites]
I went to a high school that happened to be a small school that attracted a small cohort of good math students because we had an excellent math teacher. The year I graduated, we had a larger cohort of good students than most years, so whoever was on the bottom would be downgraded in this system. I was definitely in the top 10 or so students my year out of 100, but it would be arbitrary where in the top 10 I wound up; honestly, I'd probably be somewhere between 4th and 8th, and I was a bit of a trouble maker, so I'd probably be ranked 6th to 10th. I won the city-wide math championship that year.
The problem with the adjustment procedure is given away when they described using the previous several years of school grade distributions. Why several years? It's more work, so that's not the reason. It brings in older, more out-of-date data, so that's not the reason. The obvious reason is that school grade distributions vary substantially from year to year. Which means that using the average will by definition not produce accurate results.
A more fair solution, taking everything else in the system as a given would be to allow teachers to rank students with ties, and then produce 20 random rankings for each class, breaking the ties randomly. Each of the 20 random rankings is then adjusted not by the average of the past several years for that school, but by each of the past years individually. Say the past 5 years are used, now there are 100 grades for each student. If every time the same grade comes up, then great, that's the grade. If multiple different grades come up, then the student gets all of them.
So instead of a school having 10 A* students, 20 A students, 30 B students and so on, there might be 3 A* students, 13 A*/A students, 6 A students, 18 A/B students, 10 B students, 22 B/C students, and so on. It would acknowledge that there's actual uncertainty in the real world (honestly more than the existing system). It would make things a little more complicated for universities, but welcome to 2020, shit's complicated everywhere.
posted by Superilla at 12:15 PM on August 13, 2020 [3 favorites]
These poor kids. It also looks like the government have successfully recruited an entire generation of anti-Tory voters. Well done.
posted by like_neon at 12:38 PM on August 13, 2020 [5 favorites]
posted by like_neon at 12:38 PM on August 13, 2020 [5 favorites]
a couple of twitter threads, one being a useful overview from a former senior education policy advisor (emphasising the importance of having a functioning appeals process), and one on the ranking-related issues from the chair in statistics at imperial.
posted by inire at 12:40 PM on August 13, 2020 [1 favorite]
posted by inire at 12:40 PM on August 13, 2020 [1 favorite]
I'm just gonna post this, then I promise to stop threadsitting.
Equalities body warns it may step in after A-level downgrades in England
"Analysis by the FFT Education Datalab showed that the quirk in Ofqual’s method – which relied heavily on teacher assessments for grading very small classes with up to five pupils – resulted in higher grades for less popular subjects such as German or music, and for schools that had much smaller average course entries.
About one in 10 A-level entries from independent schools were in subjects in which the schools entered just five or fewer students. In contrast, state academies – the largest school type – had just one entry in 20 within small classes. Sixth form and further education colleges had A-level course sizes more than three times as big as the average private school.
Ofqual’s data showed that 49% of entries by students at private schools in England received an A grade or above, compared with 20% awarded to students at state academies or comprehensive schools.
Johnson insisted he had confidence in Williamson, prompting Labour’s deputy leader, Angela Rayner, to respond on Twitter: “Nobody else does.” Speaking during a trip to Belfast, the prime minister added: “Let’s be in no doubt about it: the exam results that we’ve got today are robust, they’re good, they’re dependable for employers.”
posted by Absolutely No You-Know-What at 1:16 PM on August 13, 2020 [2 favorites]
Equalities body warns it may step in after A-level downgrades in England
"Analysis by the FFT Education Datalab showed that the quirk in Ofqual’s method – which relied heavily on teacher assessments for grading very small classes with up to five pupils – resulted in higher grades for less popular subjects such as German or music, and for schools that had much smaller average course entries.
About one in 10 A-level entries from independent schools were in subjects in which the schools entered just five or fewer students. In contrast, state academies – the largest school type – had just one entry in 20 within small classes. Sixth form and further education colleges had A-level course sizes more than three times as big as the average private school.
Ofqual’s data showed that 49% of entries by students at private schools in England received an A grade or above, compared with 20% awarded to students at state academies or comprehensive schools.
Johnson insisted he had confidence in Williamson, prompting Labour’s deputy leader, Angela Rayner, to respond on Twitter: “Nobody else does.” Speaking during a trip to Belfast, the prime minister added: “Let’s be in no doubt about it: the exam results that we’ve got today are robust, they’re good, they’re dependable for employers.”
posted by Absolutely No You-Know-What at 1:16 PM on August 13, 2020 [2 favorites]
An interesting contrast between Scotland and the rest of the UK in how they have handled this: in elections to the Scottish parliament (which controls education policy), 16 and 17 year olds can vote. Not to mention the next election is May 2021. The Scottish government is directly answerable to these students in a way Westminster isn't.
posted by eykal at 1:42 PM on August 13, 2020 [4 favorites]
posted by eykal at 1:42 PM on August 13, 2020 [4 favorites]
Speaking during a trip to Belfast, the prime minister added: “Let’s be in no doubt about it: the exam results that we’ve got today are robust, they’re good, they’re dependable for employers.”
They're not even fucking exam results, Boris, you clown.
posted by reynir at 1:48 PM on August 13, 2020 [4 favorites]
They're not even fucking exam results, Boris, you clown.
posted by reynir at 1:48 PM on August 13, 2020 [4 favorites]
Given the retweets of various government figures doing the usual "You might not have got the results you hoped for, but don't despair!" crap they always spout, I don't think they seem to understand what's going on.
Williamson tweeted that it's all right because all the kids who were fucked over today could get jobs in the wonderful post-Brexit economy. Then that his son just got four As.
Which is an astounding, Marie Antoinette level of tone deafness.
But then is there any other logic to government policy than trolling people any more?
posted by Grangousier at 2:05 PM on August 13, 2020 [6 favorites]
Williamson tweeted that it's all right because all the kids who were fucked over today could get jobs in the wonderful post-Brexit economy. Then that his son just got four As.
Which is an astounding, Marie Antoinette level of tone deafness.
But then is there any other logic to government policy than trolling people any more?
posted by Grangousier at 2:05 PM on August 13, 2020 [6 favorites]
The sheer gall of Williamson saying "Increasing the A Level grades will mean a whole generation could end up promoted beyond their abilities" when he's a former fireplace salesman with a Bsc in social sciences who ended up as Minister for Education after previously being sacked as Defence Minister for leaking information from the National Security Council. Plus the amount of his cabinet colleagues who have previously been fired for incompetence or lying (or both).
Then it sounds like he knows a thing or two about the dangers promotion beyond ones ability presents society!
posted by pwnguin at 2:20 PM on August 13, 2020 [1 favorite]
Then it sounds like he knows a thing or two about the dangers promotion beyond ones ability presents society!
posted by pwnguin at 2:20 PM on August 13, 2020 [1 favorite]
Interesting that there were any gains in top grades given the model used and that gains were actually the second smallest at selective state secondaries and relatively high at state comprehensives.
Ah, actually it's because they have more small classes. Comps have very few classes with 4 people taking an exam but private schools have a lot more, that's why.
An interesting contrast between Scotland and the rest of the UK in how they have handled this: in elections to the Scottish parliament (which controls education policy), 16 and 17 year olds can vote. Not to mention the next election is May 2021. The Scottish government is directly answerable to these students in a way Westminster isn't.
Aren't these particular students now 17 or 18? In other words, they're voters at the next possible election in any case.
posted by atrazine at 3:21 PM on August 13, 2020
Ah, actually it's because they have more small classes. Comps have very few classes with 4 people taking an exam but private schools have a lot more, that's why.
An interesting contrast between Scotland and the rest of the UK in how they have handled this: in elections to the Scottish parliament (which controls education policy), 16 and 17 year olds can vote. Not to mention the next election is May 2021. The Scottish government is directly answerable to these students in a way Westminster isn't.
Aren't these particular students now 17 or 18? In other words, they're voters at the next possible election in any case.
posted by atrazine at 3:21 PM on August 13, 2020
I think you can still leave school at 16 in Scotland. Been a while since I knew for sure.
posted by scruss at 5:25 PM on August 13, 2020
posted by scruss at 5:25 PM on August 13, 2020
I do think it would have saved a lot of aggravation if they'd just accepted that grades would be inflated in this one year because it doesn't matter if they all are
Yes, exactly. This year, the most important thing was to be *kind*, even if that meant lifting the cap on university admissions.
Marking down 40% of predicted grades is not kind. Sticking rigidly to the usual timetable is not kind. Disallowing individual appeals is not kind.
Williamson tweeted that it's all right because all the kids who were fucked over today could get jobs in the wonderful post-Brexit economy. Then that his son just got four As.
Thankfully, that one was a parody account. The real one just offered congratulations.
posted by ManyLeggedCreature at 2:05 AM on August 14, 2020 [12 favorites]
Yes, exactly. This year, the most important thing was to be *kind*, even if that meant lifting the cap on university admissions.
Marking down 40% of predicted grades is not kind. Sticking rigidly to the usual timetable is not kind. Disallowing individual appeals is not kind.
Williamson tweeted that it's all right because all the kids who were fucked over today could get jobs in the wonderful post-Brexit economy. Then that his son just got four As.
Thankfully, that one was a parody account. The real one just offered congratulations.
posted by ManyLeggedCreature at 2:05 AM on August 14, 2020 [12 favorites]
Bear in mind that the cap was only announced in late March to stop universities over-recruiting. There hasn't been one in the five years preceding. Then the government announced they would increase it by 5%. The universities would take more if they were allowed, I have little doubt. And if international students don't show then there will be a decent amount of capacity. Might not work out so well for the lower ranked institutions though.
posted by biffa at 2:35 AM on August 14, 2020
posted by biffa at 2:35 AM on August 14, 2020
one being a useful overview from a former senior education policy advisor
Here's another twitter thread from the same former senior education policy advisor explaining why it's to be expected that predicted grades are going to be in aggregate higher than typical exam results. How do you predict which student would be the one to misread a question, or sleep badly the night before, if the exam was actually held?
posted by scorbet at 3:02 AM on August 14, 2020 [2 favorites]
Here's another twitter thread from the same former senior education policy advisor explaining why it's to be expected that predicted grades are going to be in aggregate higher than typical exam results. How do you predict which student would be the one to misread a question, or sleep badly the night before, if the exam was actually held?
posted by scorbet at 3:02 AM on August 14, 2020 [2 favorites]
I do think it would have saved a lot of aggravation if they'd just accepted that grades would be inflated in this one year because it doesn't matter if they all are
Yes, exactly. This year, the most important thing was to be *kind*, even if that meant lifting the cap on university admissions.
On the other hand though, despite what I said about all expected grades being equally inflated, one of the twitter threads posted by inire links to this excerpt from the Ofqual report which states that it's known that the difference between "predicted" and actual results is always slightly higher at the most deprived schools. Partially because it's a closed end scale, if someone's "real expected grade" is A* then there is no higher grade that a teacher can predict for them, even if that was their natural inclination.
Also, some schools may have been harsher with their predicted grades and accepting predicted grades for everyone is in relative terms unfair to them.
posted by atrazine at 3:43 AM on August 14, 2020
Yes, exactly. This year, the most important thing was to be *kind*, even if that meant lifting the cap on university admissions.
On the other hand though, despite what I said about all expected grades being equally inflated, one of the twitter threads posted by inire links to this excerpt from the Ofqual report which states that it's known that the difference between "predicted" and actual results is always slightly higher at the most deprived schools. Partially because it's a closed end scale, if someone's "real expected grade" is A* then there is no higher grade that a teacher can predict for them, even if that was their natural inclination.
Also, some schools may have been harsher with their predicted grades and accepting predicted grades for everyone is in relative terms unfair to them.
posted by atrazine at 3:43 AM on August 14, 2020
Thankfully, that one was a parody account.
Yes, that's embarrassing. Sorry, all. That said, it was quite difficult to spot the parody, and I hope was an understandable mistake given that the Home Secretary has said that immigration policy is to be changed (possibly to flout international humanitarian standards) in order to annoy lefties.
And Lord Bethell is real.
So, this Twitter thread. Definitely a real person, Alex Hern.
Every year, it's a difficult day for me, and I get swept up in emotions enough that I really ought to be more careful, possibly avoid social media altogether - even in a good year, I have real and debilitating PTSD on A-Level results day - dread, depression, flashbacks, rage, all sorts. Usually I channel it into attacking the fact that newspapers' eagerness to publish photographs of attractive teenage girls give the impression that no boys actually pass (boys tend to be represented by a sad looking bloke who, as the phrase goes, didn't get the grades he was expecting). Obviously the fact that newspapers leap any excuse to publish pictures of attractive teenage girls is worthy of criticism in itself, but I suspect other people have that covered.
And I took A-Levels thirty-seven years ago. And thirty-six years ago. (No, I didn't do any better second time round. Yes, it wasn't a lot of fun.)
I hate the British educational system. It's full of decent people, but the actual system is designed only to have the purpose of legitimating the exclusion of huge swathes of the population. The thing is, this debacle isn't a random result: it was designed. All it has done is accentuate what the system is designed to achieve less ostentatiously in other years.
In a sense, that person who said the results weren't systematically biased was right: they were systemically biased.
I'm not generally a revolutionary type, but if this pointless, stinking country could be quietly wiped from the earth and replaced with something decent I would be pleased.
posted by Grangousier at 6:04 AM on August 14, 2020 [6 favorites]
Yes, that's embarrassing. Sorry, all. That said, it was quite difficult to spot the parody, and I hope was an understandable mistake given that the Home Secretary has said that immigration policy is to be changed (possibly to flout international humanitarian standards) in order to annoy lefties.
And Lord Bethell is real.
So, this Twitter thread. Definitely a real person, Alex Hern.
Every year, it's a difficult day for me, and I get swept up in emotions enough that I really ought to be more careful, possibly avoid social media altogether - even in a good year, I have real and debilitating PTSD on A-Level results day - dread, depression, flashbacks, rage, all sorts. Usually I channel it into attacking the fact that newspapers' eagerness to publish photographs of attractive teenage girls give the impression that no boys actually pass (boys tend to be represented by a sad looking bloke who, as the phrase goes, didn't get the grades he was expecting). Obviously the fact that newspapers leap any excuse to publish pictures of attractive teenage girls is worthy of criticism in itself, but I suspect other people have that covered.
And I took A-Levels thirty-seven years ago. And thirty-six years ago. (No, I didn't do any better second time round. Yes, it wasn't a lot of fun.)
I hate the British educational system. It's full of decent people, but the actual system is designed only to have the purpose of legitimating the exclusion of huge swathes of the population. The thing is, this debacle isn't a random result: it was designed. All it has done is accentuate what the system is designed to achieve less ostentatiously in other years.
In a sense, that person who said the results weren't systematically biased was right: they were systemically biased.
I'm not generally a revolutionary type, but if this pointless, stinking country could be quietly wiped from the earth and replaced with something decent I would be pleased.
posted by Grangousier at 6:04 AM on August 14, 2020 [6 favorites]
To elaborate on Lord Bethell's tweet that Grangousier linked to, in which Bethell says that fluffing his A level 'taught me how to hustle. First to get a place in university. And haven't stopped ever since. Grades are great, but grit and perseverance win every time."
- James Bethell went to Harrow school, current fees £42,000 per year.
- He still got a place at Edinburgh University (and has not made clear what 'hustling' to get in means)
- He acquired his title of Lord through a laudable act of perseverance of...waiting for his father to die.
- He is now a Junior Minister, despite having failed in a parliamentary election, by dint of his lifelong membership in the House of Lords, which he owes entirely to a timely coincidence of ejaculation and ovulation, and the ejaculator's later death.
- He lost his sense of self-awareness at a cruel initiation ceremony at Harrow that went too far.
posted by reynir at 6:22 AM on August 14, 2020 [11 favorites]
- James Bethell went to Harrow school, current fees £42,000 per year.
- He still got a place at Edinburgh University (and has not made clear what 'hustling' to get in means)
- He acquired his title of Lord through a laudable act of perseverance of...waiting for his father to die.
- He is now a Junior Minister, despite having failed in a parliamentary election, by dint of his lifelong membership in the House of Lords, which he owes entirely to a timely coincidence of ejaculation and ovulation, and the ejaculator's later death.
- He lost his sense of self-awareness at a cruel initiation ceremony at Harrow that went too far.
posted by reynir at 6:22 AM on August 14, 2020 [11 favorites]
Do I understand this right? The Tories massively screwed up the UK's COVID response, as a result graduating students can't take the standardized tests, and in response the Tory government decreed that students at prestigious schools for rich kids all passed with the best possible grades while mere peasants at schools for regular kids either failed or got grades appropriate to their lowly birth? And they did this claiming it was to prevent grade inflation?
posted by sotonohito at 6:40 AM on August 14, 2020 [7 favorites]
posted by sotonohito at 6:40 AM on August 14, 2020 [7 favorites]
Yes.
It's probably worse than that, but yes.
posted by Grangousier at 6:46 AM on August 14, 2020 [3 favorites]
It's probably worse than that, but yes.
posted by Grangousier at 6:46 AM on August 14, 2020 [3 favorites]
to be fair, as marina hyde points out, no one could have seen this coming.
posted by inire at 7:29 AM on August 14, 2020
posted by inire at 7:29 AM on August 14, 2020
to be fair, as marina hyde points out, no one could have seen this coming.
In fine style and showcasing that the entire British establishment is equally incompetent, that article has now been amended to remove an erroneous quote.
Also, the majority of the reporting on this in any British newspaper has been outrageously incompetent. Lots of outrage, the occasional bit of humour, but no serious consideration of what exactly has happened, why, and what should have been done instead.
posted by atrazine at 7:59 AM on August 14, 2020 [1 favorite]
In fine style and showcasing that the entire British establishment is equally incompetent, that article has now been amended to remove an erroneous quote.
Also, the majority of the reporting on this in any British newspaper has been outrageously incompetent. Lots of outrage, the occasional bit of humour, but no serious consideration of what exactly has happened, why, and what should have been done instead.
posted by atrazine at 7:59 AM on August 14, 2020 [1 favorite]
Turns out I feel really strongly that absolutely no-one should have missed out on their university place because an algorithm decided they would not have got the grades that their previous 6.5 years of secondary education, GCSE results, coursework, mock A-level results and teachers' assessments of their capabilities suggested they would get.
Some people would still have lost out. Most obviously, anyone who would have exceeded their teachers' expectations (NB that's what I'd expected this year's exam results scandal to be about), especially if their university offers were for slightly higher than their predicted grades. It would certainly have complicated things for the universities. It wouldn't have been perfect. But is anything perfect, this year? And what's actually happened is inexcusable.
posted by ManyLeggedCreature at 8:15 AM on August 14, 2020 [2 favorites]
Some people would still have lost out. Most obviously, anyone who would have exceeded their teachers' expectations (NB that's what I'd expected this year's exam results scandal to be about), especially if their university offers were for slightly higher than their predicted grades. It would certainly have complicated things for the universities. It wouldn't have been perfect. But is anything perfect, this year? And what's actually happened is inexcusable.
posted by ManyLeggedCreature at 8:15 AM on August 14, 2020 [2 favorites]
that article has now been amended to remove an erroneous quote
specifically, the 'pupils might be promoted beyond their competence' quote, which he did not actually say, but which was a (very good) joke by a twitter user that was recirculated as fact.
i suppose we should have realised that such exquisite irony could not exist in the material world -_-
it's enough to drive one to gnosticism
posted by inire at 8:15 AM on August 14, 2020 [3 favorites]
specifically, the 'pupils might be promoted beyond their competence' quote, which he did not actually say, but which was a (very good) joke by a twitter user that was recirculated as fact.
i suppose we should have realised that such exquisite irony could not exist in the material world -_-
it's enough to drive one to gnosticism
posted by inire at 8:15 AM on August 14, 2020 [3 favorites]
→ And they did this claiming it was to prevent grade inflation?
Grade inflation from the wrong sort of people, that is. A successful Brexit requires a return to a feudal society, and we can't have the villeins getting ideas above their station.
I'm only slightly joking. I think they're after a rematch with France for 1066, except with 1966 jingoism.
posted by scruss at 8:39 AM on August 14, 2020 [1 favorite]
Grade inflation from the wrong sort of people, that is. A successful Brexit requires a return to a feudal society, and we can't have the villeins getting ideas above their station.
I'm only slightly joking. I think they're after a rematch with France for 1066, except with 1966 jingoism.
posted by scruss at 8:39 AM on August 14, 2020 [1 favorite]
Worcester College, Oxford has announced it will now take all offer holders regardless of the A level results they got. Which is potentially quite a big deal if others go down the same route. I'd be interested to see their numbers. Any idea what Oxford's spare capacity is? I would have assumed it was fairly limited, maybe a bit more than usual due to possible reduction in international numbers. That won't influence their cap quota though.
posted by biffa at 9:01 AM on August 14, 2020 [3 favorites]
posted by biffa at 9:01 AM on August 14, 2020 [3 favorites]
For undergraduates at Oxford, about 20% normally come from outside the UK, with the bulk of those students being non-EU. So a decline in international undergraduates would free up some capacity. I would think ordinarily Oxford's capacity to increase enrolment isn't very elastic because the tutorial system is pretty resource intensive and probably doesn't easily accommodate unexpected increases in student numbers.
posted by persimmons at 9:40 AM on August 14, 2020 [1 favorite]
posted by persimmons at 9:40 AM on August 14, 2020 [1 favorite]
So the model is slightly more complex than I thought, I haven't read all 300 odd pages of the Ofqual report but this is my understanding from one of the Datalab posts:
Each centre has an expected grade distribution based on previous performance
Each centre then has that distribution adjusted based on the strength of the cohort, basically how good were the GCSE results of the current cohort vs the average?
Grades are then assigned based on the centre's ranking
Some issues:
First, the way grades are assigned seems to round down. If less than one exam taker in the adjusted distribution is expected to get a U and an A*, the U gets assigned but not the A*. I am not a statistician so cannot comment on the rigour of that approach, but that seems a little off.
Second, there seems to be no adjustment for high value added schools. If your centre has high value addition but a low GCSE intake (i.e. a really good comprehensive school with a large intake from a deprived area) then that might bias your results down. Not sure I've fully understood that bit, seems too odd to be correct.
Finally, there's not "check" to prevent people dropping too many grades to be plausible. If everyone in your cohort gets A*, A, B predicted and you are the lowest ranked B (someone has to be) and the distribution says someone gets a U, then bad luck.
It's one thing for someone predicted a B to get a C or a D - after all, lots of people actually do that when they take the exam - but does anyone seriously get predicted a B and then get a U? That must be very rare.
Do I understand this right? The Tories massively screwed up the UK's COVID response, as a result graduating students can't take the standardized tests, and in response the Tory government decreed that students at prestigious schools for rich kids all passed with the best possible grades while mere peasants at schools for regular kids either failed or got grades appropriate to their lowly birth? And they did this claiming it was to prevent grade inflation?
Conventionally, yes. In reality no.
We have the convention that it is government ministers who are fully and completely accountable for what goes on in their departments. It's useful because when something goes wrong, it's only a politician who has to climb inside the wicker man to dispel society's tension. On the other hand, everyone also knows that it doesn't really reflect reality.
It would be nice to sack over-promoted fireplace salesman Williamson, a shmuck of the first order who should have stayed sacked after May expelled him, and to leave the dedicated statisticians, kindly headmasters, and dedicated public servants at Ofqual alone. Does that really reflect who's responsible for this though? I somehow doubt that tarantula creeper was involved at any level of designing these models and validating them so his ouster would be purely symbolic.
posted by atrazine at 10:11 AM on August 14, 2020 [1 favorite]
Each centre has an expected grade distribution based on previous performance
Each centre then has that distribution adjusted based on the strength of the cohort, basically how good were the GCSE results of the current cohort vs the average?
Grades are then assigned based on the centre's ranking
Some issues:
First, the way grades are assigned seems to round down. If less than one exam taker in the adjusted distribution is expected to get a U and an A*, the U gets assigned but not the A*. I am not a statistician so cannot comment on the rigour of that approach, but that seems a little off.
Second, there seems to be no adjustment for high value added schools. If your centre has high value addition but a low GCSE intake (i.e. a really good comprehensive school with a large intake from a deprived area) then that might bias your results down. Not sure I've fully understood that bit, seems too odd to be correct.
Finally, there's not "check" to prevent people dropping too many grades to be plausible. If everyone in your cohort gets A*, A, B predicted and you are the lowest ranked B (someone has to be) and the distribution says someone gets a U, then bad luck.
It's one thing for someone predicted a B to get a C or a D - after all, lots of people actually do that when they take the exam - but does anyone seriously get predicted a B and then get a U? That must be very rare.
Do I understand this right? The Tories massively screwed up the UK's COVID response, as a result graduating students can't take the standardized tests, and in response the Tory government decreed that students at prestigious schools for rich kids all passed with the best possible grades while mere peasants at schools for regular kids either failed or got grades appropriate to their lowly birth? And they did this claiming it was to prevent grade inflation?
Conventionally, yes. In reality no.
We have the convention that it is government ministers who are fully and completely accountable for what goes on in their departments. It's useful because when something goes wrong, it's only a politician who has to climb inside the wicker man to dispel society's tension. On the other hand, everyone also knows that it doesn't really reflect reality.
It would be nice to sack over-promoted fireplace salesman Williamson, a shmuck of the first order who should have stayed sacked after May expelled him, and to leave the dedicated statisticians, kindly headmasters, and dedicated public servants at Ofqual alone. Does that really reflect who's responsible for this though? I somehow doubt that tarantula creeper was involved at any level of designing these models and validating them so his ouster would be purely symbolic.
posted by atrazine at 10:11 AM on August 14, 2020 [1 favorite]
Also, the majority of the reporting on this in any British newspaper has been outrageously incompetent.
FTFY
posted by HumuloneRanger at 10:55 AM on August 14, 2020
FTFY
posted by HumuloneRanger at 10:55 AM on August 14, 2020
It's a really difficult issue. In Scotland, grades were awarded based on teachers' estimates.
This totally glosses over the fact that Scotland fucked up Higher exam results, in the first instance, in exactly the same way as England. The FM even did a face to face interview on Reporting Scotland doubling down on the fact they were right and refusing to budge on the issue. The only thing that made them back down and allow teachers' estimates, was that the Greens (their usual allies in Holyrood) refused to back them in a pending vote of no confidence in Education Minister John Swinney. The Greens made backing down a condition of their support in the no confidence vote, and the SNP did a volte-face, making great play of how "We're not like other politicians, we've decided to listen and admit we were wrong". Never mind that it was also the only way to keep John Swinney's job.
Which makes it all the more amazing that England screwed things exactly the same way, given that A Level results come out 2 or 3 weeks after Highers.
Also came in to link the excellent Alex Hern thread linked to above, which give me the horrors, as a kid who was, for a brief while, freakishly succcessful in exams in a very non-academic school.
Most of all, I feel this all really just underlines the insanity of the pressure-cooker that we put our 16-18 year olds through. This idea that you must know at 18 what you want to do with the rest of your life, must already be capable of excelling in it, and in exam technique, and in course work, must go through the madness of UCAS and the quick-fire lottery of clearing, so that 100s of thousands of kids can all be allocated university places within the space of a few days, after being told for two years that the outcome of this moment will be make-or-break for their entire life chances... it's just bonkers.
When you see these poor kids standing there on the news, saying in all earnestness "This has ruined my life," because they've spent two years being told that their exam grades and their University choices are EVERYTHING, your heart goes out to them. Especially several decades down the line when you know that every new month, every new year, could be the time you decide what to do next in your life, and start working steadily towards it.
I don't know what the answer is, and I know lots of 18-year-olds want to experience the rite of passage that is student life by starting at the same time as their peers. But if we could meaningfully persuade people that they can do these things at any age, and have a system that supported them to do that, the mental health of our teens (and probably other ages) would be so much better.
In a week full of heartbreaking twitter threads, this has to be one of the most gutting - a lass who got poor grades last year so spent a year studying as a private student, at a cost of £2K for all the test centres, exams, tutoring etc., because she was so set on going to med school and decided to work for better grades. Despite masses of evidence, she was downgraded from AAB to EED. And of course, understandably, she says "My life is ruined."
posted by penguin pie at 4:09 PM on August 14, 2020 [11 favorites]
This totally glosses over the fact that Scotland fucked up Higher exam results, in the first instance, in exactly the same way as England. The FM even did a face to face interview on Reporting Scotland doubling down on the fact they were right and refusing to budge on the issue. The only thing that made them back down and allow teachers' estimates, was that the Greens (their usual allies in Holyrood) refused to back them in a pending vote of no confidence in Education Minister John Swinney. The Greens made backing down a condition of their support in the no confidence vote, and the SNP did a volte-face, making great play of how "We're not like other politicians, we've decided to listen and admit we were wrong". Never mind that it was also the only way to keep John Swinney's job.
Which makes it all the more amazing that England screwed things exactly the same way, given that A Level results come out 2 or 3 weeks after Highers.
Also came in to link the excellent Alex Hern thread linked to above, which give me the horrors, as a kid who was, for a brief while, freakishly succcessful in exams in a very non-academic school.
Most of all, I feel this all really just underlines the insanity of the pressure-cooker that we put our 16-18 year olds through. This idea that you must know at 18 what you want to do with the rest of your life, must already be capable of excelling in it, and in exam technique, and in course work, must go through the madness of UCAS and the quick-fire lottery of clearing, so that 100s of thousands of kids can all be allocated university places within the space of a few days, after being told for two years that the outcome of this moment will be make-or-break for their entire life chances... it's just bonkers.
When you see these poor kids standing there on the news, saying in all earnestness "This has ruined my life," because they've spent two years being told that their exam grades and their University choices are EVERYTHING, your heart goes out to them. Especially several decades down the line when you know that every new month, every new year, could be the time you decide what to do next in your life, and start working steadily towards it.
I don't know what the answer is, and I know lots of 18-year-olds want to experience the rite of passage that is student life by starting at the same time as their peers. But if we could meaningfully persuade people that they can do these things at any age, and have a system that supported them to do that, the mental health of our teens (and probably other ages) would be so much better.
In a week full of heartbreaking twitter threads, this has to be one of the most gutting - a lass who got poor grades last year so spent a year studying as a private student, at a cost of £2K for all the test centres, exams, tutoring etc., because she was so set on going to med school and decided to work for better grades. Despite masses of evidence, she was downgraded from AAB to EED. And of course, understandably, she says "My life is ruined."
posted by penguin pie at 4:09 PM on August 14, 2020 [11 favorites]
Bad results don't mean your life is over but there are certain pathways where once your are off track you will struggle to get back on. Medicine, veterinary science, architecture, all impossible or hard to get back on track. Oxbridge entry? Pretty much a one off chance and alternatives are not an equal substitute.
posted by biffa at 4:27 PM on August 14, 2020
posted by biffa at 4:27 PM on August 14, 2020
Bad results don't mean your life is over but there are certain pathways where once your are off track you will struggle to get back on. Medicine, veterinary science, architecture, all impossible or hard to get back on track. Oxbridge entry? Pretty much a one off chance and alternatives are not an equal substitute.
Aye, but that's because the system is set up to function like that. Wouldn't it be great if it was just as common for people at the grand old age of (gasp!) 23, for example, to decide they wanted to be a doctor and access that path as simply as you can from a high school? And for 18 year olds to know it would be just as straightforward to follow that path in a few years' time as it is now, so there was less pressure to ace it all between May and August of your one-chance exam year.
posted by penguin pie at 4:39 PM on August 14, 2020 [5 favorites]
Aye, but that's because the system is set up to function like that. Wouldn't it be great if it was just as common for people at the grand old age of (gasp!) 23, for example, to decide they wanted to be a doctor and access that path as simply as you can from a high school? And for 18 year olds to know it would be just as straightforward to follow that path in a few years' time as it is now, so there was less pressure to ace it all between May and August of your one-chance exam year.
posted by penguin pie at 4:39 PM on August 14, 2020 [5 favorites]
It's one thing for someone predicted a B to get a C or a D - after all, lots of people actually do that when they take the exam
True, but it isn't automatically the "worst" B that this happens to. Even the "best" can misread a question or have their weakest topic come up. It's bad enough that you can drop a grade or more thanks to your luck on the day, but at least in that case there's an identifiable reason for it. An algorithm deciding who would have a bad day doesn't really sit right with me.
Dropping someone to a U from a B is infinitely worse, particularly if the allocation is based on rounding down, as reported. Theoretically at least just one person in the last few years getting a U could mean that someone this year "has" to get a U.
posted by scorbet at 5:32 AM on August 15, 2020
True, but it isn't automatically the "worst" B that this happens to. Even the "best" can misread a question or have their weakest topic come up. It's bad enough that you can drop a grade or more thanks to your luck on the day, but at least in that case there's an identifiable reason for it. An algorithm deciding who would have a bad day doesn't really sit right with me.
Dropping someone to a U from a B is infinitely worse, particularly if the allocation is based on rounding down, as reported. Theoretically at least just one person in the last few years getting a U could mean that someone this year "has" to get a U.
posted by scorbet at 5:32 AM on August 15, 2020
Another Twitter thread explaining the algorithm via simplified worked example (this one by Alex Weatherall, "not currently a physics teacher").
I'm not getting any less angry. Did they just forget that A-level candidates are also people?
posted by ManyLeggedCreature at 6:56 AM on August 15, 2020 [1 favorite]
I'm not getting any less angry. Did they just forget that A-level candidates are also people?
posted by ManyLeggedCreature at 6:56 AM on August 15, 2020 [1 favorite]
I'm not getting any less angry. Did they just forget that A-level candidates are also people?
Based upon the sheer amount of successful appeals resulting in changed grades in previous years, I'm not they ever knew that in the first place. The marking is so random in the first place and the grade boundaries pretty arbitrary to massage the overall figures to meet government targets, that getting it redone can easily result in you moving up a grade boundary, or even two , or sometimes down, if you happened to luck out in the first place and got over ambitious.
Of course, appealling a grade costs money, which is waived if you win. So unsurprisingly, private schools also put in more appeals, and on average do quite well out of it. But at least there's the raw exam paper to work with, so you have something to correct injustices with in principle.
This year, no paper, just the statistical model and the rank order your school put you in in that subject. You can appeal the latter (to your school), but not the former. Your school appealing on your behalf to at least get your mock grade is something, but people very rarely get a better mock result than their final exam, it's much more common to go the other way round because you have months of revision for your A-levels.
posted by Absolutely No You-Know-What at 8:02 AM on August 15, 2020 [1 favorite]
Based upon the sheer amount of successful appeals resulting in changed grades in previous years, I'm not they ever knew that in the first place. The marking is so random in the first place and the grade boundaries pretty arbitrary to massage the overall figures to meet government targets, that getting it redone can easily result in you moving up a grade boundary, or even two , or sometimes down, if you happened to luck out in the first place and got over ambitious.
Of course, appealling a grade costs money, which is waived if you win. So unsurprisingly, private schools also put in more appeals, and on average do quite well out of it. But at least there's the raw exam paper to work with, so you have something to correct injustices with in principle.
This year, no paper, just the statistical model and the rank order your school put you in in that subject. You can appeal the latter (to your school), but not the former. Your school appealing on your behalf to at least get your mock grade is something, but people very rarely get a better mock result than their final exam, it's much more common to go the other way round because you have months of revision for your A-levels.
posted by Absolutely No You-Know-What at 8:02 AM on August 15, 2020 [1 favorite]
I have been hearing about a lot of people planning to appeal, the problem is, the grounds for appeal are pretty limited. It's not going to be a re-mark is it, for obvious reasons. So unless the algorithm had gone tits up, which I don't rule out in many of the really horrendous cases, but for those missing out on a grade that misses them getting the programme they want, then there is going to be a fair amount of disappointment in the next few weeks I think. Basically, I think people are expecting justice and it's going to be in shot supply unless there is a policy U-turn.
posted by biffa at 11:08 AM on August 15, 2020
posted by biffa at 11:08 AM on August 15, 2020
I suppose demanding kids return to school in the Daily Mail is a pretty forensic way to lessen the impact of the GCSE debacle on their future lives. Especially as it's more likely to save BAME and poor kids from having to deal with it.
Well, a small but heartbreaking percentage of them anyway.
posted by fullerine at 3:24 AM on August 16, 2020
Well, a small but heartbreaking percentage of them anyway.
posted by fullerine at 3:24 AM on August 16, 2020
I mean, you could make that conclusion from the article, fullerine, but it's not what it says.
I don't know how Starmer can persuade Mail readers that the Conservative government has been wildly and lethally malcompetent, but that article is actually a pretty good attempt at it.
posted by ambrosen at 11:56 AM on August 16, 2020
I don't know how Starmer can persuade Mail readers that the Conservative government has been wildly and lethally malcompetent, but that article is actually a pretty good attempt at it.
posted by ambrosen at 11:56 AM on August 16, 2020
Sorry to ask this so late in the conversation, but I'm very confused about the "teacher estimates" thing. Why are grades awarded based on estimates? Shouldn't the exams themselves just be graded, and you get a grade depending on how well you did? My poor understanding of this situation is that students take practice exams, and then real exams, and the latter exams are graded by teacher guesses + an algorithm adjusting grades for some arbitrary goal? This all sounds so bonkers to me I think I must be fundamentally understanding something basic, maybe estimates or grades don't mean what I think they do over there?
posted by GoblinHoney at 4:47 PM on August 16, 2020
posted by GoblinHoney at 4:47 PM on August 16, 2020
Shouldn't the exams themselves just be graded, and you get a grade depending on how well you did?
That’s what usually happens. However, this year, thanks to schools being closed from March onwards, the “real” exams were canceled. They then had to come up with a way to give each student a grade, even though there was no exam. This was mostly necessary to allow universities to determine which students they would take.
The solution chosen was to ask teachers to both estimate the grade each student “should” get in the exam and which rank they had in the school for that subject . But then, because the results were “too good”, they used an algorithm based on the previous results from each school to reassign grades according to the ranking. The previous performance of the current class/student seemed to be of only minor importance, more important was that the distribution in each school was the same as the average of the last three years.
It basically meant that a number of students got a lot lower grades than they were expecting just because of the school they went to. If your school didn’t have a student get an A* in the previous few years, then you couldn’t get one, no matter your own performance. Others got dropped by several grades simply because of their rank and the fact that the algorithm “decided” someone had to get that grade. The algorithm wasn’t used with smaller groups as that would be really unfair, but the smaller groups are mostly found in the “richer” schools, which meant less downgrading there.
The main problem was the lack of exams, but it was compounded by an attempt to ensure that the overall grade distribution was the same as previously, without really considering what the effects on individual students were.
posted by scorbet at 7:22 PM on August 16, 2020 [2 favorites]
That’s what usually happens. However, this year, thanks to schools being closed from March onwards, the “real” exams were canceled. They then had to come up with a way to give each student a grade, even though there was no exam. This was mostly necessary to allow universities to determine which students they would take.
The solution chosen was to ask teachers to both estimate the grade each student “should” get in the exam and which rank they had in the school for that subject . But then, because the results were “too good”, they used an algorithm based on the previous results from each school to reassign grades according to the ranking. The previous performance of the current class/student seemed to be of only minor importance, more important was that the distribution in each school was the same as the average of the last three years.
It basically meant that a number of students got a lot lower grades than they were expecting just because of the school they went to. If your school didn’t have a student get an A* in the previous few years, then you couldn’t get one, no matter your own performance. Others got dropped by several grades simply because of their rank and the fact that the algorithm “decided” someone had to get that grade. The algorithm wasn’t used with smaller groups as that would be really unfair, but the smaller groups are mostly found in the “richer” schools, which meant less downgrading there.
The main problem was the lack of exams, but it was compounded by an attempt to ensure that the overall grade distribution was the same as previously, without really considering what the effects on individual students were.
posted by scorbet at 7:22 PM on August 16, 2020 [2 favorites]
According to Lewis Goodall on twitter, even Eton was affected. He has a letter from the headmaster to parents. In particular in one subject, where they changed the syllabus, so there was no previous data to use. So they applied graded it based on the national curve instead.
I do wonder what was done with the Leaving Cert results in Ireland, which are due in a few weeks, and whether there will be similar problems there, and whether they will have learned from the problems in the UK. (The whole system is a quite different but the problem of how to assign grades without exams is the same.)
posted by scorbet at 1:40 AM on August 17, 2020
I do wonder what was done with the Leaving Cert results in Ireland, which are due in a few weeks, and whether there will be similar problems there, and whether they will have learned from the problems in the UK. (The whole system is a quite different but the problem of how to assign grades without exams is the same.)
posted by scorbet at 1:40 AM on August 17, 2020
More interesting statistical analysis:
The author's conclusion is that the statistical testing Ofqual did to evaluate the 11 models they considered and pick the one they did is fundamentally wrong.
What they wanted to do is test: how well does a combination of the teacher assigned order + the corrected grade distribution predict actual exam grades?
They used the 2019 exam results for that. Very reasonable. But wait... in 2019 there *were* no rank orders, it's not part of the normal process! So where did those come from? They used the actual grades from 2019 exams! But that's totally wrong. It's hardly a surprise that this would work relatively well in the test. It doesn't validate the actual method at all.
Other peculiarities: The way they combine multiple grade distributions looks weird and can lead to schools that have Ds and As but not Bs and Cs. Not credible.
The test of "accuracy" was whether a grade was correct or not with no weighting as to how far off it was! Again, I am neither a teacher nor a statistician but come on. That doesn't seem logical. That means that if someone's "real" grade was an A, algorithms which scored them an A*, a B, and U all have the same accuracy penalty. That's bonkers. Being off by one grade is, to be honest, no more unfair than the usual exam system. Being off by many grades must surely be considered much worse.
I accept that they had a difficult task but they've made it so much harder on themselves:
-Why couldn't the algorithm have been published for peer review months ago?
-For that matter, the actual grades could have been published months ago, it's not like they needed to mark exams over the summer.
-They could have used CAGs as a backstop - set a rule that if your algo grade is two or more grades below CAG, Ofqual will double check the ways the CAG was set since this logically should be quite rate.
-They could have sent "draft" grades back to schools months ago and let them submit appeals to obviously incorrect grades before the exam takers even knew about it, why do they have to stress when at least one round of appeals could have been done quietly.
Despite what I said earlier about it only being a political fiction that the minister is in charge of all this, I think both Gavin Williamson and the senior leadership of Ofqual should consider their positions. They may not have been doing the analysis but they could surely have exerted more control to ensure peer review, publish results early, and think through the implications.
posted by atrazine at 5:20 AM on August 17, 2020 [3 favorites]
The author's conclusion is that the statistical testing Ofqual did to evaluate the 11 models they considered and pick the one they did is fundamentally wrong.
What they wanted to do is test: how well does a combination of the teacher assigned order + the corrected grade distribution predict actual exam grades?
They used the 2019 exam results for that. Very reasonable. But wait... in 2019 there *were* no rank orders, it's not part of the normal process! So where did those come from? They used the actual grades from 2019 exams! But that's totally wrong. It's hardly a surprise that this would work relatively well in the test. It doesn't validate the actual method at all.
Other peculiarities: The way they combine multiple grade distributions looks weird and can lead to schools that have Ds and As but not Bs and Cs. Not credible.
The test of "accuracy" was whether a grade was correct or not with no weighting as to how far off it was! Again, I am neither a teacher nor a statistician but come on. That doesn't seem logical. That means that if someone's "real" grade was an A, algorithms which scored them an A*, a B, and U all have the same accuracy penalty. That's bonkers. Being off by one grade is, to be honest, no more unfair than the usual exam system. Being off by many grades must surely be considered much worse.
I accept that they had a difficult task but they've made it so much harder on themselves:
-Why couldn't the algorithm have been published for peer review months ago?
-For that matter, the actual grades could have been published months ago, it's not like they needed to mark exams over the summer.
-They could have used CAGs as a backstop - set a rule that if your algo grade is two or more grades below CAG, Ofqual will double check the ways the CAG was set since this logically should be quite rate.
-They could have sent "draft" grades back to schools months ago and let them submit appeals to obviously incorrect grades before the exam takers even knew about it, why do they have to stress when at least one round of appeals could have been done quietly.
Despite what I said earlier about it only being a political fiction that the minister is in charge of all this, I think both Gavin Williamson and the senior leadership of Ofqual should consider their positions. They may not have been doing the analysis but they could surely have exerted more control to ensure peer review, publish results early, and think through the implications.
posted by atrazine at 5:20 AM on August 17, 2020 [3 favorites]
Wales is now saying that they are going to use the predicted grades for both A-Levels and GCSEs. Meanwhile Northern Ireland is using them for GCSEs but not A-Levels (yet).
posted by scorbet at 7:02 AM on August 17, 2020
posted by scorbet at 7:02 AM on August 17, 2020
There's a 4pm press conference scheduled, I bet it's going to be England doing the same thing - overruling Ofqual's model and just going with the CAGs after all.
If they'd just done this a week ago they wouldn't have made themselves look so disorganised.
posted by atrazine at 8:01 AM on August 17, 2020
If they'd just done this a week ago they wouldn't have made themselves look so disorganised.
posted by atrazine at 8:01 AM on August 17, 2020
> I bet it's going to be England doing the same thing - overruling Ofqual's model and just going with the CAGs after all.
And, indeed, that's exactly what they've done. What an absolute shambles.
posted by parm at 8:12 AM on August 17, 2020 [1 favorite]
And, indeed, that's exactly what they've done. What an absolute shambles.
posted by parm at 8:12 AM on August 17, 2020 [1 favorite]
Northern Ireland have also U-turned on the A-Levels, however it's specifically mentioned there that "all of their AS and A level qualifications will now be awarded the higher of the grade submitted by their centre or the grade calculated by CCEA." So the few people actually upgraded by the algorithm gets to keep it.
posted by scorbet at 8:17 AM on August 17, 2020
posted by scorbet at 8:17 AM on August 17, 2020
Aaaaand that's the latest U-turn for this government.
posted by like_neon at 8:20 AM on August 17, 2020
posted by like_neon at 8:20 AM on August 17, 2020
I bet it's going to be England doing the same thing - overruling Ofqual's model and just going with the CAGs after all.
It is a great pity that whoever argued against the algorithm internally at Ofqual/DfE wasn't successful. I am beyond certain that someone did - even if it was only after the Scottish results. I'm surprised that Ofqual were allowed to own the solution and issue a statement from their Chair. Usually you have an arms-length body so that a politician can ride in on a white horse and save the population from the technocrat's mistake.
Universities are going to be in a world of pain. Most years, clearing places are snapped up pretty quickly. Do they have to accept all those over offers, plus whoever they've gotten through clearing? What happens with people who've switched to insurance places?
posted by plonkee at 8:27 AM on August 17, 2020
It is a great pity that whoever argued against the algorithm internally at Ofqual/DfE wasn't successful. I am beyond certain that someone did - even if it was only after the Scottish results. I'm surprised that Ofqual were allowed to own the solution and issue a statement from their Chair. Usually you have an arms-length body so that a politician can ride in on a white horse and save the population from the technocrat's mistake.
Universities are going to be in a world of pain. Most years, clearing places are snapped up pretty quickly. Do they have to accept all those over offers, plus whoever they've gotten through clearing? What happens with people who've switched to insurance places?
posted by plonkee at 8:27 AM on August 17, 2020
Yes, now the universities are in a shitty situation as all the automated acceptances/denials have gone out last week, and they all currently face caps in admissions.
I just don't understand how the government could have been so incompetent?? They literally had a magic crystal ball when this played out in Scotland and yet they forged ahead with this idiotic outcome because GOD FORBID they take any lessons from Sturgeon.
posted by like_neon at 8:30 AM on August 17, 2020 [2 favorites]
I just don't understand how the government could have been so incompetent?? They literally had a magic crystal ball when this played out in Scotland and yet they forged ahead with this idiotic outcome because GOD FORBID they take any lessons from Sturgeon.
posted by like_neon at 8:30 AM on August 17, 2020 [2 favorites]
I just don't understand how the government could have been so incompetent??
Honestly, I think there was a lot of sunk cost fallacy in having spent that time and effort on the algorithm. On top of that they were comparing the criticism they thought they'd get for allowing the grades to increase substantially beyond what was plausible if they exams had actually happened, with the criticism that some students had been downgraded. I'm also not at all sure that anyone really looked to see what the impact was on individual students (for example, the distribution of the individual student differences between UCAS scores with CAG and with algorithm).
posted by plonkee at 8:40 AM on August 17, 2020
Honestly, I think there was a lot of sunk cost fallacy in having spent that time and effort on the algorithm. On top of that they were comparing the criticism they thought they'd get for allowing the grades to increase substantially beyond what was plausible if they exams had actually happened, with the criticism that some students had been downgraded. I'm also not at all sure that anyone really looked to see what the impact was on individual students (for example, the distribution of the individual student differences between UCAS scores with CAG and with algorithm).
posted by plonkee at 8:40 AM on August 17, 2020
Quite a lot of universities held places open for students pending appeals, so presumably they hadn't released those places to clearing yet. Assuming the student now get the results they needed from the teacher-assessed grades, they and their uni should be OK, though the student is likely to be pretty shook up.
The real mess is going to be for students that now get the grades for their original offer, but the uni tossed them under a bus and opened up that spot to clearing. Given the caps on numbers, the uni may well not be able to actually give the original student a spot back, and there are going to be issues with campus accommodation already filled up due to social distancing. And what about students that already accepted a lower choice from their list, or went through clearing - can they turn down the new place they may have already accepted, even if they can now theoretically get into their original 1st choice?
What a clusterfuck, and heads should be rolling at Ofqual for this. And Williamson for failing to well, do anything at all useful. Situation normal I guess when all your ministers must enthusiastically back a hard brexit, and so by definition are thick as constipated pig shit.
posted by Absolutely No You-Know-What at 8:44 AM on August 17, 2020
The real mess is going to be for students that now get the grades for their original offer, but the uni tossed them under a bus and opened up that spot to clearing. Given the caps on numbers, the uni may well not be able to actually give the original student a spot back, and there are going to be issues with campus accommodation already filled up due to social distancing. And what about students that already accepted a lower choice from their list, or went through clearing - can they turn down the new place they may have already accepted, even if they can now theoretically get into their original 1st choice?
What a clusterfuck, and heads should be rolling at Ofqual for this. And Williamson for failing to well, do anything at all useful. Situation normal I guess when all your ministers must enthusiastically back a hard brexit, and so by definition are thick as constipated pig shit.
posted by Absolutely No You-Know-What at 8:44 AM on August 17, 2020
And now the cap on university places previous put in place has been dropped, so unis can expand places on their courses without worrying about crippling fines, which should help.
Still:
"Some 55,000 students were accepted into their second choice university or into the clearing system, where courses are filled by students who missed out on their first choice university. Many of these students could now be eligible for their first choice university, throwing the admission process into chaos.
There are a further 80,000 students with holding offers, so there are potentially 150,000 students who could be changing university on the back of this announcement."
At least schools can get back to the work needed to re-open in a couple of a weeks instead of fucking about with the appeals process. Hopefully a good chunk of the 150,000 will now get on the course they deserved, no thanks to ofqual.
posted by Absolutely No You-Know-What at 9:14 AM on August 17, 2020
Still:
"Some 55,000 students were accepted into their second choice university or into the clearing system, where courses are filled by students who missed out on their first choice university. Many of these students could now be eligible for their first choice university, throwing the admission process into chaos.
There are a further 80,000 students with holding offers, so there are potentially 150,000 students who could be changing university on the back of this announcement."
At least schools can get back to the work needed to re-open in a couple of a weeks instead of fucking about with the appeals process. Hopefully a good chunk of the 150,000 will now get on the course they deserved, no thanks to ofqual.
posted by Absolutely No You-Know-What at 9:14 AM on August 17, 2020
Honestly, I think there was a lot of sunk cost fallacy in having spent that time and effort on the algorithm. On top of that they were comparing the criticism they thought they'd get for allowing the grades to increase substantially beyond what was plausible if they exams had actually happened, with the criticism that some students had been downgraded. I'm also not at all sure that anyone really looked to see what the impact was on individual students (for example, the distribution of the individual student differences between UCAS scores with CAG and with lgorithm).
I'm sure that's it exactly.
They will have had numerous meetings with Ofqual and been convinced that the outcome was statistically fair but then when the results come out there's Sky News interviewing someone predicted ABA who got BCU and as a minister you're thinking, "now what the fuck do I do?".
I really don't think they thought about individual students at all, it was all about aggregate effects. Well on aggregate the grades are about right - no surprise, they literally fit them that way!
Ofqual will have spent so much time in tactical mode - trying to get the statistics right, get the algorithm to do what they wanted it to, that they failed to look up and think about why they were doing this and what would happen to individuals. Really, that's a ministerial responsibility. I don't blame "Fireplace" Williamson for not fixing the algorithm, obviously that isn't his job even if he could do such a thing but if I had been SoS for Education I would have wanted to know well ahead of time, a transition matrix between predicted grades and assigned grades. If I'm looking at a table that says 10,000 As go to Us (or whatever) then I know that politically I had a Situation. Did no-one in this process ever stop to think, "who really cares about grade inflation?".
posted by atrazine at 9:16 AM on August 17, 2020
I'm sure that's it exactly.
They will have had numerous meetings with Ofqual and been convinced that the outcome was statistically fair but then when the results come out there's Sky News interviewing someone predicted ABA who got BCU and as a minister you're thinking, "now what the fuck do I do?".
I really don't think they thought about individual students at all, it was all about aggregate effects. Well on aggregate the grades are about right - no surprise, they literally fit them that way!
Ofqual will have spent so much time in tactical mode - trying to get the statistics right, get the algorithm to do what they wanted it to, that they failed to look up and think about why they were doing this and what would happen to individuals. Really, that's a ministerial responsibility. I don't blame "Fireplace" Williamson for not fixing the algorithm, obviously that isn't his job even if he could do such a thing but if I had been SoS for Education I would have wanted to know well ahead of time, a transition matrix between predicted grades and assigned grades. If I'm looking at a table that says 10,000 As go to Us (or whatever) then I know that politically I had a Situation. Did no-one in this process ever stop to think, "who really cares about grade inflation?".
posted by atrazine at 9:16 AM on August 17, 2020
Williamson was the one that told ofqual to come up with a plan for how to assign grades without exams, and clearly didn't ask, didn't understand or didn't care about the consequences of that plan.
Then the exact same problem in Scotland a week earlier prompting a screeching U-turn should have prompted some probing questions like "this isn't going to blow up in my face like Scotland is it?", but all he announced at the last minute was that mock results could be used for appeals. Then of course the clusterfuck of Thursday and Friday, yet he still said on Saturday "No U-turn. No change." The DoE supposedly dictated to ofqual the appeals process, which they then put up on their website on Saturday evening, and then removed with no explanation a few hours later.
He's just claimed "We don’t get any detailed data before schools but when we started to see concerning outliers... that’s why I felt action had to be taken." So how did he come up with the mocks appeal process?? The unis get the results days beforehand, so they can send out the letters to candidates.
Instead, it takes a weekend of terrible headlines, growing numbers of Tory MPs complaining in public, and a phone call from Boris on Monday morning for this U-turn.
Under any normal PM his 'resignation' letter would already be on the PM's desk, even May managed to sack him with a minority government, but with Boris still in hiding, who the hell knows.
posted by Absolutely No You-Know-What at 10:01 AM on August 17, 2020 [3 favorites]
Then the exact same problem in Scotland a week earlier prompting a screeching U-turn should have prompted some probing questions like "this isn't going to blow up in my face like Scotland is it?", but all he announced at the last minute was that mock results could be used for appeals. Then of course the clusterfuck of Thursday and Friday, yet he still said on Saturday "No U-turn. No change." The DoE supposedly dictated to ofqual the appeals process, which they then put up on their website on Saturday evening, and then removed with no explanation a few hours later.
He's just claimed "We don’t get any detailed data before schools but when we started to see concerning outliers... that’s why I felt action had to be taken." So how did he come up with the mocks appeal process?? The unis get the results days beforehand, so they can send out the letters to candidates.
Instead, it takes a weekend of terrible headlines, growing numbers of Tory MPs complaining in public, and a phone call from Boris on Monday morning for this U-turn.
Under any normal PM his 'resignation' letter would already be on the PM's desk, even May managed to sack him with a minority government, but with Boris still in hiding, who the hell knows.
posted by Absolutely No You-Know-What at 10:01 AM on August 17, 2020 [3 favorites]
The unis get the results days beforehand, so they can send out the letters to candidates.
The Friday beforehand. Bear in mind also that the exam boards didn't have to mark anything this year either, so what did they do with all the extra time? Why not go early so there might be time to sort this out, rather than 4 weeks before term is due to start (in probably what will be the most difficult term for teaching that the unis have ever had)?
posted by biffa at 11:16 AM on August 17, 2020
The Friday beforehand. Bear in mind also that the exam boards didn't have to mark anything this year either, so what did they do with all the extra time? Why not go early so there might be time to sort this out, rather than 4 weeks before term is due to start (in probably what will be the most difficult term for teaching that the unis have ever had)?
posted by biffa at 11:16 AM on August 17, 2020
So these CAGs? They might be available to universities from Thursday. UCAS aiming for that but not confirmed the timeline. So not going to see any new offers until after that.
Williamson still in a job despite being a useless cockdribble.
posted by biffa at 6:04 AM on August 18, 2020 [2 favorites]
Williamson still in a job despite being a useless cockdribble.
posted by biffa at 6:04 AM on August 18, 2020 [2 favorites]
The Guardian was suggesting that Williamson might be being kept around so he can also take the shots from what seems to be the inevitable GCSE shitfest & back to school debacle.
posted by biffa at 3:39 AM on August 19, 2020
posted by biffa at 3:39 AM on August 19, 2020
There’s some speculation about a whip he posed with too, just normal whip speculation.
posted by lucidium at 7:53 AM on August 19, 2020
posted by lucidium at 7:53 AM on August 19, 2020
I just noticed that we now all of 8 month past the General Election. Yup, its only been 8 months. Only 4.3 years of this shit to go until Johnson is...returned to power with a smaller majority, if the opinion polls are to be believed.
posted by biffa at 10:35 AM on August 19, 2020 [1 favorite]
posted by biffa at 10:35 AM on August 19, 2020 [1 favorite]
Firm linked to Gove and Cummings hired to work with Ofqual on A-levels
Gosh. The shock. It shakes me to my very fibre.
posted by Grangousier at 12:10 AM on August 20, 2020 [1 favorite]
Gosh. The shock. It shakes me to my very fibre.
posted by Grangousier at 12:10 AM on August 20, 2020 [1 favorite]
And of course the inevitable.
The DfE had no choice but to let pupils keep the higher of the algo or CAG grades (since algo grades were already out and they didn't want another wave of outrage) but in some cases the algo grades were higher. Considering that the CAG grades were on the high side already this is leading to ridiculous grade inflation.
Also points to algorithm issues that some pupils are having grades upgraded by five places from CAG to algo grade which shouldn't really happen.
This is usually due to school policy changes or cohort differences.
For instance: A school that has entered a large number of native / heritage speakers of a GCSE language in past years but not this year - result is that grades were upgraded massively by the algorithm as the previous grades were very high.
Another example: GCSE allows for a triple combined science entry (counts as two grades). If a school previously only let their top pupils take that but shifted policy, guess what? A*/9s falling like snowflakes because the grade distribution was so top heavy in previous years.
6th forms will need to take this into account in their own admissions and streaming decisions.
posted by atrazine at 12:10 PM on August 20, 2020
The DfE had no choice but to let pupils keep the higher of the algo or CAG grades (since algo grades were already out and they didn't want another wave of outrage) but in some cases the algo grades were higher. Considering that the CAG grades were on the high side already this is leading to ridiculous grade inflation.
Also points to algorithm issues that some pupils are having grades upgraded by five places from CAG to algo grade which shouldn't really happen.
This is usually due to school policy changes or cohort differences.
For instance: A school that has entered a large number of native / heritage speakers of a GCSE language in past years but not this year - result is that grades were upgraded massively by the algorithm as the previous grades were very high.
Another example: GCSE allows for a triple combined science entry (counts as two grades). If a school previously only let their top pupils take that but shifted policy, guess what? A*/9s falling like snowflakes because the grade distribution was so top heavy in previous years.
6th forms will need to take this into account in their own admissions and streaming decisions.
posted by atrazine at 12:10 PM on August 20, 2020
There was apparently a case where at least one candidate got a 6 in a GCSE exam where a 5 was the highest possible mark.
posted by biffa at 7:25 AM on August 21, 2020
posted by biffa at 7:25 AM on August 21, 2020
« Older The first science fiction story | Sophia Robot, first robotic citizen Newer »
This thread has been archived and is closed to new comments
The appeals process which might (and this is very much not guaranteed) consider mock marks for the grades is likely to get some amended marks back before the cut off for students to get their original programmes and others not. Unis will hold courses until that date. Of course if you hold on and don't get the upgrade or it doesn't come in time then clearing will be pretty much done with, especially at the Russell Group unis.
Williamson is apparently concerned (behind a telegraph paywall) that some student might get promoted beyond their competence. Introspection not being a strong suit. He means poor kids obviously.
posted by biffa at 6:03 AM on August 13, 2020 [14 favorites]