The inhumanity of Ofqual's algorithm
August 25, 2020 1:12 AM Subscribe
The inhumanity of Ofqual's algorithm is a deep dive into the fiasco where British high school students were awarded grades by algorithm with exams cancelled due to COVID, until the government reversed course under pressure. But what happens to universities now?
And because the algorithm was not understood by whoever was in ultimate charge of the process (was there anyone in ultimate charge?), the u-turn now means that in addition to f**king up the lives of many A-level students this year, A-level students next year will now find fewer places available at universities suffering even deeper financial woes. As universities have now accepted too many students this year - both those given offers after the original results and those who met their offer following the change to predicted grades - or deferred many offers until next year, those taking A-levels in 2021 will face higher offers and lower acceptance rates and have zero confidence in the entire process.
posted by humuhumu at 2:21 AM on August 25, 2020 [4 favorites]
posted by humuhumu at 2:21 AM on August 25, 2020 [4 favorites]
Presumably, where a university course builds on knowledge acquired during your A-levels, the universities are also going to need to extend their first term's teaching to cover what should have been taught in the Easter and summer terms of the upper sixth, because they can't safely assume that all students will have been able to complete the syllabus. I can't imagine that having a big gap in your knowledge is going to help you find your feet in a maths degree, for instance.
posted by ManyLeggedCreature at 2:32 AM on August 25, 2020 [1 favorite]
posted by ManyLeggedCreature at 2:32 AM on August 25, 2020 [1 favorite]
This was spectacularly stupid even by this government's standards. The Scottish example could not have been more ideally timed for the rest of the UK to learn from it. The Scots used the same type of approach, with the same results, leading to outrage and backlash just a couple of weeks before rUK did the exact same. Surely someone should have seen they would inevitably have to ditch the algorithm, and also pay a high price.
It has been amusing to see the Scottish Tories try to criticise the Scottish Government's handling (which was poor) while not criticising the Uk govt doing the exact same, two weeks later. However amusing it underlines a really frustrating thing. In so many casees this year we have seen bad things happen elsewhere, and no-one seems able to take the lessons and get in front of it. The UK could see what was happening in Italy and Spain who got hit with the pandemic first, and did little to use this information to mitigate the coming disaster. The same with these exam results.
Scottish schools have gone back for the new term as of last week (the Scottish education terms have always been earlier than rUK) and there have been some negative consequences (eg 17 teachers in a school in Dundee getting Corona). There will be lessons that can and should be learned from the Scottish experience by the rUK in the coming weeks but given the incompetence demonstrated I am fairly sure that this opportunity will also be squandered. (Not that I am calling for Scotland to be used as a lab rat, as so often happens, and in my own personal view the return to school here in Scotland will be seen as a mistake in couple of weeks, when cases rise again and we belatedly realise that kids are actually humans, and therefore can be vectors for this disease, even if seemingly somewhat less of a vector than adults.)
posted by Gratishades at 2:46 AM on August 25, 2020 [7 favorites]
It has been amusing to see the Scottish Tories try to criticise the Scottish Government's handling (which was poor) while not criticising the Uk govt doing the exact same, two weeks later. However amusing it underlines a really frustrating thing. In so many casees this year we have seen bad things happen elsewhere, and no-one seems able to take the lessons and get in front of it. The UK could see what was happening in Italy and Spain who got hit with the pandemic first, and did little to use this information to mitigate the coming disaster. The same with these exam results.
Scottish schools have gone back for the new term as of last week (the Scottish education terms have always been earlier than rUK) and there have been some negative consequences (eg 17 teachers in a school in Dundee getting Corona). There will be lessons that can and should be learned from the Scottish experience by the rUK in the coming weeks but given the incompetence demonstrated I am fairly sure that this opportunity will also be squandered. (Not that I am calling for Scotland to be used as a lab rat, as so often happens, and in my own personal view the return to school here in Scotland will be seen as a mistake in couple of weeks, when cases rise again and we belatedly realise that kids are actually humans, and therefore can be vectors for this disease, even if seemingly somewhat less of a vector than adults.)
posted by Gratishades at 2:46 AM on August 25, 2020 [7 favorites]
I don't think universities usually assume that all of the students will arrive with all of the necessary understanding in place. Sometimes places are given to students that didn't study a key subject at A-level, for instance. And students who did take the A-level won't all have been on the same syllabus, so there can be differences in their base knowledge. I can remember a math(s) course being part of the first year of my physics degree, for those reasons.
The conservatives own this mess, one way or another. And to try to hide that fact, they'll try to palm off the blame on someone else (as with PHE over the covid disaster) and use it to justify centralising control of education even more than it already is. But that centralisation, with its over-reliance on exam results rather than gradual assessment, was a big part of the problem to begin with.
posted by pipeski at 2:48 AM on August 25, 2020
The conservatives own this mess, one way or another. And to try to hide that fact, they'll try to palm off the blame on someone else (as with PHE over the covid disaster) and use it to justify centralising control of education even more than it already is. But that centralisation, with its over-reliance on exam results rather than gradual assessment, was a big part of the problem to begin with.
posted by pipeski at 2:48 AM on August 25, 2020
In so many cases this year we have seen bad things happen elsewhere, and no-one seems able to take the lessons and get in front of it.If the same inexplicable thing keeps happening then it's not inexplicable.
They're not trying to get in front of it.
Why would they?
What repercussions have there been for 65K dead PLUS economic disaster?
It works.
posted by fullerine at 3:21 AM on August 25, 2020 [2 favorites]
This is affecting sixth-form places as well. Our son appears to have been one of the beneficiaries, to an extent. The grades that mattered look like they are teacher assessments. Within what he's capable of, but towards the top end. He is fortunate and has gained a place at sixth form, which we thought was unlikely. He already had an offer, though. But like universities, the sixth-form sector is also dealing with a flood of unexpected good grades and additional students who want to take up places that may not exist, some of whom won't have the abilities their grades suggest. It's putting schools under a lot of pressure from local parents and there's the additional pressure of students wanting to follow friends to the same destination.
Talking with him, he and his friends know that they now have to prove themselves, and they are daunted by that. They know that if they don't live up to these grades, they lose their place and may struggle to then get a place in an FE college because those will also be overfilled. I can only hope that the school is going to take steps to help them recover all the knowledge and understanding lost from five months at home. I can remember distinctly how much students used to forget over just a normal summer.
So ex fireplace salesman Williamson's idiotic comment about not wanting to create the generation that's promoted beyond its abilities is bullshit (we knew that anyway). Far from coasting onwards and upwards, these students know they need to deliver or they will fail at the next hurdle, and possibly without a safety net because there are no alternative places.
posted by dowcrag at 3:23 AM on August 25, 2020 [2 favorites]
Talking with him, he and his friends know that they now have to prove themselves, and they are daunted by that. They know that if they don't live up to these grades, they lose their place and may struggle to then get a place in an FE college because those will also be overfilled. I can only hope that the school is going to take steps to help them recover all the knowledge and understanding lost from five months at home. I can remember distinctly how much students used to forget over just a normal summer.
So ex fireplace salesman Williamson's idiotic comment about not wanting to create the generation that's promoted beyond its abilities is bullshit (we knew that anyway). Far from coasting onwards and upwards, these students know they need to deliver or they will fail at the next hurdle, and possibly without a safety net because there are no alternative places.
posted by dowcrag at 3:23 AM on August 25, 2020 [2 favorites]
In terms of difficulty with number of university places for 2021 entry, while there will be more 18 year olds in 2021 than in this year or last we will still be in a bit of a demographic trough, there will be less than in the years 2005-2015, plus way more uni spaces than in that period. Whether there are more still will depend on how government and the universities follow up on this year. Many of the Russell Group will be swamped this year and while some institutions and some individual programmes will push deferral most students will start in September. Some institutions will still be below what they need economically and institutional failure probably represents the biggest threat to overall places. However, if this year the RG unis are taking 20-40% above previous capacity then there's every chance they will aim to take an above typical number of students next year too, especially given the hit they will take on European research income from 2021. The question is thus whether the government asserts a cap again for 2021 entry.
posted by biffa at 3:55 AM on August 25, 2020 [1 favorite]
posted by biffa at 3:55 AM on August 25, 2020 [1 favorite]
This past few months has just been
Scotland: we're doing a thing
England: pfft, we're definitely not doing that thing everyone!
a week or two later
England: we're doing the thing.
posted by stillnocturnal at 4:12 AM on August 25, 2020 [9 favorites]
Scotland: we're doing a thing
England: pfft, we're definitely not doing that thing everyone!
a week or two later
England: we're doing the thing.
posted by stillnocturnal at 4:12 AM on August 25, 2020 [9 favorites]
I say "inhuman", because for me that is what is fundamentally wrong with the "algorithm", or rather with the logical process of which the mathematical model is part. Both the model itself and its attendent processes were biased and unfair. But they are merely illustrative of the underlying - and fatal - problem. The algorithm did not treat people as individuals. It reduced them to points on a curve. It was, in short, inhuman.
This is exactly right, the linked post is really an excellent summary.
Ofqual and DfE people really thought they'd cracked the problem. When they were asked whether there was any bias in the algorithm they answered honestly (in their way of thinking) that there was not, schools were getting comparable results to previous years. Schools with high levels of deprivation were not being unfairly penalised because they were getting the same results as before. So, sure, technically there was no bias in the algorithm.
Except...
They missed the human aspect by only looking at statistical aggregates. Someone who gets a C when they were predicted an A by their school is unlikely to derive any comfort from the fact that statistically groups always underperform their "expected" grades because those grades are not actually expectation values in a statistical sense. Ofqual, which is mostly statisticians, can *almost* be forgiven for missing this fundamental question but this is really where ministers need to take responsibility. It is precisely to represent the perspective of the public that we have elected MPs serving as ministers who are directly accountable to parliament.
Similarly, the fact that schools with high levels of deprivation have bigger gaps between their expected grades and achieved grades every year not just this one makes perfect sense to someone who looks at the data all the time. It's mostly because you can't "overpredict" the grades of someone who gets an A* on the exam, therefore the gap between expected and achieved is lower the better the exam performance because both the predicted and actual grades are maxed out and therefore equal. Ofqual could again be partially forgiven for not understanding how that would look to the public which is that the algorithm "downgrades" (not really technically correct but people won't see it that way) more at schools with high deprivation. Ministers cannot be forgiven for missing that, it is their job to know better. For all the jokes about Williamson being a fireplace salesman, the fundamental role of elected politicians and of ministers is to look at things from a non-expert but highly briefed (and hopefully representative) point of view. It's not his fault that he can't test an algorithm for statistical correctness, it is his fault that he didn't arrange for external review (NDAs and other obstacles be damned) and it is flagrantly and absolutely his fault that he didn't spot that they were rolling into a whole warehouse full of political and social bear traps with this approach.
It very effectively pissed off a substantial number of moderate-information home counties professionals - essentially the base of the Conservative party. Remember, they don't give a shit what Metafilter thinks of them, why would they? UK voters who are MeFi members are not likely Conservative voters anyway.
Also: they missed the effect of groups which were not algorithmically scored because they were too small. While that's totally justifiable on statistical grounds, it is private schools that have virtually all of the 5 or less groups and most of the 15 or less groups.
There's loads of other technical issues as well which have been extensively discussed in that previous thread.
Ultimately Williamson and Gibb should resign for not spotting and handling the fundamental problem which is that you cannot simultaneously fit results to prevent grade inflation and not create individual if not aggregate injustices and people will not tolerate individual injustice for their own children on the basis of statistical arguments.
Senior people at Ofqual and probably DfE should resign for the technical issues with the algorithm itself.
posted by atrazine at 4:29 AM on August 25, 2020 [10 favorites]
This is exactly right, the linked post is really an excellent summary.
Ofqual and DfE people really thought they'd cracked the problem. When they were asked whether there was any bias in the algorithm they answered honestly (in their way of thinking) that there was not, schools were getting comparable results to previous years. Schools with high levels of deprivation were not being unfairly penalised because they were getting the same results as before. So, sure, technically there was no bias in the algorithm.
Except...
They missed the human aspect by only looking at statistical aggregates. Someone who gets a C when they were predicted an A by their school is unlikely to derive any comfort from the fact that statistically groups always underperform their "expected" grades because those grades are not actually expectation values in a statistical sense. Ofqual, which is mostly statisticians, can *almost* be forgiven for missing this fundamental question but this is really where ministers need to take responsibility. It is precisely to represent the perspective of the public that we have elected MPs serving as ministers who are directly accountable to parliament.
Similarly, the fact that schools with high levels of deprivation have bigger gaps between their expected grades and achieved grades every year not just this one makes perfect sense to someone who looks at the data all the time. It's mostly because you can't "overpredict" the grades of someone who gets an A* on the exam, therefore the gap between expected and achieved is lower the better the exam performance because both the predicted and actual grades are maxed out and therefore equal. Ofqual could again be partially forgiven for not understanding how that would look to the public which is that the algorithm "downgrades" (not really technically correct but people won't see it that way) more at schools with high deprivation. Ministers cannot be forgiven for missing that, it is their job to know better. For all the jokes about Williamson being a fireplace salesman, the fundamental role of elected politicians and of ministers is to look at things from a non-expert but highly briefed (and hopefully representative) point of view. It's not his fault that he can't test an algorithm for statistical correctness, it is his fault that he didn't arrange for external review (NDAs and other obstacles be damned) and it is flagrantly and absolutely his fault that he didn't spot that they were rolling into a whole warehouse full of political and social bear traps with this approach.
It very effectively pissed off a substantial number of moderate-information home counties professionals - essentially the base of the Conservative party. Remember, they don't give a shit what Metafilter thinks of them, why would they? UK voters who are MeFi members are not likely Conservative voters anyway.
Also: they missed the effect of groups which were not algorithmically scored because they were too small. While that's totally justifiable on statistical grounds, it is private schools that have virtually all of the 5 or less groups and most of the 15 or less groups.
There's loads of other technical issues as well which have been extensively discussed in that previous thread.
Ultimately Williamson and Gibb should resign for not spotting and handling the fundamental problem which is that you cannot simultaneously fit results to prevent grade inflation and not create individual if not aggregate injustices and people will not tolerate individual injustice for their own children on the basis of statistical arguments.
Senior people at Ofqual and probably DfE should resign for the technical issues with the algorithm itself.
posted by atrazine at 4:29 AM on August 25, 2020 [10 favorites]
This past few months has just been
Scotland: we're doing a thing
England: pfft, we're definitely not doing that thing everyone!
a week or two later
England: we're doing the thing.
It's really quite remarkable how the comparison with England makes the otherwise pretty poor Scottish response look excellent.
posted by atrazine at 5:31 AM on August 25, 2020 [2 favorites]
Scotland: we're doing a thing
England: pfft, we're definitely not doing that thing everyone!
a week or two later
England: we're doing the thing.
It's really quite remarkable how the comparison with England makes the otherwise pretty poor Scottish response look excellent.
posted by atrazine at 5:31 AM on August 25, 2020 [2 favorites]
ex fireplace salesman Williamson
Were they made of chocolate?
posted by Cardinal Fang at 5:45 AM on August 25, 2020 [1 favorite]
Were they made of chocolate?
posted by Cardinal Fang at 5:45 AM on August 25, 2020 [1 favorite]
the Scottish example could not have been more ideally timed for the rest of the UK to learn from it. The Scots used the same type of approach, with the same results, leading to outrage and backlash just a couple of weeks before rUK did the exact same.
The Tories ignoring a warning of an impending disaster that was playing out nearby and pretending that the same thing wasn't about to affect them too? Hmm, what does that remind me of in early 2020...
posted by EndsOfInvention at 6:18 AM on August 25, 2020 [1 favorite]
The Tories ignoring a warning of an impending disaster that was playing out nearby and pretending that the same thing wasn't about to affect them too? Hmm, what does that remind me of in early 2020...
posted by EndsOfInvention at 6:18 AM on August 25, 2020 [1 favorite]
There are thorny philosophical issues of individual vs. group fairness, predictive accuracy, etc. that come up and will continue to come up as long as statistical models are used to make decisions. This is not a new 21st century phenomenon; many of these debates have been going on, unresolved, since the 1970s.
I think the mistake, from a public relations perspective, may have been actually reporting the fictional exam results. If they had been able to go directly from predicted scores/statistical model to university acceptance in a more opaque way, it might not have tipped people's sense of unfairness so much.
Whereas making up specific imaginary exam results for each individual student makes it very concrete to everyone that they are being evaluated based on nothing. Exam scores are in some narrow sense based on individual performance, there are normally immense efforts spent to make them as (procedurally) fair as possible. So to get what look like objective exam scores, that were actually just made up, I think would immediately make almost anyone angry.
posted by vogon_poet at 6:50 AM on August 25, 2020 [3 favorites]
I think the mistake, from a public relations perspective, may have been actually reporting the fictional exam results. If they had been able to go directly from predicted scores/statistical model to university acceptance in a more opaque way, it might not have tipped people's sense of unfairness so much.
Whereas making up specific imaginary exam results for each individual student makes it very concrete to everyone that they are being evaluated based on nothing. Exam scores are in some narrow sense based on individual performance, there are normally immense efforts spent to make them as (procedurally) fair as possible. So to get what look like objective exam scores, that were actually just made up, I think would immediately make almost anyone angry.
posted by vogon_poet at 6:50 AM on August 25, 2020 [3 favorites]
If they had been able to go directly from predicted scores/statistical model to university acceptance in a more opaque way, it might not have tipped people's sense of unfairness so much.
How might this have worked? A students gets 5 offers, accepts say Oxford as their Firm and Bristol as their Insurance. So if they get the grades they go to Oxford, if they miss but are high enough they go to Bristol, if neither then into clearing or maybe they'll opt to resit. Would you just send a letter to all the comp students telling them they're going to Lincoln? I think that might still have led to pushback. And it would still be an outrageous injustice. And its justice that is the think here I think, and the injustice of saying to someone: You can't change your stars, no matter how hard you work, how you meet the hurdles we put in front of you, then we'll make sure your options remain limited.
posted by biffa at 7:04 AM on August 25, 2020 [3 favorites]
How might this have worked? A students gets 5 offers, accepts say Oxford as their Firm and Bristol as their Insurance. So if they get the grades they go to Oxford, if they miss but are high enough they go to Bristol, if neither then into clearing or maybe they'll opt to resit. Would you just send a letter to all the comp students telling them they're going to Lincoln? I think that might still have led to pushback. And it would still be an outrageous injustice. And its justice that is the think here I think, and the injustice of saying to someone: You can't change your stars, no matter how hard you work, how you meet the hurdles we put in front of you, then we'll make sure your options remain limited.
posted by biffa at 7:04 AM on August 25, 2020 [3 favorites]
The calculated results for the Leaving Certificate in Ireland will be coming out in early September, based on teacher predictions/schools aligning all teacher results/the departments applying standardisation and validation based on previous years, afaik. It has the upside of there not being predicted results (just mock exams, if they did those prior to lockdown) but I really hope someone's sweating over every detail of what Ofqual did and making sure they're avoiding as many pitfalls as possible.
I kind of had a nervous breakdown during my mocks (my final exams were better thanks mostly to my mum taking weeks off to literally drive me to a quiet river and feed me lunch and say calm things between exams) and know that's sadly not unique - the stage where the lockdown hit the exam cycle, and the suspicions of teacher bias, would be making me so apprehensive.
posted by carbide at 7:17 AM on August 25, 2020 [2 favorites]
I kind of had a nervous breakdown during my mocks (my final exams were better thanks mostly to my mum taking weeks off to literally drive me to a quiet river and feed me lunch and say calm things between exams) and know that's sadly not unique - the stage where the lockdown hit the exam cycle, and the suspicions of teacher bias, would be making me so apprehensive.
posted by carbide at 7:17 AM on August 25, 2020 [2 favorites]
Senior people at Ofqual and probably DfE should resign for the technical issues with the algorithm itself.
Ofqual head Sally Collier resigns over exams fiasco
posted by EndsOfInvention at 7:23 AM on August 25, 2020 [1 favorite]
Ofqual head Sally Collier resigns over exams fiasco
posted by EndsOfInvention at 7:23 AM on August 25, 2020 [1 favorite]
I was wondering why they took an algorithm that looked like they didn't even try to take into account individual students qualifications for predicting outcomes. Data science has come a long way, hy are they using such a simple that obviously can't be right. Did they even check how accurate their results were?
But reading the articles, yes they did check. They found it was pretty terrible. And weren't even able to calculate how inaccurate it was because teachers didn't rank their students in 2019, so there was nothing to compare that data to. All the maths in the world won't save you if you haven't collected the data.
So as far as I can tell, the only individualized data available that included historical results to compare against was the mock exam data. So literally the only honest model you can make with that is to "just use the practice exam grades." Because statistics are not magic - they cannot make information where there is none. But I bet that sort of advice was not making the politicians and people up top happy.
The people making this model got told to do something actually impossible. No wonder it didn't work.
posted by Zalzidrax at 7:45 AM on August 25, 2020 [17 favorites]
But reading the articles, yes they did check. They found it was pretty terrible. And weren't even able to calculate how inaccurate it was because teachers didn't rank their students in 2019, so there was nothing to compare that data to. All the maths in the world won't save you if you haven't collected the data.
So as far as I can tell, the only individualized data available that included historical results to compare against was the mock exam data. So literally the only honest model you can make with that is to "just use the practice exam grades." Because statistics are not magic - they cannot make information where there is none. But I bet that sort of advice was not making the politicians and people up top happy.
The people making this model got told to do something actually impossible. No wonder it didn't work.
posted by Zalzidrax at 7:45 AM on August 25, 2020 [17 favorites]
I think there are two really fundamental points made by atrazine and Zalzidrax above:
And, while I've read a lot of stuff about what is wrong with the algorithm, I've not really seen anyone make a convincing case for an alternative that doesn't either arbitrarily penalise some individuals, or result in universities having a very challenging distribution of students (for any given university either too many having high enough grades, or not having enough at all). Anyone who made any claim that this was doable was wrong.
With the benefit of hindsight, maybe they should have cancelled all UCAS offers, gotten grades out in April/May and done a post-grades admissions round. Would have had to make a decision in March really on that though.
posted by plonkee at 8:40 AM on August 25, 2020 [4 favorites]
- people will not tolerate individual injustice for their own children on the basis of statistical arguments
- the people making this model got told to do something actually impossible
And, while I've read a lot of stuff about what is wrong with the algorithm, I've not really seen anyone make a convincing case for an alternative that doesn't either arbitrarily penalise some individuals, or result in universities having a very challenging distribution of students (for any given university either too many having high enough grades, or not having enough at all). Anyone who made any claim that this was doable was wrong.
With the benefit of hindsight, maybe they should have cancelled all UCAS offers, gotten grades out in April/May and done a post-grades admissions round. Would have had to make a decision in March really on that though.
posted by plonkee at 8:40 AM on August 25, 2020 [4 favorites]
One exceedingly stupid but predictable result of the algorithm is that the amount of variation depends on how much data they have. Looking at the results it's clear that something is messed up. My partner teaches logic at Warwick (ranked fourth in the UK for maths), and the offer is very high: three A*s and two of those must be Maths and Further Maths. More students sit for the A level in Maths as compared to Further Maths, and so there were some exceedingly weird results where students got an A* in Further Maths but only an A in Maths. This is... really unlikely to reflect their actual competence.
posted by tractorfeed at 11:05 AM on August 25, 2020 [4 favorites]
posted by tractorfeed at 11:05 AM on August 25, 2020 [4 favorites]
I really hope someone's sweating over every detail of what Ofqual did and making sure they're avoiding as many pitfalls as possible.
I hope for the sake of the Leaving Cert kids that they haven't messed up, but at the same time, I'm not that optimistic. And to be honest, I think that given the last few weeks, there will probably be an outcry even if it all goes mostly well, and just a few people are "downgraded". I'm also not entirely sure how it will play out with the points system - making me really glad that my various niblings are too young for this to have an impact on them.
posted by scorbet at 3:35 AM on August 26, 2020
I hope for the sake of the Leaving Cert kids that they haven't messed up, but at the same time, I'm not that optimistic. And to be honest, I think that given the last few weeks, there will probably be an outcry even if it all goes mostly well, and just a few people are "downgraded". I'm also not entirely sure how it will play out with the points system - making me really glad that my various niblings are too young for this to have an impact on them.
posted by scorbet at 3:35 AM on August 26, 2020
Head of Ofqual, Sally Collier - resigned
Permanent Secretary (most senior civil servant) at the Department of Education, Jonathan Collier - just resigned
Government ministers responsible for education - ¯\_(ツ)_/¯
posted by reynir at 8:54 AM on August 26, 2020
Permanent Secretary (most senior civil servant) at the Department of Education, Jonathan Collier - just resigned
Government ministers responsible for education - ¯\_(ツ)_/¯
posted by reynir at 8:54 AM on August 26, 2020
Permanent Secretary (most senior civil servant) at the Department of Education, Jonathan Collier
Jonathan Slater (in case anyone else was wondering if they were related.)
posted by scorbet at 9:16 AM on August 26, 2020 [1 favorite]
Jonathan Slater (in case anyone else was wondering if they were related.)
posted by scorbet at 9:16 AM on August 26, 2020 [1 favorite]
« Older Republican National Convention Begins | It’s Time to Abolish Nursing Homes Newer »
This thread has been archived and is closed to new comments
(This is such an omnishambles that I think it's fair to have an update!)
posted by adrianhon at 1:45 AM on August 25, 2020