Students needed more support, not more surveillance.
January 10, 2021 11:41 AM   Subscribe

"Proctoring software is some of the most outrageous "cop shit" in schools right now." Educational technology critic Audrey Watters on school surveillance.

These tools gather and analyze far more data than just a student's exam responses. They require a student show photo id to their laptop camera, then match that data that to the student's 'biometric faceprint'. They capture audio and video from the session — the background sounds and scenery from a student's home. Some ask for a tour of the student's room to make sure there aren't "suspicious items" on the walls or nearby. Some also capture a student's keystrokes, track location data, pinpointing where the student is working.

From a Dec 2020 #AgainstSurveillance talk "Behaviorism, Surveillance, and (School) Work: "The co-founders of Blackboard, incidentally, are back with a new startup: an administrative layer on top of Zoom to make it work "better" for school, they say, with features like attendance-taking, test-proctoring, and eye-tracking."

The Dec talk was in support of Ian Linkletter, an ed-tech specialist being sued by online test proctoring company Proctorio: “This video from Proctorio’s YouTube channel shows how the Abnormal Eye Movement function works... This is the one that will show you, beyond a doubt, the emotional harm you are doing to students by using this technology.”

Audrey Watters & hackeducation.com previously.
posted by spamandkimchi (43 comments total) 51 users marked this as a favorite
 
Remote, real-time exams are an absurd delusion sold by snake-oil dealers to instructors who either hate their students or know nothing about pedagogy. If that's how your choose to structure your exams, you should probably quit your job and let someone who might be competent have a try at it.
posted by eotvos at 12:00 PM on January 10, 2021 [28 favorites]


Certain large midwestern software companies (*cough* hospital software *cough*) use this stuff on job applicants as well. Not sure what they think they're getting out of it.
posted by aramaic at 12:05 PM on January 10, 2021 [3 favorites]


There's a good science fiction story about this: Not Smart, Not Clever, by E. Saxey.
posted by doctornemo at 12:13 PM on January 10, 2021 [12 favorites]


the best exams--the exams that best test real knowledge and practical skills--are and have always been open book. this would translate easily into remote learning because an open book exam doesnt care where you are or what else you are looking at. in the real world, in a real job, you have an open book but limited time to apply it, and that's what exams should test. testing memory and recall just encourages cramming and proves nothing.
posted by wibari at 12:40 PM on January 10, 2021 [51 favorites]


Products of this nature should be illegal.
posted by wordless reply at 12:43 PM on January 10, 2021 [21 favorites]


eotvos: I would love to hear more on the topic. What is your solution? I hate exams for many different reasons and know they aren't good for my students. I just haven't figured it out. We have rampant cheating always and are just playing wack-a-mole. Open book exams are great - as long as you don't have four people sitting in a room doing the exam together with a paid tutor (true story). Huge test banks are great - as long as the questions aren't posted to chegg (fucking chegg). Remote exams just open a huge door for cheating. Many students prefer proctorio to an actual zoom room where I watch them myself. It also doesn't feel fair to the honest students to not make some effort. I teach 1000+ person classes (which is really the root of the problem, but it's the system I'm in for now).
posted by lab.beetle at 12:54 PM on January 10, 2021 [15 favorites]


instructors who either hate their students or know nothing about pedagogy.... let someone who might be competent have a try at it.

Can we not be so quick to jump to trashing teachers, especially when they(we) are often under immense pressures without adequate support and/or training? Also, school boards and administrations often with little connection to classroom teaching are the ones making the decisions to buy and implement these programs.
posted by Saxon Kane at 12:59 PM on January 10, 2021 [44 favorites]


I'm so glad this crap didn't exist when I was a troubled but smart kid in school. I'd probably be in jail under some bullshit electronic tampering/hacking charges because I sure would have fought back and would have been unable to resist messing with it.

On the other hand and on a much darker note they probably would have seen my room and would have rightfully sent CPS out for a welfare check which might have been a good idea, in theory at least.

On the other other hand I can't imagine the pressure of being on camera like that and invasively inspected like that. My childhood room naturally looked like a tornado happened in a weird laboratory mixed with a new wave or darkwave band or something, and to be frank it kind of still does look like that.
posted by loquacious at 1:36 PM on January 10, 2021 [6 favorites]


in the real world, in a real job, you have an open book but limited time to apply it, and that's what exams should test.

I'm inclined to like this idea, but then again, I'm also good at taking timed tests. And I've been around enough educators to know that this doesn't quite cut it. Many timed tests aren't so great for people who aren't neurotypical or have learning disabilities. In the current lockdown situation, many students are learning from home where they have expectations from their families to do things like childcare: are they allowed to stop the timer when their little brother needs attention?

If I were teaching, I think I'd prefer open book tests with a time limit that was *way* more than sufficient, so that I am testing students' mastery of the material rather than their ability to decipher a test properly under time limits.

I'm sure that there are some areas where being able to do something quickly is an important factor, but I don't believe that is generally true in the educational setting.

I would absolutely have hated software that spied on me, and am happy I went to college at a place where, for the most part, the honor code worked, and I regularly had tests that took a couple of hours to do but had 24-48 hour time limits.
posted by grae at 1:45 PM on January 10, 2021


Hey! Don't forget! It's also enormously expensive and not very effective!
Seriously, the associate dean called me at the beginning of last semester an nervously asked me if my exams would need online proctoring, and to please not request it unless I "really needed" it, because we just cant afford it. Apparently the engineering folks are convinced they have a terrible cheating problem (which they really might, they would know better than I), and are demanding online proctoring left and right.
Anyway, joke's on them; I don't use exams.
posted by Adridne at 1:52 PM on January 10, 2021 [11 favorites]


I'm in the field of "ed tech" and I hate all proctoring software with a passion, luckily, the organization I'm in doesn't have to work with it.

The story linked above regarding Ian Linkletter deserves massive amplification. Linkletter was sued by Proctorio, makers of academic surveillance software.

Why was he sued? Because he tweeted out the links for 7 unlisted YouTube videos made by Proctorio for training purposes, as well as a screenshot of its website. These videos demonstrated the degree to which administrators can require test-takers to comply with various intrusive requirements such as being asked to document the room you are taking the test in with video.

The company claimed he’d infringed its copyright and distributed confidential material. (If you're not familiar with how unlisted YouTube videos work, they are public URLS available to anyone with the correct address, but are not visible to Google or YT's search engines)

Linkletter has been under a court-ordered injunction for months. I contributed to his GoFundMe legal defense fund and you may want to consider doing the same.
  • To those asking "Why would an online proctoring company go after a lone professor to such a degree?" Well, besides being inherent assholes, there is BIG money in this field. Ed tech in general has been susceptible to snake oil salesmen with questionable technology since the beginning.
  • If I were to guess, I would also say that companies like Proctorio have set their sights on bigger targets. Just imagine the amounts of data they are gathering to refine their "suspicion level" algorithms and how they are planning to expand their reach. Want to figure out if your job applicants are lying on job interviews? Buy Proctorio.
Many of Proctorio's competitors rely on human proctors to check in on test-takers. They are just doing what many others have done before them: Use the promise of superior technology to appeal to the cost-cutters and bean counters of (an admittedly) underfunded industry.
posted by jeremias at 2:11 PM on January 10, 2021 [27 favorites]


Not sure what they think they're getting out of it.

Well, whoever convinced the hospital system to buy it is getting a pretty little kickback and the proctoring service is getting juicy resellable data. Neither of them give a shit what anybody else gets out of it.

This is true of school boards too. Don't blame teachers for this, they got no say in it.
posted by Lyn Never at 2:11 PM on January 10, 2021 [4 favorites]


This is also deeply unfair under Covid to people with children and pets. Of course my eye is looking off the screen. I may even need to leave the room. I literally can't lock myself away from my family for two hours.
posted by corb at 2:24 PM on January 10, 2021 [21 favorites]


If that's how your choose to structure your exams, you should probably quit your job and let someone who might be competent have a try at it.

Like, I want to favorite this - and did for a moment - but then I remembered that I used remote, real-time exams last semester because I didn't know what to do instead. I care about pedagogy, but didn't have enough time or training to figure out how to do assessments differently.

Even with more time, I don't know how I'd do assessments differently for the types of classes that I've taught. I read a lot about how to best do assessments for online classes, and never saw an alternative where I thought, "yes... that would work for this class, is fair to the students, and I have the resources and knowledge to do it."

Honestly, the disconnect between how much I cared about doing it right, versus the training, resources, and time I was given to help me do it right ... that's one of the reasons I'm probably leaving academia. Someone who cares less can have my job.

(I didn't use proctoring software though, because it's unethical.)
posted by Kutsuwamushi at 2:29 PM on January 10, 2021 [21 favorites]


Ai surveillance isn't just for the remote teacher, it's also for the remote employer. I supervise delivery drivers for a startup. We use an app called "samsara" which hooks into the cars computer, and has cameras facing in and out under the rear-view mirror.

My boss showed me the dashboard. It's a bunch of photos of my face looking at my phone, looking out the window, etc. It pings him every time I go over the GPS speed limit, don't stop fully at a stop sign, or get into an accident.

I suspect that there is a liability reason for using this softwear.

I think delivery companies probably despise their drivers. They add variability. More aggressive drivers reduce time to deliver but increase risk. They're going to be so happy when the car drives itself and the humans can be in charge of loading and unloading the back. Then the car will be the supervisor, and can punish the employees for, I donno, not walking on ice safe enough.

And I worry what this means for education systems as well. A robot teacher preparing our youth for careers being observed by robots.
posted by rebent at 3:06 PM on January 10, 2021 [20 favorites]


Seconding what rebent said. There's Mobile Device Management software specifically targeted at gig workers and delivery workers both for company issued mobile devices and packaged in workplace apps for bring your own devices for gig workers.

If you've ever offered a gig/delivery worker a cash tip or material reward or snack like a bottle of water or something and they've immediately and vocally refused or seemed awkward about it - it's likely that they know or suspect that this MDM software is spying on them or monitoring the hand-off and transaction with video, photos and/or audio.

It's written into the terms of employment and service of a lot of gig worker jobs like Postmates and I remember Postmates app and company issued phone had some really heavy MDM software installed on it that I investigated and poked at, and companies like Postmates don't like cash tips because they don't get their cut.
posted by loquacious at 3:19 PM on January 10, 2021 [6 favorites]


A family member took a praxis test yesterday and it was like a grooming test for malware installation with all of the software requiring full admin access . They lied about supporting Firefox but the second remote control system they want you to install doesn’t work in anything but Chrome, their “support” system tries to burn your time in a chatbot before they’re forced to incur the unbearable cost of having a human answer questions, and the face tracking & room monitoring software seems tailor made to cause problems for some users. I’m also just assuming that we’ll be getting another breach notification at some point.
posted by adamsc at 3:21 PM on January 10, 2021 [8 favorites]


This stuff is so gross. I'm thankful that if the school I teach at tried this there's no way teachers would go along with it.

Remote assessment is a hard problem and hasn't been solved in a way I am happy with. I've tried open-book free response with a relaxed time limit. Got (some) responses that were clearly copied. I've used multiple-choice question-randomizing software but it just makes the content feel so dumbed down. And I've spent a lot of energy just being front-line tech support for the various platforms.

I fear that as soon as my school goes back in-person (end of spring maybe?) that every teacher will simultaneously say "aha now we can give tests again" and students lives will just be non-stop quizzing until the summer. I hope I'm wrong.

Back to the eye tracking, it's not all about online school either. Some of the more dystopian schools I observed at (mainly big urban charters) had rules requiring student to track the teacher with their eyes. Students that didn't could get demerits or other consequences. Combine harsh rules like that with tech-based surveillance and you get into a scary place quick.
posted by Wulfhere at 3:22 PM on January 10, 2021 [7 favorites]


They capture audio and video from the session — the background sounds and scenery from a student's home. Some ask for a tour of the student's room to make sure there aren't "suspicious items" on the walls or nearby. Some also capture a student's keystrokes, track location data, pinpointing where the student is working.

The horrific part of this is that every suburban helicopter parent probably thinks this all makes good sense.
posted by Thorzdad at 3:42 PM on January 10, 2021 [2 favorites]


I've been using open book, open notes, no collaboration tests during the pandemic. Each student gets a randomized set of multiple choice questions and then a set of short answer questions that are not googleable (because I wrote them and use made-up scenarios). The tests are open for ~3 days. I provide a class session within those 3 days for students to use to take the test, helpful for their scheduling purposes and in case they have questions for me while taking it, but they are welcome to take it at any other time.

My grade distribution has looked essentially the same as back in the pre-pandemic times when students took similar tests closed book in the classroom. Students who come to Zoom class, do the homework, and study do well on the tests. Students who don't cannot possibly make up for that and just take the test cold, open book or no.

Has there been cheating? Probably. Has it been effective, useful cheating that helped their grades? No. When several students who haven't studied or done the homework or prepared in any way are texting each other about one of my made up short answer questions, it indeed goes no better than if they had done it on their own.

I hate online class and can't wait to go back to in person. But this testing has worked so well, I may not go back to in class tests. No "Respondus Lock Down Browser" or "Proctorio" required.
posted by hydropsyche at 5:12 PM on January 10, 2021 [17 favorites]


I teach fairly small classes (for a big state schoole) of about 30-40 students, and since I wasn't happy with the options provided by our courseware, and fancy myself as a programmer, decided to write my own test software. For added challenge, I chose to use a brand new IDE still in beta. I kinda knew this was a bad idea going into it, but it kept me really busy, and actually had some benefits (when the entire university system crashed, my home-brew thing was still running... haha!).

By going fully DIY, I could make the quizzes & tests work exactly as I wanted: I had quizzes which were low stakes: a deadline, but open for a week, credit for taking them regardless of score, everyone got the same questions, and questions had a "hint" mode to give some feedback, so there was basically zero motive to cheat. I made the quiz module responsive to work on cellphones, since I reazlied many of my students had marginal internet and limited access to laptops, and the university system is not great in that area.

Exams were higher stakes: timed, simultaneous, random order, forward navigation only, and I wrote all the questions myself. Open notes, but the time limit was fast enough that relying purely on notes wouldn't help.

Absolutely none of the surveillance / spyware crap.

Not strictly impossible to cheat, but a significant alignment between "able to cheat" and "able to master the coursework".

Like hydropsyche, I saw something similar: the kind of student who did well in person did well online, and the overall distribution of scores was quite similar to my prior semesters in-person.

It was a ton of work, though, and I, too, hate online teaching, and don't want to do it again.

Another thing I noticed was a pretty clear signal of inequity - in a normal semester I would have zero or 1 students drop out or fail. This semester it was over 5%, and concentrated in my minority students.
posted by soylent00FF00 at 5:45 PM on January 10, 2021 [8 favorites]


Back in my teaching-gigantic-classes days, I used WebWork for online homework: Its big feature is randomized problems, so that every student has the same problem with different numbers in it. Thus, you can't simply copy a classmates answers, but have to have them re-do the problem for you if you want to 'cheat.' In practice, this turns into peer teaching, which was fine by me... One could also set webwork to allow up to (say) 5 tries to get a problem right: this let the students take honest multiple shots at getting it on their own before going to another student for help.

I feel like this really got a lot of the incentives right: The 'helpful' students have finite time, so the 'needy' students want to make good use of it. So they put in a real effort before calling in a favor.

And that helps change perspective a bit, too: Why are your students trying to cheat in the first place? Because there's a high-pressure, single chance to get things right, which really just means that the incentives are all in the wrong place. And the incentives are often misaligned because teacher time is finite, and grading is an extremely time-consuming activity.

The usual criticism of online/automated grading is that there's no 'real' feedback. But this is a false dichotomy: one can set up some assignments specifically for getting 'real' feedback (eg, proof writing), and use regular online homework for any mechanical or rote work.

In short, if we use tech to relieve grading burdens, we can rearrange to get better incentives for everyone involved and hopefully avoid any 'need' for surveillance.

And, finally, as someone who had a really shit high school experience, I'll say that many teachers are heroes, but some of them definitely have cop-brains. Those ones suck, even if they do have a hard and mostly thankless job.
posted by kaibutsu at 5:56 PM on January 10, 2021 [13 favorites]


These types of software are being used for remote professional exams in the UK (exams that would be in person if not for Covid). A relative had to take an exam via this system, and it crashed three times during the exam. That may be unrelated to the surveillance part of the exam, but it came across as broken enough that anyone who has a motivation to cheat would be able to do so. Especialy as these exams cost hundreds (some well over a thousand) pounds to sit.
posted by Vortisaur at 6:01 PM on January 10, 2021 [1 favorite]


My instructors put a ton of effort into writing excellent exams that tested real, creative applications of scientific knowledge.

Aaaaand the previous year's students posted those exams in their entirety on Chegg. Now we get to take shitty randomized time-pressured test bank multiple choice exams with no opportunity to check our work, as a way to avoid cheating. It sucks.
posted by cnidaria at 6:25 PM on January 10, 2021 [12 favorites]


I have feelings about this! I'm not a big fan of Proctorio and their ilk, and I've chosen not to use them, but it's tough. Cheating is rampant - once a question is posted on Chegg, one of their "tutors" answers it. All of my test questions are their, I've filed takedown notices for hundreds of my questions. (Which you can do pretty easily). So I switched to fully algorithmic questions - new numbers and nouns for everyone! Only, our LMS doesn't support this, so I needed to use an outside company and the students had to pay. So like others, I switched to programming my own, which means that I spend hours each week scrambling to properly code up each quiz and test question (lower stakes assessments = more assessments), and I'm not a programmer! It's a lot of time and debugging on top of a more than full time job.

In short - I don't have the answers. Though one thing that never gets mentioned about Proctorio is that the human who goes through the video feed after suspicious behaviour is flagged by the AI is your actual instructor, and my colleagues who use it tell me that they almost never decide it's cheating when the video is flagged for suspicious eye movements.
posted by Valancy Rachel at 7:17 PM on January 10, 2021 [2 favorites]


I AM PROCTORIO
I NEED EYE TRACKING FOR MY BUNGHOLE

... Ahem.

kaibutsu, I used WebWork with my precalculus class this fall! I didn't use it for quizzes, just for practice problems during class, but it was great. 80% of the time, students could get what they needed from the instant feedback and unlimited attempts the system offered; this freed me up (and gave me the information I needed) to zero in on the other 20% and help them in real time. Partly I did this by tracking the student progress page to see who wasn't advancing or had made ten attempts to solve a problem -- but I largely didn't even need to do that, as it turned out students were much more likely to ask for help after making ten attempts that they knew were wrong than after making one attempt that they weren't sure was right.

My previous experience with WebWork was as a GSI for a large calculus course ten years ago, and consisted largely of assisting students with technical problems and non-intuitive syntax and precision requirements. I didn't expect to ever use it of my own accord, but I was glad I remembered it in 2020 because it made pandemic teaching suck less.
posted by aws17576 at 7:24 PM on January 10, 2021 [1 favorite]


I teach high school and use testing software where you can see the answers being inputted in real time. I've found that kids aren't that resourceful on a timed test, and you can tell who is cheating by who doesn't answer any questions at all in the first 20-30mins. Asking everyone to have their cameras on and setting a time limit has worked fine, in some cases (like if there's a technical problem) I just extend the time limit for only those students.

They could be lying about the technical problems, of course, but in general they don't bother - if they can't cheat DURING the testing time itself they just shrug and take the low grade. But I also don't care that much if they do cheat because it's remote instruction and they are teenagers and their mental health is more important.

These software products that do eye tracking sound dystopian. I've heard they're being implemented in some office environments, too. My gut is that as long as employees have options, the best ones definitely won't work in those environments. Seems like you're just driving the talent away.
posted by subdee at 8:40 PM on January 10, 2021 [4 favorites]


Well, it is one reason why I’m not at the aforementioned Midwestern software company... not the only reason, obvs, but it definitely didn’t help their image and the competition wasn’t so uptight so guess who won?
posted by aramaic at 8:57 PM on January 10, 2021 [2 favorites]


I withdrew from one of my classes last quarter rather than having to do a proctored exam. The whole week before, I was having panic attacks about it and when I started crying in front of my advisor over Zoom she told me to just withdraw and take it in person later to avoid it affecting my mental health so much.
posted by mollywas at 11:29 PM on January 10, 2021 [5 favorites]


Many students prefer proctorio to an actual zoom room where I watch them myself

Why not neither?

Those of us who teach essay courses have years of experience detecting academic misconduct when students write essays. The same antiplagiarism tactics work just as well for detecting substantially similar exams, or so I've found. (Context: short-medium- and essay length answers). I haven't done a zoom moderated exam all year.

As for chegg, I think I prefer it when cheaters do their cheating in public on a website I can access. Real interesting coincidence if a crudely patchwritten copy of a chegg tutor's answer ended up on someone's exam.

Seriously though, if your exams have paragraph+ answer questions and your institution doesn't require zoom/proctoio moderation then don't do that to your students. Let them write without a webcam in peace and privacy, then hunt cheaters afterwards.
posted by justsomebodythatyouusedtoknow at 2:59 AM on January 11, 2021 [3 favorites]


My instructors put a ton of effort into writing excellent exams that tested real, creative applications of scientific knowledge.

Aaaaand the previous year's students posted those exams in their entirety on Chegg. Now we get to take shitty randomized time-pressured test bank multiple choice exams with no opportunity to check our work, as a way to avoid cheating. It sucks.


I seem to be missing why Chegg is an issue - my university had past papers available for study anyway. Is that uncommon in the US? Its not as if you can get away with copying and pasting someone else answer.
posted by stillnocturnal at 4:55 AM on January 11, 2021 [2 favorites]


I don't have personal experience with these systems, but I heard a fairly in-depth discussion on radio which pointed out that the tracking:

- is an issue for students who already have test/exam anxiety
- penalize students who fidget/move/look around when they are thinking - a particular issue for students with ADHD but can just be people who are 'gazers'
- penalize students who have less control of their environments - share bedrooms, don't have quiet spots at home, have younger siblings running around, etc.
- and of course as with all remote learning tools, connectivity is an issue

I wonder if "catching cheaters" in the context of a well-structured course with multiple ways of assessing students is really worth the continued marginalization of students whose home environments or learning challenges aren't set up for these very rough tools. I get that in a traditionally proctored exam there are people walking around the room and checking IDs, etc.

I just wish some of the resources put into this kind of thing would be applied to increasing teaching resources.

The horrific part of this is that every suburban helicopter parent probably thinks this all makes good sense.

Well, first, the traditional 'helicopter parent' is one who doesn't want their child to fail an exam and who swoops in to save them, so I think you are talking about some other kind of privilege. But secondly, parents have very little say in online tools and platforms even at the secondary school level, never mind tertiary education. From the amount of angst in my parent advocacy groups over the mess that is Google Classroom implementation, most parents just want their kids to learn. A few parents want their kids to 'win' for sure, but I don't think proctoring software is going to help their cause.
posted by warriorqueen at 6:40 AM on January 11, 2021 [4 favorites]


I seem to be missing why Chegg is an issue - my university had past papers available for study anyway. Is that uncommon in the US? Its not as if you can get away with copying and pasting someone else answer.

If you don't know in advance that "Apply Ostrom's institutional analysis to the following fictional example to determine whether this common pool resource management scheme is likely to succeed in the long term" will appear on the exam, then it's a reasonable tool for assessing your ability to use that particular tool. If you know that it's coming and have three examples of answers that received A's, then the only thing it's assessing is your skill at English composition, particularly if coming up with new fictional examples every semester isn't feasible.
posted by GCU Sweet and Full of Grace at 7:51 AM on January 11, 2021 [3 favorites]


>> Also, school boards and administrations often with little connection to classroom teaching are the ones making the decisions to buy and implement these programs.

Absolutely. And what can I tell you about most of the well-meaning but not-too-with-it members of most school boards? And here I thought standardized-testing was a scandal.

As a ex-teacher and a parent, I would *never* expose my kids to this bullshit. We can find other solutions. The first step is not to play.
posted by Twang at 9:43 AM on January 11, 2021 [2 favorites]


This is the future of employment if it isn't curtailed in school.
posted by snuffleupagus at 10:37 AM on January 11, 2021 [8 favorites]


I teach pre-service elementary teachers in college. Although I’ve had a few instances of stupid plagiarism in papers, testing is different.

That is, when I give them a “learning quiz” about things they don’t know yet, that I haven’t taught them, and TELL THEM TO CHEAT (it’s a way of introducing new material that makes it stick better), they won’t. They’re so used to the idea of solitary text taking they cannot bring themselves to ask a classmate for the answer.

It’s unnerving.

I’m not saying they wouldn’t cheat on some other test. It’s just that they won’t cheat if the teacher tells them to.

When I used to teach middle school English I spent the week before exams having my students put everything that they thought was going to be on the exam on a 4x6 index card (both sides) that they could take with them into the exam. They invariably realized when they started taking the exam that they didn’t need the card. The process of preparing it had been an effective study method.

I think exams are bogus. They are not a valid wat of assessing learning in most cases, and what learning they do encourage is often highly temporary.
posted by Peach at 1:52 PM on January 11, 2021 [8 favorites]


I think exams are bogus. They are not a valid wat of assessing learning in most cases, and what learning they do encourage is often highly temporary.

This.
posted by jeremias at 2:51 PM on January 11, 2021 [3 favorites]


If you don't know in advance that "Apply Ostrom's institutional analysis to the following fictional example to determine whether this common pool resource management scheme is likely to succeed in the long term" will appear on the exam, then it's a reasonable tool for assessing your ability to use that particular tool. If you know that it's coming and have three examples of answers that received A's, then the only thing it's assessing is your skill at English composition, particularly if coming up with new fictional examples every semester isn't feasible.

Call me naive, but reading other people's answers for a question as complex as that and formulating an answer in your own words for a similar question sounds an awful lot like studying for a test.
posted by Mr.Encyclopedia at 6:00 PM on January 11, 2021 [5 favorites]


My high school student is doing remote learning but she has to go into the school to take the final math end of course test, COVID or nor COVID. This seems pretty reasonable to me. But the bigger question is whether there are better ways to measure learning than adversarial exams with an arms race of cheaters and teachers. For college specifically, the real issue is that degree X from college Y is a gatekeeper for jobs. Better would be “let’s have a wide ranging unscripted chat about (cell biology) right here in this interview and I don’t give a shit if you went to MIT or learned it on the internet.”
posted by freecellwizard at 3:46 PM on January 12, 2021 [2 favorites]


Call me naive, but reading other people's answers for a question as complex as that and formulating an answer in your own words for a similar question sounds an awful lot like studying for a test.

You're imagining a student sitting down with those and trying to figure out what they think about it themselves; I'm imagining someone sitting down with an A essay and a thesaurus and trying to memorize the "in their own words" result.
posted by GCU Sweet and Full of Grace at 4:20 PM on January 12, 2021


A few thoughts:

Standardised exams have a great benefit, which is that they're a fair and trusted way of assessing all students equally, whoever they are.
For them to be fair and trusted, we have always needed security measures; both to prevent cheating from happening, and to make it visible that cheating is not happening. That has applied to students, and to their teachers, and to the exam-markers, and to exam-setters, etc. We don't have police involvement in UK schools at all, but we do have standardised exams, so maybe the difference between 'cop-shit' and necessary exam security is more obvious from the perspective of this environment.

When I was in school and at university, I didn't enjoy exams much. I got good grades from them, but the process is stressful and unpleasant. More recently in professional life, I've taken remote tests with invigilation software, and the process was stressful and unpleasant, in some similar ways and some dissimilar ways.
Being asked to take photos of my work area, close the room's curtains, and use the laptop camera to show myself were kind of odd, but not alarming. However as an IT professional, the visible flaws in the invigilation software were more disturbing – it wasn't the worst-written software I've ever used, but it was definitely not the best – as was the intrusiveness with which it tried to take over the running of my laptop.

There's probably not a lot of visibility of invigilation software; it's used by one person at a time, in a room by themselves, where nobody else can see it. By comparison, the security in a school exam hall is much more visible; all the pupils and all the invigilators can see what's going on, as can anybody passing by who glances through the window. That visibility would quickly expose any unfairness or ineffectiveness in the exam security, so the exam security is generally trusted. With invigilation software there isn't that visibility, so it doesn't surprise me that it didn't seem all that good when I had to use it, nor that it might be widely mistrusted.

Asking all pupils to sit an exam on paper, in a school hall, is a way of assessing them which doesn't make excessive demands on poorer pupils; the financial cost is low, and the time can be spared by all but the most deprived. By contrast, asking a pupil to sit an exam at home on their own computer using 3rd-party invigilation software is a much bigger ask; poorer pupils' families may well not have the money spare to buy a computer with the required specs, nor the space and time and quiet to let their child take hours of exams without interruption. That should certainly be a concern to those in charge of the education system.

Overall, I'd say that there's a clear benefit to invigilation software; but the downsides are also clear, and it's also clear that it'd be difficult and expensive to deal with those downsides. I'm glad that people like Audrey Watters are raising the problems with this kind of software, and to be honest I hope that educators find a way to avoid it if possible - until it can be made better, more accessible, and more trustworthy.
posted by vincebowdren at 9:41 AM on January 13, 2021 [1 favorite]


It's not just proctoring though.

Eg, a friend of mine is a lecturer at a local (New Zealand) university. Their lectures are recorded, including student questions, to assist students. And, an automatic transcript is generated Well, now they have realised that all student chatter is recorded... including private conversations between students before they get there. So now my friend has to warn the students that anything they say will be recorded, transcribed, and made available to her, other students, and maybe anyone else with access to the system. Which, I should point out, the university administration didn't suggest let alone mandate. It's up to my friend to do it. I find this appalling: it should be obvious that there are big privacy issues in this system but somehow it's an "enterprise" commercial product now.
posted by i_am_joe's_spleen at 8:54 PM on January 14, 2021 [5 favorites]


> I would absolutely have hated software that spied on me, and am happy I went to college at a place where, for the most part, the honor code worked, and I regularly had tests that took a couple of hours to do but had 24-48 hour time limits.

I have both college and grad degrees but haven't had to take an academic test since I was 17. I wonder how many classes really need testing in the first place.
posted by The corpse in the library at 5:35 PM on January 18, 2021 [1 favorite]


« Older Do you think the ’90s were five years ago?   |   F I S S I O N M A I L E D Newer »


This thread has been archived and is closed to new comments