Algorithms and exam grades; should you be worried?

After 2020’s grading fiasco, ‘algorithm’ has become a bit of a dirty word in education.

Mention the idea of using an algorithm anywhere in the grading process, and you’ll have panicked parents demanding answers.

But…

Algorithms aren’t the enemy. Here’s why.

An algorithm is a set of explicit instructions for completing a task.

That’s it.

If you taught your child to tie their shoelaces using a set of instructions, they would be using an algorithm.

Following a recipe to bake a cake? Another algorithm.

An algorithm is a means to an end – the problems arise when the person designing the algorithm doesn’t do a great job.

That’s what happened in 2020 with the exam grades.

The ‘rogue’ algorithm we heard so much about in the media was just a badly designed one.

It assumed too much about the link between a school’s past performance and its current performance, and it failed to allow for external factors like the impact of illness, disruption due to Covid, and the fact that every year group is different.

The main problem was that the government chose to ignore the help offered by statisticians, teachers and other experts, and pushed ahead with a faulty set of instructions for checking the fairness of results.

They’d been warned it was likely to cause issues, but they assumed that teachers would overestimate everyone’s grades and they would need to be adjusted.

(It’s worth mentioning that teachers are asked to predict student grades every year and most of us are pretty good at it. While we usually want to err on the side of optimism, we also get judged on the accuracy of our predictions compared to actual exam results, so it tends to balance out!)

The algorithm tried to compare the submitted Centre Assessed Grades for each school with what that school got the year before, and then adjusted if it wasn’t similar enough. That caused massive problems for schools that had made big improvements over the last year, or had a particularly strong cohort in 2020 (or a weak one in 2019).

Basically, it assumed too much, and in the end, the algorithm downgraded 39% of the A-level grades predicted by teachers in England.

It’s understandable that even the slightest hint of an algorithm in the process now makes parents and students nervous, but the algorithm isn’t the enemy. Nor are schools.

Results in 2021 aren’t going to be perfect. There have been too many unknowns, too many disruptions for that. Schools have done their absolute best, in the face of incredible pressure, to assign grades fairly, without any clear guidance. (Any problems will be due to the non-committal, often late, let’s-make-it-school’s-problem decisions taken by Gavin Williamson this year.)