In spring 2020 exams in England were cancelled. Teachers were asked to predict each pupil's grade and rank their class. Then a computer adjusted the grades so that each school's results looked like its results from the three years before. On results day, 13 August, about 39 in 100 A-level grades came out lower than the teacher had said.
Lowered39 in 100 grades
Unchanged59 in 100 grades
Raised2 in 100 grades
A-level grades against the teacher's, England, 13 August 2020.
Lesson 1: a pattern copied from the past. The past here was each school's old results. A pupil at a school that had done badly before was pulled down toward that past, whatever the teacher had said. The pupil had never sat the exam. Their grade now carried their school's history.
A strong pupil
++
A school with poor past results
A strong pupil at a school with poor past results. Which way did the computer push her grade?
Up, because the teacher ranked her first.
Not yet. Being ranked first kept her at the top of her class. It did not stop the whole class being pulled toward the school's history, top included.
Down, toward her school's past.
Right. The computer's job was to make the school's results look like its past. Her teacher's grade was the thing being adjusted; her school's history was the thing it was adjusted to.
It left her alone. The computer only touched weak pupils.
Not yet. The adjustment was to the school's spread of grades, not to any one pupil's ability. Strong pupils at weak schools were exactly who it moved most.
Big class, state schooladjusted down
vsvs
Tiny class, private schoolteacher grade kept
Lesson 5, who got counted. Classes of five or fewer were too small to adjust, so those pupils kept the teacher's grade. Tiny classes are common at private schools, so their pupils were likeliest to keep a generous grade, and their top grades rose more than twice as much as comprehensives'. That fell out of a rule about numbers.
Move the control through the size of a class. The shares moved are made up and round; the cut-offs at 5 and 15 are the real ones.
Grades the computer moved0 in 100
Grades left as the teacher said100 in 100
Pupils in the class: 3
A class of 3. Too small to adjust, so the teacher's grades stand as given.
Grades the computer moved0 in 100
Grades left as the teacher said100 in 100
Pupils in the class: 5
A class of 5. Too small to adjust, so the teacher's grades stand as given.
Grades the computer moved20 in 100
Grades left as the teacher said80 in 100
Pupils in the class: 8
A class of 8. The computer blends the teacher's grades with the school's past, and about 20 in 100 of them move, nearly all downward.
Grades the computer moved40 in 100
Grades left as the teacher said60 in 100
Pupils in the class: 15
A class of 15. The computer blends the teacher's grades with the school's past, and about 40 in 100 of them move, nearly all downward. Past 15, the school's history decides.
Grades the computer moved50 in 100
Grades left as the teacher said50 in 100
Pupils in the class: 30
A class of 30. The computer blends the teacher's grades with the school's past, and about 50 in 100 of them move, nearly all downward. Past 15, the school's history decides.
Which pupils did the computer touch least?
Those in the biggest classes, because big classes are hard to adjust.
Not yet. Big classes are the easiest to adjust: plenty of past results to match. Past 15 the school's history took over almost entirely.
Those in the smallest classes, who were mostly at small or private schools.
Right. Five or fewer, and the computer stepped back. Which pupils sit in classes of five is not random, so a rule about numbers became a rule about schools.
It touched everyone equally.
Not yet. The frames run from nothing moved to half moved. The rule was the same for all; the classes were not.
Move the control.
One rule→Run once→Hundreds of thousands of grades→No reason for any one pupil
Lesson 8, one mistake at scale. One rule, run once, changed hundreds of thousands of grades on one morning. Lesson 9: a pupil who asked why could be told the school's past results and their rank. Not why they had dropped from a B to a D. The reason was in the rule, not in anything they had done.
Put that summer in order.
Tap them in the order they happen, first to last. Each tap gets a number.
Cancelled→Predicted→Adjusted→Published→Dropped
Right. Cancelled, predicted, adjusted, published, dropped. Four days between the fourth and the fifth. The rule took months to build and a long weekend to undo, once hundreds of thousands of families were counting together.
Ministersno grade inflation
↓→
The regulatorwrites the rule
↓→
Teachersrank
↓→
The computeradjusts
Lesson 12, the chain of hands. Ministers asked for grades that matched past years. The regulator built the rule. Schools ranked. The computer ran. When it went wrong, the regulator's chief resigned within two weeks and its chair by December. The ministers stayed. Lesson 14: the teachers were the humans in the loop, and the machine overrode them.
Each is a thing that went wrong that summer. Which lesson does it belong to?
One card at a time. Tap the pile it belongs to.
Card 1 of 6
Grades pulled toward the school's old results.
Tiny classes kept their teacher's grades.
The regulator resigned; the ministers stayed.
A pupil's grade carried her school's history.
Private schools' top grades rose most.
Nobody could say which person chose the rule.
Grades pulled toward the school's old results. → The past copied
Not yet. The past copied. The school's three previous years were the examples, and the rule made this year look like them.
Tiny classes kept their teacher's grades. → Who got counted
Not yet. Who got counted. A rule about small numbers decided which pupils the computer touched, and small numbers sat mostly in one kind of school.
The regulator resigned; the ministers stayed. → Many hands
Not yet. Many hands. The chain ran from ministers to regulator to schools to computer, and one link was held for it.
A pupil's grade carried her school's history. → The past copied
Not yet. The past copied. Her own work was not in the examples. Her school's past was.
Private schools' top grades rose most. → Who got counted
Not yet. Who got counted. The pupils left out of the adjustment were not spread evenly across schools.
Nobody could say which person chose the rule. → Many hands
Not yet. Many hands. Ministers asked, the regulator built, a computer ran. Each did a part.
2The past copied
++
2Who got counted
++
2Many hands
Right. Two from each lesson. One summer, one rule, and three of the course's problems at once, which is how machine decisions usually arrive.
Ten years of admissions
↓→
The model ranks
↓→
A tutor signs
A university plans to rank next year's applicants with a model trained on ten years of its own admissions, each decision signed off by a tutor. From the whole course, what does the plan carry?
All of it: the past copied, a stand-in label, a line, scale, no reason, a chain of hands, and a signature in the loop.
Right. Ten years of admissions are old decisions, lesson 4. Admitted is a stand-in for will do well, lesson 6. One run ranks everyone, lesson 8. Nobody can say why, lesson 9. A tutor signing at speed is lesson 14.
Only the speed. A model is faster than tutors.
Not yet. Speed is lesson 8, one part of it. The other six lessons apply just as much to a slow model.
None of it, because a tutor signs each one.
Not yet. A tutor signing is a human in the loop, and lesson 14 asks what the tutor has time to do. The signature does not remove the past from the model.
Lesson complete
A machine decision is the past copied, a stand-in, a line, scale, a chain of hands and a loop.