Picture the people running a video app in 2012. They wanted viewers to be glad they came. You cannot count glad. So they told their machine to raise a number they could count: minutes watched. That is what YouTube did in March 2012, when it told the system that suggests videos to rank mainly by time watched, not by clicks.
What people want→A number that stands for it→The machine raises the number
Every model is given a target: one number to make bigger or smaller. People call it the objective. The word just means the score the machine is trying to raise. The machine does not know what the people wanted. It knows the score, and it finds whatever raises it.
The peoplewant glad viewers
↓→
The machinetold to raise…
The app's people wanted viewers to be glad they came. What was the machine actually told to raise?
Gladness. That is what the people wanted.
Not yet. That is what the people wanted. It is not what the machine was told, because nobody could put a number on it. The machine only ever gets a number.
Minutes watched. That is the number it could count.
Right. Glad cannot be counted, so a number that could was chosen to stand in for it. The machine was handed the stand-in, not the wish.
Both at once.
Not yet. A machine gets one score to raise. Glad was the hope behind the score, and the score was minutes. Only one of those was given to the machine.
Calm video: minutes8
Calm video: glad8
Upsetting video: minutes20
Upsetting video: glad3
Made up and round. Minutes watched, and glad out of 10.
A stand-in number is the thing you can count, picked because it usually moves with the thing you care about. Usually. A video that upsets you can hold you for twenty minutes. The counter goes up. You are not glad. The machine sees the counter.
Two videos, made up. Tap the one the machine pushes to the top when it is told to raise minutes watched.
Tap the part of the picture that answers it.
Calm video8 minutes watched
Upsetting video20 minutes watched
The upsetting video
Right. It holds people for twenty minutes against eight. The machine was told to raise minutes, and this is the video that raises them. Glad was never in the score.
The calm video
Not yet. People were gladder about this one, but the machine was not told about glad. It was told minutes, and this video has fewer of them.
Move the control through how hard the machine pushes the minutes. The figures are made up and round.
Minutes watched a day40
Glad they came, out of 107
How hard it pushes minutes: Gently
Pushed gently, viewers watch 40 minutes a day and rate the app 7 out of 10 for being glad they came.
Minutes watched a day60
Glad they came, out of 108
How hard it pushes minutes: Firmly
Pushed firmly, 60 minutes and a 8. So far the number and the thing move together.
Minutes watched a day90
Glad they came, out of 106
How hard it pushes minutes: Hard
Pushed hard, 90 minutes. Glad has slipped to 6. The videos that hold people longest are not the ones they are glad about.
Minutes watched a day120
Glad they came, out of 104
How hard it pushes minutes: As hard as it can
As hard as it can: 120 minutes a day, and glad is down to 4. The number kept rising after the thing it stood for turned round.
Pushed as hard as it can, minutes are at their highest. What happened to glad?
It rose with the minutes, as it always does.
Not yet. It rose for the first two frames and then fell for the last two. A stand-in follows the real thing only up to a point, and the machine keeps pushing past that point.
It fell. The number kept rising after the thing it stood for turned round.
Right. Up to firmly, minutes and glad rose together, which is why minutes was a fair stand-in. Pushed harder, the videos that hold people longest stopped being ones they were glad about.
It stayed the same. Minutes and glad have nothing to do with each other.
Not yet. They moved together at first, from 7 to 8. If they had nothing to do with each other, minutes would never have been chosen as a stand-in.
Move the control.
Test scoresup
++
Waiting timesdown
++
The thing they stood for?
This happens wherever a stand-in is pushed hard. A school told to raise test scores can teach the test and nothing else. A hospital told to cut waiting times can leave patients waiting in ambulances outside the door. The number is reached. The thing it stood for is not.
Each is a number somebody is told to raise. Is it the thing people wanted, or a stand-in for it?
One card at a time. Tap the pile it belongs to.
Card 1 of 6
A rescue team: people pulled out alive.
A news site: seconds spent on each page.
A school: pupils who can read by the end of the year.
A call centre: calls finished per hour.
A delivery firm: parcels marked delivered.
A hospital: patients who go home well.
A rescue team: people pulled out alive. → The thing itself
Not yet. The thing. Pulling people out alive is the whole point of a rescue team, not a sign of it.
A news site: seconds spent on each page. → A stand-in
Not yet. A stand-in. The hope is that people are informed. Seconds on a page also go up when a page is confusing or upsetting.
A school: pupils who can read by the end of the year. → The thing itself
Not yet. The thing. Reading is what the year was for; there is no gap between the count and the wish.
A call centre: calls finished per hour. → A stand-in
Not yet. A stand-in. The wish is problems solved. Calls finished also goes up when callers are cut off quickly.
A delivery firm: parcels marked delivered. → A stand-in
Not yet. A stand-in. The wish is parcels in people's hands. Marked delivered is a tap on a screen, and a tap can happen on a doorstep in the rain.
A hospital: patients who go home well. → The thing itself
Not yet. The thing. Going home well is what the patient came for, so raising it raises the point itself.
3The thing itself
vsvs
3A stand-in
Right. Three counts are the thing people wanted. Three are a nearby number that usually moves with it. A machine pushes the nearby number past where it still moves with the thing.
The number it raised→The thing they hoped for→The gap between them
So when a machine has decided something, two things are worth knowing: the number it was told to raise, and what the people hoped that number stood for. The gap between those two is where most of the trouble in this course lives.
The firmmarked delivered: up 30
↓→
Parcels in hands?
A delivery firm tells its machine to raise parcels marked delivered on time. After a month the number is up 30 in 100. Which is the machine most likely to have found?
Faster vans. It must have made the fleet quicker.
Not yet. A machine that ranks and schedules cannot change an engine. It can only change what drivers are pushed to do, and tapping sooner is the easiest thing to push.
Ways to get parcels marked delivered sooner: drivers tapping early, parcels left on doorsteps. The number moves without the parcel reaching a hand.
Right. Marked delivered is a tap, and the machine was told to raise taps. Early taps and doorstep drops raise it. A parcel in a hand was never in the score.
Nothing. The number went up, so the service is 30 in 100 better.
Not yet. The number is a stand-in for the service. It went up. Whether the service did is a separate question, and the whole lesson is that the two come apart when the number is pushed.
Lesson complete
A machine is told to raise one number. It raises that number, not the thing the people wanted.