Lesson 07 · 4 min · 6 things to do
A number loses its error bars
A study reports somewhere between 8,000 and 34,000. A week later everybody is saying 30,000. Nobody lied at any point in between.
A study reports "between 8,000 and 34,000". Which number ends up in the headline?
- Yes.A headline has no room for a range, and the bigger end is the more striking claim. The range survives inside the article, for the readers who reach paragraph nine.
- Not quite.Ranges do appear in careful writing. They do not survive a headline, a bulletin summary, or one person repeating the story to another.
- Not quite.Averaging the two ends is a calculation the reporter has no basis for, and it produces a weaker number than either end. It almost never happens.
Britain was said for years to have 4.2 million CCTV cameras. How much was counted to produce that number?
Commit to one. Nothing is scored — this is here so you have a number in your head before you see the real one.
Two streetsAll of LondonThe whole country"One camera per 14 people"Every arrow multiplies the guess. None of them adds a count. Two streets.Both of them in south London, counted for a working paper in 2002. The count was scaled up to all of London, then to the whole country, and quoted as fact for a decade. Later estimates put the real figure nearer one to two million.An estimate is a number with a fence around it. The fence is the first thing to fall off when the number gets repeated.
One estimate, retold. The counts here are invented; what happens to the fence around them is not.
Lowest figure still being quoted8,000Highest figure being quoted34,000As publishedBetween 8,000 and 34,000. Both ends are real, and the study cannot say where inside them the answer sits.
Lowest figure still being quoted20,000Highest figure being quoted34,000Retold twiceThe low end has crept up. Each retelling reaches for the more striking half.
Lowest figure still being quoted30,000Highest figure being quoted34,000Retold five timesThere is barely a range left. The number is now simply "about 30,000".
Nobody lied at any point in that chain. So what was lost?
- Yes.The gap between the ends is information — it is the researchers telling you how much they do not know. Dropping it makes the claim sound better established than it ever was.
- Not quite.30,000 may well be inside the range and perfectly defensible. What went missing is not the number but the honesty about how uncertain it is.
- Not quite.It is easier to remember, and it is also a different claim. "Somewhere between 8,000 and 34,000" and "about 30,000" are not the same statement about the world.
Move the control to see what changes.
A study reports somewhere between 8,000 and 34,000. How many times bigger is the high end than the low end?
timesYes.More than four times. A range that wide is the study saying plainly that it does not know the answer within a factor of four — which is real information, and it is exactly what a single headline figure deletes.Which of these numbers could you trace back to somebody's count, and which could you not?
"Official figures published on Tuesday show 4,312 cases."
"Experts estimate up to a million."
"A survey of 1,004 adults found 38 per cent."
"It is thought to be the worst in a generation."
"Deaths rose from 210 to 244 last year, the department said."
Yes.The traceable ones name who counted, when, and how many. The rest are numbers with the fence taken off — "up to" and "it is thought" are the places a range used to be.A story says "up to 40 per cent of cases". What do you now know, carefully stated?
- Yes."Up to" is the top of a range, quoted alone. It tells you where the ceiling was and nothing whatever about where the floor was.
- Not quite.That would be "about 40 per cent". "Up to" is equally true if the real figure is 2 per cent, and it gets written that way because the bigger number is the more striking one.
- Not quite.It may be a perfectly good estimate. What has been removed is the other end of it, and that removal happened after the researchers had finished.
Lesson complete
Numbers harden as they travel, because every retelling drops a qualifier.
Next: The headline is the product →