Picture two hospitals in one town. The big one delivers about 45 babies a day, the small one about 15. About half of all babies are boys. For a year, each hospital writes down every day when more than 60 in 100 of its babies were boys.
10 flips37.7 in 100 tries
100 flips2.8 in 100 tries
1,000 flips0 in 100 tries
Chance of 60% or more heads from a fair coin, computed.
Here is the same question with a coin. Flip it ten times. Six or more heads, which is 60 per cent, happens about 38 times in 100 tries. Flip it a thousand times. Six hundred or more heads almost never happens. A small count swings wide. A big count settles near the truth.
Back to the two hospitals. Which one wrote down more days when over 60 in 100 of its babies were boys?
Pick one. Nothing is scored.
15 a dayswings wide
vsvs
45 a daysettles
The small one.The small one, and by a long way, for the same reason ten flips swing more than a thousand. When this was put to people in a 1972 study, most said about the same. The swing of a small count is not something a mind feels; it has to be counted.
Move the control through the number of flips. The chance of 60 per cent or more heads is computed exactly for a fair coin.
60% or more heads37.7 in 100 tries
Fewer than 60%62.3 in 100 tries
Flips of a fair coin: 10
10 flips. Getting 6 or more heads, which is 60 per cent, happens about 37.7 times in 100 tries.
60% or more heads25.2 in 100 tries
Fewer than 60%74.8 in 100 tries
Flips of a fair coin: 20
20 flips. Getting 12 or more heads, which is 60 per cent, happens about 25.2 times in 100 tries.
60% or more heads10.1 in 100 tries
Fewer than 60%89.9 in 100 tries
Flips of a fair coin: 50
50 flips. Getting 30 or more heads, which is 60 per cent, happens about 10.1 times in 100 tries.
60% or more heads2.8 in 100 tries
Fewer than 60%97.2 in 100 tries
Flips of a fair coin: 100
100 flips. Getting 60 or more heads, which is 60 per cent, happens about 2.8 times in 100 tries.
60% or more heads0 in 100 tries
Fewer than 60%100 in 100 tries
Flips of a fair coin: 1,000
1000 flips. Getting 600 or more heads happens less than once in 100 tries. At this size, a fair coin almost never looks unfair.
A fair coin is flipped 20 times and shows 12 heads. Somebody says the coin is rigged. By the chances, how surprised should they be?
Very. Twelve of twenty is well above half.
Not yet. Above half is what a small count does half the time. The chance of this exact swing or bigger is about 25 in 100, which is far too common to be a sign.
Completely. A fair coin gives ten of twenty.
Not yet. Ten is the average, not the result. A fair coin gives exactly ten of twenty less than one time in five; the rest of the time it lands somewhere else.
Not very. Twelve of twenty happens about one time in four with a fair coin.
Right. A quarter of all twenty-flip runs land at 60 per cent or higher. Twelve heads is a Tuesday for a fair coin, and it says nothing about rigging.
Move the control.
11 of 15
boys in one small-hospital day
Eleven of fifteen: a day that happens often at the small hospital.
The part of the world you counted is called a sample. The town is the whole; the fifteen babies are the sample. The smaller the sample, the wilder it swings, and nothing about the babies, the hospital or the coin changes that. It is in the counting itself.
4 of 4
customers asked, all said yes
A café asks its first four customers if they like the new cup. All four say yes. If really only half of customers like it, the chance all four say yes is 1 in how many?
Each customer is a coin flip. Halve the chance once for each of the four.
Right. One in sixteen. A half, of a half, of a half, of a half. Four yeses in a row is not rare: ask sixteen groups of four and one group gives it even when the cup is only liked by half. The café has learned very little from four people.
Each is a claim with a count behind it. Does it rest on a few, or on many?
One card at a time. Tap the pile it belongs to.
Card 1 of 6
My gran smoked every day and lived to 95, so smoking is fine.
A survey of 2,000 adults found a third skip breakfast.
Three of my friends felt ill after the new jab.
The school's 800 pupils averaged two marks higher this year.
A YouTuber tried the diet for a week and lost three kilos.
The bus was late both times I took it this week.
My gran smoked every day and lived to 95, so smoking is fine. → A few
Not yet. A few. One gran is a sample of one, and a sample of one swings as wide as a sample can.
A survey of 2,000 adults found a third skip breakfast. → Many
Not yet. Many. Two thousand is a count big enough to settle; the swing left in it is a point or two.
Three of my friends felt ill after the new jab. → A few
Not yet. A few. Three friends is three flips, and three heads in a row happens one time in eight for no reason at all.
The school's 800 pupils averaged two marks higher this year. → Many
Not yet. Many. Eight hundred pupils is a count that barely swings; two marks on that many is a real move.
A YouTuber tried the diet for a week and lost three kilos. → A few
Not yet. A few. One person for one week is the smallest count there is, and a week of lost water can be most of three kilos.
The bus was late both times I took it this week. → A few
Not yet. A few. Two rides is two flips, and a bus that is late a quarter of the time is late twice in a row one time in sixteen.
4A few
vsvs
2Many
Right. Four of the six rest on a handful, and each of those handfuls could have come out the other way by chance alone. The two that rest on many are the two that could carry a claim.
Of all schools3 in 100
Of the top fifty12 in 100
Of the bottom fifty18 in 100
The share that are among the smallest schools. Pennsylvania, 2007 article.
The top and bottom of almost any list are full of small groups. A 2007 article ranked 1,662 schools in one US state by test scores. The smallest schools, 3 in 100 of all of them, were 12 in 100 of the top fifty and 18 in 100 of the bottom fifty. The same swing put them at both ends.
Few peoplelowest rates
vsvs
Few peoplehighest rates
A map colours US counties by their rate of kidney cancer. The lowest-rate counties are mostly rural, with few people. So are the highest-rate counties. What is going on?
The map is wrong. The same kind of county cannot be at both ends.
Not yet. The map is right. A small sample is at both ends of any list, which is exactly what lesson 5 says a small count does.
Country life protects some people and harms others.
Not yet. That would need two opposite causes living in the same kind of place. The simpler story is that a county with few people swings wide, and the swing has two ends.
Small counts swing both ways. A county of 2,000 people lands at an extreme on a case or two.
Right. One case in a tiny county is a high rate; no case is a rate of zero. Both ends of the map are the same thing: the small-hospital effect, drawn in colour.
Lesson complete
A small count swings wide. The wild results in any list come from the smallest groups, at both ends.