Lesson 02 · 4 min · 6 things to do
Randomising is the whole trick
Say what a coin toss does that careful matching cannot.
Doctors give a new drug to patients they judge most likely to benefit. Those patients do better than the ones who did not get it. What is wrong?
- Yes.The judgement that picked the group is exactly the thing that also predicts the outcome. The drug and the selection are tangled together and no amount of analysis afterwards can fully separate them.
- Not quite.It is real, and it cannot distinguish the drug's effect from the doctors' skill at picking.
- Not quite.Size does not help. A bigger version of the same design gives you a more precise estimate of the wrong thing.
Randomising decides who gets what by chance. That is the only method that balances the things nobody thought to measure.
Two ways of splitting 400 patients into two groups. Slide between them.
Doctors chooseMatched carefullyRandomisedGiven the drug22% severe casesNot given it41% severe casesDoctors chooseAge and sex look balanced. The groups also differ in how ill they were — and in things nobody recorded.
Given the drug31% severe casesNot given it33% severe casesMatched carefullyBalanced on everything measured. Still unbalanced on everything that was not.
Given the drug32% severe casesNot given it31% severe casesRandomisedBalanced on the measured things — and, on average, on the unmeasured ones too.
Careful matching balanced everything that was measured. What can randomising do that it cannot?
- Yes.You can only match on what you wrote down. A coin does not need to know what it is balancing — which is why randomisation is the one design feature that handles unknown differences.
- Not quite.It does not. It makes any remaining differences a matter of chance, which is something statistics can account for.
- Not quite.That is what blinding does, and it is the next lesson.
Move the control to see what changes.
Which of these differences between groups can careful matching handle?
Age.
Recorded severity.
How determined a patient is to recover.
An unknown genetic difference.
Whether they smoke, where it was asked.
Yes.Matching works on the columns in your spreadsheet. The right-hand items are the ones that were never in it — and there is no way to know what those are in advance.Why do trials conceal which group the next patient will be assigned to?
- Yes.It is usually not deliberate — a doctor delaying enrolment until the sequence favours their patient means well. Allocation concealment is what keeps the randomisation intact after it has been designed.
- Not quite.That is about the results, not the assignment. This is about who gets into which group.
- Not quite.Patients consent to be randomised. The risk being managed is at the point of assignment.
A trial randomises 800 patients into two equal groups. If a hidden factor affects 20% of patients, roughly how many end up in each group?
patientsYes.160 affected patients, split by chance into two groups of about 80. Nobody knew the factor existed, and it is balanced anyway — that is the entire argument for randomisation.When is a randomised trial not the right tool?
- Yes.Some of the strongest findings in medicine come from observation, built up carefully across many designs. Recognising when randomisation is unavailable is part of reading the evidence, not an excuse to skip it.
- Not quite.'Obviously effective' has been wrong many times, and randomised trials are how that was discovered.
- Not quite.Harder, and it is a problem of numbers rather than of method.
Lesson complete
A coin toss balances the differences nobody thought to measure.
Next: Expectation is itself a treatment →