Lesson 05 · 4 min · 6 things to do
Measuring what you can reach
Tell apart a marker that moved and a life that improved.
A trial reports that a drug lowered a blood marker by 30%. Why is that not the same as helping patients?
- Yes.Several drugs have improved their marker beautifully and made patients worse. The marker is chosen because it is quick and cheap to measure — which is a reason about the study, not about the patient.
- Not quite.Size is not the issue. A 90% change in the wrong quantity is still the wrong quantity.
- Not quite.They are often measured very precisely. The question is whether moving them moves anything a patient would notice.
A surrogate is something measurable that stands in for the thing you care about. It is a bet that the two move together, and the bet is sometimes lost.
Which of these are outcomes a patient would feel, and which are stand-ins?
Living longer.
A tumour appearing smaller on a scan.
Being able to walk further without pain.
A blood pressure reading falling.
Fewer hospital admissions.
Yes.The right column is measurable in weeks; the left often takes years. That is the whole reason surrogates are used — and the whole reason they mislead.A drug moves a marker. Slide to see how well the marker tracks what patients actually experience.
TightlyLooselyNot at allImprovement in the marker30%Improvement in the outcome26%TightlyThe marker moves and the outcome follows. Here the stand-in has earned its place.
Improvement in the marker30%Improvement in the outcome4%LooselyA large change in the marker, a small one in the outcome. Reporting the first is technically true.
Improvement in the marker30%Improvement in the outcome−9%Not at allThe marker improves and patients do worse. This has happened, in more than one field.
From the trial's own numbers, how would you tell the third case from the first?
- Yes.A trial that only measured the marker reports an identical headline in all three frames. That is the argument for insisting on outcomes, even when they take years.
- Not quite.Identical in all three frames — that is what makes them indistinguishable.
- Not quite.More patients measure the wrong thing more precisely.
Move the control to see what changes.
Why are surrogate outcomes used at all, if they can mislead?
- Yes.A drug for a fast-killing disease cannot wait for a ten-year survival result. The honest position is that a surrogate buys speed and owes a follow-up, and the follow-up is what often does not arrive.
- Not quite.The trials using surrogates are often enormous efforts. The choice is about time.
- Not quite.Regulators accept them in specific circumstances, usually with a commitment to confirm later.
A drug improves five-year survival from 32% to 36%. Out of 100 patients, how many extra are alive at five years?
patientsYes.Four. Reported as a 12.5% relative improvement it sounds much larger — the same finding, two framings, and one of them is usually the one in the headline.You read that a treatment "halves the risk". What must you find before that means anything?
- Yes.Halving 40% to 20% is transformative; halving 0.02% to 0.01% is not. The relative figure is always the larger-sounding one, and it is what press offices reach for.
- Not quite.Important for whether the finding is real, and no help at all in judging how big it is.
- Not quite.Also important, and again a different question from the size of the effect.