State what an approval does and does not establish.
A regulator approves a drug. What has it decided?
Yes.It is a judgement about a group, made from group averages. It says nothing about what will happen to any particular person taking it.
Not quite.No drug is safe in the sense that word usually carries. Approval says the harms found were acceptable against the benefit found.
Not quite.Many approvals are against placebo rather than against the best existing option. Being better than nothing is a lower bar than being better than what people already take.
An approval is a decision about a population, made from averages. Everything about how it lands on one person is outside what the trial measured.
A trial reports an average benefit. Slide to see how that average can be made.
EvenConcentratedSplit
Small benefit85% of patients
No change12% of patients
Worse3% of patients
EvenNearly everyone gets a small benefit. The average describes almost every patient in it.
Large benefit20% of patients
No change72% of patients
Worse8% of patients
ConcentratedSame average. A fifth are helped a great deal and most feel nothing.
Large benefit34% of patients
No change33% of patients
Worse33% of patients
SplitSame average again — a third much better, a third much worse, and the two cancel out.
Three completely different situations report the same average. What does that tell you about an average benefit?
Yes.It is why research into who responds — and why — matters as much as whether the average moved. The average is where the evidence stops, not where the question does.
Not quite.The average is the honest summary of what was measured across the whole group, and it is the right basis for a population decision.
Not quite.All three are correctly run. The spread is a fact about the drug and the patients, not an error.
Move the control to see what changes.
Which of these does a positive trial establish?
The average effect in people like those studied.
That harms common enough to appear were recorded.
What will happen to one particular patient.
That it works in people who were excluded from the trial.
That it beat whatever the comparison group received.
Yes.Trials exclude a great deal — the very old, the pregnant, people with several conditions at once. The evidence applies to the people in it, and extending it beyond them is a judgement rather than a finding.
Two drugs are approved for the same condition. Which is better?
Yes.Both may have beaten a placebo, in different trials, in different populations, with different outcomes measured. Head-to-head trials are the only thing that answers this, and they are expensive and rarer than you would expect.
Not quite.Newness carries less evidence about rare harms, not more evidence about benefit.
Not quite.Effects measured against different comparison groups, in different populations, are not comparable numbers.
Put in order what a piece of medical news should be checked against.
Tap them in order — first to last.
Compared with what?→Randomised?→A real outcome?→How big, absolutely?
Four questions, in the only order that works.Yes.Each question is only worth asking if the one before it was answered. Together they take about a minute, and they dispose of most of what is written about health.
What is the honest summary of what all this machinery achieves?
Yes.It is not a truth machine and it is a very good filter. Every part of it — the control group, the coin toss, the blinding, the registry — exists to close one route by which nothing can look like something.
Not quite.It establishes average effects in studied populations, with known uncertainty. Proof is a stronger word than the method supports.
Not quite.It is slow, and the things it catches are the reason. Most candidates that fail were drugs that did not work.
Lesson complete
An approval is a judgement about a population, made from averages.