Lesson 06 · 4 min · 6 things to do
What bigger buys
Say what scale improves and what it leaves untouched.
A model is made ten times bigger and trained on ten times more text. What reliably improves?
- Yes.The improvement is broad and gradual rather than a new capability switching on. What was occasionally wrong becomes rarely wrong, across nearly everything at once.
- Not quite.It makes things up less often. The mechanism that produces invention is the same one that produces correct answers, so it cannot be removed by scale.
- Not quite.Nothing about size changes when the training stopped.
Slide the size and watch what you get, and what you pay.
SmallMediumLargeAnswer quality (rough index)45Cost to run one answer1SmallFast and cheap. Fine for tidying text; loses the thread on anything long.
Answer quality (rough index)75Cost to run one answer10MediumMost of the useful behaviour, at a fraction of the top-end cost.
Answer quality (rough index)88Cost to run one answer100LargeBetter again on hard reasoning. Each step of improvement costs far more than the last.
The quality bars are climbing and the cost bars are exploding. What shape is this?
- Yes.This is why the industry runs several sizes rather than one. Most tasks sit where the medium model already wins, and the top model earns its cost on a narrow band of hard work.
- Not quite.If it were straight there would be no reason to run anything but the largest model.
- Not quite.It is still improving. It is the price of each improvement that is running away.
Move the control to see what changes.
Scale is a dial on quality, and a steep one to turn. It does not change what kind of thing the system is.
Which of these does making the model bigger help with, and which does it leave untouched?
Following a long, multi-part instruction.
Knowing what happened after training ended.
Writing working code for a common task.
Knowing a fact nobody ever wrote down.
Holding a harder argument together.
Yes.Scale improves how well the pattern is learned. It cannot supply material that was never there — and the two failures look identical from outside.A team needs to classify a million short messages. Which model should they reach for first?
- Yes.Classification is a narrow, repetitive job — exactly where small models are strong. Measuring on a sample turns the choice into a number rather than a habit.
- Not quite.A hundredfold cost for a few points on an easy task, repeated a million times.
- Not quite.Newest often means best-per-pound rather than biggest — worth checking, and still no substitute for measuring on your own sample.
A large model costs 100 units per answer and a medium one 10. You need 200,000 answers a month. How many units a month do you save by using the medium model?
unitsYes.200,000 × 90. At volume, the choice of model is the whole budget — which is why serious systems route easy work to small models and keep the large one for what needs it.What is the honest summary of what scale has bought so far?
- Yes.Both halves are needed. Dismissing the capability is as wrong as expecting the growth to turn it into something with a different relationship to truth.
- Not quite.Nothing in the training loop reaches for understanding, and the failures are exactly the ones a very good predictor would have.
- Not quite.The difference between the small and large frames above is enormous, and it shows up in work people do every day.
Lesson complete
Scale makes it a better predictor. It does not make it a different kind of thing.
Next: Knowing what it cannot know →