The training loop: guess, check, nudge, repeat

You now have every piece except the one that does the work. A model is a shape with knobs. Error is a number that says how wrong the current knobs are. Good knobs are the ones that make that number small. The only thing missing is how the machine finds them on its own — with no eyes, no understanding, and sometimes a billion knobs to set. The answer is a loop so simple it feels like it can’t possibly be enough. It is enough. It is the whole engine.

Blindfolded on a hill

Imagine you’re standing somewhere on a hillside, blindfolded, and you want to reach the lowest point. You can’t see the valley. But you can feel the ground under your feet — you can tell which way slopes down. So you do the obvious thing: feel for the downhill direction, take a small step that way, and stop. Now you’re a little lower. Feel again, step again. Keep going, and step by step you walk your way down to the bottom, having never once seen where you were headed.

That is exactly what a machine does to train a model. The “hill” is the error. Standing at the current knob settings, the machine works out which way to nudge each knob to make the error a little smaller — the downhill direction. It takes a small step that way. Then it re-checks from the new spot and steps again. Feel the slope, step downhill, repeat. Do it enough times and the knobs arrive at settings with very low error. The machine has learned, and at no point did it understand a thing. It just kept walking downhill.

The loop, in four words

Here is the entire training loop:

1. Guess — run the model on the examples with the current knobs.
2. Check — measure the error: how wrong were the guesses?
3. Nudge — move each knob a small step in the direction that lowers the error.
4. Repeat — go back to step one, thousands or millions of times.

That’s it. That is what “training a model” means, top to bottom. When you hear that a model trained for three weeks on a warehouse of computers, this is what those computers were doing the whole time: guess, check, nudge, repeat — an unfathomable number of tiny downhill steps, across a landscape with billions of dimensions instead of a hillside’s two.

The size of the step matters more than you’d think

There’s one dial on this loop that decides whether it works at all: how big a step to take each time. It’s called the learning rate, and it’s the difference between reaching the valley and never getting there.

Too small a step, and you crawl. You’re heading the right way, but you inch downhill so slowly it could take forever to arrive. Wasted time, wasted computing.

Too big a step, and you overshoot. You feel downhill, then leap so far that you sail clean over the valley floor and land partway up the opposite slope. Next step you leap back over again. You bounce from wall to wall, and if the steps are wild enough you don’t just fail to settle — you fly higher each time, the error exploding instead of shrinking. The training falls apart.

A step that’s about right, and you descend smoothly to the bottom in a sensible number of moves. Finding that middle setting is one of the quiet arts of making AI actually work.

A worked example: watching it walk

Keep the house model, and start it badly on purpose: a flat line that guesses the same price for every flat, with a big error. The machine measures the slope of the error and finds it should make the line steeper and lift it a bit. So it nudges: the line tilts up slightly, the error drops. It checks again from there — still too shallow — and nudges again. Tilt, drop. Tilt, drop. After a couple of dozen little steps the line has walked itself right into the middle of the five flats, the error has fallen to about the lowest a straight line can reach, and the steps get tiny because there’s almost nowhere lower to go. Nobody adjusted a knob by hand. The loop did it.

Now crank the step size up too high and run it again. The line lurches past the good fit, overshoots to something far too steep, then lurches back too shallow, wobbling around the answer it can’t settle into — or, if you push the step higher still, flinging itself further off every move until the error runs away entirely. Same loop, one bad dial, and learning turns into thrashing.

What just happened, and what didn’t

Step back and notice the trade. We got a machine to learn — genuinely, on its own, no rules written for the task — out of nothing grander than rolling numbers downhill to shrink a score. That is the real, and slightly deflating, secret of the whole field. There is no spark of understanding in there. There’s a slope and a step, repeated past the point of human patience.

And there’s a shadow already forming. The loop drives the error on the examples down toward zero. But fitting the examples you were shown, and actually understanding the thing behind them, are not the same — and a machine that only knows how to make one number small has no idea which of the two it’s doing. Hold that thought. It’s the crack that the next several items pry open, and it’s where a lot of AI’s real trouble lives. For now, you’ve seen the engine turn. Everything else is this loop, scaled up until it looks like magic.