State what a language model computes when you ask it something.
You type "The capital of France is" into a language model. What is it computing?
Yes.It scores the whole vocabulary and picks from the top. "Paris" wins because that pattern is overwhelming in the text it learned from — not because it looked anything up.
Not quite.There is no database inside it. The weights are millions of adjusted numbers; you cannot open one and find a row for France.
Not quite.Some products do search, as an added tool. The model underneath is doing something else, and it is that something else this course is about.
One operation, repeated: given the text so far, what comes next? Everything a language model appears to do is built out of that.
Put in order what happens when a model answers a question.
Tap them in order — first to last.
Text in→Through the layers→Score every next word→Pick one, repeat
The same four steps, once per word.Yes.The loop is the answer. A hundred-word reply is that cycle run a hundred times, each time with one more word of context than the last.
The model has scored the next word. Slide the setting that decides how adventurously it picks.
Always the topMildAdventurous
"Paris"99% chance
"a"1% chance
"located"0% chance
Always the topPredictable, repetitive, and it gets stuck in loops on long answers.
"Paris"92% chance
"a"5% chance
"located"3% chance
MildUsually the top word, occasionally the second. Reads like a person writing carefully.
"Paris"60% chance
"a"22% chance
"located"18% chance
AdventurousUnlikely words get a real chance. More surprising, and more often wrong.
Why does the same question sometimes get different answers?
Yes.That randomness is deliberate — it is what stops long text collapsing into repetition. It also means an answer being repeatable is not evidence that it is right.
Not quite.The weights do not change while you talk to it. What varies is the draw, and what you put in the context.
Not quite.With no tool attached there are no sources to search. The variation happens inside the pick itself.
Move the control to see what changes.
The model answers a maths question correctly. What does that prove about how it got there?
Yes.Both routes produce the same text. This is why models can ace a famous problem and fail a slightly reworded one: the second is genuinely a different task.
Not quite.Sometimes it can, in a limited way. A single right answer cannot tell you which of the two things happened.
Not quite.Also possible, and also not shown by one correct answer. The honest position is that you cannot tell from the output alone.
Which of these tasks fit "predict the next word" well, and which fight it?
Rewriting a paragraph in a plainer style.
Counting the letters in a long word.
Drafting an email in a familiar format.
Telling you whether a specific person is still alive today.
Summarising a document you supplied.
Yes.Prediction is strong where the answer is a shape it has seen many times, and weak where it needs a fact about right now or an operation on symbols it only sees in chunks.
A model says "I think" or "I remember". What is happening?
Yes.The training text is full of humans saying "I think". Predicting that phrasing is not a claim about an inner life — and it is very hard to read it any other way.
Not quite.There is no state to describe between messages. The only thing carried forward is the text in front of it.
Not quite.Some of it is shaped by later training. Most of it is simply what the pattern of human writing predicts.
Lesson complete
A language model computes one thing: how likely each next word is.