Predict a model's blind spots from what it was trained on.
A model gives noticeably better answers about London than about a city of the same size in Central Asia. What is the most likely reason?
Yes.Written material is not distributed like people are. Whole regions and languages are thin on the internet, and thin in the training as a result.
Not quite.Products are aimed, and the underlying gap is about the text itself. It shows up even when you ask in the local language.
Not quite.Importance and documentation are different things. Plenty of consequential places are barely written about in the languages that dominate the training.
The training text is the teacher. What is abundant in it becomes fluent; what is thin becomes vague; what is absent becomes invention.
Slide across topics and watch how much material the model had to learn from.
Web programmingEuropean historyLocal council rulesThis month's events
48 of 50 · how well covered in training text
Web programmingEnormous, current, and full of worked examples with the answers attached.
38 of 50 · how well covered in training text
European historyDeep, well edited, and heavily weighted to a few countries' accounts of it.
8 of 50 · how well covered in training text
Local council rulesScattered, often in PDFs, rarely in the training and frequently out of date.
0 of 50 · how well covered in training text
This month's eventsAbsent. Training ended before they happened.
The model answers with the same confident tone in all four cases. Why is that dangerous?
Yes.You cannot hear the difference between the first frame and the third. The one signal a person would rely on is exactly the one that is not connected to anything.
Not quite.Systems are trained toward hedging on thin ground, and it is unreliable — because the model has no separate record of how much it saw.
Not quite.Careful readers are fooled by this just as often. Nothing in the text distinguishes the cases.
Move the control to see what changes.
For each question, would you expect the model's answer to be well grounded or thin?
"Explain how a for-loop works."
"What are the bin collection days for my street?"
"Summarise the plot of a famous novel."
"What did this small company announce last week?"
"What is the difference between a virus and a bacterium?"
Yes.The test is not how hard the question sounds. It is how much has been written about it, and how long ago.
The training text contains a stereotype repeated across millions of documents. What happens to it?
Yes.The loop has no opinion about the world; it only makes the guess closer to what was written. Later training can push against it, and the pattern underneath does not disappear.
Not quite.Filtering removes some categories of material. It cannot remove an association carried by ordinary sentences across the whole corpus.
Not quite.It averages toward whatever is more common, which is the problem rather than the fix.
Put in order how a gap in the training becomes a wrong answer you believe.
Tap them in order — first to last.
Thin material→Must still answer→Plausible-sounding text→Same confident voice
The gap never announces itself. That is the whole failure.Yes.There is no step at which the system could stop. Producing nothing is not one of the options the loop leaves it.
You want to know how well grounded an answer is. What actually helps?
Yes.It converts an untestable claim into a testable one. On thin ground the citation is often the first thing to break — which is itself the signal you wanted.
Not quite.The stated confidence is generated the same way as everything else. It is a plausible-sounding number, not a measurement.
Not quite.You will usually get the same shape of answer, because the same patterns are driving it. Agreement with itself is not evidence.
Lesson complete
Whatever was thin in the training text is thin in the model, in the same confident voice.