How chat models work
Tokens, next-word prediction, training and the context window — what happens inside an LLM, without formulas.
When you talk to ChatGPT or Claude you are using a large language model (LLM). Knowing how it works makes you much better at using it.
1. Text is split into tokens
A model reads text not by letters or words but by tokens — words or pieces of words. “Learning” might become “learn” + “ing”.
Uzbek and Russian usually split into more tokens than English, so the same text costs a little more through an API and fills more of the context.
2. The model predicts the next token
An LLM’s core job is simple: “Which token most likely comes after this text?” It picks one, appends it and repeats. A long answer is thousands of those steps.
3. Where the ability comes from
Pre-training: the model practises predicting the next token across trillions of tokens of books, articles and code. Along the way it absorbs grammar, facts and reasoning patterns.
Tuning: people then use example conversations and ratings to turn it into a helpful, polite and safe assistant.
4. The context window
The model only “sees” what’s in its context window: the system instruction, the chat history, uploaded documents and your question. In a very long chat the beginning can be “forgotten”. Restate what matters.
5. Why it makes mistakes
The model produces likely text, not truth. If the training data had no clear answer, it may invent a convincing but wrong one. So:
- check important facts;
- give it a document and say “answer only from this document” (the idea behind RAG);
- add “if you don’t know, say so”.
Go deeper
If you code, build a small GPT from scratch with Andrej Karpathy’s Neural Networks: Zero to Hero. 3Blue1Brown’s videos explain the same ideas without formulas. Both are in the catalogue.
CC BY-SA 4.0