How large language models work
An LLM writes one small piece of text at a time by predicting what comes next. Learn how that prediction works, how models are trained in stages, and why this explains both their surprising skill and their strange mistakes.
When you type a question into ChatGPT, Gemini or Claude, it feels like talking to someone who looks up the answer. That is not what happens. A large language model (LLM) does one thing, very well: given some text, it predicts what text is likely to come next. Then it adds that piece and predicts again, and again, until the answer is finished.
Your phone keyboard does a tiny version of this when it suggests the next word. An LLM is the same idea scaled up enormously: trained on a large share of the public internet, books and code, with billions of parameters, and then taught to follow instructions.
Step 1: predicting the next token
Models don't read words exactly. They read tokens: short chunks of text, often a whole common word or a piece of a longer one. (The next lesson, *Tokens, context windows and why chats forget*, looks at tokens closely.)
For each step, the model looks at all the text so far and gives every possible next token a probability. Here is an illustration of what that might look like. The numbers are made up to show the idea; real models choose among tens of thousands of tokens.
Illustration: possible next tokens after "The best time to drink filter coffee is in the"
| Possible next token | Illustrative probability |
|---|---|
| morning | 0.71 |
| evening | 0.09 |
| afternoon | 0.08 |
| monsoon | 0.02 |
| (thousands of others) | the rest |
The app then samples one token: usually a likely one, sometimes a less likely one. A setting called temperature controls this. Low temperature means it almost always picks the top choice (more predictable). Higher temperature spreads the choices out (more varied, sometimes more creative, sometimes more wrong).
This is why pressing "regenerate" gives you a different answer to the same question. It is also why an LLM can write a fluent paragraph on almost anything: it has seen patterns of how such paragraphs go.
Step 2: how the model is trained
The usual training stages
- 1Pre-training. The model reads a huge amount of text and practises one task billions of times: hide the next token, guess it, compare with the real one, adjust the parameters slightly. Nobody labels anything; the text itself is the answer key. This is where it picks up grammar, facts, coding patterns and styles of writing. It is extremely expensive and done only by large labs.
- 2Instruction tuning (fine-tuning). A pre-trained model just continues text. Ask it a question and it might write five more questions. So it is trained further on many examples of instructions paired with good answers, written or checked by people. Now it behaves like an assistant.
- 3Learning from feedback. People (and sometimes other models) compare pairs of answers and say which is better: more helpful, more honest, safer. The model is trained to prefer the better kind. You may see this called RLHF (reinforcement learning from human feedback).
- 4Release and use (inference). Training stops and the parameters are frozen. When you chat, the model is only predicting, not learning. A newer version needs another round of training.
Knowledge cutoff
Because pre-training uses text collected up to a certain date, a model knows nothing after that date unless the app gives it fresh information, for example by searching the web. Ask a model about this week's news without search turned on and it may guess, or describe an older situation as if it were current.
Step 3: what is inside (the short version)
Almost all modern LLMs use a neural network design called the transformer, introduced by Google researchers in a 2017 paper titled "Attention Is All You Need". The key idea is attention: when predicting the next token, the model can weigh every earlier token by how relevant it is.
In "Priya ordered a cappuccino, but she said it was too cold", attention is what lets the model link "she" to Priya and "it" to the cappuccino. Stack many layers of attention and calculation, train on enough text, and the network learns surprisingly rich patterns: translation, summarising, step-by-step reasoning, writing code.
What this explains
Behaviours you will notice, and why
| What you see | Why it happens |
|---|---|
| Fluent, confident tone even when wrong | It produces likely-sounding text; confidence of style is not a check on truth |
| Different answer each time | Sampling picks among likely tokens, so outputs vary |
| Good at common topics, weak on rare or local ones | It saw far more text about popular topics than about your college's syllabus |
| Doesn't know recent events | Training data stops at a cutoff date; only search or pasted text adds newer facts |
| Struggles to count letters in a word | It sees tokens, not individual letters |
| Better answers when you give context | Your text changes which continuations are likely |
Modern chat apps also wrap the model with extra tools: web search, a code runner, file reading, memory of your preferences. When an answer cites today's cricket score, a search tool fetched it and the model wrote around it. The model alone could not know.
Two myths to drop
"It looks things up in a database of facts." No. Unless a search or document tool is used, everything comes from patterns stored in its parameters. That is why it can invent a book title that sounds real.
"It is just autocomplete, so it can't be useful." Also wrong. Predicting the next token well, at this scale, requires learning a lot about language, logic and the world. The right view sits in between: very capable, not reliable by default, so you check what matters.
How this comes up in interviews
Interviewers for software, data and even non-tech roles now ask "How does ChatGPT work, in simple words?". A good answer in under a minute: it is a transformer neural network trained to predict the next token on a very large amount of text, then fine-tuned on instructions and human feedback so it behaves like an assistant; it generates answers one token at a time, which is why it can be fluent but wrong. Mentioning the knowledge cutoff and hallucinations shows you understand the limits.
Complete this sentence in 5 different ways, numbered 1 to 5. Make the first one the most likely ending and the last one the least likely but still sensible:
"[Your sentence start, e.g. The best thing about studying in Bengaluru is]"What to expect: You'll see the model can produce a range from predictable to surprising. This is roughly what temperature changes, though in a normal chat you can't see or set the probabilities directly.
Try it: Run the same prompt three times
In any free chatbot, ask exactly the same question in three new chats: "Give me a one-line slogan for a coffee shop in Bengaluru called Brew & Bean." Then, in a fourth chat, ask: "How many letter r's are in the word strawberry? Spell it out letter by letter, then count."
Write down: how different were the three slogans, and did spelling it out change the counting answer compared with asking directly?
Key takeaways
- An LLM generates text one token at a time by predicting what is likely to come next.
- Training has stages: pre-training on huge text, instruction tuning, then learning from human feedback.
- After training the parameters are frozen; chatting does not teach the model.
- Transformers use attention to relate every token to the earlier ones.
- Fluency is not truth: the same mechanism that makes answers smooth also produces confident mistakes.
- Knowledge stops at a cutoff date unless the app adds search or you paste the information.
Check yourself
Finishing the quiz marks the lesson as done.
1. What is an LLM fundamentally trained to do during pre-training?
2. Why does clicking "regenerate" give a different answer?
3. A model confidently describes a cricket match from last week, but the details are wrong and no search was used. What is the most likely reason?
4. What does instruction tuning add to a pre-trained model?
5. Does a chatbot's model learn from your conversation while you chat?
Saved on this device. Create a free account to keep it on every device.
Practise this with Sam
A real mock interview, out loud, with an honest scorecard. Free to start.