LLMs in Plain English

Every agent you'll use or build in this course has a large language model at its center. If you treat that model as a black box, its behavior seems random: brilliant one minute, confidently wrong the next, oddly bad at counting letters in a word while writing working software.

None of that is random. Once you understand four ideas, tokens, prediction, training, and context, you can predict most of what a model will do well, where it will slip, and what to change when it does. This lesson explains those ideas without any math. By the end, you'll know why models make things up, why the same prompt gives different answers, why they forget everything between conversations, and what "thinking" or "reasoning" actually means when a model does it.

Tokens: how models read and write#

Language models don't see letters or words the way you do. They see tokens: chunks of text that are often a whole common word, part of a longer word, a piece of punctuation, or a space plus a word. Before a model reads your message, a component called a tokenizer splits it into these chunks and turns each one into a number.

Here's an illustration of how a sentence might be split. The exact splits vary between models:

Text:    The agent refactored unbelievably messy code.
Tokens:  The | agent | ref | actored | un | believ | ably | messy | code | .

A few practical rules of thumb:

• Common words are usually one token. Rare words, names, and made-up words get split into several.
• In English, a token averages somewhat less than a word, so a 1,000-word document is more than 1,000 tokens. The exact ratio depends on the model and the language.
• Code, numbers, and non-English text often use more tokens per word than ordinary English prose.

Why tokens matter to you#

Tokens are the unit of everything in the model's world. Cost is billed per token. Speed depends on how many tokens the model must write. And the context window, the model's working memory, is measured in tokens. You'll go deep on all three in Context Windows, Tokens, and Cost.

Tokens also explain some strange failures. Ask a model how many times the letter "r" appears in a long, unusual word and it may get it wrong, because it never saw the individual letters, just a few chunks. The same goes for reversing strings or doing exact arithmetic on long numbers in its head. The fix is easy once you know the cause: ask it to use a tool, such as running a line of code, for anything that requires character-level precision.

Prediction: one token at a time#

At its core, a language model does one thing: given some text, it predicts what token comes next. To write a reply, it predicts a token, appends it to the text, predicts the next one, and repeats until it predicts a "stop" signal.

That sounds like autocomplete, and mechanically it is. The surprise is how much the model has to understand to predict well. To continue "The capital of Australia is", it needs geography. To continue a half-written function, it needs to understand the code. To continue "Here's a plan to migrate our database with zero downtime:", it needs to know how databases, migrations, and downtime work. Prediction at this level of quality requires something that behaves a lot like understanding, even though it's produced by pattern recognition over vast amounts of text.

Why the same prompt gives different answers#

At each step the model doesn't produce a single next token. It produces a list of likely candidates with probabilities: perhaps "Canberra" is overwhelmingly likely, but in a creative sentence there might be dozens of reasonable options. The system then samples one, usually favoring the likely ones but not always picking the top choice.

That small amount of randomness is why two runs of the same prompt differ. It's helpful for creative work and a nuisance for anything that needs consistency. You'll handle it the way professionals do: with clear instructions, examples of the output format you want, and checks on the result, not by hoping for the same answer twice.

Training: where the knowledge comes from#

Models learn in two broad phases.

Pretraining: reading a library#

First, the model is trained on an enormous collection of text: books, websites, articles, code, and more. The training process repeatedly hides the next token and adjusts the model to predict it better, across trillions of examples. Along the way the model absorbs grammar, facts, reasoning patterns, coding conventions, and the styles of countless kinds of documents.

The result is powerful but raw. A pretrained model will happily continue any text, but it doesn't reliably follow instructions or behave like a helpful assistant.

Post-training: learning to be useful#

Next, the model is trained on examples of good assistant behavior: following instructions, answering helpfully, declining harmful requests, admitting uncertainty. This phase uses human feedback on which responses are better, and increasingly reinforcement learning, where the model attempts tasks with checkable results, like making a test suite pass or solving a math problem, and is rewarded for success.

This second phase is where tool use and agentic skills come from. Models trained on many examples of calling tools, reading results, and recovering from errors become much better agents. It's a big part of why agents became practical, as you saw in What Is an AI Agent, Really?.

The knowledge cutoff#

Training happens on data collected up to a certain date, called the knowledge cutoff. The model knows nothing about events after it unless you tell it. Ask about last week's news without a search tool and it will either say it doesn't know or, worse, guess. When freshness matters, give the agent a way to look things up.

Context: the only thing the model sees#

Here's the idea that changes how you use models more than any other: a model has no memory between requests. Each time it runs, it sees only the text sent to it in that request, called the context, and nothing else.

When you chat with an assistant, it feels like the model remembers your conversation. What's really happening is that the app sends the entire conversation so far with every new message. The model reads it all from the top, every time, and predicts the next reply.

Request 1:  [system instructions] [your message 1]
Request 2:  [system instructions] [your message 1] [reply 1] [your message 2]
Request 3:  [system instructions] [msg 1] [reply 1] [msg 2] [reply 2] [msg 3]

"Memory" features in chat apps work the same way: they save notes about you and insert them into the context of future conversations. Agents work this way too. Every file an agent reads and every command result it sees is added to its context, which is why context fills up during long tasks.

This has huge practical consequences:

• If information isn't in the context, the model doesn't have it. Your project's conventions, your preferences, yesterday's decisions: write them down and include them.
• Everything in the context influences the output, including stale, irrelevant, or wrong information. More isn't always better.
• Starting a fresh conversation genuinely gives you a clean slate, which is often the fastest fix for a session that has gone off the rails.

Module 3 is devoted to doing this well, under the name context engineering.

Reasoning: thinking before answering#

Because a model produces its answer one token at a time, it can't go back and reconsider a word once it's written. For hard problems, writing out intermediate steps helps: each step becomes context the model can build on, the way scratch paper helps you with long division.

Modern models build this in. Before giving a final answer, they can generate reasoning (often called "thinking"): a stretch of working-out where they break down the problem, consider approaches, and check their logic. Depending on the product, you may see a summary of it, a "thinking" indicator, or nothing at all.

Reasoning matters most for multi-step problems: debugging, planning, math, and tricky analysis. Many models let you adjust how much effort they put into it. More reasoning generally means better answers on hard tasks, at the cost of more time and more tokens. For simple tasks, it's wasted. You'll learn to set this deliberately in Choosing the Right Model for the Job.

Why models make things up#

Put the pieces together and hallucination, the model stating false things confidently, makes sense. The model is trained to produce plausible continuations. When it knows the answer, the plausible continuation is the true one. When it doesn't, the most plausible-sounding continuation might be a realistic but invented citation, function name, or statistic.

Post-training has made models much better at saying "I don't know", but the underlying tendency remains, especially for specific details like names, numbers, dates, quotes, and URLs. The most reliable fixes are all about context and tools:

• Provide the source. Paste the document, attach the file, or point the agent at the folder.
• Give it a way to look things up, such as web search or a documentation tool, and ask it to cite what it found.
• Ask for verification: "Quote the exact sentence that supports each claim."

Predicting model behavior: a cheat sheet#