What Is an LLM?
The concept
A large language model (LLM) — the technology behind ChatGPT, Claude, Gemini, and most of the AI tools you've heard about — is, at its core, a program trained on enormous amounts of text to do one thing very well: predict the next most likely token (a chunk of text, often part of a word) given everything written so far. It generates a reply one token at a time, each time asking "given this context, what comes next?" and picking from a ranked list of candidates, weighted by probability.
That sounds almost too simple to produce the fluent, often strikingly useful answers you get from a modern chat assistant — but scale changes everything. These models are trained on a meaningful fraction of all digitized human writing: books, articles, code, forums, documentation, transcripts. By learning to predict text across that much material, the model absorbs grammar, facts, reasoning patterns, writing styles, and even some problem-solving strategies, all as statistical regularities in how words follow other words. No one hand-wrote rules for grammar or tone into the system — every capability you see emerged from this one training objective, repeated at enormous scale.
This framing — "it's a very good next-word predictor" — explains both the magic and the risk of LLMs. The magic: they can write in almost any style, answer almost any question, translate, summarize, code, and brainstorm, because all of those are patterns found somewhere in the training data. The risk: the model is optimizing for plausible continuations, not verified ones. It doesn't have a database it looks facts up in while answering you. It reconstructs an answer from learned patterns, which means it can sound exactly as confident when it's right as when it's confidently wrong. You'll see this idea again in the hallucinations lesson later in this module — it starts here, with understanding what the model is actually doing.
It also helps to know what an LLM is not. It is not a search engine indexing live web pages (unless a tool explicitly gives it that ability). It is not a calculator with guaranteed arithmetic (though it's often right, since arithmetic patterns are heavily repeated in training data). It is not conscious, and it does not "want" anything — every output is the result of a probability calculation over the next token, repeated until the response is complete. There is no persistent memory of you between separate conversations unless a product specifically adds that feature on top of the base model.
Finally, it's worth naming why this matters for you practically, before the course goes deeper: every technique in this course — prompting well, giving examples, checking outputs, designing agent workflows — is really a way of working with this prediction mechanism instead of against it. Once you internalize "next-token prediction over learned patterns," most AI behavior that otherwise seems mysterious (inconsistency, confidence on wrong answers, sensitivity to exact wording) stops being surprising.
Worked example
Take the prompt: "The capital of France is". An LLM processes this text and, internally, computes a probability distribution over every possible next token. "Paris" will have an overwhelmingly high probability, because in essentially all the training text, "The capital of France is" is followed by "Paris." Other candidates like "a," "one," or "beautiful" might get a sliver of probability because they also appear in similar contexts, but "Paris" dominates the distribution by a wide margin.
Now imagine a trickier prompt: "The best programming language is". Here there's no single dominant answer in the training data — opinions vary widely across forums, blog posts, and books, and different communities argue for Python, JavaScript, Rust, or something else entirely. The model will produce a plausible-sounding answer, phrased with the same fluency as the Paris example, but it's really synthesizing a popular-sounding opinion from the spread of views it saw, not stating a settled fact. Same mechanism, very different reliability, depending entirely on whether the question has one dominant answer in the training data or a contested one.
A third case sits in between: "Water boils at ___ degrees Celsius at sea level." This has a precise, well-documented answer (100), repeated consistently across chemistry textbooks and reference material, so the model converges on it reliably — closer to the Paris case than the programming-language case, even though both are "factual-sounding" questions.
Exercise
Open any AI chat assistant you have access to (ChatGPT, Claude, Gemini, or similar) and ask it the same factual question three different ways:
- As a direct question: "What year did the Berlin Wall fall?"
- As a fill-in-the-blank: "The Berlin Wall fell in ___."
- As an "explain like I'm five": "Explain to a 5-year-old when the Berlin Wall fell and why."
Compare the three answers. Did the facts stay consistent across phrasings? Did the tone and level of detail change more than you expected? Now repeat the exercise with a contested, opinion-based question instead (for example, "What's the best productivity method?"), phrased the same three ways. Notice how much more the content, not just the tone, shifts across phrasings when there's no single dominant answer in the training data. Write two sentences: one noting whether you trust the model more or less after seeing how phrasing affected the factual answers, and one noting the difference in how the model handled the factual question versus the opinion question.
Recap
- An LLM predicts the next token based on patterns learned from enormous amounts of text — it does not look facts up live, and has no memory beyond the current conversation.
- This explains why it can be fluent and useful across almost any topic, and also why it can be confidently wrong, especially on contested or thinly-documented questions.
- Every later lesson on prompting, trust, and verification builds on this one idea: you are talking to a pattern-completion engine, not a database, and knowing that changes how you should use it.
Resources
- Wikipedia: "Large language model" — https://en.wikipedia.org/wiki/Large_language_model
- Anthropic: "Claude's constitution" (how Claude's model is described) — https://www.anthropic.com/news/claudes-constitution
- OpenAI: "Introduction to large language models" — https://platform.openai.com/docs/guides/text-generation
- Google: "Introduction to large language models" — https://developers.google.com/machine-learning/resources/intro-llms