What is an LLM, in plain words (and why it guesses instead of knows)

Here's a surprising thing. When you ask ChatGPT for the capital of France, it doesn't look it up in a reference or "recall" a fact. It guesses which word most likely comes next. Then the next one. And the next. The answer being correct is a side effect, not the goal.

That's an LLM — a large language model. Grasp this one idea and you'll instantly see why it can be brilliant and why it sometimes spouts nonsense with a straight face.

What an LLM actually is

An LLM is a very large program trained to continue text. You give it a beginning; it builds the rest.

Think of your phone's autocomplete, but cranked up to the extreme. Your phone suggests one next word. An LLM suggests the next word, then another, then another — until a whole answer, paragraph, or chunk of code takes shape.

"Large" is about size. The model is billions of number-settings (called parameters, or weights), tuned automatically until it got good at guessing. "Language" because it works with text — or more precisely with tokens, the chunks of words the text is sliced into.

How it does it

There isn't a single rule inside like "if asked for a capital, reply with a city name." Nobody wrote rules like that. Instead, there was training.

• The model was shown a huge amount of text: books, articles, forums, code.
• Each time, the next word was hidden and it had to guess.
• A miss nudged millions of settings slightly. Trillions of times over.

So it absorbed the patterns of language: that "the capital of France is" is usually followed by "Paris," and def in code is followed by a function name. It didn't memorize facts as a list — it learned how text about those facts is shaped.

That's why, for each prompt, it doesn't recall — it recomputes the most likely continuation. The same question can give slightly different answers; that's temperature, the dial for "how risky to be when picking words."

Why it's confidently wrong

Here's the trap. The model always produces a plausible continuation — but plausible isn't the same as true.

If your question mentions a book that doesn't exist, the model will happily "continue": invent an author, a year, a plot. It sounds convincing because, grammatically and stylistically, it's a perfect continuation. There's just no fact-check behind the words. That's a hallucination — and now you see where it comes from: the model isn't lying on purpose, it's finishing the text the only way it knows.

The practical takeaway: an LLM is strong where form matters (rephrase, explain, sketch code, translate) and needs checking where fact matters (dates, numbers, names, quotes).

Where you meet it

Almost everywhere there's "smart" text. ChatGPT, Claude, Gemini are LLMs with a friendly shell. Code autocomplete in your editor, support chatbots, email drafting — same text-continuation machine under the hood.

When you give an LLM access to tools — search, your files, code — it becomes an AI agent: not just talking, but acting. And to make it answer from your data rather than memory, you bolt on RAG — feeding the right text straight into the prompt.

Get the core idea and you stop either fearing the LLM as magic or trusting it like an encyclopedia. It's a tool with a clear principle: a brilliant text generator that needs you to keep watch over the facts.

How is an LLM different from a normal program?

A normal program is written with rules: "if this, do that." An LLM got no rules — it found the patterns itself from examples. So it's flexible but unpredictable: the same prompt can yield different answers.

Does an LLM actually understand anything?

Even researchers argue over this. The safe view: it brilliantly models how people write about things, but it has no experience of its own and no check against reality. That's both its strength and its weakness.