What AI Is Actually Doing When It Answers You

Most explanations of AI either assume you already know or drift into metaphors about brains that leave you no better off. Here is the version that actually helps.

It predicts what comes next

The tools you have used — ChatGPT, Claude, the assistant in your email — are built on large language models. A model like this was trained on an enormous quantity of writing until it got very good at one narrow skill: predicting what text should come next.

That is genuinely all it does. Answering, summarising, translating, drafting — all of it falls out of doing that prediction extremely well.

When you type a question, the model is not looking anything up. It is producing the words that most plausibly follow your question, based on patterns absorbed during training.

Why that explains almost everything

Hold that idea and the odd behaviour stops being mysterious.

It writes fluently because fluency is precisely what it learned.

It sounds confident because confident writing is what it mostly saw. There is no mechanism converting internal uncertainty into hedged language, the way there is in a person.

It sometimes invents things because producing something plausible and recalling something true are, from its point of view, the same operation. Where the patterns were thin, plausible and true come apart.

It is not lying when it gets something wrong. It has no concept of true or false.

The practical consequence

This is where it stops being philosophy.

Because it predicts rather than retrieves, you cannot read reliability off the writing. A confident paragraph and a hallucinated one look identical. Experience does not help; the tone carries no signal.

So the useful habit is not "distrust everything" — that wastes the tool. It is knowing where errors concentrate: names, numbers, dates, citations, anything recent, anything local. Vague claims are usually fine. Specifics are where the risk lives.

Where it is strong

The same idea tells you where it earns its place. It is excellent when it transforms something you gave it:

• Summarising a long report or a transcript
• Rewriting for a different reader, or for length
• Generating twenty options so you can pick three
• Explaining something dense in plain language

And weak when it must supply what you did not give it: specific facts, recent events, your context, anything requiring judgement with consequences.

That single distinction — transform versus supply — will steer you correctly more often than any list of use cases.

What to do with this

Two things.

First, brief it properly. Most disappointing output is a briefing problem: a model given a category writes about the category, and category-level prose always reads as hollow. Give it the audience, the constraint, the actual facts.

Second, decide before you ask what happens if the answer is wrong. If the answer is "nothing much", use it freely. If the answer is "a client sees it" or "I make a decision I cannot reverse", the output is a starting point and nothing more.

That is most of it. The rest is practice.