What an LLM actually does

Autocomplete, run in a loop

Your phone's keyboard suggests the next word as you type. A large language model (LLM) does the same job at an enormous scale. You give it some text. It works out how likely each possible next piece of text is.

Those pieces are called tokens. As in your first LLM call, a token is a word or a piece of a word. The model reads tokens, which are chunks of characters, so individual letters aren't directly visible to it: that's why it can miscount the letters in a word. The full list of tokens a model knows is its vocabulary. The model gives every token in its vocabulary a probability. That's a number from 0 to 1 saying how likely the token is to come next. A probability of 0.31 means "about 31 times in 100".

Take the text Our spring sale ends. A model might score the candidates like this (numbers invented for illustration):

Notice the space before Sunday in the table. Tokens often include the space in front of a word. The probabilities across the whole vocabulary add up to 1, the same way the slices of a pie add up to the whole pie.

In plain words: the model looks at your text and asks "what usually comes next?" Its answer is a list of options, each with a score for how likely it is.

The model picks one token and adds it to the end of the text. Then it runs again on the longer text to score the token after that. That's the whole generation process:

1. Read all the text so far: your prompt plus everything generated already.
2. Give every possible next token a probability.
3. Pick one token and add it to the end.
4. Repeat. Stop when the model picks a special end-of-reply token, or when the reply reaches max_tokens (the length limit you set in your request).

A 300-word reply is a few hundred trips around that loop. The model doesn't write the whole answer in private first and then type it out. It chooses each token in light of the ones before it.

The step-through below runs the same loop on a toy "model". The toy is a Python dictionary. For each word, it lists the words that can follow and their probabilities. The loop always takes the most likely option. That's called greedy picking, because it grabs the biggest number every time.

"sale": {"ends": 0.3, "<end>": 0.7},

A real model differs from this toy in two big ways:

• It looks at all the text so far, not only the last word. So "sale ends" after "Black Friday" gets different scores than "sale ends" after "spring".
• Its probabilities come from billions of learned numbers, not a hand-written table.

The loop has the same shape.

The top choice doesn't always win

If a model always took the most likely token, the same prompt would give the same reply every time. Chat models usually sample instead. Sampling means picking at random, but weighted by the probabilities. Think of a raffle where Sunday holds 31 tickets out of every 100 and soon holds 22. So Sunday wins about 31% of the time and soon about 22%.

One different pick early on changes everything after it. That's why asking twice can get you two different subject lines.

Python's random.choices makes the same kind of weighted pick:

import random

options = ["Sunday", "soon", "tomorrow", "March"]
weights = [0.31, 0.22, 0.18, 0.09]

for attempt in range(5):
    pick = random.choices(options, weights=weights)[0]
    print("Our spring sale ends", pick)

random.choices can pick several items at once, so it returns a list. [0] takes the single pick out of that list. Run the cell a few times. The likely words show up most often, but not every time. (The weights here don't add up to 1. random.choices only cares how big they are compared with each other.)

Note: Earlier versions of the API let you tune how adventurous the picking is with a setting called temperature. The current version (2026-10-21) doesn't accept it. Phase 5 explains how sampling works inside a model.

chance = 0.6 * 0.7 * 0.6 * 0.7 * 1.0
print(round(chance, 4))

Where the probabilities come from

The probabilities come from training. Training happens in two broad stages.

Pretraining. The model reads a huge amount of text: web pages, books, articles and code. At each spot in the text, it guesses the next token. Then it compares its guess with the real next token. It nudges its internal numbers so it would guess a little better next time. Those numbers are called parameters, or weights, and a large model has billions of them.

Repeat that over trillions of tokens and the model picks up grammar, facts, writing styles and patterns of reasoning. It learns all of those because they all help predict text.

Post-training. A pretrained model continues text. It doesn't reliably follow instructions. Give it "Write a subject line for our spring sale" and it might carry on with more lines that look like a list of homework prompts. So it gets more training on example conversations, plus feedback on which replies are better. That turns it into an assistant. It answers the request, follows a system prompt, declines harmful tasks and stops when it's done. Phase 6 covers both stages in depth.

Three practical consequences follow:

• Knowledge has a cutoff. The model knows what was in its training data, up to the date that data was collected. It doesn't know this week's news, yesterday's campaign results or anything in your company's files. It only sees those if you put them in the request.
• It doesn't look things up. By default, nothing gets searched or checked. A model uses a search engine, a database or your documents only when the app around it provides them. That happens through tools or retrieval, both covered later in Phase 1.
• It doesn't learn from your requests. Every request runs on the same fixed model. Your conversations don't change it.

Reasoning models such as Claude Opus 5.5 fit the same picture. The thinking block you saw in your first call is text the model generates, token by token, before its answer. Writing out the steps in between gives the later tokens more to build on. That's why thinking helps on hard problems. Those thinking tokens are billed as output.

Why models get some things wrong

Next-token prediction explains most of the odd behavior you'll run into. The best known is a hallucination: a confident, specific answer that's false.

Tip: Use the model for language work: explaining, rewriting, classifying and summarizing. Use code for anything that must be exact.

import requests

campaigns = [
    {"name": "Spring Sale", "impressions": 182000, "clicks": 5470, "conversions": 301},
    {"name": "Brand Video", "impressions": 410000, "clicks": 3800, "conversions": 41},
]

lines = []
for c in campaigns:
    ctr = c["clicks"] / c["impressions"] * 100
    conversion_rate = c["conversions"] / c["clicks"] * 100
    lines.append(f"{c['name']}: CTR {ctr:.2f}%, conversion rate {conversion_rate:.2f}%")
facts = "\n".join(lines)

prompt = f"""Here are last week's campaign results, already calculated:
{facts}

In two sentences, explain to a busy manager which campaign performed better and why."""

response = requests.post(
    "https://api.anthropic.com/v1/messages",
    headers={"x-api-key": "added-by-the-academy-server", "anthropic-version": "2026-10-21"},
    json={
        "model": "claude-opus-5-5",
        "max_tokens": 1000,
        "messages": [{"role": "user", "content": prompt}],
    },
    timeout=60,
)
response.raise_for_status()

print(facts)
print()
for block in response.json()["content"]:
    if block["type"] == "text":
        print(block["text"])