Index Ai Academy
Phase 6 · Editing models

Editing one fact

A rank-one change to one matrix rewrites what a product is made from; then reliability, generalization, locality, and what breaks after many edits.

About 40 minutes · Optional lesson: deeper material that no phase or certificate requires · A free account opens the whole lesson.
Already have an account? Sign in to open this lesson

What it covers10

Why it matters

A model has learned that tents are made from silnylon, and you want it to say titanium. Retraining for one fact is absurd, and a prompt only lasts for one conversation. Model editing aims between the two: find where the fact is stored and rewrite just that.

It is the most precise tool in this course, and the easiest to overrate. In this lesson you make such an edit by hand on the tiny GPT, with no gradient descent, and then test it the way the research community learned to: did it take, does it survive a rewording, what else changed, and what broke. The last question has the most interesting answer.

What you'll be able to do

  • Describe a feed-forward layer's output matrix as a lookup from keys to values, and what a rank-one change does to it
  • Change one of the tiny GPT's facts with a rank-one edit, choosing its strength with a few forward passes
  • Measure an edit four ways: reliability, generalization, locality and damage
  • Say what happens after many edits, and what research on larger models found
Quick check
You change a model's answer to one question by rewriting one of its weight matrices in the smallest sensible way. The matrix holds 4,096 numbers. A guess: how many of them change?
  1. All 4,096, though the change is fully described by 160 numbers
  2. None: facts are stored in the prompt, not the weights
  3. 32: one row of the matrix
  4. One: the weight that stores the fact
Show the answer

All 4,096, though the change is fully described by 160 numbers

A fact isn't kept in one weight. The smallest useful edit adds one pattern to the whole matrix: a column of 32 numbers times a row of 128. That is 160 numbers to describe, and it touches every entry.

The idea: a fact is a key and a value

This lesson rests on A matrix times a vector and Independence, span and rank: a matrix maps a vector to a vector, and rank counts independent rows.

A fact is a key and a value

The feed-forward lesson described each block's feed-forward layer as a memory. Its first half turns the stream into a wide list of hidden activations: 128 numbers here, each saying how strongly some pattern is present. Call that list the key. Its second half, the matrix mlp.2.weight, turns the key into 32 numbers that are added to the stream. Call those the value. One matrix holds many key-to-value pairs at once.

So "the model knows tents are made from silnylon" has a mechanical reading. At the end of the prompt "The River tent is made from", the last block's feed-forward layer holds a certain key, and its output matrix maps that key to a value which pushes the stream toward "silnylon."

wsp.mlp_keys(model, tok, prompt) returns the keys the last block holds at every position of a prompt. The next cell looks at the fact before it is edited.

Worked example

The fact, its key and its value

What the base model predicts after the prompt, then the key at the last position, and the value the output matrix turns it into.

import numpy as np
import wsp

base, info = wsp.load_state("tiny-gpt")
model, tok = wsp.build(base, info)
prompt = "The River tent is made from"

p = wsp.next_probs(model, tok, prompt)
print("predictions:",
      ", ".join(f"{tok.vocab[i]} {p[i]:.2f}" for i in np.argsort(-p)[:3]))

# 32 by 128: the last block's key-to-value matrix
W = base["blocks.1.mlp.2.weight"]
key = wsp.mlp_keys(model, tok, prompt)[-1]        # the key at the last position
value = W @ key
print("matrix", W.shape, "- key", key.shape, "- value", value.shape)
print(f"hidden units clearly on (above 0.5): {(key > 0.5).sum()} of {key.size}")
print(f"length of the value written to the stream: {np.linalg.norm(value):.2f}")
It runs right here in your browser, with nothing to install.

The model gives "silnylon" 0.59 and "recycled" 0.40: the catalog's two tent materials, and almost nothing else. The key is 128 numbers and the value 32. Only 37 of the 128 hidden units are clearly on for this prompt, which is the "pattern present" reading of a key. To change the fact, you change what this one key maps to, and leave the matrix's treatment of other keys alone as far as you can.

Quick check
From the feed-forward layer lesson: the layer expands the stream to four times its width and back. With a stream 32 wide, how many hidden activations make up a key?
  1. 8
  2. 128
  3. 4,096
  4. 32
Show the answer

128

Four times 32 is 128. The first half of the layer maps 32 numbers to 128, and the second half maps those 128 back to 32. The 128 in the middle are the key.

Here is the smallest change that does that. Say you want the value for a key kk to move by a chosen amount Δ\Delta. Add one pattern to the matrix:

W′=W+Δ k⊤k⋅kW' = W + \frac{\Delta \, k^{\top}}{k \cdot k}

Read it as: Δ k⊤\Delta \, k^{\top} is a column (Δ\Delta, 32 numbers) times a row (kk, 128 numbers): a full-size matrix built from one pattern. Dividing by k⋅kk \cdot k, the key's length squared, sets the scale so that the key now maps to exactly Δ\Delta more than before.

Two things follow, and both are easy to check with numbers. For the key itself, the new value is the old value plus Δ\Delta. For any key at right angles to kk, nothing changes at all. Because the added matrix is one column times one row, it has rank 1, and this is called a rank-one edit.

Worked example

A rank-one edit on a matrix small enough to read

A 2 by 3 matrix, a key, and a change to make. After the edit: what the key maps to, what a key at right angles to it maps to, and the rank of what was added.

import numpy as np

W = np.array([[1.0, 0.0, 2.0],
              [0.0, 3.0, 1.0]])
key = np.array([1.0, 2.0, 0.0])
other = np.array([2.0, -1.0, 5.0])               # at right angles to key: 2 - 2 + 0 = 0
delta = np.array([10.0, -4.0])                   # the change we want for key's value

W2 = W + np.outer(delta, key) / (key @ key)
print("key:   before", W @ key, "after", W2 @ key)
print("other: before", W @ other, "after", W2 @ other)
print("entries changed:", int((W2 != W).sum()), "of", W.size, "- rank of the change:",
      np.linalg.matrix_rank(W2 - W))
In the lesson you can change this and run it.

The key's value moved from [1, 6] to [11, 2]: exactly the change asked for. The other key's value is untouched, because it is at right angles to the edited one. Four of the six entries changed, and the change has rank 1. In a real model, other keys are rarely at exact right angles to yours. They overlap a little, and each gets a share of the change in proportion to the overlap. That overlap is where side effects come from.

Quick check
What does this print?
import numpy as np

W = np.zeros((2, 2))
k = np.array([1.0, 1.0])
delta = np.array([4.0, 0.0])
W2 = W + np.outer(delta, k) / (k @ k)
print(W2 @ k, W2 @ np.array([1.0, -1.0]))
  1. [4. 0.] [4. 0.]
  2. [8. 0.] [0. 0.]
  3. [0. 0.] [4. 0.]
  4. [4. 0.] [0. 0.]
Show the answer

[4. 0.] [0. 0.]

The key [1, 1] now maps to delta, [4, 0]. The second vector, [1, −1], is at right angles to the key (1 − 1 = 0), so it still maps to [0, 0].

Try it now

Then divide every cell by k · k = 5. That is the added matrix of the edit below.

Drill

Write the rank-one edit

Write rank_one_edit(W, key, delta): return a new matrix that maps key to delta more than W did, and maps every vector at right angles to key exactly as before. Don't change W.

import numpy as np

def rank_one_edit(W, key, delta):
    return W

W = np.array([[1.0, 0.0, 2.0], [0.0, 3.0, 1.0]])
key = np.array([1.0, 2.0, 0.0])
print(rank_one_edit(W, key, np.array([10.0, -4.0])) @ key)
In the lesson, your code is checked as you write it.
In your own words
In your own words: what does a rank-one edit change, and what does it leave alone?

Write a sentence or two, then compare it with a good answer in the lesson.

In code: make the edit, then measure it

Make the edit

Three choices remain: where, in which direction, and how hard.

Where. The last block's output matrix, blocks.1.mlp.2.weight, with the key taken at the end of the prompt.

Direction. The value should move the stream toward "titanium." The output layer turns the stream into a score for each token, so ask it backwards: which stream direction gives a score pattern closest to "titanium up, everything else down"? That is a least-squares problem, which NumPy solves in one call. No gradient descent is involved.

How hard. The stream passes through a LayerNorm on its way to the output layer, so the effect of a push isn't proportional to its size. The honest approach is to try a few strengths and run the model each time. Each try costs one forward pass, and wsp.Budget counts them. A fixed budget of test runs is the difference between designing an edit and tuning it until the number looks good.

Worked example

Tents are now made from titanium

Make the edit at strengths 8, 16, 32 and 64, stopping at the first one that gives "titanium" a chance of at least 0.8. Then describe the change to the weights.

import numpy as np
import wsp

base, info = wsp.load_state("tiny-gpt")
model, tok = wsp.build(base, info)
NAME = "blocks.1.mlp.2.weight"
prompt, target = "The River tent is made from", tok.token_id("titanium")

key = wsp.mlp_keys(model, tok, prompt)[-1]
want = -np.ones(len(tok.vocab)) / len(tok.vocab)
# "titanium up, everything else down"
want[target] += 1
direction = np.linalg.lstsq(base["head.weight"] * base["ln_f.weight"], want,
                            rcond=None)[0]
direction /= np.linalg.norm(direction)

def edit(strength):
    out = {name: t.copy() for name, t in base.items()}
    out[NAME] = (base[NAME]
                 + strength * np.outer(direction, key) / (key @ key)).astype(np.float32)
    return out

budget = wsp.Budget(4)
for strength in (8, 16, 32, 64):
    budget.spend()
    edited = edit(strength)
    chance = wsp.next_probs(wsp.build(edited, info)[0], tok, prompt)[target]
    print(f"strength {strength:2d}: chance of 'titanium' {chance:.2f}")
    if chance >= 0.8:
        break

change = edited[NAME].astype(float) - base[NAME]
print(f"forward passes used: {budget.used} of {budget.limit}")
print(f"entries changed: {(change != 0).sum():,} of {change.size:,}; rank of the "
      f"change: {np.linalg.matrix_rank(change, tol=1e-4)}")
p = wsp.next_probs(wsp.build(edited, info)[0], tok, prompt)
print("predictions now:",
      ", ".join(f"{tok.vocab[i]} {p[i]:.2f}" for i in np.argsort(-p)[:3]))
In the lesson you can change this and run it.

At strength 8 the chance of "titanium" is 0.09, at 16 it is 0.61, and at 32 it reaches 0.83, so the search stops after three forward passes. The model now completes the sentence with a material no tent in its training data ever had. The change touches all 4,096 entries of the matrix, about a tenth of the model's weights, yet it has rank 1: it is one column of 32 numbers times one row of 128. Counted by entries it is a large edit. Counted by what it takes to describe, it is tiny. Both counts are true, and an honest report gives both.

Quick check
The edit changed all 4,096 entries of the matrix but has rank 1. The same thing in miniature: a column of 3 numbers times a row of 4. What does this print?
import numpy as np

change = np.outer([1.0, 2.0, 3.0], [1.0, 5.0, -1.0, 2.0])
print(change.shape, int((change != 0).sum()), np.linalg.matrix_rank(change))
  1. (3, 4) 12 3
  2. (3, 4) 12 1
  3. (3, 4) 7 1
  4. (4, 3) 12 1
Show the answer

(3, 4) 12 1

A column of 3 times a row of 4 gives a number for every one of the 12 positions, so all 12 entries are nonzero. But every row is the same pattern at a different strength, so the rank is 1. Seven numbers, 3 + 4, describe it completely, as 32 + 128 = 160 describe the edit to the tiny GPT.

In your own words
In your own words: why does the edit need forward passes at all, if the formula gives the new matrix directly?

Write a sentence or two, then compare it with a good answer in the lesson.

Did it work? Four measurements

"The model now says titanium" is one measurement, and on its own it is close to worthless: you tuned the strength until it came out. Research on model editing settled on a small set of questions. A 2023 survey, "Editing Large Language Models: Problems, Methods, and Opportunities," sets out the first three below. The fourth is this lesson's own, because it is easy to measure here.

  • Reliability. Does the edited prompt give the new answer?
  • Generalization. Does a rewording give it too? "Meet the River tent, made from..." states the same fact.
  • Locality. Do unrelated facts stay as they were? A fleece should still be made from fleece.
  • Damage. Does the model still write ordinary text as well as before?

There is also a way to make the edit kinder to other keys. The plain formula spreads the change across all 128 hidden units in proportion to the key. But some of those units are busy in almost every sentence, and loading the change onto them disturbs everything. If you collect keys from ordinary text first, you can work out which combinations of units are common and steer the row of the edit away from them. The best-known editing method, called ROME, does exactly this with a table of statistics about the keys. The next cell tries both versions.

Worked example

The same edit, plain and careful, measured four ways

Both edits at strength 32. The careful one uses statistics from 150 catalog descriptions to choose its row. For each: the edited prompt, two rewordings, the other 29 product names, how many of 110 other products keep their answer, and the loss on ordinary text.

import numpy as np
import wsp

base, info = wsp.load_state("tiny-gpt")
model, tok = wsp.build(base, info)
facts = wsp.facts()
NAME = "blocks.1.mlp.2.weight"
prompt, target = "The River tent is made from", tok.token_id("titanium")
docs = wsp.catalog()

key = wsp.mlp_keys(model, tok, prompt)[-1]
want = -np.ones(len(tok.vocab)) / len(tok.vocab)
want[target] += 1
direction = np.linalg.lstsq(base["head.weight"] * base["ln_f.weight"], want,
                            rcond=None)[0]
direction /= np.linalg.norm(direction)

# keys from ordinary text
K = np.concatenate([wsp.mlp_keys(model, tok, d) for d in wsp.catalog(0, 150)])
# which units fire together
C = K.T @ K / len(K)
careful_row = np.linalg.solve(C + 0.01 * np.trace(C) / len(C) * np.eye(len(C)), key)

def edit(row, strength=32):
    out = {name: t.copy() for name, t in base.items()}
    out[NAME] = (base[NAME]
                 + strength * np.outer(direction, row) / (row @ key)).astype(np.float32)
    return out

others = [f"The {name} {p['type']} is made from" for name in facts["names"][::3]
          for p in facts["products"] if p["type"] != "tent"]
before = [int(wsp.next_probs(model, tok, t).argmax()) for t in others]
print("before the edit: catalog loss", round(wsp.text_loss(model, tok, docs), 2))
for label, row in [("plain", key), ("careful", careful_row)]:
    m, _ = wsp.build(edit(row), info)
    chance = lambda text: wsp.next_probs(m, tok, text)[target]
    reworded = np.mean([chance("Meet the River tent , made from"),
                        chance("Our River tent is made from")])
    other_names = np.mean([chance(f"The {name} tent is made from")
                           for name in facts["names"] if name != "River"])
    kept = sum(int(wsp.next_probs(m, tok, t).argmax()) == b
               for t, b in zip(others, before))
    print(f"{label:8s} edited prompt {chance(prompt):.2f} | reworded {reworded:.2f} | "
          f"other tents {other_names:.2f} | other products unchanged {kept} of "
          f"{len(others)} | catalog loss {wsp.text_loss(m, tok, docs):.2f}")
In the lesson you can change this and run it.

Both edits are reliable, 0.83, and both generalize to the rewordings. Then they part. The plain edit leaves only 39 of 110 other products with their old answer and raises the loss on ordinary text from 0.47 to 0.92. The careful one keeps 89 of 110 and a loss of 0.51. Same key, same direction, same strength: the only difference is which hidden units carry the change. Even the careful edit altered 21 products it had no business touching. Locality is the hard part of editing, and no method gets it for free.

Look at the "other tents" column: 0.83 or more. You edited a prompt about the River tent, and every tent now comes in titanium. Is that generalization or a failure of locality? It depends on what the fact was. This model learned that a product's material depends on its kind. It has no separate entry for the River tent, so the key at the end of the prompt is mostly "tent." The smallest fact this model lets you edit here is "tents are made from titanium."

Editing methods for real models work at an earlier point for this reason. They take the key at the last token of the subject, in a middle layer, where the model is still assembling what it knows about that one thing. The principle carries over: you can only edit a fact at the grain the model stored it.

Quick check
After an edit, a model answers the edited question correctly, but half of its answers to unrelated questions have changed. Which measurement failed?
  1. None: the edit did its job
  2. Reliability
  3. Locality
  4. Generalization
Show the answer

Locality

Locality asks whether everything the edit wasn't aimed at stayed the same. Reliability passed here, since the edited question gets the new answer. An edit that fixes one fact and disturbs fifty others has not done its job.

Drill

Score locality

Write locality(before, after). Each argument is a list of probability arrays, one per prompt, in the same order: the model's next-token probabilities before and after an edit. Return the share, from 0 to 1, of prompts whose most likely token is the same in both.

import numpy as np

def locality(before, after):
    return 0.0

before = [np.array([0.7, 0.2, 0.1]), np.array([0.1, 0.8, 0.1]),
          np.array([0.3, 0.3, 0.4])]
after = [np.array([0.6, 0.3, 0.1]), np.array([0.5, 0.4, 0.1]),
         np.array([0.2, 0.2, 0.6])]
print(locality(before, after))
In the lesson, your code is checked as you write it.
In your own words
In your own words: the edit was aimed at the River tent and changed every tent. Why, and what does that teach about editing facts?

Write a sentence or two, then compare it with a good answer in the lesson.

In production: after many edits

One edit went well enough. The obvious next step is to fix many facts, one after another, each a rank-one change to the same matrix. Each edit's key overlaps a little with the keys of earlier edits, so each one disturbs the others. The next cell makes twelve careful edits, one per kind of product, and watches what happens.

Worked example

Twelve edits to one matrix

Give each of the twelve kinds of product a material it never has in the catalog, one careful edit at a time. After 1, 2, 4, 8 and 12 edits: how many of the edits so far still hold, and the loss on ordinary text.

import numpy as np
import matplotlib.pyplot as plt
import wsp

base, info = wsp.load_state("tiny-gpt")
model, tok = wsp.build(base, info)
NAME = "blocks.1.mlp.2.weight"
docs = wsp.catalog()
K = np.concatenate([wsp.mlp_keys(model, tok, d) for d in wsp.catalog(0, 150)])
C = K.T @ K / len(K)
C = C + 0.01 * np.trace(C) / len(C) * np.eye(len(C))

NEW = {"tent": "titanium", "camp mug": "silnylon", "fleece": "stainless",
       "daypack": "merino", "headlamp": "waxed", "water bottle": "enamel",
       "rain jacket": "titanium", "base layer": "stainless", "down vest": "silnylon",
       "sun hoodie": "enamel", "sleeping bag": "waxed", "hiking pants": "merino"}

def careful_edit(state, prompt, target):
    m, _ = wsp.build(state, info)
    key = wsp.mlp_keys(m, tok, prompt)[-1]
    want = -np.ones(len(tok.vocab)) / len(tok.vocab)
    want[target] += 1
    direction = np.linalg.lstsq(state["head.weight"] * state["ln_f.weight"], want,
                                rcond=None)[0]
    direction /= np.linalg.norm(direction)
    row = np.linalg.solve(C, key)
    for strength in (8, 16, 32, 64):
        out = {name: t.copy() for name, t in state.items()}
        out[NAME] = (state[NAME] + strength * np.outer(direction, row)
                     / (row @ key)).astype(np.float32)
        if wsp.next_probs(wsp.build(out, info)[0], tok, prompt)[target] >= 0.8:
            break
    return out

state, done, rows = base, [], []
print(f"before: catalog loss {wsp.text_loss(model, tok, docs):.2f}")
for count, (kind, material) in enumerate(NEW.items(), 1):
    state = careful_edit(state, f"The River {kind} is made from",
                         tok.token_id(material))
    done.append((kind, material))
    if count in (1, 2, 4, 8, 12):
        m, _ = wsp.build(state, info)
        held = sum(int(wsp.next_probs(m, tok, f"The River {k} is made from").argmax())
                   == tok.token_id(v) for k, v in done)
        loss = wsp.text_loss(m, tok, docs)
        print(f"after {count:2d} edits: {held:2d} of {count:2d} still hold; catalog "
              f"loss {loss:.2f}")
        rows.append((count, held / count, loss))

x, share, loss = zip(*rows)
plt.plot(x, share, marker="o", label="share of edits that hold")
plt.plot(x, loss, marker="s", label="catalog loss")
plt.xlabel("edits made")
plt.title("Twelve edits to one matrix")
plt.legend()
plt.show()
In the lesson you can change this and run it.

The first four edits all hold, while the loss on ordinary text creeps up: 0.51, 0.53, 0.66. After eight edits two have slipped and the loss is 1.24. After twelve, ten hold and the loss is 1.69, more than three times the 0.47 it started at. The edits were each "careful", and each was fine alone. Stacked in one small matrix they crowd each other. A model this size has little room, so the effect shows after a dozen edits. Larger models have far more room, and the same thing happens to them later.

That matches what researchers report. The method ROME (2022) introduced the rank-one edit with key statistics that you just built. Its successor MEMIT spreads a batch of edits across several layers to fit thousands at once. The 2023 survey compared these and other approaches and found two patterns worth remembering.

First, methods that rewrite weights lose quality as edits accumulate one after another, as yours did. Methods that keep each edit outside the weights, in a memory the model consults, held up better. Second, an edited fact often fails to carry through to reasoning, which the survey calls portability: methods of this kind scored under half on questions that needed one step of inference from the new fact. Tell a model that a tent is made from titanium and it may still not conclude that the tent is heavy.

So fact editing is a tool for one fact, or a few, that you will test afterward. When many facts change, or change often, the usual answer is to keep them outside the model and retrieve the ones a question needs.

Quick check
Twelve careful edits to one matrix each worked when made alone, yet together some failed and ordinary text got worse. Why?
  1. The model ran out of vocabulary
  2. Each edit's key overlaps the others a little, so every new edit also shifts the earlier ones and the model's other keys
  3. A matrix can hold at most ten facts
  4. The twelfth edit used a different formula from the first eleven, and that upset all the others
Show the answer

Each edit's key overlaps the others a little, so every new edit also shifts the earlier ones and the model's other keys

A rank-one edit leaves a key unchanged only when that key is at right angles to the edited one. Real keys overlap, so each edit leaks into the others. The leaks are small one at a time and add up.

Where it shows up in your week

When a product fact changes, someone asks whether to edit the model. You run the four measurements before anyone ships it, you keep the budget of test runs fixed, and for many facts that change often you keep them outside the model and retrieve them.

In your own words
A team wants to update 5,000 product facts in a deployed model every month. In your own words, would you use weight edits, and what would you suggest?

Write a sentence or two, then compare it with a good answer in the lesson.

Goes deeper in

Practice what you just read

  • 5 code examples you can edit and run
  • a practice exercise that checks your code
  • a 5-question quiz
  • 9 review cards for spaced repetition

Try a quiz question

In a feed-forward layer, what are the "key" and the "value"?

  1. The key is the hidden activations for an input; the value is what the output matrix writes to the stream for that key
  2. The key is the token's id; the value is its embedding
  3. The key is the whole prompt that was typed in; the value is the reply the model gives to it
  4. The key is the bias; the value is the weight

Answer it at the end of the lesson to see if you're right, and why.

Before this lesson

Go deeper, if you like

Optional lessons that build on this one. They never block a phase or a certificate.

Keep learning

Where this lesson leads.

575 lessons in thirteen phases. 176 are free without an account, and a free account opens the rest. See every lesson.

Phase 6 · about 35 hours of study

Modern LLM engineering

Post-training, fine-tuning, serving and safety 48 lessons 2 free
  1. Base models vs assistants FreeWhat post-training adds to a pretrained model.30 min
  2. Supervised fine-tuningTeach behavior from example conversations.40 min
  3. Instruction and preference dataThe datasets that shape assistant behavior.30 min
  4. RLHFReward models and reinforcement learning from human feedback.40 min
  5. DPO and direct preference methodsPreference training without a separate reward model.40 min
  6. Constitutional AITraining with AI feedback guided by written principles.35 min
  7. Training reasoning modelsReinforcement learning on verifiable problems.40 min
  8. Prompting vs RAG vs fine-tuningPick the cheapest approach that works.30 min
  9. LoRAFine-tune a big model by training small low-rank updates.40 min
  10. Fine-tune an open modelRun a real fine-tune and evaluate the result.45 min
  11. DistillationTrain a small model to imitate a large one.35 min
  12. Tuning an embedding model for retrievalWhen search fails on your own vocabulary: train an adapter with a contrastive loss on question and passage pairs, test on held-out passages, use batch size and hard negatives, and measure what tuning costs elsewhere.45 min
  13. QuantizationStore weights in fewer bits to run models cheaply.35 min
  14. Quantization in practice: zero points, calibration and formatsScale and zero point, per-channel scales, calibration sets, error on a layer's output, quantization-aware training and the file formats you meet at work.35 min
  15. Serving modelsBatching, throughput and latency in production.35 min
  16. One request through a serving stack: latency and costFollow one request through queue, prefill, a prefix cache, decode and the stream, and price a token from a GPU's hourly cost.35 min
  17. Speculative decodingDraft with a small model, verify with a big one.35 min
  18. Running open models locallyDownload, quantize and run a model on your own machine.30 min
  19. Will it fit? Memory for serving and fine-tuningWeights, KV cache, runtime and adapter states added up for a model card and a GPU, with a calculator.30 min
  20. InterpretabilityLooking inside models to see what they compute.40 min
  21. Confident and wrongWhy a model answers with the same assurance whether it is right, wrong or asked about something that does not exist.30 min
  22. Why it can't spell or count lettersThe model sees token IDs, and an embedding learned from usage carries nothing about spelling.25 min
  23. Copying from the promptIn-context learning as a two-head copying circuit: the base tiny GPT cannot copy a repeated pattern, a model trained on repeats grows an induction head and can.30 min
  24. Why temperature changes the answerThe same prompt at 0, 0.7 and 1.5: deterministic loops, plausible variety, broken grammar, measured on a real model.25 min
  25. Why long prompts diluteAttention is a budget shared by every token, so a fact far from the question loses its share; measured, with the cliff beyond the trained window.25 min
  26. Why it never says "I don't know"A chat model answers every request because its fine-tuning data always did; preference training adds flattery; how to build abstention in.25 min
  27. Benchmarks and their limitsWhat leaderboards measure, and what they miss.38 min
  28. Alignment and safetyMaking capable models behave as intended.35 min
  29. Evaluating agentsMeasure multi-step, tool-using behavior.35 min
  30. Reading AI papersGet the idea out of a research paper in 30 minutes.30 min
  31. Capstone: a tuned, evaluated assistantFine-tune, evaluate and deploy a specialist assistant.45 min
  32. Weight-space programming: what it is, and what has been shown Free OptionalA language model reads and writes another network's weights with no training loop: the published experiments, their limits and the criticism, and one edit you make by hand.30 min
  33. Anatomy of a checkpoint OptionalA model file is a set of named tensors: count them, find the ones that write to the residual stream, and save one to bytes and back.35 min
  34. Reading weights without running the model OptionalSingular values, and an attention head's token-to-token circuit, read from the matrices alone and then checked against a measurement.35 min
  35. Writing weights by hand OptionalSet a small network's 55 weights so it catches every ball in a paddle game, the way a program would be written, and meet compiled transformers.40 min
  36. Edits are deltas: task vectors, patches and hashes OptionalSubtract a base from a fine-tune and the edit becomes an object: scale it, take it away, ship it as a patch, and prove by hash what was applied.35 min
  37. Merging models OptionalAverage two fine-tunes, add their task vectors, try SLERP, TIES and DARE, measure the interference, and compare with a model trained on both tasks.40 min
  38. Low-rank edits OptionalCut a fine-tune's change down to its largest directions with the SVD: how few a behavior needs, and how to pull an adapter out of a delta.35 min
  39. Directions: steer the run or edit the weights OptionalFind a behavior's direction in the residual stream, add it at run time, project it out of the weights, and see the effect change from input to input.40 min
  40. Editing one fact OptionalA rank-one change to one matrix rewrites what a product is made from; then reliability, generalization, locality, and what breaks after many edits.40 min
  41. The editor's harness OptionalWhat an LLM that edits weights is given and what it is denied: weights as text and their cost in tokens, a sandbox, a budget of forward passes, and rules the harness enforces.40 min
  42. An LLM reads a network's weights OptionalSix recorded sessions: an LLM is handed 1,315 weights as text and asked what the network was trained to do. Score each answer against the network, and study one it got wrong.40 min
  43. An LLM fixes a model's weights OptionalFind a tiny GPT's loop by hand, then replay recorded sessions in which an LLM hunts for it with ten forward passes, and measure every fix on held-out questions.45 min
  44. An LLM writes a network's weights OptionalFive recorded sessions: an LLM writes every weight of a paddle network from the game's source and one sentence. Score each against its specification, test its design claims, and edit one by hand.40 min
  45. Did it work? Testing an edit OptionalReview a patch for a tiny GPT that can't stop talking: held-out questions, a paired comparison, a same-size control, and what the edit cost.40 min
  46. The published edit, on a real model OptionalRead the edit published for an open model of 870 million weights, apply and verify one in the same form, see what 16-bit and 4-bit storage do to a fix, and run the real thing from a notebook.45 min
  47. Safety, licenses and provenance OptionalBefore a custom model is released: trace its lineage by hash, work out what its parents' licenses pass on, see what an edit can do to a safeguard, and write its card.40 min
  48. Capstone: ship a custom model OptionalSet release gates, then diagnose, edit, measure and document a tiny GPT with a fault no lesson has fixed, and write the record a stranger could check.45 min
Part of a free course 17 lessons

Creating Custom LLMs: Editing Open Weights Using Weight-Space Programming

Read a model's weights, change them by hand and judge an edit someone else made: deltas, merging, low-rank edits, directions and fact edits on a tiny GPT in your browser. Then replay recorded sessions of an LLM reading, fixing and writing weights, check the published edit of a real open model, and ship a model of your own with its patch, verdict and card. Every number is printed by the lesson's own code, and the method's limits are taught with it.

Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write