Index Ai Academy
Phase 6 · Weight-space programming

An LLM writes a network's weights

Five recorded sessions: an LLM writes every weight of a paddle network from the game's source and one sentence. Score each against its specification, test its design claims, and edit one by hand.

About 40 minutes · Optional lesson: deeper material that no phase or certificate requires · A free account opens the whole lesson.
Already have an account? Sign in to open this lesson

What it covers10

Why it matters

In Writing weights by hand you wrote 55 weights by hand and your network caught every ball. It took you most of a lesson. The post that named weight-space programming claims an LLM can do that job from a description: read the rules, think, and write the weights, with no training and no test run.

Here you check that claim on a network small enough to check completely. Five sessions were recorded for this lesson. You open them, score what the LLM wrote against what it was asked for, test what it said about its own weights, and then edit one of its networks yourself.

What you'll be able to do

  • Read a recorded session: what the LLM was given, what it was denied and what it replied
  • Score a written network against its specification on every possible input
  • Test a claim about what one unit computes, and edit a written network by hand
  • Say what five exact networks show about writing weights, and what they leave open
Quick check
An LLM gets the paddle game's source code and one sentence: "catch the ball." It must write every weight of a 4-32-32-3 network in one reply, with no code to run and no practice game. A guess: how many of 200 balls will its network catch?
  1. All 200: the rule can be worked out from the source and written as weights
  2. None: a network has to be trained before its weights can mean anything
  3. About 60: weights written as text are too imprecise to act as a policy
  4. About 120: without a test run it will get the wall bounces slightly wrong
Show the answer

All 200: the rule can be worked out from the source and written as weights

It caught all 200, in three sessions of three. The game is a short program, and where the ball lands is a formula. A network that computes the formula needs no practice. Whether this result carries over to larger tasks is a separate question, and the lesson ends on it.

The claim: an LLM writes every weight, from a rule

This lesson rests on Writing weights by hand: weights written by hand, with ReLU units as switches.

Five recorded sessions

An LLM answers differently each time it is asked, and a lesson has to print the same thing on every run. So the sessions here were run once, on November 10, 2026, and saved. What you load is a recording: the prompt, the reply and an audit, exactly as they were. Nothing is generated when you run a cell.

The editor was Claude Fable 5.2, working inside Claude Code, in a fresh session each time. It could see the name of this course and nothing of its contents. It got one file: the paddle game's source code, the shape of the network with every weight at zero, and a specification. It was allowed no tools: no code, no test run, no second try. It had to answer in one reply, with one line for each weight that isn't zero.

Those are the conditions of the post's fifth experiment, in which GPT-6 Astra wrote networks for three small video games from their rules and source code. One difference matters. The post's harness switched the editor's tools off. Here the editor was told not to use them, and its transcript was read afterward. The recording's audit lists every tool call it made, so you can see for yourself.

Worked example

Open one recording

Load the first session. Print who the editor was, what the audit found, the specification it was given and the start of its reply.

import wsp

session = wsp.load_session("write-catch-1")
print("recorded:", session["recorded"], "| editor:", session["editor"], "| date:",
      session["date"])
audit = session["audit"]
print("tool calls:", audit["tool_calls"], "| outside the rules:",
      audit["outside_the_rules"])
print("time:", audit["seconds"], "seconds | output tokens:", audit["output_tokens"])

spec = session["prompt"].split("## The specification")[1].split("Scoring:")[0]
print("specification:", " ".join(spec.split()))
print("reply, first lines:", session["reply"].split("\n")[:4])

layers, info = wsp.load_network(session["made"])
print(session["made"], "| lines of weights:", info["training"]["entries_written"],
      "of", info["parameters"])
It runs right here in your browser, with nothing to install.

The audit shows one tool call, reading the prompt, and none outside the rules. The session took 61 seconds and 6,196 output tokens, most of them thinking that the recording doesn't hold. The reply names 31 weights out of 1,315. Every other weight stays at zero. The practice API serves the result as a network called written-catch-1, next to the trained ones you already know.

Score it against the specification

A specification is a rule you can run. "Catch the ball" means: work out where the ball will land, and step toward that column. The game has 9 ball columns, 8 rows, 3 directions and 9 paddle columns, which makes 1,944 possible observations. That is few enough to test every one.

So a written network gets two scores. Agreement is the number of observations where its move is the rule's move. Caught is the number of balls it catches in play. A network can catch every ball in a sample and still break the rule on an observation the sample never reached. Agreement on all 1,944 leaves no such gap.

The next cell scores four networks: paddle-catch, which was trained by gradient descent, and the three networks from the three recorded sessions.

Worked example

Three written networks and a trained one

For each network: how many of its weights aren't zero, how many units each layer uses, its agreement with the rule on all 1,944 observations, and balls caught out of 200 and out of another 1,000.

import paddle
import wsp

STATES = [(bx, by, dx, px) for bx in range(9) for by in range(8) for dx in (-1, 0, 1)
          for px in range(9)]

def catch(bx, by, dx, px):
    target = paddle.landing_column(bx, by, dx)
    return 0 if target < px else 2 if target > px else 1

print(len(STATES), "possible observations")
for name in ["paddle-catch", "written-catch-1", "written-catch-2", "written-catch-3"]:
    layers, info = wsp.load_network(name)
    policy = wsp.network_policy(layers)
    nonzero = sum(int((W != 0).sum() + (b != 0).sum()) for W, b in layers)
    units = [int(((abs(W).sum(axis=1) + abs(b)) > 0).sum()) for W, b in layers]
    agree = sum(policy(list(s)) == catch(*s) for s in STATES)
    print(f"{name:16s} nonzero {nonzero:4d}  units {units}  agrees {agree}  caught "
          f"{paddle.play(policy)} and {paddle.play(policy, 1000, 123)}")
In the lesson you can change this and run it.

All four agree with the rule on all 1,944 observations and catch every ball. The difference is in how. The trained network uses all 1,315 weights and every unit. The written ones use 31, 46 and 26 weights, and between 4 and 9 of the 32 first-layer units. Your own network from Writing weights by hand used 29 nonzero weights in a 4-4-4-3 shape. The third session's network has the same design as yours: the same four first-layer units, found again from the source code.

Quick check
The second session's network is run on one observation: ball at column 3, row 2, moving right, paddle at column 6. The scores come out in the order left, stay, right. What does this print?
import wsp

layers, info = wsp.load_network("written-catch-2")
x = [3.0, 2.0, 1.0, 6.0]
for i, (W, b) in enumerate(layers):
    x = W @ x + b
    if i < 2:
        x = x.clip(min=0)
print([float(v) for v in x])
  1. [1.0, 0.0, 0.5]
  2. [1.0, 0.5, 0.0]
  3. [0.5, 1.0, 0.0]
  4. [0.0, 0.5, 1.0]
Show the answer

[0.0, 0.5, 1.0]

The ball has 6 steps left. Moving right from column 3 it would reach column 9, one past the wall, so it bounces and lands in column 7. The paddle is at 6, one column to the left of that. So right scores 1, the size of the gap, left scores 0, and stay keeps its constant 0.5. The paddle moves right.

Try it now

The first layer the editor wrote in session 1: six rows of four weights. The other 26 rows are zero and aren't shown. Move the column box to 2 to pick out ball_dx.

Quick check
Unit 0 of the first layer in session 1's network, on a ball in row 3 with a paddle at column 6. Only the ball's direction changes between the three numbers. What does this print?
import wsp

layers, info = wsp.load_network("written-catch-1")
W, b = layers[0]
print([max(0.0, float(W[0] @ [4.0, 3.0, dx, 6.0] + b[0])) for dx in (-1, 0, 1)])
  1. [5.0, 5.0, 5.0]
  2. [0.0, 5.0, 5.0]
  3. [-15.0, -5.0, 5.0]
  4. [0.0, 0.0, 5.0]
Show the answer

[0.0, 0.0, 5.0]

The unit computes -ball_y + 10 × ball_dx - 2. For a ball moving right that is -3 + 10 - 2 = 5, the steps left from row 3. For the other two directions the sum is negative, and ReLU makes it 0. The unit holds the steps left, for balls moving right only.

In your own words
In your own words: the trained network and the written ones all agree with the rule on 1,944 observations. What is different about them?

Write a sentence or two, then compare it with a good answer in the lesson.

The replay: test the design, then change the specification

A score tells you that the network works. It doesn't tell you that the editor understood why. The reply has a second part, DESIGN, where the editor says what each unit computes. For session 1 it reads, in part:

Unit 0 is r when dx=+1, else 0. Unit 1 is r when dx=-1, else 0. Unit 2 is relu(ball_x - ball_y) when dx=+1, else 0; this is the right-wall overshoot. Unit 3 is relu(8 - ball_x - ball_y) when dx=-1, else 0; this is the left-wall overshoot. Unit 4 is ball_x. Unit 5 is paddle_x.

Here r is the number of steps left, 8 - ball_y. Each sentence is a claim about numbers, so it can be tested the way the network was: on every observation. Compute the unit's value from the weights, compute what the editor said it should be, and count the observations where the two differ.

Worked example

Six claims, tested on every observation

Turn each sentence of the design into a small function. Then compare it with the unit's real value, after ReLU, on all 1,944 observations.

import numpy as np
import wsp

STATES = [(bx, by, dx, px) for bx in range(9) for by in range(8) for dx in (-1, 0, 1)
          for px in range(9)]
layers, info = wsp.load_network("written-catch-1")
W, b = layers[0]

def unit(i, obs):
    return max(0.0, float(W[i] @ np.array(obs, dtype=np.float32) + b[i]))

claims = {
    0: lambda bx, by, dx, px: 8 - by if dx == 1 else 0,
    1: lambda bx, by, dx, px: 8 - by if dx == -1 else 0,
    2: lambda bx, by, dx, px: max(0, bx - by) if dx == 1 else 0,
    3: lambda bx, by, dx, px: max(0, 8 - bx - by) if dx == -1 else 0,
    4: lambda bx, by, dx, px: bx,
    5: lambda bx, by, dx, px: px,
}
for i, claim in claims.items():
    wrong = sum(unit(i, s) != claim(*s) for s in STATES)
    print(f"unit {i}: differs from the claim on {wrong} of {len(STATES)}")
In the lesson you can change this and run it.

Every claim holds on every observation. The editor's account of its own first layer is exact, and you checked it without trusting a word of it. That is a stronger position than the one you have with a trained network, where nobody wrote down what unit 17 is for. It is also only possible because the editor was asked to explain itself in terms that can be run.

Drill

Find where a claim fails

Write wrong_positions(layers, i, claim). It returns the observations, in the order of STATES, where first-layer unit i (after ReLU) differs from claim(bx, by, dx, px). The starter tries it on session 2's network with a claim that is false: its design says unit 0 is ball_x always, not only for balls moving right.

import numpy as np
import wsp

STATES = [(bx, by, dx, px) for bx in range(9) for by in range(8) for dx in (-1, 0, 1)
          for px in range(9)]

def wrong_positions(layers, i, claim):
    return []

layers, info = wsp.load_network("written-catch-2")
false_claim = lambda bx, by, dx, px: bx if dx == 1 else 0
misses = wrong_positions(layers, 0, false_claim)
print(len(misses), "observations; the first is", misses[:1])
In the lesson, your code is checked as you write it.
In your own words
In your own words: the network already agreed with the rule on every observation. What did testing the design claims add?

Write a sentence or two, then compare it with a good answer in the lesson.

A specification nobody trained for

"Catch the ball" has a trained network to compare with. The second specification has none. It asks for a paddle that catches the ball while moving as late as it can:

  • if the paddle is steps_left or more columns away from the landing column, step toward it;
  • otherwise stay.

Such a paddle waits, then moves on every remaining step and arrives exactly as the ball lands. To train a network for this you would first write the rule as code, label all 1,944 observations with it and then run gradient descent. The editor skips the last two steps. It got the rule in words, in the same prompt as before, and wrote the weights. Two sessions were recorded.

Worked example

The late paddle, scored

Write the second rule as code. Score both written networks against it, with a catch network beside them. The last number counts the moves made while the ball is still in the top four rows, over 200 balls.

import paddle
import wsp

STATES = [(bx, by, dx, px) for bx in range(9) for by in range(8) for dx in (-1, 0, 1)
          for px in range(9)]

def lazy(bx, by, dx, px):
    target = paddle.landing_column(bx, by, dx)
    if abs(target - px) >= 8 - by:
        return 0 if target < px else 2
    return 1

def early_moves(policy):
    count = 0
    for start in paddle.episodes():
        game = paddle.Game(*start)
        while not game.done:
            action = policy(game.observe())
            count += action != 1 and game.ball_y < 4
            game.step(action)
    return count

for name in ["written-lazy-1", "written-lazy-2", "written-catch-1"]:
    layers, info = wsp.load_network(name)
    policy = wsp.network_policy(layers)
    agree = sum(policy(list(s)) == lazy(*s) for s in STATES)
    print(f"{name:16s} agrees with the late rule {agree:4d}  caught "
          f"{paddle.play(policy)}  early moves {early_moves(policy)}")
In the lesson you can change this and run it.

Both sessions wrote the late rule exactly: 1,944 of 1,944, and every ball caught. The catch network catches every ball too, yet it agrees with the late rule on only 912 observations, because it starts moving at once. In the top four rows it makes 481 moves over the 200 balls, and the late networks make 97. Balls caught can't tell these policies apart. Agreement with the rule can.

Quick check
From Writing weights by hand: a layer can't multiply two of its inputs. How does a ReLU unit still compute ball_dx × steps?
  1. It can't: the product has to be worked out by the game and passed in as an input
  2. It squares the sum of the two inputs, which holds their product as one of its terms
  3. It adds a large penalty when the direction is wrong, and ReLU turns that case into 0
  4. It stores the answer for each pair of inputs as a separate weight and looks it up
Show the answer

It adds a large penalty when the direction is wrong, and ReLU turns that case into 0

ball_dx is only ever -1, 0 or 1, so multiplying by it means choosing between plus steps, nothing and minus steps. One unit passes steps through for right-moving balls and another for left-moving ones. Every network in this lesson, written by a person or an LLM, uses that switch.

Edit a written network by hand

You now have a catch network whose design you have verified, and a rule that differs from "catch" in one way: wait until the gap equals the steps left. That is a small change to a program, so it should be a small change to the weights.

In session 1's network the second layer has two units. Unit 0 is relu(landing - paddle_x) and unit 1 is the same with the sign flipped. A move wins when its unit reaches 1. For the late rule, unit 0 should instead reach 1 only when landing - paddle_x is at least steps_left. That is relu(landing - paddle_x - steps_left + 1).

So the second layer needs steps_left for every ball. The first layer doesn't have it: units 0 and 1 hold the steps left only for diagonal balls. But 26 first-layer units are unused. Give one of them the job.

Drill

Turn the catch network into the late one

Finish make_late(state). It gets session 1's network as a state (named arrays) and returns an edited copy that follows the late rule. Use first-layer unit 6 for steps_left = 8 - ball_y. Then change second-layer units 0 and 1 so each computes its old value minus steps_left plus 1. Six entries change.

import paddle
import wsp

STATES = [(bx, by, dx, px) for bx in range(9) for by in range(8) for dx in (-1, 0, 1)
          for px in range(9)]

def lazy(bx, by, dx, px):
    target = paddle.landing_column(bx, by, dx)
    if abs(target - px) >= 8 - by:
        return 0 if target < px else 2
    return 1

def make_late(state):
    new = {name: tensor.copy() for name, tensor in state.items()}
    # first layer: new["0.weight"][unit, input] and new["0.bias"][unit]
    # second layer: new["1.weight"][unit, first-layer unit] and new["1.bias"][unit]
    return new

layers, info = wsp.load_network("written-catch-1")
state = wsp.network_state(layers)
edited = make_late(state)
policy = wsp.network_policy(wsp.network_layers(edited))
print("agrees with the late rule on", sum(policy(list(s)) == lazy(*s) for s in STATES),
      "of", len(STATES))
print("entries changed:", wsp.make_patch(state, edited)["changed"])
In the lesson, your code is checked as you write it.
In your own words
In your own words: why was this edit six weights, when retraining the network for the new rule would have moved all 1,315?

Write a sentence or two, then compare it with a good answer in the lesson.

The grading: what breaks written weights, and what they show

In Writing weights by hand, small random noise on your 55 weights cost catches, and the text said trained networks tend to be sturdier. Now there is a trained network of the same shape to test that against.

The next cell adds noise to every one of the 1,315 entries, zeros included, at three sizes and with three seeds each. Noise on a zero matters here: it wakes up a unit the editor left switched off.

Worked example

Noise on every weight

The same random noise is added to the trained network and to each written one. Each list holds the balls caught, of 200, for three seeds.

import matplotlib.pyplot as plt
import numpy as np
import paddle
import wsp

def with_noise(layers, size, seed):
    rng = np.random.default_rng(seed)
    return [(W + rng.normal(0, size, W.shape), b + rng.normal(0, size, b.shape))
            for W, b in layers]

sizes = (0.01, 0.03, 0.1)
for name in ["paddle-catch", "written-catch-1", "written-catch-2", "written-catch-3"]:
    layers, info = wsp.load_network(name)
    print(f"{name:16s}", end=" ")
    means = []
    for size in sizes:
        scores = [paddle.play(wsp.network_policy(with_noise(layers, size, seed)))
                  for seed in range(3)]
        print(size, scores, end="  ")
        means.append(sum(scores) / len(scores))
    print()
    plt.plot(sizes, means, marker="o", label=name)
plt.xscale("log")
plt.xlabel("noise added to every weight")
plt.ylabel("balls caught of 200 (mean of 3)")
plt.title("Written and trained networks under noise")
plt.legend()
plt.show()
In the lesson you can change this and run it.

No clear winner. At 0.01 two written networks still catch 198 to 200 balls, and the trained one drops to 185 on one seed. At 0.03 every network loses balls, and at 0.1 none catches half. The outlier is session 2's network, which loses up to 83 balls at the smallest noise. Its second layer multiplies one unit by 8, and that weight multiplies the noise as well. So how sturdy a written network is depends on its design. The three sessions differ from one another more than writing differs from training.

Quick check
Session 2's network was the most fragile under noise, though it scored a perfect 1,944 of 1,944. What does that tell a reviewer?
  1. That the noise was too large, since a fair test would never lower a perfect score
  2. That agreement with the specification says nothing about how the network behaves off it
  3. That written networks are always more fragile than trained ones and shouldn't be used
  4. That the network was scored wrongly, since a perfect network can't lose balls to noise
Show the answer

That agreement with the specification says nothing about how the network behaves off it

The specification covered exact inputs and exact weights, and on those the network is perfect. Sturdiness wasn't in the specification, so the editor had no reason to design for it. If you need a property, put it in the specification and measure it.

What five exact networks show

Five sessions, two specifications, five networks that match their rule on every observation. Before you carry that anywhere, look at what made it possible.

The rule could be written down. Where the ball lands is a formula, and you found it yourself. The editor turned a short program into weights, which is what a compiler does. A critic of the post would say a skilled person can do the same, and here one did: you. What the LLM added was speed, about a minute against most of a lesson.

The network was tiny, and so was the test. With 1,944 inputs you could check everything. No language model has an input space you can list.

One model, one day. All five sessions used the same editor on the same date. They say nothing about other editors, and five runs can't measure how often one fails.

The post reports larger results, and they are its authors' numbers, not ones this lesson checked.

Quick check
You checked all five written networks completely. Which fact about this task made a complete check possible?
  1. The network has only 1,315 weights, so each of its weights can be inspected one by one
  2. The game is played by a fixed seed, so the same 200 balls come up on every run
  3. The editor explained its design, and a design that is explained can't hold mistakes
  4. The network has four small whole-number inputs, so every possible observation can be listed
Show the answer

The network has four small whole-number inputs, so every possible observation can be listed

A complete check runs the network on every input it could ever see. Here that is 9 × 8 × 3 × 9 = 1,944 observations. A language model's inputs are all possible texts, which nobody can list, so its tests are always samples.

What the post reports at larger scale

In the post's fifth experiment the networks were 600-32-32-6 and the games were three from MinAtar. The authors report that the written networks beat networks trained with PPO for 100 million steps on all three. Against 500 million steps the picture is mixed: far ahead on Breakout (517.4 against 25.3), level on Asterix (53.2 against 55.2) and behind on Seaquest (53.1 against 88.9).

For Go, with about 1.5 million weights, the editor no longer wrote text. It wrote a program that builds the weights, and it did better as it was given more forward passes to see its own games.

So the published pattern matches what you saw. Writing works where the task has rules someone can state, and leans on feedback as the task grows. For a task with no stated rule, such as "answer like a helpful assistant," nobody has shown weights written from scratch.

Go deeper Why the post's Breakout score is so far ahead

The authors explain it by design: the first layer works out where the ball will land and the second moves the paddle there. That is the network you just read, for a larger court. A trained policy has to discover the landing formula from reward alone, and 500 million steps of PPO hadn't found it. The gap shows how hard that formula is to find by trial. On Seaquest, where no single formula wins, training was ahead.

Where it shows up in your week

When a model writes something that must match a specification, such as a parser, a policy or a set of weights, you test it against the specification on every input you can list, you test the claims it makes about itself, and you test it under a small change. Agreement alone says nothing about robustness.

In your own words
In your own words: a colleague says "LLMs can now write neural networks without training." What would you add?

Write a sentence or two, then compare it with a good answer in the lesson.

Goes deeper in

Practice what you just read

  • 5 code examples you can edit and run
  • a practice exercise that checks your code
  • a 5-question quiz
  • 9 review cards for spaced repetition

Try a quiz question

A written network catches 200 of 200 sample balls. Why score it on all 1,944 observations as well?

  1. A sample can miss the observations where the network breaks its rule
  2. Catching balls is a matter of luck, so the sample score means nothing
  3. The 1,944 observations are harder than the ones met in a real game
  4. A network has to be scored twice before its weights can be published

Answer it at the end of the lesson to see if you're right, and why.

Before this lesson

Keep learning

Where this lesson leads.

575 lessons in thirteen phases. 176 are free without an account, and a free account opens the rest. See every lesson.

Phase 6 · about 35 hours of study

Modern LLM engineering

Post-training, fine-tuning, serving and safety 48 lessons 2 free
  1. Base models vs assistants FreeWhat post-training adds to a pretrained model.30 min
  2. Supervised fine-tuningTeach behavior from example conversations.40 min
  3. Instruction and preference dataThe datasets that shape assistant behavior.30 min
  4. RLHFReward models and reinforcement learning from human feedback.40 min
  5. DPO and direct preference methodsPreference training without a separate reward model.40 min
  6. Constitutional AITraining with AI feedback guided by written principles.35 min
  7. Training reasoning modelsReinforcement learning on verifiable problems.40 min
  8. Prompting vs RAG vs fine-tuningPick the cheapest approach that works.30 min
  9. LoRAFine-tune a big model by training small low-rank updates.40 min
  10. Fine-tune an open modelRun a real fine-tune and evaluate the result.45 min
  11. DistillationTrain a small model to imitate a large one.35 min
  12. Tuning an embedding model for retrievalWhen search fails on your own vocabulary: train an adapter with a contrastive loss on question and passage pairs, test on held-out passages, use batch size and hard negatives, and measure what tuning costs elsewhere.45 min
  13. QuantizationStore weights in fewer bits to run models cheaply.35 min
  14. Quantization in practice: zero points, calibration and formatsScale and zero point, per-channel scales, calibration sets, error on a layer's output, quantization-aware training and the file formats you meet at work.35 min
  15. Serving modelsBatching, throughput and latency in production.35 min
  16. One request through a serving stack: latency and costFollow one request through queue, prefill, a prefix cache, decode and the stream, and price a token from a GPU's hourly cost.35 min
  17. Speculative decodingDraft with a small model, verify with a big one.35 min
  18. Running open models locallyDownload, quantize and run a model on your own machine.30 min
  19. Will it fit? Memory for serving and fine-tuningWeights, KV cache, runtime and adapter states added up for a model card and a GPU, with a calculator.30 min
  20. InterpretabilityLooking inside models to see what they compute.40 min
  21. Confident and wrongWhy a model answers with the same assurance whether it is right, wrong or asked about something that does not exist.30 min
  22. Why it can't spell or count lettersThe model sees token IDs, and an embedding learned from usage carries nothing about spelling.25 min
  23. Copying from the promptIn-context learning as a two-head copying circuit: the base tiny GPT cannot copy a repeated pattern, a model trained on repeats grows an induction head and can.30 min
  24. Why temperature changes the answerThe same prompt at 0, 0.7 and 1.5: deterministic loops, plausible variety, broken grammar, measured on a real model.25 min
  25. Why long prompts diluteAttention is a budget shared by every token, so a fact far from the question loses its share; measured, with the cliff beyond the trained window.25 min
  26. Why it never says "I don't know"A chat model answers every request because its fine-tuning data always did; preference training adds flattery; how to build abstention in.25 min
  27. Benchmarks and their limitsWhat leaderboards measure, and what they miss.38 min
  28. Alignment and safetyMaking capable models behave as intended.35 min
  29. Evaluating agentsMeasure multi-step, tool-using behavior.35 min
  30. Reading AI papersGet the idea out of a research paper in 30 minutes.30 min
  31. Capstone: a tuned, evaluated assistantFine-tune, evaluate and deploy a specialist assistant.45 min
  32. Weight-space programming: what it is, and what has been shown Free OptionalA language model reads and writes another network's weights with no training loop: the published experiments, their limits and the criticism, and one edit you make by hand.30 min
  33. Anatomy of a checkpoint OptionalA model file is a set of named tensors: count them, find the ones that write to the residual stream, and save one to bytes and back.35 min
  34. Reading weights without running the model OptionalSingular values, and an attention head's token-to-token circuit, read from the matrices alone and then checked against a measurement.35 min
  35. Writing weights by hand OptionalSet a small network's 55 weights so it catches every ball in a paddle game, the way a program would be written, and meet compiled transformers.40 min
  36. Edits are deltas: task vectors, patches and hashes OptionalSubtract a base from a fine-tune and the edit becomes an object: scale it, take it away, ship it as a patch, and prove by hash what was applied.35 min
  37. Merging models OptionalAverage two fine-tunes, add their task vectors, try SLERP, TIES and DARE, measure the interference, and compare with a model trained on both tasks.40 min
  38. Low-rank edits OptionalCut a fine-tune's change down to its largest directions with the SVD: how few a behavior needs, and how to pull an adapter out of a delta.35 min
  39. Directions: steer the run or edit the weights OptionalFind a behavior's direction in the residual stream, add it at run time, project it out of the weights, and see the effect change from input to input.40 min
  40. Editing one fact OptionalA rank-one change to one matrix rewrites what a product is made from; then reliability, generalization, locality, and what breaks after many edits.40 min
  41. The editor's harness OptionalWhat an LLM that edits weights is given and what it is denied: weights as text and their cost in tokens, a sandbox, a budget of forward passes, and rules the harness enforces.40 min
  42. An LLM reads a network's weights OptionalSix recorded sessions: an LLM is handed 1,315 weights as text and asked what the network was trained to do. Score each answer against the network, and study one it got wrong.40 min
  43. An LLM fixes a model's weights OptionalFind a tiny GPT's loop by hand, then replay recorded sessions in which an LLM hunts for it with ten forward passes, and measure every fix on held-out questions.45 min
  44. An LLM writes a network's weights OptionalFive recorded sessions: an LLM writes every weight of a paddle network from the game's source and one sentence. Score each against its specification, test its design claims, and edit one by hand.40 min
  45. Did it work? Testing an edit OptionalReview a patch for a tiny GPT that can't stop talking: held-out questions, a paired comparison, a same-size control, and what the edit cost.40 min
  46. The published edit, on a real model OptionalRead the edit published for an open model of 870 million weights, apply and verify one in the same form, see what 16-bit and 4-bit storage do to a fix, and run the real thing from a notebook.45 min
  47. Safety, licenses and provenance OptionalBefore a custom model is released: trace its lineage by hash, work out what its parents' licenses pass on, see what an edit can do to a safeguard, and write its card.40 min
  48. Capstone: ship a custom model OptionalSet release gates, then diagnose, edit, measure and document a tiny GPT with a fault no lesson has fixed, and write the record a stranger could check.45 min
Part of a free course 17 lessons

Creating Custom LLMs: Editing Open Weights Using Weight-Space Programming

Read a model's weights, change them by hand and judge an edit someone else made: deltas, merging, low-rank edits, directions and fact edits on a tiny GPT in your browser. Then replay recorded sessions of an LLM reading, fixing and writing weights, check the published edit of a real open model, and ship a model of your own with its patch, verdict and card. Every number is printed by the lesson's own code, and the method's limits are taught with it.

Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write