Index Ai Academy
Phase 6 · Weight-space programming

Writing weights by hand

Set a small network's 55 weights so it catches every ball in a paddle game, the way a program would be written, and meet compiled transformers.

About 40 minutes · Optional lesson: deeper material that no phase or certificate requires · A free account opens the whole lesson.
Already have an account? Sign in to open this lesson

What it covers9

Why it matters

The reading lessons took weights that training made and asked what they do. This lesson goes the other way. You decide what a network should compute, then choose every weight so that it does. Nothing is trained.

It is the "write" third of weight-space programming, and it is older than the name: researchers have compiled small programs into transformer weights for years. Doing it once by hand changes how you see a weight matrix. A layer stops being a blur of numbers and becomes a line of a program.

What you'll be able to do

  • Write a one-layer policy's weights by hand and score it on a fixed set of games
  • Use a ReLU unit as a switch, and as a detector for "past the wall"
  • Explain what each layer of a hand-written three-layer network computes, and finish one yourself
  • Say what a compiled transformer is, and how hand-written weights differ from trained ones
Quick check
A small network has to catch a falling ball that bounces off the walls. Training one takes thousands of games. A guess: if a person writes every weight by hand instead, how many of 200 balls can it catch?
  1. None: weights only work when a training loop finds them
  2. All 200, but only with millions of weights
  3. About 60: hand-written weights can only follow simple rules
  4. All 200, with 55 weights
Show the answer

All 200, with 55 weights

You will write that network in this lesson. A weight doesn't know how it was chosen. If you can describe the computation, you can set the numbers that perform it.

The idea: a rule written as weights

This lesson rests on Multi-layer networks: a layer is a matrix times its input plus a bias, and ReLU lets layers do more than add.

The game, and a policy of fifteen numbers

The course's paddle module is a small game. The court is 9 columns wide and 9 rows tall. A ball starts in the top row and falls one row per step. It goes straight down or diagonally, and it bounces off the side walls. The paddle sits in the bottom row and moves one column per step. After 8 steps the ball lands, and it is caught if the paddle is under it.

A policy decides the paddle's move. It sees four numbers, the observation: [ball_x, ball_y, ball_dx, paddle_x]. ball_dx is the ball's sideways speed: −1, 0 or +1 columns per step. The policy answers 0 for left, 1 for stay or 2 for right. paddle.play(policy) runs it on 200 fixed balls and counts the catches, so every policy faces the same test.

Start with plain rules, before any network.

Worked example

Two rules, scored

A policy that never moves, and one that moves toward the ball's current column. The code also draws one starting position and counts which kind of ball the second rule misses.

Setup (runs before every cell) · 14 lines

The lesson runs this once, before its first cell, and every cell, drill and example starts with what it defines. A cell that changes a setup object changes it for the cells after it, until the page reloads.

import random
import paddle

L1 = ([[1, 0, 0, 0], [0, 0, 0, 1], [0, -1, 8, 0], [0, -1, -8, 0]], [0, 0, 0, 0])
L2 = ([[1, 0, 1, -1], [1, 0, 1, -1], [-1, 0, -1, 1], [0, 1, 0, 0]], [8, -8, 0, 0])
L3 = ([[-1, 2, -2, 1], [0, 0, 0, 0], [1, -2, 2, -1]], [8, 0.5, -8])
net = [L1, L2, L3]

def with_noise(net, size, seed):
    rng = random.Random(seed)
    def jitter(v):
        return v + rng.gauss(0, size)
    return [([[jitter(w) for w in row] for row in W], [jitter(b) for b in bias])
            for W, bias in net]
import paddle

stay = lambda obs: paddle.STAY
def follow(obs):
    if obs[0] < obs[3]:
        return paddle.LEFT
    return paddle.RIGHT if obs[0] > obs[3] else paddle.STAY

print(paddle.render(paddle.Game(7, 1, 2)))
print("never move:", paddle.play(stay), "of 200")
print("follow the ball:", paddle.play(follow), "of 200")

missed = [start for start in paddle.episodes() if not paddle.run(follow, start).caught]
straight = sum(dx == 0 for _, dx, _ in missed)
print("missed while following:", len(missed))
print("  of those, falling straight down:", straight)
It runs right here in your browser, with nothing to install.

Standing still catches 21 balls by luck. Following the ball catches 62, which is every ball that falls straight down and none of the others: all 138 misses are diagonal balls. The reason is speed. A diagonal ball moves one column per step, and so does the paddle, so chasing it never closes the gap. To catch those, the paddle has to go where the ball will be.

Now write the "follow" rule as a network. One layer is enough: three rows of weights, one per move. Each row's score is its weights times the four inputs, added up, plus a bias. The move with the highest score wins.

"Move left when the ball is to the paddle's left" becomes a row that scores paddle_x − ball_x: weight −1 on the ball's column, +1 on the paddle's column, 0 on the other two inputs. "Right" is the mirror image. "Stay" ignores everything and has a bias of 0.5, so it wins only when the other two score 0.

Quick check
The layer below is the "follow" rule. paddle.forward runs it on one observation and returns the three scores. What does this print?
import paddle

layer = ([[-1, 0, 0, 1], [0, 0, 0, 0], [1, 0, 0, -1]], [0, 0.5, 0])
print(paddle.forward([layer], [2, 0, 0, 5]))
  1. [3.0, 0.5, -3.0]
  2. [-3.0, 0.5, 3.0]
  3. [3.0, 0.0, -3.0]
  4. [7.0, 0.5, -7.0]
Show the answer

[3.0, 0.5, -3.0]

The ball is in column 2 and the paddle in column 5. Left scores −2 + 5 = 3, stay scores its bias 0.5, and right scores 2 − 5 = −3. Left wins, which is toward the ball.

Try it now

The "follow" layer as a picture. Two whole columns are blank: this policy never looks at how high the ball is or which way it is moving.

Drill

Lead the ball by one step

Following aims at where the ball is. Write a one-layer network that aims one step ahead: at ball_x + ball_dx. Fill in the weights of the left and right rows. The left row should score paddle_x − (ball_x + ball_dx) and the right row the opposite. It should catch about twice as many balls as following did.

Uses the setup above, 14 lines.

import paddle

W = [[-1, 0, 0, 1],      # left
     [ 0, 0, 0, 0],      # stay
     [ 1, 0, 0, -1]]     # right
b = [0, 0.5, 0]
lead = [(W, b)]
print(paddle.play(paddle.mlp_policy(lead)), "of 200")
In the lesson, your code is checked as you write it.
In your own words
In your own words: how do you turn the rule "move left when the ball is to the paddle's left" into a row of weights?

Write a sentence or two, then compare it with a good answer in the lesson.

In code: switches, folds and three layers

A ReLU unit is a switch

To catch diagonal balls the paddle needs the landing column. Ignore the walls for a moment. A ball in column ball_x with steps steps to fall ends up at:

  • ball_x + steps if it moves right (ball_dx is +1)
  • ball_x − steps if it moves left (−1)
  • ball_x if it falls straight down (0)

That is ball_x + ball_dx × steps, and here is the difficulty. A layer can add its inputs with weights. It cannot multiply two inputs together. The way out is the activation function. ReLU keeps a positive number and turns a negative one into 0, and that makes it a switch. Take

right_steps = relu(steps + 8 × (ball_dx − 1))

When ball_dx is +1 the bracket is 0 and the unit passes steps through. When it is 0 or −1, the unit subtracts 8 or 16. steps is never more than 8, so the result is zero or negative, and ReLU makes it 0. One unit computes "steps left, if the ball is moving right." A mirror-image unit handles moving left.

Worked example

Two switch units and where the ball is heading

Compute both switch units for a ball 5 steps from landing, for each direction. Then combine them: column plus the right-moving steps minus the left-moving steps.

Uses the setup above, 14 lines.

relu = lambda v: max(0, v)

ball_x, steps = 3, 5
for ball_dx in (-1, 0, 1):
    right_steps = relu(steps + 8 * (ball_dx - 1))
    left_steps = relu(steps + 8 * (-ball_dx - 1))
    position = ball_x + right_steps - left_steps
    state = f"right_steps {right_steps}, left_steps {left_steps}"
    print(f"ball_dx {ball_dx:+d}: {state}, heading for column {position}")
In the lesson you can change this and run it.

Exactly one switch is on for a diagonal ball and neither for a straight one, so the last column of the output is ball_x + ball_dx × steps: −2, 3 and 8. No multiplication happened. Two units with a ReLU did the job of one multiply, because one of the two numbers could only be −1, 0 or +1. Notice the −2: with no walls, the left-moving ball would leave the court. That is the next thing to fix.

Quick check
What does this print?
relu = lambda v: max(0, v)
steps = 6
print(relu(steps + 8 * (1 - 1)), relu(steps + 8 * (0 - 1)), relu(steps + 8 * (-1 - 1)))
  1. 6 -2 -10
  2. 0 0 6
  3. 6 0 0
  4. 6 6 6
Show the answer

6 0 0

With ball_dx = 1 the unit passes the 6 steps through. With 0 it computes 6 − 8 = −2, and with −1 it computes 6 − 16 = −10. ReLU turns both negatives into 0.

The second trick handles the walls. A ball that would end up past the right wall, say at column 11 on a court whose last column is 8, has bounced. It lands as far inside the wall as it would have gone outside: 8 − 3 = 5. In general a position past 8 lands at 16 − position, and a position below 0 lands at −position.

ReLU can measure "how far past the wall": relu(position − 8) is 0 inside the court and counts the columns beyond the right wall. Subtract that overshoot twice and the ball is folded back:

landing = position − 2 × relu(position − 8) + 2 × relu(−position)

With 8 steps and a court 9 wide, a ball bounces once at most, so one fold on each side is enough.

Drill

Fold a position back into the court

Write landing(position) using only +, −, × and relu: no if, no abs, no min or max of your own. position is where a ball would end up with no walls, from −8 to 16. Return the column where it really lands.

Uses the setup above, 14 lines.

relu = lambda v: max(0, v)

def landing(position):
    return position

print([landing(p) for p in (-8, -2, 0, 5, 8, 11, 16)])
In the lesson, your code is checked as you write it.
In your own words
In your own words: a layer can't multiply two of its inputs. How did a ReLU unit get around that for ball_dx × steps?

Write a sentence or two, then compare it with a good answer in the lesson.

Fifty-five weights that never miss

Now put the pieces in order. Each layer hands four numbers to the next.

  1. Layer 1 keeps ball_x and paddle_x and computes the two switch units. The game gives ball_y, and steps is 8 − ball_y, so the right-moving switch relu(steps + 8 × (ball_dx − 1)) simplifies to relu(8 × ball_dx − ball_y).
  2. Layer 2 forms position = ball_x + right_steps − left_steps and outputs four things: position + 8 (shifted so it is never negative and survives the ReLU), the overshoot past the right wall, the overshoot past the left wall, and paddle_x again.
  3. Layer 3 has no ReLU. It folds the position into the landing column, subtracts paddle_x, and scores left, stay and right from that difference, exactly as the one-layer policy did.

Each row below is one unit, and its comment says what it computes.

Worked example

The whole network, and its score

Three layers written as lists of numbers, scored on the 200 fixed balls and on a second set. The last lines trace one bouncing ball through the layers.

Uses the setup above, 14 lines.

import paddle

L1 = ([[1,  0,  0, 0],      # ball_x
       [0,  0,  0, 1],      # paddle_x
       [0, -1,  8, 0],      # steps left, only if moving right: relu(8*dx - ball_y)
       [0, -1, -8, 0]],     # steps left, only if moving left
      [0, 0, 0, 0])
L2 = ([[ 1, 0,  1, -1],     # position + 8, where position = ball_x + right - left
       [ 1, 0,  1, -1],     # overshoot past the right wall: relu(position - 8)
       [-1, 0, -1,  1],     # overshoot past the left wall:  relu(-position)
       [ 0, 1,  0,  0]],    # paddle_x
      [8, -8, 0, 0])
L3 = ([[-1,  2, -2,  1],    # left:  paddle_x - landing
       [ 0,  0,  0,  0],    # stay
       [ 1, -2,  2, -1]],   # right: landing - paddle_x
      [8, 0.5, -8])
net = [L1, L2, L3]

print("weights:", sum(len(W) * len(W[0]) + len(b) for W, b in net))
caught = paddle.play(paddle.mlp_policy(net))
other = paddle.play(paddle.mlp_policy(net), seed=5)
print("caught:", caught, "of 200; on another 200:", other)

obs = [7, 0, 1, 2]                      # ball at column 7 moving right, paddle at 2
print("after layer 1:", [max(0.0, v) for v in paddle.forward([L1], obs)])
print("after layer 2:", [max(0.0, v) for v in paddle.forward([L1, L2], obs)])
lands = paddle.landing_column(7, 0, 1)
print("scores:", paddle.forward(net, obs), "- the game says column", lands)
In the lesson you can change this and run it.

Fifty-five numbers, and the network catches all 200 balls, then all 200 of a set it has never seen. The trace shows the program running. Layer 1 passes on the ball's column 7, the paddle's column 2 and 8 right-moving steps. Layer 2 turns those into 23 (a position of 15, shifted by 8), an overshoot of 7 past the right wall, 0 past the left, and the paddle's 2. Layer 3 computes 15 − 2 × 7 = 1 for the landing column and scores left at 2 − 1 = 1. The paddle moves left, toward column 1, where the game says the ball will land.

Quick check
In the three-layer network, which layer works out how far past a wall the ball would go?
  1. Layer 2: two of its units are relu(position − 8) and relu(−position)
  2. Layer 3: it has the largest biases
  3. Layer 1: its switch units detect the walls
  4. None of them: the game tells the network where the walls are
Show the answer

Layer 2: two of its units are relu(position − 8) and relu(−position)

Layer 1 only decides which direction's steps to pass on. Layer 2 forms the straight-line position and measures the overshoot on each side. Layer 3 uses those overshoots to fold the position back.

Quick check
From the activation functions lesson: what would happen to this network if every ReLU were removed?
  1. It would still work, but need twice as many weights
  2. The three layers would collapse into one linear layer, which can't compute the landing column
  3. Nothing: ReLU only speeds up training
  4. It would catch more balls, because no information would be thrown away between layers
Show the answer

The three layers would collapse into one linear layer, which can't compute the landing column

Layers with nothing between them multiply out to a single matrix. One linear layer can't switch on ball_dx or fold at a wall. Both tricks in this lesson are the ReLU doing something a weighted sum can't.

In your own words
In your own words: what does each of the three layers hand to the next?

Write a sentence or two, then compare it with a good answer in the lesson.

In production: compiled transformers, and why hand-written weights are brittle

What you did by hand has been automated for transformers. RASP is a small programming language whose operations match what attention heads and feed-forward layers can do. Tracr, released in 2023, is a compiler that takes a RASP program and produces the actual weights of a transformer that runs it. Researchers use those models as a laboratory: when you know exactly what the weights compute, you can test whether an interpretability method finds it.

The post that named weight-space programming reports the same move made by a language model. Given only the rules of three small games and a network of zeros, the model wrote every weight in one reply. Commentary on the post describes its best network as working like a program: one layer finds where the ball will land, and the next moves the paddle there. That is the design you just built.

Hand-written weights are not like trained ones. Yours are whole numbers, mostly zeros, and every row has a comment. Trained weights for the same game would be dense and unlabeled. There is a price for the neatness, which the last cell measures.

Worked example

Exact, readable, and brittle

Add a little random noise to every weight of the hand-written network and score it again, for three noise sizes and three random seeds each. The network and the noise helper are in the setup.

Uses the setup above, 14 lines.

for size in (0.01, 0.02, 0.05):
    runs = [with_noise(net, size, seed) for seed in range(3)]
    caught = [paddle.play(paddle.mlp_policy(n)) for n in runs]
    print(f"noise {size}:", caught, "of 200")
In the lesson you can change this and run it.

Noise of 0.01, a hundredth of the smallest weight, already costs up to 23 catches. At 0.05 the network catches half or fewer. The program depends on exact arithmetic: "stay" wins by a margin of 0.5 over a difference of whole columns, and an error of a few hundredths, multiplied by positions as large as 24, eats that margin. Trained networks tend to settle where small changes matter less. Hand-written and compiled weights are a way to know what a network computes. They are not a recipe for sturdy ones.

Quick check
Why did such small noise break the hand-written network?
  1. Hand-written weights are always wrong for some inputs
  2. Its decisions rest on exact whole-number arithmetic with a margin of only 0.5
  3. Noise changed the network's shape, so the layers no longer fit
  4. The game's balls change when the weights change
Show the answer

Its decisions rest on exact whole-number arithmetic with a margin of only 0.5

The network compares a landing column with the paddle's column, and "stay" wins only when they differ by exactly 0. A weight that is off by 0.02 turns a position of 23 into 23.46, and the comparison tips the wrong way.

Plot the score against the noise, averaged over three seeds. The setup holds the network.

Uses the setup above, 14 lines.

import matplotlib.pyplot as plt

sizes = [0, 0.01, 0.02, 0.05, 0.1]
mean_caught = []
for size in sizes:
    runs = [with_noise(net, size, seed) for seed in range(3)]
    scores = [paddle.play(paddle.mlp_policy(n)) for n in runs]
    mean_caught.append(sum(scores) / len(scores))
plt.plot(sizes, mean_caught, marker="o")
plt.xlabel("noise added to every weight")
plt.ylabel("balls caught of 200")
plt.title("The hand-written network under noise")
plt.show()

Where it shows up in your week

Compiled and hand-written networks are test fixtures. You use one when you want a model whose behavior you know exactly, to check whether an interpretability tool or an editing method finds what is really there.

In your own words
In your own words: what is a compiled transformer good for, if nobody would ship one?

Write a sentence or two, then compare it with a good answer in the lesson.

Goes deeper in

Practice what you just read

  • 5 code examples you can edit and run
  • a practice exercise that checks your code
  • a 5-question quiz
  • 9 review cards for spaced repetition

Try a quiz question

A one-layer policy's "left" row is [−1, 0, 0, 1] over the inputs [ball_x, ball_y, ball_dx, paddle_x]. What does the row score?

  1. How far the paddle is to the right of the ball
  2. How far the ball has fallen toward the paddle
  3. The ball's sideways speed
  4. The sum of the two columns

Answer it at the end of the lesson to see if you're right, and why.

Before this lesson

Go deeper, if you like

Optional lessons that build on this one. They never block a phase or a certificate.

Keep learning

Where this lesson leads.

575 lessons in thirteen phases. 176 are free without an account, and a free account opens the rest. See every lesson.

Phase 6 · about 35 hours of study

Modern LLM engineering

Post-training, fine-tuning, serving and safety 48 lessons 2 free
  1. Base models vs assistants FreeWhat post-training adds to a pretrained model.30 min
  2. Supervised fine-tuningTeach behavior from example conversations.40 min
  3. Instruction and preference dataThe datasets that shape assistant behavior.30 min
  4. RLHFReward models and reinforcement learning from human feedback.40 min
  5. DPO and direct preference methodsPreference training without a separate reward model.40 min
  6. Constitutional AITraining with AI feedback guided by written principles.35 min
  7. Training reasoning modelsReinforcement learning on verifiable problems.40 min
  8. Prompting vs RAG vs fine-tuningPick the cheapest approach that works.30 min
  9. LoRAFine-tune a big model by training small low-rank updates.40 min
  10. Fine-tune an open modelRun a real fine-tune and evaluate the result.45 min
  11. DistillationTrain a small model to imitate a large one.35 min
  12. Tuning an embedding model for retrievalWhen search fails on your own vocabulary: train an adapter with a contrastive loss on question and passage pairs, test on held-out passages, use batch size and hard negatives, and measure what tuning costs elsewhere.45 min
  13. QuantizationStore weights in fewer bits to run models cheaply.35 min
  14. Quantization in practice: zero points, calibration and formatsScale and zero point, per-channel scales, calibration sets, error on a layer's output, quantization-aware training and the file formats you meet at work.35 min
  15. Serving modelsBatching, throughput and latency in production.35 min
  16. One request through a serving stack: latency and costFollow one request through queue, prefill, a prefix cache, decode and the stream, and price a token from a GPU's hourly cost.35 min
  17. Speculative decodingDraft with a small model, verify with a big one.35 min
  18. Running open models locallyDownload, quantize and run a model on your own machine.30 min
  19. Will it fit? Memory for serving and fine-tuningWeights, KV cache, runtime and adapter states added up for a model card and a GPU, with a calculator.30 min
  20. InterpretabilityLooking inside models to see what they compute.40 min
  21. Confident and wrongWhy a model answers with the same assurance whether it is right, wrong or asked about something that does not exist.30 min
  22. Why it can't spell or count lettersThe model sees token IDs, and an embedding learned from usage carries nothing about spelling.25 min
  23. Copying from the promptIn-context learning as a two-head copying circuit: the base tiny GPT cannot copy a repeated pattern, a model trained on repeats grows an induction head and can.30 min
  24. Why temperature changes the answerThe same prompt at 0, 0.7 and 1.5: deterministic loops, plausible variety, broken grammar, measured on a real model.25 min
  25. Why long prompts diluteAttention is a budget shared by every token, so a fact far from the question loses its share; measured, with the cliff beyond the trained window.25 min
  26. Why it never says "I don't know"A chat model answers every request because its fine-tuning data always did; preference training adds flattery; how to build abstention in.25 min
  27. Benchmarks and their limitsWhat leaderboards measure, and what they miss.38 min
  28. Alignment and safetyMaking capable models behave as intended.35 min
  29. Evaluating agentsMeasure multi-step, tool-using behavior.35 min
  30. Reading AI papersGet the idea out of a research paper in 30 minutes.30 min
  31. Capstone: a tuned, evaluated assistantFine-tune, evaluate and deploy a specialist assistant.45 min
  32. Weight-space programming: what it is, and what has been shown Free OptionalA language model reads and writes another network's weights with no training loop: the published experiments, their limits and the criticism, and one edit you make by hand.30 min
  33. Anatomy of a checkpoint OptionalA model file is a set of named tensors: count them, find the ones that write to the residual stream, and save one to bytes and back.35 min
  34. Reading weights without running the model OptionalSingular values, and an attention head's token-to-token circuit, read from the matrices alone and then checked against a measurement.35 min
  35. Writing weights by hand OptionalSet a small network's 55 weights so it catches every ball in a paddle game, the way a program would be written, and meet compiled transformers.40 min
  36. Edits are deltas: task vectors, patches and hashes OptionalSubtract a base from a fine-tune and the edit becomes an object: scale it, take it away, ship it as a patch, and prove by hash what was applied.35 min
  37. Merging models OptionalAverage two fine-tunes, add their task vectors, try SLERP, TIES and DARE, measure the interference, and compare with a model trained on both tasks.40 min
  38. Low-rank edits OptionalCut a fine-tune's change down to its largest directions with the SVD: how few a behavior needs, and how to pull an adapter out of a delta.35 min
  39. Directions: steer the run or edit the weights OptionalFind a behavior's direction in the residual stream, add it at run time, project it out of the weights, and see the effect change from input to input.40 min
  40. Editing one fact OptionalA rank-one change to one matrix rewrites what a product is made from; then reliability, generalization, locality, and what breaks after many edits.40 min
  41. The editor's harness OptionalWhat an LLM that edits weights is given and what it is denied: weights as text and their cost in tokens, a sandbox, a budget of forward passes, and rules the harness enforces.40 min
  42. An LLM reads a network's weights OptionalSix recorded sessions: an LLM is handed 1,315 weights as text and asked what the network was trained to do. Score each answer against the network, and study one it got wrong.40 min
  43. An LLM fixes a model's weights OptionalFind a tiny GPT's loop by hand, then replay recorded sessions in which an LLM hunts for it with ten forward passes, and measure every fix on held-out questions.45 min
  44. An LLM writes a network's weights OptionalFive recorded sessions: an LLM writes every weight of a paddle network from the game's source and one sentence. Score each against its specification, test its design claims, and edit one by hand.40 min
  45. Did it work? Testing an edit OptionalReview a patch for a tiny GPT that can't stop talking: held-out questions, a paired comparison, a same-size control, and what the edit cost.40 min
  46. The published edit, on a real model OptionalRead the edit published for an open model of 870 million weights, apply and verify one in the same form, see what 16-bit and 4-bit storage do to a fix, and run the real thing from a notebook.45 min
  47. Safety, licenses and provenance OptionalBefore a custom model is released: trace its lineage by hash, work out what its parents' licenses pass on, see what an edit can do to a safeguard, and write its card.40 min
  48. Capstone: ship a custom model OptionalSet release gates, then diagnose, edit, measure and document a tiny GPT with a fault no lesson has fixed, and write the record a stranger could check.45 min
Part of a free course 17 lessons

Creating Custom LLMs: Editing Open Weights Using Weight-Space Programming

Read a model's weights, change them by hand and judge an edit someone else made: deltas, merging, low-rank edits, directions and fact edits on a tiny GPT in your browser. Then replay recorded sessions of an LLM reading, fixing and writing weights, check the published edit of a real open model, and ship a model of your own with its patch, verdict and card. Every number is printed by the lesson's own code, and the method's limits are taught with it.

Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write