Index Ai Academy
Phase 6 · Editing models

Merging models

Average two fine-tunes, add their task vectors, try SLERP, TIES and DARE, measure the interference, and compare with a model trained on both tasks.

About 40 minutes · Optional lesson: deeper material that no phase or certificate requires · A free account opens the whole lesson.
Already have an account? Sign in to open this lesson

What it covers10

Why it matters

Thousands of fine-tunes of the same open models are published. Merging combines several into one model with no training: you do arithmetic on the weights and hope the skills add up. It is the most widely used kind of weight editing, and model hubs are full of merges.

It is also oversold. This lesson merges two specialists that you can score exactly, so you will see what merging really buys, what gets lost, and why. You will write the four best-known methods yourself and find, as a large independent study did, that the simplest one is hard to beat.

What you'll be able to do

  • Average fine-tunes of one base model and measure what the merged model can and can't do
  • Measure how much two task vectors agree, with a cosine and a count of sign conflicts
  • Write task arithmetic, SLERP, TIES and DARE in a few lines each and compare them on the same test
  • Say when merging can't work, and what the evidence says about simple and elaborate methods
Quick check
Two fine-tunes of one base model each learned a different question. Each answers all 72 test questions on its own skill and none on the other's. You average their weights. A guess: what can the averaged model do?
  1. Neither skill at all: averaged weights are noise
  2. Some of each: neither skill survives in full
  3. Only the first model's skill: the first model dominates
  4. Both skills perfectly: averaging adds them up
Show the answer

Some of each: neither skill survives in full

You will measure it: 20 of 72 on one skill and 48 of 72 on the other. That is far more than either parent managed on the other's question, and far less than a model trained on both. Most of this lesson is about that gap.

The idea: merging is arithmetic on task vectors

Purpose. Combine fine-tunes of one base with arithmetic on their weights, and predict how it will go before you run it.

This lesson rests on Edits are deltas: task vectors, patches and hashes, where an edit is a task vector, and on Angle, cosine similarity and projection.

To merge two models you need them to have the same architecture, so that every tensor in one has a partner of the same shape in the other. The simplest merge is then the average: add each pair of tensors and divide by two.

When does that work? Start with the easy case. The course serves two fine-tunes of the same base on the same question, "What is the River tent for?". They saw the same examples in a different random order. tiny-gpt-uses and tiny-gpt-uses-b are two answers to one problem. Then the hard case: tiny-gpt-uses and tiny-gpt-materials, which learned different questions.

wsp.score asks 72 questions about products held out of every fine-tune and counts good answers. The next section of code runs both cases.

Quick check
Two models are to be averaged, tensor by tensor. What must be true of every tensor in one model?
  1. It is stored in another file format
  2. It has a same-shaped partner in the other
  3. It was trained on a different task
  4. It holds larger numbers than its partner
Show the answer

It has a same-shaped partner in the other

Averaging adds each pair of tensors and halves the sum, so every tensor needs a partner of the same shape. That is why merging needs the same architecture.

Measuring the conflict

You can predict how a merge will go before you run it. Each fine-tune has a task vector: its weights minus the base's. If two task vectors point the same way, the fine-tunes agree about what to change. If they point in unrelated directions, each leaves the other's changes alone. If they pull the same weights in opposite directions, they fight.

Two cheap measurements show which case you are in.

  • The cosine between the two task vectors, from Angle, cosine similarity and projection: 1 means the same direction, 0 means unrelated, and negative means opposed.
  • The share of weights where the two changes have opposite signs. One fine-tune raised that weight and the other lowered it. An average of the two shrinks both changes toward zero.
Quick check
From the dot product lesson: two vectors have a cosine of 0. What does that mean?
  1. One of them has no length
  2. They are at right angles: moving along one doesn't move you along the other
  3. They are the same vector
  4. They point in exactly opposite directions, so each one undoes the other
Show the answer

They are at right angles: moving along one doesn't move you along the other

A cosine of 0 means perpendicular. For two task vectors that is the friendly case: each fine-tune's change leaves the other's direction untouched, so adding both does little harm to either.

Averaging isn't the only arithmetic. Task arithmetic adds the task vectors to the base with a strength you choose, written λ\lambda (lambda):

θmerged=θbase+λ (ΔA+ΔB)\theta_{\text{merged}} = \theta_{\text{base}} + \lambda \, (\Delta_A + \Delta_B)

In plain words: start from the base, then add both fine-tunes' changes, each scaled by the same number. With λ=1\lambda = 1 you add both changes in full. With λ=0.5\lambda = 0.5 you add half of each, which is exactly the average of the two models.

Adding both in full sounds better than halving each. The next cell tries four strengths.

Quick check
Two task vectors for the same three weights. What does their average print?
import numpy as np

delta_a = np.array([2.0, -1.0, 3.0])
delta_b = np.array([4.0, 1.0, -1.0])
print((delta_a + delta_b) / 2)
  1. [6. 0. 2.]
  2. [2. -1. 3.]
  3. [3. 1. 2.]
  4. [3. 0. 1.]
Show the answer

[3. 0. 1.]

The first weight's two changes agree, so the average keeps a large change. The second weight's cancel exactly: one fine-tune's −1 and the other's +1 leave nothing. The third is a conflict too, and what survives is smaller than either change.

Try it now

Two toy task vectors in two dimensions. Their sum is the merge at strength 1; half the sum is the average.

In your own words
In your own words: what do the cosine between two task vectors and their share of sign conflicts tell you before you merge?

Write a sentence or two, then compare it with a good answer in the lesson.

In code: merge the tiny GPTs and score them

Purpose. Run each method on the course's fine-tunes and read the scores on 72 + 72 held-out questions.

Worked example

Average two models: same task, then different tasks

Score each parent, then the average of the two same-task models, then the average of the two specialists.

import wsp

base, info = wsp.load_state("tiny-gpt")
uses, _ = wsp.load_state("tiny-gpt-uses")
uses_b, _ = wsp.load_state("tiny-gpt-uses-b")
materials, _ = wsp.load_state("tiny-gpt-materials")

def report(label, state):
    model, tok = wsp.build(state, info)
    print(f"{label:34s} uses {wsp.score(model, tok, 'uses'):2d} of 72   materials "
          f"{wsp.score(model, tok, 'materials'):2d} of 72")

average = lambda a, b: {k: (a[k] + b[k]) / 2 for k in a}
report("uses specialist", uses)
report("materials specialist", materials)
report("average of two uses models", average(uses, uses_b))
report("average of uses and materials", average(uses, materials))
It runs right here in your browser, with nothing to install.

Each specialist scores 72 on its own skill and 0 on the other. Averaging the two uses models loses nothing: still 72. This is the idea behind "model soups": fine-tunes of one base on one task sit close together, and the point between them is as good as either. Averaging the two specialists is another story. The result answers 20 uses questions and 48 materials questions. It has some of both skills, which neither parent had, and all of neither.

Quick check
What does this print?
import numpy as np

a = {"w": np.array([1.0, -2.0, 4.0])}
b = {"w": np.array([3.0, 2.0, 4.0])}
print({k: (a[k] + b[k]) / 2 for k in a}["w"])
  1. [2. 2. 4.]
  2. [2. 0. 4.]
  3. [4. 0. 8.]
  4. [1. -2. 4.]
Show the answer

[2. 0. 4.]

Each entry is the average of its pair: (1 + 3)/2 = 2, (−2 + 2)/2 = 0 and (4 + 4)/2 = 4. Look at the middle entry. The two models disagreed about its sign, and the average erased it.

Drill

Average any number of models

Write average(states): states is a list of one or more states with the same names and shapes. Return a new state where each tensor is the mean of that tensor across the list. This is a model soup.

import numpy as np

def average(states):
    return states[0]

a = {"w": np.array([1.0, 2.0]), "b": np.array([0.0])}
b = {"w": np.array([3.0, 0.0]), "b": np.array([1.0])}
c = {"w": np.array([2.0, 4.0]), "b": np.array([2.0])}
print(average([a, b, c]))
In the lesson, your code is checked as you write it.

Code flow. Every method here starts from the same three lines: task vectors, a strength, an addition. Step through them on three weights.

Worked example

How much do the task vectors agree?

For the same-task pair and for the two specialists: the cosine between their task vectors, and the share of weights the two fine-tunes moved in opposite directions.

import numpy as np
import wsp

base, info = wsp.load_state("tiny-gpt")
uses, _ = wsp.load_state("tiny-gpt-uses")
uses_b, _ = wsp.load_state("tiny-gpt-uses-b")
materials, _ = wsp.load_state("tiny-gpt-materials")

def agreement(one, two):
    a, b = wsp.flatten(wsp.delta(one, base)), wsp.flatten(wsp.delta(two, base))
    return wsp.cosine(wsp.delta(one, base),
                      wsp.delta(two, base)), float((np.sign(a) != np.sign(b)).mean())

for label, one, two in [("two uses models", uses, uses_b),
                        ("uses and materials", uses, materials)]:
    cos, conflict = agreement(one, two)
    print(f"{label:20s} cosine {cos:.3f}   opposite signs on {conflict:.0%} of weights")
In the lesson you can change this and run it.

The two uses models have a cosine of 0.965 and disagree in sign on 7% of weights: they made nearly the same edit, which is why their average kept the skill. The two specialists have a cosine of 0.401 and pull 35% of the weights in opposite directions. A cosine of 0.4 is not zero. The two skills share a lot, since both answer a question of the same shape, and that shared part is where they compete. Merging works best on task vectors that are close to unrelated. These are not.

Drill

Count the weights two edits fight over

Write sign_conflicts(delta_a, delta_b) for two deltas (states of differences). Return the share, from 0 to 1, of all entries where one delta is positive and the other negative. An entry where either delta is exactly 0 is not a conflict.

import numpy as np

def sign_conflicts(delta_a, delta_b):
    return 0.0

a = {"w": np.array([0.5, -0.2, 0.1, 0.0]), "b": np.array([1.0])}
b = {"w": np.array([0.3, 0.4, -0.1, 0.7]), "b": np.array([-2.0])}
print(sign_conflicts(a, b))
In the lesson, your code is checked as you write it.
Worked example

Task arithmetic at four strengths

Add both specialists' task vectors to the base at each strength and score both skills.

import wsp

base, info = wsp.load_state("tiny-gpt")
uses, _ = wsp.load_state("tiny-gpt-uses")
materials, _ = wsp.load_state("tiny-gpt-materials")
d_uses, d_materials = wsp.delta(uses, base), wsp.delta(materials, base)

for lam in (0.3, 0.5, 0.7, 1.0):
    merged = wsp.add(wsp.add(base, d_uses, lam), d_materials, lam)
    model, tok = wsp.build(merged, info)
    print(f"lambda {lam}: uses {wsp.score(model, tok, 'uses'):2d} of 72, materials "
          f"{wsp.score(model, tok, 'materials'):2d} of 72")
In the lesson you can change this and run it.

At 0.5 the result is the average again, 20 and 48. Adding both changes in full, at 1.0, is much worse: 8 and 13. Where the two task vectors agree, their sum moves those weights twice as far as either fine-tune did, and the deltas lesson showed what overshooting does. At 0.3 the edits are too weak, and uses nearly vanishes. So the strength is a dial here too, with its best setting near the plain average.

SLERP, TIES and DARE

Researchers have proposed many ways to merge more carefully. Three names come up constantly.

  • SLERP (spherical interpolation) treats the two models' weights as two points on a sphere and moves along the arc between them, where an average cuts straight through. It keeps the result's length from shrinking.
  • TIES tries to remove the conflicts. It keeps only each task vector's largest changes, picks one sign for every weight (the sign with more total size behind it), and averages only the changes that agree with that sign.
  • DARE drops a random share of each task vector's entries and scales the survivors up to make up for it, betting that most of the small changes aren't needed.

Each is a few lines of NumPy. The next cell writes all three and scores them next to the one model that had an unfair advantage: tiny-gpt-both, fine-tuned on both questions with real training.

Worked example

Three careful merges, and the trained answer

SLERP halfway between the specialists, TIES keeping the largest half of each task vector, DARE dropping half of each at random, and the model trained on both tasks. All on the same 72 + 72 questions.

import numpy as np
import wsp

base, info = wsp.load_state("tiny-gpt")
uses, _ = wsp.load_state("tiny-gpt-uses")
materials, _ = wsp.load_state("tiny-gpt-materials")
both, _ = wsp.load_state("tiny-gpt-both")
a, b = wsp.flatten(wsp.delta(uses, base)), wsp.flatten(wsp.delta(materials, base))
from_delta = lambda d: wsp.add(base, wsp.unflatten(d, base))

def slerp(t=0.5):
    x, y = wsp.flatten(uses).astype(float), wsp.flatten(materials).astype(float)
    angle = np.arccos(x @ y / (np.linalg.norm(x) * np.linalg.norm(y)))
    print(f"angle between the two specialists: {np.degrees(angle):.1f} degrees")
    return wsp.unflatten((np.sin((1 - t) * angle) * x + np.sin(t * angle) * y)
                         / np.sin(angle), base)

def ties(keep=0.5):
    trim = lambda d: np.where(np.abs(d) >= np.quantile(np.abs(d), 1 - keep), d, 0.0)
    ta, tb = trim(a), trim(b)
    # the sign with more size behind it
    sign = np.sign(ta + tb)
    agree_a, agree_b = ((np.sign(ta) == sign) & (ta != 0),
                        (np.sign(tb) == sign) & (tb != 0))
    count = np.maximum(agree_a.astype(float) + agree_b, 1)
    return from_delta((ta * agree_a + tb * agree_b) / count)

def dare(drop=0.5, lam=0.5):
    rng = np.random.default_rng(0)
    keep = lambda d: np.where(rng.random(d.size) >= drop, d / (1 - drop), 0.0)
    return from_delta(lam * (keep(a) + keep(b)))

average = {k: (uses[k] + materials[k]) / 2 for k in uses}
results = {}
for label, state in [("average", average), ("SLERP", slerp()), ("TIES", ties()),
                     ("DARE", dare()), ("trained on both", both)]:
    model, tok = wsp.build(state, info)
    results[label] = (wsp.score(model, tok, "uses"), wsp.score(model, tok, "materials"))
    print(f"{label:16s} uses {results[label][0]:2d} of 72   materials",
          f"{results[label][1]:2d} of 72")

import matplotlib.pyplot as plt

xs = range(len(results))
plt.bar([x - 0.2 for x in xs], [r[0] for r in results.values()], 0.4, label="uses")
plt.bar([x + 0.2 for x in xs], [r[1] for r in results.values()], 0.4, label="materials")
plt.xticks(list(xs), list(results))
plt.ylabel("answers out of 72")
plt.title("Every merge, both skills")
plt.legend()
plt.show()
In the lesson you can change this and run it.

SLERP gives 20 and 48, the same as the plain average. The two specialists are only about 5 degrees apart as vectors, and over so small an arc the curve and the straight line are nearly the same path. TIES gives 13 and 49. DARE gives 15 and 55: seven more materials answers and five fewer uses answers. None of the three beats averaging on both skills. The model trained on both questions scores 72 and 72. Merging is free and training isn't, but when you have the data, training is the stronger tool, and here by a wide margin.

Quick check
On this pair of specialists, how did SLERP, TIES and DARE compare with the plain average (20 and 48)?
  1. They can't be compared, because each needs different models
  2. TIES removed the conflicts and reached 72 on both
  3. None of them did better on both skills; SLERP matched it exactly
  4. All three roughly doubled both scores
Show the answer

None of them did better on both skills; SLERP matched it exactly

SLERP: 20 and 48. TIES: 13 and 49. DARE: 15 and 55. The elaborate methods moved answers from one skill to the other and stayed near the same total. Only the model trained on both tasks reached 72 and 72.

Worked example

Averaging two models that don't share a base

tiny-gpt and tiny-gpt-induction have the same architecture but were each trained from scratch. Measure the loss per token on catalog text for the base, and for its average with the other model. For comparison: its average with one of its own fine-tunes.

import torch
import wsp

base, info = wsp.load_state("tiny-gpt")
# trained from scratch on other data
other, _ = wsp.load_state("tiny-gpt-induction")
tuned, _ = wsp.load_state("tiny-gpt-uses")               # a fine-tune of base

text = ("The River tent is made from silnylon. It is light and roomy. Built for "
        "backpacking trips.")
average = lambda a, b: {k: (a[k] + b[k]) / 2 for k in a}

def loss(state):
    model, tok = wsp.build(state, info)
    ids = torch.tensor([tok.encode(text)])
    with torch.no_grad():
        logp = torch.log_softmax(model(ids[:, :-1]), dim=-1)
    return float(-logp[0, torch.arange(ids.shape[1] - 1), ids[0, 1:]].mean())

print(f"base alone:                    {loss(base):.2f}")
print(f"base averaged with a fine-tune: {loss(average(base, tuned)):.2f}")
print(f"base averaged with a stranger:  {loss(average(base, other)):.2f}")
In the lesson you can change this and run it.

The base predicts this catalog sentence with a loss of 0.64. Averaged with its own fine-tune it moves to 0.74, because the fine-tune is a short step from the base. Averaged with a model trained separately, the loss is 5.38, more than eight times higher: the result has forgotten the catalog. The two models' weights mean nothing to each other. This is why every merge recipe starts with "fine-tunes of the same base," and why a merge of unrelated checkpoints with matching shapes loads without error and then produces nonsense.

In your own words
In your own words: why did averaging two fine-tunes of the same task lose nothing, while averaging two different specialists lost a lot?

Write a sentence or two, then compare it with a good answer in the lesson.

In production: merging with real tools

Purpose. What a published merge is, what the evidence says about elaborate methods, and how to run one.

One pair of tiny models proves little on its own, so here is the larger evidence. A study first posted in late 2025, "A Systematic Study of In-the-Wild Model Merging for Large Language Models," tested six merging methods on four open models, each with twelve published fine-tunes, across sixteen benchmarks. Its finding matches what you just saw. The oldest and simplest method, task arithmetic, was the only one that reliably improved on the base model. The newer methods built to handle interference generally did not.

The authors point to the kind of case you met in the cosine cell. Earlier evidence for merging came from fine-tunes with clearly separate tasks. Fine-tunes found in the wild overlap and can conflict, and they conclude that current methods mostly fail to draw useful changes out of such checkpoints.

One case is worse than conflict. Merging needs a shared base. Two models trained separately, from different random starting weights, can do the same job with their internal units in a different order. Averaging them blends unit 3 of one with an unrelated unit 3 of the other.

Quick check
Two models have the same architecture but were trained separately from different random starting weights. Why does averaging them fail?
  1. Their units learned in different orders, so each weight is averaged with one that means something else
  2. One of them must have been trained wrongly
  3. Their files have different sizes, so the tensors don't line up
  4. Averaging only works for models with more than a billion weights, and these are too small
Show the answer

Their units learned in different orders, so each weight is averaged with one that means something else

Nothing forces unit 3 of one model to do the job of unit 3 in the other. Fine-tunes of one base keep the base's layout, which is why their weights can be mixed. Separately trained models have no such agreement.

Where this shows up in an engineer's week

A merge appears on the model hub with a good-looking leaderboard score. Before you use it, check that its parents share a base, re-score it on your own held-out set for every skill you need, and compare it with the plain average. If the plain average does as well, take the average.

Goes deeper in

In your own words
A model hub lists a merge of four fine-tunes made with an elaborate method. In your own words, what would you check before believing it is better than its parents?

Write a sentence or two, then compare it with a good answer in the lesson.

Practice what you just read

  • 5 code examples you can edit and run
  • a practice exercise that checks your code
  • a 5-question quiz
  • 9 review cards for spaced repetition

Try a quiz question

What does merging two models require before any method can work?

  1. The same architecture, and in practice a shared base model they were both fine-tuned from
  2. The same training data
  3. The same score on a benchmark
  4. That both were trained by the same team, on the same hardware and software

Answer it at the end of the lesson to see if you're right, and why.

Before this lesson

Go deeper, if you like

Optional lessons that build on this one. They never block a phase or a certificate.

Keep learning

Where this lesson leads.

575 lessons in thirteen phases. 176 are free without an account, and a free account opens the rest. See every lesson.

Phase 6 · about 35 hours of study

Modern LLM engineering

Post-training, fine-tuning, serving and safety 48 lessons 2 free
  1. Base models vs assistants FreeWhat post-training adds to a pretrained model.30 min
  2. Supervised fine-tuningTeach behavior from example conversations.40 min
  3. Instruction and preference dataThe datasets that shape assistant behavior.30 min
  4. RLHFReward models and reinforcement learning from human feedback.40 min
  5. DPO and direct preference methodsPreference training without a separate reward model.40 min
  6. Constitutional AITraining with AI feedback guided by written principles.35 min
  7. Training reasoning modelsReinforcement learning on verifiable problems.40 min
  8. Prompting vs RAG vs fine-tuningPick the cheapest approach that works.30 min
  9. LoRAFine-tune a big model by training small low-rank updates.40 min
  10. Fine-tune an open modelRun a real fine-tune and evaluate the result.45 min
  11. DistillationTrain a small model to imitate a large one.35 min
  12. Tuning an embedding model for retrievalWhen search fails on your own vocabulary: train an adapter with a contrastive loss on question and passage pairs, test on held-out passages, use batch size and hard negatives, and measure what tuning costs elsewhere.45 min
  13. QuantizationStore weights in fewer bits to run models cheaply.35 min
  14. Quantization in practice: zero points, calibration and formatsScale and zero point, per-channel scales, calibration sets, error on a layer's output, quantization-aware training and the file formats you meet at work.35 min
  15. Serving modelsBatching, throughput and latency in production.35 min
  16. One request through a serving stack: latency and costFollow one request through queue, prefill, a prefix cache, decode and the stream, and price a token from a GPU's hourly cost.35 min
  17. Speculative decodingDraft with a small model, verify with a big one.35 min
  18. Running open models locallyDownload, quantize and run a model on your own machine.30 min
  19. Will it fit? Memory for serving and fine-tuningWeights, KV cache, runtime and adapter states added up for a model card and a GPU, with a calculator.30 min
  20. InterpretabilityLooking inside models to see what they compute.40 min
  21. Confident and wrongWhy a model answers with the same assurance whether it is right, wrong or asked about something that does not exist.30 min
  22. Why it can't spell or count lettersThe model sees token IDs, and an embedding learned from usage carries nothing about spelling.25 min
  23. Copying from the promptIn-context learning as a two-head copying circuit: the base tiny GPT cannot copy a repeated pattern, a model trained on repeats grows an induction head and can.30 min
  24. Why temperature changes the answerThe same prompt at 0, 0.7 and 1.5: deterministic loops, plausible variety, broken grammar, measured on a real model.25 min
  25. Why long prompts diluteAttention is a budget shared by every token, so a fact far from the question loses its share; measured, with the cliff beyond the trained window.25 min
  26. Why it never says "I don't know"A chat model answers every request because its fine-tuning data always did; preference training adds flattery; how to build abstention in.25 min
  27. Benchmarks and their limitsWhat leaderboards measure, and what they miss.38 min
  28. Alignment and safetyMaking capable models behave as intended.35 min
  29. Evaluating agentsMeasure multi-step, tool-using behavior.35 min
  30. Reading AI papersGet the idea out of a research paper in 30 minutes.30 min
  31. Capstone: a tuned, evaluated assistantFine-tune, evaluate and deploy a specialist assistant.45 min
  32. Weight-space programming: what it is, and what has been shown Free OptionalA language model reads and writes another network's weights with no training loop: the published experiments, their limits and the criticism, and one edit you make by hand.30 min
  33. Anatomy of a checkpoint OptionalA model file is a set of named tensors: count them, find the ones that write to the residual stream, and save one to bytes and back.35 min
  34. Reading weights without running the model OptionalSingular values, and an attention head's token-to-token circuit, read from the matrices alone and then checked against a measurement.35 min
  35. Writing weights by hand OptionalSet a small network's 55 weights so it catches every ball in a paddle game, the way a program would be written, and meet compiled transformers.40 min
  36. Edits are deltas: task vectors, patches and hashes OptionalSubtract a base from a fine-tune and the edit becomes an object: scale it, take it away, ship it as a patch, and prove by hash what was applied.35 min
  37. Merging models OptionalAverage two fine-tunes, add their task vectors, try SLERP, TIES and DARE, measure the interference, and compare with a model trained on both tasks.40 min
  38. Low-rank edits OptionalCut a fine-tune's change down to its largest directions with the SVD: how few a behavior needs, and how to pull an adapter out of a delta.35 min
  39. Directions: steer the run or edit the weights OptionalFind a behavior's direction in the residual stream, add it at run time, project it out of the weights, and see the effect change from input to input.40 min
  40. Editing one fact OptionalA rank-one change to one matrix rewrites what a product is made from; then reliability, generalization, locality, and what breaks after many edits.40 min
  41. The editor's harness OptionalWhat an LLM that edits weights is given and what it is denied: weights as text and their cost in tokens, a sandbox, a budget of forward passes, and rules the harness enforces.40 min
  42. An LLM reads a network's weights OptionalSix recorded sessions: an LLM is handed 1,315 weights as text and asked what the network was trained to do. Score each answer against the network, and study one it got wrong.40 min
  43. An LLM fixes a model's weights OptionalFind a tiny GPT's loop by hand, then replay recorded sessions in which an LLM hunts for it with ten forward passes, and measure every fix on held-out questions.45 min
  44. An LLM writes a network's weights OptionalFive recorded sessions: an LLM writes every weight of a paddle network from the game's source and one sentence. Score each against its specification, test its design claims, and edit one by hand.40 min
  45. Did it work? Testing an edit OptionalReview a patch for a tiny GPT that can't stop talking: held-out questions, a paired comparison, a same-size control, and what the edit cost.40 min
  46. The published edit, on a real model OptionalRead the edit published for an open model of 870 million weights, apply and verify one in the same form, see what 16-bit and 4-bit storage do to a fix, and run the real thing from a notebook.45 min
  47. Safety, licenses and provenance OptionalBefore a custom model is released: trace its lineage by hash, work out what its parents' licenses pass on, see what an edit can do to a safeguard, and write its card.40 min
  48. Capstone: ship a custom model OptionalSet release gates, then diagnose, edit, measure and document a tiny GPT with a fault no lesson has fixed, and write the record a stranger could check.45 min
Part of a free course 17 lessons

Creating Custom LLMs: Editing Open Weights Using Weight-Space Programming

Read a model's weights, change them by hand and judge an edit someone else made: deltas, merging, low-rank edits, directions and fact edits on a tiny GPT in your browser. Then replay recorded sessions of an LLM reading, fixing and writing weights, check the published edit of a real open model, and ship a model of your own with its patch, verdict and card. Every number is printed by the lesson's own code, and the method's limits are taught with it.

Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write