About 40 minutes · Optional lesson: deeper material that no phase or certificate requires · A free account opens the whole lesson. Already have an account? Sign in to open this lesson
Thousands of fine-tunes of the same open models are published. Merging combines several into one model with no training: you do arithmetic on the weights and hope the skills add up. It is the most widely used kind of weight editing, and model hubs are full of merges.
It is also oversold. This lesson merges two specialists that you can score exactly, so you will see what merging really buys, what gets lost, and why. You will write the four best-known methods yourself and find, as a large independent study did, that the simplest one is hard to beat.
What you'll be able to do
Average fine-tunes of one base model and measure what the merged model can and can't do
Measure how much two task vectors agree, with a cosine and a count of sign conflicts
Write task arithmetic, SLERP, TIES and DARE in a few lines each and compare them on the same test
Say when merging can't work, and what the evidence says about simple and elaborate methods
Quick check
Two fine-tunes of one base model each learned a different question. Each answers all 72 test questions on its own skill and none on the other's. You average their weights. A guess: what can the averaged model do?
Neither skill at all: averaged weights are noise
Some of each: neither skill survives in full
Only the first model's skill: the first model dominates
Both skills perfectly: averaging adds them up
Show the answer
Some of each: neither skill survives in full
You will measure it: 20 of 72 on one skill and 48 of 72 on the other. That is far more than either parent managed on the other's question, and far less than a model trained on both. Most of this lesson is about that gap.
The idea: merging is arithmetic on task vectors
Purpose. Combine fine-tunes of one base with arithmetic on their weights, and predict how it will go before you run it.
To merge two models you need them to have the same architecture, so that every tensor in one has a partner of the same shape in the other. The simplest merge is then the average: add each pair of tensors and divide by two.
When does that work? Start with the easy case. The course serves two fine-tunes of the same base on the same question, "What is the River tent for?". They saw the same examples in a different random order. tiny-gpt-uses and tiny-gpt-uses-b are two answers to one problem. Then the hard case: tiny-gpt-uses and tiny-gpt-materials, which learned different questions.
wsp.score asks 72 questions about products held out of every fine-tune and counts good answers. The next section of code runs both cases.
Quick check
Two models are to be averaged, tensor by tensor. What must be true of every tensor in one model?
It is stored in another file format
It has a same-shaped partner in the other
It was trained on a different task
It holds larger numbers than its partner
Show the answer
It has a same-shaped partner in the other
Averaging adds each pair of tensors and halves the sum, so every tensor needs a partner of the same shape. That is why merging needs the same architecture.
Measuring the conflict
You can predict how a merge will go before you run it. Each fine-tune has a task vector: its weights minus the base's. If two task vectors point the same way, the fine-tunes agree about what to change. If they point in unrelated directions, each leaves the other's changes alone. If they pull the same weights in opposite directions, they fight.
Two cheap measurements show which case you are in.
The cosine between the two task vectors, from Angle, cosine similarity and projection: 1 means the same direction, 0 means unrelated, and negative means opposed.
The share of weights where the two changes have opposite signs. One fine-tune raised that weight and the other lowered it. An average of the two shrinks both changes toward zero.
Quick check
From the dot product lesson: two vectors have a cosine of 0. What does that mean?
One of them has no length
They are at right angles: moving along one doesn't move you along the other
They are the same vector
They point in exactly opposite directions, so each one undoes the other
Show the answer
They are at right angles: moving along one doesn't move you along the other
A cosine of 0 means perpendicular. For two task vectors that is the friendly case: each fine-tune's change leaves the other's direction untouched, so adding both does little harm to either.
Averaging isn't the only arithmetic. Task arithmetic adds the task vectors to the base with a strength you choose, written λ (lambda):
θmerged=θbase+λ(ΔA+ΔB)
In plain words: start from the base, then add both fine-tunes' changes, each scaled by the same number. With λ=1 you add both changes in full. With λ=0.5 you add half of each, which is exactly the average of the two models.
Adding both in full sounds better than halving each. The next cell tries four strengths.
Quick check
Two task vectors for the same three weights. What does their average print?
The first weight's two changes agree, so the average keeps a large change. The second weight's cancel exactly: one fine-tune's −1 and the other's +1 leave nothing. The third is a conflict too, and what survives is smaller than either change.
Try it now
Two toy task vectors in two dimensions. Their sum is the merge at strength 1; half the sum is the average.
In your own words
In your own words: what do the cosine between two task vectors and their share of sign conflicts tell you before you merge?
Write a sentence or two, then compare it with a good answer in the lesson.
In code: merge the tiny GPTs and score them
Purpose. Run each method on the course's fine-tunes and read the scores on 72 + 72 held-out questions.
Worked example
Average two models: same task, then different tasks
Score each parent, then the average of the two same-task models, then the average of the two specialists.
import wsp
base, info = wsp.load_state("tiny-gpt")
uses, _ = wsp.load_state("tiny-gpt-uses")
uses_b, _ = wsp.load_state("tiny-gpt-uses-b")
materials, _ = wsp.load_state("tiny-gpt-materials")
defreport(label, state):
model, tok = wsp.build(state, info)
print(f"{label:34s} uses {wsp.score(model, tok, 'uses'):2d} of 72 materials "f"{wsp.score(model, tok, 'materials'):2d} of 72")
average = lambda a, b: {k: (a[k] + b[k]) / 2for k in a}
report("uses specialist", uses)
report("materials specialist", materials)
report("average of two uses models", average(uses, uses_b))
report("average of uses and materials", average(uses, materials))
It runs right here in your browser, with nothing to install.
Each specialist scores 72 on its own skill and 0 on the other. Averaging the two uses models loses nothing: still 72. This is the idea behind "model soups": fine-tunes of one base on one task sit close together, and the point between them is as good as either. Averaging the two specialists is another story. The result answers 20 uses questions and 48 materials questions. It has some of both skills, which neither parent had, and all of neither.
Quick check
What does this print?
import numpy as np
a = {"w": np.array([1.0, -2.0, 4.0])}
b = {"w": np.array([3.0, 2.0, 4.0])}
print({k: (a[k] + b[k]) / 2for k in a}["w"])
[2. 2. 4.]
[2. 0. 4.]
[4. 0. 8.]
[1. -2. 4.]
Show the answer
[2. 0. 4.]
Each entry is the average of its pair: (1 + 3)/2 = 2, (−2 + 2)/2 = 0 and (4 + 4)/2 = 4. Look at the middle entry. The two models disagreed about its sign, and the average erased it.
Drill
Average any number of models
Write average(states): states is a list of one or more states with the same names and shapes. Return a new state where each tensor is the mean of that tensor across the list. This is a model soup.
import numpy as np
defaverage(states):
return states[0]
a = {"w": np.array([1.0, 2.0]), "b": np.array([0.0])}
b = {"w": np.array([3.0, 0.0]), "b": np.array([1.0])}
c = {"w": np.array([2.0, 4.0]), "b": np.array([2.0])}
print(average([a, b, c]))
In the lesson, your code is checked as you write it.
Code flow. Every method here starts from the same three lines: task vectors, a strength, an addition. Step through them on three weights.
Worked example
How much do the task vectors agree?
For the same-task pair and for the two specialists: the cosine between their task vectors, and the share of weights the two fine-tunes moved in opposite directions.
import numpy as np
import wsp
base, info = wsp.load_state("tiny-gpt")
uses, _ = wsp.load_state("tiny-gpt-uses")
uses_b, _ = wsp.load_state("tiny-gpt-uses-b")
materials, _ = wsp.load_state("tiny-gpt-materials")
defagreement(one, two):
a, b = wsp.flatten(wsp.delta(one, base)), wsp.flatten(wsp.delta(two, base))
return wsp.cosine(wsp.delta(one, base),
wsp.delta(two, base)), float((np.sign(a) != np.sign(b)).mean())
for label, one, two in [("two uses models", uses, uses_b),
("uses and materials", uses, materials)]:
cos, conflict = agreement(one, two)
print(f"{label:20s} cosine {cos:.3f} opposite signs on {conflict:.0%} of weights")
In the lesson you can change this and run it.
The two uses models have a cosine of 0.965 and disagree in sign on 7% of weights: they made nearly the same edit, which is why their average kept the skill. The two specialists have a cosine of 0.401 and pull 35% of the weights in opposite directions. A cosine of 0.4 is not zero. The two skills share a lot, since both answer a question of the same shape, and that shared part is where they compete. Merging works best on task vectors that are close to unrelated. These are not.
Drill
Count the weights two edits fight over
Write sign_conflicts(delta_a, delta_b) for two deltas (states of differences). Return the share, from 0 to 1, of all entries where one delta is positive and the other negative. An entry where either delta is exactly 0 is not a conflict.
import numpy as np
defsign_conflicts(delta_a, delta_b):
return0.0
a = {"w": np.array([0.5, -0.2, 0.1, 0.0]), "b": np.array([1.0])}
b = {"w": np.array([0.3, 0.4, -0.1, 0.7]), "b": np.array([-2.0])}
print(sign_conflicts(a, b))
In the lesson, your code is checked as you write it.
Worked example
Task arithmetic at four strengths
Add both specialists' task vectors to the base at each strength and score both skills.
import wsp
base, info = wsp.load_state("tiny-gpt")
uses, _ = wsp.load_state("tiny-gpt-uses")
materials, _ = wsp.load_state("tiny-gpt-materials")
d_uses, d_materials = wsp.delta(uses, base), wsp.delta(materials, base)
for lam in (0.3, 0.5, 0.7, 1.0):
merged = wsp.add(wsp.add(base, d_uses, lam), d_materials, lam)
model, tok = wsp.build(merged, info)
print(f"lambda {lam}: uses {wsp.score(model, tok, 'uses'):2d} of 72, materials "f"{wsp.score(model, tok, 'materials'):2d} of 72")
In the lesson you can change this and run it.
At 0.5 the result is the average again, 20 and 48. Adding both changes in full, at 1.0, is much worse: 8 and 13. Where the two task vectors agree, their sum moves those weights twice as far as either fine-tune did, and the deltas lesson showed what overshooting does. At 0.3 the edits are too weak, and uses nearly vanishes. So the strength is a dial here too, with its best setting near the plain average.
SLERP, TIES and DARE
Researchers have proposed many ways to merge more carefully. Three names come up constantly.
SLERP (spherical interpolation) treats the two models' weights as two points on a sphere and moves along the arc between them, where an average cuts straight through. It keeps the result's length from shrinking.
TIES tries to remove the conflicts. It keeps only each task vector's largest changes, picks one sign for every weight (the sign with more total size behind it), and averages only the changes that agree with that sign.
DARE drops a random share of each task vector's entries and scales the survivors up to make up for it, betting that most of the small changes aren't needed.
Each is a few lines of NumPy. The next cell writes all three and scores them next to the one model that had an unfair advantage: tiny-gpt-both, fine-tuned on both questions with real training.
Worked example
Three careful merges, and the trained answer
SLERP halfway between the specialists, TIES keeping the largest half of each task vector, DARE dropping half of each at random, and the model trained on both tasks. All on the same 72 + 72 questions.
import numpy as np
import wsp
base, info = wsp.load_state("tiny-gpt")
uses, _ = wsp.load_state("tiny-gpt-uses")
materials, _ = wsp.load_state("tiny-gpt-materials")
both, _ = wsp.load_state("tiny-gpt-both")
a, b = wsp.flatten(wsp.delta(uses, base)), wsp.flatten(wsp.delta(materials, base))
from_delta = lambda d: wsp.add(base, wsp.unflatten(d, base))
defslerp(t=0.5):
x, y = wsp.flatten(uses).astype(float), wsp.flatten(materials).astype(float)
angle = np.arccos(x @ y / (np.linalg.norm(x) * np.linalg.norm(y)))
print(f"angle between the two specialists: {np.degrees(angle):.1f} degrees")
return wsp.unflatten((np.sin((1 - t) * angle) * x + np.sin(t * angle) * y)
/ np.sin(angle), base)
defties(keep=0.5):
trim = lambda d: np.where(np.abs(d) >= np.quantile(np.abs(d), 1 - keep), d, 0.0)
ta, tb = trim(a), trim(b)
# the sign with more size behind it
sign = np.sign(ta + tb)
agree_a, agree_b = ((np.sign(ta) == sign) & (ta != 0),
(np.sign(tb) == sign) & (tb != 0))
count = np.maximum(agree_a.astype(float) + agree_b, 1)
return from_delta((ta * agree_a + tb * agree_b) / count)
defdare(drop=0.5, lam=0.5):
rng = np.random.default_rng(0)
keep = lambda d: np.where(rng.random(d.size) >= drop, d / (1 - drop), 0.0)
return from_delta(lam * (keep(a) + keep(b)))
average = {k: (uses[k] + materials[k]) / 2for k in uses}
results = {}
for label, state in [("average", average), ("SLERP", slerp()), ("TIES", ties()),
("DARE", dare()), ("trained on both", both)]:
model, tok = wsp.build(state, info)
results[label] = (wsp.score(model, tok, "uses"), wsp.score(model, tok, "materials"))
print(f"{label:16s} uses {results[label][0]:2d} of 72 materials",
f"{results[label][1]:2d} of 72")
import matplotlib.pyplot as plt
xs = range(len(results))
plt.bar([x - 0.2for x in xs], [r[0] for r in results.values()], 0.4, label="uses")
plt.bar([x + 0.2for x in xs], [r[1] for r in results.values()], 0.4, label="materials")
plt.xticks(list(xs), list(results))
plt.ylabel("answers out of 72")
plt.title("Every merge, both skills")
plt.legend()
plt.show()
In the lesson you can change this and run it.
SLERP gives 20 and 48, the same as the plain average. The two specialists are only about 5 degrees apart as vectors, and over so small an arc the curve and the straight line are nearly the same path. TIES gives 13 and 49. DARE gives 15 and 55: seven more materials answers and five fewer uses answers. None of the three beats averaging on both skills. The model trained on both questions scores 72 and 72. Merging is free and training isn't, but when you have the data, training is the stronger tool, and here by a wide margin.
Quick check
On this pair of specialists, how did SLERP, TIES and DARE compare with the plain average (20 and 48)?
They can't be compared, because each needs different models
TIES removed the conflicts and reached 72 on both
None of them did better on both skills; SLERP matched it exactly
All three roughly doubled both scores
Show the answer
None of them did better on both skills; SLERP matched it exactly
SLERP: 20 and 48. TIES: 13 and 49. DARE: 15 and 55. The elaborate methods moved answers from one skill to the other and stayed near the same total. Only the model trained on both tasks reached 72 and 72.
Worked example
Averaging two models that don't share a base
tiny-gpt and tiny-gpt-induction have the same architecture but were each trained from scratch. Measure the loss per token on catalog text for the base, and for its average with the other model. For comparison: its average with one of its own fine-tunes.
import torch
import wsp
base, info = wsp.load_state("tiny-gpt")
# trained from scratch on other data
other, _ = wsp.load_state("tiny-gpt-induction")
tuned, _ = wsp.load_state("tiny-gpt-uses") # a fine-tune of base
text = ("The River tent is made from silnylon. It is light and roomy. Built for ""backpacking trips.")
average = lambda a, b: {k: (a[k] + b[k]) / 2for k in a}
defloss(state):
model, tok = wsp.build(state, info)
ids = torch.tensor([tok.encode(text)])
with torch.no_grad():
logp = torch.log_softmax(model(ids[:, :-1]), dim=-1)
returnfloat(-logp[0, torch.arange(ids.shape[1] - 1), ids[0, 1:]].mean())
print(f"base alone: {loss(base):.2f}")
print(f"base averaged with a fine-tune: {loss(average(base, tuned)):.2f}")
print(f"base averaged with a stranger: {loss(average(base, other)):.2f}")
In the lesson you can change this and run it.
The base predicts this catalog sentence with a loss of 0.64. Averaged with its own fine-tune it moves to 0.74, because the fine-tune is a short step from the base. Averaged with a model trained separately, the loss is 5.38, more than eight times higher: the result has forgotten the catalog. The two models' weights mean nothing to each other. This is why every merge recipe starts with "fine-tunes of the same base," and why a merge of unrelated checkpoints with matching shapes loads without error and then produces nonsense.
In your own words
In your own words: why did averaging two fine-tunes of the same task lose nothing, while averaging two different specialists lost a lot?
Write a sentence or two, then compare it with a good answer in the lesson.
In production: merging with real tools
Purpose. What a published merge is, what the evidence says about elaborate methods, and how to run one.
One pair of tiny models proves little on its own, so here is the larger evidence. A study first posted in late 2025, "A Systematic Study of In-the-Wild Model Merging for Large Language Models," tested six merging methods on four open models, each with twelve published fine-tunes, across sixteen benchmarks. Its finding matches what you just saw. The oldest and simplest method, task arithmetic, was the only one that reliably improved on the base model. The newer methods built to handle interference generally did not.
The authors point to the kind of case you met in the cosine cell. Earlier evidence for merging came from fine-tunes with clearly separate tasks. Fine-tunes found in the wild overlap and can conflict, and they conclude that current methods mostly fail to draw useful changes out of such checkpoints.
One case is worse than conflict. Merging needs a shared base. Two models trained separately, from different random starting weights, can do the same job with their internal units in a different order. Averaging them blends unit 3 of one with an unrelated unit 3 of the other.
Quick check
Two models have the same architecture but were trained separately from different random starting weights. Why does averaging them fail?
Their units learned in different orders, so each weight is averaged with one that means something else
One of them must have been trained wrongly
Their files have different sizes, so the tensors don't line up
Averaging only works for models with more than a billion weights, and these are too small
Show the answer
Their units learned in different orders, so each weight is averaged with one that means something else
Nothing forces unit 3 of one model to do the job of unit 3 in the other. Fine-tunes of one base keep the base's layout, which is why their weights can be mixed. Separately trained models have no such agreement.
Where this shows up in an engineer's week
A merge appears on the model hub with a good-looking leaderboard score. Before you use it, check that its parents share a base, re-score it on your own held-out set for every skill you need, and compare it with the plain average. If the plain average does as well, take the average.
A model hub lists a merge of four fine-tunes made with an elaborate method. In your own words, what would you check before believing it is better than its parents?
Write a sentence or two, then compare it with a good answer in the lesson.
11DistillationTrain a small model to imitate a large one.35 min
12Tuning an embedding model for retrievalWhen search fails on your own vocabulary: train an adapter with a contrastive loss on question and passage pairs, test on held-out passages, use batch size and hard negatives, and measure what tuning costs elsewhere.45 min
13QuantizationStore weights in fewer bits to run models cheaply.35 min
23Copying from the promptIn-context learning as a two-head copying circuit: the base tiny GPT cannot copy a repeated pattern, a model trained on repeats grows an induction head and can.30 min
24Why temperature changes the answerThe same prompt at 0, 0.7 and 1.5: deterministic loops, plausible variety, broken grammar, measured on a real model.25 min
25Why long prompts diluteAttention is a budget shared by every token, so a fact far from the question loses its share; measured, with the cliff beyond the trained window.25 min
26Why it never says "I don't know"A chat model answers every request because its fine-tuning data always did; preference training adds flattery; how to build abstention in.25 min
32Weight-space programming: what it is, and what has been shownFreeOptionalA language model reads and writes another network's weights with no training loop: the published experiments, their limits and the criticism, and one edit you make by hand.30 min
33Anatomy of a checkpointOptionalA model file is a set of named tensors: count them, find the ones that write to the residual stream, and save one to bytes and back.35 min
34Reading weights without running the modelOptionalSingular values, and an attention head's token-to-token circuit, read from the matrices alone and then checked against a measurement.35 min
35Writing weights by handOptionalSet a small network's 55 weights so it catches every ball in a paddle game, the way a program would be written, and meet compiled transformers.40 min
36Edits are deltas: task vectors, patches and hashesOptionalSubtract a base from a fine-tune and the edit becomes an object: scale it, take it away, ship it as a patch, and prove by hash what was applied.35 min
37Merging modelsOptionalAverage two fine-tunes, add their task vectors, try SLERP, TIES and DARE, measure the interference, and compare with a model trained on both tasks.40 min
38Low-rank editsOptionalCut a fine-tune's change down to its largest directions with the SVD: how few a behavior needs, and how to pull an adapter out of a delta.35 min
39Directions: steer the run or edit the weightsOptionalFind a behavior's direction in the residual stream, add it at run time, project it out of the weights, and see the effect change from input to input.40 min
40Editing one factOptionalA rank-one change to one matrix rewrites what a product is made from; then reliability, generalization, locality, and what breaks after many edits.40 min
41The editor's harnessOptionalWhat an LLM that edits weights is given and what it is denied: weights as text and their cost in tokens, a sandbox, a budget of forward passes, and rules the harness enforces.40 min
42An LLM reads a network's weightsOptionalSix recorded sessions: an LLM is handed 1,315 weights as text and asked what the network was trained to do. Score each answer against the network, and study one it got wrong.40 min
43An LLM fixes a model's weightsOptionalFind a tiny GPT's loop by hand, then replay recorded sessions in which an LLM hunts for it with ten forward passes, and measure every fix on held-out questions.45 min
44An LLM writes a network's weightsOptionalFive recorded sessions: an LLM writes every weight of a paddle network from the game's source and one sentence. Score each against its specification, test its design claims, and edit one by hand.40 min
45Did it work? Testing an editOptionalReview a patch for a tiny GPT that can't stop talking: held-out questions, a paired comparison, a same-size control, and what the edit cost.40 min
46The published edit, on a real modelOptionalRead the edit published for an open model of 870 million weights, apply and verify one in the same form, see what 16-bit and 4-bit storage do to a fix, and run the real thing from a notebook.45 min
47Safety, licenses and provenanceOptionalBefore a custom model is released: trace its lineage by hash, work out what its parents' licenses pass on, see what an edit can do to a safeguard, and write its card.40 min
48Capstone: ship a custom modelOptionalSet release gates, then diagnose, edit, measure and document a tiny GPT with a fault no lesson has fixed, and write the record a stranger could check.45 min
Read a model's weights, change them by hand and judge an edit someone else made: deltas, merging, low-rank edits, directions and fact edits on a tiny GPT in your browser. Then replay recorded sessions of an LLM reading, fixing and writing weights, check the published edit of a real open model, and ship a model of your own with its patch, verdict and card. Every number is printed by the lesson's own code, and the method's limits are taught with it.
See the course →
Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write