Before a custom model is released: trace its lineage by hash, work out what its parents' licenses pass on, see what an edit can do to a safeguard, and write its card.
About 40 minutes · Optional lesson: deeper material that no phase or certificate requires · A free account opens the whole lesson. Already have an account? Sign in to open this lesson
You can change what a model does by changing its numbers, and you can show that the change worked. One step is left before anyone else may rely on your model: releasing it. The person who downloads it can't see how you worked. They get a file, and whatever you wrote beside it.
Three questions decide whether they can trust that file. Where did it come from? Whose rules come with it? And what did the edit do to behavior they count on? An edited model is easy to make and easy to pass around, so these questions are asked of every team that ships one.
What you'll be able to do
Trace a model's lineage from its files, and check each link by hash
Work out what a patched or merged model inherits from the licenses of its parents
Say what an edit can do to a safeguard kept in the weights, and add one outside them
Write the record a derived model ships with: base, change, hashes, measurements and limits
Quick check
A model's page says it passed a safety review. You download the file from a mirror site. A guess: what ties that review to the file now on your disk?
The file's name, which the mirror copied from the original
A hash published with the review, which your file has to match
The file's size, since any change would make the file larger
Nothing is needed: a copy of a file is always the same file
Show the answer
A hash published with the review, which your file has to match
A review describes one exact set of numbers. A hash is a short fingerprint of those numbers. If your file's hash matches the published one, you hold what was reviewed. A name or a size can match on a file that someone changed.
The model you reviewed in Did it work? Testing an edit has a history. The base model tiny-gpt was pretrained on catalog text. tiny-gpt-uses was fine-tuned from it. tiny-gpt-loop is that fine-tune with a fault planted in it. The patch loop-fix makes a fourth model from the third.
That history is the model's lineage: every model it was made from, in order. A written record of the lineage, with proof, is its provenance. The proof is the hash from Edits are deltas: task vectors, patches and hashes. wsp.state_hash turns all 40,195 numbers into 12 characters, and a change to any one number gives different characters.
Each model the practice API serves says which model it was made from. So you can rebuild the lineage by following those notes back to the start.
Worked example
Rebuild a lineage from the files
Start at the faulty model and follow each model's base note until there is none. Print the chain oldest first, with each model's hash. Then add the patch as the last link, and check its two hashes.
import wsp
chain = []
name = "tiny-gpt-loop"while name:
state, info = wsp.load_state(name)
parent = info["training"].get("base")
chain.append((name, wsp.state_hash(state), parent))
name = parent
for name, digest, parent inreversed(chain):
print(f"{name:13s} {digest} made from {parent or 'catalog text, by pretraining'}")
loop, info = wsp.load_state("tiny-gpt-loop")
patch = wsp.load_patch("loop-fix")
edited = wsp.apply_patch(loop, patch)
print(f"{'+ loop-fix':13s} {wsp.state_hash(edited)} made from {patch['base']} by a "f"patch of {patch['changed']:,} entries")
print("the patch names its parent:", patch["base"] == chain[0][1], "| and its result:",
patch["result"] == wsp.state_hash(edited))
It runs right here in your browser, with nothing to install.
Read the last two lines first. The patch says it fits a model with the hash ced602187d4e, and that is the hash of tiny-gpt-loop as served. The patch says it makes 5fedae1a442e, and applying it gives exactly that. So this link is proved by arithmetic on the files. The links above it are weaker. "Made from tiny-gpt" is a note written by whoever built the model. A hash proves which file you hold. It can't prove how the file was made. Good provenance says which of its links are proved and which are only stated.
A hash sees what tests can't
You could skip the hash and run your tests on the download instead. But tests sample a model's behavior, and a hash covers every number in it. The next cell makes a copy of the published fine-tune and moves one weight by a thousandth.
Worked example
A copy that passes every test
The published tiny-gpt-uses and a copy with one weight changed. For each: held-out answers right, replies that end, loss on catalog text, and the hash.
import wsp
uses, info = wsp.load_state("tiny-gpt-uses")
copy = {name: tensor.copy() for name, tensor in uses.items()}
# one weight of 40,195, moved by a thousandth
copy["blocks.0.mlp.2.weight"][0, 0] += 0.001
docs = wsp.catalog()
for label, state in [("published", uses), ("the copy", copy)]:
model, tok = wsp.build(state, info)
right, ended = wsp.score(model, tok, "uses"), sum(wsp.endings(model, tok, "uses"))
print(f"{label:9s} right {right} of 72 | replies that end {ended} | catalog loss "f"{wsp.text_loss(model, tok, docs):.2f} | hash {wsp.state_hash(state)}")
In the lesson you can change this and run it.
The copy answers all 72 questions, ends all 72 replies and writes catalog text with the same loss of 1.15. No check in this course can tell the two models apart. The hash can: 44214fa59350 against ff715f7bbcd8. This change was harmless. A change made on purpose could alter the answer to one input you never thought to test, and it would look the same on every test you ran. That is why an evaluation, a license check or a safety review is tied to a hash. It covers the file with that fingerprint and no other.
Try it now
How short can a fingerprint be? Start: the 12 characters this course prints, and a hub holding a million model files. Try 6 characters, then 16.
Quick check
Each step of a lineage is written as (parent hash, result hash). What does this print?
steps = [("a1", "b2"), ("b2", "c3"), ("c9", "d4")]
print([i for i inrange(1, len(steps)) if steps[i][0] != steps[i - 1][1]])
[2]
[0]
[1, 2]
[]
Show the answer
[2]
A chain holds when each step's parent is the result of the step before it. Step 1 starts from b2, which step 0 made. Step 2 claims to start from c9, but step 1 made c3. So the chain breaks at index 2: something happened between those two steps that nobody recorded.
Drill
Find the first broken link
Write first_break(steps). Each step is a dictionary with "name", "parent" and "result", oldest first. Return the name of the first step whose parent isn't the result of the step before it. The first step has nothing before it, so it can't break the chain. If every link holds, return None.
In the lesson, your code is checked as you write it.
In your own words
In your own words: a downloaded model passes every test you have. Why check its hash anyway?
Write a sentence or two, then compare it with a good answer in the lesson.
Whose rules follow the weights
The next question is who allows you to release this model at all.
Quick check
From Running open models locally: before you build a product on an open model, where do you look for the rules on using it?
In its benchmark scores, which show what it may do
In its license, which the model card names
In its weights, where the rules are kept as numbers
Nowhere: open weights are free for any use at all
Show the answer
In its license, which the model card names
Every open model comes with a license, and the model card says which. Some licenses allow almost anything. Others set conditions. The same license matters again now, because it follows the weights into anything you make from them.
A changed model keeps its parent's license
A model you edited is still mostly someone else's work. Your patch changed 1,024 numbers. The other 39,171 came from the people who trained the base. So the base model's license comes along with your patched model. Open-model licenses come in three rough families:
Permissive, such as Apache 2.0 and MIT. You may use, change and sell the model. The duties are light: keep the license text and its notices with the model, and under Apache 2.0 say what you changed. Qwen3.5-0.8B, the model edited in the post, is listed as Apache 2.0 on its model card.
Community licenses with conditions. Meta's Llama 3.1 license is one. It lets you build products, and it adds duties for anyone who shares a changed model: pass on the license, whose acceptable-use policy still applies, display "Built with Llama," and begin the new model's name with "Llama."
Non-commercial, such as CC BY-NC 4.0. You may study and share the model, with credit to its authors. You may not use it to make money.
A merged model has more than one parent, as in Merging models, and it has to meet every parent's terms at once. One non-commercial parent makes the whole merge non-commercial, and the parents' duties add up.
Watch out: This is an engineer's map, not legal advice. Before a real release, read the license itself and ask someone qualified.
Worked example
What a derived model inherits
Four licenses with their terms written as data. The first three are real, cut down to two fields. The fourth, trail-community-1.0, is invented for this lesson and modeled on community licenses. Then four derived models, and what each one inherits.
TERMS = {
"apache-2.0": {"commercial": True,
"duties": ["keep the license and notices", "say what you changed"]},
"mit": {"commercial": True, "duties": ["keep the license and notices"]},
"cc-by-nc-4.0": {"commercial": False,
"duties": ["credit the authors", "say what you changed"]},
"trail-community-1.0": {"commercial": True,
"duties": ["keep the license and notices",
"pass on the use policy",
"start the model's name with Trail"]},
}
definherited(parents):
commercial = all(TERMS[p]["commercial"] for p in parents)
duties = sorted({duty for p in parents for duty in TERMS[p]["duties"]})
return commercial, duties
cases = [("a patched Apache model", ["apache-2.0"]),
("a merge of an Apache and an MIT model", ["apache-2.0", "mit"]),
("a merge with one non-commercial parent", ["apache-2.0", "cc-by-nc-4.0"]),
("a patched community-license model", ["trail-community-1.0"])]
for label, parents in cases:
commercial, duties = inherited(parents)
print(f"{label}: commercial use {'allowed' if commercial else 'NOT allowed'}")
for duty in duties:
print(" -", duty)
In the lesson you can change this and run it.
A patch on an Apache model may be sold, and inherits two duties. Adding an MIT parent changes nothing here, because its one duty was already on the list. One non-commercial parent flips the answer for the whole merge, and brings a third duty: credit the authors. The community license allows commercial use, and it adds two duties that a quick reader misses. The use policy has to be passed on, and the name has to start a certain way. The function is short. The work is in reading each license closely enough to fill in the table.
Quick check
Three parents of a merge, and whether each one allows commercial use. What does this print?
parents = [{"commercial": True}, {"commercial": False}, {"commercial": True}]
print(all(p["commercial"] for p in parents), sum(p["commercial"] for p in parents))
True 3
False 2
True 2
False 1
Show the answer
False 2
all() is True only when every item is True. One parent says no, so the merge can't be sold, though two of the three would have allowed it. sum() counts the True values, and there are 2. A vote among the parents doesn't decide this. Each parent's license applies in full.
Drill
Find what blocks a commercial release
Write blockers(lineage, terms). lineage maps each parent model's name to its license id. terms maps a license id to a dictionary with a "commercial" flag. Return the sorted names of the parents that stand in the way of commercial use: those whose license says False, and also those whose license isn't in terms at all. A license nobody on your team has read is a blocker until someone reads it.
In the lesson, your code is checked as you write it.
In your own words
In your own words: why does one non-commercial parent decide the answer for a whole merge?
Write a sentence or two, then compare it with a good answer in the lesson.
In code: what an edit can do to a safeguard, and a guard outside the weights
Chat models are trained to behave well in certain ways: to stop when the answer is complete, to stay on the task, to refuse some harmful requests. Those habits are stored where everything else is stored, in the weights. You have spent this course changing weights.
Researchers have tested how firmly such a habit sits. A 2024 study looked at 13 open chat models. In each, the habit of refusing harmful requests depended on one direction in the residual stream. A small edit to the weights that removed the direction nearly ended the refusals, and the models' scores on ordinary benchmarks barely moved. An earlier study found that fine-tuning can do the same. Ten chosen examples were enough, and even ordinary, harmless training data wore the safeguards down to some extent.
This course doesn't do that to any model, and it gives no steps for it. What a builder needs is the consequence, and a harmless stand-in shows it. The tiny GPT has one trained-in good habit: it stops. tiny-gpt-uses learned in fine-tuning to end each reply with <end>. The patch's recipe names a direction tied to stopping. The next cell removes that one direction from every writer of the healthy model.
Worked example
One direction, and the habit of stopping
The healthy fine-tune, and the same model with one direction removed from all ten writer tensors. For each: held-out answers right, replies that end, and loss on catalog text.
import numpy as np
import wsp
uses, info = wsp.load_state("tiny-gpt-uses")
direction = np.array(wsp.load_patch("loop-fix")["recipe"]["direction"])
edited = wsp.project_out(uses, direction)
change = wsp.make_patch(uses, edited)
print(f"entries changed: {change['changed']:,} of {wsp.count(uses):,}, in "f"{len(change['tensors'])} tensors")
docs = wsp.catalog()
for label, state in [("as published", uses), ("direction removed", edited)]:
model, tok = wsp.build(state, info)
right, ended = wsp.score(model, tok, "uses"), sum(wsp.endings(model, tok, "uses"))
print(f"{label:17s} right {right:2d} of 72 | replies that end {ended:2d} | catalog "f"loss {wsp.text_loss(model, tok, docs):.2f}")
In the lesson you can change this and run it.
The edit touches 18,656 weights in the 10 writer tensors. It uses no data and no training. The healthy model answered 72 of 72 and ended every reply. Now it answers 9, and only 20 replies end. A habit that took 500 training steps to teach went in one subtraction. This blunt version leaves marks: loss on catalog text rose from 1.15 to 1.55. The study above reports edits to real models that left few. Either way the lesson for a builder is the same. What training puts into open weights, an edit can take out.
Quick check
A model card says the model was trained to refuse harmful requests. You downloaded a copy that someone has patched. What does the card's claim tell you about your copy?
It holds for any copy that has the same number of weights and loads without an error
It is void for every copy of the model, which is why open models can never be trusted
It still holds in full, since training a model to refuse can't be undone by a later edit
Nothing certain: it covers the published weights, so a patched copy has to be measured again
Show the answer
Nothing certain: it covers the published weights, so a patched copy has to be measured again
A claim on a card describes the weights its authors published, and those have a hash. A patched copy has a different hash and may behave differently. The claim may still be true of it, but only a new measurement can say so.
Three habits for a builder
Three working habits follow from that result.
Tie every claim to a hash. A safety review covers the weights it was run on. Publish the hash with the claim, and check the hash before you rely on someone else's.
Run guard checks around every edit. A guard check measures a behavior the model must keep: replies end, refusals hold, ordinary text still reads well. Measure before and after. Your own edit can weaken a safeguard without your meaning it to, as harmless training data did in the study. In Did it work? Testing an edit, the widened patch did that to stopping.
Keep a safeguard outside the weights. A rule in the program around the model can't be edited out of the model. Using more than one layer of protection is called defense in depth: if one layer fails, another is still there.
The next cell tries the third habit on the looping model. It leaves every weight alone and adds one rule in ordinary code.
Worked example
A safeguard outside the weights
A rule in the program: cut each reply after its first sentence. The looping model and the patched model are each scored raw, then with the rule applied to their replies.
import wsp
f = wsp.facts()
defguarded(model, tok, prompt):
"""The reply, cut after its first sentence: a rule the program applies, whatever the
weights do."""return wsp.answer(model, tok, prompt).split(".")[0].strip() + "."defright_with_guard(model, tok):
right = 0for name in f["held_out"]:
for product in f["products"]:
reply = guarded(model, tok, wsp.question("uses", name, product["type"]))
right += reply in [use + "."for use in product["uses"]]
return right
loop, info = wsp.load_state("tiny-gpt-loop")
model, tok = wsp.build(loop, info)
prompt = wsp.question("uses", "Basin", "tent")
print("raw: ", wsp.answer(model, tok, prompt))
print("guarded:", guarded(model, tok, prompt))
print(f"looping model: {wsp.score(model, tok, 'uses')} of 72 raw, "f"{right_with_guard(model, tok)} of 72 with the guard")
fixed, _ = wsp.build(wsp.apply_patch(loop, wsp.load_patch("loop-fix")), info)
print(f"patched model: {wsp.score(fixed, tok, 'uses')} of 72 raw, "f"{right_with_guard(fixed, tok)} of 72 with the guard")
In the lesson you can change this and run it.
The looping model, with no weight changed, goes from 0 right to 72 of 72. That beats the patch, which scores 68 raw. The rule lifts the patched model to 71 as well. A rule outside the weights is cheap and easy to test, and for a service you run yourself it is often the first thing to reach for. Its limit is where it lives. The rule is in your program. Anyone who downloads the weights gets the model without it. A published file carries only the safeguards that are in its numbers, and you have seen what an edit can do to those.
Quick check
Your team runs a patched model behind its own app, and also publishes the weights. Which safeguards does a person who downloads the weights receive?
The app's rules only, since trained habits can't be copied from one machine to another
All of them, since rules in the app are saved into the weights when a file is published
Only those in the weights, which an edit can weaken; the app's rules stay with the app
None at all, since publishing a model's weights always strips out its trained habits
Show the answer
Only those in the weights, which an edit can weaken; the app's rules stay with the app
A weights file holds numbers and nothing else. Trained habits travel with it, and so does anyone's ability to edit them. The rules in your app are code on your servers. They protect the people who use your service, and nobody else.
In your own words
In your own words: why isn't a safeguard that lives only in the weights enough for a model you publish?
Write a sentence or two, then compare it with a good answer in the lesson.
In production: the record you publish
Everything so far ends up in one short document that travels with the model. For a base model it is the model card from Running open models locally. A derived model's card has more to say, because the model has a history:
Base and hash. The exact model you started from.
License. The base's license, and the duties you met.
Change. How many weights, in which tensors, made how.
Result hash. So a stranger can confirm they hold this model.
Measurements. Held-out results with the verdict, and the guard checks before and after.
Limits. What you didn't measure.
The authors of the weight-space programming post released their Qwen edit this way. Their public logs hold the edit as an exact difference from the public base weights at one pinned revision. They give a hash of every edited tensor before and after, and a script that rebuilds the edited model and checks each hash.
Most of a card can be printed by the code that did the work. Then it can't drift from the facts.
Worked example
A card printed by the code that did the work
Apply the patch, measure, and build the card's lines from the results. Nothing in it is typed by hand except the labels and the last line.
import wsp
loop, info = wsp.load_state("tiny-gpt-loop")
patch = wsp.load_patch("loop-fix")
edited = wsp.apply_patch(loop, patch)
before, tok = wsp.build(loop, info)
after, _ = wsp.build(edited, info)
v = wsp.verdict(wsp.scores(before, tok, "uses"), wsp.scores(after, tok, "uses"))
docs = wsp.catalog()
card = [
"Model: tiny-gpt-loop + loop-fix",
f"Base: tiny-gpt-loop, hash {patch['base']}",
f"Change: {patch['changed']:,} of {wsp.count(loop):,} weights in "f"{len(patch['recipe']['columns'])} tensors, {patch['recipe']['made_by']}",
f"Result: hash {wsp.state_hash(edited)}",
f"Held-out answers right: {v['base'] * 72:.0f} -> {v['edited'] * 72:.0f} of 72 "f"({v['verdict']}: {v['better']} better, {v['worse']} worse)",
f"Guard checks: replies that end {sum(wsp.endings(before, tok, 'uses'))} -> "f"{sum(wsp.endings(after, tok, 'uses'))} of 72; catalog loss "f"{wsp.text_loss(before, tok, docs):.2f} -> {wsp.text_loss(after, tok, docs):.2f}",
"Not measured: any question except what a product is for",
]
for line in card:
print(line)
In the lesson you can change this and run it.
Seven lines, and every number in them came from a function call. A reader learns what the model was, what was done to it, and how to confirm they hold the result: hash 5fedae1a442e. The guard checks show both sides. All 72 replies end now, and catalog loss fell from 1.93 to 1.52, which is still above the 1.15 of the model that never had the fault. The last line matters as much as the others. This model was only ever tested on one kind of question, and the card says so. A real card would add the license line. These practice models don't carry one.
Quick check
Which line of the card lets a stranger confirm that the file they downloaded is the model the card describes?
The held-out score, which they can compare with a first impression
The result hash, which they can compute again from their own copy
The count of weights changed, which fits any patch of the same size
The model's name, which is copied along with the file to any mirror
Show the answer
The result hash, which they can compute again from their own copy
Anyone can hash the file they hold and compare. If the 12 characters match, every number matches. The other lines describe the model, and each of them would read the same on a copy that had been changed.
Who answers for it
A release also has an owner. If you changed a model and published it, you are the person who knows what changed, and questions about it come to you. Some laws now say this in formal terms.
Go deeper What the EU's rules say about modified models
The EU's AI Act puts duties on the provider of a general-purpose model. Guidelines from the European Commission, published in July 2025, cover people who modify someone else's model. A modifier becomes the provider of the modified model when the change is significant for the model's generality, capabilities or risk, and then only for the change they made. As a rough sign, the guidelines point to a modification that used more than a third of the computing the original training used. A weight edit uses almost none, so by that sign alone it falls below the line. The deciding question is still what the change did to the model. Rules like these get revised, so read the current text before a release in Europe.
The card is how you answer in advance. It says what you started from, what you did, what you measured and what you didn't.
Where it shows up in your week
Before you publish or deploy a model you changed, you list its parents and their licenses, you hash the file, you run the guard checks before and after, and you print the card from the code that ran them. A claim with no hash is a claim about some other file.
In your own words
In your own words: why does a card for a derived model say what was not measured?
Write a sentence or two, then compare it with a good answer in the lesson.
11DistillationTrain a small model to imitate a large one.35 min
12Tuning an embedding model for retrievalWhen search fails on your own vocabulary: train an adapter with a contrastive loss on question and passage pairs, test on held-out passages, use batch size and hard negatives, and measure what tuning costs elsewhere.45 min
13QuantizationStore weights in fewer bits to run models cheaply.35 min
23Copying from the promptIn-context learning as a two-head copying circuit: the base tiny GPT cannot copy a repeated pattern, a model trained on repeats grows an induction head and can.30 min
24Why temperature changes the answerThe same prompt at 0, 0.7 and 1.5: deterministic loops, plausible variety, broken grammar, measured on a real model.25 min
25Why long prompts diluteAttention is a budget shared by every token, so a fact far from the question loses its share; measured, with the cliff beyond the trained window.25 min
26Why it never says "I don't know"A chat model answers every request because its fine-tuning data always did; preference training adds flattery; how to build abstention in.25 min
32Weight-space programming: what it is, and what has been shownFreeOptionalA language model reads and writes another network's weights with no training loop: the published experiments, their limits and the criticism, and one edit you make by hand.30 min
33Anatomy of a checkpointOptionalA model file is a set of named tensors: count them, find the ones that write to the residual stream, and save one to bytes and back.35 min
34Reading weights without running the modelOptionalSingular values, and an attention head's token-to-token circuit, read from the matrices alone and then checked against a measurement.35 min
35Writing weights by handOptionalSet a small network's 55 weights so it catches every ball in a paddle game, the way a program would be written, and meet compiled transformers.40 min
36Edits are deltas: task vectors, patches and hashesOptionalSubtract a base from a fine-tune and the edit becomes an object: scale it, take it away, ship it as a patch, and prove by hash what was applied.35 min
37Merging modelsOptionalAverage two fine-tunes, add their task vectors, try SLERP, TIES and DARE, measure the interference, and compare with a model trained on both tasks.40 min
38Low-rank editsOptionalCut a fine-tune's change down to its largest directions with the SVD: how few a behavior needs, and how to pull an adapter out of a delta.35 min
39Directions: steer the run or edit the weightsOptionalFind a behavior's direction in the residual stream, add it at run time, project it out of the weights, and see the effect change from input to input.40 min
40Editing one factOptionalA rank-one change to one matrix rewrites what a product is made from; then reliability, generalization, locality, and what breaks after many edits.40 min
41The editor's harnessOptionalWhat an LLM that edits weights is given and what it is denied: weights as text and their cost in tokens, a sandbox, a budget of forward passes, and rules the harness enforces.40 min
42An LLM reads a network's weightsOptionalSix recorded sessions: an LLM is handed 1,315 weights as text and asked what the network was trained to do. Score each answer against the network, and study one it got wrong.40 min
43An LLM fixes a model's weightsOptionalFind a tiny GPT's loop by hand, then replay recorded sessions in which an LLM hunts for it with ten forward passes, and measure every fix on held-out questions.45 min
44An LLM writes a network's weightsOptionalFive recorded sessions: an LLM writes every weight of a paddle network from the game's source and one sentence. Score each against its specification, test its design claims, and edit one by hand.40 min
45Did it work? Testing an editOptionalReview a patch for a tiny GPT that can't stop talking: held-out questions, a paired comparison, a same-size control, and what the edit cost.40 min
46The published edit, on a real modelOptionalRead the edit published for an open model of 870 million weights, apply and verify one in the same form, see what 16-bit and 4-bit storage do to a fix, and run the real thing from a notebook.45 min
47Safety, licenses and provenanceOptionalBefore a custom model is released: trace its lineage by hash, work out what its parents' licenses pass on, see what an edit can do to a safeguard, and write its card.40 min
48Capstone: ship a custom modelOptionalSet release gates, then diagnose, edit, measure and document a tiny GPT with a fault no lesson has fixed, and write the record a stranger could check.45 min
Read a model's weights, change them by hand and judge an edit someone else made: deltas, merging, low-rank edits, directions and fact edits on a tiny GPT in your browser. Then replay recorded sessions of an LLM reading, fixing and writing weights, check the published edit of a real open model, and ship a model of your own with its patch, verdict and card. Every number is printed by the lesson's own code, and the method's limits are taught with it.
See the course →
Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write