Index Ai Academy
Phase 6 · Editing models

Safety, licenses and provenance

Before a custom model is released: trace its lineage by hash, work out what its parents' licenses pass on, see what an edit can do to a safeguard, and write its card.

About 40 minutes · Optional lesson: deeper material that no phase or certificate requires · A free account opens the whole lesson.
Already have an account? Sign in to open this lesson

What it covers11

Why it matters

You can change what a model does by changing its numbers, and you can show that the change worked. One step is left before anyone else may rely on your model: releasing it. The person who downloads it can't see how you worked. They get a file, and whatever you wrote beside it.

Three questions decide whether they can trust that file. Where did it come from? Whose rules come with it? And what did the edit do to behavior they count on? An edited model is easy to make and easy to pass around, so these questions are asked of every team that ships one.

What you'll be able to do

  • Trace a model's lineage from its files, and check each link by hash
  • Work out what a patched or merged model inherits from the licenses of its parents
  • Say what an edit can do to a safeguard kept in the weights, and add one outside them
  • Write the record a derived model ships with: base, change, hashes, measurements and limits
Quick check
A model's page says it passed a safety review. You download the file from a mirror site. A guess: what ties that review to the file now on your disk?
  1. The file's name, which the mirror copied from the original
  2. A hash published with the review, which your file has to match
  3. The file's size, since any change would make the file larger
  4. Nothing is needed: a copy of a file is always the same file
Show the answer

A hash published with the review, which your file has to match

A review describes one exact set of numbers. A hash is a short fingerprint of those numbers. If your file's hash matches the published one, you hold what was reviewed. A name or a size can match on a file that someone changed.

The idea: lineage, hashes and inherited licenses

This lesson rests on Edits are deltas: task vectors, patches and hashes: a hash covers every number in a model, and a patch names its parent and its result by hash.

Where a model came from

The model you reviewed in Did it work? Testing an edit has a history. The base model tiny-gpt was pretrained on catalog text. tiny-gpt-uses was fine-tuned from it. tiny-gpt-loop is that fine-tune with a fault planted in it. The patch loop-fix makes a fourth model from the third.

That history is the model's lineage: every model it was made from, in order. A written record of the lineage, with proof, is its provenance. The proof is the hash from Edits are deltas: task vectors, patches and hashes. wsp.state_hash turns all 40,195 numbers into 12 characters, and a change to any one number gives different characters.

Each model the practice API serves says which model it was made from. So you can rebuild the lineage by following those notes back to the start.

Worked example

Rebuild a lineage from the files

Start at the faulty model and follow each model's base note until there is none. Print the chain oldest first, with each model's hash. Then add the patch as the last link, and check its two hashes.

import wsp

chain = []
name = "tiny-gpt-loop"
while name:
    state, info = wsp.load_state(name)
    parent = info["training"].get("base")
    chain.append((name, wsp.state_hash(state), parent))
    name = parent
for name, digest, parent in reversed(chain):
    print(f"{name:13s} {digest}  made from {parent or 'catalog text, by pretraining'}")

loop, info = wsp.load_state("tiny-gpt-loop")
patch = wsp.load_patch("loop-fix")
edited = wsp.apply_patch(loop, patch)
print(f"{'+ loop-fix':13s} {wsp.state_hash(edited)}  made from {patch['base']} by a "
      f"patch of {patch['changed']:,} entries")
print("the patch names its parent:", patch["base"] == chain[0][1], "| and its result:",
      patch["result"] == wsp.state_hash(edited))
It runs right here in your browser, with nothing to install.

Read the last two lines first. The patch says it fits a model with the hash ced602187d4e, and that is the hash of tiny-gpt-loop as served. The patch says it makes 5fedae1a442e, and applying it gives exactly that. So this link is proved by arithmetic on the files. The links above it are weaker. "Made from tiny-gpt" is a note written by whoever built the model. A hash proves which file you hold. It can't prove how the file was made. Good provenance says which of its links are proved and which are only stated.

A hash sees what tests can't

You could skip the hash and run your tests on the download instead. But tests sample a model's behavior, and a hash covers every number in it. The next cell makes a copy of the published fine-tune and moves one weight by a thousandth.

Worked example

A copy that passes every test

The published tiny-gpt-uses and a copy with one weight changed. For each: held-out answers right, replies that end, loss on catalog text, and the hash.

import wsp

uses, info = wsp.load_state("tiny-gpt-uses")
copy = {name: tensor.copy() for name, tensor in uses.items()}
# one weight of 40,195, moved by a thousandth
copy["blocks.0.mlp.2.weight"][0, 0] += 0.001
docs = wsp.catalog()
for label, state in [("published", uses), ("the copy", copy)]:
    model, tok = wsp.build(state, info)
    right, ended = wsp.score(model, tok, "uses"), sum(wsp.endings(model, tok, "uses"))
    print(f"{label:9s} right {right} of 72 | replies that end {ended} | catalog loss "
          f"{wsp.text_loss(model, tok, docs):.2f} | hash {wsp.state_hash(state)}")
In the lesson you can change this and run it.

The copy answers all 72 questions, ends all 72 replies and writes catalog text with the same loss of 1.15. No check in this course can tell the two models apart. The hash can: 44214fa59350 against ff715f7bbcd8. This change was harmless. A change made on purpose could alter the answer to one input you never thought to test, and it would look the same on every test you ran. That is why an evaluation, a license check or a safety review is tied to a hash. It covers the file with that fingerprint and no other.

Try it now

How short can a fingerprint be? Start: the 12 characters this course prints, and a hub holding a million model files. Try 6 characters, then 16.

Quick check
Each step of a lineage is written as (parent hash, result hash). What does this print?
steps = [("a1", "b2"), ("b2", "c3"), ("c9", "d4")]
print([i for i in range(1, len(steps)) if steps[i][0] != steps[i - 1][1]])
  1. [2]
  2. [0]
  3. [1, 2]
  4. []
Show the answer

[2]

A chain holds when each step's parent is the result of the step before it. Step 1 starts from b2, which step 0 made. Step 2 claims to start from c9, but step 1 made c3. So the chain breaks at index 2: something happened between those two steps that nobody recorded.

Drill

Write first_break(steps). Each step is a dictionary with "name", "parent" and "result", oldest first. Return the name of the first step whose parent isn't the result of the step before it. The first step has nothing before it, so it can't break the chain. If every link holds, return None.

def first_break(steps):
    return None

steps = [
    {"name": "tiny-gpt-uses", "parent": "8f40d347103f", "result": "44214fa59350"},
    {"name": "tiny-gpt-loop", "parent": "44214fa59350", "result": "ced602187d4e"},
    {"name": "loop-fix", "parent": "ced602187d4e", "result": "5fedae1a442e"},
]
print(first_break(steps))
steps[2]["parent"] = "ff715f7bbcd8"
print(first_break(steps))
In the lesson, your code is checked as you write it.
In your own words
In your own words: a downloaded model passes every test you have. Why check its hash anyway?

Write a sentence or two, then compare it with a good answer in the lesson.

Whose rules follow the weights

The next question is who allows you to release this model at all.

Quick check
From Running open models locally: before you build a product on an open model, where do you look for the rules on using it?
  1. In its benchmark scores, which show what it may do
  2. In its license, which the model card names
  3. In its weights, where the rules are kept as numbers
  4. Nowhere: open weights are free for any use at all
Show the answer

In its license, which the model card names

Every open model comes with a license, and the model card says which. Some licenses allow almost anything. Others set conditions. The same license matters again now, because it follows the weights into anything you make from them.

A changed model keeps its parent's license

A model you edited is still mostly someone else's work. Your patch changed 1,024 numbers. The other 39,171 came from the people who trained the base. So the base model's license comes along with your patched model. Open-model licenses come in three rough families:

  • Permissive, such as Apache 2.0 and MIT. You may use, change and sell the model. The duties are light: keep the license text and its notices with the model, and under Apache 2.0 say what you changed. Qwen3.5-0.8B, the model edited in the post, is listed as Apache 2.0 on its model card.
  • Community licenses with conditions. Meta's Llama 3.1 license is one. It lets you build products, and it adds duties for anyone who shares a changed model: pass on the license, whose acceptable-use policy still applies, display "Built with Llama," and begin the new model's name with "Llama."
  • Non-commercial, such as CC BY-NC 4.0. You may study and share the model, with credit to its authors. You may not use it to make money.

A merged model has more than one parent, as in Merging models, and it has to meet every parent's terms at once. One non-commercial parent makes the whole merge non-commercial, and the parents' duties add up.

Watch out: This is an engineer's map, not legal advice. Before a real release, read the license itself and ask someone qualified.

Worked example

What a derived model inherits

Four licenses with their terms written as data. The first three are real, cut down to two fields. The fourth, trail-community-1.0, is invented for this lesson and modeled on community licenses. Then four derived models, and what each one inherits.

TERMS = {
    "apache-2.0": {"commercial": True,
                   "duties": ["keep the license and notices", "say what you changed"]},
    "mit": {"commercial": True, "duties": ["keep the license and notices"]},
    "cc-by-nc-4.0": {"commercial": False,
                     "duties": ["credit the authors", "say what you changed"]},
    "trail-community-1.0": {"commercial": True,
                            "duties": ["keep the license and notices",
                                       "pass on the use policy",
                                       "start the model's name with Trail"]},
}

def inherited(parents):
    commercial = all(TERMS[p]["commercial"] for p in parents)
    duties = sorted({duty for p in parents for duty in TERMS[p]["duties"]})
    return commercial, duties

cases = [("a patched Apache model", ["apache-2.0"]),
         ("a merge of an Apache and an MIT model", ["apache-2.0", "mit"]),
         ("a merge with one non-commercial parent", ["apache-2.0", "cc-by-nc-4.0"]),
         ("a patched community-license model", ["trail-community-1.0"])]
for label, parents in cases:
    commercial, duties = inherited(parents)
    print(f"{label}: commercial use {'allowed' if commercial else 'NOT allowed'}")
    for duty in duties:
        print("   -", duty)
In the lesson you can change this and run it.

A patch on an Apache model may be sold, and inherits two duties. Adding an MIT parent changes nothing here, because its one duty was already on the list. One non-commercial parent flips the answer for the whole merge, and brings a third duty: credit the authors. The community license allows commercial use, and it adds two duties that a quick reader misses. The use policy has to be passed on, and the name has to start a certain way. The function is short. The work is in reading each license closely enough to fill in the table.

Quick check
Three parents of a merge, and whether each one allows commercial use. What does this print?
parents = [{"commercial": True}, {"commercial": False}, {"commercial": True}]
print(all(p["commercial"] for p in parents), sum(p["commercial"] for p in parents))
  1. True 3
  2. False 2
  3. True 2
  4. False 1
Show the answer

False 2

all() is True only when every item is True. One parent says no, so the merge can't be sold, though two of the three would have allowed it. sum() counts the True values, and there are 2. A vote among the parents doesn't decide this. Each parent's license applies in full.

Drill

Find what blocks a commercial release

Write blockers(lineage, terms). lineage maps each parent model's name to its license id. terms maps a license id to a dictionary with a "commercial" flag. Return the sorted names of the parents that stand in the way of commercial use: those whose license says False, and also those whose license isn't in terms at all. A license nobody on your team has read is a blocker until someone reads it.

def blockers(lineage, terms):
    return []

terms = {"apache-2.0": {"commercial": True}, "cc-by-nc-4.0": {"commercial": False}}
lineage = {"trail-base": "apache-2.0", "trail-poems": "cc-by-nc-4.0",
           "trail-chat": "summit-open-2.0"}
print(blockers(lineage, terms))
In the lesson, your code is checked as you write it.
In your own words
In your own words: why does one non-commercial parent decide the answer for a whole merge?

Write a sentence or two, then compare it with a good answer in the lesson.

In code: what an edit can do to a safeguard, and a guard outside the weights

Chat models are trained to behave well in certain ways: to stop when the answer is complete, to stay on the task, to refuse some harmful requests. Those habits are stored where everything else is stored, in the weights. You have spent this course changing weights.

Researchers have tested how firmly such a habit sits. A 2024 study looked at 13 open chat models. In each, the habit of refusing harmful requests depended on one direction in the residual stream. A small edit to the weights that removed the direction nearly ended the refusals, and the models' scores on ordinary benchmarks barely moved. An earlier study found that fine-tuning can do the same. Ten chosen examples were enough, and even ordinary, harmless training data wore the safeguards down to some extent.

This course doesn't do that to any model, and it gives no steps for it. What a builder needs is the consequence, and a harmless stand-in shows it. The tiny GPT has one trained-in good habit: it stops. tiny-gpt-uses learned in fine-tuning to end each reply with <end>. The patch's recipe names a direction tied to stopping. The next cell removes that one direction from every writer of the healthy model.

Worked example

One direction, and the habit of stopping

The healthy fine-tune, and the same model with one direction removed from all ten writer tensors. For each: held-out answers right, replies that end, and loss on catalog text.

import numpy as np
import wsp

uses, info = wsp.load_state("tiny-gpt-uses")
direction = np.array(wsp.load_patch("loop-fix")["recipe"]["direction"])
edited = wsp.project_out(uses, direction)
change = wsp.make_patch(uses, edited)
print(f"entries changed: {change['changed']:,} of {wsp.count(uses):,}, in "
      f"{len(change['tensors'])} tensors")
docs = wsp.catalog()
for label, state in [("as published", uses), ("direction removed", edited)]:
    model, tok = wsp.build(state, info)
    right, ended = wsp.score(model, tok, "uses"), sum(wsp.endings(model, tok, "uses"))
    print(f"{label:17s} right {right:2d} of 72 | replies that end {ended:2d} | catalog "
          f"loss {wsp.text_loss(model, tok, docs):.2f}")
In the lesson you can change this and run it.

The edit touches 18,656 weights in the 10 writer tensors. It uses no data and no training. The healthy model answered 72 of 72 and ended every reply. Now it answers 9, and only 20 replies end. A habit that took 500 training steps to teach went in one subtraction. This blunt version leaves marks: loss on catalog text rose from 1.15 to 1.55. The study above reports edits to real models that left few. Either way the lesson for a builder is the same. What training puts into open weights, an edit can take out.

Quick check
A model card says the model was trained to refuse harmful requests. You downloaded a copy that someone has patched. What does the card's claim tell you about your copy?
  1. It holds for any copy that has the same number of weights and loads without an error
  2. It is void for every copy of the model, which is why open models can never be trusted
  3. It still holds in full, since training a model to refuse can't be undone by a later edit
  4. Nothing certain: it covers the published weights, so a patched copy has to be measured again
Show the answer

Nothing certain: it covers the published weights, so a patched copy has to be measured again

A claim on a card describes the weights its authors published, and those have a hash. A patched copy has a different hash and may behave differently. The claim may still be true of it, but only a new measurement can say so.

Three habits for a builder

Three working habits follow from that result.

  1. Tie every claim to a hash. A safety review covers the weights it was run on. Publish the hash with the claim, and check the hash before you rely on someone else's.
  2. Run guard checks around every edit. A guard check measures a behavior the model must keep: replies end, refusals hold, ordinary text still reads well. Measure before and after. Your own edit can weaken a safeguard without your meaning it to, as harmless training data did in the study. In Did it work? Testing an edit, the widened patch did that to stopping.
  3. Keep a safeguard outside the weights. A rule in the program around the model can't be edited out of the model. Using more than one layer of protection is called defense in depth: if one layer fails, another is still there.

The next cell tries the third habit on the looping model. It leaves every weight alone and adds one rule in ordinary code.

Worked example

A safeguard outside the weights

A rule in the program: cut each reply after its first sentence. The looping model and the patched model are each scored raw, then with the rule applied to their replies.

import wsp

f = wsp.facts()

def guarded(model, tok, prompt):
    """The reply, cut after its first sentence: a rule the program applies, whatever the
    weights do."""
    return wsp.answer(model, tok, prompt).split(".")[0].strip() + "."

def right_with_guard(model, tok):
    right = 0
    for name in f["held_out"]:
        for product in f["products"]:
            reply = guarded(model, tok, wsp.question("uses", name, product["type"]))
            right += reply in [use + "." for use in product["uses"]]
    return right

loop, info = wsp.load_state("tiny-gpt-loop")
model, tok = wsp.build(loop, info)
prompt = wsp.question("uses", "Basin", "tent")
print("raw:    ", wsp.answer(model, tok, prompt))
print("guarded:", guarded(model, tok, prompt))
print(f"looping model: {wsp.score(model, tok, 'uses')} of 72 raw, "
      f"{right_with_guard(model, tok)} of 72 with the guard")
fixed, _ = wsp.build(wsp.apply_patch(loop, wsp.load_patch("loop-fix")), info)
print(f"patched model: {wsp.score(fixed, tok, 'uses')} of 72 raw, "
      f"{right_with_guard(fixed, tok)} of 72 with the guard")
In the lesson you can change this and run it.

The looping model, with no weight changed, goes from 0 right to 72 of 72. That beats the patch, which scores 68 raw. The rule lifts the patched model to 71 as well. A rule outside the weights is cheap and easy to test, and for a service you run yourself it is often the first thing to reach for. Its limit is where it lives. The rule is in your program. Anyone who downloads the weights gets the model without it. A published file carries only the safeguards that are in its numbers, and you have seen what an edit can do to those.

Quick check
Your team runs a patched model behind its own app, and also publishes the weights. Which safeguards does a person who downloads the weights receive?
  1. The app's rules only, since trained habits can't be copied from one machine to another
  2. All of them, since rules in the app are saved into the weights when a file is published
  3. Only those in the weights, which an edit can weaken; the app's rules stay with the app
  4. None at all, since publishing a model's weights always strips out its trained habits
Show the answer

Only those in the weights, which an edit can weaken; the app's rules stay with the app

A weights file holds numbers and nothing else. Trained habits travel with it, and so does anyone's ability to edit them. The rules in your app are code on your servers. They protect the people who use your service, and nobody else.

In your own words
In your own words: why isn't a safeguard that lives only in the weights enough for a model you publish?

Write a sentence or two, then compare it with a good answer in the lesson.

In production: the record you publish

Everything so far ends up in one short document that travels with the model. For a base model it is the model card from Running open models locally. A derived model's card has more to say, because the model has a history:

  • Base and hash. The exact model you started from.
  • License. The base's license, and the duties you met.
  • Change. How many weights, in which tensors, made how.
  • Result hash. So a stranger can confirm they hold this model.
  • Measurements. Held-out results with the verdict, and the guard checks before and after.
  • Limits. What you didn't measure.

The authors of the weight-space programming post released their Qwen edit this way. Their public logs hold the edit as an exact difference from the public base weights at one pinned revision. They give a hash of every edited tensor before and after, and a script that rebuilds the edited model and checks each hash.

Most of a card can be printed by the code that did the work. Then it can't drift from the facts.

Worked example

A card printed by the code that did the work

Apply the patch, measure, and build the card's lines from the results. Nothing in it is typed by hand except the labels and the last line.

import wsp

loop, info = wsp.load_state("tiny-gpt-loop")
patch = wsp.load_patch("loop-fix")
edited = wsp.apply_patch(loop, patch)
before, tok = wsp.build(loop, info)
after, _ = wsp.build(edited, info)
v = wsp.verdict(wsp.scores(before, tok, "uses"), wsp.scores(after, tok, "uses"))
docs = wsp.catalog()
card = [
    "Model: tiny-gpt-loop + loop-fix",
    f"Base: tiny-gpt-loop, hash {patch['base']}",
    f"Change: {patch['changed']:,} of {wsp.count(loop):,} weights in "
    f"{len(patch['recipe']['columns'])} tensors, {patch['recipe']['made_by']}",
    f"Result: hash {wsp.state_hash(edited)}",
    f"Held-out answers right: {v['base'] * 72:.0f} -> {v['edited'] * 72:.0f} of 72 "
    f"({v['verdict']}: {v['better']} better, {v['worse']} worse)",
    f"Guard checks: replies that end {sum(wsp.endings(before, tok, 'uses'))} -> "
    f"{sum(wsp.endings(after, tok, 'uses'))} of 72; catalog loss "
    f"{wsp.text_loss(before, tok, docs):.2f} -> {wsp.text_loss(after, tok, docs):.2f}",
    "Not measured: any question except what a product is for",
]
for line in card:
    print(line)
In the lesson you can change this and run it.

Seven lines, and every number in them came from a function call. A reader learns what the model was, what was done to it, and how to confirm they hold the result: hash 5fedae1a442e. The guard checks show both sides. All 72 replies end now, and catalog loss fell from 1.93 to 1.52, which is still above the 1.15 of the model that never had the fault. The last line matters as much as the others. This model was only ever tested on one kind of question, and the card says so. A real card would add the license line. These practice models don't carry one.

Quick check
Which line of the card lets a stranger confirm that the file they downloaded is the model the card describes?
  1. The held-out score, which they can compare with a first impression
  2. The result hash, which they can compute again from their own copy
  3. The count of weights changed, which fits any patch of the same size
  4. The model's name, which is copied along with the file to any mirror
Show the answer

The result hash, which they can compute again from their own copy

Anyone can hash the file they hold and compare. If the 12 characters match, every number matches. The other lines describe the model, and each of them would read the same on a copy that had been changed.

Who answers for it

A release also has an owner. If you changed a model and published it, you are the person who knows what changed, and questions about it come to you. Some laws now say this in formal terms.

Go deeper What the EU's rules say about modified models

The EU's AI Act puts duties on the provider of a general-purpose model. Guidelines from the European Commission, published in July 2025, cover people who modify someone else's model. A modifier becomes the provider of the modified model when the change is significant for the model's generality, capabilities or risk, and then only for the change they made. As a rough sign, the guidelines point to a modification that used more than a third of the computing the original training used. A weight edit uses almost none, so by that sign alone it falls below the line. The deciding question is still what the change did to the model. Rules like these get revised, so read the current text before a release in Europe.

The card is how you answer in advance. It says what you started from, what you did, what you measured and what you didn't.

Where it shows up in your week

Before you publish or deploy a model you changed, you list its parents and their licenses, you hash the file, you run the guard checks before and after, and you print the card from the code that ran them. A claim with no hash is a claim about some other file.

In your own words
In your own words: why does a card for a derived model say what was not measured?

Write a sentence or two, then compare it with a good answer in the lesson.

Goes deeper in

Practice what you just read

  • 6 code examples you can edit and run
  • a practice exercise that checks your code
  • a 5-question quiz
  • 9 review cards for spaced repetition

Try a quiz question

You hash a downloaded model and the result matches the hash on its model card. What have you shown?

  1. That your file holds exactly the numbers the card's results and reviews were about
  2. That the model is safe to use for any purpose, since a hash only matches for safe files
  3. That the model was trained in the way its card says, with the data the card describes
  4. That the model will give correct answers to the questions you plan to put to it

Answer it at the end of the lesson to see if you're right, and why.

Before this lesson

Go deeper, if you like

Optional lessons that build on this one. They never block a phase or a certificate.

Keep learning

Where this lesson leads.

575 lessons in thirteen phases. 176 are free without an account, and a free account opens the rest. See every lesson.

Phase 6 · about 35 hours of study

Modern LLM engineering

Post-training, fine-tuning, serving and safety 48 lessons 2 free
  1. Base models vs assistants FreeWhat post-training adds to a pretrained model.30 min
  2. Supervised fine-tuningTeach behavior from example conversations.40 min
  3. Instruction and preference dataThe datasets that shape assistant behavior.30 min
  4. RLHFReward models and reinforcement learning from human feedback.40 min
  5. DPO and direct preference methodsPreference training without a separate reward model.40 min
  6. Constitutional AITraining with AI feedback guided by written principles.35 min
  7. Training reasoning modelsReinforcement learning on verifiable problems.40 min
  8. Prompting vs RAG vs fine-tuningPick the cheapest approach that works.30 min
  9. LoRAFine-tune a big model by training small low-rank updates.40 min
  10. Fine-tune an open modelRun a real fine-tune and evaluate the result.45 min
  11. DistillationTrain a small model to imitate a large one.35 min
  12. Tuning an embedding model for retrievalWhen search fails on your own vocabulary: train an adapter with a contrastive loss on question and passage pairs, test on held-out passages, use batch size and hard negatives, and measure what tuning costs elsewhere.45 min
  13. QuantizationStore weights in fewer bits to run models cheaply.35 min
  14. Quantization in practice: zero points, calibration and formatsScale and zero point, per-channel scales, calibration sets, error on a layer's output, quantization-aware training and the file formats you meet at work.35 min
  15. Serving modelsBatching, throughput and latency in production.35 min
  16. One request through a serving stack: latency and costFollow one request through queue, prefill, a prefix cache, decode and the stream, and price a token from a GPU's hourly cost.35 min
  17. Speculative decodingDraft with a small model, verify with a big one.35 min
  18. Running open models locallyDownload, quantize and run a model on your own machine.30 min
  19. Will it fit? Memory for serving and fine-tuningWeights, KV cache, runtime and adapter states added up for a model card and a GPU, with a calculator.30 min
  20. InterpretabilityLooking inside models to see what they compute.40 min
  21. Confident and wrongWhy a model answers with the same assurance whether it is right, wrong or asked about something that does not exist.30 min
  22. Why it can't spell or count lettersThe model sees token IDs, and an embedding learned from usage carries nothing about spelling.25 min
  23. Copying from the promptIn-context learning as a two-head copying circuit: the base tiny GPT cannot copy a repeated pattern, a model trained on repeats grows an induction head and can.30 min
  24. Why temperature changes the answerThe same prompt at 0, 0.7 and 1.5: deterministic loops, plausible variety, broken grammar, measured on a real model.25 min
  25. Why long prompts diluteAttention is a budget shared by every token, so a fact far from the question loses its share; measured, with the cliff beyond the trained window.25 min
  26. Why it never says "I don't know"A chat model answers every request because its fine-tuning data always did; preference training adds flattery; how to build abstention in.25 min
  27. Benchmarks and their limitsWhat leaderboards measure, and what they miss.38 min
  28. Alignment and safetyMaking capable models behave as intended.35 min
  29. Evaluating agentsMeasure multi-step, tool-using behavior.35 min
  30. Reading AI papersGet the idea out of a research paper in 30 minutes.30 min
  31. Capstone: a tuned, evaluated assistantFine-tune, evaluate and deploy a specialist assistant.45 min
  32. Weight-space programming: what it is, and what has been shown Free OptionalA language model reads and writes another network's weights with no training loop: the published experiments, their limits and the criticism, and one edit you make by hand.30 min
  33. Anatomy of a checkpoint OptionalA model file is a set of named tensors: count them, find the ones that write to the residual stream, and save one to bytes and back.35 min
  34. Reading weights without running the model OptionalSingular values, and an attention head's token-to-token circuit, read from the matrices alone and then checked against a measurement.35 min
  35. Writing weights by hand OptionalSet a small network's 55 weights so it catches every ball in a paddle game, the way a program would be written, and meet compiled transformers.40 min
  36. Edits are deltas: task vectors, patches and hashes OptionalSubtract a base from a fine-tune and the edit becomes an object: scale it, take it away, ship it as a patch, and prove by hash what was applied.35 min
  37. Merging models OptionalAverage two fine-tunes, add their task vectors, try SLERP, TIES and DARE, measure the interference, and compare with a model trained on both tasks.40 min
  38. Low-rank edits OptionalCut a fine-tune's change down to its largest directions with the SVD: how few a behavior needs, and how to pull an adapter out of a delta.35 min
  39. Directions: steer the run or edit the weights OptionalFind a behavior's direction in the residual stream, add it at run time, project it out of the weights, and see the effect change from input to input.40 min
  40. Editing one fact OptionalA rank-one change to one matrix rewrites what a product is made from; then reliability, generalization, locality, and what breaks after many edits.40 min
  41. The editor's harness OptionalWhat an LLM that edits weights is given and what it is denied: weights as text and their cost in tokens, a sandbox, a budget of forward passes, and rules the harness enforces.40 min
  42. An LLM reads a network's weights OptionalSix recorded sessions: an LLM is handed 1,315 weights as text and asked what the network was trained to do. Score each answer against the network, and study one it got wrong.40 min
  43. An LLM fixes a model's weights OptionalFind a tiny GPT's loop by hand, then replay recorded sessions in which an LLM hunts for it with ten forward passes, and measure every fix on held-out questions.45 min
  44. An LLM writes a network's weights OptionalFive recorded sessions: an LLM writes every weight of a paddle network from the game's source and one sentence. Score each against its specification, test its design claims, and edit one by hand.40 min
  45. Did it work? Testing an edit OptionalReview a patch for a tiny GPT that can't stop talking: held-out questions, a paired comparison, a same-size control, and what the edit cost.40 min
  46. The published edit, on a real model OptionalRead the edit published for an open model of 870 million weights, apply and verify one in the same form, see what 16-bit and 4-bit storage do to a fix, and run the real thing from a notebook.45 min
  47. Safety, licenses and provenance OptionalBefore a custom model is released: trace its lineage by hash, work out what its parents' licenses pass on, see what an edit can do to a safeguard, and write its card.40 min
  48. Capstone: ship a custom model OptionalSet release gates, then diagnose, edit, measure and document a tiny GPT with a fault no lesson has fixed, and write the record a stranger could check.45 min
Part of a free course 17 lessons

Creating Custom LLMs: Editing Open Weights Using Weight-Space Programming

Read a model's weights, change them by hand and judge an edit someone else made: deltas, merging, low-rank edits, directions and fact edits on a tiny GPT in your browser. Then replay recorded sessions of an LLM reading, fixing and writing weights, check the published edit of a real open model, and ship a model of your own with its patch, verdict and card. Every number is printed by the lesson's own code, and the method's limits are taught with it.

Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write