Curation, active learning and synthetic data
Three levers on a training set: removing copies and broken rows, letting a model choose which rows to label, and manufacturing rows, which helps only when their source knows more than you do.
What it covers6
Why it matters
Fine-tuning a model, or training one from scratch, starts with a pile of candidate examples and a budget too small to label or keep them all. Which rows go in decides more than the choice of model does. This lesson covers the three levers you have: curation, which removes rows that teach nothing or teach the wrong thing; active learning, which lets the model point at the rows worth labeling; and synthetic data, which manufactures rows, and helps only when the thing making them knows more than the data you already hold.
What you'll be able to do
- Curate a training set: remove duplicates and broken rows, and say why copies add little
- Run an active learning loop that spends a labeling budget on the rows a model is least sure about
- Explain why an actively chosen set can't be used to estimate accuracy or class balance
- Tell apart three kinds of synthetic data, and predict which of them can add information
- Check a batch of generated examples for diversity and leakage before it reaches training
Show the answer
The ones the current model is least sure about
A row the model already classifies with confidence tells it little it doesn't know. A row it scores near fifty-fifty sits close to its decision boundary, and a label there moves the boundary. Choosing rows this way is called active learning, and this lesson measures how much it buys and what it costs.
The idea: fewer, better rows, and labels chosen by the model
A copy of a row teaches nothing new, so the first sums here are counts of distinct rows and shares of a pool. They use Percentages. The first check recalls what a good test set is.
Curation: fewer, better rows
Curation is deciding which candidate rows go into a training set. It sounds like housekeeping, and it changes results, because the usual assumption that more rows mean more information fails in two common cases: the rows are copies of each other, or they are broken.
Start with copies. Scraped pages, retried requests and templated messages fill real datasets with rows that differ by a space or a capital letter. In the cell below, a pool of 5,000 rows contains a growing share of near copies of the same 40 originals. A model is trained on 200 rows drawn at random from the pool, as if 200 labels had been paid for.
import numpy as np from sklearn.linear_model import LogisticRegression W = np.array([2.0, -1.5, 1.0, 0.5, 0, 0, 0, 0, 0, 0]) def make(n, seed): g = np.random.default_rng(seed) X = g.normal(size=(n, 10)) return X, (X @ W + g.normal(scale=0.5, size=n) > 0).astype(int) X_test, y_test = make(5000, 1) for copies in (0.0, 0.5, 0.8, 0.95): scores, distinct = [], [] for rep in range(20): g = np.random.default_rng(100 + rep) X, y = make(5000, 50 + rep) n = int(copies * 5000) # every copy is of one of 40 originals source = g.integers(0, 40, size=n) X_pool = np.vstack( [X[: 5000 - n], X[source] + g.normal(scale=0.01, size=(n, 10))] ) y_pool = np.concatenate([y[: 5000 - n], y[source]]) origin = np.concatenate([np.arange(5000 - n) + 1000, source]) # the 200 rows that get labeled chosen = g.permutation(5000)[:200] model = LogisticRegression().fit(X_pool[chosen], y_pool[chosen]) scores.append(model.score(X_test, y_test)) distinct.append(len(set(origin[chosen]))) print( "share of the pool that is copies:", str(copies).ljust(4), "| distinct rows among the 200:", str(round(float(np.mean(distinct)))).rjust(3), "| accuracy:", round(float(np.mean(scores)), 3), )
The budget was 200 labels on every line. With no copies it bought 200 distinct rows and an accuracy of 0.926. With 95% of the pool made of copies it bought 50 distinct rows and 0.87, while the invoice still said 200. A copy costs as much to label, store and train on as a new row, and teaches nothing.
Copies do a second kind of harm. They weight the model toward whatever happens to be repeated, and, as in Datasheets, they leak between the training and test splits.
Show the answer
The 200 labeled rows held fewer distinct examples
Each copy carried its original's label, so the labels were right. What shrank was the number of different examples the model saw: 200 with no copies, 50 when nearly all the pool was copies. A model learns from distinct examples, and the budget was paying for repeats.
Distinct rows among the 200 labeled, and the accuracy each bought, from the cell above.
texts = [
"Reset my password", "reset my password", "Reset my password ", "Export invoices",
"Reset my passwords", "EXPORT INVOICES",
]
def normalize(text):
return " ".join(text.lower().split())
print(len(texts), len({normalize(t) for t in texts}))Show the answer
6 3
Lower-casing and collapsing spaces makes the first three texts identical, and the two about invoices identical. "Reset my passwords" has an extra letter, which normalizing doesn't remove, so it stays separate: three distinct texts.
Active learning
Curation removes rows. Active learning chooses which rows to label next, and lets the model do the choosing. It is a loop.
- Train a model on the rows labeled so far.
- Score every unlabeled row in the pool.
- Send the rows the model is least sure about to the annotators.
- Add their labels and go back to step 1.
For a two-class model, "least sure" has a simple meaning: the predicted probability is nearest to 0.5. Picking those rows is called uncertainty sampling.
Show the answer
The row at 0.48
0.48 is nearest to 0.5: the model can barely choose between the two classes there. The rows at 0.03 and 0.91 are its most confident answers, one for each class, and the row at 0.70 is in between.
Write a sentence or two, then compare it with a good answer in the lesson.
In code: dedup, uncertainty sampling and a gate for synthetic rows
Each cell below is small enough to read in a minute and runs in the page: a normalizing deduplicator, the active loop, and the checks a generated batch has to pass.
Text for fine-tuning needs the same care, plus a look at the answers. A model fine-tuned on question and answer pairs repeats the habits of its answers, including the bad ones. Here is a small curation pass over ten candidate pairs for a support assistant.
pairs = [
(
"How do I reset my password?",
"Open Settings, choose Security, then Reset password."
),
(
"how do I reset my password?",
"Open Settings, choose Security, then Reset password."
),
(
"Can I export my invoices?",
"Yes. Go to Billing, then Export, and pick a date range."
),
("Where is the dark theme?", ""),
(
"How do I add a teammate?",
"Open Team, choose Invite, and enter their email address."
),
(
"Can I export my invoices?",
"Yes. Go to Billing, then Export, and pick a date range."
),
("Why was I charged twice?", "ok"),
("How do I cancel my plan?", "I'm sorry, I can't help with that."),
(
"How do I change my email?",
"Open Settings, choose Profile, and edit the email field."
),
(
"HOW DO I ADD A TEAMMATE?",
"Open Team, choose Invite, and enter their email address."
),
]
def normalize(text):
return " ".join(text.lower().split()) # lower case, single spaces
kept, seen, dropped = [], set(), {"duplicate": 0, "too short": 0, "refusal": 0}
for question, answer in pairs:
key = normalize(question)
if key in seen:
dropped["duplicate"] += 1
elif len(answer.split()) < 3:
dropped["too short"] += 1
elif normalize(answer).startswith("i'm sorry"):
dropped["refusal"] += 1
else:
kept.append((question, answer))
seen.add(key)
print(len(pairs), "pairs in |", len(kept), "kept | dropped:", dropped)
Ten pairs went in and four came out. Three were the same question written with different spacing or capitals. Two had an answer too short to teach anything. One answered a question the product should handle with a refusal, and a model trained on that row learns to refuse it.
Every filter is a rule that someone chose, so a curation pass should count what each rule removed, as this one does, and the counts belong in the datasheet. A filter that quietly drops a third of the data deserves a look before anyone trains on the rest.
Keep the first of each near duplicate
Write keep_first(texts). Two texts are the same when they are equal after lower-casing and collapsing every run of whitespace to one space, with none at the ends. Return a tuple: the list of indexes to keep, which is the first occurrence of each distinct text, in order, and the number of texts dropped as duplicates.
def keep_first(texts): return list(range(len(texts))), 0 sample = [ "Reset my password", "reset my password", "Export invoices", "RESET MY PASSWORD", ] print(keep_first(sample))
The cell below compares uncertainty sampling with labeling at random, 20 rows a round, on a task where about one row in ten is positive. Each strategy is run on ten different pools and the results are averaged. The test set is a separate random sample.
import numpy as np from sklearn.linear_model import LogisticRegression W = np.array([2.0, -1.5, 1.0, 0.5, 0, 0, 0, 0, 0, 0]) def make(n, seed): g = np.random.default_rng(seed) X = g.normal(size=(n, 10)) # about one row in ten is positive return X, (X @ W + g.normal(scale=0.5, size=n) > 3.5).astype(int) # a random sample, labeled separately X_test, y_test = make(5000, 1) def learn(strategy, seed, rounds=9, step=20): # y[i] is what an annotator would answer if asked X, y = make(5000, seed) g = np.random.default_rng(1000 + seed) labeled = list(g.permutation(5000)[:step]) # the first batch has to be random while len(set(y[labeled])) < 2: # and has to contain both classes labeled.append(int(g.integers(0, 5000))) history = {} for r in range(rounds + 1): model = LogisticRegression().fit(X[labeled], y[labeled]) history[ len(labeled) // step * step ] = (model.score(X_test, y_test), y[labeled].mean()) rest = np.setdiff1d(np.arange(5000), labeled) if strategy == "random": chosen = g.permutation(rest)[:step] # the rows whose probability is nearest 0.5 else: p = model.predict_proba(X[rest])[:, 1] chosen = rest[np.argsort(np.abs(p - 0.5))[:step]] labeled += list(chosen) return history runs = {s: [learn(s, seed) for seed in range(10)] for s in ("random", "uncertain")} print( "labels | random: accuracy, share positive | least sure: accuracy, share positive" ) for n in (20, 60, 100, 140, 200): cells = [] for s in ("random", "uncertain"): acc = round(float(np.mean([h[n][0] for h in runs[s]])), 3) pos = round(float(np.mean([h[n][1] for h in runs[s]])), 2) cells.append(str(acc).ljust(6) + " " + str(pos).ljust(4)) print(str(n).rjust(6), "|", cells[0].ljust(32), "|", cells[1]) print("share positive in the real data:", round(float(y_test.mean()), 3))
Both strategies start from the same random batch. From there the gap opens: at 100 labels, random labeling reaches 0.95 and uncertainty sampling 0.972. Random labeling needs all 200 labels to reach 0.964, which uncertainty sampling passed before its hundredth. That is the same accuracy for about half the labels, and the gain is usually largest when one class is rare.
The right-hand columns show why, and show the price. Random labeling finds positives at the rate they occur, about one in ten, so most of its budget goes on easy negatives. Uncertainty sampling goes to the boundary, where the two classes meet in equal numbers, and its labeled set ends up 0.48 positive in a world that is 0.094 positive.
p = [0.97, 0.55, 0.08, 0.41, 0.62, 0.50] order = sorted(range(len(p)), key=lambda i: abs(p[i] - 0.5)) print(order[:3])
Show the answer
[5, 1, 3]
The distance from 0.5 is 0 for ticket 5, 0.05 for ticket 1 and 0.09 for ticket 3. Those are the three the model can't decide about. Tickets 0 and 2, at 0.97 and 0.08, are the ones it is surest of, and labeling them would mostly confirm what it already believes.
That lopsided labeled set is the main hazard of active learning, and it has a firm rule attached: an actively chosen set is not a sample of the world. Don't estimate the class balance from it. Don't hold part of it out as a test set, since it is made of the hardest rows and would understate the model. The test set is drawn at random and labeled separately, before the loop starts.
Three smaller cautions.
- The first batch must be random. With no model there is no uncertainty to rank by.
- Uncertain rows come in clumps. The twenty least certain rows are often near copies of each other. Deduplicate the pool first, or spread each batch out.
- Some rows are uncertain because they are ambiguous. No label will settle them. They are the split votes of the lesson on labeling, and they belong with an adjudicator, not in a loop that keeps asking.
Pick the rows the model is least sure about
Write least_sure(probabilities, k, skip). probabilities holds a two-class model's probability for each row of a pool. skip is a set of indexes that are already labeled. Return the indexes of the k rows, not in skip, whose probability is nearest to 0.5, nearest first. When two rows are equally near, the lower index goes first. If fewer than k rows are available, return them all.
def least_sure(probabilities, k, skip): return [] print(least_sure([0.97, 0.55, 0.08, 0.41, 0.62, 0.50], 3, {5}))
Show the answer
The rows were chosen for being hard, so they aren't a sample of real traffic
Uncertainty sampling fills the labeled set with rows near the decision boundary and with far more of the rare class than the world contains. A test set cut from it measures the model on unusually hard, unusually balanced rows. An honest test set is sampled at random and labeled on its own.
The gap is easy to see when it is drawn:
import matplotlib.pyplot as plt labels = [20, 60, 100, 140, 200] random_acc = [0.915, 0.942, 0.95, 0.955, 0.964] active_acc = [0.915, 0.957, 0.972, 0.974, 0.976] plt.plot(labels, random_acc, marker="o", label="random") plt.plot(labels, active_acc, marker="o", label="least sure") plt.axhline(0.964, linestyle=":", color="gray") plt.xlabel("labels bought") plt.ylabel("accuracy") plt.legend() plt.show()
The dotted line is where random labeling ends, at 200 labels. The least-sure curve crosses it before 100.
Show the answer
About 60 to 100
At 60 labels uncertainty sampling is at 0.957, just short of random's 0.964 at 200. At 100 it is at 0.972, past it. So it crosses between 60 and 100.
Synthetic data
Synthetic data is training data that was manufactured, not collected. It comes in three kinds, and they are worth keeping apart, because they differ in whether they can add anything.
- Resampled. Fit a simple generator to the rows you have, and draw new rows from it.
- Augmented. Take each real row and make altered copies of it: a little noise, a reworded sentence, a cropped image.
- Labeled by a stronger model. Take real inputs that have no labels, and have a model that knows more than yours answer them. You met this as distillation.
Show the answer
Augmented: altered copies of real rows
Each new row is a real row with its wording changed and its label kept. That is augmentation. It can teach a model that wording doesn't change the label, and it can't teach it about a kind of question the 500 never contained.
The test below starts from 40 real labeled rows and adds 2,000 synthetic ones in each of the three ways. The stronger model, the teacher, was trained on 5,000 real labels that the small team doesn't have.
import numpy as np from sklearn.linear_model import LogisticRegression W = np.array([2.0, -1.5, 1.0, 0.5, 0, 0, 0, 0, 0, 0]) def make(n, seed): g = np.random.default_rng(seed) X = g.normal(size=(n, 10)) return X, (X @ W + g.normal(scale=0.5, size=n) > 0).astype(int) X_test, y_test = make(5000, 1) teacher = LogisticRegression().fit(*make(5000, 2)) names = [ "40 real rows", "+ 2,000 resampled from a generator fitted to them", "+ 2,000 augmented copies with small noise", "+ 2,000 real inputs labeled by the teacher", ] scores = {name: [] for name in names} for rep in range(20): g = np.random.default_rng(300 + rep) X, y = make(40, 400 + rep) def fit(extra_X, extra_y): both_X, both_y = np.vstack([X, extra_X]), np.concatenate([y, extra_y]) return LogisticRegression().fit(both_X, both_y).score(X_test, y_test) scores[names[0]].append(LogisticRegression().fit(X, y).score(X_test, y_test)) drawn = [g.normal(X[y == c].mean(axis=0), X[y == c].std(axis=0), size=(1000, 10)) for c in (0, 1)] scores[names[1]].append(fit(np.vstack(drawn), np.repeat([0, 1], 1000))) noisy = np.repeat(X, 50, axis=0) + g.normal(scale=0.1, size=(2000, 10)) scores[names[2]].append(fit(noisy, np.repeat(y, 50))) unlabeled = g.normal(size=(2000, 10)) scores[names[3]].append(fit(unlabeled, teacher.predict(unlabeled))) for name in names: print(name.ljust(50), round(float(np.mean(scores[name])), 3)) print("the teacher itself".ljust(50), round(float(teacher.score(X_test, y_test)), 3))
Forty real rows give 0.868. Resampling from a generator fitted to those forty makes the model worse, 0.844: the generator knows only what the forty rows told it, less the detail it smoothed away, and 2,000 rows of that outvote the real ones. Augmenting with small noise gives 0.875, a rounding error of a gain, since a nudged copy is still the same example.
The third line is different in kind. The teacher's labels on fresh real inputs lift the model to 0.942, which is the teacher's own score. Those rows carried information that the forty didn't: the teacher's knowledge of 5,000 labels.
That is the rule for synthetic data. It can only pass on what its source knows. A generator fitted to your data knows less than your data. A large model asked to write or label examples can know more, and then your model's ceiling is that model's accuracy, mistakes included.
Show the answer
The teacher: the student learns its mistakes along with its skill
The student is fitted to the teacher's outputs, so the teacher's outputs are its truth. Where the teacher is wrong, the student is taught to be wrong the same way. Synthetic labels behave the same: the run above topped out at the teacher's 0.942.
Show the answer
Little or no gain, since the generator knows only the 300
A generator fitted to 300 rows contains at most what those rows contain, minus whatever it smoothed over. Drawing 10,000 rows from it repeats that knowledge at greater volume, and can drown the real rows. New information has to come from somewhere: new real data, new labels, or a source that knows more.
Worked example: a gate for a batch of generated examples
A large model has written a batch of labeled support questions for fine-tuning. Before the batch is allowed near training, run it through four checks: variety, a dominant opening, class balance, and overlap with the real test set.
test_questions = [
"How do I reset my password?", "Why was I charged twice?",
"Can I export my invoices?",
]
batch = [ # (generated question, label)
(
"Certainly! How can I reset my password?", "account"
), ("How do I reset my password?", "account"),
(
"Certainly! How can I export my invoices?", "billing"
), ("Certainly! How can I reset my password?", "account"),
(
"Certainly! How can I add a teammate?", "account"
), ("Where can I change my email address?", "account"),
(
"Certainly! How can I reset my password?", "account"
), ("Certainly! How can I change the theme?", "account"),
(
"Why was I charged twice?", "billing"
), ("Certainly! How can I update my profile?", "account"),
]
def normalize(text):
return " ".join(text.lower().split())
texts = [question for question, label in batch]
distinct = len({normalize(t) for t in texts}) / len(texts)
openers = {}
for t in texts:
openers[t.split()[0].lower()] = openers.get(t.split()[0].lower(), 0) + 1
top_opener = max(openers.values()) / len(texts)
labels = [label for question, label in batch]
largest_class = max(labels.count(k) for k in set(labels)) / len(labels)
leaked = sum(normalize(t) in {normalize(q) for q in test_questions} for t in texts)
checks = [
("at least 0.9 of the texts are distinct", distinct >= 0.9, round(distinct, 2)),
(
"no first word opens more than 0.3 of the texts", top_opener <= 0.3,
round(top_opener, 2)
),
(
"no class holds more than 0.7 of the rows", largest_class <= 0.7,
round(largest_class, 2)
),
("no text is also in the real test set", leaked == 0, leaked),
]
for name, passed, value in checks:
print("pass" if passed else "FAIL", "|", name, "| measured:", value)
print(
"verdict:",
"accept" if all(passed for name, passed, value in checks) else "send back",
)The batch fails all four checks. Only 0.8 of its texts are distinct, since one question appears three times. Seven of the ten open with the same word. Eight of the ten are labeled account, so a model would see billing questions too rarely. And two of the generated questions are word for word in the real test set, which would let the model score on them by memory. None of this needed a model to be trained: every check is a count over the batch, cheap enough to run on every batch a generator produces. The thresholds are choices, and they belong in the datasheet beside the counts.
Write a sentence or two, then compare it with a good answer in the lesson.
On the job: filters you can count, loops you can stop and rows you can trace
In a real pipeline curation is a job that runs before training, and every filter reports how many rows it removed.
Where it comes up in a week
A team wants to triple its fine-tuning set with examples a large model wrote. You run the gate from the worked example on the batch, keep a column that says which rows are synthetic and from which generator, and train twice, with and without them, scored on a test set that stays real. For labels, the first batch is random, the test set is labeled separately before the loop starts, and the ambiguous rows go to an adjudicator, as in Labels and annotator agreement.
Goes deeper in
- Fine-tuning with supervised data: how the curated rows are used to train.
- Distillation: the teacher's labels as synthetic data.
- Datasheets: where the counts of each filter are recorded.
Write a sentence or two, then compare it with a good answer in the lesson.
Practice what you just read
- 6 code examples you can edit and run
- a practice exercise that checks your code
- a 5-question quiz
- 12 review cards for spaced repetition
Try a quiz question
A scraped dataset of 100,000 examples turns out to contain 20,000 distinct ones. What is the fair description of its size for training?
Before this lesson
Where this lesson leads.
575 lessons in thirteen phases. 176 are free without an account, and a free account opens the rest. See every lesson.
Phase 11 · about 34 hours of study
Professional practice
Design, ship and run real systems, work well in a team, and get hired
43 lessons 1 free
- System design fundamentals FreeRequirements, rough numbers, the standard parts and where systems break.40 min
- Caches, queues and background jobsSpeed reads with caches, and move slow work to queues that retry safely.40 min
- Designing AI systemsLatency, cost, routing, fallbacks and guardrails for features built on models.40 min
- Containers and deploymentPackage an app as an image, configure it with the environment, and roll out safely.45 min
- CI/CD pipelinesTest every change automatically, and ship with gates, flags and rollbacks.40 min
- Docker ComposeSeveral containers together: service names, volumes, secrets and start order.40 min
- Kubernetes basicsDesired state, Deployments and Services, labels, probes and rolling updates.40 min
- Cloud for AI engineersAccess policies and least privilege, storage, compute choices, managed model endpoints, secrets and budgets.30 min
- Metrics, dashboards and alert rulesCounters, gauges and histograms, a rate from a counter, a percentile from buckets, a dashboard panel and an alert rule that fires on a sustained condition.40 min
- Reliability and incidentsSLOs, error budgets, alerts, on-call and blameless postmortems.40 min
- Incident lab: from the page to the postmortemOne incident run from the page to the all-clear: decide the mitigation, work out the impact in failed requests and error budget, and write a blameless postmortem, a status update and a runbook entry.40 min
- Measuring an AI feature from its logsTyped views over event logs, metric specs, cost from a price table, and rollups you can rerun.40 min
- A/B testing a model changeHash assignment, per-user results from raw logs, sample ratio checks, guardrails and a decision rule.40 min
- Columnar files and a local warehouseRows against columns, the encodings that shrink a column, Parquet's row groups and the ranges that let a query skip them, and SQL over files with DuckDB.40 min
- Data quality checks and contractsSix properties of a delivery you can test, each as a count of rows that break it, a contract with the producer, and when to block, quarantine or warn.40 min
- Transformations: staging, models and testsRaw, staging, facts and marts as SQL models run in dependency order in DuckDB, tests as queries that count the rows that break a rule, an incremental load that is safe to rerun, and the same models as dbt files.45 min
- Streams and windowsEvents that never stop arriving: event time against arrival time, tumbling, sliding and session windows, a watermark for late events, and counting each event once.40 min
- Labels and annotator agreementLabels as measurements: agreement and Cohen's kappa, the pair of classes annotators confuse, votes and gold questions, and what wrong labels do to a model and a test score.45 min
- Experiment tracking and reproducible runsA run record of settings, seed, data and code behind every score, a run log in a dozen lines, and comparing settings across seeds.40 min
- Hyperparameter searchWhat a grid costs, random search on a log scale, stopping weak settings early, and why the winner's validation score flatters it.40 min
- Versioning data and modelsFiles stored under the hash of their contents, a manifest that names a whole dataset, and lineage from data and code to each model.35 min
- A model registry and promotionVersions under one name, an alias for the live one, gates a candidate must pass, group by group, and a rollback that moves a pointer.40 min
- Feature pipelines and training-serving skewWhy a model that passes offline fails live: statistics refit at serving, two code paths, and features from the future, with the fix for each.40 min
- Batch and online inferencePredict ahead of time or on request: when the inputs exist, latency near a server's limit, batching what waits, cost, and a hybrid with a freshness limit.40 min
- Monitoring a model: drift and decayWhy a model worsens with no code change: data drift and concept drift, a measure of how far inputs have moved, thresholds set from noise, and labels that arrive late.45 min
- Retraining and pipelinesWhen to retrain and on which weeks, training as steps in dependency order, a cache that reruns only what changed, and guards before and after.45 min
- Designing ML systemsOne method for an ML design, goal and metric to retraining, with the sums that size it: requests a second at peak, a year of feature storage, a label delay and a latency budget.45 min
- Datasheets: documenting a datasetThe document that travels with a dataset: a profile by column and by group, claims a program can check, splits that don't leak, a scan for personal data, and the license and limits only people can write.45 min
- Curation, active learning and synthetic dataThree levers on a training set: removing copies and broken rows, letting a model choose which rows to label, and manufacturing rows, which helps only when their source knows more than you do.45 min
- Measuring fairnessBreak a model's results down by group: selection rate, true and false positive rates and precision, why they can't all be equal, what each fix costs, and when a group is too small to judge.40 min
- Privacy engineeringWhy removing names isn't anonymity, how to hold less (minimization, retention, keyed pseudonyms), the idea behind differential privacy and its budget, what a model remembers, and a deletion that reaches every store.40 min
- A red-team labAttack a simulated assistant methodically: eight kinds of attack run as a suite, success rates beside ordinary requests blocked, fresh attacks to test the fixes, where a defense belongs, and findings kept as release tests.45 min
- What regulation asks of an engineerRisk tiers that follow a system's use, the requirements for high-risk systems as artifacts an engineer builds, evidence tied to a model version, a model card a program can check, and legal dates kept as data.35 min
- Git for teamsBranches, pull requests, rebasing and how a three-way merge finds conflicts.40 min
- Writing for engineersREADMEs, design docs, commit messages and bug reports people can act on.35 min
- Scoping and estimating workA vague request cut into tickets with acceptance criteria and a first shippable slice, estimates as ranges, an honest update for a slip, and a ticket linked to its pull request.30 min
- Roles, résumé and the searchWhat AI, ML and data engineering postings ask for, résumé bullets that carry a number, a 100-word project summary, a take-home as a portfolio piece and how to read an offer.35 min
- Interviews and your portfolioShow your work, and solve interview problems the way professionals do.40 min
- Code reviewWhat to look for, how to give and take feedback, and automating the routine checks.35 min
- Reading unfamiliar codeFind your way around a big codebase fast, with tools that map it for you.35 min
- Working with AI coding toolsGive agents good context, keep tasks small, and check every change they make.35 min
- Capstone: ship a production AI serviceAn AI-backed service with validation, caching, rate limits, fallbacks, logging and evals.45 min
- Capstone: take a model to productionA churn model through a data contract, a tracked run, a registry gate by slice, hybrid serving, a PSI monitor and one retrain, shipped with a datasheet, a model card, a fairness audit, a design doc and a runbook.45 min
Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write