Privacy engineering
Why removing names isn't anonymity, how to hold less (minimization, retention, keyed pseudonyms), the idea behind differential privacy and its budget, what a model remembers, and a deletion that reaches every store.
What it covers6
Why it matters
Every AI system runs on data about people: the logs of what they asked, the tickets they wrote, the records a model was trained on. Privacy law says what you owe those people. Privacy engineering is how you actually deliver it, and most of it is unglamorous: fewer columns, shorter retention, identifiers that can't be reversed, and knowing where every copy lives. This lesson measures the main risks with small experiments, so that "we removed the names" never again sounds like enough.
What you'll be able to do
- Tell direct identifiers from quasi-identifiers, and show how a few ordinary columns single a person out
- Reduce what a system holds: collect less, keep it for less time, and replace identifiers with keyed pseudonyms
- Measure how well a table hides people with the size of its smallest group
- Explain the idea behind differential privacy: noise sized to hide any one person, and a budget that is spent
- Describe how a trained model can leak who was in its training data, and plan a deletion across every store
Show the answer
No: those three together single out most people
Each of the three columns is shared by thousands of people. Together they are nearly a fingerprint: very few people in one ZIP code share a birth date and a sex. Anyone holding another list with the same three columns and names, such as a voter register, can put the names back. That the columns are public is what makes the attack possible.
The idea: data that points at a person
Uniqueness is a counting argument: more combinations of details than people. It uses Counting. The first check recalls why free text needs a scan.
Data that points at a person
It helps to sort the columns of any table into three kinds.
- A direct identifier names one person by itself: an email address, a phone number, an account id.
- A quasi-identifier doesn't, and narrows things down: a ZIP code, a birth date, a job title. Several together can be as good as a name.
- A sensitive attribute is what the person would not want attached to them: a diagnosis, a salary.
Removing the direct identifiers is the obvious first step, and it is the one people stop at. The cell builds a city of 20,000 people and counts how many are the only person with their combination of quasi-identifiers.
import numpy as np rng = np.random.default_rng(0) N = 20000 # everyone in a small city zip_code = rng.integers(0, 400, N) # 400 ZIP codes, in 20 areas of 20 year = rng.integers(1940, 2006, N) # year of birth day = rng.integers(0, 365, N) # day of the year sex = rng.integers(0, 2, N) def groups(*columns): """For each person, how many people share their combination of these columns.""" keys = np.stack(columns, axis=1) _, where, counts = np.unique(keys, axis=0, return_inverse=True, return_counts=True) return counts[where.ravel()] print("columns kept".ljust(32), "| people who are the only one | smallest group") for name, columns in (("ZIP code, birth date, sex", (zip_code, year, day, sex)), ("ZIP code, birth year, sex", (zip_code, year, sex)), ("ZIP area, birth year, sex", (zip_code // 20, year, sex)), ("ZIP area, decade, sex", (zip_code // 20, year // 10, sex))): size = groups(*columns) print( name.ljust(32), "|", f"{100 * (size == 1).mean():5.1f}%".rjust(27), "|", int(size.min()), ) # A hospital shares 2,000 patients' records with the names removed. The city's public # register has everyone's # name, ZIP code, birth date and sex. patients = rng.choice(N, 2000, replace=False) alone = groups(zip_code, year, day, sex) == 1 print() print( "patients in the shared table:", len(patients), "| matched to exactly one named person in the register:", int(alone[patients].sum()), )
With ZIP code, full birth date and sex, 99.9% of the city is unique. That is why the hospital's table fails: of 2,000 patients, 1,999 match exactly one named person in the public register. Their diagnoses now have names on them.
Blurring helps, slowly. The birth year alone still leaves 68.8% of people unique. Widening the ZIP code to an area gets that near zero, and yet the smallest group is still 1: somebody remains alone. Only at area, decade and sex does every person share their combination with at least 31 others.
That last number has a name. A table is k-anonymous when every combination of quasi-identifiers is shared by at least k people. The last row is 32-anonymous. It is a useful measure with a known weakness: if all 32 people in a group have the same diagnosis, finding the group is finding the diagnosis.
Each row keeps fewer details than the one above. k is the size of the smallest group.
Show the answer
People type names, emails and phone numbers into free text
Personal data arrives uninvited: a customer pastes their email address or phone number into the message. A dataset built from that text carries it along, into training sets and logs. A scan with patterns finds the kinds that have a recognizable shape, and the datasheet records what was found and what was done about it.
Write a sentence or two, then compare it with a good answer in the lesson.
In code: blurring, keyed hashes, noise and memorization
Each cell runs a small attack and its defense, with the numbers printed, so you can see what a protection buys.
How well does this table hide people?
Write smallest_group(rows, keys). rows is a list of dictionaries and keys is the list of quasi-identifier columns. Group the rows by their values in those columns, and return the size of the smallest group: the k of k-anonymity. Return 0 for a table with no rows. With no keys at all, every row is in one group.
def smallest_group(rows, keys): return 0 rows = [ {"zip": "10001", "decade": 1980, "diagnosis": "flu"}, {"zip": "10001", "decade": 1980, "diagnosis": "asthma"}, {"zip": "10001", "decade": 1990, "diagnosis": "flu"}, ] print(smallest_group(rows, ["zip", "decade"]))
Collect less, keep less, blur the rest
The strongest protections are the plainest.
- Minimization. Don't collect a column unless a feature needs it. Data you don't hold can't leak, be subpoenaed or be misused. Ask of every field: what breaks without it?
- Retention. Give every store a lifetime, and delete on schedule. A debug log kept forever is a growing copy of everything users typed.
- Coarser values. Keep the birth decade, the ZIP area, the day and not the second.
- Pseudonyms. Where rows must still be linked to the same person, replace the identifier with a stand-in.
That last one has a classic mistake. The cell replaces email addresses with a hash, which looks irreversible, and then attacks it.
import hashlib, hmac # addresses an attacker could list addresses = ["user" + str(i) + "@example.com" for i in range(5000)] customers = addresses[:300] # Attempt 1: replace each address with its SHA-256 hash. stored = [hashlib.sha256(a.encode()).hexdigest() for a in customers] print("stored:", stored[0][:24] + "...") # the attacker hashes every address they can think of table = {hashlib.sha256(a.encode()).hexdigest(): a for a in addresses} print( "plain hash | addresses recovered:", sum(h in table for h in stored), "of", len(stored), ) # Attempt 2: a keyed hash. The key lives in a secrets manager, away from the data. KEY = b"kept out of the data store" def pseudonym(address): return "u_" + hmac.new( KEY, address.lower().encode(), hashlib.sha256, ).hexdigest()[:12] stored = [pseudonym(a) for a in customers] guesses = {"u_" + hashlib.sha256(a.encode()).hexdigest()[:12] for a in addresses} print( "keyed hash | addresses recovered:", sum(p in guesses for p in stored), "of", len(stored), ) print( "the same person always gets the same pseudonym:", pseudonym("Sam@Example.com"), pseudonym("sam@example.com"), )
A plain hash hid nothing: all 300 addresses were recovered. The attacker didn't reverse the hash. They hashed every address they could list and looked for matches. Email addresses, phone numbers and names are guessable, so hashing them only obscures them from honest readers.
A keyed hash fixes that. Without the key the guesses don't match, and 0 of 300 are recovered. The same person still gets the same pseudonym every time, so counts and joins keep working. The key lives apart from the data, in a secrets manager, and whoever holds both can link pseudonyms back to people.
For that reason pseudonymized data is still personal data, in law and in fact: it is safer to hold, and it still needs protecting. Only data that can't be linked back by anyone counts as anonymous, and the last part showed how hard that is to reach.
import hashlib stored = hashlib.sha256(b"5550142").hexdigest() found = [n for n in range(5550000, 5560000) if hashlib.sha256(str(n).encode()).hexdigest() == stored] print(found)
Show the answer
[5550142]
The attacker hashes every candidate and compares. Here ten thousand candidates take a moment, and every phone number in a country is within reach of one ordinary computer. A hash is only hard to undo when the input is hard to guess, and phone numbers aren't. A keyed hash, with the key held elsewhere, is what stops this.
Show the answer
Ask "are you 18 or over?" and store only the answer
The feature needs one bit, so collect one bit. A date of birth that was never collected can't be leaked, matched against another list, or asked for in a deletion request. A hash of a birth date is trivially reversed, since there are only a few tens of thousands of possible dates, and hiding or moving the column leaves it just as collected.
Answers leak too
Suppose nobody ever sees a row. People can only ask for totals. That still leaks, by subtraction: ask for the count over everyone, then for the count over everyone but one person.
Differential privacy is the idea that answers this. Add random noise to each answer, sized so that the answer would look much the same whether or not any one person were in the data. Then no answer can reveal much about anyone. The amount of noise is set by a number called epsilon: small epsilon, more noise, more privacy.
import numpy as np rng = np.random.default_rng(1) # 500 employees, and a private fact about each has_condition = rng.random(500) < 0.3 # the person an attacker wants to know about target = int(np.argmax(has_condition)) everyone = int(has_condition.sum()) all_but_one = int(np.delete(has_condition, target).sum()) print( "How many have the condition?", everyone, "| How many, leaving out one person?", all_but_one, "| so that person:", everyone - all_but_one, ) print() print("the same two questions, answered with noise added:") for epsilon in (0.1, 0.5, 1.0, 5.0): # 2,000 tries at the attack a = everyone + rng.laplace(0, 1 / epsilon, 2000) b = all_but_one + rng.laplace(0, 1 / epsilon, 2000) right = ((a - b) > 0.5).mean() print( "epsilon", str(epsilon).ljust(3), "| attacker right", f"{100 * right:5.1f}% of the time", "| the count is off by", f"{np.abs(a - everyone).mean():4.1f}", "on average, out of", everyone, ) print() print("asking the same two questions again and again, at epsilon 0.5:") for times in (1, 10, 100, 1000): a = everyone + rng.laplace(0, 2, times) b = all_but_one + rng.laplace(0, 2, times) print( str(times).rjust(4), "times | averaged answers differ by", f"{a.mean() - b.mean():+.2f}", "| privacy budget spent:", 2 * times * 0.5, )
With exact answers, two questions give the person away: 159 minus 158 is 1.
With noise at epsilon 0.1 the attacker is right 52.3% of the time, barely better than a coin, and the count is off by about 10 out of 159. At epsilon 5 the count is nearly exact, off by 0.2, and the attacker is right 91.5% of the time. Privacy and accuracy are traded against each other, and epsilon is the dial. Notice that a count over more people would tolerate the same noise far better: being off by 10 matters little in 159,000.
The last four lines show why noise alone isn't enough. Ask the same two questions a thousand times and average, and the noise washes out: the difference comes back as +1.03. So every answer spends some privacy, the epsilons add up, and a system must keep a privacy budget and refuse to answer once it is gone. One question at epsilon 0.5 is a strong protection. A thousand of them is none.
budget, each = 3.0, 0.2 answered = 0 while budget >= each - 1e-9: budget -= each answered += 1 print(answered, round(abs(budget), 2))
Show the answer
15 0.0
Each answer spends 0.2, and the epsilons add, so 3.0 buys 15 answers and nothing is left. The sixteenth query has to be refused, or answered from a cached earlier answer, which costs nothing more. A budget that is never enforced is the same as having none.
Noise sized to hide one person
The noise used above comes from the Laplace distribution, and can be made from one uniform random number u between 0 and 1. Write laplace_noise(u, epsilon, sensitivity). Let scale = sensitivity / epsilon. If u is below 0.5, return scale * log(2 * u). Otherwise return -scale * log(2 * (1 - u)). Raise a ValueError if epsilon isn't above 0. (sensitivity is how much one person can change the true answer: 1 for a count.)
import math def laplace_noise(u, epsilon, sensitivity): return 0.0 print(round(laplace_noise(0.25, 0.5, 1), 3), round(laplace_noise(0.75, 0.5, 1), 3))
The attacker's success against epsilon, on a log axis:
import matplotlib.pyplot as plt epsilons = [0.1, 0.5, 1.0, 5.0] attacker = [52.3, 56.2, 62.7, 91.5] plt.plot(epsilons, attacker, marker="o") plt.axhline(50, linestyle=":", color="gray") plt.xscale("log") plt.xlabel("epsilon (log scale)") plt.ylabel("attacker right (%)") plt.show()
The curve stays near the coin-flip line until epsilon passes 1, and then climbs fast.
Write a sentence or two, then compare it with a good answer in the lesson.
Models remember
A trained model is made from its training data, and can give some of it back. One measurable form of this is a membership inference attack: given a person's record, decide whether the model was trained on it. That alone can be the secret, if the training set was "patients with this condition".
The attack needs nothing clever. A model is usually more confident on examples it was trained on. The cell trains the same small model on more and more people, and lets an attacker guess membership from confidence alone.
import numpy as np def train(X, y, steps=300, rate=0.5): w = np.zeros(X.shape[1]) for step in range(steps): w -= rate * X.T @ (1 / (1 + np.exp(-X @ w)) - y) / len(y) return w def loss(X, y, w): p = np.clip(1 / (1 + np.exp(-X @ w)), 1e-9, 1 - 1e-9) return -(y * np.log(p) + (1 - y) * np.log(1 - p)) rng = np.random.default_rng(2) truth = rng.normal(0, 1, 100) * (rng.random(100) < 0.2) def people(n): # 100 facts about each person X = rng.normal(0, 1, (n, 100)) return X, (rng.random(n) < 1 / (1 + np.exp(-X @ truth))).astype(float) # people the model never saw X_out, y_out = people(1000) print( "trained on | accuracy on them | on new people | attacker says who was in the " "training data" ) for n in (50, 200, 1000, 10000): X_in, y_in = people(n) w = train(X_in, y_in) inside, outside = loss(X_in[:1000], y_in[:1000], w), loss(X_out, y_out, w) # the attack: "the model is very sure about this person, so it was probably trained # on them" cuts = np.quantile(np.concatenate([inside, outside]), np.linspace(0.02, 0.98, 49)) attack = max(((inside < cut).mean() + (outside >= cut).mean()) / 2 for cut in cuts) print( str(n).rjust(10), "|", f"{((X_in @ w > 0) == y_in).mean():.2f}".rjust(16), "|", f"{((X_out @ w > 0) == y_out).mean():.2f}".rjust(13), "|", f"right {100 * attack:.0f}% of the time", )
Trained on 50 people, the model is perfect on them and poor on anyone new, and the attacker is right 87% of the time. That gap between training and new data is the leak: the model has memorized individuals. Trained on 10,000 people it does as well on new people as on its own, and the attacker is at 52%, which is guessing. (Picking the best cut-off after the fact is worth a point or two above 50.)
So the defenses are the ones that reduce overfitting: more data, a smaller model, regularization. There is a version of training with differential privacy built in, which adds noise to the gradients. Large language models have the same weakness in another form: text that appears often in training can be recited back.
The practical consequences reach beyond the model.
- Don't train on what you couldn't publish, unless you have measured what comes back out.
- Prompts and outputs are personal data when a person typed them. They follow the same rules of minimization and retention as any log.
- Deletion has to reach every copy. A model can't forget one row. When someone asks to be erased, the honest plan is to delete the rows everywhere and retrain on schedule.
Show the answer
It has memorized examples, so membership can likely be inferred
A large gap means the model treats the examples it trained on very differently from others: it is far more confident about them. An attacker can use exactly that difference to tell who was in the training set. Closing the gap, with more data or regularization, closes most of the leak.
On the job: deletion across stores, logs and the tools
In a real system the work is finding every copy of a person's data, and keeping new logs from becoming one more copy.
Worked example: "please delete my data"
A user asks to be erased. The product keeps things in five stores, some keyed by email address and some by pseudonym, each with its own lifetime. The code finds every row that belongs to the person and prints what has to happen to it.
import hmac, hashlib KEY = b"kept out of the data store" def pseudonym(address): return "u_" + hmac.new( KEY, address.lower().encode(), hashlib.sha256, ).hexdigest()[:12] person = "sam@example.com" pid = pseudonym(person) ana_id = pseudonym("ana@example.com") # Where one product keeps things. Each store: what a row is keyed by, how long it is # kept, and its rows. stores = { "accounts": { "key": "email", "kept": "until deleted", "rows": [{"email": "sam@example.com"}, {"email": "ana@example.com"}], }, "events": {"key": "pseudonym", "kept": "13 months", "rows": [{"pseudonym": pid}] * 412 + [{"pseudonym": ana_id}] * 77}, "tickets": { "key": "email", "kept": "3 years", "rows": [{"email": "sam@example.com"}] * 3 + [{"email": "ana@example.com"}], }, "training set": {"key": "pseudonym", "kept": "until rebuilt", "rows": [{"pseudonym": pid}] * 3 + [{"pseudonym": ana_id}]}, "backups": { "key": "email", "kept": "30 days", "rows": [{"email": "sam@example.com"}], }, } ACTIONS = {"accounts": "delete now", "events": "delete now", "tickets": "delete now", "training set": "delete, and mark the set as changed", "backups": "can't edit: gone when the backup expires"} found = 0 for name, store in stores.items(): wanted = person if store["key"] == "email" else pid rows = [r for r in store["rows"] if r[store["key"]] == wanted] found += len(rows) print( name.ljust(12), "|", str(len(rows)).rjust(3), "rows | kept", store["kept"].ljust(13), "|", ACTIONS[name], ) print("rows found:", found, "in", len(stores), "stores") print( "model fine-tuned on the training set | no rows to delete | retrain without them " "at the next scheduled build" )
One person, 420 rows, five stores. Three stores can be cleaned at once. Two things are worth noticing. The events and the training set hold no email address, and the rows are still found, since the pseudonym can be recomputed from the address with the key. Pseudonyms made the stores safer to hold without making the person unfindable, and that is the reason pseudonymized data still counts as personal data. Then the two awkward cases. The backup can't be edited, so the answer is its lifetime: thirty days and it is gone, which is only true if backups really do expire. And the model was fine-tuned on three of this person's tickets. There is no row to delete in a model. The honest plan is the one printed: remove the rows from the training set, mark it changed, and retrain at the next scheduled build. None of this works without the inventory in the first place. A team that can't list where personal data lives can't delete it.
Where it comes up in a week
A product manager wants every user message kept "in case we need it". You ask what breaks without it, propose a retention period for the log, redact the obvious identifiers on the way in, and key the pseudonyms with a secret held in a secrets manager. When a user asks to be erased, the deletion job runs against every store, and the training set is marked changed so the next scheduled build leaves them out.
Goes deeper in
- Datasheets: documenting a dataset: the personal-data section of a card.
- What regulation asks of an engineer: deletion and retention as duties.
- Red-team lab: attacks that pull data back out of a model.
- Capstone: where the keyed hash replaces the plain one.
Write a sentence or two, then compare it with a good answer in the lesson.
Practice what you just read
- 6 code examples you can edit and run
- a practice exercise that checks your code
- a 5-question quiz
- 13 review cards for spaced repetition
Try a quiz question
Why is a quasi-identifier a privacy risk, when it doesn't identify anyone by itself?
Before this lesson
Where this lesson leads.
575 lessons in thirteen phases. 176 are free without an account, and a free account opens the rest. See every lesson.
Phase 11 · about 34 hours of study
Professional practice
Design, ship and run real systems, work well in a team, and get hired
43 lessons 1 free
- System design fundamentals FreeRequirements, rough numbers, the standard parts and where systems break.40 min
- Caches, queues and background jobsSpeed reads with caches, and move slow work to queues that retry safely.40 min
- Designing AI systemsLatency, cost, routing, fallbacks and guardrails for features built on models.40 min
- Containers and deploymentPackage an app as an image, configure it with the environment, and roll out safely.45 min
- CI/CD pipelinesTest every change automatically, and ship with gates, flags and rollbacks.40 min
- Docker ComposeSeveral containers together: service names, volumes, secrets and start order.40 min
- Kubernetes basicsDesired state, Deployments and Services, labels, probes and rolling updates.40 min
- Cloud for AI engineersAccess policies and least privilege, storage, compute choices, managed model endpoints, secrets and budgets.30 min
- Metrics, dashboards and alert rulesCounters, gauges and histograms, a rate from a counter, a percentile from buckets, a dashboard panel and an alert rule that fires on a sustained condition.40 min
- Reliability and incidentsSLOs, error budgets, alerts, on-call and blameless postmortems.40 min
- Incident lab: from the page to the postmortemOne incident run from the page to the all-clear: decide the mitigation, work out the impact in failed requests and error budget, and write a blameless postmortem, a status update and a runbook entry.40 min
- Measuring an AI feature from its logsTyped views over event logs, metric specs, cost from a price table, and rollups you can rerun.40 min
- A/B testing a model changeHash assignment, per-user results from raw logs, sample ratio checks, guardrails and a decision rule.40 min
- Columnar files and a local warehouseRows against columns, the encodings that shrink a column, Parquet's row groups and the ranges that let a query skip them, and SQL over files with DuckDB.40 min
- Data quality checks and contractsSix properties of a delivery you can test, each as a count of rows that break it, a contract with the producer, and when to block, quarantine or warn.40 min
- Transformations: staging, models and testsRaw, staging, facts and marts as SQL models run in dependency order in DuckDB, tests as queries that count the rows that break a rule, an incremental load that is safe to rerun, and the same models as dbt files.45 min
- Streams and windowsEvents that never stop arriving: event time against arrival time, tumbling, sliding and session windows, a watermark for late events, and counting each event once.40 min
- Labels and annotator agreementLabels as measurements: agreement and Cohen's kappa, the pair of classes annotators confuse, votes and gold questions, and what wrong labels do to a model and a test score.45 min
- Experiment tracking and reproducible runsA run record of settings, seed, data and code behind every score, a run log in a dozen lines, and comparing settings across seeds.40 min
- Hyperparameter searchWhat a grid costs, random search on a log scale, stopping weak settings early, and why the winner's validation score flatters it.40 min
- Versioning data and modelsFiles stored under the hash of their contents, a manifest that names a whole dataset, and lineage from data and code to each model.35 min
- A model registry and promotionVersions under one name, an alias for the live one, gates a candidate must pass, group by group, and a rollback that moves a pointer.40 min
- Feature pipelines and training-serving skewWhy a model that passes offline fails live: statistics refit at serving, two code paths, and features from the future, with the fix for each.40 min
- Batch and online inferencePredict ahead of time or on request: when the inputs exist, latency near a server's limit, batching what waits, cost, and a hybrid with a freshness limit.40 min
- Monitoring a model: drift and decayWhy a model worsens with no code change: data drift and concept drift, a measure of how far inputs have moved, thresholds set from noise, and labels that arrive late.45 min
- Retraining and pipelinesWhen to retrain and on which weeks, training as steps in dependency order, a cache that reruns only what changed, and guards before and after.45 min
- Designing ML systemsOne method for an ML design, goal and metric to retraining, with the sums that size it: requests a second at peak, a year of feature storage, a label delay and a latency budget.45 min
- Datasheets: documenting a datasetThe document that travels with a dataset: a profile by column and by group, claims a program can check, splits that don't leak, a scan for personal data, and the license and limits only people can write.45 min
- Curation, active learning and synthetic dataThree levers on a training set: removing copies and broken rows, letting a model choose which rows to label, and manufacturing rows, which helps only when their source knows more than you do.45 min
- Measuring fairnessBreak a model's results down by group: selection rate, true and false positive rates and precision, why they can't all be equal, what each fix costs, and when a group is too small to judge.40 min
- Privacy engineeringWhy removing names isn't anonymity, how to hold less (minimization, retention, keyed pseudonyms), the idea behind differential privacy and its budget, what a model remembers, and a deletion that reaches every store.40 min
- A red-team labAttack a simulated assistant methodically: eight kinds of attack run as a suite, success rates beside ordinary requests blocked, fresh attacks to test the fixes, where a defense belongs, and findings kept as release tests.45 min
- What regulation asks of an engineerRisk tiers that follow a system's use, the requirements for high-risk systems as artifacts an engineer builds, evidence tied to a model version, a model card a program can check, and legal dates kept as data.35 min
- Git for teamsBranches, pull requests, rebasing and how a three-way merge finds conflicts.40 min
- Writing for engineersREADMEs, design docs, commit messages and bug reports people can act on.35 min
- Scoping and estimating workA vague request cut into tickets with acceptance criteria and a first shippable slice, estimates as ranges, an honest update for a slip, and a ticket linked to its pull request.30 min
- Roles, résumé and the searchWhat AI, ML and data engineering postings ask for, résumé bullets that carry a number, a 100-word project summary, a take-home as a portfolio piece and how to read an offer.35 min
- Interviews and your portfolioShow your work, and solve interview problems the way professionals do.40 min
- Code reviewWhat to look for, how to give and take feedback, and automating the routine checks.35 min
- Reading unfamiliar codeFind your way around a big codebase fast, with tools that map it for you.35 min
- Working with AI coding toolsGive agents good context, keep tasks small, and check every change they make.35 min
- Capstone: ship a production AI serviceAn AI-backed service with validation, caching, rate limits, fallbacks, logging and evals.45 min
- Capstone: take a model to productionA churn model through a data contract, a tracked run, a registry gate by slice, hybrid serving, a PSI monitor and one retrain, shipped with a datasheet, a model card, a fairness audit, a design doc and a runbook.45 min
Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write