Index Ai Academy
Phase 11 · Data engineering

Data quality checks and contracts

Six properties of a delivery you can test, each as a count of rows that break it, a contract with the producer, and when to block, quarantine or warn.

About 40 minutes · A free account opens the whole lesson.
Already have an account? Sign in to open this lesson

What it covers7

Why it matters

Code that is wrong usually fails loudly. Data that is wrong doesn't fail at all. A file arrives with a customer listed twice, or with prices in cents where they used to be in dollars, and every query runs, every chart draws and every model trains. The numbers are just wrong, and the first person to notice is often a customer or an executive, weeks later. The defense is unglamorous and it works: write down what the data is supposed to look like, test each delivery against that before anything reads it, and agree in advance what happens when a test fails.

What you'll be able to do

  • Name the properties of a batch of data that can be tested: volume, uniqueness, completeness, validity, references and freshness
  • Write each property as a check that counts the rows that break it
  • Put the checks in a contract between the team that produces the data and the teams that use it, and decide what a failure does
  • Tell a schema change that is safe for readers from one that breaks them
Quick check
A guess before you start. A revenue report joins an orders table to a customers table. One night the customers table arrives with one customer listed twice. The orders table is untouched. What happens to the revenue total?
  1. It goes up: every order of that customer is now counted twice
  2. It goes down: the duplicated customer's orders are dropped
  3. The query fails with an error about a duplicate key
  4. It is unchanged: the orders table holds the amounts, and it didn't change
Show the answer

It goes up: every order of that customer is now counted twice

A join pairs each order with every matching customer row. With the customer listed twice, each of their orders finds two matches and appears twice in the result, so its amount is added twice. Nothing errors. The lesson's first cell runs it: 200 becomes 280.

The idea: wrong data raises no error

This lesson rests on Describing data: center, spread and scale (a mean can be dragged by a few values, and a count is plain) and Ratios, rates and proportion (a share of bad rows is bad ÷ all). Nothing here is clever, which is the point.

Wrong data doesn't raise errors

Here is the opening question, run.

Setup (runs before every cell) · 22 lines

The lesson runs this once, before its first cell, and every cell, drill and example starts with what it defines. A cell that changes a setup object changes it for the cells after it, until the page reloads.

import duckdb

def make_orders():
    con = duckdb.connect()
    con.execute("""
        create table orders as
        select range as order_id,
               1 + range % 400 as customer_id,
               case when range % 50 = 7 then null
                    else ['de', 'fr', 'us'][1 + range % 3] end as country,
               case when range % 25 = 3 then (500 + range % 90) * 100.0
                    else 5 + range % 90 end as amount,
               case when range % 67 = 5 then 'refunded'
                    else ['paid', 'shipped'][1 + range % 2] end as status,
               date '2026-11-06' - cast(range % 3 as integer) as created
        from range(1000)
        union all select 17, 18, 'de', 22, 'paid', date '2026-11-06'
        union all select 250, 251, 'fr', 75, 'paid', date '2026-11-05'
    """)
    con.execute("create table customers as "
                "select range + 1 as customer_id from range(400)")
    return con
import duckdb

con = duckdb.connect()
con.execute("create table orders (order_id int, customer_id int, amount double)")
con.execute("insert into orders values "
            "(1, 10, 50.0), (2, 10, 30.0), (3, 11, 20.0), (4, 12, 100.0)")
con.execute("create table customers (customer_id int, country text)")
con.execute("insert into customers values (10, 'de'), (11, 'fr'), (12, 'us')")

revenue = ("select round(sum(o.amount)) "
           "from orders o join customers c using (customer_id)")
print("revenue:", con.execute(revenue).fetchone()[0])

# tonight's export delivered customer 10 twice
con.execute("insert into customers values (10, 'de')")
print("revenue after one duplicated customer row:", con.execute(revenue).fetchone()[0])
print("the orders table is untouched:",
      con.execute("select count(*), sum(amount) from orders").fetchone())
It runs right here in your browser, with nothing to install.

Revenue went from 200 to 280. No order changed. One row was repeated in another table, the join matched customer 10's two orders twice each, and 80 was counted that never existed. The query is correct. The answer is wrong. And if this table fed a model, the model would now be training on customer 10's orders at double weight.

Three things make this kind of failure expensive.

  • It is silent. Nothing crashes, so nothing pages anyone.
  • It travels. Every table, chart and model built downstream inherits it.
  • It comes from somewhere else. The data was produced by another team's system, changed for their own good reasons, and they had no idea who was reading it.

So the people who use a dataset have to be the ones who test it, at the door, before anything else reads it.

Quick check
A team's dashboard has shown an average order value of about 49 for a year. One morning it shows 2,223. Every query ran without error. Which is the most likely cause?
  1. Some rows arrived with wrong values, such as amounts in another unit
  2. The database ran out of space while the query was running
  3. Customers began spending forty-five times as much overnight
  4. The dashboard's code was broken by last night's deployment
Show the answer

Some rows arrived with wrong values, such as amounts in another unit

A sudden, enormous jump with no error points at the data. A share of rows carrying values a hundred times too large, cents delivered where dollars were expected, is enough to drag an average that far. This lesson builds that table: 40 bad rows in 1,002 move the mean from 49 to 2,223.

Quick check
From the lesson on describing data: the amounts are 10, 20, 30 and one that was entered in cents, 4,000. What is the mean?
  1. 20
  2. 1,015
  3. 4,060
  4. 25
Show the answer

1,015

(10 + 20 + 30 + 4,000) ÷ 4 = 1,015. One bad value drags a mean far from the typical 20, which is why a quality check counts rows out of range instead of trusting an average.

Checks you can state

"Is the data good?" can't be tested. These six properties can, and between them they catch most of what goes wrong with a delivered table.

PropertyThe questionTypical failure
VolumeIs there about as much as expected?Half a file; a job that ran twice
UniquenessDoes each key appear once?Duplicated rows from a retry
CompletenessAre required values present?A field the source stopped filling
ValidityAre values in range and of an allowed kind?A unit change; a new category
ReferencesDoes each id point at something that exists?Orders for customers nobody has
FreshnessIs the newest data recent enough?A feed that silently stopped
Try it now

Pick a day and step through. Row problems go to quarantine; a problem with the batch blocks the whole delivery. These are the three days the worked example in the last part runs.

In your own words
Why is bad data harder to catch than bad code, and what makes a check useful?

Write a sentence or two, then compare it with a good answer in the lesson.

In code: queries that count the rows that break a rule

Each of the six properties is one SQL query that counts bad rows. Zero passes. The cell below runs all of them on a day's delivery with several things wrong, then plots the counts.

Each one becomes a query that counts the rows that break the rule. Zero means the check passes. Here is a day's delivery of orders with several things wrong with it, and a check for each property.

Uses the setup above, 22 lines.

import duckdb

con = make_orders()

checks = {
    "at least 1,500 rows": "select count(*) < 1500 from orders",
    "order_id is never repeated":
        "select count(*) - count(distinct order_id) from orders",
    "country is never missing": "select count(*) from orders where country is null",
    "amount is between 0 and 500":
        "select count(*) from orders where amount not between 0 and 500",
    "status is paid or shipped":
        "select count(*) from orders where status not in ('paid', 'shipped')",
    "every customer_id is a known customer":
        "select count(*) from orders "
        "where customer_id not in (select customer_id from customers)",
    "newest row is at most 1 day old":
        "select date '2026-11-10' - max(created) > 1 from orders",
}
for name, sql in checks.items():
    bad = con.execute(sql).fetchone()[0]
    print("FAIL" if bad else "ok  ", name, "|", bad)

print("rows:", con.execute("select count(*) from orders").fetchone()[0],
      "| mean amount:",
      con.execute("select round(avg(amount), 2) from orders").fetchone()[0],
      "| mean of the rows in range:",
      con.execute("select round(avg(amount), 2) from orders "
                  "where amount <= 500").fetchone()[0])

Six of the seven checks fail, and every failure names something specific a person can go and look at:

  • Only 1,002 rows where at least 1,500 are normal: part of the day is missing.
  • Two order ids are repeated.
  • Twenty rows have no country.
  • Forty amounts are far out of range. They are in cents, and they drag the mean from 49.06 to 2,222.75.
  • Fifteen rows carry a status nobody was told about, "refunded".
  • The newest row is four days old.

None of these checks is clever, and that is the point. Tools such as dbt and Great Expectations are, at heart, a tidy way to write lists like this one, run them on every delivery, and keep the history.

Two habits make a check useful. Count, don't just pass or fail: "40 rows out of range" tells you whether this is a stray record or a broken source. And set the limits from history: "at least 1,500 rows" came from looking at how many rows a normal day brings.

Quick check
From the lesson on cleaning messy data: duplicated() marks each row whose key repeats an earlier one, so the first copy is never marked. A table of payments has the column payment_id. What does this print: the number of extra rows that duplicates have added, and the ids that repeat?
payment_ids = [501, 502, 502, 503, 504, 504, 504]
repeated = sorted(p for p in set(payment_ids) if payment_ids.count(p) > 1)
print(len(payment_ids) - len(set(payment_ids)), repeated)
  1. 3 [502, 504]
  2. 3 [502, 502, 504, 504, 504]
  3. 2 [502, 504]
  4. 4 [502, 504]
Show the answer

3 [502, 504]

Seven rows hold four distinct ids, so three rows are extra: one extra 502 and two extra 504s. Two ids are repeated. "Rows minus distinct keys" is the count the SQL check used, and it is the number by which a row count is inflated. It is also what duplicated().sum() gives in pandas, since every copy after the first is marked.

Drill

Find the repeated keys

Write duplicate_keys(rows, key). rows is a list of dictionaries. Return a dictionary from each value of row[key] that appears more than once to the number of times it appears. Keys that appear once aren't included.

Uses the setup above, 22 lines.

def duplicate_keys(rows, key):
    return {}


rows = [{"order_id": 17, "amount": 22}, {"order_id": 18, "amount": 5},
        {"order_id": 17, "amount": 22}]
print(duplicate_keys(rows, "order_id"))
In the lesson, your code is checked as you write it.
Worked example

Plot the bad rows per check

The same table, counted again, as a bar chart: how many rows each rule rejects.

Uses the setup above, 22 lines.

import matplotlib.pyplot as plt

con = make_orders()
counts = {
    "repeated id": "select count(*) - count(distinct order_id) from orders",
    "no country": "select count(*) from orders where country is null",
    "amount out of range":
        "select count(*) from orders where amount not between 0 and 500",
    "unknown status":
        "select count(*) from orders where status not in ('paid', 'shipped')",
}
bad = {name: con.execute(sql).fetchone()[0] for name, sql in counts.items()}
print(bad)

plt.barh(list(bad), list(bad.values()))
plt.xlabel("rows that break the rule")
plt.tight_layout()
plt.show()
In the lesson you can change this and run it.

Forty rows are out of range, twenty have no country, fifteen carry a new status and two ids repeat. The bar lengths say which problem to chase first: the amounts, which look like a unit change.

In your own words
Why should a check report a count and not just pass or fail, and when does a repeated key block a delivery?

Write a sentence or two, then compare it with a good answer in the lesson.

On the job: a contract at the door

In a week, this lesson shows up when a new upstream table appears, or when a producer says they will change a column. You write down what each delivery must look like, run the checks before anything reads it, and decide what each failure does. The same file tells the producer who is affected when they change something.

A contract, and what a failure does

A list of checks kept privately by the team that reads the data is a guard. Shared with the team that produces the data, it becomes a data contract: a written agreement about what each delivery will look like. The producer promises it and runs the checks before publishing. The consumer verifies it at the door. When the producer wants to change something, the contract is how they find out who will be affected.

A contract is best written as data, so that the same file is documentation, test suite and change log:

Uses the setup above, 22 lines.

contract = {
    "owner": "checkout team",
    "delivered": "daily, by 06:00 UTC",
    "min_rows": 1500,
    "unique": "order_id",
    "columns": {
        "order_id": {"type": "int"},
        "country": {"type": "str", "allowed": ["de", "fr", "us"]},
        "amount": {"type": "float", "min": 0, "max": 500, "unit": "dollars"},
        "note": {"type": "str", "nullable": True},
    },
}

TYPES = {"int": int, "float": (int, float), "str": str}

def row_problems(row, columns):
    problems = []
    for name, rule in columns.items():
        value = row.get(name)
        if value is None:
            if not rule.get("nullable"):
                problems.append(name + ": missing")
            continue
        if not isinstance(value, TYPES[rule["type"]]):
            problems.append(name + ": wrong type")
        elif "allowed" in rule and value not in rule["allowed"]:
            problems.append(name + ": not allowed")
        elif "min" in rule and value < rule["min"]:
            problems.append(name + ": too low")
        elif "max" in rule and value > rule["max"]:
            problems.append(name + ": too high")
    return problems

rows = [
    {"order_id": 1, "country": "de", "amount": 20.0, "note": None},
    {"order_id": 2, "country": None, "amount": 35.5},
    {"order_id": 3, "country": "fr", "amount": 52000.0},
    {"order_id": "4", "country": "it", "amount": 12.0},
]
for row in rows:
    print(row["order_id"], row_problems(row, contract["columns"]) or "ok")

Notice what the contract carries beyond types: who owns the data, when it arrives, and that amount is in dollars. A unit written down is a unit someone has to change on purpose.

Block, quarantine or warn

A failed check has to do something, and what it should do depends on the failure.

  • Block the whole delivery when the batch itself can't be trusted: too few rows, duplicated keys, a stale feed, or a large share of bad rows. Yesterday's good data stays in place, and nothing downstream runs on the new one. Late and right beats on time and wrong.
  • Quarantine the bad rows when a few records are wrong and the rest are fine. The good rows go through. The bad ones go to a side table with the reason, for someone to inspect and replay.
  • Warn when something is unusual but usable, such as a row count 20% under normal.

The split between the first two needs a number: the share of bad rows above which you stop trusting the rest.

Uses the setup above, 22 lines.

TYPES = {"int": int, "float": (int, float), "str": str}
columns = {"order_id": {"type": "int"},
           "amount": {"type": "float", "min": 0, "max": 500}}

def row_problems(row, columns):
    problems = []
    for name, rule in columns.items():
        value = row.get(name)
        if value is None:
            problems.append(name + ": missing")
        elif not isinstance(value, TYPES[rule["type"]]):
            problems.append(name + ": wrong type")
        elif "max" in rule and value > rule["max"]:
            problems.append(name + ": too high")
    return problems

def deliver(rows, max_bad_share):
    good = [r for r in rows if not row_problems(r, columns)]
    bad = [(r, row_problems(r, columns)) for r in rows if row_problems(r, columns)]
    share = len(bad) / len(rows)
    decision = "BLOCK" if share > max_bad_share else "accept"
    return decision, len(good), len(bad), round(share, 3)

mostly_fine = ([{"order_id": i, "amount": 20.0} for i in range(998)]
               + [{"order_id": 998, "amount": None},
                  {"order_id": 999, "amount": 52000.0}])
unit_changed = [{"order_id": i, "amount": 20.0 if i % 2 else 2000.0}
                for i in range(1000)]
for name, rows in (("two stray rows", mostly_fine),
                   ("half the file in cents", unit_changed)):
    print(name, "->", deliver(rows, max_bad_share=0.01))

Two stray rows in a thousand are 0.2% of the file: they are set aside and the other 998 are accepted. Half the file out of range is not five hundred bad records. It is one broken source, and the right move is to refuse the delivery and call its owner.

Quick check
A delivery of eight rows has been checked, and each row's list of problems is below. The limit on the share of bad rows is 0.25. What does this print?
problems = [[], ["amount: too high"], [], [], ["country: missing"], [], [], []]
bad = sum(1 for p in problems if p)
share = bad / len(problems)
print(bad, share, "block" if share > 0.25 else "quarantine")
  1. 6 0.75 block
  2. 2 0.25 quarantine
  3. 2 0.2 quarantine
  4. 2 0.25 block
Show the answer

2 0.25 quarantine

Two of the eight rows have a problem, a share of 0.25. The delivery is blocked only when the share is more than the limit, and 0.25 is not more than 0.25, so the two rows are quarantined and the other six go through.

Quick check
A contract says a daily file has at least 80,000 rows. Today's has 79,100 rows, and every row passes its checks. Yesterday's had 91,000. What is the most sensible response?
  1. Warn and accept: it is slightly under normal, and the rows are sound
  2. Quarantine 900 rows to bring the count back into line
  3. Block the delivery: any breach of the contract must stop everything
  4. Accept silently: a row count can't indicate a problem
Show the answer

Warn and accept: it is slightly under normal, and the rows are sound

The shortfall is about 1% under the limit and the rows themselves are fine, so the data is usable. It is still unusual enough that someone should be told. Blocking would hold up everything downstream for a small wobble. If the file had 8,000 rows, the same check should block.

Quick check
An automatic retrain is about to fit a model on today's data. Where does a data quality check belong?
  1. Before training: a failed check stops the pipeline and calls a person
  2. Inside the model: let it learn to ignore the bad rows
  3. Nowhere: the promotion gate will reject a model trained on bad data
  4. After deployment: check the data once the new model is live
Show the answer

Before training: a failed check stops the pipeline and calls a person

A retraining pipeline puts its data check directly after fetching, so that a broken column is caught before a model is fitted to it. The gate after training is a second line, and a model trained on a broken feature can still pass it by luck. Retraining and pipelines builds such a pipeline.

When the schema changes

Data sources change, and they should. The danger is a change the readers weren't ready for. Changes to a table's schema, its columns and their types, fall into two groups.

Safe for existing readers:

  • Adding a new column. A reader that asks for the columns it knows never sees it.
  • Making a rule looser on the reader's side, such as allowing a wider range.

Breaking:

  • Removing or renaming a column. Every query that names it fails, or worse, quietly gets nothing.
  • Changing a column's type, such as an id from a number to text.
  • Changing what a column means while keeping its name: dollars to cents, local time to UTC. This is the worst kind, because no type check can see it.
  • Adding a value to a fixed list, such as a new status. Code that handled "paid" and "shipped" has no branch for "refunded".

The rule that keeps everyone working: add, don't change. If amounts must move to cents, add amount_cents beside amount, give readers time to switch, and remove the old column only when the contract's list of readers says nobody uses it.

Uses the setup above, 22 lines.

old = {"order_id": "int", "country": "str", "amount": "float"}
new = {"order_id": "str", "country": "str", "amount_cents": "int", "channel": "str"}

removed = sorted(c for c in old if c not in new)
retyped = sorted(c for c in old if c in new and new[c] != old[c])
added = sorted(c for c in new if c not in old)
print("removed:", removed, "| type changed:", retyped, "| added:", added)
print("safe for existing readers:", not removed and not retyped)

This change removed amount, changed the type of order_id and added two columns. The additions are harmless. The other two would break every reader on the day it shipped.

Quick check
A producer wants to change a table. Which of these changes is safe for teams already reading it?
  1. Renaming amount to amount_usd to make the unit clearer
  2. Changing created from local time to UTC, keeping the name
  3. Adding a new column, channel, that existing queries don't mention
  4. Changing customer_id from a number to a text code
Show the answer

Adding a new column, channel, that existing queries don't mention

A new column is invisible to readers that ask for the columns they already knew. A rename removes the name their queries use. A type change breaks comparisons and joins. And a change of meaning under the same name is the most dangerous of all, because everything still runs.

Drill

Is a schema change safe?

Write breaking_changes(old, new). Each schema maps a column name to a type name. Return a sorted list of strings describing what would break an existing reader: "removed: <column>" for a column in old that isn't in new, and "retyped: <column>" for a column in both whose type differs. Columns that were only added are safe and aren't listed.

Uses the setup above, 22 lines.

def breaking_changes(old, new):
    return []


old = {"order_id": "int", "country": "str", "amount": "float"}
new = {"order_id": "str", "country": "str", "amount_cents": "int", "channel": "str"}
print(breaking_changes(old, new))
In the lesson, your code is checked as you write it.
Worked example

Worked example: a delivery at the door

Put the pieces together for one delivery: row checks against the contract, batch checks, and one decision.

Uses the setup above, 22 lines.

TYPES = {"int": int, "float": (int, float), "str": str}
contract = {
    "min_rows": 8, "unique": "order_id", "max_bad_share": 0.2,
    "columns": {"order_id": {"type": "int"},
                "country": {"type": "str", "allowed": ["de", "fr", "us"]},
                "amount": {"type": "float", "min": 0, "max": 500}},
}

def row_problems(row, columns):
    problems = []
    for name, rule in columns.items():
        value = row.get(name)
        if value is None:
            problems.append(name + ": missing")
        elif not isinstance(value, TYPES[rule["type"]]):
            problems.append(name + ": wrong type")
        elif "allowed" in rule and value not in rule["allowed"]:
            problems.append(name + ": not allowed")
        elif "max" in rule and value > rule["max"]:
            problems.append(name + ": too high")
    return problems

def at_the_door(rows, contract):
    checked = [(r["order_id"], row_problems(r, contract["columns"])) for r in rows]
    quarantined = [(order, problems) for order, problems in checked if problems]
    keys = [r[contract["unique"]] for r in rows]
    batch = []
    if len(rows) < contract["min_rows"]:
        batch.append("too few rows: " + str(len(rows)))
    if len(keys) != len(set(keys)):
        repeated = sorted({k for k in keys if keys.count(k) > 1})
        batch.append("duplicate keys: " + str(repeated))
    share = len(quarantined) / len(rows)
    if batch or share > contract["max_bad_share"]:
        decision = "BLOCK"
    else:
        decision = "accept, with quarantine" if quarantined else "accept"
    return decision, batch, quarantined

monday = [{"order_id": i, "country": "de", "amount": 20.0 + i} for i in range(10)]
tuesday = monday[:9] + [{"order_id": 9, "country": "it", "amount": 31.0}]
wednesday = monday + [{"order_id": 3, "country": "fr", "amount": 12.0}]
for day, rows in (("monday", monday), ("tuesday", tuesday), ("wednesday", wednesday)):
    print(day, "->", at_the_door(rows, contract))
In the lesson you can change this and run it.

Monday is clean and is accepted. Tuesday has one row with a country the contract doesn't allow: a tenth of the file, under the limit of a fifth, so that row is quarantined with its reason and the other nine go through. Wednesday repeats order 3. No individual row is wrong, which is why row checks alone would have passed it, but a duplicated key makes every count and sum over the batch untrustworthy, so the whole delivery is blocked. The three outcomes are the three responses from the lesson, chosen by rules written before the data arrived.

In your own words
When should a failed check block a delivery, quarantine rows or only warn, and what changes are safe for readers?

Write a sentence or two, then compare it with a good answer in the lesson.

Practice what you just read

  • 7 code examples you can edit and run
  • a practice exercise that checks your code
  • a 5-question quiz
  • 12 review cards for spaced repetition

Try a quiz question

Why are data quality checks run by the team that consumes a dataset, and not left to the team that produces it?

  1. The producer can't know every use of the data, or what would break each one
  2. The producer isn't able to run SQL against its own tables
  3. The consumer's checks run faster than the producer's would
  4. Checks only work on data after it has been copied

Answer it at the end of the lesson to see if you're right, and why.

Before this lesson

Keep learning

Where this lesson leads.

575 lessons in thirteen phases. 176 are free without an account, and a free account opens the rest. See every lesson.

Phase 11 · about 34 hours of study

Professional practice

Design, ship and run real systems, work well in a team, and get hired 43 lessons 1 free
  1. System design fundamentals FreeRequirements, rough numbers, the standard parts and where systems break.40 min
  2. Caches, queues and background jobsSpeed reads with caches, and move slow work to queues that retry safely.40 min
  3. Designing AI systemsLatency, cost, routing, fallbacks and guardrails for features built on models.40 min
  4. Containers and deploymentPackage an app as an image, configure it with the environment, and roll out safely.45 min
  5. CI/CD pipelinesTest every change automatically, and ship with gates, flags and rollbacks.40 min
  6. Docker ComposeSeveral containers together: service names, volumes, secrets and start order.40 min
  7. Kubernetes basicsDesired state, Deployments and Services, labels, probes and rolling updates.40 min
  8. Cloud for AI engineersAccess policies and least privilege, storage, compute choices, managed model endpoints, secrets and budgets.30 min
  9. Metrics, dashboards and alert rulesCounters, gauges and histograms, a rate from a counter, a percentile from buckets, a dashboard panel and an alert rule that fires on a sustained condition.40 min
  10. Reliability and incidentsSLOs, error budgets, alerts, on-call and blameless postmortems.40 min
  11. Incident lab: from the page to the postmortemOne incident run from the page to the all-clear: decide the mitigation, work out the impact in failed requests and error budget, and write a blameless postmortem, a status update and a runbook entry.40 min
  12. Measuring an AI feature from its logsTyped views over event logs, metric specs, cost from a price table, and rollups you can rerun.40 min
  13. A/B testing a model changeHash assignment, per-user results from raw logs, sample ratio checks, guardrails and a decision rule.40 min
  14. Columnar files and a local warehouseRows against columns, the encodings that shrink a column, Parquet's row groups and the ranges that let a query skip them, and SQL over files with DuckDB.40 min
  15. Data quality checks and contractsSix properties of a delivery you can test, each as a count of rows that break it, a contract with the producer, and when to block, quarantine or warn.40 min
  16. Transformations: staging, models and testsRaw, staging, facts and marts as SQL models run in dependency order in DuckDB, tests as queries that count the rows that break a rule, an incremental load that is safe to rerun, and the same models as dbt files.45 min
  17. Streams and windowsEvents that never stop arriving: event time against arrival time, tumbling, sliding and session windows, a watermark for late events, and counting each event once.40 min
  18. Labels and annotator agreementLabels as measurements: agreement and Cohen's kappa, the pair of classes annotators confuse, votes and gold questions, and what wrong labels do to a model and a test score.45 min
  19. Experiment tracking and reproducible runsA run record of settings, seed, data and code behind every score, a run log in a dozen lines, and comparing settings across seeds.40 min
  20. Hyperparameter searchWhat a grid costs, random search on a log scale, stopping weak settings early, and why the winner's validation score flatters it.40 min
  21. Versioning data and modelsFiles stored under the hash of their contents, a manifest that names a whole dataset, and lineage from data and code to each model.35 min
  22. A model registry and promotionVersions under one name, an alias for the live one, gates a candidate must pass, group by group, and a rollback that moves a pointer.40 min
  23. Feature pipelines and training-serving skewWhy a model that passes offline fails live: statistics refit at serving, two code paths, and features from the future, with the fix for each.40 min
  24. Batch and online inferencePredict ahead of time or on request: when the inputs exist, latency near a server's limit, batching what waits, cost, and a hybrid with a freshness limit.40 min
  25. Monitoring a model: drift and decayWhy a model worsens with no code change: data drift and concept drift, a measure of how far inputs have moved, thresholds set from noise, and labels that arrive late.45 min
  26. Retraining and pipelinesWhen to retrain and on which weeks, training as steps in dependency order, a cache that reruns only what changed, and guards before and after.45 min
  27. Designing ML systemsOne method for an ML design, goal and metric to retraining, with the sums that size it: requests a second at peak, a year of feature storage, a label delay and a latency budget.45 min
  28. Datasheets: documenting a datasetThe document that travels with a dataset: a profile by column and by group, claims a program can check, splits that don't leak, a scan for personal data, and the license and limits only people can write.45 min
  29. Curation, active learning and synthetic dataThree levers on a training set: removing copies and broken rows, letting a model choose which rows to label, and manufacturing rows, which helps only when their source knows more than you do.45 min
  30. Measuring fairnessBreak a model's results down by group: selection rate, true and false positive rates and precision, why they can't all be equal, what each fix costs, and when a group is too small to judge.40 min
  31. Privacy engineeringWhy removing names isn't anonymity, how to hold less (minimization, retention, keyed pseudonyms), the idea behind differential privacy and its budget, what a model remembers, and a deletion that reaches every store.40 min
  32. A red-team labAttack a simulated assistant methodically: eight kinds of attack run as a suite, success rates beside ordinary requests blocked, fresh attacks to test the fixes, where a defense belongs, and findings kept as release tests.45 min
  33. What regulation asks of an engineerRisk tiers that follow a system's use, the requirements for high-risk systems as artifacts an engineer builds, evidence tied to a model version, a model card a program can check, and legal dates kept as data.35 min
  34. Git for teamsBranches, pull requests, rebasing and how a three-way merge finds conflicts.40 min
  35. Writing for engineersREADMEs, design docs, commit messages and bug reports people can act on.35 min
  36. Scoping and estimating workA vague request cut into tickets with acceptance criteria and a first shippable slice, estimates as ranges, an honest update for a slip, and a ticket linked to its pull request.30 min
  37. Roles, résumé and the searchWhat AI, ML and data engineering postings ask for, résumé bullets that carry a number, a 100-word project summary, a take-home as a portfolio piece and how to read an offer.35 min
  38. Interviews and your portfolioShow your work, and solve interview problems the way professionals do.40 min
  39. Code reviewWhat to look for, how to give and take feedback, and automating the routine checks.35 min
  40. Reading unfamiliar codeFind your way around a big codebase fast, with tools that map it for you.35 min
  41. Working with AI coding toolsGive agents good context, keep tasks small, and check every change they make.35 min
  42. Capstone: ship a production AI serviceAn AI-backed service with validation, caching, rate limits, fallbacks, logging and evals.45 min
  43. Capstone: take a model to productionA churn model through a data contract, a tracked run, a registry gate by slice, hybrid serving, a PSI monitor and one retrain, shipped with a datasheet, a model card, a fairness audit, a design doc and a runbook.45 min

Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write