Index Ai Academy
Phase 11 · Working in a team

Scoping and estimating work

A vague request cut into tickets with acceptance criteria and a first shippable slice, estimates as ranges, an honest update for a slip, and a ticket linked to its pull request.

About 30 minutes · A free account opens the whole lesson.
Already have an account? Sign in to open this lesson

What it covers8

Why it matters

Most missed deadlines are not slow engineers: they are work that was never cut small enough to see. A request such as "let customers get a refund" hides a migration, an endpoint, business rules, an email and a screen, and every one of those can slip. This lesson gives you the habits of a person nobody has to chase: tickets you can finish and check, estimates written as ranges, and an honest update when the range was wrong.

What you'll be able to do

  • Cut a vague request into small tickets with acceptance criteria and a first slice that ships alone
  • Estimate a ticket as a range, and combine ranges without adding every worst case
  • Write a status update for a slip that gives a number, a range and a request
  • Link a ticket to its pull request and say what you are doing at stand-up
Quick check
A guess before you start. A request says "let customers get a refund". What is the first thing to do with it?
  1. Cut it into small pieces that can each be finished and checked
  2. Estimate the whole thing as one number
  3. Start with the part you like best
  4. Wait until the request is fully specified by everyone
Show the answer

Cut it into small pieces that can each be finished and checked

A request this size hides several pieces of work. Cutting it up is what makes it plannable.

The practice: a request cut into tickets

You met the habit of small, reviewable changes in Git for teams: one thing per pull request. The same habit applies a step earlier, to the work itself.

A ticket is one piece of work with four parts: a title that starts with a verb ("Add a refunds table"), acceptance criteria that say how anyone can check it is done, a size small enough to finish in about three days, and an estimate written as a range. Criteria are easiest to write as given, when, then: given an order that was paid, when a support agent refunds it, then its status is "refunded" and the customer's balance goes up by the amount.

Here is the same piece of work, written weakly and strongly.

blankWeakStrong
Title"Refunds""Add a refunds endpoint that marks a paid order refunded"
Done means"It works""Given a paid order, when POST /refunds is called with its id, then the order's status is refunded and the amount is returned"
Size"A couple of weeks""About 3 days, with a migration ticket before it"
Estimate"5 days""Best 1, likely 3, worst 8"
SliceAll five pieces, shipped togetherThe migration and endpoint, behind a flag, first

The strong ticket can be checked by someone who did not write it, and it can ship before the rest.

Try it now

Each bar is a ticket at its likely size. Two people can work in parallel only once the endpoint exists.

Quick check
What does this print: the total of the likely sizes for the five refund tickets (2, 3, 3, 1 and 4 days)?
likely = [2, 3, 3, 1, 4]
print(sum(likely))
  1. 13
  2. 14
  3. 7
  4. 30
Show the answer

13

2 + 3 + 3 + 1 + 4 is 13 days of likely work. That is not yet a plan: it ignores that two people share it and that estimates are ranges.

The first shippable slice

Cut the work along dependencies, and find the smallest set of tickets that gives a user something real. For refunds, the migration, the endpoint and its rules can ship behind a feature flag: support agents can already refund by hand. The email and the admin button can follow. A first slice has two virtues. It puts working code in front of real use early, where problems show up. And if the work slips, you can still ship something, because the slice stands alone.

Quick check
Which set of tickets is the best first slice for refunds?
  1. The migration, the endpoint and its rules and tests
  2. The admin button alone
  3. All five at once, so the feature is complete
  4. The email and the admin button
Show the answer

The migration, the endpoint and its rules and tests

Those three give agents a working refund, and the other two build on it. The email and button cannot work without the endpoint.

Quick check
From Git for teams: how big should a pull request be?
  1. One thing, small enough to review in about 15 minutes
  2. One line, always
  3. Everything for the feature, so it is reviewed together
  4. As big as the author likes, if CI is green
Show the answer

One thing, small enough to review in about 15 minutes

Small pull requests are reviewed faster and merge with fewer conflicts. A ticket small enough for three days usually gives a pull request of that size.

In your own words
In your own words: what makes a ticket "small enough", and what does a first shippable slice protect you from?

Write a sentence or two, then compare it with a good answer in the lesson.

Your turn: estimate a range and tell the truth about a slip

A single estimate hides what you do not know. Write three: best if it goes well, likely and worst. A common way to combine them weights the likely value:

expected = (best + 4 × likely + worst) / 6 and spread = (worst − best) / 6.

The spread is a rough measure of how unsure you are. A wide range is not a failure to estimate: it is the honest answer, and it points at the part of the ticket you should look into first.

Setup (runs before every cell) · 17 lines

The lesson runs this once, before its first cell, and every cell, drill and example starts with what it defines. A cell that changes a setup object changes it for the cells after it, until the page reloads.

import math

TICKETS = [
    ("Migration", 1, 2, 5),
    ("Endpoint", 1, 3, 8),
    ("Rules and tests", 2, 3, 4),
    ("Refund email", 1, 1, 3),
    ("Admin button", 2, 4, 10),
]


def expected(best, likely, worst):
    return (best + 4 * likely + worst) / 6


def spread(best, worst):
    return (worst - best) / 6
for name, best, likely, worst in TICKETS:
    print(
        name.ljust(16), round(expected(best, likely, worst), 2), "+/-",
        round(spread(best, worst), 2),
    )
total = sum(expected(b, l, w) for n, b, l, w in TICKETS)
sd = math.sqrt(sum(spread(b, w) ** 2 for n, b, l, w in TICKETS))
print("total", round(total, 2), "days, spread", round(sd, 2))
print("range", round(total - 2 * sd, 1), "to", round(total + 2 * sd, 1))
print("sum of worst cases:", sum(w for n, b, l, w in TICKETS))

The expected sizes are 2.33, 3.5, 3.0, 1.33 and 4.67 days, which add to 14.83. The spreads do not add the same way. Independent errors partly cancel, so the total spread is the square root of the sum of the squares: 1.95 days. A range of two spreads either side is 10.9 to 18.7 days. Adding every worst case gives 30 days, which is far too pessimistic: it assumes all five tickets go as badly as possible at once. So you plan for about 15 days, tell people "between 11 and 19", and watch the widest tickets.

Quick check
Why is the total range for five tickets narrower than the sum of their ranges?
  1. Rounding makes the range smaller
  2. The best cases add up and the worst cases never happen
  3. Every ticket is estimated exactly
  4. Errors are independent, so some go over while others come in under
Show the answer

Errors are independent, so some go over while others come in under

It is unlikely that every ticket hits its worst case together. Independent errors partly cancel, which is why the spread adds as a square root.

Quick check
What does this print: the slip of a 15-day plan that took 19 days, as a percent of the plan?
plan, actual = 15, 19
print(round((actual - plan) / plan * 100))
  1. 27
  2. 4
  3. 0.27
  4. 21
Show the answer

27

The 4 extra days over the plan of 15 is 0.267, which is 27 percent.

Worked example

Worked example: dependencies set the calendar

Thirteen days of likely work does not mean thirteen days on the calendar. With two people, what matters is the longest chain of tickets that depend on each other.

Uses the setup above, 17 lines.

tickets = {
    "a": (2, []), "b": (3, ["a"]), "c": (3, ["b"]), "d": (1, ["b"]), "e": (4, ["b"]),
}
finish = {}
for name in ["a", "b", "c", "d", "e"]:
    days, after = tickets[name]
    start = max((finish[x] for x in after), default=0)
    finish[name] = start + days
print(finish)
print("longest chain:", max(finish.values()), "days of calendar time")
print("work in all:", sum(days for days, after in tickets.values()), "days")
It runs right here in your browser, with nothing to install.

The migration (2 days) comes first, the endpoint (3) after it, and then the other three can run side by side: the admin button takes 4 days, the rules take 3 and the email 1, so two people finish them in the same 4 days. The longest chain is a, b, e: 9 days on the calendar for 13 days of work. That chain is the one to watch. A day lost on the migration, the endpoint or the admin button moves the end date, and a day lost on the email does not.

Drill

The expected size of a plan

Write plan(tickets). tickets is a list of (name, best, likely, worst). Return a tuple (total, spread): total is the sum of each ticket's (best + 4 * likely + worst) / 6, and spread is the square root of the sum of the squares of each ticket's (worst - best) / 6. Round both to 2 places.

Uses the setup above, 17 lines.

import math


def plan(tickets):
    return 0.0, 0.0


print(plan([("a", 1, 2, 5), ("b", 1, 3, 8)]))
In the lesson, your code is checked as you write it.
Drill

Is this ticket ready?

Write ticket_problems(ticket). ticket is a dictionary with "title", "criteria" (a list of strings) and "estimate" (a tuple (best, likely, worst)). Return a list of problems, in this order, or an empty list: "no criteria" if the list is empty, "criteria not in given/when/then form" if any criterion lacks given, when or then (any case), "estimate not ordered" unless best <= likely <= worst, and "too big" if the expected size, (best + 4 * likely + worst) / 6, is over 3 days.

Uses the setup above, 17 lines.

def ticket_problems(ticket):
    return []


print(ticket_problems({"title": "Refunds", "criteria": [], "estimate": (1, 3, 8)}))
In the lesson, your code is checked as you write it.
Written brief · Tickets

Split a feature into tickets

Your product manager asks: "Let customers get a refund on an order, and email them when it is done." Write the tickets you would put on the board, each small enough to finish in about three days, and name the first slice that can ship by itself.

It must

  • Gives each ticket an acceptance criterion in given, when, then form
  • Gives each ticket three estimates
  • Names a first slice that works on its own

Go further

  • Orders the tickets by what depends on what

The template

## First shippable slice
Which tickets, and what a user can do after them.

## Ticket 1
Title, acceptance criteria (given, when, then), best, likely and worst days.

## Ticket 2
Same again.

## Ticket 3
Same again.

## Not in this work
What you are leaving out on purpose.
In the lesson, you write over it, 140 to 340 words, and the tutor can mark it.
Written brief · Status update

Write the status update for a slip

The refunds work was planned at 15 days and you are now on day 17 with the admin button and the email still to do. Write the update to your manager: what is done, what slipped and why, the new date, and what you need.

It must

  • States the slip in days against the plan
  • Gives a new date as a range
  • Asks for one specific thing

Go further

  • Offers a smaller slice that could ship on the original date

The template

## Where it stands
Done, and not done.

## What slipped
By how much, and the reason in one sentence.

## New date
A range, and what would move it.

## What I need
One decision or one piece of help.
In the lesson, you write over it, 60 to 160 words, and the tutor can mark it.

What a good update sounds like

At stand-up and in writing, the same rules apply. Lead with the number. "I am two days behind the plan" is clearer than "things have been a bit tricky". Say why in one sentence, about the work and not about people. Give a range for the new date, and name what would move it, such as "if the review comes back by Wednesday". End with a request: one decision, one review, one piece of information. Bad news early is a problem someone can help with; the same news at the deadline is an incident.

In your own words
The plan said 15 days and you are at day 17 with work left. What goes in the update, and what must you not do?

Write a sentence or two, then compare it with a good answer in the lesson.

On the job: the ticket, the pull request and stand-up

On a team the ticket lives in a tracker, and the work leaves a trail. An issue template makes every ticket start with the same four parts, so none arrives without criteria. A pull request names the ticket it closes, so the board moves when the code merges and a reviewer can read the criteria beside the diff. At stand-up you say three things: what you finished, what you will do next, and what is blocking you, in about thirty seconds. A blocker is a request: name who can unblock it.

Where it comes up in a week

A product manager drops a one-line request into the team channel. You reply with questions, not a date: who uses it, what does done look like, what can wait? An hour later there is a list of five tickets, each with criteria and a range, and a first slice marked. At stand-up on Thursday you are a day behind on the endpoint, so you say so with the new range, and ask the reviewer you need for the pull request that is up.

Goes deeper in

In your own words
What are the three things you say at stand-up, and why is a blocker a request?

Write a sentence or two, then compare it with a good answer in the lesson.

Practice what you just read

  • 2 code examples you can edit and run
  • a practice exercise that checks your code
  • a 5-question quiz
  • 8 review cards for spaced repetition

Try a quiz question

What makes acceptance criteria useful?

  1. Someone who did not write the ticket can check that it is done
  2. They make the ticket longer
  3. They replace testing
  4. They only matter to managers and product owners, who read them when planning the quarter

Answer it at the end of the lesson to see if you're right, and why.

Before this lesson

Keep learning

Where this lesson leads.

575 lessons in thirteen phases. 176 are free without an account, and a free account opens the rest. See every lesson.

Phase 11 · about 34 hours of study

Professional practice

Design, ship and run real systems, work well in a team, and get hired 43 lessons 1 free
  1. System design fundamentals FreeRequirements, rough numbers, the standard parts and where systems break.40 min
  2. Caches, queues and background jobsSpeed reads with caches, and move slow work to queues that retry safely.40 min
  3. Designing AI systemsLatency, cost, routing, fallbacks and guardrails for features built on models.40 min
  4. Containers and deploymentPackage an app as an image, configure it with the environment, and roll out safely.45 min
  5. CI/CD pipelinesTest every change automatically, and ship with gates, flags and rollbacks.40 min
  6. Docker ComposeSeveral containers together: service names, volumes, secrets and start order.40 min
  7. Kubernetes basicsDesired state, Deployments and Services, labels, probes and rolling updates.40 min
  8. Cloud for AI engineersAccess policies and least privilege, storage, compute choices, managed model endpoints, secrets and budgets.30 min
  9. Metrics, dashboards and alert rulesCounters, gauges and histograms, a rate from a counter, a percentile from buckets, a dashboard panel and an alert rule that fires on a sustained condition.40 min
  10. Reliability and incidentsSLOs, error budgets, alerts, on-call and blameless postmortems.40 min
  11. Incident lab: from the page to the postmortemOne incident run from the page to the all-clear: decide the mitigation, work out the impact in failed requests and error budget, and write a blameless postmortem, a status update and a runbook entry.40 min
  12. Measuring an AI feature from its logsTyped views over event logs, metric specs, cost from a price table, and rollups you can rerun.40 min
  13. A/B testing a model changeHash assignment, per-user results from raw logs, sample ratio checks, guardrails and a decision rule.40 min
  14. Columnar files and a local warehouseRows against columns, the encodings that shrink a column, Parquet's row groups and the ranges that let a query skip them, and SQL over files with DuckDB.40 min
  15. Data quality checks and contractsSix properties of a delivery you can test, each as a count of rows that break it, a contract with the producer, and when to block, quarantine or warn.40 min
  16. Transformations: staging, models and testsRaw, staging, facts and marts as SQL models run in dependency order in DuckDB, tests as queries that count the rows that break a rule, an incremental load that is safe to rerun, and the same models as dbt files.45 min
  17. Streams and windowsEvents that never stop arriving: event time against arrival time, tumbling, sliding and session windows, a watermark for late events, and counting each event once.40 min
  18. Labels and annotator agreementLabels as measurements: agreement and Cohen's kappa, the pair of classes annotators confuse, votes and gold questions, and what wrong labels do to a model and a test score.45 min
  19. Experiment tracking and reproducible runsA run record of settings, seed, data and code behind every score, a run log in a dozen lines, and comparing settings across seeds.40 min
  20. Hyperparameter searchWhat a grid costs, random search on a log scale, stopping weak settings early, and why the winner's validation score flatters it.40 min
  21. Versioning data and modelsFiles stored under the hash of their contents, a manifest that names a whole dataset, and lineage from data and code to each model.35 min
  22. A model registry and promotionVersions under one name, an alias for the live one, gates a candidate must pass, group by group, and a rollback that moves a pointer.40 min
  23. Feature pipelines and training-serving skewWhy a model that passes offline fails live: statistics refit at serving, two code paths, and features from the future, with the fix for each.40 min
  24. Batch and online inferencePredict ahead of time or on request: when the inputs exist, latency near a server's limit, batching what waits, cost, and a hybrid with a freshness limit.40 min
  25. Monitoring a model: drift and decayWhy a model worsens with no code change: data drift and concept drift, a measure of how far inputs have moved, thresholds set from noise, and labels that arrive late.45 min
  26. Retraining and pipelinesWhen to retrain and on which weeks, training as steps in dependency order, a cache that reruns only what changed, and guards before and after.45 min
  27. Designing ML systemsOne method for an ML design, goal and metric to retraining, with the sums that size it: requests a second at peak, a year of feature storage, a label delay and a latency budget.45 min
  28. Datasheets: documenting a datasetThe document that travels with a dataset: a profile by column and by group, claims a program can check, splits that don't leak, a scan for personal data, and the license and limits only people can write.45 min
  29. Curation, active learning and synthetic dataThree levers on a training set: removing copies and broken rows, letting a model choose which rows to label, and manufacturing rows, which helps only when their source knows more than you do.45 min
  30. Measuring fairnessBreak a model's results down by group: selection rate, true and false positive rates and precision, why they can't all be equal, what each fix costs, and when a group is too small to judge.40 min
  31. Privacy engineeringWhy removing names isn't anonymity, how to hold less (minimization, retention, keyed pseudonyms), the idea behind differential privacy and its budget, what a model remembers, and a deletion that reaches every store.40 min
  32. A red-team labAttack a simulated assistant methodically: eight kinds of attack run as a suite, success rates beside ordinary requests blocked, fresh attacks to test the fixes, where a defense belongs, and findings kept as release tests.45 min
  33. What regulation asks of an engineerRisk tiers that follow a system's use, the requirements for high-risk systems as artifacts an engineer builds, evidence tied to a model version, a model card a program can check, and legal dates kept as data.35 min
  34. Git for teamsBranches, pull requests, rebasing and how a three-way merge finds conflicts.40 min
  35. Writing for engineersREADMEs, design docs, commit messages and bug reports people can act on.35 min
  36. Scoping and estimating workA vague request cut into tickets with acceptance criteria and a first shippable slice, estimates as ranges, an honest update for a slip, and a ticket linked to its pull request.30 min
  37. Roles, résumé and the searchWhat AI, ML and data engineering postings ask for, résumé bullets that carry a number, a 100-word project summary, a take-home as a portfolio piece and how to read an offer.35 min
  38. Interviews and your portfolioShow your work, and solve interview problems the way professionals do.40 min
  39. Code reviewWhat to look for, how to give and take feedback, and automating the routine checks.35 min
  40. Reading unfamiliar codeFind your way around a big codebase fast, with tools that map it for you.35 min
  41. Working with AI coding toolsGive agents good context, keep tasks small, and check every change they make.35 min
  42. Capstone: ship a production AI serviceAn AI-backed service with validation, caching, rate limits, fallbacks, logging and evals.45 min
  43. Capstone: take a model to productionA churn model through a data contract, a tracked run, a registry gate by slice, hybrid serving, a PSI monitor and one retrain, shipped with a datasheet, a model card, a fairness audit, a design doc and a runbook.45 min

Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write