AI agent interview questions and answers
Questions on designing and running agents: tools, the agent loop, permissions, context, multi-agent systems, security and evaluation.
Coordinating many agents on one large job.
The harness you have built in this course runs one agent: one model calling tools in a loop. Bigger jobs are now often split across many agents that work at the same time. Think of researching a market from dozens of sources, checking every page of a large website, or reading every module of a codebase. Done well, a team of agents is faster and more thorough. Done badly, it multiplies cost and mistakes. This lesson gives you the arithmetic for telling the two apart, before a later lesson builds a subagent.
When the remaining jobs have to happen one after another
You can't load the van before the boxes are packed. Steps that depend on each other set a floor on the time, whatever the team's size. Multi-agent systems hit the same limit, and pay more tokens for every helper.
Picture a big house move. One person can do it alone, but it takes all weekend. With friends, one person packs the kitchen while another packs the books. Someone has to say who does what. Someone also has to check that nothing got left behind.
An AI agent is a model that calls tools in a loop until a task is done. A multi-agent system puts several agents on one job. Most fit one of four patterns:
votes = ["refund", "refund", "replace"] winner = max(set(votes), key=votes.count) print(winner, round(votes.count(winner) / len(votes), 2))
refund 0.67
Two of the three runs said "refund", so it wins with two thirds of the votes. Voting is the simplest kind of team: copies of one worker, and a rule for combining them. This lesson is about teams whose members do different jobs, and about what that costs.
A pipeline
Each step feeds the next in a fixed order, like a production line: that's a pipeline.
Why go to the trouble? There are three main gains:
It can pay off. Anthropic built a research system in which a lead agent directs subagents working in parallel. The company reported that it did about 90% better than a single agent on an internal research test. It also used roughly 15 times as many tokens as a chat conversation. You pay for tokens, so that matters.
Parallel sections
The same instructions run on separate pieces at the same time. A pipeline would pass the work along a line, one agent after another.
Think about cooking a big dinner with friends. Extra cooks can chop vegetables and set the table at the same time. But the roast still needs its full time in the oven. No number of helpers changes that.
Agent work behaves the same way. Here's a research job as a list of tasks. Each task has a length in minutes. It also has prerequisites, the tasks that must finish before it can start:
The code below simulates agents, where is the number of agents working at once. Whenever an agent is free, it picks up any task whose prerequisites are done. With more than one agent, each task also costs an extra half minute for handing the work over.
tasks = { # name: (minutes, prerequisites)
"plan": (3, []),
"pricing": (9, ["plan"]), "reviews": (12, ["plan"]),
"social": (8, ["plan"]), "ads": (10, ["plan"]),
"seo": (11, ["plan"]), "partners": (7, ["plan"]),
"compare": (6, ["pricing", "reviews"]),
"synthesis": (5, ["compare", "social", "ads", "seo", "partners"]),
}
def finish_time(agents, overhead_per_task=0.5):
done, running, clock = {}, [], 0.0 # done: name -> finish time
while len(done) < len(tasks):
busy = [r for r, _ in running]
ready = [t for t, (_, pre) in tasks.items()
if t not in done and t not in busy and all(p in done for p in pre)]
while ready and len(running) < agents: # start ready tasks
t = ready.pop(0)
extra = overhead_per_task if agents > 1 else 0 # a hand-over costs a little
running.append((t, clock + tasks[t][0] + extra))
running.sort(key=lambda r: r[1])
t, clock = running.pop(0) # jump to the next finish
done[t] = clock
return clock
total = sum(m for m, _ in tasks.values())
for agents in [1, 2, 4, 8]:
t = finish_time(agents)
label = f"{agents} agent{'s' if agents > 1 else ''}:"
print(f"{label:<10}{t:5.1f} minutes speedup {total / t:.1f}x")
path = 3 + 12 + 6 + 5
print(f"\nthe critical path (plan, reviews, compare, synthesis): {path} minutes")
Speedup is how many times faster the job gets. It's the one-agent time divided by the new time. With 4 agents, that's 71 ÷ 29, about 2.4.
Going from 1 agent to 4 cuts the job from 71 minutes to 29. But 8 agents save only one more minute. Look at the chain plan → reviews → compare → synthesis. Each step has to wait for the one before it. That chain takes 3 + 12 + 6 + 5 = 26 minutes, however many agents you add. Add the half-minute hand-off for each of its four tasks, and you get the 28 minutes that 8 agents need.
The longest chain of tasks that depend on each other is called the critical path. It can't be split up. The same holds for any team, human or AI. Split work only where the pieces are truly independent, and expect the longest chain to set the pace.
An arrow points from a task to a task that waits for it. The numbers are minutes.
The remaining time is a chain of dependent steps (the critical path) that can't run in parallel
Once the independent work is spread out, the longest chain of dependent steps sets the finish time.
34 minutes
The chain plan → reviews → compare → synthesis now takes 3 + 20 + 6 + 5 = 34 minutes. The other chains are shorter: plan → pricing → compare → synthesis is 23, and plan → SEO → synthesis is 19.
Write finish_time(durations, agents): hand out tasks longest first, each to the agent with the least work so far, and return when the last agent finishes (the biggest total). This simple rule is close to the best possible.
def finish_time(durations, agents): return sum(durations) / agents print(finish_time([30, 10, 20, 20, 10], 2), finish_time([30, 10, 20, 20, 10], 10))
Write a sentence or two, then compare it with a good answer in the lesson.
Think of the party game telephone. A message passes from person to person. Each person usually gets it right, but small slips pile up. By the end, the message is often garbled.
Agent hand-offs work the same way. Each one is a chance to drop or bend information. Say each step in a chain works with probability . That's a number between 0 and 1, so 0.95 means 95% of the time. Now say the chain has steps, and each one succeeds or fails on its own. Then the chance that the whole chain works is:
In plain words: write down the step's success rate once for each step, then multiply them all together. Two steps at 90% give 0.9 × 0.9 = 0.81. So the pair works only 81% of the time, even though each step works 90% of the time.
This is a simplification. Real steps don't always succeed or fail on their own, so treat it as a rough guide.
Each step in a chain gets it right 90% of the time. How often does a chain of one, two or three steps get everything right?
p = 0.9 # each step succeeds 90% of the time for n in [1, 2, 3]: print("steps:", n, " chance all succeed:", round(p ** n, 3))
One step works 0.9 of the time. Two steps: 0.9 × 0.9 = 0.81. Three steps: 0.81 × 0.9 = 0.729.
In Python, p ** n means to the power . So 0.9 ** 3 is 0.9 × 0.9 × 0.9. Each extra step takes away another 10% of whatever chance was left.
print(0.5 ** 3)
0.125
0.5 × 0.5 × 0.5 = 0.125. The whole chain works only 1 time in 8.
Write a sentence or two, then compare it with a good answer in the lesson.
A verifier is a checker placed at a hand-off. It might be another agent or a set of tests. It might also check that the output fits a fixed format, called a schema. When it spots a mistake, the step runs again. That second try is a retry.
Here's how much one checker helps. Take a step that's right 95% of the time. The checker catches 80% of the mistakes, and the step gets one retry. Now the step can work in two ways:
Together that's 98.8%, up from 95%. The code below works this out for chains of different lengths. Its catch setting is the share of mistakes the checker spots. Its retries setting is how many retries a step gets.
def chain_success(p, steps, catch=0.0, retries=0): # one step: right first time, or wrong but caught and retried (same chances) step = p miss = 1 - p for _ in range(retries): step += miss * catch * p miss = miss * catch * (1 - p) return step ** steps for steps in [1, 3, 6, 10]: plain = chain_success(0.95, steps) checked = chain_success(0.95, steps, catch=0.8, retries=1) print(f"{steps:>2} steps at 95% each: {plain:.0%} end to end", f" with a verifier (catches 80%, one retry): {checked:.0%}")
Ten steps at 95% each work together only 60% of the time. With a checker and one retry, that rises to 89%. So well-built multi-agent systems put checks at their hand-offs. They use schemas for structured outputs and tests for code. They use citations that can be checked. And an evaluator agent reviews the work before anything reaches a person.
With one retry, the chance that a single step works is:
Here is the catch rate, the share of mistakes the checker spots.
In plain words: the step works if it's right the first time (). It also works if it's wrong (), the mistake is caught (), and the retry is right ( again).
With and , that's 0.95 + 0.05 × 0.8 × 0.95 = 0.988. For ten steps, multiply 0.988 by itself ten times. You get about 0.886, the 89% in the table.
p, c = 0.95, 0.8 step = p + (1 - p) * c * p print(round(step, 3), round(step ** 10, 3))
In the chain_success code, the line step += miss * catch * p adds this second route once per retry. After each retry, miss becomes the chance of needing yet another one: wrong, caught, and wrong again.
94.5%
Right the first time is 90%. Wrong, caught, then right on the retry is 10% × 50% × 90% = 4.5%. Together that's 94.5%.
More agents aren't always better. Watch for three warning signs:
Each subagent re-reads its own instructions and context, so token use rises sharply
You pay for every token each agent reads and writes. Five agents reading their own instructions and context means many more tokens than one agent.
Each subagent reads its own context, so tokens multiply. Write run_cost(subagents, tokens_each, coordinator_tokens, price_per_million): the dollars for the coordinator plus every subagent, rounded to 2 places. Compare one agent doing it all with five subagents.
def run_cost(subagents, tokens_each, coordinator_tokens, price_per_million): return coordinator_tokens * price_per_million / 1_000_000 print(run_cost(0, 0, 120_000, 3), run_cost(5, 80_000, 40_000, 3))
Write a sentence or two, then compare it with a good answer in the lesson.
A typical orchestrator gives the lead model a tool for starting subagents. Each subagent is a separate model call with its own instructions and a fresh context. The picture shows one run: three research subagents at the same time, a checker at the hand-off, and a last call that combines what passed.
In practice the subagents also get tools, such as web search or an MCP server, and the lead chooses the subtasks itself. A later lesson, Subagents, builds that into your harness: a brief out, a short answer back.
On the job: split work across agents when it breaks into independent pieces that you merge at the end: research across many sources, an audit of hundreds of pages, a search through every module of a codebase. Keep one agent on work where every decision depends on the others, such as one design or one bug fix.
The trail club sends a short invitation for every trip, and it wants them written by a small team of model calls instead of by a tired volunteer. Build that team: three drafters write an invitation from the trip's details, each with a different style, a picker chooses the best against a rubric, a guardrail checks the winner, and the result goes out through the practice email outbox.
Multi-agent systems are easy to make expensive and hard to make reliable, so this one is measured: every run reports what it cost, how long it took, and why the winner won. Two traps to design for. A picker that sees the drafts in the same order every time tends to favor a position, so show it the drafts under neutral labels in a shuffled order. And with simulated replies (no API key) the picker gives every draft the same score, so you need a tie-break rule.
Sending uses smtplib as in the notifications lesson: connect to the starter's host and port, starttls(), login(), then send_message(). The academy catches the message in a practice outbox, so nothing really goes out.
TRIP, and TRIP['meet'] appears word for word; a winner that fails falls back to the next-best draftsmtplib (the academy's practice outbox) with a plain-text bodyPRICES), total time, the time the drafts would take in parallel (the slowest drafter), the picker's scores, and whether the guardrail stepped inGET /v1/forecast?trailhead=fern-hollow and require a weather line that the guardrail also verifiesimport smtplib from email.message import EmailMessage import requests TRIP = {"name": "Fern Hollow loop", "day": "Saturday, June 14", "meet": "Fern Hollow trailhead parking lot, 8:30 a.m.", "miles": 7, "bring": ["water", "rain jacket", "lunch"], "leader": "Bo"} FROM, TO = "trips@example.org", "members@example.org" SMTP_HOST, SMTP_PORT, SMTP_PASSWORD = "smtp.example.com", 587, "practice-app-password" PRICES = { # dollars per million tokens: (input, output) "claude-opus-5-5": (4.00, 20.00), "claude-sonnet-5-5": (2.00, 10.00), "claude-haiku-5-5": (0.10, 0.50), } def call_claude(prompt, system, model="claude-opus-5-5"): r = requests.post("https://api.anthropic.com/v1/messages", headers={"x-api-key": "added-by-the-academy-server", "anthropic-version": "2026-10-21"}, json={"model": model, "max_tokens": 500, "system": system, "messages": [{"role": "user", "content": prompt}]}, timeout=90) r.raise_for_status() body = r.json() text = "".join(b["text"] for b in body["content"] if b["type"] == "text") return text, body["usage"] draft, usage = call_claude(f"Write a short invitation for this trip: {TRIP}", "You write warm, brief trip invitations.") print(draft) print(usage) # your code: three drafters, the picker, the guardrail, sending and the run summary
Write a sentence or two, then compare it with a good answer in the lesson.
Which task is the best fit for an orchestrator with parallel subagents?
575 lessons in thirteen phases. 176 are free without an account, and a free account opens the rest. See every lesson.
Questions on designing and running agents: tools, the agent loop, permissions, context, multi-agent systems, security and evaluation.
Learn how to build AI agents by building one: tools, the agent loop, MCP, permissions, context management and evals, ending with a coding agent that stops when its tests pass.
Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write