Glossary · Phase 6 · Modern LLM engineering
pass^k
The chance an agent succeeds on all k tries at the same task: a test of reliability, where pass@k tests whether it can succeed at all.
Try it now
A 90% agent looks perfect if you only need one success, and shaky if you need it to work every time. Which number you report changes the story.
Learn it properly
Evaluating agents · Measure multi-step, tool-using behavior. About 35 minutes.
Other terms from the same lesson
- sandbox: A sealed-off test environment where an agent can take real actions without touching real systems or data.
- trajectory: The full record of what an agent did on a task: every message, tool call and result, in order.
Never trained an AI?
Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account.