Glossary · Phase 1 · Building with LLMs
A/B test
Show two versions to two random groups and compare the results, to learn which one really works better.
Try it now
Use the cheapest grader that measures what you care about.
Learn it properly
Evals · Measure output quality with test sets instead of vibes. About 30 minutes.
Other terms from the same lesson
- baseline: A simple reference result to beat, such as always guessing the most common answer. A model is only useful if it does better.
- eval: A set of test cases with expected results, used to measure how well an AI system does its job.
- held-out: Data kept aside and never used for training, so you can test the model on examples it hasn't seen.
Never trained an AI?
Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account.