Glossary · Phase 1 · Building with LLMs
Micro average
Pooling every item from every example before scoring, so examples with more items count for more; macro averaging counts each example equally.
Try it now
Use the cheapest grader that measures what you care about.
Learn it properly
Evals · Measure output quality with test sets instead of vibes. About 30 minutes.
Other terms from the same lesson
- A/B test: Show two versions to two random groups and compare the results, to learn which one really works better.
- baseline: A simple reference result to beat, such as always guessing the most common answer. A model is only useful if it does better.
- eval: A set of test cases with expected results, used to measure how well an AI system does its job.
- held-out: Data kept aside and never used for training, so you can test the model on examples it hasn't seen.
Never trained an AI?
Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account.