Glossary · Phase 6 · Modern LLM engineering
Specification gaming
When a system scores well by exploiting a loophole in how its goal was written, instead of doing what was meant.
Every trained behavior comes from an objective: a scoring rule someone wrote down. Training then hunts for the cheapest way to score well on that rule. It follows what was written, not what was meant. Doing well on the written rule in a way the designer didn't intend is called specification gaming.
Try it now
The fixes differ: access controls for misuse, evals for mistakes, and alignment research for the last kind.
Learn it properly
Alignment and safety · Making capable models behave as intended. About 35 minutes.
Other terms from the same lesson
- alignment: Making AI systems reliably do what people intend, safely and honestly.
- Q-learning: A simple reinforcement-learning method that learns a table of how good each action is in each situation.
- red-teaming: Deliberately trying to make a model misbehave, before release, to find and fix the problems first.
- system card: A report published with a model describing its safety testing: what was tried, what went wrong and what was done about it.
Never trained an AI?
Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account.