Glossary · Phase 6 · Modern LLM engineering
Policy gradient
Training by making high-scoring answers more likely and low-scoring ones less likely, in proportion to how good they were.
Try it now
Nobody writes the reasoning. It emerges because answers with good reasoning get rewarded more often.
Learn it properly
Training reasoning models · Reinforcement learning on verifiable problems. About 40 minutes.
Other terms from the same lesson
- advantage: In reinforcement learning, how much better one answer scored than expected (in GRPO, than its group's average).
- GRPO: Group relative policy optimization: sample several answers to one problem and push up the ones that beat the group's average.
- pass@k: The chance that at least one of k attempts at a problem is correct.
- RLVR: Reinforcement learning with verifiable rewards: training on problems whose answers a program can check, such as math or code with tests.
- self-consistency: Asking a model the same question several times and taking the most common answer.
- test-time compute: Extra computation spent while answering, such as thinking longer or trying several answers, rather than during training.
Never trained an AI?
Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account.