Glossary · Phase 6 · Modern LLM engineering
Draft model
In speculative decoding, the small fast model that guesses the next few tokens for the big model to check.
Try it now
The first rejected token ends the round, so acceptance rate matters far more than draft length. This is a simplified model of the speed-up.
Learn it properly
Speculative decoding · Draft with a small model, verify with a big one. About 35 minutes.
Other terms from the same lesson
- lossless: Changing how something is computed without changing the result at all, such as speculative decoding producing exactly what the big model would have.
- speculative decoding: Speeding up generation by letting a small model draft several tokens and a big model check them at once.
Never trained an AI?
Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account.