Glossary · Phase 5 · Transformers
N-gram
A model that predicts the next item from the previous n − 1 items. A bigram uses one, a trigram two.
Try it now
A bigram model has one row like this for each letter. Generating text is just sampling from the current row, again and again.
Learn it properly
A bigram model · The simplest language model, built from counts. About 35 minutes.
Other terms from the same lesson
- bigram: A pair of neighboring tokens. A bigram model predicts the next token from only the one before.
Never trained an AI?
Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account.