The glossary
Every word you'll meet in AI and machine learning, explained in everyday language.
Dividing a sequence's log probability by its length, so that a search doesn't always prefer the shortest output.
The usual repair is in the last column: divide the score by the length. Per word, "I think that it works" is the best of the three, at −0.43. Real systems use this length normalization, often with a tunable penalty.
Press Next three times. The lit boxes are what the beam keeps; the pale ones are thrown away.
Beam search and constrained decoding · Why the likeliest word isn't the likeliest sentence, beam search with log probabilities, its taste for short and repetitive text, and forcing output into a grammar: what that guarantees and what it doesn't. About 40 minutes.
1286 terms in the glossary, each with the lesson that teaches it.
Every word you'll meet in AI and machine learning, explained in everyday language.
Why the likeliest word isn't the likeliest sentence, beam search with log probabilities, its taste for short and repetitive text, and forcing output into a grammar: what that guarantees and what it doesn't.
Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write