Index Ai Academy
Glossary · Phase 4 · Deep learning

Sequence to sequence

Also written: seq2seq, sequence-to-sequence, encoder-decoder

A model in two parts: an encoder that reads a whole input, and a decoder that writes an output of a different length, one piece at a time.

Try it now

The three attention lines on three encoder states. The context has the size of one state.

Learn it properly

Sequence to sequence, and the invention of attention · Build an encoder and decoder, measure the bottleneck of a fixed-size handoff, add attention in three lines, read an attention map with care, and see how one mistake compounds after teacher forcing. About 45 minutes.

Other terms from the same lesson

  • exposure bias: The gap between training, where a model is always given the correct previous output, and use, where it is given its own and can be thrown off by its mistakes.
Keep learning

More in plain English.

1286 terms in the glossary, each with the lesson that teaches it.

1286 terms

The glossary

Every word you'll meet in AI and machine learning, explained in everyday language.

Phase 4 · lesson 45 min

Sequence to sequence, and the invention of attention

Build an encoder and decoder, measure the bottleneck of a fixed-size handoff, add attention in three lines, read an attention map with care, and see how one mistake compounds after teacher forcing.

Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write