Glossary · Phase 4 · Deep learning
Hidden state
An RNN's running memory: a list of numbers it updates after each word and passes on to the next step.
Try it now
The same weights run at every step, so any length of text works. The memory is what fades over long distances.
Learn it properly
Recurrent networks · Process sequences step by step, and their limits. About 40 minutes.
Other terms from the same lesson
- GRU: A simpler cousin of the LSTM with fewer gates. It also helps an RNN remember across long sequences.
- LSTM: An RNN with gates that decide what to keep, forget and output, so it can carry information across many more steps.
- RNN: A network that reads a sequence one step at a time, carrying a memory forward.
Never trained an AI?
Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account.