Glossary · Phase 4 · Deep learning

LSTM

Also written: long short-term memory

An RNN with gates that decide what to keep, forget and output, so it can carry information across many more steps.

The LSTM (long short-term memory, 1997) was the fix. Alongside the hidden state, it keeps a second memory called the cell state. Think of it as a notebook. A plain RNN rewrites its whole memory at every step, through tanh. An LSTM mostly adds new notes to its notebook and crosses a few old ones out.
From the lesson Recurrent networks

Try it now

The same weights run at every step, so any length of text works. The memory is what fades over long distances.

Learn it properly

Recurrent networks · Process sequences step by step, and their limits. About 40 minutes.

Create a free account See what's in the lesson

Other terms from the same lesson

GRU: A simpler cousin of the LSTM with fewer gates. It also helps an RNN remember across long sequences.
hidden state: An RNN's running memory: a list of numbers it updates after each word and passes on to the next step.
RNN: A network that reads a sequence one step at a time, carrying a memory forward.

Never trained an AI?

Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account.

Teach it to write