Glossary · Phase 4 · Deep learning
Residual block
A layer that adds its result to its own input (x + f(x)), so the input always has a clean path through.
Try it now
Thirty plain layers lose almost all the gradient. With skip connections, it arrives intact.
Learn it properly
Residual connections · Skip connections that let gradients flow. About 30 minutes.
Other terms from the same lesson
- residual connection: A shortcut that adds a layer's input to its output, so information and gradients flow easily through deep networks.
- residual stream: The running total that flows through a transformer: each layer reads it and adds its result back in, instead of replacing it.
Never trained an AI?
Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account.