Glossary · Phase 4 · Deep learning
dead ReLU
A ReLU neuron whose input is always negative, so it always outputs 0, gets no gradient and stops learning.
Try it now
Most training bugs are silent, so learn their fingerprints.
Learn it properly
Debugging training · Read loss curves and fix what's wrong. About 35 minutes.
Never trained an AI?
Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account.