The glossary
Every word you'll meet in AI and machine learning, explained in everyday language.
A trick that lets gradients pass through rounding as if it were the identity, so quantization-aware training can learn.
Everything so far rounds after training, which is post-training quantization. If you can train again, you can let the model adapt to the rounding. In quantization-aware training the forward pass uses the rounded weights, so the loss sees the damage. The gradient has a problem: rounding is flat almost everywhere, so its slope is zero. The fix is the straight-through estimator: compute the gradient at the rounded weights and apply it to the float weights as if rounding were not there.
The largest possible error is half a gap, 0.2. Zero sits exactly on a mark.
Quantization in practice: zero points, calibration and formats · Scale and zero point, per-channel scales, calibration sets, error on a layer's output, quantization-aware training and the file formats you meet at work. About 35 minutes.
1286 terms in the glossary, each with the lesson that teaches it.
Every word you'll meet in AI and machine learning, explained in everyday language.
Scale and zero point, per-channel scales, calibration sets, error on a layer's output, quantization-aware training and the file formats you meet at work.
Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write