The glossary
Every word you'll meet in AI and machine learning, explained in everyday language.
The speedup from adding processors is limited by the part of the work that can't be shared between them.
The gap near 1 is 2 to the power of minus 10, about 0.001.
Training at scale: precision, accumulation, data and many GPUs · Gradient accumulation and what it really saves, scaling the learning rate with the batch, mixed precision and loss scaling, a data loader that starves the GPU, and why eight GPUs aren't eight times as fast. About 45 minutes.
1286 terms in the glossary, each with the lesson that teaches it.
Every word you'll meet in AI and machine learning, explained in everyday language.
Gradient accumulation and what it really saves, scaling the learning rate with the batch, mixed precision and loss scaling, a data loader that starves the GPU, and why eight GPUs aren't eight times as fast.
Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write