Very big and very small numbers
Why this matters
AI work mixes numbers of wildly different sizes: billions of parameters, learning rates of a few ten-thousandths, tiny numbers added to keep a division safe. Written out in full they're hard to read and easy to get wrong by a zero or two. Scientific notation writes each one as a small number times a power of ten, so sizes can be read, compared and multiplied at a glance. Python writes it with an e, as in 3e-4.
- Write numbers in scientific notation, a × 10ᵏ, and back again
- Compare and rank numbers by order of magnitude, and know the prefixes from nano to tera
- Read and write e-notation in Python, and find an order of magnitude with math.log10
- Work out a model's memory from its parameter count, and compare learning rates
The math: powers of ten
Purpose. Write any number, however big or small, as a short number times a power of ten.
Powers of ten are ones with zeros: 10³ = 1,000 and 10⁶ = 1,000,000. With negative exponents they're one over: 10⁻³ = 1/1,000 = 0.001.
A number in scientific notation is written a × 10ᵏ, where a is at least 1 and less than 10:
In plain words: move the decimal point until one digit that isn't zero sits in front of it. The power of ten counts the moves: positive for a big number, negative for a small one.