Powers, roots and powers of ten

print(10 ** 3, 2 ** 10)

A power is repeated multiplication

2 ** 5 means "multiply 2 by itself, five 2s in all": 2 × 2 × 2 × 2 × 2 = 32. The number being multiplied is the base and the count is the exponent. On paper it's written small and raised, 2⁵, and said "two to the fifth". Two exponents have their own names: x ** 2 is "x squared" (the area of a square with side x), and x ** 3 is "x cubed".

print(2 ** 5)
print(7 ** 2, 7 ** 3)
print(5 ** 1, 5 ** 0)    # anything to the power 1 is itself; to the power 0 is 1

Why is 5 ** 0 equal to 1? Count down: 5³ is 125, 5² is 25, 5¹ is 5. Each step divides by 5, so the next step, 5⁰, is 5 ÷ 5 = 1. That pattern holds for any base except 0.

Exponents add when you multiply powers of the same base: 2³ × 2⁴ = 2⁷, because it's three 2s and then four more. That's why 10³ × 10⁶ = 10⁹: a thousand millions is a billion.

Powers of ten and e-notation

Powers of ten are how big and small numbers are written: 10 ** 3 is a thousand, 10 ** 6 a million and 10 ** 9 a billion. Negative exponents go the other way, dividing instead of multiplying: 10 ** -1 is 0.1 and 10 ** -3 is 0.001.

Writing 70,000,000,000 is error-prone, so scientists write 7 × 10¹⁰, and code writes e-notation: 7e10. The number after the e is the power of ten, so 3e-4 is 3 × 10⁻⁴, which is 0.0003:

print(7e10)
print(3e-4)
print(1e6 == 1000000)
print(0.00001)        # Python switches to e-notation for very small floats

A handy way to read e-notation: a positive exponent moves the decimal point that many places right, a negative one moves it left. For long whole numbers in your own code, Python also lets you group digits with underscores: 70_000_000_000 is the same number as 7e10, but stays a whole number.

print(2.5e3, 4e-2)

bytes_per_param = 2

params = 0          # 70 billion, written in e-notation
total_bytes = 0
gigabytes = 0

print(params, total_bytes, gigabytes)

Orders of magnitude

When two numbers are far apart, the useful question is often "how many powers of ten apart?" That count is the order of magnitude. A thousand (10³) and a million (10⁶) are three orders of magnitude apart. A model with 7 × 10⁹ parameters and one with 7 × 10¹¹ are two apart: one is a hundred times bigger than the other.

Rough sizes are often all a decision needs. If one design costs about 10² dollars a month and another about 10⁴, the details hardly matter: one is a hundred times cheaper. Engineers call this a back-of-the-envelope estimate, and powers of ten are what make it quick.

Roots undo powers

A square root answers "what number, multiplied by itself, gives this?" Since 5 × 5 = 25, the square root of 25 is 5, written √25. In Python, raising to the power 0.5 takes a square root:

print(25 ** 0.5)
print(2 ** 0.5)       # the side of a square with area 2
print(8 ** (1/3))     # a cube root: 2 × 2 × 2 = 8

Squaring and square roots undo each other, the way multiplying and dividing do: square 5 to get 25, take the root to get 5 back. Roots always give a float, and most aren't whole numbers. The square root of 2 goes on forever. Python's math module has a sqrt() function too, which you'll meet in the modules lesson; ** 0.5 does the same job. One trap: no real number multiplied by itself gives a negative, so (-4) ** 0.5 gives a strange "complex" result. You won't need those here; if one appears, a negative number got into a square root by mistake.

print(144 ** 0.5)

Doubling, halving and growth

Growth that builds on itself is a power. The numbers lesson grew sign-ups by 5% a week with 1.05 ** 4. Doubling is the same idea with a factor of 2: something that doubles every month is 2³ = 8 times bigger after three months.

A quick estimate, the rule of 72, says how long a steady percentage growth takes to double: about 72 divided by the percentage. At 10% a period, it doubles in about 7 periods:

print(1.10 ** 7)      # about 1.95: nearly doubled
print(72 / 10)        # the rule of 72's estimate
print(1000 * 0.5 ** 3)   # halving three times

Doubling is easy to underestimate. Folding a sheet of paper doubles its thickness each time, and about 42 folds would make a stack thick enough to reach the Moon. That's why "it doubles every year" is a far bigger claim than "it grows by the same amount every year".

Halving works the same way in reverse, with a factor of 0.5. Each halving cuts what's left in half, so after three halvings 1,000 becomes 125.

print(round(1.10 ** 7, 2))

months_per_doubling = 6

doublings = 0
factor = 0
decade_factor = 0

print(doublings, factor, decade_factor)

Powers all through AI

You'll see these ideas everywhere. Model sizes are quoted in powers of ten: a 7B model has about 7 × 10⁹ parameters. Learning rates are written in e-notation, like 3e-4. Squaring appears in the most common way to score a prediction, the squared error, which makes big misses count much more than small ones. A square root undoes it again in the standard deviation, coming up two lessons from now. Attention, the core of every modern language model, divides its scores by the square root of a vector's length to keep them a sensible size. And charts of training progress often use a log scale, where each step along the axis is ten times the last, so that a thousand and a billion fit on one picture. Even computer memory runs on powers: 8 bits make a byte, and 2 ** 8 = 256 is why one byte can hold a number from 0 to 255.

print(10 ** -2)