Bits, bytes and floats: how numbers are stored
Why this matters
Inside a computer, every number is a row of tiny switches, each on or off. How many switches a number gets decides how big it can be and how exact. That's why 0.1 + 0.2 isn't quite 0.3, why a model's memory is its number of weights times 4 bytes, and why engineers run models in "half precision" or squeeze them into 8 bits. This lesson opens the box: counting in twos, then floats, then what it all costs a model.
- Read and write whole numbers in base 2, and say how many values n bits can hold (2ⁿ)
- Explain a float as a sign, a power of two and a fixed number of binary digits, and why that limits precision
- Use bin() and int(text, 2) in Python, and see why testing floats with == goes wrong
- Work out a model's memory from its count of numbers and the bytes each takes, and the error 8-bit rounding adds
The math: counting in twos
Purpose. Write numbers with only two digits, 0 and 1, and see what a fixed number of them can hold.
In base ten, from lesson 1, each place is worth ten times the place on its right: ones, tens, hundreds. Base two works the same way with twos. Each place is worth twice the place on its right: 1, 2, 4, 8, 16 and so on, the powers of two from lesson 10. Its only digits are 0 and 1, so each place is either used or not. A binary digit is called a bit.
In plain words: label the places from the right 1, 2, 4, 8. Add up the worth of every place that holds a 1. The small 2 says the digits are in base 2.