Two variables at once: covariance and correlation
Why this matters
So far every random variable has stood alone. Real data comes in pairs and more: temperature and rentals, height and weight, two pixels of the same picture. The question is whether they move together, and how strongly. Covariance measures it in the data's units; correlation rescales it to a number from −1 to 1 that's easy to read. In ML this tells you which features carry the same information, which ones a model can do without, and it ties back to two earlier lessons: the covariance matrix whose eigenvectors gave PCA's directions (lessons 62 and 63), and correlation, which turns out to be exactly lesson 51's cosine similarity, applied to centered data.
- Read a joint table, get each variable's own distribution (the marginals) by summing, and test independence
- Compute covariance by centering, multiplying pair by pair and averaging, and correlation from −1 to 1
- Build a covariance matrix as XᵀX/n on centered data, and check it against np.cov and torch.cov
- Find features that carry the same information, and see that correlation is the cosine of centered data
The math: moving together
Purpose. Do two quantities move together?
A joint table counts two things at once. 100 days, by weather and by rentals:
| blank | high rentals | low rentals |
|---|---|---|
| sunny | 30 | 10 |
| rainy | 15 | 45 |
The marginals are each variable on its own: sum each row for weather (40 sunny, 60 rainy) and each column for rentals (45 high, 55 low). If weather and rentals were independent, P(sunny and high) would equal P(sunny) × P(high) = 0.4 × 0.45 = 0.18. It's actually 0.30, so they're not independent: sun goes with high rentals.