Functions and graphs

A marketing function: diminishing returns

Think about cups of coffee. The first one in the morning makes a big difference. The second helps a little. By the fifth you feel almost nothing extra. Each cup adds less than the one before. That pattern is called diminishing returns.

Ad spend often behaves the same way. The first dollars reach the people most likely to buy. Later dollars reach people who care less. So conversions (sales or sign-ups from the ads) don't grow in a straight line with spend. They climb fast, then level off toward a ceiling.

A common model for this is a saturating curve:

Take the symbols one at a time:

• $s$ is the monthly ad spend in dollars. It's the input.
• $c(s)$ is the number of conversions you get for that spend. It's the output.
• $c_{\max}$, said "c max", is the ceiling: the level conversions creep toward but never quite reach.
• $k$ is the spend at which you reach half the ceiling.
• The dot $\cdot$ means multiply.

In plain words: divide your spend by (spend plus k). That gives a fraction between 0 and 1. Multiply the fraction by the ceiling to get your conversions.

Here's a worked example. Say the ceiling is 900 conversions and k is $4,000.

The first $4,000 brings 450 conversions. The next $4,000 brings only 150 more. The $4,000 after that brings 75 more. Each step buys less. The same calculation in Python:

def conversions(spend, c_max=900, k=4000):
    return c_max * spend / (spend + k)

for spend in [0, 4000, 8000, 12000]:
    print(spend, conversions(spend))

Next to each spend it prints the table's last column: 0.0, 450.0, 600.0 and 675.0.

Finding the best budget

More conversions isn't the same as more profit. Say each conversion earns you $60 of margin, which is what's left after costs. Then your profit is that margin minus what you spent on ads:

profit = 60 × conversions − spend

Use the numbers from the table. At $4,000 of spend, profit is 60 × 450 − 4,000 = $23,000. At $12,000 it's 60 × 675 − 12,000 = $28,500. Spend a lot more and profit drops again, because the extra conversions no longer cover the extra spend.

The code below tries 300 budgets from $0 to $30,000. It works out the profit for each and keeps the best one.

import numpy as np
import matplotlib.pyplot as plt

def conversions(spend, c_max=900, k=4000):
    return c_max * spend / (spend + k)

spend = np.linspace(0, 30000, 300)
profit = 60 * conversions(spend) - spend            # $60 margin per conversion, minus spend
best = spend[np.argmax(profit)]
plt.plot(spend, conversions(spend), label="conversions")
plt.plot(spend, profit / 60, label="profit ÷ 60")
plt.axvline(best, linestyle="--", color="gray")
plt.xlabel("monthly ad spend ($)")
plt.legend()
print(f"best spend ≈ ${best:,.0f}, profit ≈ ${profit.max():,.0f}")

It prints a best spend of about $10,736 and a profit of about $28,606. np.argmax finds the position of the largest profit. Then spend[...] looks up the budget at that position.

Now read the graph. Conversions keep rising but flatten out. Profit rises, peaks, then falls. The dashed line marks the peak: the best budget.

Trying every candidate and keeping the best is called grid search. It's simple and reliable when there are one or two inputs. For an AI model it's hopeless. A model can have billions of weights, the numbers it adjusts while it learns. Try 10 values for each of 2 inputs and you need 10 × 10 = 100 tries. Try 10 values for each of a million weights and the count is 1 followed by a million zeros. That's why the next lessons need derivatives. They point toward better settings without trying them all.

Shapes you'll keep meeting

A handful of shapes turn up again and again in machine learning, which is the craft of fitting such shapes to data. Once you can spot them, many formulas start to look familiar.

Line

In $f(x) = ax + b$, the number $a$ sets how steep the line is. The number $b$ is the output when the input is 0.

In plain words: multiply the input by one fixed number, then add another.

With a = 2 and b = 1, input 2 gives 2 × 2 + 1 = 5, and input −2 gives −3.

Bowl

In plain words: subtract 2 from the input, square the result, then add 1.

Input 0 gives (0 − 2)² + 1 = 5. Input 2 gives 0 + 1 = 1, the lowest output there is. Any other input lands higher up the sides of the bowl. Training a model means looking for the bottom of a bowl like this one. Real loss bowls are bumpier, but the idea is the same.

S-curve (sigmoid)

The S-curve is called the sigmoid. Its name is written $\sigma$, the Greek letter sigma. The formula also uses $e$, a fixed number about 2.718 that you'll meet properly in the next lesson.

In plain words: raise $e$ to the power of minus the input, add 1, then divide 1 by that. Big negative inputs give almost 0. Big positive inputs give almost 1. An input of 0 gives exactly 0.5.

Input 2 gives about 0.88. Input −2 gives about 0.12. That squashing is how a model turns any score into something that can be read as a probability.

Hinge (ReLU)

ReLU is short for "rectified linear unit". The $\max(0, x)$ part means "the larger of 0 and x".

In plain words: if the input is positive, keep it. If it's negative, output 0.

Input 2 gives 2. Input −2 gives 0.

Words for describing a graph

• Increasing: the graph goes up as you move left to right.
• Decreasing: it goes down.
• Minimum: its lowest point. Maximum: its highest point.
• Steep or flat: how fast the output changes as the input moves.

Steepness has a proper name, the derivative. It's coming next.

import math

def line(x):    return 2 * x + 1
def bowl(x):    return (x - 2) ** 2 + 1
def sigmoid(x): return 1 / (1 + math.exp(-x))
def relu(x):    return max(0, x)

for x in [-2, 0, 2]:
    print(x, line(x), bowl(x), round(sigmoid(x), 2), relu(x))