Features and labels

The model is a function

A model takes one row's features and gives back a prediction. It's like a recipe: ingredients go in, a dish comes out. In math you write it like this:

Take the symbols one at a time:

• $\mathbf{x}$, a bold x, is one row's features as a list of numbers. You met lists like this as vectors in Phase 2.
• $f$ is the model. It's a function that turns the features into an answer.
• $\hat{y}$, said "y-hat", is the prediction. The little hat marks a guess.
• $y$ with no hat is the true answer from the label column.

In plain words: feed one row's features into the model, and what comes out is the prediction.

Training means choosing the model so its guesses land close to the true answers. You train on examples where you already know those answers.

A worked example

Here's a very simple model. It predicts churn (1) when the last order was more than 90 days ago. Otherwise it predicts "stays" (0).

One customer placed 2 orders and last ordered 120 days ago. So $\mathbf{x} = [2, 120]$. Since 120 is more than 90, the model's guess is $\hat{y} = 1$. In fact this customer stayed, so $y = 0$. The guess was wrong. Training would adjust the model to make fewer mistakes like this one.

The same steps in Python:

def f(x):
    orders, days_since = x
    return 1 if days_since > 90 else 0   # 1 means "will churn"

x = [2, 120]   # one customer: 2 orders, 120 days since the last one
y = 0          # what really happened: they stayed
y_hat = f(x)
print("prediction:", y_hat, "truth:", y, "right?", y_hat == y)

It prints prediction: 1 truth: 0 right? False.

X and y in code

A dataset has many rows, so code keeps them together. By custom, the whole table of features is called X, with one row per example. The whole column of labels is called y. You'll see these two names in almost every piece of machine learning code.

def f(days):
    return 1 if days > 90 else 0

print(f(30), f(200))

Two kinds of target

"How much rain will fall tomorrow?" asks for a number. "Will it rain tomorrow?" asks for yes or no. The weather data is the same, but the question is a different kind. Machine learning splits targets the same way:

"Regression" is an old word from statistics. Here it means "predict a number". A category is one choice from a fixed list, such as yes or no, or A, B or C.

Classification with two classes, like churn yes or no, is called binary classification. It's the most common kind. These models usually don't give a bare yes or no. They give a probability for the "yes" class, like "72% likely to churn".

Watch out: a column of numbers isn't always a quantity. If regions are stored as 1, 2 and 3, region 3 isn't "three times" region 1. Those are categories that happen to use numbers as names.

Working with pandas

pandas is Python's spreadsheet library. A DataFrame is a table with named columns. It's the usual way to hold a dataset before you hand it to a model as NumPy arrays.

The code below fetches the course's practice dataset of 600 customers from the practice API. It loads them into a DataFrame and prints three things:

import pandas as pd
import requests

df = pd.DataFrame(requests.get("https://api.gradient.test/v1/customers").json()["data"])
print(df.shape)
print(df[["tenure_months", "orders_last_year", "days_since_last_order", "churned"]].head())
print(df["churned"].value_counts())

• df.shape is the size of the table as (rows, columns): 600 customers and 14 columns.
• .head() shows the first five rows of the columns you picked.
• value_counts() counts how often each value appears. 484 customers stayed (0) and 116 churned (1). That's about 19% who churned.

Next, choose the features and the label, and turn them into NumPy arrays:

import pandas as pd
import requests

df = pd.DataFrame(requests.get("https://api.gradient.test/v1/customers").json()["data"])
features = ["tenure_months", "orders_last_year", "avg_order_value", "days_since_last_order", "support_tickets", "trail_club"]
X = df[features].to_numpy()
y = df["churned"].to_numpy()
print(X.shape, y.shape)
print(df.groupby("churned")[features].mean().round(1))

Here's what each part does:

• features is your list of input columns. trail_club is 1 if the customer is in the loyalty program, and 0 if not.
• .to_numpy() turns those columns into a NumPy array. X has shape (600, 6): one row per customer, one column per feature.
• y is the churned column, 600 labels in a row. Its shape prints as (600,).
• groupby("churned") splits the customers into two groups: stayed and churned. Then .mean() averages each feature inside each group.

The group averages already hint at what a model will learn. Churners ordered less: about 3.6 orders last year, against 6.8 for those who stayed. Their last order was longer ago: about 157 days, against 70. And they filed more support tickets.

Leakage: the answer hiding in the inputs

Imagine a practice exam with the answer key printed faintly on the back of each page. You'd score 100%. Then the real exam arrives with no answer key, and you struggle. A model can fall into the same trap.

Here's the rule. A feature is allowed only if you'd know it at the moment you make the prediction. Breaking that rule is called leakage. It's the most common way machine learning projects fool themselves.

Say you predict who will churn next quarter. These features would leak:

• "number of orders next quarter" (it's the future),
• "received a win-back email" (sent because they churned),
• "account closed date" (only exists for churners),
• in this dataset, lifetime_value needs a careful look. It's spend to date. That's fine if it's measured before the churn window starts. It leaks if it includes the months after.

A leaky model scores beautifully in testing, because the answer is in the inputs. Then it fails in production, meaning real use, where that information doesn't exist yet. When a model seems too good to be true, look for leakage first.

For marketers: the same trap shows up in attribution reports. A feature like "visited the order confirmation page" predicts purchase perfectly, and tells you nothing you can act on.

# Four past customers. Each dictionary is one example (one row).
customers = [
    {"orders": 8, "days_since": 20, "tickets": 0, "winback_email": 0, "churned": 0},
    {"orders": 2, "days_since": 150, "tickets": 3, "winback_email": 1, "churned": 1},
    {"orders": 5, "days_since": 45, "tickets": 1, "winback_email": 0, "churned": 0},
    {"orders": 1, "days_since": 200, "tickets": 2, "winback_email": 1, "churned": 1},
]

# winback_email is sent after a customer leaves, so it is left out
features = ["orders", "days_since", "tickets"]
X = [[c[f] for f in features] for c in customers]
y = [c["churned"] for c in customers]

print(len(X), "examples,", len(features), "features")
print(X[1], "->", y[1])
print(sorted(set(y)))