Clustering

Scale the features first

Distances drive every step of k-means. So the units of each feature matter a lot.

Say you cluster customers on two features. Average order value is in dollars, from tens to hundreds. Discount share is the fraction of orders bought on discount, from 0 to 1. Take two customers whose order values differ by $50 and whose discount shares differ by 0.5.

• Order value adds 50 × 50 = 2,500 to the squared distance.
• Discount share adds 0.5 × 0.5 = 0.25.

Order value swamps discount share, and you'd get clusters by order value alone. The fix, from the Preparing data lesson, is to standardize each feature first. That rescales a column so its average is 0 and its typical spread (its standard deviation) is 1. Then every feature gets a fair say. In scikit-learn, StandardScaler does this.

How many clusters?

k-means never picks k for you. So you run it for several values of k and compare two numbers.

Inertia always falls as k grows. More centers means every point has a center closer by. In the extreme, give every customer their own center and the inertia is zero. So the lowest inertia can't be the goal. Instead, look for the elbow: the point where adding another cluster stops helping much. Picture a bent arm. The curve drops steeply, then bends and flattens.

The silhouette score asks a different question. For each point, is it much closer to its own cluster than to the next nearest cluster? The score runs from −1 to 1:

• near 1: the point sits snugly inside its own cluster;
• near 0: it's on the border between two clusters;
• below 0: it's probably in the wrong cluster.

The silhouette score for a whole clustering is the average over all points. Higher is better.

x = 3                  # the customer we're scoring
own = [1, 2]           # the other customers in its cluster
other = [10, 11, 12]   # the customers in the nearest other cluster

a = sum(abs(x - p) for p in own) / len(own)
b = sum(abs(x - p) for p in other) / len(other)
print("a =", a, " b =", b, " silhouette =", (b - a) / max(a, b))

The code below loads the shop's customers and uses three features: orders last year, average order value and discount share. It scales them, then runs k-means for k from 1 to 7. For each k, it prints the inertia and the silhouette score, and plots the inertia. The silhouette needs at least two clusters, so k = 1 shows a dash.

import pandas as pd
import requests
import matplotlib.pyplot as plt
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score
from sklearn.preprocessing import StandardScaler

df = pd.DataFrame(requests.get("https://api.gradient.test/v1/customers").json()["data"])
features = ["orders_last_year", "avg_order_value", "discount_share"]
X = StandardScaler().fit_transform(df[features])

ks, inertia = list(range(1, 8)), []
for k in ks:
    km = KMeans(n_clusters=k, n_init=10, random_state=0).fit(X)
    inertia.append(km.inertia_)
    sil = f"{silhouette_score(X, km.labels_):.3f}" if k > 1 else "  -  "
    print(f"k={k}   inertia {km.inertia_:6.0f}   silhouette {sil}")

plt.plot(ks, inertia, marker="o")
plt.xlabel("number of clusters k")
plt.ylabel("inertia")

The biggest drops in inertia come at k = 2 and k = 3. After that, the curve flattens. The silhouette score is nearly tied between 2 and 3. So the numbers narrow it down to two choices, and usefulness decides between them. The next step is to look at what three clusters mean.

From clusters to segments

k-means gives each customer a cluster number: 0, 1 or 2. On its own, that number means nothing. To make it useful, you profile each cluster. Take the average of every feature per cluster. Include features the clustering didn't use, like churn and lifetime value (the total a customer spends with you over time).

The code below runs k-means with k = 3 and adds the cluster number as a new column. Then groupby averages the columns per cluster. The churn column is the share of customers in the cluster who left.

import pandas as pd
import requests
from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler

df = pd.DataFrame(requests.get("https://api.gradient.test/v1/customers").json()["data"])
features = ["orders_last_year", "avg_order_value", "discount_share"]
X = StandardScaler().fit_transform(df[features])
df["cluster"] = KMeans(n_clusters=3, n_init=10, random_state=0).fit_predict(X)

profile = df.groupby("cluster").agg(
    customers=("id", "count"),
    orders=("orders_last_year", "mean"),
    order_value=("avg_order_value", "mean"),
    discount=("discount_share", "mean"),
    churn=("churned", "mean"),
    ltv=("lifetime_value", "mean"),
).round(2)
pd.set_option("display.width", 120)
print(profile)

Three recognizable groups, or segments, fall out:

• Bargain hunters: the most orders, the smallest baskets, and over half their orders discounted.
• Loyal regulars: fewer orders but the biggest baskets. They rarely use discounts and have by far the highest lifetime value.
• Occasional shoppers: about two orders a year and mid-sized baskets. Around 40% of them churn.

Now compare k = 2. The loyal regulars and the occasional shoppers land in one cluster with a 25% churn rate. That number describes neither group, and the at-risk segment disappears. That's why k = 3 is the better choice here, even though the scores barely separated the options.

For marketers: this is where clustering earns its keep. Send a discount-led message to the first group and a premium or loyalty message to the second. Give the third a re-engagement program. Each segment is described in terms a campaign can use.

Clustering text with embeddings

k-means works on any list of numbers, so it can group text too. First you turn each text into an embedding, which you met in Phase 1. It's a list of numbers that captures a text's meaning. Texts that mean similar things get nearby lists. Then k-means groups the texts by meaning.

Here are a dozen support messages with no labels. The code asks the embeddings service for one vector per message. Then it runs k-means with k = 3 and prints the messages in each cluster.

import numpy as np
import requests
from sklearn.cluster import KMeans

messages = [
    "Where is my parcel? Tracking hasn't changed in a week.",
    "How do I return an unworn sweater?",
    "The jacket runs small. Should I size up?",
    "Delivery was promised Friday and nothing has arrived.",
    "Still waiting on my refund for the boots.",
    "What size fits a 34 inch waist?",
    "Can I get express shipping to Denver?",
    "Can I exchange this for a different color?",
    "Sleeves are too tight and the length is short.",
    "My package says delivered but it isn't here.",
    "I returned it two weeks ago. Where's my refund?",
    "Is the medium a loose fit?",
]
data = requests.post("https://api.gradient.test/v1/embeddings", json={"input": messages}).json()["data"]
E = np.array([d["embedding"] for d in data])
labels = KMeans(n_clusters=3, n_init=10, random_state=0).fit_predict(E)
for c in range(3):
    print(f"cluster {c}:")
    for m, l in zip(messages, labels):
        if l == c:
            print("   ", m)

Shipping, returns and sizing questions separate cleanly. Real teams have thousands of tickets, not twelve. The usual next step is to send a few messages from each cluster to an LLM and ask it to name the theme. That turns a pile of text into a labeled map of what customers are asking about.

Limits worth knowing

• You choose k, and k-means always delivers. Ask for 4 clusters and you get 4, even from data with no real groups at all.
• It expects round blobs of similar size. Other methods handle other shapes. DBSCAN grows clusters from crowded areas, so it finds odd shapes and flags lonely points as outliers. Hierarchical clustering builds a family tree of groups that you can cut at any level. Gaussian mixtures give each point a probability of belonging to each cluster, instead of one firm answer.
• Clusters describe the data you have. Before you build a strategy on them, re-run on newer data and check that the same segments come back.