Phase 3 · Machine learning · Data analysis

Time series with pandas

Resample, smooth, compare across years, and test whether one series leads another.

About 45 minutes · A free account opens the whole lesson

Create a free account Try part of it here

Already have an account? Sign in to open this lesson

Why it matters

Sales, sign-ups, server load, model accuracy over time: much of the data a business watches is a time series, a measurement taken again and again. Time adds its own traps: unfinished periods, seasons that repeat every year, one-off shocks. It also adds its own power: a measure that moves first can warn you about one that moves later. The forecasting models you'll meet later are built on exactly these tools.

What you'll be able to do

Index a table by real dates, select by date, and resample days into weeks or months
Smooth noisy daily data with rolling windows, and compare like with like across years
Shift a series to test whether one measure leads another
Chart two measures with different units on twin axes

The brief

The shop orders trail-running shoes from its supplier two months ahead. Order too few and the shelves are empty in May; order too many and boxes sit in the warehouse all winter. The buyer has daily online shoe orders since 2019 and a monthly search-interest index, which scores how often people search for "trail shoes", from 0 to 100. Can the past tell you anything about next month?

import pandas as pd

daily = pd.read_csv("https://api.gradient.test/v1/datasets/shoe-orders.csv")
daily["date"] = pd.to_datetime(daily["date"])
daily = daily.set_index("date")
print(daily.index.min(), "to", daily.index.max(), len(daily), "days")

print(daily.loc["2020-04-01":"2020-04-07"])
print(daily.loc["2024-12"].sum())
print(daily.index[:3].day_name().tolist())

It runs right here in your browser, with nothing to install.

With real dates as the index, you can select by date. .loc["2020-04-01":"2020-04-07"] is a date range, both ends included. A partial date like "2024-12" means "all of December 2024". The index also knows calendar facts, such as day_name(), month, year and days_in_month.

Resampling: days into months

resample groups rows into regular time buckets, a groupby for time. The rule names the bucket: "W" for weeks, "MS" for months (labeled by their first day), "QS" for quarters, "YS" for years. Then you say how to combine the rows in each bucket:

import pandas as pd

daily = pd.read_csv("https://api.gradient.test/v1/datasets/shoe-orders.csv")
daily["date"] = pd.to_datetime(daily["date"])
daily = daily.set_index("date")

monthly = daily["orders"].resample("MS").sum()
days = daily["orders"].resample("MS").size()
print(pd.DataFrame({"orders": monthly, "days": days, "days_in_month": monthly.index.days_in_month}).tail(3))

complete = monthly[days == monthly.index.days_in_month]
print(len(monthly), "months,", len(complete), "complete")
print(daily["orders"].resample("W").sum().tail(3))

This is the unfinished-period check from the charts lesson, done properly. Instead of "at least 28 days", compare each month's day count with days_in_month, so a February with all 28 days counts as complete and a July with 30 doesn't. The data stops on June 15, 2025, so June is dropped. A weekly total ends on Sundays: "W" is short for "W-SUN".

Watch for missing days, too. If the export skipped a day, there's simply no row for it, and a monthly sum quietly comes out low. daily.asfreq("D") puts in a row for every calendar day, with NaN where data is missing, so daily.asfreq("D")["orders"].isna().sum() counts the gaps. Here it's 0. On a day with no sales, the data should say 0, which is different from saying nothing at all.

Quick check

What does this print?

import pandas as pd
s = pd.Series([1, 2, 3, 4], index=pd.to_datetime(["2025-01-30", "2025-01-31", "2025-02-01", "2025-02-02"]))
print(s.resample("MS").sum().tolist())

[10]
[4, 6]
[3, 7]
[1, 2, 3, 4]

Show the answer

[3, 7]

Two January days (1 + 2) and two February days (3 + 4) make two monthly buckets: 3 and 7.

Rolling windows

Daily orders bounce around: 31 one day, 18 the next. A rolling mean replaces each day with the average of the last few days, smoothing the noise so the trend shows:

import matplotlib.pyplot as plt
import pandas as pd

daily = pd.read_csv("https://api.gradient.test/v1/datasets/shoe-orders.csv")
daily["date"] = pd.to_datetime(daily["date"])
daily = daily.set_index("date")

recent = daily.loc["2024-01":"2024-12", "orders"]
fig, ax = plt.subplots(figsize=(9, 3.2))
ax.plot(recent.index, recent, alpha=0.35, label="daily")
ax.plot(recent.index, recent.rolling(7).mean(), label="7-day average")
ax.plot(recent.index, recent.rolling(28).mean(), label="28-day average")
ax.set_title("Trail-shoe orders, 2024")
ax.set_ylabel("orders per day")
ax.legend()
print(recent.rolling(7).mean().head(8).round(1).tolist())

rolling(7) looks at the current day and the six before it. The first six values are NaN, because there aren't seven days yet. A 7-day window is a natural choice for daily data, because it always holds each day of the week exactly once, so a busy-weekend pattern can't make the line wobble. A longer window is smoother, but it reacts later to real changes. The 28-day line turns down weeks after the 7-day one.

Seasons, and comparing like with like

Shoe orders rise every spring and fall every autumn. That seasonality makes month-to-month changes misleading: orders in May always beat April, so "up 25% on last month" says little. Compare each month with the same month a year earlier instead:

import pandas as pd

daily = pd.read_csv("https://api.gradient.test/v1/datasets/shoe-orders.csv")
daily["date"] = pd.to_datetime(daily["date"])
monthly = daily.set_index("date")["orders"].resample("MS").sum().iloc[:-1]

season = monthly.groupby(monthly.index.month).mean().round(0)
print(season.to_dict())

growth = (monthly.pct_change(12) * 100).round(1)
print(growth.loc["2025"].tolist())
print(growth.loc["2020-03":"2020-06"].tolist())

Grouping by index.month averages every January together, every February together, and so on. That gives the seasonal shape: May to July are roughly twice January. pct_change(12) compares each month with the one 12 rows earlier, the same month last year. It's called year-over-year growth.

The 2020 rows show a shock: April 2020 fell 56% below April 2019, then May and June were back to normal growth straight away. A shock is a one-off event, not part of the seasonal pattern. Keep it in mind before you average it into "a typical April".

Try it now

resample changes the time step, rolling smooths without changing it, and shift moves a series in time so you can compare it with another.

Drill

The busiest week

Using daily (indexed by date), set weekly to the weekly totals of orders, busiest to the date that labels the busiest week (a Timestamp, from idxmax), and top_2023 to the busiest week's total within 2023 only.

import pandas as pd

daily = pd.read_csv("https://api.gradient.test/v1/datasets/shoe-orders.csv")
daily["date"] = pd.to_datetime(daily["date"])
daily = daily.set_index("date")

weekly = None
busiest = None
top_2023 = None

In the lesson, your code is checked as you write it.

Does one series lead another?

If search interest climbs a month before orders do, the buyer gets a month's warning. To test it, shift the search series forward so each month's orders sit next to the previous month's interest, then measure how closely they move together with a correlation:

import pandas as pd

def dataset(name):
    return pd.read_csv(f"https://api.gradient.test/v1/datasets/{name}.csv")

daily = dataset("shoe-orders")
daily["date"] = pd.to_datetime(daily["date"])
orders = daily.set_index("date")["orders"].resample("MS").sum().iloc[:-1]

search = dataset("search-interest")
interest = pd.Series(search["interest"].values, index=pd.to_datetime(search["month"]))

for lag in range(4):
    print(lag, "months:", round(orders.corr(interest.shift(lag)), 3))

Correlation runs from -1 to 1, where 1 means the two rise and fall perfectly together. shift(1) moves every value one row later: February's row now holds January's interest. pandas lines the two series up by date before correlating, so mismatched months never get compared by accident. The strongest link is at one month: interest leads orders by about a month.

Two cautions. A strong correlation over six years doesn't make every bump meaningful: in March 2022, interest jumped to 100, a one-off viral moment, and orders barely noticed. And a lead is only useful if it holds up on months the analysis hasn't seen. Check it on the latest year before you rely on it.

Worked example

Worked example: two units on one chart

Show orders and search interest together so the lead is visible. They're measured in different units, so each gets its own y-axis.

import matplotlib.pyplot as plt
import pandas as pd

def dataset(name):
    return pd.read_csv(f"https://api.gradient.test/v1/datasets/{name}.csv")

daily = dataset("shoe-orders")
daily["date"] = pd.to_datetime(daily["date"])
orders = daily.set_index("date")["orders"].resample("MS").sum().iloc[:-1]
search = dataset("search-interest")
interest = pd.Series(search["interest"].values, index=pd.to_datetime(search["month"]))

window = slice("2023-01", "2025-05")
fig, ax = plt.subplots(figsize=(9, 3.5))
ax.plot(orders.loc[window].index, orders.loc[window], color="C0", label="orders")
ax.set_ylabel("orders per month", color="C0")
ax2 = ax.twinx()
ax2.plot(interest.loc[window].index, interest.loc[window], color="C1", linestyle="--", label="search interest")
ax2.set_ylabel("search interest (0-100)", color="C1")
ax.set_title("Search interest turns up about a month before orders")
print(orders.loc["2025-02":"2025-05"].tolist(), interest.loc["2025-01":"2025-04"].tolist())

In the lesson you can change this and run it.

ax.twinx() makes a second axes that shares the x-axis but has its own y-axis on the right. Coloring each axis label like its line tells the reader which scale belongs to which line. The title states the finding. Twin axes need care: by stretching either scale you can make two lines look as close or as far apart as you like. So use them to show timing, as here, not to claim the two are "the same size". The printout shows the lead in numbers: each month's orders follow the previous month's interest.

Practice what you just read

6 code examples you can edit and run
a practice exercise that checks your code
a 5-question quiz
5 review cards for spaced repetition

Create a free account

Try a quiz question

Orders in May were 25% higher than in April. Why is that weak evidence that the business is growing?

May is seasonally busier than April every year; compare with last May instead
Percentages can't be used for orders
April has fewer days than May
A 25% rise is too small to matter

Answer it at the end of the lesson to see if you're right, and why.