Index Ai Academy
Project tutorial · 4 steps

Ship a small machine learning library in Python

The core of a typed, tested training package: a config that rejects unknown settings, mini-batches as a generator, a seeded training loop and an evaluation function.

Free to read, with no account. The code comes from the lesson Capstone: ship a small model library, and has been run.

The steps6

What you'll build

A model library is more than a training loop. It reads settings it can trust, splits data into batches without copying, gives the same answer for the same seed, and reports an error you can compare. This tutorial writes that core in plain Python. The lesson adds tests, a profile, a command line and a wheel around it.

What you'll learn

  • Plan a small library from a brief: a dataset iterator, a typed config, a train and an eval command
  • Load a YAML config into a typed object, and avoid the way YAML turns 3e-4 into text
  • Test numeric code with a seed, and profile a training loop before and after one fix
  • Ship it: a command line with exit codes, a wheel built from its parts, a README and a portfolio brief

What you need

Python functions, dataclasses and generators, and what a mean squared error is. It needs pip install pyyaml on your own machine, and nothing else. In the lesson it runs in your browser.

Step 1: A config that rejects what it does not know

A dataclass holds the settings with their defaults. load_config reads YAML, refuses any key it does not know (a typo should fail loudly, not be ignored), and converts each value to its type, because YAML reads 3e-4 as text.

import random
from dataclasses import dataclass

import yaml


@dataclass
class Config:
    lr: float = 0.1
    epochs: int = 100
    batch: int = 4
    seed: int = 0


def load_config(text):
    raw = yaml.safe_load(text) or {}
    types = {"lr": float, "epochs": int, "batch": int, "seed": int}
    for key in raw:
        if key not in types:
            raise ValueError(f"unknown setting: {key}")
    return Config(**{key: types[key](value) for key, value in raw.items()})

Step 2: Mini-batches as a generator

batches yields one slice at a time and never builds the whole list of batches, so a data set too big for memory works the same way.

def batches(xs, ys, size):
    for start in range(0, len(xs), size):
        yield xs[start:start + size], ys[start:start + size]

Rather write it yourself?

In the lesson you write this from a starter, in your browser with nothing to install, and checks tell you what's wrong as you go. A free account opens it and all 575 lessons.

Step 3: A seeded training loop

The shuffle comes from random.Random(seed), so the same config gives the same weights. Each batch computes the errors once, then moves the weight and the bias downhill along the mean squared error.

def train(cfg, xs, ys):
    rng = random.Random(cfg.seed)
    w = b = 0.0
    order = list(range(len(xs)))
    for _ in range(cfg.epochs):
        rng.shuffle(order)
        sx, sy = [xs[i] for i in order], [ys[i] for i in order]
        for bx, by in batches(sx, sy, cfg.batch):
            errs = [w * x + b - y for x, y in zip(bx, by)]
            w -= cfg.lr * 2 * sum(e * x for e, x in zip(errs, bx)) / len(bx)
            b -= cfg.lr * 2 * sum(errs) / len(bx)
    return w, b

Step 4: Score it, and read a config

evaluate is the mean squared error on the rows you pass it. The last line reads a one-line config: the other settings keep their defaults.

def evaluate(w, b, xs, ys):
    return sum((w * x + b - y) ** 2 for x, y in zip(xs, ys)) / len(xs)


xs = [x / 10 for x in range(10)]
ys = [2 * x + 1 for x in xs]
print(load_config("lr: 3e-4"))
What it prints
Config(lr=0.0003, epochs=100, batch=4, seed=0)

In the lesson the same functions live in a src package with tests, a profile of the training step, a train and an eval command, logging, and a wheel you can install.

Where to take it next

  • Write a test that loads "lr: oops" and expects a clean error.
  • Train twice with the same seed and assert the two weights are equal.
  • Add shuffle: false as a setting and a test that proves it keeps the order.

Mistakes to watch for

  • Writing the learning rate as 3e-4 in YAML and trusting it to be a number.
  • Building a list of all the batches instead of yielding them.
  • Printing a failure message and still exiting 0.
  • Reporting a speedup without checking that the result is unchanged.
  • Leaving the settings in the code so every experiment is an edit.
The complete program
import random
from dataclasses import dataclass

import yaml


@dataclass
class Config:
    lr: float = 0.1
    epochs: int = 100
    batch: int = 4
    seed: int = 0


def load_config(text):
    raw = yaml.safe_load(text) or {}
    types = {"lr": float, "epochs": int, "batch": int, "seed": int}
    for key in raw:
        if key not in types:
            raise ValueError(f"unknown setting: {key}")
    return Config(**{key: types[key](value) for key, value in raw.items()})


def batches(xs, ys, size):
    for start in range(0, len(xs), size):
        yield xs[start:start + size], ys[start:start + size]


def train(cfg, xs, ys):
    rng = random.Random(cfg.seed)
    w = b = 0.0
    order = list(range(len(xs)))
    for _ in range(cfg.epochs):
        rng.shuffle(order)
        sx, sy = [xs[i] for i in order], [ys[i] for i in order]
        for bx, by in batches(sx, sy, cfg.batch):
            errs = [w * x + b - y for x, y in zip(bx, by)]
            w -= cfg.lr * 2 * sum(e * x for e, x in zip(errs, bx)) / len(bx)
            b -= cfg.lr * 2 * sum(errs) / len(bx)
    return w, b


def evaluate(w, b, xs, ys):
    return sum((w * x + b - y) ** 2 for x, y in zip(xs, ys)) / len(xs)


xs = [x / 10 for x in range(10)]
ys = [2 * x + 1 for x in xs]
print(load_config("lr: 3e-4"))

Write it yourself

In the lesson you write this from a starter, in your browser with nothing to install, and checks tell you what's wrong as you go. A free account opens it and all 575 lessons.

Keep building

More tutorials.

This is one stage of the AI engineer roadmap: the bigger picture, in order.

5 steps

Build an AI ad-copy reviewer in Python

A small Python tool that checks ad headlines against your rules, saves a JSON report, and asks a language model to rewrite the worst one.

3 steps

Train a neural network classifier in PyTorch

Standardize from the training rows only, train a small network with a validation score after every epoch, and compare it fairly with logistic regression.

6 steps

Build a GPT from scratch in Python

Write a tiny GPT in PyTorch, train it on a product catalog, and watch it write new descriptions. The whole program is about 70 lines.

9 steps

Build a REST API in Python from scratch

Authentication, validation, pagination, idempotency and clean errors, in a working API built with the standard library so you can see how each one works.

6 steps

Build an MCP server in Python from scratch

A working Model Context Protocol server with tools, resources and prompts, written without an SDK so you can see exactly what goes over the wire.

7 steps

Build a coding agent in Python

An agent that reads and edits a project, runs its tests and stops when they pass, with the safety checks that keep it inside the project.

8 steps

Ship a production AI service in Python

API keys, rate limits, validation, caching, model fallbacks, structured logs and an eval, around a service that classifies support tickets.

7 steps

Build a neural network from scratch with NumPy

A two-layer classifier for trail signs written straight from the formulas: forward pass, cross-entropy, a backward pass, training, an honest test score with its error bar, and next steps in PyTorch.

Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write