Index Ai Academy
Phase 11 · Professional practice · Shipping

Docker Compose

Several containers together: service names, volumes, secrets and start order.

About 40 minutes · A free account opens the whole lesson

Create a free account Try part of it here

Already have an account? Sign in to open this lesson

Why it matters

A real app is rarely one container. The shop's API needs a database, a cache and a worker that sends emails, and every developer on the team needs all four running the same way. Docker Compose describes the whole set in one file, starts it with one command, and is how most teams run their stack on a laptop, in CI, and often on a small server. Reading and fixing Compose files is an everyday job skill, and the mistakes it invites (localhost, lost data, passwords in git) are the same ones that cause outages at larger scale.

What you'll be able to do

  • Read a Compose file: services, images, ports, environment, volumes and dependencies
  • Explain how containers reach each other by service name, and why localhost fails
  • Keep data in volumes and secrets out of the file
  • Work out a safe start order, and spot the mistakes that break a stack

What it covers

  1. One file for the whole stack
  2. Containers find each other by name
  3. Two services, one port
  4. Data lives in volumes, secrets live elsewhere
  5. Start order, and waiting for "ready"
  6. Beyond the laptop
Quick check
Your app needs an API, a database, a cache and a worker. How does every developer get all four running the same way?
  1. Each developer installs whatever versions they like
  2. Run them all on one shared developer's laptop
  3. A long README listing the steps to start each one by hand
  4. One Compose file that describes all four, started with one command
Show the answer

One Compose file that describes all four, started with one command

A Compose file describes every service, how they connect and what they need, so docker compose up starts the same stack for everyone.

One file for the whole stack

A Compose file is written in YAML, a format for nested settings where indentation shows what belongs to what: key: value pairs, and lists with a - in front of each item. Under services, each entry is one container:

import yaml

compose = """
services:
  api:
    build: .
    ports:
      - "8000:8000"
    environment:
      DATABASE_URL: postgresql://shop:${DB_PASSWORD}@db:5432/shop
      CACHE_URL: redis://cache:6379
    depends_on:
      - db
      - cache
  db:
    image: postgres:16
    environment:
      POSTGRES_USER: shop
      POSTGRES_PASSWORD: ${DB_PASSWORD}
    volumes:
      - dbdata:/var/lib/postgresql/data
  cache:
    image: redis:7
volumes:
  dbdata:
"""

stack = yaml.safe_load(compose)
for name, service in stack["services"].items():
    print(name, "| image:", service.get("image", "built from " + str(service.get("build"))),
          "| ports:", service.get("ports", []), "| needs:", service.get("depends_on", []))
It runs right here in your browser, with nothing to install.

yaml.safe_load turns the text into ordinary dictionaries and lists (always safe_load: the full loader, yaml.load with yaml.Loader, can run code hidden in a file). Reading it as data is what tools that check Compose files do. The pieces:

  • image names a ready-made image, pinned to a version (postgres:16); build: . builds one from the Dockerfile in this folder.
  • ports: "8000:8000" publishes a container's port on your machine, host:container. Keep the quotes: some YAML readers treat an unquoted 22:22 as a base-60 number (1342), so your browser can reach the API at localhost:8000.
  • environment sets the variables the app reads its settings from.
  • depends_on says which services should start first.
  • volumes keeps data somewhere that outlives the container (more below).
Quick check
A service has ports: ["5433:5432"]. From your laptop, which address reaches the database inside it?
  1. Neither: those ports are for containers
  2. db:5433, using the service name
  3. localhost:5432, where Postgres listens
  4. localhost:5433: host port first
Show the answer

localhost:5433: host port first

The left number is on your machine, the right one inside the container. Teams pick a different host port, like 5433, when something on the laptop already uses 5432.

Containers find each other by name

Compose puts every service on a shared private network where each one is reachable by its service name. That's why the API's settings say db:5432 and cache:6379: from inside the api container, db is the database.

The classic mistake is localhost. Inside a container, localhost means that container itself, so an API told to find its database at localhost:5432 looks inside its own box, finds nothing, and fails with "connection refused". It worked on the developer's laptop because there the database really was local. A second point follows: the database needs no ports entry at all, since the API reaches it over the private network. Publishing it only exposes it to everyone who can reach your machine.

Try it now

Inside a container, localhost is the container itself; other services go by name.

Drill

Two services, one port

Two services can't publish the same port on your machine: the second fails to start with "port is already allocated". Write port_clashes(services): services is the parsed services dictionary. Return a sorted list of (host_port, [service names]) for every host port used by more than one service, with the names sorted. A port entry is a string like "8000:8000" (host first) or just "8000", which publishes the same number on both sides. Keep the host port as a string, as it appears in the file.

def port_clashes(services):
    return []


services = {
    "api": {"ports": ["8000:8000"]},
    "admin": {"ports": ["8000:3000"]},
    "db": {"ports": ["5432:5432"]},
    "cache": {},
}
print(port_clashes(services))
In the lesson, your code is checked as you write it.

Data lives in volumes, secrets live elsewhere

A container's own files vanish when it's removed, and Compose removes and recreates containers all the time. Anything that must survive, above all a database's files, goes in a volume: dbdata:/var/lib/postgresql/data mounts a named volume, managed by Docker, at the folder where Postgres keeps its data. A bind mount such as ./src:/app maps a folder on your machine instead, which is handy in development so code changes show up without rebuilding.

Passwords are the other thing that doesn't belong in the file, because Compose files go into git. Write POSTGRES_PASSWORD: ${DB_PASSWORD} and Compose fills it in from your environment or from a .env file next to the Compose file, which stays out of git (add it to .gitignore). For production, use the platform's secret store.

The same ${...} mechanism is how one Compose file serves several environments: image: shop-api:${API_VERSION} lets CI run the exact version it just built, and a developer's .env can point the app at a local mail catcher instead of a real mail service. Keep a committed .env.example that lists every variable with a harmless sample value, so a new teammate knows what to set.

Quick check
Where should the database password used in a Compose file live?
  1. In the service name, such as db-hunter2
  2. Written in the Compose file, which goes into git
  3. In a .env file kept out of git, read as ${DB_PASSWORD}
  4. In the Dockerfile, so it's baked into the image
Show the answer

In a .env file kept out of git, read as ${DB_PASSWORD}

Compose files and Dockerfiles are shared and committed, so anything written in them is public to everyone with the repository. ${DB_PASSWORD} pulls the value from your environment or an uncommitted .env file at start time.

Start order, and waiting for "ready"

depends_on makes Compose start the database before the API. But "started" isn't "ready": Postgres takes a few seconds to accept connections, and an API that connects immediately crashes. The fix is a health check on the database and condition: service_healthy on the dependency, or an app that retries its first connection, which is good practice anyway.

Working out a start order from depends_on is the topological sort problem: everything must come after the things it needs, and a loop (A needs B, B needs A) has no valid order at all. Here's a short version that starts whatever has no unstarted needs, over and over:

def start_order(needs):
    order, started = [], set()
    while len(order) < len(needs):
        ready = sorted(s for s in needs if s not in started and all(d in started for d in needs[s]))
        if not ready:
            raise ValueError("these services depend on each other in a loop: " + ", ".join(sorted(set(needs) - started)))
        for s in ready:
            order.append(s)
            started.add(s)
    return order

print(start_order({"api": ["db", "cache"], "worker": ["db", "cache"], "db": [], "cache": []}))
try:
    start_order({"api": ["worker"], "worker": ["api"]})
except ValueError as e:
    print(e)

Here's the health check that makes "ready" mean ready, in the Compose file itself:

  db:
    image: postgres:16
    healthcheck:
      test: ["CMD", "pg_isready", "-U", "shop"]
      interval: 5s
      retries: 10
  api:
    depends_on:
      db:
        condition: service_healthy

pg_isready is a small tool that ships with Postgres and answers "accepting connections" or not. Docker runs it every 5 seconds and marks the database healthy once it passes; only then does it start the API. Notice that depends_on changed shape: from a list of names to a dictionary of names with a condition each. Both forms are valid, which is why a checker has to handle either.

Quick check
What does this print?
def start_order(needs):
    order, started = [], set()
    while len(order) < len(needs):
        ready = sorted(s for s in needs if s not in started and all(d in started for d in needs[s]))
        if not ready:
            raise ValueError("loop")
        for s in ready:
            order.append(s)
            started.add(s)
    return order

print(start_order({"web": ["api"], "api": ["db"], "db": [], "worker": ["db"]}))
  1. ['db', 'api', 'web', 'worker']
  2. ['db', 'api', 'worker', 'web']
  3. ['web', 'api', 'db', 'worker']
  4. ['api', 'db', 'web', 'worker']
Show the answer

['db', 'api', 'worker', 'web']

db needs nothing, so it starts first. Then api and worker both have their needs met, and start in alphabetical order. web needs api, so it comes last.

Beyond the laptop

The same file earns its keep in two more places. In CI, a test job runs docker compose up -d db cache, waits for the health checks, runs the test suite against real services instead of fakes, and tears everything down with docker compose down -v (the -v removes the volumes too, so every run starts clean). On a small server, Compose can run a whole product: restart: unless-stopped brings containers back after a crash or a reboot, and docker compose pull && docker compose up -d updates to newly published images. What it doesn't do is spread containers over several machines or replace a failed machine; when one server is no longer enough, that's the job Kubernetes takes on.

Practice what you just read

  • 2 code examples you can edit and run
  • a practice exercise that checks your code
  • a 5-question quiz
  • 5 review cards for spaced repetition

Create a free account

Try a quiz question

The API works when run on a laptop but fails in Compose with "connection refused" to its database. The setting is DATABASE_URL=postgresql://shop@localhost:5432/shop. What's wrong?

  1. In a container, localhost is itself; use db
  2. Postgres can't run inside a container
  3. The database needs a published port first
  4. Compose ignores environment variables

Answer it at the end of the lesson to see if you're right, and why.

Before this lesson