Running APIs in production

import hashlib, json

def respond(data, if_none_match=None):
    body = json.dumps(data)
    etag = hashlib.sha256(body.encode()).hexdigest()[:12]
    if if_none_match == etag:
        return 304, etag, ""
    return 200, etag, body

catalog = {"items": ["tent", "stove", "lantern"] * 100}

status, etag, body = respond(catalog)
print(status, "bytes sent:", len(body))

status, etag, body = respond(catalog, if_none_match=etag)
print(status, "bytes sent:", len(body))

catalog["items"].append("kettle")
status, etag, body = respond(catalog, if_none_match=etag)
print(status, "bytes sent:", len(body))

Request ids and structured logs

A user reports "checkout failed at 2:14". Your servers handled 40,000 requests that minute. Which log lines were theirs?

A parcel carries a tracking number so anyone can find it among millions. Give every request a request id for the same reason. Generate one at the edge, or accept one from the caller or load balancer. Then put it in every log line for that request, return it in an X-Request-Id response header, and include it in error bodies. Support can then ask for that id and find the exact trail in seconds. Anthropic's API does this: every response carries a request-id header.

Write logs as structured JSON, one object per line. It's the difference between a spreadsheet and a diary, because each fact sits in its own named column:

{"ts": "2026-10-30T14:14:03Z", "request_id": "9f2c1e", "method": "POST", "path": "/v1/checkout", "status": 502, "ms": 1840, "user": "u_42"}

Log tools can then filter, count and graph them: "all 5xx errors on /checkout in the last hour". Never log secrets, tokens, passwords or full card numbers, and keep personal data to what you need. Across many services, the W3C traceparent header carries the id from one service to the next, so a distributed trace shows the whole journey.

Health checks

Load balancers and orchestrators such as Kubernetes poll your servers, and they ask two different questions. "Is anyone home?" and "Is the shop open?" have different answers when the owner is inside, waiting for a delivery.

• Liveness (/healthz): is the process running at all? If not, restart it. Keep it trivial.
• Readiness (/readyz): can it serve traffic right now? Is the database reachable, the startup work done? If not, stop sending it requests until it recovers.

Mixing them up hurts. A liveness check that tests the database restarts every server when the database blips, which turns a small outage into a big one.

Scaling and shutting down

• Stateless servers keep nothing between requests. Any checkout lane can serve you, because your loyalty points live in a central system. In the same way, sessions, caches and rate-limit buckets live in a shared database or Redis. Then a load balancer can send any request to any copy, and you add copies to handle more traffic (horizontal scaling).
• Graceful shutdown: a restaurant at closing time stops seating people and lets the diners finish. On a deploy, stop accepting new connections, finish the requests already running, then exit. Otherwise every deploy cuts off some users mid-request.
• Limits: cap body sizes (answer 413 Content Too Large), set timeouts, and compress large responses (Content-Encoding: gzip).

Retiring old versions

A shop that's closing puts a sign in the window months ahead. When an old version must go, warn first. Two standard headers say so on every response. Deprecation says it's on the way out, and Sunset says when it stops working:

Deprecation: @1788134400
Sunset: Wed, 01 Jul 2026 00:00:00 GMT
Link: <https://docs.shop.test/migrate-v2>; rel="deprecation"

Announce it in the changelog and by email, watch which keys still call the old version, and contact them before the sunset date.

import json, time, uuid

LOG = []

def handler(request):
    return {"status": 200, "headers": {}, "body": {"ok": True}}

def traced(request):
    rid = request["headers"].get("x-request-id") or uuid.uuid4().hex[:12]
    start = time.perf_counter()
    response = handler(request)
    response["headers"]["X-Request-Id"] = rid
    LOG.append(json.dumps({"request_id": rid, "method": request["method"], "path": request["path"],
                           "status": response["status"], "ms": round((time.perf_counter() - start) * 1000, 2)}))
    return response

print(traced({"method": "GET", "path": "/v1/orders", "headers": {"x-request-id": "edge-7f3a"}})["headers"])
print(LOG[-1])

What CORS doesn't do

A guest list at the front door does nothing about the back door. Only browsers enforce CORS. A script can still call your API freely, so CORS is no substitute for authentication.