LLMs in automations

Retry the right errors

Requests to any API sometimes fail for reasons that fix themselves:

• you hit a rate limit, a cap on how many requests you can send per minute (status 429),
• the service is briefly overloaded (529 on Anthropic's API, or 503),
• a server error (500, 502).

The standard response is to wait and retry, waiting longer each time: 1 second, then 2, then

1. That's called exponential backoff, because the wait doubles each time. Real code also adds

a little random extra wait, called jitter, so a thousand scripts don't all retry at the same instant. And if the API sends a retry-after header, wait that long.

Never retry a 400 (bad request), 401 or 403 (problems with your key) or 404 (not found). The request itself is wrong, so it will fail the same way every time. Retrying just wastes time and hides the bug.

import requests

for code in [429, 503, 400]:
    response = requests.get(f"https://api.gradient.test/v1/status/{code}")
    try:
        response.raise_for_status()
    except requests.exceptions.HTTPError as e:
        retry = e.response.status_code in {429, 500, 502, 503, 529}
        print(code, "retry" if retry else "give up and report it")

It prints 429 retry, 503 retry and 400 give up and report it.

Note: the official Anthropic SDKs already retry these errors, twice by default, with backoff. You write retries yourself when you call the HTTP API directly, as these lessons do, or to wrap a whole multi-step job.

waits = [2 ** n for n in range(4)]
print(sum(waits))

Batches that survive failure

Two more habits make a batch job safe to leave running:

• Isolate failures. Wrap each item in its own try. One broken lead should produce one error line in the report, not stop the other 999.
• Make reruns safe. Record the ids you've finished, and skip them next time. If the job crashes at item 600, the rerun starts at 601. It doesn't email those 600 customers again. A job that's safe to rerun like this is called idempotent.

done = {"rev_01", "rev_02"}
todo = ["rev_01", "rev_02", "rev_03", "rev_04"]
waits = [2 ** attempt for attempt in range(4)]
print(waits)
print([r for r in todo if r not in done])

Big jobs that can wait: the Message Batches API

Sometimes thousands of items can wait, such as tagging a year of reviews or rewriting 5,000 product descriptions. Send them as one batch instead of one request at a time. Anthropic's Message Batches API works through them in the background at half the normal price. Results arrive within 24 hours, and most batches finish much sooner:

batch = requests.post("https://api.anthropic.com/v1/messages/batches", headers=HEADERS, json={
    "requests": [
        {"custom_id": review["id"],
         "params": {"model": "claude-haiku-5-5", "max_tokens": 50,
                    "messages": [{"role": "user", "content": review["text"]}]}}
        for review in reviews
    ]
}).json()
# later: check the batch's status, then download results matched up by custom_id

Each request carries your own custom_id, so you can match each result back to its row. (The academy's proxy only passes on single messages, so this one is for your own projects.)

For marketers: a good first automation is small and boring. A daily script that tags new reviews and emails you the negative ones. Get retries, logging and safe reruns right on that, then scale up.

Don't rerun from the start

If a job crashes at item 800 of 1,000, rerunning everything repeats 800 side effects, such as emails sent twice. Save progress per item, and let one bad item fail on its own without stopping the rest.

Keep a record per item

Writing each item's outcome to a file as you go means you can see what failed, and rerun only those items once you've fixed the cause.