Concurrency and async

Race conditions

Two people share a tally on a whiteboard. Both read 10, both add one in their heads, and both write 11. Two people counted, and the board went up by one. When two threads share data, their steps can interleave in any order in the same way. Take count += 1. It looks like one step, but it's three: read count, add 1, write it back. If two threads both read before either writes, one update is lost. That's a race condition: the result depends on timing.

This simulation plays one unlucky interleaving step by step:

count = 0
steps = [
    ("A", "read"), ("B", "read"),      # both read 0 before either writes
    ("A", "write"), ("B", "write"),
]
seen = {}
for thread, action in steps:
    if action == "read":
        seen[thread] = count
        print(f"{thread} reads {count}")
    else:
        count = seen[thread] + 1
        print(f"{thread} writes {count}")
print("final count:", count, "(two increments, but one was lost)")

Race bugs are nasty because they're rare. The test passes a thousand times, then production loses an order. The fix is a lock, which works like the lock on a bathroom door: only the thread holding it may run the protected steps, so the read, add and write can't be split.

Locks bring their own risk. Picture two cars meeting on a narrow bridge, each waiting for the other to back up. A deadlock happens when thread A holds lock 1 and waits for lock 2, while B holds lock 2 and waits for lock 1. Both wait forever.

The safest plan is to share less: give each task its own data, and pass results back through a queue.

count = 0
read_a = count
read_b = count
count = read_a + 1
count = read_b + 1
print(count)

How async works: an event loop

A single waiter can serve many tables, because most of the time each table is waiting for the kitchen. Async code runs on one thread like that waiter, so there are no surprise interleavings. Tasks switch only where the code says await. The machine that does the switching is an event loop. Its core is small enough to build here, using generators.

A generator is a function with yield, and it keeps its place like a bookmark. Each yield pauses it and hands a value to whoever is running it. Calling next() resumes it from where it stopped. So a generator can say "I'm waiting; wake me later" with a yield, and a scheduler can run something else in the meantime:

def download(name, chunks):
    for i in range(chunks):
        print(f"{name}: chunk {i + 1} of {chunks}")
        yield                      # "I'm waiting for the network, run someone else"
    print(f"{name}: done")

ready = [download("report.pdf", 2), download("photo.jpg", 3)]
while ready:
    task = ready.pop(0)
    try:
        next(task)                 # run the task until its next yield
        ready.append(task)         # it paused: put it back in line
    except StopIteration:
        pass                       # it finished

The two downloads take turns. Python's asyncio is this idea, grown up: await is a pause point like yield, and the loop wakes each task when its network data actually arrives.

async and await in real Python

Here's the shape of real async code. It sends several model requests at once, with a semaphore capping how many are in flight, to stay under the API's rate limit. A semaphore works like the door staff at a busy club, letting in only so many at a time:

import asyncio
import httpx

LIMIT = asyncio.Semaphore(8)            # at most 8 requests in flight

async def ask(client, prompt):
    async with LIMIT:
        r = await client.post("https://api.anthropic.com/v1/messages", json={...}, timeout=60)
        return r.json()

async def main(prompts):
    async with httpx.AsyncClient(headers={...}) as client:
        return await asyncio.gather(*(ask(client, p) for p in prompts))

results = asyncio.run(main(prompts))

async def makes a coroutine, a function that can pause. await pauses it until the result is ready, letting the loop run others. asyncio.gather starts them all and waits for every result. The Anthropic Python SDK has an AsyncAnthropic client for exactly this.

Note: The academy's in-browser Python can't run asyncio.run or start threads, so that block is for reading. The exercise builds the same scheduling logic by hand.

Choosing the right tool

Threads won't speed up heavy calculation in standard Python. The global interpreter lock lets only one thread run Python code at a time. Threads help when your program is waiting on a network or a disk. For CPU-heavy work, use processes, or a library like NumPy that does its work outside that lock.

One slow call can freeze all your async code. If the only waiter stops to chat at one table, every other table waits. An event loop runs one task at a time, and it switches only at an await. Call something slow that doesn't await, such as time.sleep or requests.get, and every other task stops too. Use the async version, like await asyncio.sleep, or hand the slow call off to a thread.

Passing once doesn't prove a race is gone. Races depend on timing, so a test can pass a thousand times and fail on the next run. Don't rely on luck: protect shared data with a lock, or don't share it.

Limit how much you run at once. It's tempting to fire off 5,000 model calls together, but you'll hit rate limits, and most of them will fail. A semaphore caps how many run at the same time. Ten or twenty at once usually gets through a big job faster than 5,000 that fail and retry.