Running code safely

Running a process, properly

On a normal machine, an execution tool looks like this. (It can't run in your browser, which has no processes.)

import os, signal, subprocess

SAFE_ENV = {"PATH": "/usr/bin:/bin", "HOME": "/workspace", "LANG": "C.UTF-8"}

def run_command(cmd, cwd, timeout=60, max_output=20_000):
    proc = subprocess.Popen(cmd, shell=True, cwd=cwd, env=SAFE_ENV,
                            stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
                            start_new_session=True)   # its own process group
    try:
        out, _ = proc.communicate(timeout=timeout)
    except subprocess.TimeoutExpired:
        os.killpg(proc.pid, signal.SIGKILL)            # kill children too
        return f"Timed out after {timeout}s", True
    text = out.decode(errors="replace")
    if len(text) > max_output:
        text = text[:max_output] + f"\n[output truncated at {max_output} characters]"
    return f"exit code {proc.returncode}\n{text}", proc.returncode != 0

Notice: an explicit environment, a new process group, a hard kill, a cap, and the exit code in the result so the model knows whether it worked.

In-process limits are for mistakes, not attacks

A "please knock" sign stops a polite visitor. It won't stop a burglar. Inside one Python process you can count steps with sys.settrace and stop a runaway loop. That's handy for catching mistakes, and it's what the exercise does, because a browser has no processes to kill. But it's not a security wall. Code running in your process can reach your objects, your files and your secrets, and tricks like removing __builtins__ have well-known escapes. Untrusted code belongs in a separate process inside a sandbox.

In plain words: to contain code you don't trust, run it somewhere it can't reach what you care about.

allowed = {"__builtins__": {}}

try:
    eval("open('secrets.txt')", allowed)
except NameError as err:
    print("blocked:", err)

reached = eval("().__class__.__base__.__subclasses__()", allowed)
print("it can still reach", len(reached), "classes, starting from an empty tuple")

Persistent shells

Some harnesses keep one shell alive between calls, so cd and export carry over, like a person's terminal. That's convenient, but state builds up: a bad cd affects every later command, the way a messy desk carries into tomorrow. Others start fresh each time and tell the model the working folder. Either way, report the current folder in each result so the model doesn't get lost.

import sys

class StepLimit(Exception):
    pass

def limited(code, max_steps):
    count = 0
    def tracer(frame, event, arg):
        nonlocal count
        if event == "line":
            count += 1
            if count > max_steps:
                raise StepLimit()
        return tracer
    sys.settrace(tracer)
    try:
        exec(compile(code, "<agent>", "exec"), {})
        return f"finished in {count} steps"
    except StepLimit:
        return f"stopped after {max_steps} steps"
    finally:
        sys.settrace(None)

print(limited("total = sum(range(10))", 100))
print(limited("while True:\n    pass", 100))

env = {"PATH": "/bin", "HOME": "/w", "API_KEY": "sk-1"}
allow = ("PATH", "HOME", "LANG")
print(sorted(k for k in allow if k in env))

Why a blocklist can't work

Ban knives from a building and someone will still bring scissors. Blocking commands that contain rm sounds sensible until you count the ways around it: /bin/rm, python -c "import os; os.remove('x')", a script written to a file and run, or find . -delete. You can't list every way to do something harmful, because any general tool can do almost anything. Blocklists catch accidents. They don't stop anything determined, and a prompt injection can be very determined. Real protection comes from limits the process can't get around, set by the operating system.

Kill the whole family

When a command times out, killing only its parent often leaves its children running. Run npm test, and it may start a browser and a server, and those stay up, holding ports and using memory, long after your harness has moved on. Start each command in its own process group, and kill the group.

Think about what leaks

You wouldn't hand your wallet to everyone who runs an errand for you. A harness holds API keys. If it starts commands with its own environment, those keys pass to every command the model chooses, including a curl that a poisoned web page suggested. So start from an empty environment and add only what's needed. It's one of the cheapest protections you can have, and one of the most valuable.