Working with AI coding tools

How AI-written code goes wrong

A confident tour guide can make up a street name and never blink. AI-written code fails in similar, predictable ways, so you can learn what to look for:

Invented packages deserve special mention. Attackers register package names that models tend to invent, so a careless install runs their code. It's like a fake shop opening under a name people often misremember. This is sometimes called slopsquatting.

def weakened(diff):
    removed = [l for l in diff.splitlines() if l.startswith("-") and "assert" in l]
    added = [l for l in diff.splitlines() if l.startswith("+") and "assert" in l]
    return removed, added

diff = """--- a/tests/test_cart.py
+++ b/tests/test_cart.py
-    assert total(cart) == 42.50
-    assert len(cart) == 3
+    assert total(cart) > 0"""

removed, added = weakened(diff)
print("assertions removed:", len(removed), " added:", len(added))
print("look closely" if len(removed) > len(added) else "fine")

Safety habits

• No secrets in prompts. Use placeholders; keep keys in the environment.
• Untrusted repositories can carry prompt injection in files and comments. Use plan mode or a sandbox, and approve commands.
• Keep your skills sharp. Pilots who fly on autopilot still train by hand. Use AI to explain and to check, but write enough code yourself that you can judge its output. If you can't explain a change, don't merge it.

In plain words: the tool writes it, but you're the one who answers for it.

def review_size(files, limit_lines=400, limit_files=10):
    lines = sum(added + removed for added, removed in files.values())
    if lines > limit_lines or len(files) > limit_files:
        return f"too big: {lines} lines in {len(files)} files. Split it."
    return f"ok: {lines} lines in {len(files)} files"

print(review_size({"cart.py": (12, 3), "tests/test_cart.py": (20, 0)}))
print(review_size({f"module_{n}.py": (40, 25) for n in range(30)}))

import ast, sys

def imports(source):
    names = set()
    for node in ast.walk(ast.parse(source)):
        if isinstance(node, ast.Import):
            names |= {a.name.split(".")[0] for a in node.names}
        elif isinstance(node, ast.ImportFrom) and node.level == 0:
            names.add(node.module.split(".")[0])
    return names

before = "import json\nimport requests\n"
after = "import json, hashlib\nimport requests\nfrom fastjsonx import loads\n"
new = imports(after) - imports(before)
print(sorted(new))
print(sorted(n for n in new if n not in sys.stdlib_module_names))

import sys

print("tomllib" in sys.stdlib_module_names, "numpy" in sys.stdlib_module_names)

Green tests aren't enough

An agent's code can pass the tests and still be wrong in ways the tests don't check. It may have changed behavior elsewhere, added a slow query, or quietly weakened the test itself. You're responsible for what you merge, whoever typed it. So read the diff as if a new teammate wrote it, and ask the same questions you'd ask them.

Small asks get better results

"Rebuild the checkout" invites a huge change that you can't review and the agent can't get right in one go. "Add a test for empty carts, then make it pass" is something you can check in two minutes. Large work is fine, as long as it's broken into steps that each leave the code working. You stay in control, and a mistake only costs one small step.

Check every new package

Models sometimes suggest packages that don't exist, because the name sounds right. Attackers have started registering those names with malware inside, and this has a name: slopsquatting. Before you install anything new, check that it's real, that it's maintained, and that it's the one you meant.

You still have to understand it

If you can't explain what a change does, you can't review it, debug it or tell when it's wrong. The more code you generate, the more this skill matters. A fair test is to ask yourself whether you could fix this at 3 a.m. if it broke.