Command-line programs

Subcommands

Tools like git have subcommands: git add, git commit, each with its own arguments. argparse builds them with add_subparsers. A neat trick connects each subcommand to the function that handles it: set_defaults(func=...).

import argparse

def cmd_add(args):
    print(f"added note: {args.text!r} (priority {args.priority})")
    return 0

def cmd_list(args):
    print("listing notes", "including done ones" if args.all else "")
    return 0

def build_parser():
    parser = argparse.ArgumentParser(prog="notes")
    sub = parser.add_subparsers(dest="command", required=True)
    add = sub.add_parser("add", help="add a note")
    add.add_argument("text")
    add.add_argument("-p", "--priority", type=int, choices=[1, 2, 3], default=2)
    add.set_defaults(func=cmd_add)
    show = sub.add_parser("list", help="list notes")
    show.add_argument("--all", action="store_true")
    show.set_defaults(func=cmd_list)
    return parser

def main(argv):
    args = build_parser().parse_args(argv)
    return args.func(args)

main(["add", "Check stove fuel", "-p", "1"])
main(["list", "--all"])

required=True makes running the tool with no subcommand an error with a helpful message. After parsing, args.func is whichever handler belongs to the subcommand used, so main stays two lines however many commands you add.

Behaving like a good tool

Command-line tools are glued together by scripts and pipelines, which rely on a few conventions:

• Exit codes. 0 means success; anything else means failure. Scripts check it (if ./notes.py list; then ...) and CI fails a build on it. Return an int from main, and end the file with raise SystemExit(main(sys.argv[1:])) (or sys.exit(main(...))).
• stdout for results, stderr for messages. Print the actual output to stdout so it can be piped into another program or a file; print errors and progress to stderr (print(..., file=sys.stderr)) so they don't end up in the data.
• Read stdin when it's useful. Accepting input on stdin lets a tool sit in a pipeline: cat orders.csv | python totals.py. sys.stdin is a file-like object you can loop over line by line.
• Be quiet on success. Many good tools print nothing when everything worked, so they stay useful inside other scripts.

import io
import sys

def totals(lines, out, err):
    total, bad = 0.0, 0
    for n, line in enumerate(lines, start=1):
        try:
            total += float(line.split(",")[-1])
        except ValueError:
            bad += 1
            print(f"line {n}: not a number, skipped", file=err)
    print(f"{total:.2f}", file=out)
    return 1 if bad else 0

out, err = io.StringIO(), io.StringIO()
code = totals(["TE-101,200", "PA-108,55.5", "oops,abc"], out, err)
print("stdout:", out.getvalue().strip(), "| stderr:", err.getvalue().strip(), "| exit:", code)

Passing the streams in, as with argv, keeps the function testable: a real run passes sys.stdin, sys.stdout and sys.stderr.

import argparse


def main(argv):
    return 0


main(["10"])
main(["16.09", "--to", "mi"])

Settings, help and other libraries

Real tools take settings from more than one place. A common order of precedence, from strongest to weakest, is: a command-line option, then an environment variable, then a config file, then a built-in default. argparse handles the first and last directly; for the middle two, read the environment or the file yourself and pass the value as the option's default, for example default=os.environ.get("NOTES_FILE", "notes.json"). Secrets such as API keys belong in the environment, never in command-line options, because options show up in shell history and in process lists that other users can see.

Good help text is part of the interface. Give the parser a description, give every argument a short help string, and add an epilog with an example or two. Users run --help before anything else, and an example usually answers their question faster than a list of options.

argparse is in the standard library, so it's always available. Two popular third-party libraries build on the same ideas with less code: click, which uses decorators on functions, and typer, which reads the arguments from a function's type hints. Learn argparse first; the others will then feel familiar.

Finally, a tool should be easy to run. The packaging lesson shows how a [project.scripts] entry in pyproject.toml turns main into a real command, so people type notes add ... instead of python path/to/notes.py add ....

import argparse
import contextlib
import io

def build_parser():
    parser = argparse.ArgumentParser(prog="stock")
    sub = parser.add_subparsers(dest="command", required=True)
    check = sub.add_parser("check")
    check.add_argument("sku")
    return parser

STOCK = {"TE-101": 4, "PA-108": 0}

def main(argv):
    args = build_parser().parse_args(argv)
    if args.sku not in STOCK:
        print(f"unknown product {args.sku}", file=sys.stderr)
        return 1
    print(f"{args.sku}: {STOCK[args.sku]} in stock")
    return 0 if STOCK[args.sku] > 0 else 3

import sys

def run(argv):
    out, err = io.StringIO(), io.StringIO()
    with contextlib.redirect_stdout(out), contextlib.redirect_stderr(err):
        try:
            code = main(argv)
        except SystemExit as e:
            code = e.code
    return code, out.getvalue().strip(), err.getvalue().strip()

for argv in [["check", "TE-101"], ["check", "PA-108"], ["check", "XX-000"], []]:
    code, out, err = run(argv)
    print(argv, "->", code, "|", out or "-", "|", err.splitlines()[-1] if err else "-")