Config files: YAML and TOML

The traps

YAML guesses types from how a value looks, and its guesses can surprise you:

• yes, no, on and off become booleans in PyYAML. GitHub Actions files start with on:, which PyYAML reads as the key True. (GitHub's own parser knows better; your Python script may not.)
• python: 3.10 is the number 3.1, because a number drops its trailing zero. Versions should be quoted strings.
• port: 08080 and other values with leading zeros may not be the numbers you expect.
• A colon followed by a space inside text, as in title: Note: read me, breaks parsing unless the text is quoted.
• A tab used for indentation is an error.

import yaml
print(yaml.safe_load("python: 3.10\nversion: '3.10'\non: push\ncountry: NO"))

The fix is the same each time: quote the value, then load the file and look.

These traps come from YAML 1.1, the version PyYAML implements. The newer YAML 1.2 dropped the yes/no booleans, but plenty of tools still read files the old way, so write as if they all do. When a value is meant to be text, quoting it costs nothing and removes the guesswork for every reader, human or program.

import yaml
print(yaml.safe_load("python: 3.10"))

import yaml

WORKFLOW = '''
name: CI: tests
on: push
jobs:
  test:
    python: 3.10
'''

try:
    print(yaml.safe_load(WORKFLOW))
except yaml.YAMLError as e:
    print("Doesn't parse:", type(e).__name__)

safe_load, never load

PyYAML can also build arbitrary Python objects from special tags, if you use the full loader. Load an untrusted file that way and it can run code on your machine. yaml.safe_load only ever builds plain dictionaries, lists, strings, numbers, booleans and None, which is all a configuration needs. Use it every time, including for files you wrote yourself, so the habit never fails you on one you didn't. Security reviewers search code for yaml.load( without a safe loader for exactly this reason.

TOML: types you can see

TOML spells its types out. Strings are always quoted, so a version can't turn into a number. Square brackets start a table, which becomes a dictionary, and key = value lines fill it. Lists use square brackets with commas.

import tomllib

text = '''
[project]
name = "trail-notes"
version = "0.1.0"
requires-python = ">=3.11"
dependencies = ["fastapi>=0.110", "httpx"]

[tool.pytest.ini_options]
addopts = "-q"
'''
data = tomllib.loads(text)
print(data["project"]["dependencies"], data["tool"]["pytest"]["ini_options"])

The dotted header [tool.pytest.ini_options] builds nested dictionaries. tomllib only reads TOML; writing it needs a separate package, which is why most people edit pyproject.toml by hand.

Two details trip people up. A key belongs to the most recent table header above it, so the order of sections matters. And a table can only be defined once: writing [project] twice is an error rather than a merge.

import tomllib
print(tomllib.loads('[tool.black]\nline-length = 88'))

import tomllib

PYPROJECT = '''
[project]
name = trip-planner
version = 1.2.0
'''

try:
    print(tomllib.loads(PYPROJECT))
except tomllib.TOMLDecodeError as e:
    print("Doesn't parse:", e)

Validate what you load

Parsing proves the file is well formed, not that it says what your program needs. Check the result straight after loading: the keys that must be there, their types, and their ranges. Collect every problem and report them together, with the key's name, so whoever edits the file can fix it in one go instead of finding one mistake per deployment. A program that fails at start-up with "max_tokens must be a whole number from 1 to 8192" saves someone an hour of guessing.

Defaults belong here too. If a setting is optional, fill in its default in one place, the loader, so the rest of the program can rely on every key being present. And keep secrets out of these files entirely: configuration is committed and reviewed, while keys belong in environment variables or a secret manager.

import yaml
print(repr(yaml.safe_load("max_tokens: 1e3")["max_tokens"]))