Glossary · Phase 1 · Building with LLMs
Token bucket
A way to rate-limit: each user has a bucket of request tokens that slowly refills. Each request takes one, and an empty bucket means wait.
Try it now
Three quick requests empty the bucket, the fourth is refused, and two seconds later one token has refilled. It prints ['ok', 'ok', 'ok', '429', 'ok'].
Learn it properly
Shipping an LLM app · Deploy, secure keys, rate-limit and handle failures. About 30 minutes.
Other terms from the same lesson
- backend: The part of an app that runs on your own server, out of users' sight. It's where secrets like API keys belong.
- front end: The part of an app that runs in the user's browser or phone. Anyone can read its code.
- health check: A simple web address your host calls now and then to check your app is still running.
- staging: A private copy of your app where you try changes before real users see them.
Never trained an AI?
Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account.