Glossary · Phase 1 · Building with LLMs
Latency
How long you wait for a response.
Try it now
One extra write already costs more than it saves if nothing reads it. By two requests it has paid for itself, and 20 requests save about 84%.
Learn it properly
Cost and latency · Prompt caching, batching and routing to cheaper models. About 30 minutes.
Other terms from the same lesson
- prompt caching: Reusing the work a model already did on a repeated prompt start, which makes calls cheaper and faster.
Never trained an AI?
Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account.