Glossary · Phase 1 · Building with LLMs
Prompt caching
Reusing the work a model already did on a repeated prompt start, which makes calls cheaper and faster.
Try it now
One extra write already costs more than it saves if nothing reads it. By two requests it has paid for itself, and 20 requests save about 84%.
Learn it properly
Cost and latency · Prompt caching, batching and routing to cheaper models. About 30 minutes.
Other terms from the same lesson
- latency: How long you wait for a response.
Never trained an AI?
Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account.