Glossary · Phase 1 · Building with LLMs
Model gateway
The one place an app sends model requests through, which picks a provider for each request and falls back to another when one fails.
Try it now
Format differences belong in adapters, provider choice in the gateway, and quality differences in your evals.
Learn it properly
One app, several model providers · Two API formats side by side, adapters to one internal format, a gateway with fallbacks, and evals before switching. About 40 minutes.
Other terms from the same lesson
- chat completions: A request and response format for model APIs that many providers and local model servers share: the system prompt is a message, and tool calls come back with their arguments as a JSON string.
- provider adapter: A pair of functions that translate an app's own request and reply format to and from one model provider's API.
Never trained an AI?
Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account.