Glossary · Phase 1 · Building with LLMs
Server-sent events
A simple way for a server to keep a connection open and push a stream of small updates, used to send a reply token by token.
Add
"stream": trueto the request body. You no longer get one JSON response back. Instead, the connection stays open. The API sends a series of server-sent events (SSE). SSE is a simple text format for pushing updates over HTTP. HTTP is the system that browsers and web servers use to talk.
Try it now
Watch the text grow with each text delta. The ping is skipped, and stop_reason only arrives at the end.
Learn it properly
Streaming responses · Show output as it's generated instead of waiting. About 25 minutes.
Other terms from the same lesson
- streaming: Showing a model's reply as it is written, word by word, instead of waiting for all of it.
- time to first token: How long you wait before the first word of a reply appears. Most of it is the model reading the prompt.
Never trained an AI?
Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account.