Glossary · Phase 1 · Building with LLMs
Time to first token
How long you wait before the first word of a reply appears. Most of it is the model reading the prompt.
Try it now
Watch the text grow with each text delta. The ping is skipped, and stop_reason only arrives at the end.
Learn it properly
Streaming responses · Show output as it's generated instead of waiting. About 25 minutes.
Other terms from the same lesson
- server-sent events: A simple way for a server to keep a connection open and push a stream of small updates, used to send a reply token by token.
- streaming: Showing a model's reply as it is written, word by word, instead of waiting for all of it.
Never trained an AI?
Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account.