Index Ai Academy
Phase 1 · Building with LLMs · Prompting

Streaming responses

Show output as it's generated instead of waiting.

About 25 minutes · A free account opens the whole lesson

Create a free account Try part of it here

Already have an account? Sign in to open this lesson

Why it matters

A long reply from a reasoning model can take 30 seconds or more to finish. Without streaming, your user stares at a spinner the whole time. With streaming, words start to appear in a second or two. That's why chat products stream. Streaming also matters behind the scenes. Requests with a very large max_tokens value need it to avoid network timeouts. And the event format you'll learn here is the same one that tool use and agents build on.

What you'll be able to do

  • Explain what streaming changes (how soon text appears) and what it doesn't (the reply, the cost)
  • Read a stream of server-sent events and rebuild the reply from its text deltas
  • Get the stop reason and token usage from a stream's events, and catch errors mid-stream
  • Decide when a feature should stream and when it should wait for the full reply

What it covers

  1. Waiting versus watching
  2. What arrives
  3. Reading a stream in Python
  4. Rebuilding the reply
  5. Worked example: rebuild the reply and its metadata
  6. Streaming on your own machine
  7. Letting the SDK do the work

Waiting versus watching

Picture two restaurants serving the same three-course meal. The first kitchen holds every plate until dessert is ready, then brings it all at once. The second brings each course as soon as it's cooked. The food is the same and so is the bill. At the second place, you start eating much sooner.

So far every request you've sent has worked like the first kitchen. You send it, you wait, and the whole reply arrives at once. The model still wrote that reply one token at a time. A token is a small chunk of text, often a short word or part of one. The API held on to the tokens until the last one was done.

With streaming, the API works like the second kitchen. It sends the reply in small pieces as they're generated. Two numbers describe the difference:

  • Time to first token: how long until the first piece of text arrives. Streaming cuts this from the full generation time to a second or two.
  • Total time: how long until the reply is complete. Streaming doesn't change it.

The model, the reply and the price are the same either way. You get the same tokens, only sooner. The typing effect in chat apps is real. You're watching tokens arrive as the model produces them.

How to ask for a stream

Add "stream": true to the request body. You no longer get one JSON response back. Instead, the connection stays open. The API sends a series of server-sent events (SSE). SSE is a simple text format for pushing updates over HTTP. HTTP is the system that browsers and web servers use to talk.

Each event is two lines of text, then a blank line:

  • an event: line that names the event;
  • a data: line that holds JSON with the details.
Quick check
What does adding "stream": true change?
  1. The model writes a shorter reply
  2. The reply costs less
  3. The reply arrives in small events as it's written, with the same tokens at the same price
  4. The model thinks harder
Show the answer

The reply arrives in small events as it's written, with the same tokens at the same price

Streaming changes when you see the words, not what they are or what they cost.

What arrives

Here's a short stream, trimmed, for a request that asked for an email subject line:

event: message_start
data: {"type":"message_start","message":{"id":"msg_01Example","model":"claude-opus-5-5","content":[],"usage":{"input_tokens":31,"output_tokens":1}}}

event: content_block_start
data: {"type":"content_block_start","index":0,"content_block":{"type":"thinking","thinking":""}}

event: content_block_stop
data: {"type":"content_block_stop","index":0}

event: content_block_start
data: {"type":"content_block_start","index":1,"content_block":{"type":"text","text":""}}

event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"text_delta","text":"Rain jackets"}}

event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"text_delta","text":" 30% off,"}}

event: content_block_stop
data: {"type":"content_block_stop","index":1}

event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"end_turn"},"usage":{"output_tokens":87}}

event: message_stop
data: {"type":"message_stop"}

It looks busy, but there are only a few kinds of event. Here's what each one tells you:

EventWhat it tells you
message_startThe reply has begun. Holds the message id, the model and usage.input_tokens.
content_block_startA new block begins: thinking, text and, later in Phase 1, tool_use.
content_block_deltaA piece of a block. For text, delta.type is "text_delta" and delta.text is the new text. Thinking blocks send their own delta types.
content_block_stopThat block is finished.
message_deltaNear the end: delta.stop_reason and the final usage.output_tokens.
message_stopThe stream is over.
pingKeeps the connection alive. Ignore it.
errorSomething went wrong partway through, such as the API being overloaded.

A delta means "the change since last time." Each text delta carries a few new words. You rebuild the reply by adding every text delta together, in order. It's like a friend sending one long message as several chat bubbles. You read them top to bottom as one message.

Claude Opus 5.5 thinks before it answers, so a thinking block usually comes first. Its text stays empty unless you ask for a summary of the thinking. The real reply starts in the next block. The rule is the same as for a normal reply: keep the text, skip everything else.

Quick check
In the stream above, which events carry the words of the reply?
  1. message_delta
  2. content_block_delta events whose delta type is text_delta
  3. message_start
  4. content_block_start
Show the answer

content_block_delta events whose delta type is text_delta

Each text_delta holds a few new words in delta.text. The content_block_start for a text block arrives with empty text, and the other events carry ids, counts and the stop reason.

Reading a stream in Python

The academy's proxy doesn't pass streams through. A request with "stream": true gets a 400 error from lesson code. So this lesson works with a recording: the exact text a stream sends. The code that reads it is the same code you'd run on a live stream, one line at a time.

import json

RECORDED = """event: message_start
data: {"type":"message_start","message":{"id":"msg_01Example","model":"claude-opus-5-5","content":[],"usage":{"input_tokens":31,"output_tokens":1}}}

event: content_block_start
data: {"type":"content_block_start","index":0,"content_block":{"type":"text","text":""}}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"Rain jackets"}}

event: ping
data: {"type":"ping"}

event: content_block_delta
data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" 30% off,"}}

event: message_stop
data: {"type":"message_stop"}"""

for line in RECORDED.splitlines():
    if line.startswith("data: "):
        event = json.loads(line[6:])
        print(event["type"])
It runs right here in your browser, with nothing to install.

Here's what each piece does:

  • splitlines() breaks a string into a list of its lines.
  • Only the data: lines matter. Each one repeats the event's name in its "type" field.
  • line[6:] slices off the six characters of "data: ", counting the space. What's left is JSON that json.loads can read.
Quick check
Look at the code above. What does it print on its fourth line?
  1. Rain jackets
  2. content_block_delta
  3. message_stop
  4. ping
Show the answer

ping

It prints one event type per data: line, in order: message_start, content_block_start, content_block_delta, ping, content_block_delta, message_stop. It prints types, not the reply text.

Rebuilding the reply

Printing event names is a start. To get the reply itself, you need a loop that adds each text delta to a string. Step through it and watch the string grow:

Quick check
What does this print?
pieces = ["The ", "sky ", "is ", "blue"]
text = ""
for p in pieces:
    text += p
print(len(text.split()))
  1. 3
  2. 4
  3. 13
  4. 1
Show the answer

4

The loop joins the pieces into "The sky is blue", and split() cuts that into 4 words. Each streaming event carries a small piece like these. Join them in order and you have the whole reply.

Try it now

Watch the text grow with each text delta. The ping is skipped, and stop_reason only arrives at the end.

Worked example

Worked example: rebuild the reply and its metadata

Read a full recording. Show each piece as it would arrive. Then report the stop reason and the token counts.

import json

RECORDED = """event: message_start
data: {"type":"message_start","message":{"id":"msg_01Example","model":"claude-opus-5-5","content":[],"usage":{"input_tokens":31,"output_tokens":1}}}

event: content_block_start
data: {"type":"content_block_start","index":0,"content_block":{"type":"thinking","thinking":""}}

event: content_block_stop
data: {"type":"content_block_stop","index":0}

event: content_block_start
data: {"type":"content_block_start","index":1,"content_block":{"type":"text","text":""}}

event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"text_delta","text":"Rain jackets"}}

event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"text_delta","text":" 30% off,"}}

event: content_block_delta
data: {"type":"content_block_delta","index":1,"delta":{"type":"text_delta","text":" through Sunday"}}

event: content_block_stop
data: {"type":"content_block_stop","index":1}

event: message_delta
data: {"type":"message_delta","delta":{"stop_reason":"end_turn"},"usage":{"output_tokens":87}}

event: message_stop
data: {"type":"message_stop"}"""

text = ""
stop_reason = None
input_tokens = 0
output_tokens = 0

for line in RECORDED.splitlines():
    if not line.startswith("data: "):
        continue
    event = json.loads(line[6:])
    if event["type"] == "message_start":
        input_tokens = event["message"]["usage"]["input_tokens"]
    elif event["type"] == "content_block_delta" and event["delta"]["type"] == "text_delta":
        piece = event["delta"]["text"]
        print(piece, end="")
        text += piece
    elif event["type"] == "message_delta":
        stop_reason = event["delta"]["stop_reason"]
        output_tokens = event["usage"]["output_tokens"]

print()
print("stop_reason:", stop_reason)
print("tokens:", input_tokens, "in,", output_tokens, "out")
In the lesson you can change this and run it.

print(piece, end="") prints without starting a new line. The pieces join up on screen the way they would in a chat window. In a live app, each print happens the moment its event arrives.

The token counts come from two places. Input tokens are in message_start. Output tokens are in message_delta, because the output count isn't known until the model stops writing.

The empty thinking block sends no text deltas. So the elif chain passes over it, with no special code needed.

Quick check
You're streaming a reply. Where do you find out whether it was cut off by max_tokens?
  1. In each content_block_delta
  2. In the message_delta event near the end of the stream
  3. In the HTTP status code
  4. In the first event, message_start
Show the answer

In the message_delta event near the end of the stream

The stop reason can't be known until the model stops writing. So it arrives in message_delta, along with the final output token count.

Streaming on your own machine

Outside the academy, the requests library can read a live stream line by line:

import json
import os
import requests

response = requests.post(
    "https://api.anthropic.com/v1/messages",
    headers={
        "x-api-key": os.environ["ANTHROPIC_API_KEY"],
        "anthropic-version": "2026-10-21",
        "content-type": "application/json",
    },
    json={
        "model": "claude-opus-5-5",
        "max_tokens": 2000,
        "stream": True,
        "messages": [{"role": "user", "content": "Write a short welcome email for new newsletter subscribers."}],
    },
    stream=True,
    timeout=60,
)
response.raise_for_status()
for line in response.iter_lines(decode_unicode=True):
    if line.startswith("data: "):
        event = json.loads(line[6:])
        if event["type"] == "content_block_delta" and event["delta"]["type"] == "text_delta":
            print(event["delta"]["text"], end="", flush=True)

There are two stream settings here, and you need both:

  • "stream": True inside the JSON body asks the API to stream.
  • stream=True as an argument to requests.post tells requests to hand you the response as it arrives. Without it, requests waits for all of it.

Then iter_lines gives you one line at a time. flush=True makes each piece show up on screen right away instead of waiting in a queue.

Quick check
Your code puts "stream": True in the JSON body but leaves out stream=True in requests.post. What does the user see?
  1. The whole reply appearing at once at the end
  2. A 400 error, because the API refuses the request
  3. Only the first piece of the reply
  4. The text appearing piece by piece, as normal
Show the answer

The whole reply appearing at once at the end

The API still streams, but requests downloads the full response before handing it over. Your loop only starts once everything has arrived, so nothing shows up early.

Letting the SDK do the work

An SDK (software development kit) is a ready-made library for calling an API. The official SDK handles the events for you:

import anthropic

client = anthropic.Anthropic()   # reads ANTHROPIC_API_KEY from the environment
with client.messages.stream(
    model="claude-opus-5-5",
    max_tokens=2000,
    messages=[{"role": "user", "content": "Write a short welcome email for new newsletter subscribers."}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)
    final = stream.get_final_message()

print()
print(final.stop_reason, final.usage.output_tokens)

Reading it from the top:

  • with ... as stream: opens the stream. It also makes sure the connection closes when the block ends, even if something fails.
  • text_stream gives you only the text pieces.
  • get_final_message() gives you the complete response, as if you hadn't streamed. It has content, stop_reason and usage.

If you only want streaming to avoid timeouts, that last call is all you need.

Errors in the middle of a stream

A streaming response sends its 200 OK status before the model has written anything. So a problem partway through can't change the status code. It arrives as an event instead:

event: error
data: {"type":"error","error":{"type":"overloaded_error","message":"Overloaded"}}

Think of a phone call that drops halfway through a sentence. The call connected fine, but you didn't hear the whole message. Your loop has to watch for this event. When one arrives, the text so far is incomplete. Throw it away or label it as cut off. Then retry, or tell the user.

The same goes for a stop_reason of "max_tokens". That means the reply hit your length limit, exactly as without streaming.

When to stream

Stream in three cases:

  • A person is watching the reply being written. Think of a chat, a drafting tool or an assistant in a dashboard.
  • The replies are long.
  • max_tokens is large enough that a single waiting request could time out.

Don't bother for background jobs nobody watches, such as tagging 5,000 reviews overnight. Short replies don't need it either.

Structured output is a special case. Half a JSON object can't be parsed, so your code waits for the end either way. Stream it only to avoid timeouts, and parse the final message.

Streaming changes how a reply is delivered, not whether it's right. Check the finished result before anything acts on it.

Quick check
Which of these features gains the most from streaming?
  1. A chat assistant that drafts long campaign briefs while the marketer watches
  2. A nightly job that tags 5,000 reviews with a single-word topic
  3. A function that returns one JSON object your code saves to a database
  4. A script that counts tokens in last month's prompts
Show the answer

A chat assistant that drafts long campaign briefs while the marketer watches

Streaming helps when a person is waiting on long text. Nobody watches the nightly job, and the JSON can't be used until it's complete anyway.

Practice what you just read

  • 4 code examples you can edit and run
  • a practice exercise that checks your code
  • a 5-question quiz
  • 5 review cards for spaced repetition

Create a free account

Try a quiz question

What does turning on streaming change?

  1. The total cost of the reply
  2. How soon the first words appear, while the reply's tokens and price stay the same
  3. The quality of the reply, because the model gets feedback as it writes
  4. Which model answers the request

Answer it at the end of the lesson to see if you're right, and why.

Before this lesson