Streaming LLM Responses in Next.js: From Spinner to First Token

How to stream model output through a Next.js route handler to the browser: ReadableStream on the server, incremental rendering on the client, and the abort and error states people forget.

A complete LLM answer can take 10 to 30 seconds to generate. If your UI waits for the whole thing, every AI feature feels broken, no matter how good the answer is. Streaming fixes the perception problem without touching the model: the first words show up in well under a second and the user reads while the model writes.

Here is the full path, server to client, with the edge cases that only show up once real users touch it.

The server side: pipe the provider stream through

Model providers all support server-sent events. In a Next.js route handler you do not need a library: request the stream from the provider, transform it into plain text chunks, and return a ReadableStream in the Response. The route stays a dumb pipe, which is exactly what you want.

// app/api/chat/route.ts
export async function POST(req: Request) {
    const { prompt } = await req.json();

    const upstream = await fetch('https://api.anthropic.com/v1/messages', {
        method: 'POST',
        headers: {
            'x-api-key': process.env.ANTHROPIC_API_KEY!,
            'anthropic-version': '2023-06-01',
            'content-type': 'application/json',
        },
        body: JSON.stringify({
            model: 'claude-sonnet-5',
            max_tokens: 1024,
            stream: true,
            messages: [{ role: 'user', content: prompt }],
        }),
    });

    // forward the SSE bytes as they arrive
    return new Response(upstream.body, {
        headers: { 'content-type': 'text/event-stream' },
    });
}

The client side: read chunks, append to state

On the client, fetch the route, grab the reader from response.body, and decode chunks as they arrive. Append parsed text deltas to a state variable and React re-renders as the answer grows. Keep the parsing tolerant: a chunk boundary can split an SSE event in half, so buffer until you have a full line.

const res = await fetch('/api/chat', {
    method: 'POST',
    body: JSON.stringify({ prompt }),
    signal: controller.signal,
});

const reader = res.body!.getReader();
const decoder = new TextDecoder();

while (true) {
    const { done, value } = await reader.read();
    if (done) break;
    const text = parseSseChunk(decoder.decode(value, { stream: true }));
    if (text) setAnswer((prev) => prev + text);
}

The states people forget

• Abort. Wire an AbortController to a stop button and to component unmount. Without it, a user who navigates away keeps paying for tokens they will never see.
• Mid-stream errors. A stream can fail after half the answer has rendered. Keep the partial text, show a small "response interrupted" note, and offer a retry instead of blanking the screen.
• Autoscroll with an escape hatch. Follow the bottom while new text arrives, but stop following the moment the user scrolls up to reread something.
• Disable resubmission while a stream is open, or queue it. Two concurrent streams writing into one state variable produce interleaved nonsense.

Takeaways

• Streaming is a UX fix for latency you cannot remove; ship it for anything longer than a sentence.
• Keep the route handler a pass-through pipe and do rendering logic on the client.
• Abort handling is not optional; it is your cost control and your cleanup.
• Design the interrupted state on purpose; it will happen in production.