Adding AI Features to a Next.js App: What Actually Works in Production

Practical patterns for shipping LLM-powered features in a Next.js application: API route proxies, streaming, cost control, and graceful failure.

Most tutorials about AI features stop at "call the API and render the response." That works in a demo. In production, the interesting problems start after that: keys leaking to the client, requests that take 20 seconds, bills that grow faster than usage, and models that occasionally return something you did not expect.

This post collects the patterns I reach for when adding LLM-powered features to a Next.js application, based on building and maintaining full stack apps where AI is a feature, not the product.

Never call the model from the browser

The first rule is boring but non-negotiable: all model calls go through your backend. In Next.js that means an API route or a server action. The browser never sees your API key, and you get one place to add auth, rate limiting, logging, and cost tracking.

// app/api/assist/route.ts
export async function POST(req: Request) {
    const { prompt } = await req.json();

    // auth + rate limit checks live here, before any tokens are spent
    const response = await fetch('https://api.anthropic.com/v1/messages', {
        method: 'POST',
        headers: {
            'x-api-key': process.env.ANTHROPIC_API_KEY!,
            'anthropic-version': '2023-06-01',
            'content-type': 'application/json',
        },
        body: JSON.stringify({
            model: 'claude-sonnet-5',
            max_tokens: 1024,
            messages: [{ role: 'user', content: prompt }],
        }),
    });

    return Response.json(await response.json());
}

Stream, because latency is the real UX problem

A full LLM response can take 5 to 30 seconds. Users will not wait that long staring at a spinner, but they will happily read text as it arrives. Streaming turns the same latency into a feature: the first words appear in under a second.

Next.js API routes can return a ReadableStream directly, and the model providers all support server-sent events. On the client, read the stream chunk by chunk and append to state. It is less code than most people expect, and it is the single biggest UX improvement you can make.

Control cost before you have a cost problem

• Pick the smallest model that does the job. Classification, extraction, and rewriting rarely need the flagship model. The price difference is often 10 to 20 times.
• Cache aggressively. Identical or near-identical requests (same document, same question) should never hit the API twice. A hash of the normalized input makes a fine cache key.
• Cap output tokens. max_tokens is a budget, set it to what the feature actually needs.
• Log token usage per feature from day one. When the bill grows, you want to know which feature grew it.

Design for the model being wrong

Treat model output as untrusted input. If you expect JSON, validate it against a schema and retry once on failure. If the output goes into your UI, sanitize it like user content. If the feature makes a decision, keep a human-visible trail of why.

And always ship a fallback: if the AI feature times out or fails, the app should degrade to its non-AI behavior, not break. Users forgive a missing suggestion; they do not forgive a broken page.

Takeaways

• Proxy every model call through your backend.
• Stream responses; first-token latency beats total latency.
• Small models plus caching solve most cost problems.
• Validate output, retry once, and always have a non-AI fallback.