Choosing the Right Model Tier: Stop Defaulting to the Flagship

Most AI features do not need the biggest model. Matching tasks to model tiers, with routing and fallback, cuts cost and latency without a visible quality drop.

Every provider ships a ladder of models: small and fast, medium and balanced, large and expensive. Almost every codebase I have looked at uses exactly one of them, chosen once, usually the flagship, and never revisited. That single default is often the biggest line on the AI bill and the biggest chunk of user-facing latency.

Model selection is an engineering decision per task, not a one-time setting.

What each tier is actually for

• Small models: classification, routing, extraction from clean input, formatting, yes/no decisions. Sub-second, and roughly 10 to 20 times cheaper than the flagship.
• Mid-tier models: summarization, drafting, RAG answers over decent context, most chat. The default for user-facing text.
• Flagship models: multi-step reasoning, code generation, ambiguous inputs where a wrong answer is expensive, and anything where the mid-tier measurably fails.

Decide with an eval set, not a feeling

For each feature, run your evaluation set (20 to 50 real inputs) against the tier below the one you are using. If the pass rate holds, downgrade and keep the savings. If it drops, you have a documented reason for the bigger model. The flagship demo bias is real: everything feels like it needs the best model until you measure.

Routing and fallback

Two patterns cover most production needs. Escalation: try the small model first, and re-run on the larger one when the output fails validation or the confidence signal is weak. Routing: a cheap classifier (small model or plain rules) sends each request to the tier its category needs. Add a provider-outage fallback to a different model family and you get resilience from the same switch statement.

const MODEL_BY_TASK = {
    classify: 'claude-haiku-4-5',
    summarize: 'claude-sonnet-5',
    codereview: 'claude-opus-4-8',
} as const;

async function run(task: keyof typeof MODEL_BY_TASK, input: string) {
    const result = await callModel(MODEL_BY_TASK[task], input);
    if (task === 'classify' && !isValid(result)) {
        return callModel('claude-sonnet-5', input); // escalate once
    }
    return result;
}

Takeaways

• One hard-coded flagship model is a cost and latency bug, not a safe default.
• Downgrade experiments with an eval set turn model choice into a measurement.
• Escalation and routing give you small-model prices with big-model reliability.
• Revisit the mapping when providers ship new tiers; the ladder moves every few months.