Every team that watches an LLM bill for a month arrives at the same idea: most of these requests are easy, so send the easy ones somewhere cheaper. It is correct, it usually works, and it has a failure mode that is invisible until you do the arithmetic.
The failure mode is that an escalation pays twice. When the cheap model cannot handle a request, you have already paid for its attempt — the tokens went out, the answer came back, it was not good enough. So the saving is not "the price difference on the requests we routed"; it is the price difference minus everything you spent on attempts that did not work.
That gives a break-even escalation rate, and it is lower than people assume.
The arithmetic
One request under routing costs the cheap model, always, plus the expensive model on the fraction that escalates:
routed = cheap + escalation_rate × expensive
unrouted = expensiveSetting them equal gives the break-even:
escalation_rate = 1 − cheap / expensiveThat is the whole thing. The break-even is not 100% and it is not related to how good the models are. It is one minus the price ratio, and you can read it straight off a rate card.
At list prices, Haiku 4.5 is a fifth of Opus 5 on both input and output — the rate cards are published by Anthropic, OpenAI and Google, and the calculator reads them from one data file so the prose and the tool cannot disagree — so the break-even escalation rate is 80%. That is generous: you could send four requests in five to the expensive model and routing would still be ahead. Two models a factor of 1.5 apart give you 33%, and at that point a routing layer is fragile — a shift in the traffic mix can quietly turn it into a tax.
Work out your own
calculator · model routing
Does routing to a cheaper model actually save money?
daily cost at different escalation rates
where the routed bill goes
The arithmetic that decides this is that an escalation pays twice: the cheap attempt already happened and its tokens are already billed. So the break-even escalation rate is the gap between the two models expressed as a fraction, and it is not near 100% unless the models are far apart in price. Cached input pricing is ignored here because it depends on how repetitive your prompts are; where a long system prompt is reused, caching moves this in routing's favour. Prices from each provider's published rate card, checked 12 Sep 2026, and several are introductory.
Three things worth trying:
Pick two models close in price. Sonnet 5 against Opus 5 is a 2.5× gap, so the break-even drops to 60%. Still workable. Now try two models a factor of 1.3 apart and watch the headroom vanish. The right question before building a router is not "can I route" but "is the gap big enough to be worth the machinery".
Add a model router. Set the router to a cheap model and watch the break-even fall. A classifier on every request is a third charge on the 80% that were going to be handled by the cheap model anyway, and it has to be small enough to disappear into the rounding.
Set escalation to 60%. Still saving, but the margin is thin enough that a bad week of traffic wipes it out. That is the number to alert on, not the bill.
Three ways to decide, and the one that usually wins
Static rules cost nothing and are surprisingly effective. Route by endpoint, by the length of the input, by whether the user is on a paid plan, by whether the task is extraction or generation. The information you need to route is usually already in the request, and a rule is auditable in a way a classifier is not.
Try cheap, verify, escalate is the version that generalises. Run the cheap model, apply a test to its answer, escalate when the test fails. The test is where all the value is, and it is specific to the task: does the JSON parse and match the schema, does the extracted date exist, does the answer cite a document that was actually retrieved, is the refusal genuine. A cheap deterministic verifier beats an expensive judge almost every time.
This is also the strategy whose cost the calculator models exactly, because there is no router charge — only the escalations that pay twice.
A model classifies first is the most popular architecture diagram and the least often justified. It adds a call to every request, including the large majority that the cheap model was going to handle. For it to pay, the classifier has to be a lot cheaper than the cheap model and the classification has to avoid enough escalations to cover itself. It is worth it when the expensive path is very expensive — a long-context or reasoning model — and not worth it when the gap is one tier.
| Strategy | Cost per request | When it wins |
|---|---|---|
| Static rules | Zero | The routing signal is already in the request |
| Verify and escalate | Cheap model, plus big model on failures | You can write a cheap, reliable test for the answer |
| Model classifier | Classifier on every request, plus the above | The expensive path is several tiers up, not one |
| No routing | The expensive model, every time | The gap is small, or the quality risk is not worth it |
What routing does to quality, and how to find out
The cost arithmetic is the easy half. The hard half is that a cheaper model gets some requests wrong, and the wrong answers are not distributed the way your test set is.
The measurement that matters is not "does the cheap model score well on our evals". It is on which requests does it fail, and whether the verifier catches those. A cheap model that is 95% as good on average but fails on exactly the requests your most important customers send is worse than useless, and an aggregate accuracy number will not show it.
Two practical rules. Sample the escalation decisions and read them: a sample of a hundred routed-and-not-escalated requests, judged by someone who knows the domain, is worth more than any automated metric. And measure the escalation rate continuously, because it is the early-warning signal for a traffic shift — a rising escalation rate means the mix has changed, which is worth knowing before the bill tells you.
The sample size you need to tell two models apart is larger than most teams use, which is the usual reason routing decisions get made on a vibe.
quick check
Your cheap model costs a third of your expensive one. At what escalation rate does routing stop saving money?
Break-even is one minus the price ratio. At a third of the price, that is 1 − 0.33 = 67%. Past that, the accumulated cost of cheap attempts you threw away exceeds the saving on the ones that worked. The narrower the price gap, the less headroom you have — and the escalation rate is not a constant.
What the arithmetic does not settle
Latency is not in the ledger. An escalation is two sequential model calls, so the requests that escalate are roughly twice as slow. If 20% escalate, your p95 is now the escalation path, not the cheap path. For a user-facing feature that can be the reason not to route, whatever the bill says.
Output tokens dominate and vary. These calculations assume a fixed output length. In practice a cheap model is often more verbose, or fails by producing a long wrong answer, and output tokens are priced at four to five times input. Measure your actual token distribution rather than assuming the two models write the same amount.
The escalation rate is a moving target. It is set by your traffic and it drifts. Every product change, every new customer segment and every prompt revision moves it. The headroom between your current rate and the break-even is the only thing standing between a saving and a tax, so treat it as a monitored number.
Failure is not binary. "Escalate on failure" assumes you can tell. For extraction and classification, you usually can. For open-ended generation, the verifier is nearly as hard as the generator, and that is exactly the case where routing is least likely to work.
Prices move and the ratio moves with them. Several of the prices in the calculator are introductory. A router built around a 5× gap is a router built around a number that a provider can change in a blog post, so the break-even is worth re-checking when rate cards move — the arithmetic survives, the conclusion might not. The same caveat applies to any per-user LLM cost model.
questions people ask
Does routing to a cheaper LLM actually save money?
Usually yes, but less than the price difference suggests, because an escalation pays for both models. At a 20% escalation rate with a 5× price gap, the saving is about 60%. The break-even escalation rate is one minus the price ratio — 80% for a 5× gap, 33% for a 1.5× gap.
What is the break-even escalation rate?
One minus the ratio of the cheap model's price to the expensive one's. Above it, the cost of cheap attempts you discarded exceeds what you saved on the ones that worked, and routing costs more than not routing.
Should the router be a model or a set of rules?
Start with rules, because they are free, deterministic and usually sufficient — endpoint, user tier, input length and task type carry most of the signal. A model classifier adds a charge to every request, including the majority the cheap model would have handled anyway, so it only pays when the expensive path is several tiers up.
How do I decide when to escalate?
Run the cheap model and test its answer, rather than predicting in advance. Schema validation, a parse check, a citation check or a confidence threshold are cheap and specific. The test is where the value is; a good verifier is far cheaper than a good generator.
Does routing hurt quality?
It can, and the aggregate accuracy number will not tell you. What matters is which requests the cheap model fails on, and whether your verifier catches them. Sample the requests that were not escalated and have someone who knows the domain read them.
How does prompt caching interact with routing?
It helps routing. Cached input is billed at a fraction of the normal rate, which reduces the cost of the cheap attempt more than it reduces the escalation, so the break-even rate moves up. Where a long system prompt is shared, caching alone sometimes removes the need for a router.
The short version
Routing to a cheaper model first saves real money, and the saving is smaller than the price difference because every escalation pays for the attempt that failed. The break-even escalation rate is one minus the price ratio: 80% for a 5× gap, 33% for a 1.5× gap. Check that number before building anything, because it decides whether the architecture is worth having.
Prefer static rules, then try-verify-escalate. A model that classifies every request in advance is the most common design and the one that most often fails to pay for its own call.
Then watch the escalation rate rather than the bill. It is the number that drifts, and the distance between it and your break-even is the entire margin.