Most agent demos use one model for everything. That works until the bill arrives, or the model goes down, or one provider quietly changes behavior on a Tuesday. Then you learn the cost of betting the whole system on a single API.
Provider routing is the answer, and it is also a trap. Done well, it cuts cost and adds resilience. Done badly, it becomes the most fragile part of your stack. This post is about doing it well.
Why route at all
One model is simpler. Simpler is usually right. So start by being honest about whether you need routing.
You need it when three things are true. Your volume is high enough that price-per-token matters. Your tasks vary in difficulty. And an outage on one provider would actually hurt you. If any of those is false, pick one good model and move on.
When all three are true, the math changes fast. A classification step and a multi-file refactor have wildly different needs. Sending both to the same frontier model means overpaying on the easy half. Routing lets each task land where it fits.
The three lanes
Think in lanes, not in a ranked list. There is no best model. There is a best model for a task, a budget, and a deadline.
Claude is the lane we reach for on agentic work and long-horizon coding. Adaptive thinking and high effort settings carry multi-step tasks that other models drop halfway through. It is not the cheapest token, so we do not waste it on trivial calls.
GPT is the lane for broad coverage and ecosystem fit. When a task needs a specific tool integration or a format the OpenAI side handles cleanly, that is where it goes. We treat it as a peer, not a fallback.
Local is the lane for privacy, latency, and floor cost. A small model running on your own hardware answers in milliseconds and never sends data off-box. The catch is real: capability drops, and ops burden rises. You now own the GPU, the quantization, and the 2 a.m. page.
The router is the risk
Here is the part the demos skip. The routing layer is code, and code fails.
The common failure mode is a router that picks the wrong lane confidently. A task that needed the strong model goes to the cheap one, the output looks plausible, and nobody notices until a customer does. Silent quality regressions are worse than outages. An outage you see.
The second failure mode is fan-out cost. Some teams route to two providers and compare answers. That doubles spend and latency on every call. Sometimes worth it for a verifier step. Usually not. Measure before you assume.
The third is the abstraction tax. A normalization layer that pretends every provider has the same API will eventually lie to you. Tool-call formats differ. Token counting differs. Thinking and effort controls differ per model. Hide those differences and you lose the controls that make each lane good.
How we wire it
We keep routing rules boring and explicit. A task type maps to a lane. The map lives in config, not in a model's head. You can read it, diff it, and roll it back.
Cheap tasks get a cheap default with a strong fallback on failure. Hard tasks get the strong model first, no fallback to a weaker one, because a wrong answer is worse than a retry. Privacy-flagged tasks never leave local, full stop, regardless of cost.
Every call gets logged with the lane it took, the token count, and the latency. That log is the whole game. Without it you cannot tell whether routing is saving money or quietly degrading quality. With it, you tune in days instead of guessing for months.
We also pin behavior we depend on. Tool-call parsing reads structured output, never raw string matching, because escaping differs across providers and changes between model versions. Small thing. Saves a class of 2 a.m. bugs.
What it actually buys you
The payoff is not magic. It is margin and uptime. A support agent we built answers in 12 minutes, down from 4 hours, and routing keeps that running when a single provider has a bad day.
This is the same discipline behind everything we ship. We built 10 production products in 7 months as a 2-person team, ex-AWS Federal, with deep cloud certs behind us. None of that came from clever abstractions. It came from boring config, real logs, and refusing to route a task to a lane it cannot handle.
If you are standing up agents, our OpenClaw Installation sets up the routing layer with the guardrails above baked in. Fixed price, you own the code, we exit. No retainer, no lock-in.
Provider routing is worth it when your volume and risk justify the complexity. The trick is treating the router as the riskiest component, not the smartest one.
Want to pressure-test your own setup? Book a free 30-minute call and we will look at where your agent spend is going.