One Model on the Client: the Gateway Tiers Automatically
Take model choice out of the UI — the client exposes a single model name, planning runs on the strong model and execution lands on the cost-efficient one, tiered on the hop the request was already taking.
What the community actually complains about
"Everyone has a different model in their settings and the same prompt comes out differently for two people," "the most common question in someone's first week is which model should I pick," "retiring a model means editing env vars on every dev machine." Model choice has been handed to the people with the least information to make it: they can't see their own spend, and they have no way to know whether this particular request deserves the strong model.
Drop everyone to a cheap model instead and the work degrades — hand planning and architecture judgment to a small model and the rework costs more than the savings. The split that actually matters is what this request is doing, planning or execution, and the client has never told the gateway that.
How GateLLM does it
The client keeps a single model name; a catch-all mapping (from "*") on the gateway normalizes whatever name the client asks for, so editing personal settings no longer changes where the request lands. There is nothing left to explain about which model to pick.
The tiering signal is already in the conversation: Claude Code's plan mode injects Plan mode is active on entry and Exited Plan Mode on exit. A before-slot script scans backwards from the last message, takes the last marker seen to determine the current state, then calls context.useModel() to switch routing. The decision is a string-position comparison — no separate classification call, no extra network hop.
Tiering has to live in the before slot: the after slot sees the translated upstream body and cannot read the client-injected markers, and after-slot useModel only switches within the same protocol, while plan to execution is a cross-model route change. request_payload rules (prompt_cache_key and friends) must be configured on both models: the entry model keeps its own while planning, the target model's apply once the switch happens.
5 levers that actually land
- Single ingress name: the client exposes one model name and a from "*" catch-all mapping normalizes every requested name to one landing model
- Automatic tiering: a before-slot script takes the last marker scanning backwards — the entry marker stays in history forever, so a "contains" check reads the state as plan-mode permanently
- Protocol guard: the script checks context.sourceProtocol first and returns early for non-Anthropic traffic, so one entry model serving several clients never gets mis-routed
- Observable: an SW tag in the log model column marks a script switch and MM marks an identity mapping; statistics and billing attribute to the final model name, so every tiering decision is auditable
- Zero client rollout: config changes need no per-machine edits and mappings hot-reload (the one exception is the script heap cap SCRIPT_MEMORY_LIMIT_MB, which needs a restart)