Give every model web search — without standing up RAG
Gateway-layer search injection: native search blocks synthesized per client protocol, so open models get search grounding too. Graceful degradation, and hijacked search is not billed.
What the community actually complains about
Open / self-hosted models have no search, so anything time-sensitive exposes them; standing up a RAG stack just for that is too heavy. Vendor search APIs each have their own format and per-call pricing, so every model switch means re-integrating.
"Can the gateway just add search for all models" is a recurring request from integrators.
How GateLLM does it
GateLLM hijacks search at the gateway layer: configure a search engine (serper, testable inline), then a per-model three-way toggle (follow capability declaration / force on / force off) decides which models get search. On a hit the gateway retrieves first, injects the results into the user message as reference material, and only then calls the upstream model — vendor text is never returned directly.
Injected content is synthesized into native search blocks per client protocol (Anthropic server_tool_use / OpenAI url_citation / Responses web_search_call), so clients render citations in each SDK's native format.
Search failures degrade gracefully without blocking the request, and hijacked search is excluded from usage billing.
5 levers that actually land
- Engine management: serper API key + inline test + multi-engine priority
- Per-model three-way toggle: follow capability / force on / force off, precedence model > upstream > capability declaration
- Native search blocks per protocol: Anthropic / OpenAI / Responses each render their own citation format
- Safe degradation: search failure never blocks the request; degradation reasons land in the log detail notice area
- Billing-transparent: hijacked search excluded from usage billing; memory usage has a documented ceiling formula