Glossary

Unifies the seven cross-vendor coupling points (tool calling, structured output, prompt cache, reasoning tokens, multimodal, billing, error semantics) behind one interface. This is the prerequisite for mixing models across vendors — otherwise each model has its own calling convention and every switch requires glue code.

Routing different request phases to different models at the gateway — identity-scoped model mapping plus plan-mode switching. The canonical pattern: a top-tier closed model (e.g. Claude Opus) plans while open SOTA models (GLM / Qwen / DeepSeek) execute — ~10% cost for ~98% capability. Configured in the console (rules + scripts), zero client-code change.

Running the same prompt through several models in parallel and merging the results — the fan-out lives in your orchestration layer (e.g. GitHub Actions), while the gateway supplies protocol interop and unified metering. Yields quality and reliability a single model cannot — suited for high-stakes calls (code review, fraud detection, compliance judgment).

The usage-side operating layer for LLM APIs — observability, client admission, audit trails, and programmatic management. It answers "who is using what, how much, and can they get in"; Token FinOps (its counterpart) answers "how tokens become money, who it is charged to, and what stops it". GateLLM integrates these into the gateway — never a per-token markup.

The financial control layer that applies FinOps visibility, allocation, and governance to token spend: a self-owned rate card turns tokens into money, tags land spend on the right cost center, and daily / monthly cost quotas turn budgets into request-blocking gates. Complements Token Ops — the former sees usage, this one sees money.

You bring your own model-vendor API keys. Keys live in gateway memory only — never persisted or transmitted; requests route through the gateway straight to vendor APIs. GateLLM takes no cut.