Glossary

Unifies the seven cross-vendor coupling points (tool calling, structured output, prompt cache, reasoning tokens, multimodal, billing, error semantics) behind one interface. This is the prerequisite for Composition and Ensemble — otherwise each model has its own calling convention and composition requires glue code per model.

Mixing multiple models in one workflow: different steps run different models. The canonical pattern: a top-tier closed model (e.g. Claude Opus) plans while open SOTA models (GLM / Qwen / DeepSeek) execute — ~10% cost for ~98% capability. Invoked via client.compose([...]).

A single logical call fans out to N models; outputs are aggregated via vote / judge / merge / best-of-N. Yields quality and reliability a single model cannot — suited for high-stakes calls (fraud detection, medical assist, compliance judgment).

The umbrella for LLM-API observability, cost attribution, budget throttling, client admission, and audit trails. GateLLM integrates these into the gateway — never a per-token markup.

You bring your own model-vendor API keys. Keys live in gateway memory only — never persisted or transmitted; requests route through the gateway straight to vendor APIs. GateLLM takes no cut.