Multi-Tenant Isolation: Key Groups, Header Rules, Per-Tenant Billing
Key groups split models and load balancers, header rules tell clients apart, per-tenant rate limit and billing — multiple teams on one gateway, costs and permissions crystal clear.
What the community actually complains about
Reddit: "50 engineers, finance asks who spent the $400k." Multiple teams on one API key means no one can tell who spent what. Relays also mix all tenant keys together, let tenant data cross, and cannot rate-limit per tenant.
How GateLLM does it
GateLLM uses key groups to split models and load balancers — each tenant gets independent keys and quotas; header rules (Header ACL) tell clients apart by User-Agent / custom header for differentiated policy; per-tenant rate limit and billing are attributed independently. Under BYOK, keys live in memory only, so tenants are isolated by construction — never mixed, never exfiltrated.
5 levers that actually land
- Key groups: split models and load balancers per tenant
- Header rules (Header ACL): tell clients apart by User-Agent / custom header
- Per-tenant rate limit: independent RPM / TPM / budget quotas
- Per-tenant billing: independent cost attribution and invoicing
- Keys in memory only: tenant isolation by construction, never mixed or exfiltrated