Multi-Tenant Isolation: Key Groups, Header Rules, Per-Tenant Billing
Key groups split models and load balancers, header rules tell clients apart, per-tenant rate limit and billing — multiple teams on one gateway, costs and permissions crystal clear.
What the community actually complains about
Reddit: "50 engineers, finance asks who spent the $400k." Multiple teams on one API key means no one can tell who spent what.
Relays also mix all tenant keys together, let tenant data cross, and cannot rate-limit per tenant.
How GateLLM does it
GateLLM uses key groups to split models and load balancers — each tenant gets independent keys and quotas; header rules (Header ACL) tell clients apart by User-Agent / custom header for differentiated policy; per-tenant rate limit and billing are attributed independently. Tag each tenant key and you can also aggregate cost across tenants by tag.
Under BYOK, keys live in memory only, so tenants are isolated by construction — never mixed, never exfiltrated.
5 levers that actually land
- Key groups: split models and load balancers per tenant
- Header rules (Header ACL): tell clients apart by User-Agent / custom header
- Per-tenant rate limit: independent RPM / TPM / budget quotas
- Per-tenant billing: independent cost attribution and invoicing
- Keys in memory only: tenant isolation by construction, never mixed or exfiltrated