# SDK & Framework Integrations - GateLLM

> Connect any SDK or framework to GateLLM: OpenAI, Anthropic, Gemini, DashScope, LangChain, LlamaIndex, CrewAI, Vercel AI SDK, OpenAI Agents SDK, Ollama and 30+ more — one base_url swap, per-SDK gotchas included.

Canonical: https://www.gatellm.io/integrations/

## SDK & Framework Integrations
Every SDK connects to GateLLM the same three steps. Find your SDK, note its base_url convention and gotcha, then copy the pattern.
The gateway is protocol-oriented: whatever SDK you use, as long as it speaks OpenAI / Anthropic / Gemini / DashScope / Realtime / MCP, pointing its base_url at the gateway and swapping in an access key lets you call any configured model — including cross-protocol calls to other vendors' models.
## Works with the SDK you already have
Point any SDK or framework at the gateway with one line. OpenAI, Anthropic, Gemini and DashScope connect natively; LangChain, LlamaIndex, CrewAI, Vercel AI SDK and 30+ more through the same base_url swap.
### Point base_url at the gateway
Default https://your-gateway.gatellm.io — whether it includes /v1 depends on the SDK (each example gives the exact value).
### Swap in a gateway access key
Not an upstream sk-… key — the same access key mounts into whichever header your SDK expects.
### Use the gateway model name
Whatever you configured in the console — not the vendor's official name.
```
# base_url convention differs by SDK — match the one you're using
from openai import OpenAI
OpenAI(base_url="https://your-gateway.gatellm.io/v1")        # OpenAI SDK: /v1

import anthropic
anthropic.Anthropic(base_url="https://your-gateway.gatellm.io")  # Anthropic SDK: root (appends /v1/messages)

# Vercel AI SDK createAnthropic: baseURL ".../v1" (appends /messages) — the reverse
# LangChain ChatOpenAI: base_url="https://your-gateway.gatellm.io/v1"
```
### `Authorization: Bearer <key>`
- SDK / framework: OpenAI · DashScope · generic
- Bearer prefix case-insensitive

### `x-api-key: <key>`
- SDK / framework: Anthropic SDK
- Authorization also accepted

### `x-goog-api-key: <key>`
- SDK / framework: Gemini SDK
- Authorization also accepted

## SDK selection matrix

### Official vendor SDKs
OpenAI / Anthropic / Google SDKs connect natively — only base_url changes
| SDK / framework | Language | Protocol | Docs |
| --- | --- | --- | --- |
| OpenAI SDK | Python, Node.js | OpenAI | https://docs.gatellm.io/en/sdk/vendor-sdks |
| Anthropic SDK | Python, Node.js | Anthropic | https://docs.gatellm.io/en/sdk/vendor-sdks |
| Google GenAI SDK | Python, JavaScript | Gemini | https://docs.gatellm.io/en/sdk/vendor-sdks |
| OpenAI-compatible third parties | Python, Node.js | OpenAI-compatible | https://docs.gatellm.io/en/sdk/vendor-sdks |

### DashScope native
Text, image, async video, embedding & rerank through one endpoint
| SDK / framework | Language | Protocol | Docs |
| --- | --- | --- | --- |
| DashScope SDK | Python | DashScope | https://docs.gatellm.io/en/quickstart/dashscope |

### Unified SDKs & gateways
Vercel AI SDK, LiteLLM, Portkey, Semantic Kernel, Spring AI
| SDK / framework | Language | Protocol | Docs |
| --- | --- | --- | --- |
| Vercel AI SDK | TypeScript | OpenAI / Anthropic | https://docs.gatellm.io/en/sdk/unified-sdks |
| LiteLLM | Python | OpenAI-compatible | https://docs.gatellm.io/en/sdk/unified-sdks |
| Portkey | Node.js, Python | OpenAI-compatible | https://docs.gatellm.io/en/sdk/unified-sdks |
| Braintrust / Langbase | TypeScript, Python | OpenAI-compatible | https://docs.gatellm.io/en/sdk/unified-sdks |
| Semantic Kernel | Python, .NET | OpenAI-compatible | https://docs.gatellm.io/en/sdk/unified-sdks |
| Spring AI | Java | OpenAI-compatible | https://docs.gatellm.io/en/sdk/unified-sdks |

### Orchestration frameworks
LangChain, LlamaIndex, CrewAI, AutoGen, DSPy, Haystack
| SDK / framework | Language | Protocol | Docs |
| --- | --- | --- | --- |
| LangChain | Python, TypeScript | OpenAI / Anthropic | https://docs.gatellm.io/en/sdk/orchestration-frameworks |
| LangGraph | Python, TypeScript | OpenAI / Anthropic | https://docs.gatellm.io/en/sdk/orchestration-frameworks |
| LlamaIndex | Python, TypeScript | OpenAI-compatible | https://docs.gatellm.io/en/sdk/orchestration-frameworks |
| CrewAI | Python | OpenAI-compatible (LiteLLM) | https://docs.gatellm.io/en/sdk/orchestration-frameworks |
| AutoGen / AG2 | Python | OpenAI-compatible | https://docs.gatellm.io/en/sdk/orchestration-frameworks |
| DSPy | Python | OpenAI-compatible | https://docs.gatellm.io/en/sdk/orchestration-frameworks |
| Haystack | Python | OpenAI-compatible | https://docs.gatellm.io/en/sdk/orchestration-frameworks |
| Llama Stack | Python | OpenAI-compatible | https://docs.gatellm.io/en/sdk/orchestration-frameworks |

### Agent SDKs
OpenAI Agents, Claude Agent, Pydantic AI, ADK, Mastra
| SDK / framework | Language | Protocol | Docs |
| --- | --- | --- | --- |
| OpenAI Agents SDK | Python, TypeScript | OpenAI Responses | https://docs.gatellm.io/en/sdk/agent-sdks |
| Claude Agent SDK | Python, TypeScript | Anthropic | https://docs.gatellm.io/en/quickstart/anthropic-interop |
| Pydantic AI | Python | OpenAI-compatible | https://docs.gatellm.io/en/sdk/agent-sdks |
| Google ADK | Python | OpenAI-compatible (LiteLLM) | https://docs.gatellm.io/en/sdk/agent-sdks |
| Mastra | TypeScript | OpenAI-compatible | https://docs.gatellm.io/en/sdk/agent-sdks |
| Microsoft Agent Framework | .NET, Python | OpenAI-compatible | https://docs.gatellm.io/en/sdk/agent-sdks |
| MCP SDK | Python, TypeScript | MCP | https://docs.gatellm.io/en/reference/mcp-config |

### Realtime & audio
Realtime WebSocket voice + openai_audio ASR / TTS
| SDK / framework | Language | Protocol | Docs |
| --- | --- | --- | --- |
| Realtime WebSocket | Any WS client | OpenAI Realtime | https://docs.gatellm.io/en/sdk/realtime-and-audio |
| openai_audio (ASR / TTS) | Python, Node.js | OpenAI audio | https://docs.gatellm.io/en/sdk/realtime-and-audio |

### Local / self-hosted
Ollama, vLLM, llama.cpp as upstreams — keep any client SDK
| SDK / framework | Language | Protocol | Docs |
| --- | --- | --- | --- |
| Ollama | REST | OpenAI-compatible (upstream) | https://docs.gatellm.io/en/sdk/local-inference |
| vLLM | REST | OpenAI-compatible (upstream) | https://docs.gatellm.io/en/sdk/local-inference |
| llama.cpp server | REST | OpenAI-compatible (upstream) | https://docs.gatellm.io/en/sdk/local-inference |
| SGLang | REST | OpenAI-compatible (upstream) | https://docs.gatellm.io/en/sdk/local-inference |
| Text Generation Inference (TGI) | REST | OpenAI-compatible (upstream) | https://docs.gatellm.io/en/sdk/local-inference |
| LM Studio | REST | OpenAI-compatible (upstream) | https://docs.gatellm.io/en/sdk/local-inference |

## Gotchas
### The model name must match the gateway config
Not the vendor's official name, not the upstream model ID — whatever you named it in the console. Wrong name → 404 model_not_found.

### Whether base_url includes /v1 varies by SDK
OpenAI SDK takes /v1; the Anthropic SDK takes the root (it appends /v1/messages); the Vercel AI SDK's createAnthropic takes /v1 (it appends /messages). Getting it wrong adds or drops a path segment.

### Fields get translated and constrained across protocols
A max_tokens may become max_output_tokens with a lower bound (e.g. ≥16) when the upstream is a Responses-protocol model — too small → 400.

### Thinking blocks are wrapped by the gateway
Claude / Gemini thinking content and signatures are passed through or stripped — take the type == "text" element of the content array, don't assume content[0] is text.

### OpenAI Agents SDK defaults to /v1/responses
It calls the Responses API, not Chat Completions. If the upstream is openai (Chat), switch explicitly with OpenAIChatCompletionsModel; if openai_response, a string model name works as-is.

### LlamaIndex validates the model name against an allowlist
The OpenAI class only accepts families it recognizes and rejects custom gateway names — use OpenAILike (Python) or declare the model info explicitly.

### CrewAI is driven by LiteLLM under the hood
base_url isn't a framework parameter — it's the OPENAI_API_BASE env var, and model needs the openai/ prefix.

### AutoGen requires explicit model_info
Unknown model names error on the default list — declare model_info explicitly to bypass the check.

### Google GenAI SDK points at the root
base_url goes via http_options.base_url (no /v1) and the key mounts in the x-goog-api-key header.

## FAQ
### Does GateLLM work with LangChain / LlamaIndex / CrewAI?
Yes — point each framework's underlying chat model at the gateway's OpenAI-compatible endpoint. Each has a specific "change base_url" entry point: LangChain uses ChatOpenAI(base_url=…), LlamaIndex uses OpenAILike, CrewAI sets the OPENAI_API_BASE env var (it's LiteLLM under the hood).
### Can I use the Claude Agent SDK or Claude Code through GateLLM?
Yes. They read the ANTHROPIC_BASE_URL / ANTHROPIC_AUTH_TOKEN env vars — point them at the gateway root and use a gateway access key. Name the gateway model exactly the ID the agent requests (or align via ANTHROPIC_MODEL).
### How do I connect the OpenAI Agents SDK?
Point its OpenAI client at the gateway and set the model. Note it defaults to /v1/responses — if your upstream is a Chat-protocol model, switch explicitly with OpenAIChatCompletionsModel. Disable tracing (set_tracing_disabled) so traces don't hit the gateway too.
### Can GateLLM front Ollama / vLLM / llama.cpp?
Yes — configure them as OpenAI-compatible upstreams and keep any client SDK. The gateway adds routing, fallback and governance on top of your self-hosted models.
### Does the base_url include /v1 or not?
It depends on the SDK: the OpenAI SDK takes /v1; the Anthropic SDK takes the root (it appends /v1/messages); the Vercel AI SDK's createAnthropic takes /v1 (it appends /messages). The docs give the exact value per SDK.
### What is a gateway access key vs an upstream key?
The gateway access key is what your client presents, in whichever header your SDK uses (Authorization, x-api-key, or x-goog-api-key). Upstream vendor keys stay in the gateway and are never exposed to your client.

## Related
- [Home](https://www.gatellm.io/index.md)
- [Pricing](https://www.gatellm.io/pricing.md)
- [Cost Calculator](https://www.gatellm.io/calculator.md)
- [About](https://www.gatellm.io/about.md)
- [Book a Demo](https://www.gatellm.io/demo.md)
- [FAQ](https://www.gatellm.io/faq.md)
- [Compare AI Gateways](https://www.gatellm.io/compare.md)
- [Compare LiteLLM](https://www.gatellm.io/compare/litellm.md)
- [Compare OpenRouter](https://www.gatellm.io/compare/openrouter.md)
- [Compare Alibaba Cloud AI Gateway](https://www.gatellm.io/compare/alibaba-cloud.md)
- [Compare Azure API Management](https://www.gatellm.io/compare/azure-apim.md)
- [Compare Apigee](https://www.gatellm.io/compare/apigee.md)
- [Compare Cloudflare AI Gateway](https://www.gatellm.io/compare/cloudflare.md)
- [Compare Kong AI Gateway](https://www.gatellm.io/compare/kong.md)
- [Compare Portkey](https://www.gatellm.io/compare/portkey.md)
- [LiteLLM vs Kong AI Gateway](https://www.gatellm.io/compare/litellm-vs-kong.md)
- [LiteLLM vs OpenRouter](https://www.gatellm.io/compare/litellm-vs-openrouter.md)
- [LiteLLM vs Portkey](https://www.gatellm.io/compare/litellm-vs-portkey.md)
- [Use Cases](https://www.gatellm.io/use-cases.md)
- [Reduce LLM Cost](https://www.gatellm.io/use-cases/reduce-cost.md)
- [High Availability](https://www.gatellm.io/use-cases/high-availability.md)
- [Multi-Tenant Isolation](https://www.gatellm.io/use-cases/multi-tenant.md)
- [Compliance & Audit](https://www.gatellm.io/use-cases/compliance-audit.md)
- [Troubleshooting by Symptom](https://www.gatellm.io/use-cases/troubleshooting.md)
- [Mixed Multi-Vendor Model Stack](https://www.gatellm.io/use-cases/mixed-model-stack.md)
- [Self-Host AI Inside Office](https://www.gatellm.io/use-cases/self-hosted-office-ai.md)
- [Token Usage Visibility & Attribution](https://www.gatellm.io/use-cases/token-ops.md)
- [Token FinOps: AI Cost Allocation & Control](https://www.gatellm.io/use-cases/token-finops.md)
- [MCP Tool Governance](https://www.gatellm.io/use-cases/mcp-tool-governance.md)
- [Web Search for Every Model](https://www.gatellm.io/use-cases/web-search-grounding.md)
- [Multi-Key Rotation & Standby](https://www.gatellm.io/use-cases/multi-key-pooling.md)
- [Admission Control for Connected Clients](https://www.gatellm.io/use-cases/client-admission-control.md)
- [Enforce Which Model Agents Actually Run](https://www.gatellm.io/use-cases/enforce-model-policy.md)
- [No Model Choice on the Client](https://www.gatellm.io/use-cases/automatic-model-tiering.md)
- [One Protocol, Any Model](https://www.gatellm.io/use-cases/one-protocol-any-model.md)
- [Office / Microsoft 365 Add-in](https://www.gatellm.io/office-addin.md)
- [Glossary](https://www.gatellm.io/glossary.md)
- [Model Catalog](https://www.gatellm.io/models.md)
- [AI Coding Agents & Harnesses](https://www.gatellm.io/coding-agents.md)
- [Security & Compliance](https://www.gatellm.io/security.md)
- [Privacy Policy](https://www.gatellm.io/privacy-policy.md)
- [Terms of Service](https://www.gatellm.io/terms.md)
- [Cookie Policy](https://www.gatellm.io/cookies.md)
- [Data Processing Agreement (DPA)](https://www.gatellm.io/dpa.md)
- [Sub-processor List](https://www.gatellm.io/subprocessors.md)
- [Security Questionnaire (CAIQ/SIG/VSA)](https://www.gatellm.io/trust.md)
- [Incident Response & Support](https://www.gatellm.io/sla.md)
- [Vulnerability Disclosure Policy](https://www.gatellm.io/vulnerability-disclosure.md)
- [Open Source License Notice](https://www.gatellm.io/open-source-licenses.md)
- [Version Support & Lifecycle (EOL)](https://www.gatellm.io/lifecycle.md)
- [Engineering Blog](https://www.gatellm.io/blog.md)
- [Your Agents Have More Permissions Than Your Employees: Governing MCP Tools at the Gateway](https://www.gatellm.io/blog/mcp-tool-governance.md)
- [The $10k Invoice Problem: Five Ways to Cut LLM Spend Without Dumbing Down Your App](https://www.gatellm.io/blog/reduce-cost.md)
- [Your Vendor's List Price Is Not Your Cost: Applying Real FinOps to Token Spend](https://www.gatellm.io/blog/token-finops.md)
- [llms.txt](https://www.gatellm.io/llms.txt)