SDK & Framework Integrations

Every SDK connects to GateLLM the same three steps. Find your SDK, note its base_url convention and gotcha, then copy the pattern.

The gateway is protocol-oriented: whatever SDK you use, as long as it speaks OpenAI / Anthropic / Gemini / DashScope / Realtime / MCP, pointing its base_url at the gateway and swapping in an access key lets you call any configured model — including cross-protocol calls to other vendors' models.

1

Point base_url at the gateway

Default https://your-gateway.gatellm.io — whether it includes /v1 depends on the SDK (each example gives the exact value).

2

Swap in a gateway access key

Not an upstream sk-… key — the same access key mounts into whichever header your SDK expects.

3

Use the gateway model name

Whatever you configured in the console — not the vendor's official name.

Authorization: Bearer <key>
OpenAI · DashScope · generic
Bearer prefix case-insensitive
x-api-key: <key>
Anthropic SDK
Authorization also accepted
x-goog-api-key: <key>
Gemini SDK
Authorization also accepted

SDK selection matrix

SDK / frameworkLanguageProtocolDocs
Official vendor SDKsGitHub examples ↗
OpenAI SDKPython, Node.jsOpenAIDocs
Anthropic SDKPython, Node.jsAnthropicDocs
Google GenAI SDKPython, JavaScriptGeminiDocs
OpenAI-compatible third partiesPython, Node.jsOpenAI-compatibleDocs
DashScope native
DashScope SDKPythonDashScopeDocs
Unified SDKs & gatewaysGitHub examples ↗
Vercel AI SDKTypeScriptOpenAI / AnthropicDocs
LiteLLMPythonOpenAI-compatibleDocs
PortkeyNode.js, PythonOpenAI-compatibleDocs
Braintrust / LangbaseTypeScript, PythonOpenAI-compatibleDocs
Semantic KernelPython, .NETOpenAI-compatibleDocs
Spring AIJavaOpenAI-compatibleDocs
Orchestration frameworksGitHub examples ↗
LangChainPython, TypeScriptOpenAI / AnthropicDocs
LangGraphPython, TypeScriptOpenAI / AnthropicDocs
LlamaIndexPython, TypeScriptOpenAI-compatibleDocs
CrewAIPythonOpenAI-compatible (LiteLLM)Docs
AutoGen / AG2PythonOpenAI-compatibleDocs
DSPyPythonOpenAI-compatibleDocs
HaystackPythonOpenAI-compatibleDocs
Llama StackPythonOpenAI-compatibleDocs
Agent SDKs
OpenAI Agents SDKPython, TypeScriptOpenAI ResponsesDocs
Claude Agent SDKPython, TypeScriptAnthropicDocs
Pydantic AIPythonOpenAI-compatibleDocs
Google ADKPythonOpenAI-compatible (LiteLLM)Docs
MastraTypeScriptOpenAI-compatibleDocs
Microsoft Agent Framework.NET, PythonOpenAI-compatibleDocs
MCP SDKPython, TypeScriptMCPDocs
Realtime & audio
Realtime WebSocketAny WS clientOpenAI RealtimeDocs
openai_audio (ASR / TTS)Python, Node.jsOpenAI audioDocs
Local / self-hosted
OllamaRESTOpenAI-compatible (upstream)Docs
vLLMRESTOpenAI-compatible (upstream)Docs
llama.cpp serverRESTOpenAI-compatible (upstream)Docs
SGLangRESTOpenAI-compatible (upstream)Docs
Text Generation Inference (TGI)RESTOpenAI-compatible (upstream)Docs
LM StudioRESTOpenAI-compatible (upstream)Docs

Gotchas

The model name must match the gateway config

Not the vendor's official name, not the upstream model ID — whatever you named it in the console. Wrong name → 404 model_not_found.

Whether base_url includes /v1 varies by SDK

OpenAI SDK takes /v1; the Anthropic SDK takes the root (it appends /v1/messages); the Vercel AI SDK's createAnthropic takes /v1 (it appends /messages). Getting it wrong adds or drops a path segment.

Fields get translated and constrained across protocols

A max_tokens may become max_output_tokens with a lower bound (e.g. ≥16) when the upstream is a Responses-protocol model — too small → 400.

Thinking blocks are wrapped by the gateway

Claude / Gemini thinking content and signatures are passed through or stripped — take the type == "text" element of the content array, don't assume content[0] is text.

OpenAI Agents SDK defaults to /v1/responses

It calls the Responses API, not Chat Completions. If the upstream is openai (Chat), switch explicitly with OpenAIChatCompletionsModel; if openai_response, a string model name works as-is.

LlamaIndex validates the model name against an allowlist

The OpenAI class only accepts families it recognizes and rejects custom gateway names — use OpenAILike (Python) or declare the model info explicitly.

CrewAI is driven by LiteLLM under the hood

base_url isn't a framework parameter — it's the OPENAI_API_BASE env var, and model needs the openai/ prefix.

AutoGen requires explicit model_info

Unknown model names error on the default list — declare model_info explicitly to bypass the check.

Google GenAI SDK points at the root

base_url goes via http_options.base_url (no /v1) and the key mounts in the x-goog-api-key header.

FAQ