SDK & Framework Integrations

Every SDK connects to GateLLM the same three steps. Find your SDK, note its base_url convention and gotcha, then copy the pattern.

The gateway is protocol-oriented: whatever SDK you use, as long as it speaks OpenAI / Anthropic / Gemini / DashScope / Realtime / MCP, pointing its base_url at the gateway and swapping in an access key lets you call any configured model — including cross-protocol calls to other vendors' models.

1

Point base_url at the gateway

Default https://your-gateway.gatellm.io — whether it includes /v1 depends on the SDK (each example gives the exact value).

2

Swap in a gateway access key

Not an upstream sk-… key — the same access key mounts into whichever header your SDK expects.

3

Use the gateway model name

Whatever you configured in the console — not the vendor's official name.

Authorization: Bearer <key>
OpenAI · DashScope · generic
Bearer prefix case-insensitive
x-api-key: <key>
Anthropic SDK
Authorization also accepted
x-goog-api-key: <key>
Gemini SDK
Authorization also accepted
SDK / frameworkLanguageProtocolDocs
Official vendor SDKs
OpenAI SDKPython, Node.jsOpenAIExamples
Anthropic SDKPython, Node.jsAnthropicExamples
Google GenAI SDKPython, JavaScriptGeminiExamples
OpenAI-compatible third partiesPython, Node.jsOpenAI-compatibleExamples
DashScope native
DashScope SDKPythonDashScopeExamples
Unified SDKs & gateways
Vercel AI SDKTypeScriptOpenAI / AnthropicExamples
LiteLLMPythonOpenAI-compatibleExamples
PortkeyNode.js, PythonOpenAI-compatibleExamples
Braintrust / LangbaseTypeScript, PythonOpenAI-compatibleExamples
Semantic KernelPython, .NETOpenAI-compatibleExamples
Spring AIJavaOpenAI-compatibleExamples
Orchestration frameworks
LangChainPython, TypeScriptOpenAI / AnthropicExamples
LangGraphPython, TypeScriptOpenAI / AnthropicExamples
LlamaIndexPython, TypeScriptOpenAI-compatibleExamples
CrewAIPythonOpenAI-compatible (LiteLLM)Examples
AutoGen / AG2PythonOpenAI-compatibleExamples
DSPyPythonOpenAI-compatibleExamples
HaystackPythonOpenAI-compatibleExamples
Llama StackPythonOpenAI-compatibleExamples
Agent SDKs
OpenAI Agents SDKPython, TypeScriptOpenAI ResponsesExamples
Claude Agent SDKPython, TypeScriptAnthropicExamples
Pydantic AIPythonOpenAI-compatibleExamples
Google ADKPythonOpenAI-compatible (LiteLLM)Examples
MastraTypeScriptOpenAI-compatibleExamples
Microsoft Agent Framework.NET, PythonOpenAI-compatibleExamples
MCP SDKPython, TypeScriptMCPExamples
Realtime & audio
Realtime WebSocketAny WS clientOpenAI RealtimeExamples
openai_audio (ASR / TTS)Python, Node.jsOpenAI audioExamples
Local / self-hosted
OllamaRESTOpenAI-compatible (upstream)Examples
vLLMRESTOpenAI-compatible (upstream)Examples
llama.cpp serverRESTOpenAI-compatible (upstream)Examples
SGLangRESTOpenAI-compatible (upstream)Examples
Text Generation Inference (TGI)RESTOpenAI-compatible (upstream)Examples
LM StudioRESTOpenAI-compatible (upstream)Examples

Gotchas

The model name must match the gateway config

Not the vendor's official name, not the upstream model ID — whatever you named it in the console. Wrong name → 404 model_not_found.

Whether base_url includes /v1 varies by SDK

OpenAI SDK takes /v1; the Anthropic SDK takes the root (it appends /v1/messages); the Vercel AI SDK's createAnthropic takes /v1 (it appends /messages). Getting it wrong adds or drops a path segment.

Fields get translated and constrained across protocols

A max_tokens may become max_output_tokens with a lower bound (e.g. ≥16) when the upstream is a Responses-protocol model — too small → 400.

Thinking blocks are wrapped by the gateway

Claude / Gemini thinking content and signatures are passed through or stripped — take the type == "text" element of the content array, don't assume content[0] is text.

OpenAI Agents SDK defaults to /v1/responses

It calls the Responses API, not Chat Completions. If the upstream is openai (Chat), switch explicitly with OpenAIChatCompletionsModel; if openai_response, a string model name works as-is.

LlamaIndex validates the model name against an allowlist

The OpenAI class only accepts families it recognizes and rejects custom gateway names — use OpenAILike (Python) or declare the model info explicitly.

CrewAI is driven by LiteLLM under the hood

base_url isn't a framework parameter — it's the OPENAI_API_BASE env var, and model needs the openai/ prefix.

AutoGen requires explicit model_info

Unknown model names error on the default list — declare model_info explicitly to bypass the check.

Google GenAI SDK points at the root

base_url goes via http_options.base_url (no /v1) and the key mounts in the x-goog-api-key header.

FAQ