OpenAI-Compatible API

An OpenAI-compatible API is an HTTP interface that follows the same request and response conventions as OpenAI's official API — most importantly the /chat/completions endpoint. Any client, SDK, or framework built to talk to OpenAI can be redirected to a different provider simply by changing two values: the base URL and the model identifier. No other code changes are required. [NVIDIA NIM Free Models Guide]

This portability is what makes the pattern so widely adopted. Providers like NVIDIA NIM (NVIDIA Inference Microservices) expose their inference endpoints in this format precisely so developers can drop them into existing stacks with minimal friction. [NVIDIA NIM Free Models Guide] As of mid-2026, NVIDIA Build hosts over 100 models at https://integrate.api.nvidia.com/v1, all accessible through this same interface. [NVIDIA NIM Free API Guide]

Because the request and response structures are identical to OpenAI's, your existing parsing logic, retry handling, and tooling all continue to work unchanged. The response object — including choices[0].message.content — is the same shape regardless of which compatible provider is serving the model. [NVIDIA NIM Free Models Guide]

This makes the OpenAI-compatible API the foundation for Model Tiering Strategy: you can route different tasks to different providers (premium vs. free) without rewriting your integration layer. [NVIDIA NIM Free Models Guide]

Everything else — message format, temperature, max_tokens, streaming flags — remains identical. [NVIDIA NIM Free Models Guide] Streaming, function calling, and tool use are all supported on NVIDIA NIM endpoints. [NVIDIA NIM Free API Guide]

The application sends the same structured request; only the destination changes. The response shape is identical, so downstream logic never needs to know which provider was used. [NVIDIA NIM Free Models Guide]

Python (OpenAI SDK)

from openai import OpenAI

client = OpenAI(
    api_key="nvapi-xxxxxxxxxxxxxxxx",
    base_url="https://integrate.api.nvidia.com/v1"
)

response = client.chat.completions.create(
    model="zhipuai/glm-4",
    messages=[
        {"role": "user", "content": "Explain this function in one paragraph."}
    ],
    temperature=0.3,
    max_tokens=512
)

print(response.choices[0].message.content)

[NVIDIA NIM Free Models Guide]

LangChain

from langchain_openai import ChatOpenAI

nim_llm = ChatOpenAI(
    model="zhipuai/glm-4",
    api_key="nvapi-xxxxxxxxxxxxxxxx",
    base_url="https://integrate.api.nvidia.com/v1",
    temperature=0.2
)

response = nim_llm.invoke("Summarize this document in three bullet points.")

[NVIDIA NIM Free Models Guide]

CrewAI

from crewai import LLM

nim_model = LLM(
    model="openai/zhipuai/glm-4",  # CrewAI prefixes with "openai/"
    api_key="nvapi-xxxxxxxxxxxxxxxx",
    base_url="https://integrate.api.nvidia.com/v1"
)

[NVIDIA NIM Free Models Guide]

AutoGen

config_list = [
    {
        "model": "zhipuai/glm-4",
        "api_key": "nvapi-xxxxxxxxxxxxxxxx",
        "base_url": "https://integrate.api.nvidia.com/v1",
        "api_type": "openai"
    }
]

[NVIDIA NIM Free Models Guide]

Direct HTTP (curl)

curl https://integrate.api.nvidia.com/v1/chat/completions \
  -H "Authorization: Bearer nvapi-xxxxxxxxxxxxxxxx" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "zhipuai/glm-4",
    "messages": [{"role": "user", "content": "Hello, what can you do?"}],
    "max_tokens": 256
  }'

[NVIDIA NIM Free Models Guide]

The OpenAI-compatible interface works with virtually every popular AI framework because they all support a configurable base URL: [NVIDIA NIM Free Models Guide]

• OpenAI Python SDK — pass base_url and api_key to the constructor
• LangChain / LangGraph — use ChatOpenAI with a custom base_url
• CrewAI — use the LLM class, prefixing the model with openai/
• AutoGen — pass a config dict with api_type: "openai"
• Any HTTP client — raw curl or fetch calls to the endpoint
• Direct NIM (no wrapper) — the openai library alone is sufficient; no NVIDIA-specific SDK or custom auth flow is required [NVIDIA NIM Free API Guide]

For a detailed walkthrough of these framework integrations in the context of NVIDIA NIM, see NVIDIA NIM Framework Integrations.

The OpenAI-compatible API is the enabling mechanism for Model Tiering Strategy. Because swapping providers requires no structural code changes, you can build an agentic pipeline where:

• High-stakes reasoning and planning — routed to a premium model (Claude, GPT-4o)
• Repetitive classification, summarization, formatting — routed to a free NIM model like GLM-4
• Multilingual processing — routed to a model with bilingual strength

According to the NVIDIA NIM Free Models Guide, this tiering approach can reduce API spend by 40–70% on typical agentic workflows without meaningful quality degradation on the tasks routed to free models. [NVIDIA NIM Free Models Guide]

[NVIDIA NIM Free Models Guide]

For teams that want multi-model workflows without managing base URLs and API keys, MindStudio provides a visual workflow builder with 200+ models available out of the box. The same model-tiering logic is built into its workflow structure, eliminating the need to write routing code. [NVIDIA NIM Free Models Guide]