**The stateful, agentic API layer for [vLLM](https://github.com/vllm-project/vllm), written in Rust 🦀** *Run OpenAI-grade agentic workloads (Responses and Messages APIs, server-side tools, Codex, Claude Code) on your own GPUs.* [](LICENSE) [](Cargo.toml) [](https://github.com/vllm-project/agentic-api/actions/workflows/rust.yml) [](https://github.com/vllm-project/agentic-api/actions/workflows/pre-commit.yml)

🧠 Overview

vLLM gives you state-of-the-art inference throughput. But real agentic applications need more than raw tokens: they need conversation state, tool-call loops, and multi-turn orchestration. Today, all of that complexity lives in your client code.

Agentic API moves it server-side. It is a Rust-native gateway that sits in front of vLLM and owns the stateful agentic APIs, starting with an OpenAI-compatible Responses API. vLLM is one supported backend, not part of the Agentic API product name. Your application makes one API call and the server handles the rest: state hydration, tool execution, streaming, and continuation.

flowchart LR
    C(["🧑‍💻 Client<br/>Codex · Claude Code · SDKs · curl"]) -->|"📮 <code>POST /v1/responses</code> · <code>/v1/messages</code><br/>🌐 HTTP&nbsp;&nbsp;📡 SSE&nbsp;&nbsp;🔌 WebSocket"| A
    subgraph A ["⚡ Agentic API (Rust 🦀)"]
        direction TB
        S["🔄 State hydration<br/><code>previous_response_id</code>"]
        T["🛠️ Server-side tools<br/>web search · web fetch · MCP · code interpreter"]
        P["💾 Persistence<br/>SQLite · PostgreSQL"]
    end
    A -->|"🚀 <code>POST /v1/responses</code> · <code>/v1/messages</code><br/>⚙️ stateless&nbsp;&nbsp;🤝 OpenAI/Anthropic-compatible"| V(["🚀 vLLM core<br/>inference engine"])

    classDef client fill:#FFE8B3,stroke:#F59E0B,stroke-width:2px,color:#7C2D12
    classDef inner fill:#E0E7FF,stroke:#6366F1,stroke-width:2px,color:#312E81
    classDef engine fill:#DCFCE7,stroke:#22C55E,stroke-width:2px,color:#14532D

    class C client
    class S,T,P inner
    class V engine
    style A fill:#F5F3FF,stroke:#8B5CF6,stroke-width:2px,color:#5B21B6
    linkStyle 0 stroke:#F59E0B,stroke-width:2px
    linkStyle 1 stroke:#22C55E,stroke-width:2px

[!TIP] Point OpenAI Codex at Agentic API and drive it entirely with open models served by vLLM. No OpenAI account required.

✨ Key Features

• 🔄 Stateful conversations: the server manages history via previous_response_id or the Conversations API. No client-side message tracking, no replaying full transcripts.
• 🛠️ Server-side tool execution: an explicit tool-ownership model (gateway / client / provider) decides exactly what runs where. Web search ships today via You.com, Brave Search, Tavily, Serply, or a self-hosted SearXNG instance, alongside MCP tools and an opt-in embedded code interpreter; the model executes multi-step tool chains automatically.
• 🤝 Multi-agent orchestration: a stored Responses request can spawn and coordinate subagents server-side over HTTP, returning their attributed work in one response (details).
• 📡 Every transport: non-streaming HTTP, server-sent events for token streaming, and full WebSocket support for interactive clients, with stream_id multiplexing.
• 🧰 Codex and Claude Code ready: accepts Codex-shaped Responses traffic and Anthropic Messages traffic, preserving the tool declarations and item shapes each client depends on.
• 🗜️ Compaction: automatic and explicit context compaction through /v1/responses/compact (guide).
• 🔭 Observability: opt-in OpenTelemetry traces and metrics over OTLP (guide).
• ✅ Compatibility tested: validated against the Open Responses compatibility suite, with replay-cassette tests for real OpenAI, vLLM, SGLang, and NVIDIA Dynamo traffic.

🧭 API Surface

See the API reference for routing, authentication, and WebSocket details.

Responses created with store: true retain a terminal snapshot for retrieval, including status, usage, and this turn's output. Retrieval does not call the upstream model. Unknown IDs return a JSON 404; older records created before snapshot storage (or through history-only APIs) return 409 because their original response cannot be reconstructed faithfully. Request-scoped MCP credentials are stripped from stored tool definitions. Responses created with store: false do not have retrievable snapshots.

Retrieval requires a valid OIDC bearer token when OIDC is enabled. Otherwise, when OPENAI_API_KEY is nonempty, callers must send that key in Authorization: Bearer <key>; missing or invalid credentials return 401 before storage is read. With neither configured, retrieval allows unauthenticated access.

🚀 Quickstart

Agentic API CLI

Build the user-facing CLI and gateway binaries together:

cargo build -p agentic-server --bins

Launch Codex or Claude Code with an isolated Agentic API configuration:

./target/debug/agentic run codex --model Qwen/Qwen3-30B-A3B-FP8
./target/debug/agentic run claude --model Qwen/Qwen3-30B-A3B-FP8

To use an existing upstream, provide its http:// or https:// base URL. The harness model defaults to the first model the upstream lists at /v1/models; pass --model to choose a different one:

./target/debug/agentic run codex --upstream http://127.0.0.1:5050
./target/debug/agentic run claude \
  --upstream http://127.0.0.1:5050 \
  --model Qwen/Qwen3-30B-A3B-FP8

For self-hosted Claude Code setup—including models that Claude Code does not recognize in its own catalog—see Configure Claude Code with a self-hosted model.

SQLite is the default storage backend. Use PostgreSQL explicitly when the session is shared:

./target/debug/agentic run codex \
  --model Qwen/Qwen3-30B-A3B-FP8 \
  --database-url postgresql://user:password@localhost/agentic_api

Run preflight checks without launching a harness:

./target/debug/agentic validate \
  --upstream http://127.0.0.1:5050 \
  --model Qwen/Qwen3-30B-A3B-FP8 \
  --harness codex

Use AGENTIC_CODEX_BIN or AGENTIC_CLAUDE_BIN to override harness binary discovery. Add --no-color for scripts or --quiet for minimal lifecycle output. Use --yolo only in an externally isolated environment; it skips Claude permission checks and disables Codex approvals and sandboxing.

Python distribution

The agentic-api Python package is available on PyPI. Version 0.9.0 includes the Rust gateway, the agentic CLI, and a small Python launcher. Prebuilt wheels support Linux x86_64 (glibc 2.17 or newer), macOS Intel, and macOS Apple Silicon; Python 3.10 or newer is required.

With uv installed, run the packaged Rust CLI without a global installation:

uvx --from agentic-api==0.9.0 agentic --version
uvx --from agentic-api==0.9.0 agentic serve --upstream http://existing-vllm:8000

Or install the Python launcher, with the optional local inference runtime:

python -m pip install agentic-api==0.9.0
agentic-api serve --vllm-base-url http://existing-vllm:8000

python -m pip install "agentic-api[local]==0.9.0"
agentic-api serve --model MODEL_ID

The base install is for remote mode and does not install vLLM. The [local] extra installs the pinned vLLM runtime so the launcher can manage a local vLLM process on supported Linux hosts.

Use agentic-api --version for a quick install check and agentic-api doctor --mode remote --json when an agent or script needs machine-readable diagnostics.

Python launcher with uv

Use uv to install the published package or run the Python launcher without a global installation:

```bash uv pip install agentic-api==0.9.0 uv pip install "agentic-api[lo