📦raullenchai / Rapid-MLX

← 返回首页

Rapid-MLX is an open-source (Apache 2.0) OpenAI- and Anthropic-compatible LLM inference server for Apple Silicon, built on MLX, focused on reliable tool calling for coding agents. Up to 4× faster than Apple's MLX (mlx-lm) on the same weights.

⚖️ NOASSERTION#apple-silicon #local-llm #macos #mlx #tool-calling 🌐 rapidmlx.com📅 创建于 2026-02-25🔄 更新于 2026-10-29

🔗 GitHub Python

📄 仓库 README

在 GitHub 查看 ↗

来自 GitHub 仓库自述文件 · 版权归原作者所有

Rapid-MLX

Run Codex, Claude Code, OpenCode, Pi, Hermes, Crush or Goose with a local model on Mac:
rapid-mlx launch <agent>

Install on a stock Apple Silicon Mac with curl -fsSL https://rapidmlx.com/install.sh | bash, restart your terminal (or run export PATH="$HOME/.local/bin:$PATH"), then install your chosen coding agent and run, for example, rapid-mlx launch codex. Run the same command tomorrow; rapid-mlx launch codex --remove removes Rapid's private profile without changing your existing Codex settings. One-command guides for each agent. A first reply can take longer while a model downloads, loads, or processes the agent's opening prompt.

Rapid-MLX is an open-source (Apache 2.0) OpenAI- and Anthropic-compatible LLM inference server for Apple Silicon, built on MLX, focused on reliable tool calling for coding agents.
Up to 4× faster than Apple's MLX (mlx-lm), 1.5× on a typical task — same model, same weights.
Decode speed, Qwen3.5-9B 4-bit on a Mac mini M4 Pro, greedy decoding, both servers at their defaults — per-task results, method, and raw data.

Faster decode than oMLX and LM Studio on Qwen3.5-9B, Qwen3.8-27B, and Qwen3.6-35B

Measured speed on a Mac Studio M5 Max (36 GB) — Rapid-MLX 0.16.1 vs oMLX 0.7.0 and LM Studio, using the same base 4-bit weights for each model.

Model	Metric	Rapid-MLX	oMLX	LM Studio
Qwen3.5-9B	Decode (tok/s)	115.5	77.9	81.0
Qwen3.5-9B	4 streams (tok/s)	248.2	224.6	230.0
Qwen3.8-27B	Decode (tok/s)	37.9	25.3	26.0
Qwen3.8-27B	4 streams (tok/s)	82.8	75.3	69.7
Qwen3.6-35B	Decode (tok/s)	173.4	145.8	124.5
Qwen3.6-35B	4 streams (tok/s)	293.9	281.9	265.6
Qwen3.6-35B	2K-prompt first token (s, lower is better)	0.55	1.14	1.49

_{Single run per engine; default serving modes with Rapid's MTP text mode, oMLX's SSD cache, and LM Studio's configured context and four parallel slots. Full results and raw data.}

_{rapidmlx.com ·
Docs ·
Model mirror ·
Desktop app ·
Discord}

How it compares

Feature facts only, taken mostly from each project's own documentation as of 2026-09-26 (sources below). For speed, see the measured results above.

	Rapid-MLX	oMLX	Ollama	LM Studio	mlx-lm (`mlx_lm.server`)
License	Apache-2.0	Apache-2.0	MIT	Closed-source app; its MLX engine and `lms` CLI are MIT	MIT
Inference engine on a Mac	MLX	MLX	GGML/GGUF engine, plus an MLX engine for safetensors models	llama.cpp (GGUF) and MLX engines	MLX
Local API	OpenAI (`/v1/chat/completions`, `/v1/responses`) and Anthropic Messages (`/v1/messages`)	OpenAI and Anthropic Messages	Ollama API, OpenAI-compatible, Anthropic-compatible (subset)	OpenAI-compatible, Anthropic-compatible, LM Studio REST API	OpenAI-style chat API
GUI	Rapid-MLX Desktop (macOS app)	macOS menu-bar app and web admin panel	Desktop app (macOS, Windows)	Desktop app	None (CLI / Python)
Concurrent requests	Continuous batching	Continuous batching (mlx-lm `BatchGenerator`)	Parallel requests per model (`OLLAMA_NUM_PARALLEL`, default 1)	Continuous batching (llama.cpp engine; MLX engine since 0.4.2)	Continuous batching (`BatchGenerator`); one request at a time with a quantized KV cache
Prompt / KV cache	Radix prefix cache in memory (with state snapshots for hybrid models), saved to disk on shutdown and restored at startup; quantized KV cache	Tiered KV cache: in-memory hot tier plus SSD cold tier, reused across restarts	Reuses the previous request's prompt KV cache; KV cache quantization (`OLLAMA_KV_CACHE_TYPE`)	—	In-memory prompt cache; quantized KV cache (`--kv-bits`)
Tool-call parsing	27 parser modules, including an auto-detect fallback (`rapid_mlx/tool_parsers`)	mlx-lm formats plus the auto-detected families listed in its README	Tool calling	Tool use via the OpenAI-compatible API	Tool calls for models whose tokenizer declares tool-calling support

_{Sources: oMLX README ·
Ollama OpenAI and Anthropic compatibility, FAQ, MLX engine ·
LM Studio Anthropic compatibility, parallel requests, 0.4.2 changelog, app terms, mlx-engine ·
mlx-lm server docs and server.py.
Ollama prompt-cache reuse is from our benchmark notes.
— = not verified for this table. Spot an error? Please open an issue.}

vs Apple's mlx-lm: per-task results and raw data at https://rapidmlx.com/compare/mlx-lm.

Works with your AI stack

Use Rapid-MLX as a local backend for agents, apps, or your own code. If a client accepts an OpenAI- or Anthropic-compatible endpoint, it can usually use Rapid-MLX without an adapter.

For Claude Desktop, set up its in-app Gateway.