Rapid-MLX

Overview

Rapid-MLX is a local AI inference engine optimized for Apple Silicon, offering drop-in OpenAI API compatibility. It achieves 4.2x faster performance than Ollama with 0.08s cached time-to-first-token and 100% tool calling support. The engine includes 17 tool parsers, prompt caching, reasoning separation, and cloud routing for hybrid local/remote execution.

Best for

Best for
Developers on Apple Silicon who need a fast, local OpenAI-compatible inference engine for tool-calling and reasoning tasks.

Use cases

• Running local LLMs with OpenAI-compatible endpoints for tools like Claude Code or Cursor
• Accelerating tool-calling workflows with 17 built-in parsers and cached responses
• Offloading reasoning tasks locally while routing other requests to cloud models

Notes

Rapid-MLX is a local AI inference engine optimized for Apple Silicon, offering drop-in OpenAI API compatibility. It achieves 4.2x faster performance than Ollama with 0.08s cached time-to-first-token and 100% tool calling support. The engine includes 17 tool parsers, prompt caching, reasoning separation, and cloud routing for hybrid local/remote execution.

2,641 stars on GitHub. Last updated 2026-06-01. Licensed Apache-2.0.

Use cases

• Running local LLMs with OpenAI-compatible endpoints for tools like Claude Code or Cursor
• Accelerating tool-calling workflows with 17 built-in parsers and cached responses
• Offloading reasoning tasks locally while routing other requests to cloud models

Pros

• Significantly faster than Ollama on Apple Silicon hardware
• Full OpenAI API compatibility simplifies integration with existing tools
• Includes advanced features like prompt caching and reasoning separation out of the box

Cons

• Limited to Apple Silicon hardware, excluding Intel Macs and other platforms
• Community-maintained project may have less support than commercial alternatives
• Performance gains depend on model caching and may not apply to all workloads

Indexed from awesome-llmops and enriched against its public facts.