Open Source Alternatives
Open source alternatives to Ollama
Open source alternatives to Ollama, ranked by GitHub stars and freshness.
5 open-source alternatives in the index, ranked by GitHub stars and freshness.
OpenLLM
Community
Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
Best for: Developers who need to serve open-source LLMs with OpenAI API compatibility
mistral.rs
Community
Fast, flexible LLM inference
Best for: Rust developers seeking a fast, flexible LLM inference framework for performance-critical or resource-constrained environments.
Rapid-MLX
Community
The fastest local AI engine for Apple Silicon. 4.2x faster than Ollama, 0.08s cached TTFT, 100% tool calling. 17 tool parsers, prompt cache, reasoning separation, cloud routing. Dr
Best for: Developers on Apple Silicon who need a fast, local OpenAI-compatible inference engine for tool-calling and reasoning tasks.
inference.sh
ac.inference.sh
Run 150+ AI apps — image, video, audio, LLMs, 3D and more. Browse, execute, stream results.
Best for: Developers who need a quick, browser-based sandbox to test and compare AI models across different tasks.
LLM
Various
LLM: A CLI utility and Python library for interacting with Large Language Models
Best for: Developers who want a lightweight, scriptable way to use LLMs locally.