Open source alternatives to Ollama

Open source alternatives to Ollama, ranked by GitHub stars and freshness.

5 open-source alternatives in the index, ranked by GitHub stars and freshness.

O OSS Framework medium

OpenLLM

Community

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

Alternative to Ollama

★ 12,346 updated 1mo ago

open-source Apache-2.0 Python

Best for: Developers who need to serve open-source LLMs with OpenAI API compatibility

O OSS Framework medium

mistral.rs

Community

Fast, flexible LLM inference

Alternative to Llama Cpp, Vllm, Ollama +1 more

★ 7,205 updated 1mo ago

open-source MIT Rust

Best for: Rust developers seeking a fast, flexible LLM inference framework for performance-critical or resource-constrained environments.

O OSS Obs medium

Rapid-MLX

Community

The fastest local AI engine for Apple Silicon. 4.2x faster than Ollama, 0.08s cached TTFT, 100% tool calling. 17 tool parsers, prompt cache, reasoning separation, cloud routing. Dr

Alternative to Ollama

★ 2,641 updated 1mo ago

open-source Apache-2.0 Python

Best for: Developers on Apple Silicon who need a fast, local OpenAI-compatible inference engine for tool-calling and reasoning tasks.

M MCP Dev low

inference.sh

ac.inference.sh

Run 150+ AI apps — image, video, audio, LLMs, 3D and more. Browse, execute, stream results.

Alternative to Ollama

open-source

Best for: Developers who need a quick, browser-based sandbox to test and compare AI models across different tasks.

P Apps Productivity low

LLM

Various

LLM: A CLI utility and Python library for interacting with Large Language Models

Alternative to Ollama

freemium

Best for: Developers who want a lightweight, scriptable way to use LLMs locally.