Directories / Alternatives / Ollama

Open Source Alternatives

Open source alternatives to Ollama

Open source alternatives to Ollama, ranked by GitHub stars and freshness.

7 open-source alternatives in the index, ranked by GitHub stars and freshness.

P Apps Productivity low

llama.cpp

Various

LLM inference in C/C++

Alternative to Ollama, Gpt4all, Bitnet Cpp

★ 114,160 updated 3mo ago

freemium MIT C++

Best for: Developers building privacy-first applications or deploying models on resource-constrained devices

P Apps Productivity low

gpt4all

Various

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

Alternative to Llama Cpp, Ollama, Privategpt

★ 77,348 updated 1y ago

freemium MIT C++

Best for: Developers building offline-first applications or deploying LLMs on consumer hardware without cloud infrastructure.

O OSS Obs medium

Triton Server (TRTIS)

Community

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

Alternative to Vllm, Ollama

★ 10,720 updated 3mo ago

open-source BSD-3-Clause Python

Best for: Teams deploying large-scale inference services that need high throughput and multi-framework support.

O OSS Framework medium

Serge

Community

A web interface for chatting with Alpaca through llama.cpp. Fully dockerized, with an easy to use API.

Alternative to Ollama

★ 5,725 updated 9mo ago

open-source Apache-2.0 Svelte

Best for: Developers who want a quick self-hosted chat interface for Alpaca models using llama.cpp

O OSS Obs medium

Rapid-MLX

Community

The fastest local AI engine for Apple Silicon. 4.2x faster than Ollama, 0.08s cached TTFT, 100% tool calling. 17 tool parsers, prompt cache, reasoning separation, cloud routing. Dr

Alternative to Ollama, Llama Cpp, Vllm

★ 2,641 updated 3mo ago

open-source Apache-2.0 Python

Best for: Developers on Apple Silicon who need a fast, local OpenAI-compatible inference engine for tool-calling and reasoning tasks.

P Apps Productivity low

LM Studio

Various

Run local AI models like gpt-oss, Llama, Gemma, Qwen, and DeepSeek privately on your computer.

Alternative to Gpt4all, Ollama, Privategpt

freemium

Best for: Developers and privacy-conscious users who want offline access to open-source LLMs without cloud dependency

P Apps Productivity low

OpenAI API

Various

Announcement of the OpenAI API for text-to-text general-purpose AI models based on GPT-3. OpenAI blog, June 11, 2020.

Alternative to Llama Cpp, Gpt4all, Ollama

freemium

Best for: Developers needing quick integration of general text generation into their applications