Open Source Alternatives
Open source alternatives to OpenLLM
Open source alternatives to OpenLLM, ranked by GitHub stars and freshness.
4 open-source alternatives in the index, ranked by GitHub stars and freshness.
vLLM
Community
A high-throughput and memory-efficient inference and serving engine for LLMs
Alternative to Tensorrt Llm, Lmdeploy, Sglang +1 more
Best for: Teams building production LLM APIs and services that need to maximize throughput and minimize latency under concurrent load.
LMDeploy
Community
LMDeploy is a toolkit for compressing, deploying, and serving LLMs.
Alternative to Vllm, Sglang, Tensorrt Llm +2 more
Best for: Developers who need to compress and serve LLMs efficiently in production
mistral.rs
Community
Fast, flexible LLM inference
Alternative to Llama Cpp, Vllm, Sglang +2 more
Best for: Rust developers seeking a fast, flexible LLM inference framework for performance-critical or resource-constrained environments.
IntelliServer
Community
AI models as scalable microservices, enabling evaluation of LLMs and offering end-to-end functions such as chatbot, semantic search, image generation and beyond.
Alternative to Openllm
Best for: JavaScript developers who need a simple microservice wrapper for deploying and evaluating AI models