Open Source
vLLM
A high-throughput and memory-efficient inference and serving engine for LLMs
Our verdict
We have not reviewed vLLM yet
No one at Enterprise DNA has checked this listing against its sources. The description comes from public listings gathered automatically. Any fact below shows where it came from and when it was last checked.
From the listing
About vLLM
vLLM is a Python framework for serving large language models with optimized throughput and memory efficiency. It uses techniques like paged attention and continuous batching to reduce latency and increase request throughput compared to standard inference servers. Designed for production deployments that need to handle multiple concurrent requests.
Best for
Teams building production LLM APIs and services that need to maximize throughput and minimize latency under concurrent load.
Use cases
- Running inference servers that handle high request volume with low latency
- Reducing GPU memory footprint when serving large models
- Batching and scheduling inference requests efficiently
Notes
81,619 stars on GitHub. Last updated 2026-06-01. Licensed Apache-2.0.
Pros
- Significantly higher throughput than standard LLM serving approaches
- Lower memory consumption enables serving larger models on same hardware
- Active community with 81k+ GitHub stars and ongoing development
Cons
- Requires Python and GPU infrastructure, not suitable for CPU-only deployments
- Steeper learning curve than simple inference libraries for basic use cases
- Performance gains depend on workload characteristics and batch patterns
Key facts
What we have checked
- Latest version
- 0.32.0 (released 18 Oct 2026)
- GitHub stars
- 92,700
Each value shows when we last confirmed it and where it came from. Values we have not confirmed are left out, never guessed.
Stars and usage
How vLLM is growing
81,619 to 92,700
Alternatives
Alternatives to vLLM
| Record | Type | How we know | Checked |
|---|---|---|---|
| exllamaAlternative to this | Open source | Named in the listing | Not dated |
| FastChatAlternative to this | Open source | Named in the listing | Not dated |
| FasterTransformerAlternative to this | Open source | Named in the listing | Not dated |
| FlexGenAlternative to this | Open source | Named in the listing | Not dated |
| InfinityAlternative to this | Open source | Named in the listing | Not dated |
| Llama 3-8|70BAlternative to this | Open source | Named in the listing | Not dated |
| llama.cppAlternative to this | Open source | Named in the listing | Not dated |
| LMDeployAlternative to this | Open source | Named in the listing | Not dated |
| MInferenceAlternative to this | Open source | Named in the listing | Not dated |
| mistral.rsAlternative to this | Open source | Named in the listing | Not dated |
| OpenLLMAlternative to this | Open source | Named in the listing | Not dated |
| Rapid-MLXAlternative to this | Open source | Named in the listing | Not dated |
| ray-llmAlternative to this | Open source | Named in the listing | Not dated |
| SGLangAlternative to this | Open source | Named in the listing | Not dated |
| ShimmyAlternative to this | Open source | Named in the listing | Not dated |
| TensorRT-LLMAlternative to this | Open source | Named in the listing | Not dated |
| Text-Embeddings-InferenceAlternative to this | Open source | Named in the listing | Not dated |
| text-generation-inferenceAlternative to this | Open source | Named in the listing | Not dated |
| TGIAlternative to this | Open source | Named in the listing | Not dated |
| Triton Server (TRTIS)Alternative to this | Open source | Named in the listing | Not dated |
Compare
Compare vLLM side by side
Sources
Where this comes from
- GitHubOfficial
- github api via nightly crawl, recovered from git history 20e874270887c1008f860ee0198e461a23c9e438Registry
A source shows what was published when we checked it. It does not certify security, performance or a working installation. Read our method.
Changelog
What changed
Released 0.32.0
Released 0.31.0
Released 0.30.0
Released 0.29.0
Released 0.28.0
Released 0.27.1
Released 0.27.0
Released 0.26.0
Released 0.25.1
Released 0.25.0
Open-source & AI alternatives
Swap-in tools that solve the same job. Weigh the trade-offs before you commit.
LMDeploy
Community
LMDeploy is a toolkit for compressing, deploying, and serving LLMs.
OpenLLM
Community
Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
SGLang
Community
SGLang is a high-performance serving framework for large language models and multimodal models.
TensorRT-LLM
Community
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NV
exllama
Community
A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.
FastChat
Community
An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
FasterTransformer
Community
Transformer related optimization, including BERT, GPT
FlexGen
Community
Running large language models on a single GPU for throughput-oriented scenarios.
Infinity
Community
Infinity is a high-throughput, low-latency serving engine for text-embeddings, reranking models, clip, clap and colpali
Llama 3-8|70B
Community
Llama 2-7 13 70B
llama.cpp
Community
LLM inference in C/C++
LMDeploy
Community
LMDeploy is a toolkit for compressing, deploying, and serving LLMs.
MInference
Community
[NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to
mistral.rs
Community
Fast, flexible LLM inference
OpenLLM
Community
Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
Rapid-MLX
Community
The fastest local AI engine for Apple Silicon. 4.2x faster than Ollama, 0.08s cached TTFT, 100% tool calling. 17 tool parsers, prompt cache, reasoning separation, cloud routing. Dr
Pairs with
Other entries in the index that connect to this one. Click through to see the chain.
DeepSeek
Various
Org profile for DeepSeek on Hugging Face, the AI community building the future.
Grok
Various
An LLM by xAI with open source and open weights. #opensource
LibreChat
Various
LibreChat brings together all your AI conversations in one unified, customizable interface.
LLaMA
Various
Llama LLM, a foundational, 65-billion-parameter large language model by Meta. Meta, February 23rd, 2023. #opensource
LLM Stats
Various
The LLM Leaderboard — independent ranking of GPT, Claude, Gemini, Llama, DeepSeek and 300+ AI models by intelligence, speed and price. Composite LLM Stats Score updated continuou
NeMo
Various
A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech
Open LLMs
Various
📋 A list of open LLMs available for commercial use.
Qwen
Various
Qwickly forging AGI, enhancing intelligence.
Jwrede/llmprobe
Various
Synthetic monitoring and CI smoke tests for LLM inference endpoints.
Agency
Community
🕵️♂️ Library designed for developers eager to explore the potential of Large Language Models (LLMs) and other generative AI through a clean, effective, and Go-idiomatic approach.
AgentField
Community
Build, run and scale AI agents like API and microservices - observable,auditable and identity-aware from day one.
ai-evaluation
Community
Evaluation Framework for all your AI related Workflows
Forefront
Various
Forefront is a platform to fine-tune and inference open-source-language-models.
Harbor
Various
Stop configuring your AI stack. Start using it. One command brings a complete pre-wired LLM stack with hundreds of services to explore.
Together AI
Various
Build what's next on the AI Native Cloud. Full-stack AI platform for inference, fine-tuning, and GPU clusters, powered by cutting-edge research.
inference.sh
ac.inference.sh
Run 150+ AI apps — image, video, audio, LLMs, 3D and more. Browse, execute, stream results.
setheerwagen/mcp-modelmanager
Various
Two Model Context Protocol (MCP) servers for self-hosted, local AI: manage and train Ollama/vLLM models, and route questions and tasks to local LLMs through Claude, ChatGPT or loca
CodeQwen1.5-7B
Community
GITHUB HUGGING FACE MODELSCOPE DEMO DISCORD Introduction The advent of advanced programming tools, which harnesses the power of large language models (LLMs), has significantly en
Command R-35B
Community
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Dify
Dify
Build applications, agents and document-backed chat workflows through a visual platform.
distilabel
Community
Distilabel is a framework for synthetic data and AI feedback for engineers who need fast, reliable and scalable pipelines based on verified research papers.
DSPy
Stanford NLP
Programming, not prompting. Declare what you want, compile prompts and weights against an objective.
Falcon 40B
Community
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
FlagAI
Community
FlagAI (Fast LArge-scale General AI models) is a fast, easy-to-use and extensible toolkit for large-scale model.
Get the free Developer’s Field Guide
A 27-page field guide to the AI coding workflow with Claude. Claude Code, MCP servers, the prompt patterns that work, and what to delegate. Free.
Enter your work email. We send it straight over, plus a few short notes worth knowing. Unsubscribe any time.