Open source alternatives to OpenLLM

Open source alternatives to OpenLLM, ranked by GitHub stars and freshness.

3 open-source alternatives in the index, ranked by GitHub stars and freshness.

O OSS Framework medium

vLLM

Community

A high-throughput and memory-efficient inference and serving engine for LLMs

★ 81,619 updated 1mo ago

open-source

Best for: Teams building production LLM APIs and services that need to maximize throughput and minimize latency under concurrent load.

O OSS Framework medium

IntelliServer

Community

AI models as scalable microservices, enabling evaluation of LLMs and offering end-to-end functions such as chatbot, semantic search, image generation and beyond.

★ 29 updated 1y ago

open-source

Best for: JavaScript developers who need a simple microservice wrapper for deploying and evaluating AI models

O OSS Framework medium

Wallaroo.AI

Community

Deploy, manage, optimize any model at scale across any environment from cloud to edge. Let's you go from python notebook to inferencing in minutes.

open-source

Best for: Teams needing fast, scalable model deployment across diverse infrastructure from cloud to edge.