OpenLLM

Overview

OpenLLM provides a framework to run any open-source large language model, such as DeepSeek and Llama, as an OpenAI-compatible API endpoint. It handles model serving in cloud environments and is built in Python.

Best for

Best for
Developers who need to serve open-source LLMs with OpenAI API compatibility

Use cases

• Deploy open-source LLMs as drop-in replacements for OpenAI endpoints
• Serve multiple open-source models behind a unified API
• Experiment with different LLMs locally or in the cloud

Notes

OpenLLM provides a framework to run any open-source large language model, such as DeepSeek and Llama, as an OpenAI-compatible API endpoint. It handles model serving in cloud environments and is built in Python.

12,346 stars on GitHub. Last updated 2026-06-01. Licensed Apache-2.0.

Use cases

• Deploy open-source LLMs as drop-in replacements for OpenAI endpoints
• Serve multiple open-source models behind a unified API
• Experiment with different LLMs locally or in the cloud

Pros

• OpenAI-compatible API simplifies integration with existing applications
• Supports a wide range of popular open-source models out of the box
• Active community with over 12,000 GitHub stars

Cons

• Requires manual cloud infrastructure setup or management
• Model performance is heavily dependent on the underlying hardware
• May not support all model architectures or custom optimizations

Indexed from awesome-llm and enriched against its public facts.