LiteLLM ๐
by Community
Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, Vertex
OSS
LiteLLM ๐
Added 1 June 2026
Overview
Python SDK and proxy server that abstracts 100+ LLM APIs behind a unified OpenAI-compatible interface. Handles cost tracking, request logging, load balancing, and guardrails across providers like Bedrock, Azure, Anthropic, VertexAI, and HuggingFace without rewriting application code.
Best for
Best for
Teams managing multiple LLM providers or needing cost visibility across model calls
Use cases
- Switch between LLM providers without changing application code
- Track token costs and usage across multiple models in production
- Distribute requests across models for load balancing and fallback
Notes
Python SDK and proxy server that abstracts 100+ LLM APIs behind a unified OpenAI-compatible interface. Handles cost tracking, request logging, load balancing, and guardrails across providers like Bedrock, Azure, Anthropic, VertexAI, and HuggingFace without rewriting application code.
48,950 stars on GitHub. Last updated 2026-06-01.
Use cases
- Switch between LLM providers without changing application code
- Track token costs and usage across multiple models in production
- Distribute requests across models for load balancing and fallback
Pros
- Supports 100+ models with standardized API interface
- Built-in cost tracking and logging for observability
- Can run as proxy server or Python SDK for flexible deployment
Cons
- Adds latency layer between application and LLM endpoints
- Requires maintenance as new model APIs and breaking changes emerge
- Community-maintained project with no commercial support guarantee
Indexed from awesome-llmops and enriched against its public facts.
Pros
- Supports 100+ models with standardized API interface
- Built-in cost tracking and logging for observability
- Can run as proxy server or Python SDK for flexible deployment
Cons
- Adds latency layer between application and LLM endpoints
- Requires maintenance as new model APIs and breaking changes emerge
- Community-maintained project with no commercial support guarantee
Open-source & AI alternatives
Swap-in tools that solve the same job. Weigh the trade-offs before you commit.
JamesANZ/cross-llm-mcp
Various
A Model Context Protocol (MCP) server that provides access to multiple Large Language Model (LLM) APIs including ChatGPT, Claude, Gemini, Mistral, Kimi K2, and DeepSeek.
sachinuppal/modelcostsaver
Various
Offline MCP server that predicts LLM call cost and recommends the cheapest capable model before you call it. No API keys, no network. npx @workswarm/modelcostsaver
smigolsmigol/llmkit
Various
Know what your AI agents cost. API gateway with budget enforcement, session tracking, and MCP tools.
ypollak2/llm-router
Various
Universal LLM router for AI coding tools. Works with Claude Code, Cursor, Codex, Gemini CLI, Copilot and more. Free-first fallback chain keeps costs 70โ85% lower.
spanlens/Spanlens
Various
Open source LLM observability and monitoring. Drop-in proxy for OpenAI, Anthropic, and Gemini with request logging, cost tracking, and agent tracing. Self-host with one Docker comm
AI Gateway
Community
A blazing fast AI Gateway with integrated guardrails. Route to 1,600+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.
AI studio
Community
[deprecated] AI Gateway - core infrastructure stack for building production-ready AI Applications
Bifrost
Community
Fastest enterprise AI gateway (50x faster than LiteLLM) with adaptive load balancer, cluster mode, guardrails, 1000+ models support & <100 ยตs overhead at 5k RPS.
Entroly
Community
Local proxy that cuts your Claude / OpenAI / Gemini bill 70%+. Drop-in for Cursor, Claude Code, Codex, Aider โ 30 seconds, no code changes.
Glide
Community
๐ฆ A open blazing-fast simple model gateway for rapid development of production GenAI apps
Hypersigil
Community
Prompt management gateway with a UI for AI-powered applications. Enables non-technical users to test, refine, and deploy prompts seamlessly across multiple AI providers.
Maxim AI
Community
At Maxim AI, we are building the production infrastructure for AI. Maximโs stack comprising gateway and governance, observability, and evals empowers AI teams to ship agents with
Mirascope
Community
The LLM Anti-Framework
Modelz-LLM
Community
OpenAI compatible API for LLMs and embeddings (LLaMA, Vicuna, ChatGLM and many others)
ReliableGPT ๐ช
Community
Handle OpenAI Errors (overloaded OpenAI servers, rotated keys, or context window errors) for your production LLM Applications.
TeamoRouter
Community
Use one API key for Claude Code, Codex, and AI coding agents. TeamoRouter helps developers reduce token costs, simplify provider setup, and pay only for usage.
TrueFoundry
Community
TrueFoundry offers an enterprise-grade AI Gateway combining LLM, MCP, and Agent Gatewaysโempowering businesses to connect, monitor, and govern agentic AI applications across prov
Vercel AI Gateway
Vercel
One API key, hundreds of LLM models. Unified endpoint with zero token markup, fallbacks, and spend monitoring.
Manifest
Various
Route every request to the most cost effective model and save up to 70% on AI tokens. Track cost and set usage limits.
OpenRouter
Various
The unified interface for LLMs. Find the best models & prices for your prompts
Pairs with
Other entries in the index that connect to this one. Click through to see the chain.
vLLM
Community
A high-throughput and memory-efficient inference and serving engine for LLMs
ollama
Community
Get up and running with Kimi-K2.5, GLM-5, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
LangChain
Community
The agent engineering platform.
Open WebUI
Various
User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
OpenHands
All Hands AI
Open-source autonomous coding agent platform. Spins up a sandboxed dev environment, ships PRs end to end.
legolev/mediamcp
Various
๐จ MCP server for AI media generation โ create and edit images, generate video via OpenRouter or any OpenAI-compatible API
zhaoyue722/llm-usage-mcp
Various
a local-first, multi-provider tool that captures LLM API spend and exposes it to coding agents via the Model Context Protocol
Agent-LLM
Community
AGiXT is a dynamic AI Agent Automation Platform that seamlessly orchestrates instruction management and complex task execution across diverse AI providers. Combining adaptive memor
Casibase
Community
โก๏ธnext-generation personal AI assistant powered by LLM, RAG and agent loops, supporting computer-use, browser-use and coding agent, demo: https://demo.openagentai.org
DemoGPT
Community
๐ค Create LLM agents in a second with your prompts. Everything you need to create an LLM Agent - tools, prompts, frameworks, and models - all in one place.
Embedbase
Community
A dead-simple API to build LLM-powered apps
Intelli
Community
Build multi-model chatbots and agents from intent.
Langroid
Community
Harness LLMs with Multi-Agent Programming
Manag.ai
Community
Your all-in-one prompt management and observability platform. Craft, track, and perfect your LLM prompts with ease.
MicroAgent
Community
Agents Capable of Self-Editing Their Prompts / Python Code
PraisonAI
Community
PraisonAI ๐ฆ โ Hire a 24/7 AI Workforce. Stop writing boilerplate and start shipping autonomous self-improving agents that research, plan, code, and execute tasks. Deployed in 5 li
R2R
Community
SoTA production-ready AI retrieval system. Agentic Retrieval-Augmented Generation (RAG) with a RESTful API.
Rhesis
Community
The testing platform for AI teams. Bring engineers, PMs, and domain experts together to generate tests, simulate (adversarial) conversations, and trace every failure to its root ca
Rigging
Community
Lightweight LLM Interaction Framework
SwarmClaw
Community
Open-source self-hosted AI agent runtime and multi-agent framework for autonomous agent swarms. Agent memory, MCP tools, schedules, delegation, and 23+ LLM providers (Claude, GPT,
TensorZero
Community
TensorZero builds open-source tools for production-grade LLM applications: LLM gateway, observability, optimization, evaluations, and experimentation.
Acacian/aegis
Various
LLM guardrails & prompt injection detection for Python. Auto-instruments LangChain, CrewAI, OpenAI, LiteLLM + 8 more frameworks. PII masking, toxicity detection, policy CI/CD. One
nikhilnt1234/TokenBurnRate
Various
[](https://glama.ai/mcp/servers/nikhilnt1234/TokenBurnRate) ๐ ๐ - Track LLM token costs across Claude, GPT and Gemini. MCP server + CLI with optimization hints and $ savings esti
szp2005/llm-prices-cn
Various
Daily-verified LLM API pricing dataset (44+ models, CN & global) with a hosted MCP server for live price queries and token cost estimation.
ai-evaluation
Community
Evaluation Framework for all your AI related Workflows
Azure OpenAI Logger
Community
"Batteries included" logging solution for your Azure OpenAI instance.
Continue
Community
โฉ Source-controlled AI checks, enforceable in CI. Powered by the open-source Continue CLI
dolly
Community
Databricksโ Dolly, a large language model trained on the Databricks Machine Learning Platform
Falcon 40B
Community
Weโre on a journey to advance and democratize artificial intelligence through open source and open science.
Helicone
Community
๐ง Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 ๐
Keywords AI
Community
Unify observability, evals, prompt optimization, and your LLM gateway in one platform.
Langfuse
Langfuse
Open-source LLM observability. Traces, evals, prompt management, all self-hostable.
LangKit
Community
๐ LangKit: An open-source toolkit for monitoring Large Language Models (LLMs). ๐ Extracts signals from prompts & responses, ensuring safety & security. ๐ก๏ธ Features include text
Literal AI
Community
Multi-modal LLM observability and evaluation platform. Create prompt templates, deploy prompts versions, debug LLM runs, create datasets, run evaluations, monitor LLM metrics and c
llm-ui
Community
The React library for LLMs
Lunary
Community
Observability and prompt management for LLM chabots and agents. Debug agents with powerful tracing and logging. Usage analytics and dive deep into the history of your requests. Dev
onWatch
Community
Track AI API quotas across Synthetic, Z.ai, Anthropic (Claude Code), Codex, GitHub Copilot & Antigravity in real time. Lightweight background daemon (<50MB RAM), SQLite storage, Ma
Open Responses
Community
OpenAI o3-mini
Community
Pushing the frontier of cost-effective reasoning.
OpenLIT
Community
Open source platform for AI Engineering: OpenTelemetry-native LLM Observability, GPU Monitoring, Guardrails, Evaluations, Prompt Management, Vault, Playground. ๐๐ป Integrates with
Parea AI
Community
The experimentation and human annotation platform for AI teams.
Pezzo ๐น๏ธ
Community
๐น๏ธ Open-source, developer-first LLMOps platform designed to streamline prompt design, version management, instant delivery, collaboration, troubleshooting, observability and more.
Plexiglass
Community
A toolkit for detecting and protecting against vulnerabilities in Large Language Models (LLMs).
Portkey
Community
Democratize and productionize Gen AI across your entire org with Portkey
PromptMage
Community
simplifies the process of creating and managing LLM workflows.
Puzzlet AI
Community
Redirecting...
Rapid-MLX
Community
The fastest local AI engine for Apple Silicon. 4.2x faster than Ollama, 0.08s cached TTFT, 100% tool calling. 17 tool parsers, prompt cache, reasoning separation, cloud routing. Dr
Semantic Cache Router
Community
Distributed semantic cache and stateful routing system that cuts LLM API costs by returning cached responses for semantically similar queries. Uses ANN vector search (cosine โฅ 0.8)
systemprompt.io
Community
The governance layer for AI agents. A single compiled Rust binary that authenticates, authorises, rate-limits, logs, and costs every AI interaction. Self-hosted, air-gap capable,
text-generation-inference
Community
Large Language Model Text Generation Inference
Vellum
Community
An assistant that knows you deeply, evolves alongside you, and belongs to no one else. Powered by memory that remembers the way you do.
Cohere
Various
Cohere builds powerful models and AI solutions enabling enterprises to automate processes, empower employees, and turn fragmented data into actionable insights.
Helicone AI
Various
AI Gateway & LLM Observability
LLM Stats
Various
The LLM Leaderboard โ independent ranking of GPT, Claude, Gemini, Llama, DeepSeek and 300+ AI models by intelligence, speed and price. Composite LLM Stats Score updated continuou
MiniMax
Various
Building AGI with our mission Intelligence with Everyone. Global leader in multi-modal models and AI-native products with over 200 million users.
Get the free Developerโs Field Guide
A 27-page field guide to the AI coding workflow with Claude. Claude Code, MCP servers, the prompt patterns that work, and what to delegate. Free.
Enter your work email. We send it straight over, plus a few short notes worth knowing. Unsubscribe any time.