LiteLLM 🚅

The agent engineering platform.

A high-throughput and memory-efficient inference and serving engine for LLMs

Get up and running with Kimi-K2.5, GLM-5, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.

LLM inference in C/C++

Fastest enterprise AI gateway (50x faster than LiteLLM) with adaptive load balancer, cluster mode, guardrails, 1000+ models support & <100 µs overhead at 5k RPS.

Build agents which are controlled by LLMs

PraisonAI 🦞 — Hire a 24/7 AI Workforce. Stop writing boilerplate and start shipping autonomous self-improving agents that research, plan, code, and execute tasks. Deployed in 5 li

Fastest enterprise AI gateway (50x faster than LiteLLM) with adaptive load balancer, cluster mode, guardrails, 1000+ models support & <100 µs overhead at 5k RPS.

Evaluation Framework for all your AI related Workflows

"Batteries included" logging solution for your Azure OpenAI instance.

⏩ Source-controlled AI checks, enforceable in CI. Powered by the open-source Continue CLI

Fiddler Auditor is a tool to evaluate language models.

Production-grade AI evaluation, prompt management & observability SDK. Automated evaluations with sub-100ms guardrails. No human-in-the-loop required. Python + TypeScript.

Checking your browser - reCAPTCHA

🐦 A open blazing-fast simple model gateway for rapid development of production GenAI apps

Learn about Guardrails AI and how it helps build reliable AI applications

🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

Multi-Agent Harness for Production AI

Humanloop is joining Anthropic to accelerate the adoption of AI, safely.

Structured output for LLMs via Pydantic. The cleanest answer to 'just give me a typed object back'.

Unify observability, evals, prompt optimization, and your LLM gateway in one platform.

Laminar - open-source observability platform purpose-built for AI agents. YC S24.

Open-source LLM observability. Traces, evals, prompt management, all self-hostable.

🔍 LangKit: An open-source toolkit for monitoring Large Language Models (LLMs). 📚 Extracts signals from prompts & responses, ensuring safety & security. 🛡️ Features include text

Multi-modal LLM observability and evaluation platform. Create prompt templates, deploy prompts versions, debug LLM runs, create datasets, run evaluations, monitor LLM metrics and c

Observability and prompt management for LLM chabots and agents. Debug agents with powerful tracing and logging. Usage analytics and dive deep into the history of your requests. Dev

Your all-in-one prompt management and observability platform. Craft, track, and perfect your LLM prompts with ease.

At Maxim AI, we are building the production infrastructure for AI. Maxim’s stack comprising gateway and governance, observability, and evals empowers AI teams to ship agents with

MLflow - Open Source AI Platform for Agents, LLMs & Models


Open source platform for AI Engineering: OpenTelemetry-native LLM Observability, GPU Monitoring, Guardrails, Evaluations, Prompt Management, Vault, Playground. 🚀💻 Integrates with

Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)

The experimentation and human annotation platform for AI teams.

🕹️ Open-source, developer-first LLMOps platform designed to streamline prompt design, version management, instant delivery, collaboration, troubleshooting, observability and more.

A toolkit for detecting and protecting against vulnerabilities in Large Language Models (LLMs).

Democratize and productionize Gen AI across your entire org with Portkey

Version, test, and monitor every prompt and agent with robust evals, tracing, and regression sets. Empower domain experts to collaborate in the visual editor.

Redirecting...

The testing platform for AI teams. Bring engineers, PMs, and domain experts together to generate tests, simulate (adversarial) conversations, and trace every failure to its root ca

Distributed semantic cache and stateful routing system that cuts LLM API costs by returning cached responses for semantically similar queries. Uses ANN vector search (cosine ≥ 0.8)

⚡ Python-free Rust inference server — OpenAI-API compatible. GGUF + SafeTensors, hot model swap, auto-discovery, single binary. FREE now, FREE forever.

Open Source AI Tracing Framework built on Opentelemetry for AI Applications and Frameworks

An assistant that knows you deeply, evolves alongside you, and belongs to no one else. Powered by memory that remembers the way you do.

A blazing fast AI Gateway with integrated guardrails. Route to 1,600+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.

[deprecated] AI Gateway - core infrastructure stack for building production-ready AI Applications

LangChain chat model abstractions for dynamic failover, load balancing, chaos engineering, and more!

Local proxy that cuts your Claude / OpenAI / Gemini bill 70%+. Drop-in for Cursor, Claude Code, Codex, Aider — 30 seconds, no code changes.

Prompt management gateway with a UI for AI-powered applications. Enables non-technical users to test, refine, and deploy prompts seamlessly across multiple AI providers.

The LLM Anti-Framework

Handle OpenAI Errors (overloaded OpenAI servers, rotated keys, or context window errors) for your production LLM Applications.

The governance layer for AI agents. A single compiled Rust binary that authenticates, authorises, rate-limits, logs, and costs every AI interaction. Self-hosted, air-gap capable,

Use one API key for Claude Code, Codex, and AI coding agents. TeamoRouter helps developers reduce token costs, simplify provider setup, and pay only for usage.

TrueFoundry offers an enterprise-grade AI Gateway combining LLM, MCP, and Agent Gateways—empowering businesses to connect, monitor, and govern agentic AI applications across prov

One API key, hundreds of LLM models. Unified endpoint with zero token markup, fallbacks, and spend monitoring.