LiteLLM 🚅

LLM inference in C/C++

⏩ Source-controlled AI checks, enforceable in CI. Powered by the open-source Continue CLI

An autonomous agent that conducts deep research on any data using any LLM providers

Prompt management gateway with a UI for AI-powered applications. Enables non-technical users to test, refine, and deploy prompts seamlessly across multiple AI providers.

Version, test, and monitor every prompt and agent with robust evals, tracing, and regression sets. Empower domain experts to collaborate in the visual editor.

Open-source self-hosted AI agent runtime and multi-agent framework for autonomous agent swarms. Agent memory, MCP tools, schedules, delegation, and 23+ LLM providers (Claude, GPT,

The AI Toolkit for TypeScript. From the creators of Next.js, the AI SDK is a free open-source library for building AI-powered applications and agents

Open source, high performance fine tuning as a service for GPT4 quality models with 5x lower latency and 3x lower cost

⛓️ build cognitive systems, pythonic

Production-grade AI evaluation, prompt management & observability SDK. Automated evaluations with sub-100ms guardrails. No human-in-the-loop required. Python + TypeScript.

Learn about Guardrails AI and how it helps build reliable AI applications

🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

Humanloop is joining Anthropic to accelerate the adoption of AI, safely.

Build multi-model chatbots and agents from intent.

Unify observability, evals, prompt optimization, and your LLM gateway in one platform.

Open-source LLM observability. Traces, evals, prompt management, all self-hostable.

🔍 LangKit: An open-source toolkit for monitoring Large Language Models (LLMs). 📚 Extracts signals from prompts & responses, ensuring safety & security. 🛡️ Features include text

Multi-modal LLM observability and evaluation platform. Create prompt templates, deploy prompts versions, debug LLM runs, create datasets, run evaluations, monitor LLM metrics and c

Observability and prompt management for LLM chabots and agents. Debug agents with powerful tracing and logging. Usage analytics and dive deep into the history of your requests. Dev

Your all-in-one prompt management and observability platform. Craft, track, and perfect your LLM prompts with ease.

Track AI API quotas across Synthetic, Z.ai, Anthropic (Claude Code), Codex, GitHub Copilot & Antigravity in real time. Lightweight background daemon (<50MB RAM), SQLite storage, Ma


Open source platform for AI Engineering: OpenTelemetry-native LLM Observability, GPU Monitoring, Guardrails, Evaluations, Prompt Management, Vault, Playground. 🚀💻 Integrates with

The experimentation and human annotation platform for AI teams.

🕹️ Open-source, developer-first LLMOps platform designed to streamline prompt design, version management, instant delivery, collaboration, troubleshooting, observability and more.

Democratize and productionize Gen AI across your entire org with Portkey

Markdown for the AI era

Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative config

simplifies the process of creating and managing LLM workflows.

PromptSite is a lightweight prompt version management package that helps you version control, track, experiment and debug with your LLM prompts with ease.

Handle OpenAI Errors (overloaded OpenAI servers, rotated keys, or context window errors) for your production LLM Applications.

Distributed semantic cache and stateful routing system that cuts LLM API costs by returning cached responses for semantically similar queries. Uses ANN vector search (cosine ≥ 0.8)

Microsoft's enterprise-flavoured framework for AI agents. .NET-first, with Python and Java siblings.

⚡ Python-free Rust inference server — OpenAI-API compatible. GGUF + SafeTensors, hot model swap, auto-discovery, single binary. FREE now, FREE forever.

The governance layer for AI agents. A single compiled Rust binary that authenticates, authorises, rate-limits, logs, and costs every AI interaction. Self-hosted, air-gap capable,

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

An assistant that knows you deeply, evolves alongside you, and belongs to no one else. Powered by memory that remembers the way you do.

A high-throughput and memory-efficient inference and serving engine for LLMs

A blazing fast AI Gateway with integrated guardrails. Route to 1,600+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.

[deprecated] AI Gateway - core infrastructure stack for building production-ready AI Applications

🐦 A open blazing-fast simple model gateway for rapid development of production GenAI apps

The LLM Anti-Framework

OpenAI compatible API for LLMs and embeddings (LLaMA, Vicuna, ChatGLM and many others)

Use one API key for Claude Code, Codex, and AI coding agents. TeamoRouter helps developers reduce token costs, simplify provider setup, and pay only for usage.

TensorZero builds open-source tools for production-grade LLM applications: LLM gateway, observability, optimization, evaluations, and experimentation.

TrueFoundry offers an enterprise-grade AI Gateway combining LLM, MCP, and Agent Gateways—empowering businesses to connect, monitor, and govern agentic AI applications across prov

One API key, hundreds of LLM models. Unified endpoint with zero token markup, fallbacks, and spend monitoring.

The de facto TypeScript SDK for AI apps. Streaming, tools, multi-model, and now an agent loop.