vLLM

Tensors and Dynamic neural networks in Python with strong GPU acceleration

Tensors and Dynamic neural networks in Python with strong GPU acceleration

The agent engineering platform.

Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, Vertex

SGLang is a high-performance serving framework for large language models and multimodal models.

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NV

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

LLM inference in C/C++

The most powerful AI platform for enterprises. Customize, fine-tune, and deploy AI assistants, autonomous agents, and multimodal AI with open models.

First-generation reasoning models from DeepSeek.

Production-ready platform for agentic workflow development.

An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

A GPU cluster manager that configures and orchestrates inference engines like vLLM and SGLang for high-performance AI model deployment.

Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes

Kubernetes operator for local LLM inference with llama.cpp, vLLM, TGI, and mlx-server — multi-GPU NVIDIA + Apple Silicon Metal, autoscaling, air-gapped, production-ready

A framework for few-shot evaluation of language models.

🌟 The Multi-Agent Framework: First AI Software Company, Towards Natural Language Programming

OpenAI compatible API for LLMs and embeddings (LLaMA, Vicuna, ChatGLM and many others)

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)

An Easy-to-use, Scalable and High-performance Agentic RL Framework based on Ray (PPO & DAPO & REINFORCE++ & VLM & TIS & vLLM & Ray & Async RL)

DEMO PAPER GITHUB HUGGING FACE MODELSCOPE DISCORD To achieve the objective of building an AGI system, the model should be capable of understanding information from different moda

RayLLM - LLMs on Ray (Archived). Read README for more info.

verl/HybridFlow: A Flexible and Efficient RL Post-Training Framework

Microsoft's framework for multi-agent conversations. Agents that talk to each other to solve hard problems.

📖A curated list of Awesome LLM/VLM Inference Papers with codes: WINT8/4, FlashAttention, PagedAttention, MLA, Parallelism, etc. 🎉🎉

AGI Large Language Models

DeepSeek Math series

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

DeepSeek-VL model series

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Fiddler Auditor is a tool to evaluate language models.

Gemma 2, our next generation of open models, is now available globally for researchers and developers.

Leaked Internal Google Document Claims Open Source AI Will Outcompete Google and OpenAI

A GPU cluster manager that configures and orchestrates inference engines like vLLM and SGLang for high-performance AI model deployment.

Grok-1-314B-MoE — indexed from awesome-llm

A guidance language for controlling large language models.

Create agentic, context engineered AI systems using Haystack’s modular and customizable building blocks, built for real-world, production-ready applications.

Open source project for data preparation for GenAI applications

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Kimi K2 is the large language model series developed by Moonshot AI team

Langchain-Chatchat（原Langchain-ChatGLM）基于 Langchain 与 ChatGLM, Qwen 与 Llama 等语言模型的 RAG 与 Agent 应用 | Langchain-Chatchat (formerly langchain-ChatGLM), local knowledge based LLM (like

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

[Llama 3.1-8 70 405B](https://llama.meta.com/)

[Llama 2-7 13 70B](https://llama.meta.com/llama2/)

Ongoing research training transformer language models at scale, including: BERT & GPT-2

A simple, performant and scalable Jax LLM!

[NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to

The most powerful AI platform for enterprises. Customize, fine-tune, and deploy AI assistants, autonomous agents, and multimodal AI with open models.

MLflow - Open Source AI Platform for Agents, LLMs & Models

Moonshot's Compute-efficient MoE LLM, first Scaling Up of Muon Optimizer

A high-performance ML model serving framework, offers dynamic batching and CPU/GPU pipelines to fully exploit your compute machine

Introducing MPT-7B, the first entry in our MosaicML Foundation Series. MPT-7B is a transformer trained from scratch on 1T tokens of text and code. It is open source, available fo

A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Artifacts for the first set of OLMo models.


Structured Outputs

Redirecting...

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

GITHUB HUGGING FACE MODELSCOPE DEMO DISCORD Introduction After months of efforts, we are pleased to announce the evolution from Qwen1.5 to Qwen2. This time, we bring to you: Pret

Qwen - a Qwen Collection

GITHUB HUGGING FACE MODELSCOPE DISCORD 🚨 This model mainly supports English. We will release bilingual (English and Chinese) math models soon. Introduction Over the past year, w

Open weights language model from Google DeepMind, based on Griffin.

Run, manage, and scale AI workloads on any AI infrastructure. Use one system to access & manage all AI compute (Kubernetes, Slurm, 20+ clouds, on-prem).

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

All models, datasets, and demos related to StarCoder!

TensorZero builds open-source tools for production-grade LLM applications: LLM gateway, observability, optimization, evaluations, and experimentation.

PyTorch native post-training library

A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit and 4-bit floating point (FP8 and FP4) precision on Hopper, Ada and Blackwell GPUs, to provide b

Playground for devs to finetune & deploy LLMs

verl/HybridFlow: A Flexible and Efficient RL Post-Training Framework

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Transformer related optimization, including BERT, GPT

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

Fast, flexible LLM inference

SGLang is a high-performance serving framework for large language models and multimodal models.

⚡ Python-free Rust inference server — OpenAI-API compatible. GGUF + SafeTensors, hot model swap, auto-discovery, single binary. FREE now, FREE forever.

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NV

A blazing fast inference solution for text embeddings models

Large Language Model Text Generation Inference

We’re on a journey to advance and democratize artificial intelligence through open source and open science.