vLLM

Tensors and Dynamic neural networks in Python with strong GPU acceleration

Tensors and Dynamic neural networks in Python with strong GPU acceleration

The agent engineering platform.

Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, Vertex

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NV

SGLang is a high-performance serving framework for large language models and multimodal models.

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

BigScience

Distilabel is a framework for synthetic data and AI feedback for engineers who need fast, reliable and scalable pipelines based on verified research papers.

A GPU cluster manager that configures and orchestrates inference engines like vLLM and SGLang for high-performance AI model deployment.

The agent engineering platform.

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

Kubernetes operator for local LLM inference with llama.cpp, vLLM, TGI, and mlx-server — multi-GPU NVIDIA + Apple Silicon Metal, autoscaling, air-gapped, production-ready

A framework for few-shot evaluation of language models.

Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

An Easy-to-use, Scalable and High-performance Agentic RL Framework based on Ray (PPO & DAPO & REINFORCE++ & VLM & TIS & vLLM & Ray & Async RL)

Structured Outputs

GITHUB HUGGING FACE MODELSCOPE DISCORD 🚨 This model mainly supports English. We will release bilingual (English and Chinese) math models soon. Introduction Over the past year, w

Playground for devs to finetune & deploy LLMs

Awesome LLM compression research papers and tools.

📖A curated list of Awesome LLM/VLM Inference Papers with codes: WINT8/4, FlashAttention, PagedAttention, MLA, Parallelism, etc. 🎉🎉

A collection of open source, actively maintained web apps for LLM applications

Go ahead and axolotl questions

AGI Large Language Models

Fastest enterprise AI gateway (50x faster than LiteLLM) with adaptive load balancer, cluster mode, guardrails, 1000+ models support & <100 µs overhead at 5k RPS.

BigScience

GITHUB HUGGING FACE MODELSCOPE DEMO DISCORD Introduction The advent of advanced programming tools, which harnesses the power of large language models (LLMs), has significantly en

The most powerful AI platform for enterprises. Customize, fine-tune, and deploy AI assistants, autonomous agents, and multimodal AI with open models.

DeepSeek Math series

First-generation reasoning models from DeepSeek.

We present DeepSeek-V2, a strong Mixture-of-Experts (MoE) language model characterized by economical training and efficient inference. It comprises 236B total parameters, of whic

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

DeepSeek-VL model series

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Fiddler Auditor is a tool to evaluate language models.

Open source, high performance fine tuning as a service for GPT4 quality models with 5x lower latency and 3x lower cost

Checking your browser - reCAPTCHA

Gemma 2, our next generation of open models, is now available globally for researchers and developers.

GLM-130B

Org profile for THUDM on Hugging Face, the AI community building the future.

Grok-1-314B-MoE — indexed from awesome-llm

Create agentic, context engineered AI systems using Haystack’s modular and customizable building blocks, built for real-world, production-ready applications.

Publications — Google DeepMind

Infinity is a high-throughput, low-latency serving engine for text-embeddings, reranking models, clip, clap and colpali

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

AI Inference Operator for Kubernetes. The easiest way to serve ML models in production. Supports VLMs, LLMs, embeddings, and speech-to-text.

[OPT-1.3 6.7 13 30 66B](https://arxiv.org/abs/2205.01068)

2023-07

[Llama 3.1-8 70 405B](https://llama.meta.com/)

[Llama 2-7 13 70B](https://llama.meta.com/llama2/)

2023-02

A simple, performant and scalable Jax LLM!

Meta Lingua: a lean, efficient, and easy-to-hack codebase to research LLMs.

The MiniCPM family of LLMs and VLLMs.

Mistral 7B

The most powerful AI platform for enterprises. Customize, fine-tune, and deploy AI assistants, autonomous agents, and multimodal AI with open models.

Moonshot's Compute-efficient MoE LLM, first Scaling Up of Muon Optimizer

Introducing MPT-7B, the first entry in our MosaicML Foundation Series. MPT-7B is a transformer trained from scratch on 1T tokens of text and code. It is open source, available fo

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Artifacts for the first set of OLMo models.

We introduce OLMoE, a fully open, state-of-the-art language model leveraging sparse Mixture-of-Experts (MoE). OLMoE-1B-7B has 7 billion (B) parameters but uses only 1B per input

Evaluation suite for LLMs

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Qwen - a Qwen Collection

GITHUB HUGGING FACE MODELSCOPE DEMO DISCORD Introduction After months of efforts, we are pleased to announce the evolution from Qwen1.5 to Qwen2. This time, we bring to you: Pret

Tech Report HuggingFace ModelScope Qwen Chat HuggingFace Demo ModelScope Demo DISCORD Introduction Two months after upgrading Qwen2.5-Turbo to support context length up to one mi

In this report, we introduce Qwen2.5, a comprehensive series of large language models (LLMs) designed to meet diverse needs. Compared to previous iterations, Qwen 2.5 has been si

QWEN CHAT API DEMO DISCORD It is widely recognized that continuously scaling both data size and model size can lead to significant improvements in model intelligence. However, th

RayLLM - LLMs on Ray (Archived). Read README for more info.

Microsoft's enterprise-flavoured framework for AI agents. .NET-first, with Python and Java siblings.

Run, manage, and scale AI workloads on any AI infrastructure. Use one system to access & manage all AI compute (Kubernetes, Slurm, 20+ clouds, on-prem).

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

All models, datasets, and demos related to StarCoder!

Modern artificial intelligence (AI) systems are powered by foundation models. This paper presents a new set of foundation models, called Llama 3. It is a herd of language models

A PyTorch native platform for training generative AI models

A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit and 4-bit floating point (FP8 and FP4) precision on Hopper, Ada and Blackwell GPUs, to provide b

verl/HybridFlow: A Flexible and Efficient RL Post-Training Framework

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Megatron-LM

Transformer related optimization, including BERT, GPT

Running large language models on a single GPU for throughput-oriented scenarios.

AI models as scalable microservices, enabling evaluation of LLMs and offering end-to-end functions such as chatbot, semantic search, image generation and beyond.

LLM inference in C/C++

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

Fast, flexible LLM inference

Get up and running with Kimi-K2.5, GLM-5, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.

RayLLM - LLMs on Ray (Archived). Read README for more info.

SGLang is a high-performance serving framework for large language models and multimodal models.

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NV

Large Language Model Text Generation Inference

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

The Triton Inference Server provides an optimized cloud and edge inferencing solution.