PyTorch

Pretrain, finetune ANY AI model of ANY size on 1 or 10,000+ GPUs with zero code changes.

Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more

An Open Source Machine Learning Framework for Everyone

Caffe: a fast open framework for deep learning.

Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more

LLM as a Chatbot Service

Accelerate your Neural Architecture Search (NAS) through fast, reproducible and modular research.

AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head

Go ahead and axolotl questions

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

2018-10

Accessible large language models via k-bit quantization for PyTorch.

BigScience

How to implement LLM attention mechanisms and GPT-style transformers.

CodeGeeX: An Open Multilingual Code Generation Model (KDD 2023)

Examples of Machine Learning code using Comet.ml

Fast inference engine for Transformer models

DeepSeek-VL model series

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Determined is an open-source machine learning platform that simplifies distributed training, hyperparameter tuning, experiment tracking, and resource management. Works with PyTorch

[ACL 2024] An Easy-to-use Knowledge Editing Framework for LLMs.

An Industrial Grade Federated Learning Framework

Transformer related optimization, including BERT, GPT

Fiddler Auditor is a tool to evaluate language models.

A PyTorch Lightning extension that accelerates and enhances foundation model experimentation with flexible fine-tuning schedules.

A fast library for AutoML and tuning. Join our Discord: https://discord.gg/Cppx2vSPVP.

Running large language models on a single GPU for throughput-oriented scenarios.

Open source, high performance fine tuning as a service for GPT4 quality models with 5x lower latency and 3x lower cost

Holistic Evaluation of Language Models (HELM) is an open source Python framework created by the Center for Research on Foundation Models (CRFM) at Stanford for holistic, reproducib

Stanford

Collection of hyperparameter optimization benchmark problems

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Machine Learning Toolkit for Kubernetes

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

[Llama 2-7 13 70B](https://llama.meta.com/llama2/)

A framework for few-shot evaluation of language models.

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

Using Low-rank adaptation to quickly fine-tune diffusion models.

CMU&Princeton

Ongoing research training transformer language models at scale, including: BERT & GPT-2

Meta Lingua: a lean, efficient, and easy-to-hack codebase to research LLMs.

The MiniCPM family of LLMs and VLLMs.

MLflow - Open Source AI Platform for Agents, LLMs & Models


Moonshot's Compute-efficient MoE LLM, first Scaling Up of Muon Optimizer

Introducing MPT-7B, the first entry in our MosaicML Foundation Series. MPT-7B is a transformer trained from scratch on 1T tokens of text and code. It is open source, available fo

Large language models have recently been shown to attain reasonable zero-shot generalization on a diverse set of tasks (Brown et al., 2020). It has been hypothesized that this is

A simple OpenAI Gym environment for Neural Architecture Search (NAS)

An open source AutoML toolkit for automate machine learning lifecycle, including feature engineering, neural architecture search, model compression and hyper-parameter tuning.

Home for OctoML PyTorch Profiler

We introduce OLMoE, a fully open, state-of-the-art language model leveraging sparse Mixture-of-Experts (MoE). OLMoE-1B-7B has 7 billion (B) parameters but uses only 1B per input

Python package for LLM compression

2022-05

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

An Automatic Model Compression (AutoMC) framework for developing smaller and faster AI applications.

Pretrain, finetune ANY AI model of ANY size on 1 or 10,000+ GPUs with zero code changes.

Tech Report HuggingFace ModelScope Qwen Chat HuggingFace Demo ModelScope Demo DISCORD Introduction Two months after upgrading Qwen2.5-Turbo to support context length up to one mi

Recurrent Neural Networks (RNNs) offer fast inference on long sequences but are hard to optimize and slow to train. Deep state-space models (SSMs) have recently been shown to per

An Efficient and User-Friendly Scaling Library for Reinforcement Learning with Large Language Models

Transformers have revolutionized almost all natural language processing (NLP) tasks but suffer from memory and computational complexity that scales quadratically with sequence le

Org profile for RWKV on Hugging Face, the AI community building the future.

SGLang is a high-performance serving framework for large language models and multimodal models.

A blazing fast inference solution for text embeddings models

Large Language Model Text Generation Inference

Minimal reproduction of DeepSeek R1-Zero

Serve, optimize and scale PyTorch models in production

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

Playground for devs to finetune & deploy LLMs

Unsloth Studio is a web UI for training and running open models like Gemma 4, Qwen3.6, DeepSeek, gpt-oss locally.

AutoML tools chain

verl/HybridFlow: A Flexible and Efficient RL Post-Training Framework

A high-throughput and memory-efficient inference and serving engine for LLMs

An Open-Ended Embodied Agent with Large Language Models

2021-12

Robust Speech Recognition via Large-Scale Weak Supervision

Real-time inference for Stable Diffusion - 0.88s latency. Covers AITemplate, nvFuser, TensorRT, FlashAttention. Join our Discord communty: https://discord.com/invite/TgHXuSJEk6

ZenML 🙏: One AI Platform from Pipelines to Agents. https://zenml.io.

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Microsoft

ML models and internal tensors 3D visualizer

🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP a

Projects

Automatic architecture search and hyperparameter optimization for PyTorch

An autoML framework & toolkit for machine learning on graphs.

Go ahead and axolotl questions

🔊 Text-Prompted Generative Audio Model

2018-10

BigScience

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Efficient Training (including pre-training and fine-tuning) for Big Models

ChatGLM2-6B: An Open Bilingual Chat LLM | 开源双语对话语言模型

🏄 Scalable embedding, reasoning, ranking for images and sentences with CLIP

CodeGeeX: An Open Multilingual Code Generation Model (KDD 2023)

GITHUB HUGGING FACE MODELSCOPE DEMO DISCORD Introduction The advent of advanced programming tools, which harnesses the power of large language models (LLMs), has significantly en

The most powerful AI platform for enterprises. Customize, fine-tune, and deploy AI assistants, autonomous agents, and multimodal AI with open models.

Home of CodeT5: Open Code LLMs for Code Understanding and Generation

Making large AI models cheaper, faster and more accessible

DeepSeek Math series

First-generation reasoning models from DeepSeek.

We present DeepSeek-V2, a strong Mixture-of-Experts (MoE) language model characterized by economical training and efficient inference. It comprises 236B total parameters, of whic

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

DeepSeek-VL model series

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

A frankensteinian amalgamation of notebooks, models and techniques for the generation of AI Art and Animations.

Distilabel is a framework for synthetic data and AI feedback for engineers who need fast, reliable and scalable pipelines based on verified research papers.

A continuous diffusion-based Vision-Language-Action model that integrates diffusion policies into autoregressive VLMs for robust and precise continuous robotic control.

Databricks’ Dolly, a large language model trained on the Databricks Machine Learning Platform

[ACL 2024] An Easy-to-use Knowledge Editing Framework for LLMs.

An easy-to-use federated learning platform

A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

🩹Editing large language models within 10 seconds⚡

A powerful tool for creating high-quality training datasets for Large Language Models (LLMs)（一个快速生成高质量LLM微调训练数据集的工具）

FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables runn

FlagAI (Fast LArge-scale General AI models) is a fast, easy-to-use and extensible toolkit for large-scale model.

Open source, high performance fine tuning as a service for GPT4 quality models with 5x lower latency and 3x lower cost

Galactica

GLM-130B

ChatGLM-6B: An Open Bilingual Dialogue Language Model | 开源双语对话语言模型

Gorilla: Training and Evaluating LLMs for Function Calls (Tool Calls)

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

Grok-1-314B-MoE — indexed from awesome-llm

Hands-On Large Language Models

A General Automated Machine Learning framework to simplify the development of End-to-end AutoML toolkits in specific domains.

Publications — Google DeepMind

Infinity is a high-throughput, low-latency serving engine for text-embeddings, reranking models, clip, clap and colpali

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

A big convergence of language, multimodal perception, action, and world modeling is a key step toward artificial general intelligence. In this work, we introduce Kosmos-1, a Mult

A PyTorch Library for Meta-learning Research

Efficient Triton Kernels for LLM Training

🤗 LeRobot: Making AI for Robotics more accessible with end-to-end learning

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

[OPT-1.3 6.7 13 30 66B](https://arxiv.org/abs/2205.01068)

2023-07

[Llama 3.1-8 70 405B](https://llama.meta.com/)

[Llama 2-7 13 70B](https://llama.meta.com/llama2/)

2023-02

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

Low-code framework for building custom LLMs, neural networks, and other AI models

Using Low-rank adaptation to quickly fine-tune diffusion models.

骆驼(Luotuo): Open Sourced Chinese Language Models. Developed by 陈启源 @ 华中师范大学 & 李鲁鲁 @ 商汤科技 & 冷子昂 @ 商汤科技

CMU&Princeton

Ongoing research training transformer models at scale

Meta Lingua: a lean, efficient, and easy-to-hack codebase to research LLMs.

[NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to

The most powerful AI platform for enterprises. Customize, fine-tune, and deploy AI assistants, autonomous agents, and multimodal AI with open models.

Official repo for MM-REACT

Moonshot's Compute-efficient MoE LLM, first Scaling Up of Muon Optimizer

Introducing MPT-7B, the first entry in our MosaicML Foundation Series. MPT-7B is a transformer trained from scratch on 1T tokens of text and code. It is open source, available fo

Large language models have recently been shown to attain reasonable zero-shot generalization on a diverse set of tasks (Brown et al., 2020). It has been hypothesized that this is

Minimalistic large language model 3D-parallelism training

A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech

An open source AutoML toolkit for automate machine learning lifecycle, including feature engineering, neural architecture search, model compression and hyper-parameter tuning.

Octo is a transformer-based robot policy trained on a diverse mix of 800k robot trajectories.

Artifacts for the first set of OLMo models.

Language models (LMs) have become ubiquitous in both NLP research and in commercial product offerings. As their commercial importance has surged, the most powerful models have be

We introduce OLMoE, a fully open, state-of-the-art language model leveraging sparse Mixture-of-Experts (MoE). OLMoE-1B-7B has 7 billion (B) parameters but uses only 1B per input

Evaluation suite for LLMs

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

An Easy-to-use, Scalable and High-performance Agentic RL Framework based on Ray (PPO & DAPO & REINFORCE++ & VLM & TIS & vLLM & Ray & Async RL)

Open-source VLA models from Physical Intelligence, including π₀ and π₀.5 — flow-based vision-language-action models pretrained on large-scale robot data with fine-tuning support.

OpenVLA: An open-source vision-language-action model for robotic manipulation.

2022-05

An optimized deep prompt tuning strategy comparable to fine-tuning across scales and tasks

🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

An Automatic Model Compression (AutoMC) framework for developing smaller and faster AI applications.

The hub for EleutherAI's work on interpretability and learning dynamics

Pretrain, finetune ANY AI model of ANY size on 1 or 10,000+ GPUs with zero code changes.

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Qwen - a Qwen Collection

GITHUB HUGGING FACE MODELSCOPE DEMO DISCORD Introduction After months of efforts, we are pleased to announce the evolution from Qwen1.5 to Qwen2. This time, we bring to you: Pret

Tech Report HuggingFace ModelScope Qwen Chat HuggingFace Demo ModelScope Demo DISCORD Introduction Two months after upgrading Qwen2.5-Turbo to support context length up to one mi

DEMO PAPER GITHUB HUGGING FACE MODELSCOPE DISCORD To achieve the objective of building an AGI system, the model should be capable of understanding information from different moda

GITHUB HUGGING FACE MODELSCOPE DISCORD 🚨 This model mainly supports English. We will release bilingual (English and Chinese) math models soon. Introduction Over the past year, w

Supercharge Your LLM Application Evaluations 🚀

Recurrent Neural Networks (RNNs) offer fast inference on long sequences but are hard to optimize and slow to train. Deep state-space models (SSMs) have recently been shown to per

An efficient VLA model leveraging State Space Models (Mamba) instead of standard self-attention, offering linear inference complexity for efficient, recurrent robotic reasoning.

Flan-T5/PaLM

The repository provides code for running inference with the SegmentAnything Model (SAM), links for downloading the trained model checkpoints, and example notebooks that show how to

SGLang is a high-performance serving framework for large language models and multimodal models.

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

A latent text-to-image diffusion model

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

All models, datasets, and demos related to StarCoder!

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NV

Large Language Model Text Generation Inference

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Modern artificial intelligence (AI) systems are powered by foundation models. This paper presents a new set of foundation models, called Llama 3. It is a herd of language models

A collection of extensions and data-loaders for few-shot learning & meta-learning in PyTorch

PyTorch native post-training library

A PyTorch native platform for training generative AI models

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Playground for devs to finetune & deploy LLMs

UQLM: Uncertainty Quantification for Language Models, is a Python package for UQ-based LLM hallucination detection

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

verl/HybridFlow: A Flexible and Efficient RL Post-Training Framework

Instruction tuning large language models (LLMs) using machine-generated instruction-following data has improved zero-shot capabilities on new tasks, but the idea is less explored

A high-throughput and memory-efficient inference and serving engine for LLMs

An Open-Ended Embodied Agent with Large Language Models

2021-12

Robust Speech Recognition via Large-Scale Weak Supervision

Build, personalize and control your own LLMs. From data pre-processing to fine-tuning, xTuring provides an easy way to personalize open-source LLMs. Join our discord community: htt

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Stanford

DreamBench++: A Human-Aligned Benchmark for Personalized Image Generation

Megatron-LM

2022-12

How do large language models (LLMs) develop and evolve over the course of training? How do these patterns change as models scale? To answer these questions, we introduce \textit{

Aim 💫 — An easy-to-use & supercharged open-source experiment tracker.

Platform created by the community to programmatically author, schedule and monitor workflows.

Transformers

Federated Learning Systems Paper List

A list of awesome compiler projects and papers for tensor computation and deep learning.

Deploy a ML inference service on a budget in less than 10 lines of code.

ClearML - Auto-Magical CI/CD to streamline your AI workload. Experiment Management, Data Management, Pipeline, Orchestration, Scheduling & Serving in one MLOps/LLMOps solution

A system-level, binary package and environment manager running on all major operating systems and platforms.

Deepchecks: Tests for Continuous Validation of ML Models & Data. Deepchecks is a holistic open-source solution for all of your AI & ML validation needs, enabling to thoroughly test

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs

Distilabel is a framework for synthetic data and AI feedback for engineers who need fast, reliable and scalable pipelines based on verified research papers.

The Open Source Feature Store for AI/ML

The Virtual Feature Store. Turn your existing data infrastructure into a feature store.

Flower: A Friendly Federated AI Framework

A library for high performance deep learning inference on NVIDIA GPUs.

A hyperparameter optimization framework, inspired by Optuna.

Experiment tracking, ML developer tools

Federated Learning Made Easy

Hopsworks - Data-Intensive AI platform with a Feature Store

Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.

a distributed Hyperband implementation on Steroids

Tuning hyperparams fast with Hyperband

Distributed Asynchronous Hyperparameter Optimization in Python

A toolset for black-box hyperparameter optimisation.

Jupyter Interactive Notebook

Machine Learning Pipelines for Kubeflow

LabNotebook is a tool that allows you to flexibly monitor, record, save, and query all your machine learning experiments.

A model-agnostic visual debugging tool for machine learning

MLRun is an open source MLOps platform for quickly building and managing continuous ML applications across their lifecycle. MLRun integrates into your development and CI/CD environ


Open Source ML Model Versioning, Metadata, and Experiment Management

ModelFox makes it easy to train, deploy, and monitor machine learning models.

A global, black box optimization engine for real world metric optimization.

A high-performance ML model serving framework, offers dynamic batching and CPU/GPU pipelines to fully exploit your compute machine

Visualizer for neural network, deep learning and machine learning models

Representation and Reference Lowering of ONNX Models in MLIR Compiler Infrastructure

Docker for Your ML/DL Models Based on OCI Artifacts

Resource scheduling and cluster management for AI

Open Source AI Infra & Engineering Control Plane

Sacred is a tool to help you configure, organize, log and reproduce experiments developed at IDSIA.

Scalene: a high-performance, high-precision CPU, GPU, and memory profiler for Python with AI-powered optimization proposals

scikit-learn: machine learning in Python

Spearmint Bayesian optimization codebase

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

Open Machine Learning Compiler Framework

Data search & enrichment library for Machine Learning → Easily find and add relevant features to your ML & AI pipeline from hundreds of public and premium external data sources, in

W&B Weave helps developers evaluate, monitor, and iterate continuously to deliver generative AI applications with confidence.

An open-source data logging library for machine learning models and data pipelines. 📚 Provides visibility into data quality & model performance over time. 🛡️ Supports privacy-pre

Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more

Caffe: a fast open framework for deep learning.

Minimalist ML framework for Rust

Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more

MegEngine 是一个快速、可拓展、易于使用且支持自动求导的深度学习框架

MindSpore is a new open source deep learning training/inference framework that could be used for mobile, edge and cloud scenarios.

ncnn is a high-performance neural network inference framework optimized for the mobile platform

OneFlow is a deep learning framework designed to be user-friendly, scalable and efficient.

PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice （『飞桨』核心框架，深度学习&机器学习高性能单机、分布式训练和跨平台部署）

An Open Source Machine Learning Framework for Everyone

TNN: developed by Tencent Youtu Lab and Guangying Lab, a uniform deep learning inference framework for mobile、desktop and server. TNN is distinguished by several outstanding featur

A minimalist neural network library optimized for sparse data and single machine environments.