DeepSpeed
by Community
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
OSS
DeepSpeed
Added 1 June 2026
Overview
DeepSpeed is a Python library for optimizing distributed training and inference of large language models and deep neural networks. It reduces memory footprint, accelerates training speed, and enables efficient multi-GPU and multi-node setups through techniques like gradient checkpointing, mixed precision, and ZeRO optimizer states partitioning.
Best for
Best for
Teams training large models who need to maximize GPU efficiency and scale across multiple devices.
Use cases
- Training large models on limited GPU memory
- Scaling training across multiple GPUs or nodes
- Reducing inference latency for deployed models
Notes
DeepSpeed is a Python library for optimizing distributed training and inference of large language models and deep neural networks. It reduces memory footprint, accelerates training speed, and enables efficient multi-GPU and multi-node setups through techniques like gradient checkpointing, mixed precision, and ZeRO optimizer states partitioning.
42,436 stars on GitHub. Last updated 2026-06-01. Licensed Apache-2.0.
Use cases
- Training large models on limited GPU memory
- Scaling training across multiple GPUs or nodes
- Reducing inference latency for deployed models
Pros
- Significant memory savings enable training larger models on existing hardware
- Production-ready with strong community adoption and Microsoft backing
- Works with existing PyTorch code with minimal integration effort
Cons
- Steep learning curve for advanced features like ZeRO stages and custom configurations
- Debugging distributed training issues remains complex despite optimizations
- Performance gains vary significantly based on hardware, model architecture, and tuning
Indexed from awesome-llm and enriched against its public facts.
Pros
- Significant memory savings enable training larger models on existing hardware
- Production-ready with strong community adoption and Microsoft backing
- Works with existing PyTorch code with minimal integration effort
Cons
- Steep learning curve for advanced features like ZeRO stages and custom configurations
- Debugging distributed training issues remains complex despite optimizations
- Performance gains vary significantly based on hardware, model architecture, and tuning
Pairs with
Other entries in the index that connect to this one. Click through to see the chain.
Accelerate
Community
🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP a
Axolotl
Community
Go ahead and axolotl questions
Flyflow
Community
Open source, high performance fine tuning as a service for GPT4 quality models with 5x lower latency and 3x lower cost
open-r1
Community
Fully open reproduction of DeepSeek-R1
TRL
Community
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
veRL
Community
verl/HybridFlow: A Flexible and Efficient RL Post-Training Framework
BLOOMZ&mT0
Community
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
DeepSeek-V2.5
Community
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Falcon 40B
Community
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
FlagAI
Community
FlagAI (Fast LArge-scale General AI models) is a fast, easy-to-use and extensible toolkit for large-scale model.
GPT-NeoX
Community
An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries
open-r1
Community
Fully open reproduction of DeepSeek-R1
Qwen2-0.5B|1.5B|7B|57B-A14B-MoE|72B
Community
GITHUB HUGGING FACE MODELSCOPE DEMO DISCORD Introduction After months of efforts, we are pleased to announce the evolution from Qwen1.5 to Qwen2. This time, we bring to you: Pret
Bloom
Various
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Large Language Model Training in 2023
Community
Learn about large language model training with insights on large language model examples, model architecture, and model training guide.
Liger-Kernel
Community
Efficient Triton Kernels for LLM Training
LLMDatahub
Community
A quick guide (especially) for trending instruction finetuning datasets
MInference
Community
[NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to
nanotron
Community
Minimalistic large language model 3D-parallelism training
NeMo Framework
Community
A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech
peft
Community
🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.
SkyPilot
Community
Run, manage, and scale AI workloads on any AI infrastructure. Use one system to access & manage all AI compute (Kubernetes, Slurm, 20+ clouds, on-prem).
Transformer Engine
Community
A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit and 4-bit floating point (FP8 and FP4) precision on Hopper, Ada and Blackwell GPUs, to provide b
veRL
Community
verl/HybridFlow: A Flexible and Efficient RL Post-Training Framework
Weights & Biases
Community
W&B Weave helps developers evaluate, monitor, and iterate continuously to deliver generative AI applications with confidence.
BMTrain
Community
Efficient Training (including pre-training and fine-tuning) for Big Models
Colossal-AI
Community
Making large AI models cheaper, faster and more accessible
FasterTransformer
Community
Transformer related optimization, including BERT, GPT
maxtext
Community
A simple, performant and scalable Jax LLM!
Megatron-LM
Community
Ongoing research training transformer models at scale
Mesh Tensorflow
Community
Mesh TensorFlow: Model Parallelism Made Easier
nanotron
Community
Minimalistic large language model 3D-parallelism training
torchtitan
Community
A PyTorch native platform for training generative AI models
Transformer Engine
Community
A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit and 4-bit floating point (FP8 and FP4) precision on Hopper, Ada and Blackwell GPUs, to provide b
Get the free Developer’s Field Guide
A 27-page field guide to the AI coding workflow with Claude. Claude Code, MCP servers, the prompt patterns that work, and what to delegate. Free.
Enter your work email. We send it straight over, plus a few short notes worth knowing. Unsubscribe any time.