Open Source
Transformer Engine
A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit and 4-bit floating point (FP8 and FP4) precision on Hopper, Ada and Blackwell GPUs, to provide b
Our verdict
We have not reviewed Transformer Engine yet
No one at Enterprise DNA has checked this listing against its sources. The description comes from public listings gathered automatically. Any fact below shows where it came from and when it was last checked.
From the listing
About Transformer Engine
Transformer Engine is a Python library that accelerates Transformer models on NVIDIA GPUs by leveraging low-precision floating point formats (FP8 and FP4). It targets Hopper, Ada, and Blackwell architectures to improve performance and reduce memory usage during both training and inference.
Best for
Developers training or deploying large transformer models on modern NVIDIA GPUs who need to maximize performance and minimize memory usage
Use cases
- Training large language models with reduced memory footprint
- Running inference on transformer models with higher throughput
- Fine-tuning transformers on GPU clusters with limited VRAM
Notes
3,374 stars on GitHub. Last updated 2026-06-01. Licensed Apache-2.0.
Pros
- Significantly reduces memory consumption compared to FP32 or FP16
- Optimized for the latest NVIDIA GPU families (Hopper, Ada, Blackwell)
- Supports both training and inference for transformer architectures
Cons
- Requires compatible NVIDIA GPUs (Hopper, Ada, or Blackwell) to use FP8/FP4
- Limited to specific precision formats; not a general-purpose optimization library
- May need code modifications to integrate into existing PyTorch workflows
Key facts
What we have checked
- Latest version
- 2.20.2 (released 5 Oct 2026)
- GitHub stars
- 3,556
Each value shows when we last confirmed it and where it came from. Values we have not confirmed are left out, never guessed.
Stars and usage
How Transformer Engine is growing
3,374 to 3,556
Sources
Where this comes from
- GitHubOfficial
- github api via nightly crawl, recovered from git history 20e874270887c1008f860ee0198e461a23c9e438Registry
A source shows what was published when we checked it. It does not certify security, performance or a working installation. Read our method.
Changelog
What changed
Released 2.20.2
Released 2.19
Released 2.18
Released 2.17.1
Released 2.17
Released 2.16.1
Released 2.16
Released 2.15
Released 2.14.1
Released 2.14
Pairs with
Other entries in the index that connect to this one. Click through to see the chain.
DeepSpeed
Community
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
Megatron-LM
Community
Ongoing research training transformer models at scale
NeMo Framework
Community
A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech
Get the free Developer’s Field Guide
A 27-page field guide to the AI coding workflow with Claude. Claude Code, MCP servers, the prompt patterns that work, and what to delegate. Free.
Enter your work email. We send it straight over, plus a few short notes worth knowing. Unsubscribe any time.