Enterprise DNAEnterprise DNA

Open Source

TensorRT-LLM

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NV

Our verdict

Not reviewed yet

We have not reviewed TensorRT-LLM yet

No one at Enterprise DNA has checked this listing against its sources. The description comes from public listings gathered automatically. Any fact below shows where it came from and when it was last checked.

How we review records

From the listing

About TensorRT-LLM

TensorRT-LLM is a Python framework for defining and optimizing large language model inference on NVIDIA GPUs. It provides a high-level API to build LLM architectures and applies state-of-the-art optimizations like quantization and kernel fusion, then generates Python and C++ runtimes to execute inference efficiently.

Best for

Teams deploying LLMs at scale on NVIDIA infrastructure who need maximum inference performance.

Use cases

  • Deploying LLMs with low latency on NVIDIA hardware
  • Optimizing inference throughput for production serving
  • Building custom inference pipelines with fine-grained control

Notes

13,781 stars on GitHub. Last updated 2026-06-01.

Pros

  • Deep NVIDIA GPU optimization built in, not bolted on
  • Supports both Python and C++ runtime generation for flexibility
  • Active community project with 13k+ stars and regular updates

Cons

  • Locked to NVIDIA GPUs, no portability to other accelerators
  • Steeper learning curve than higher-level inference frameworks
  • Requires understanding of LLM architecture and optimization techniques

Key facts

What we have checked

Latest prerelease version
1.4.0rc3 (released 6 Nov 2026)
Checked GitHub
Days since the last commit
0
Checked GitHub
GitHub forks
2,935
Checked GitHub
GitHub stars
15,076
Checked GitHub
Days to a first reply on issues
0.1
Checked GitHub
Open issues
748
Checked GitHub
Releases in the last 90 days
9
Checked GitHub

Each value shows when we last confirmed it and where it came from. Values we have not confirmed are left out, never guessed.

Stars and usage

How TensorRT-LLM is growing

GitHub stars

13,781 to 15,076

stars. 12 July 2026 to 14 Nov 2026, 5 readings.

Source: GitHub. Checked .

Health and security

Is TensorRT-LLM looked after, and what do its setup files show

Health score

100 out of 100

Does someone look after this repository? It does not say the code is good or safe. Worked out from GitHub signals checked . How it is worked out

PartPointsWhat we saw
Recent activity30 of 30Last push 0 days ago
Releases15 of 159 releases in 90 days
Issue repliesNot countedThe latest check has no issue response reading (fewer than 3 measured issues, or issues switched off)
Maintainer15 of 15Owned by an organisation
Official sourceNot countedMCP servers only

Alternatives

Alternatives to TensorRT-LLM

RecordTypeHow we knowChecked
FasterTransformerAlternative to thisOpen sourceNamed in the listingNot dated
llama.cppAlternative to thisOpen sourceNamed in the listingNot dated
LMDeployAlternative to thisOpen sourceNamed in the listingNot dated
MInferenceAlternative to thisOpen sourceNamed in the listingNot dated
mistral.rsAlternative to thisOpen sourceNamed in the listingNot dated
MNN-LLMAlternative to thisOpen sourceNamed in the listingNot dated
SGLangAlternative to thisOpen sourceNamed in the listingNot dated
vLLMAlternative to thisOpen sourceNamed in the listingNot dated

Compare

Compare TensorRT-LLM side by side

Sources

Where this comes from

  • GitHubOfficial

    Checked · Latest prerelease version

  • GitHubRegistry

    Checked · Days since the last commit, GitHub forks, GitHub stars, Days to a first reply on issues, Open issues, Releases in the last 90 days

A source shows what was published when we checked it. It does not certify security, performance or a working installation. Read our method.

Changelog

What changed

  1. Released 1.4.0rc3

    Source: GitHub

  2. Released 1.4.0rc2

    Source: GitHub

  3. Released 1.4.0rc1

    Source: GitHub

  4. Released 1.4.0rc0

    Source: GitHub

  5. Released 1.3.0rc29

    Source: GitHub

  6. Released 1.3.0rc28

    Source: GitHub

  7. Released 1.3.0rc27

    Source: GitHub

  8. Released 1.3.0rc26

    Source: GitHub

  9. Released 1.3.0rc25

    Source: GitHub

  10. Released 1.3.0rc24

    Source: GitHub

  11. Released 1.3.0rc23

    Source: GitHub

  12. Released 1.3.0rc22

    Source: GitHub

Open-source & AI alternatives

Swap-in tools that solve the same job. Weigh the trade-offs before you commit.

Pairs with

Other entries in the index that connect to this one. Click through to see the chain.

Pairs with10 entries
OOSSmediumCatalogue

Awesome-LLM-Inference

Community

📖A curated list of Awesome LLM/VLM Inference Papers with codes: WINT8/4, FlashAttention, PagedAttention, MLA, Parallelism, etc. 🎉🎉

★ 16
SSkillslowCatalogue

airunway-aks-setup

microsoft

Official agent plugin providing skills and MCP server configurations for Azure scenarios.

★ 1,628
SSkillslowCatalogue

evaluating-cosmos-policy

Orchestra-Research

Comprehensive open-source library of AI research and engineering skills for any AI model. Package the skills and your claude code/codex/gemini agent will be an AI research agent wi

★ 14,387
SSkillslowCatalogue

faiss

Orchestra-Research

Comprehensive open-source library of AI research and engineering skills for any AI model. Package the skills and your claude code/codex/gemini agent will be an AI research agent wi

★ 14,387
SSkillslowCatalogue

gguf-quantization

Orchestra-Research

Comprehensive open-source library of AI research and engineering skills for any AI model. Package the skills and your claude code/codex/gemini agent will be an AI research agent wi

★ 14,387
SSkillslowCatalogue

gptq

Orchestra-Research

Comprehensive open-source library of AI research and engineering skills for any AI model. Package the skills and your claude code/codex/gemini agent will be an AI research agent wi

★ 14,387
SSkillslowCatalogue

Knative Istio Typesense GPU

PatrickJS

📄 Configuration files that enhance Cursor AI editor experience with custom rules and behaviors

★ 41,137
SSkillslowCatalogue

Linux Nvidia Cuda Python

PatrickJS

📄 Configuration files that enhance Cursor AI editor experience with custom rules and behaviors

★ 41,137
SSkillslowCatalogue

speculative-decoding

Orchestra-Research

Comprehensive open-source library of AI research and engineering skills for any AI model. Package the skills and your claude code/codex/gemini agent will be an AI research agent wi

★ 14,387
SSkillslowCatalogue

training-llms-megatron

Orchestra-Research

Comprehensive open-source library of AI research and engineering skills for any AI model. Package the skills and your claude code/codex/gemini agent will be an AI research agent wi

★ 14,387
Free 27-page guide

Get the free Developer’s Field Guide

A 27-page field guide to the AI coding workflow with Claude. Claude Code, MCP servers, the prompt patterns that work, and what to delegate. Free.

Enter your work email. We send it straight over, plus a few short notes worth knowing. Unsubscribe any time.

No spam. Unsubscribe any time.