Open Source
lighteval
Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends
Our verdict
We have not reviewed lighteval yet
No one at Enterprise DNA has checked this listing against its sources. The description comes from public listings gathered automatically. Any fact below shows where it came from and when it was last checked.
From the listing
About lighteval
Lighteval is an open-source Python framework for evaluating large language models across multiple backends. It provides a unified toolkit to run standardized benchmarks and compare models from different providers or architectures. Developed by the community and hosted under Hugging Face's GitHub, it simplifies the evaluation workflow for LLMs.
Best for
Researchers and developers who need a unified way to evaluate and compare LLMs from different sources
Use cases
- Benchmark LLM performance on standard tasks using a single interface
- Compare outputs from different models or provider backends
- Integrate automated evaluation into development or CI pipelines
Notes
2,430 stars on GitHub. Last updated 2026-05-29. Licensed MIT.
Pros
- Open-source with an active community (over 2,400 GitHub stars)
- Supports multiple backends, enabling flexible model comparisons
- Written in Python, making it accessible to the data science ecosystem
Cons
- Limited to evaluation tasks; does not cover training or deployment
- Requires manual setup and configuration of backend integrations
- Community-maintained, without dedicated enterprise support
Key facts
What we have checked
- Latest version
- 0.13.0 (released 24 Nov 2025)
- GitHub stars
- 2,550
Each value shows when we last confirmed it and where it came from. Values we have not confirmed are left out, never guessed.
Stars and usage
How lighteval is growing
2,430 to 2,550
Alternatives
Alternatives to lighteval
| Record | Type | How we know | Checked |
|---|---|---|---|
| lm-evaluation-harnessThis is an alternative to it | Open source | Named in the listing | Not dated |
| OpenAI EvalsThis is an alternative to it | Open source | Named in the listing | Not dated |
Compare
Compare lighteval side by side
Sources
Where this comes from
- GitHubOfficial
- github api via nightly crawl, recovered from git history 20e874270887c1008f860ee0198e461a23c9e438Registry
A source shows what was published when we checked it. It does not certify security, performance or a working installation. Read our method.
Changelog
What changed
Released 0.13.0
Released 0.12.2
Released 0.12.1
Released 0.12.0
Released 0.11.0
Released 0.10.0
Released 0.9.2
Released 0.8.0
Released 0.7.0
Released 0.6.0
Open-source & AI alternatives
Swap-in tools that solve the same job. Weigh the trade-offs before you commit.
Pairs with
Other entries in the index that connect to this one. Click through to see the chain.
llama.cpp
Various
LLM inference in C/C++
SGLang
Community
SGLang is a high-performance serving framework for large language models and multimodal models.
vLLM
Community
A high-throughput and memory-efficient inference and serving engine for LLMs
Get the free Developer’s Field Guide
A 27-page field guide to the AI coding workflow with Claude. Claude Code, MCP servers, the prompt patterns that work, and what to delegate. Free.
Enter your work email. We send it straight over, plus a few short notes worth knowing. Unsubscribe any time.