Enterprise DNAEnterprise DNA

Open Source

lighteval

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

Our verdict

Not reviewed yet

We have not reviewed lighteval yet

No one at Enterprise DNA has checked this listing against its sources. The description comes from public listings gathered automatically. Any fact below shows where it came from and when it was last checked.

How we review records

From the listing

About lighteval

Lighteval is an open-source Python framework for evaluating large language models across multiple backends. It provides a unified toolkit to run standardized benchmarks and compare models from different providers or architectures. Developed by the community and hosted under Hugging Face's GitHub, it simplifies the evaluation workflow for LLMs.

Best for

Researchers and developers who need a unified way to evaluate and compare LLMs from different sources

Use cases

  • Benchmark LLM performance on standard tasks using a single interface
  • Compare outputs from different models or provider backends
  • Integrate automated evaluation into development or CI pipelines

Notes

2,430 stars on GitHub. Last updated 2026-05-29. Licensed MIT.

Pros

  • Open-source with an active community (over 2,400 GitHub stars)
  • Supports multiple backends, enabling flexible model comparisons
  • Written in Python, making it accessible to the data science ecosystem

Cons

  • Limited to evaluation tasks; does not cover training or deployment
  • Requires manual setup and configuration of backend integrations
  • Community-maintained, without dedicated enterprise support

Key facts

What we have checked

Latest version
0.13.0 (released 24 Nov 2025)
Checked GitHub
Days since the last commit
2
Checked GitHub
GitHub forks
614
Checked GitHub
GitHub stars
2,586
Checked GitHub
Days to a first reply on issues
23.7
Checked GitHub
Open issues
259
Checked GitHub
Releases in the last 90 days
0
Checked GitHub

Each value shows when we last confirmed it and where it came from. Values we have not confirmed are left out, never guessed.

Stars and usage

How lighteval is growing

GitHub stars

2,430 to 2,586

stars. 12 July 2026 to 13 Nov 2026, 5 readings.

Source: GitHub. Checked .

Health and security

Is lighteval looked after, and what do its setup files show

Health score

62 out of 100

Does someone look after this repository? It does not say the code is good or safe. Worked out from GitHub signals checked . How it is worked out

PartPointsWhat we saw
Recent activity30 of 30Last push 2 days ago
Releases0 of 150 releases in 90 days
Issue replies4.7 of 20First reply in 23.7 days (median)
Maintainer15 of 15Owned by an organisation
Official sourceNot countedMCP servers only

Alternatives

Alternatives to lighteval

RecordTypeHow we knowChecked
lm-evaluation-harnessThis is an alternative to itOpen sourceNamed in the listingNot dated
OpenAI EvalsThis is an alternative to itOpen sourceNamed in the listingNot dated

Compare

Compare lighteval side by side

Sources

Where this comes from

  • GitHubRegistry

    Checked · Days since the last commit, GitHub forks, GitHub stars, Days to a first reply on issues, Open issues, Releases in the last 90 days

  • GitHubOfficial

    Checked · Latest version

A source shows what was published when we checked it. It does not certify security, performance or a working installation. Read our method.

Changelog

What changed

  1. Released 0.13.0

    Source: GitHub

  2. Released 0.12.2

    Source: GitHub

  3. Released 0.12.1

    Source: GitHub

  4. Released 0.12.0

    Source: GitHub

  5. Released 0.11.0

    Source: GitHub

  6. Released 0.10.0

    Source: GitHub

  7. Released 0.9.2

    Source: GitHub

  8. Released 0.8.0

    Source: GitHub

  9. Released 0.7.0

    Source: GitHub

  10. Released 0.6.0

    Source: GitHub

Free 27-page guide

Get the free Developer’s Field Guide

A 27-page field guide to the AI coding workflow with Claude. Claude Code, MCP servers, the prompt patterns that work, and what to delegate. Free.

Enter your work email. We send it straight over, plus a few short notes worth knowing. Unsubscribe any time.

No spam. Unsubscribe any time.