Open LLM Leaderboard

Overview

The Open LLM Leaderboard is a community-maintained benchmark that tracks, ranks, and evaluates open-source large language models and chatbots. It uses standardized tests like ARC, HellaSwag, MMLU, and TruthfulQA to produce comparable performance scores.

Best for

Best for
Developers and researchers evaluating open LLMs for general-purpose language tasks

Use cases

• Compare open LLMs before selecting one for a project
• Track model improvements across new releases and fine-tunes
• Identify top-performing models for specific evaluation tasks

Notes

The Open LLM Leaderboard is a community-maintained benchmark that tracks, ranks, and evaluates open-source large language models and chatbots. It uses standardized tests like ARC, HellaSwag, MMLU, and TruthfulQA to produce comparable performance scores.

Use cases

• Compare open LLMs before selecting one for a project
• Track model improvements across new releases and fine-tunes
• Identify top-performing models for specific evaluation tasks

Pros

• Provides a single, standardized comparison point for many open models
• Community-driven and transparent with reproducible evaluation methodology
• Regularly updated with new models and benchmarks

Cons

• Limited to a fixed set of benchmarks that may not reflect real-world use
• Does not measure inference speed, cost, or deployment practicality
• Rankings can be gamed by models optimized specifically for these tests

Indexed from awesome-llm and enriched against its public facts.