Emergent Abilities of Large Language Models

Overview

A conceptual framework describing abilities that become present only above a certain model scale, not predictable from smaller models. It provides a lens for researchers and builders to identify and anticipate sudden capability jumps in large language models.

Best for

Best for
Researchers and engineers planning large-scale language model training or deployment

Use cases

• Planning model scaling experiments to trigger new capabilities
• Identifying which tasks require minimal model size thresholds
• Explaining sudden performance improvements in deployed LLMs

Notes

A conceptual framework describing abilities that become present only above a certain model scale, not predictable from smaller models. It provides a lens for researchers and builders to identify and anticipate sudden capability jumps in large language models.

Use cases

• Planning model scaling experiments to trigger new capabilities
• Identifying which tasks require minimal model size thresholds
• Explaining sudden performance improvements in deployed LLMs

Pros

• Offers a principled way to think about model scaling
• Helps set realistic expectations for smaller model performance
• Provides a shared vocabulary for discussing model capabilities

Cons

• Does not predict which abilities will emerge at which scale
• Relies on observational evidence rather than causal explanation
• Limited direct utility for builders without extensive scaling resources

Indexed from awesome-llm and enriched against its public facts.