Language models are few-shot learners

Overview

This 2020 paper introduced in-context learning, showing that large language models can perform tasks from a few examples without gradient updates. It demonstrated that scaling model size and number of examples improves few-shot performance across a range of NLP benchmarks.

Best for

Best for
Researchers and developers exploring few-shot learning with large language models

Use cases

• Classifying text with a handful of labeled examples
• Generating answers or completions from a prompt with demonstrations
• Evaluating model capabilities on new tasks without fine-tuning

Notes

This 2020 paper introduced in-context learning, showing that large language models can perform tasks from a few examples without gradient updates. It demonstrated that scaling model size and number of examples improves few-shot performance across a range of NLP benchmarks.

Use cases

• Classifying text with a handful of labeled examples
• Generating answers or completions from a prompt with demonstrations
• Evaluating model capabilities on new tasks without fine-tuning

Pros

• Established a foundational method for few-shot NLP tasks
• Reduced need for task-specific training data
• Influenced subsequent prompting and in-context learning research

Cons

• Performance is sensitive to prompt wording and example selection
• Requires large models to be effective, limiting accessibility
• Does not provide a mechanism for learning beyond the context window

Indexed from awesome-llm and enriched against its public facts.