AutoRAG

Overview

AutoRAG is an open-source framework that automates the evaluation and optimization of retrieval-augmented generation pipelines. It uses an AutoML-style approach to test multiple configurations of retrievers, generators, and other components and identifies the best combination for a given dataset. Users provide a dataset and a set of candidate modules, and AutoRAG runs experiments to find the optimal pipeline.

Best for

Best for
Developers building and optimizing custom RAG systems for production or research.

Use cases

• Optimizing RAG pipeline components for a custom dataset
• Benchmarking different retrievers and generators in a unified framework
• Automating hyperparameter search for RAG system deployment

Notes

AutoRAG is an open-source framework that automates the evaluation and optimization of retrieval-augmented generation pipelines. It uses an AutoML-style approach to test multiple configurations of retrievers, generators, and other components and identifies the best combination for a given dataset. Users provide a dataset and a set of candidate modules, and AutoRAG runs experiments to find the optimal pipeline.

4,802 stars on GitHub. Last updated 2026-05-19. Licensed Apache-2.0.

Use cases

• Optimizing RAG pipeline components for a custom dataset
• Benchmarking different retrievers and generators in a unified framework
• Automating hyperparameter search for RAG system deployment

Pros

• Reduces manual trial-and-error in RAG system design
• Supports a wide range of retrievers and generators out of the box
• Provides detailed evaluation metrics and reports

Cons

• Requires significant computational resources to test many configurations
• Limited to the set of supported modules (not infinitely extensible)
• Setup and configuration can be complex for new users

Indexed from awesome-llm and enriched against its public facts.