Ragas
Evaluate retrieval and generated answers using test datasets and metrics for RAG applications.
Published by Vibrant Labs
How to use it
For each test question, retain the retrieved passages, generated answer, and reference answer when available. Choose metrics for retrieval and answer quality separately so a bad result can be traced to the right stage.
Run the same questions when changing the embedder, retriever, or prompt. Document which scoring metrics use a judge model or embedding provider and keep their configuration fixed across runs.
- Compatibility
- Python, Model APIs
- Evaluation areas
- RAG evaluation, Test datasets, Model-graded scoring
- Access
- Local Python evaluator. Judge and embedding requirements vary by metric.