RAG Evaluation

A specialization of evals that decomposes a RAG system into three connected levels. Retrieval measures whether the required evidence appears and where it ranks, using signals such as precision@k, recall@k, MRR, or NDCG. Generation examines faithfulness, correctness, completeness, and attribution accuracy. The end-to-end level checks usefulness under cost and latency limits.

The final score alone cannot locate the failure. A retriever may return the right evidence while the model ignores it; a correct answer may appear despite poor context. Versioned cases, relevance ground truth, verifiable claims, and component-level traces make it possible to attribute the problem before changing the model, embedding, or search strategy.