NLP Evaluation Metrics

A family of instruments used to compare a text output with one or more references. BLEU emphasizes n-gram precision with a penalty for short outputs; ROUGE covers overlap- and recall-oriented variants; METEOR aligns terms with resources such as stemming and synonymy; Exact Match requires equality under a declared normalization. These signals are fast, deterministic, and useful when the expected form and content are constrained.

Open-ended tasks allow valid paraphrases and semantically correct answers with little lexical overlap. Embedding metrics, BERTScore, and model-based evaluators add semantic sensitivity at the cost of greater expense, variance, and evaluator dependence. Selection should follow the failure mode: lexical signals for exactness and cheap regression, semantic signals for meaning, and human review to calibrate the criterion. No single metric constitutes an eval.