Perplexity

A metric that summarizes how surprising a sequence is to a language model, derived from the probability it assigns to the observed tokens. Lower perplexity means the model assigned higher probability to the text under that tokenization and distribution.

It is useful for comparing probabilistic fit under equivalent conditions, but it does not by itself measure usefulness, factual accuracy, safety, or user satisfaction. Comparisons across tokenizers, corpora, languages, or different objectives can mislead. In AI applications, an evaluation tied to the user outcome is usually more informative than perplexity alone.