Top-P (Nucleus Sampling)

A sampling method that selects the smallest set of tokens whose cumulative probability reaches P and samples within it. Unlike Top-K, the set size changes with model confidence: a concentrated distribution yields few candidates; an uncertain distribution admits more alternatives.

Top-P regulates diversity locally but does not replace quality criteria. A high P can expand useful options or noise; a low P can remove needed phrasing. Temperature, Top-K, and Top-P act on the same decoding stage, so they should be tuned through controlled experiments rather than stacked by habit.