Autoregressive Model

A model that factorizes the probability of a sequence and predicts each element conditioned on the preceding ones. In a decoder LLM, the selected token is appended to context and influences the next prediction; generation keeps repeating this cycle until a stopping condition is reached.

The local objective of predicting the next token can produce globally coherent text, but it does not verify facts or guarantee a consistent plan. An error also becomes context for later steps and may propagate. Temperature, top-k, and top-p change the sampling path while the autoregressive architecture remains the same.