Induction Heads

Attention circuits associated with recognizing and continuing repeated patterns in context. A canonical behavior is to observe a sequence A followed by B and, when A appears again, increase the probability of B as the continuation. This provides an interpretable mechanism for part of in-context learning.

Evidence is strongly causal in small transformers and more correlational in larger models. Induction heads do not by themselves explain all ICL capability or reveal how a proprietary model solved a specific task. The term belongs to mechanistic interpretability, where hypotheses should be tested through interventions on the circuit.