Prompt Injection

An attack in which content controlled by a user or third party is interpreted by the model as an instruction and diverts the application's intended behavior. Direct injection arrives through the user's own input; indirect injection sits in pages, documents, or tool outputs that the system retrieves and adds to context. The mechanism exploits the weak boundary between instructions and data inside a token sequence.

Message hierarchy and filters reduce exposure but do not form a security boundary by themselves. Tool-using systems need least privilege, authorization outside the model, argument validation, and human approval for consequential effects. External content should remain explicitly untrusted, with guardrails, traces, and adversarial evaluations covering exfiltration, goal diversion, and tool misuse.