Prompt injection

Ordinary software separates code from data structurally: a parameterised query cannot be talked into becoming a different query. A language model has no such boundary. Your instructions and the document it was asked to summarise occupy the same context, and instructions that appear inside that document can influence what it does next.

The risk scales with capability. A model that only writes a summary can be made to write a misleading summary. A model with tools can be made to use them — which is why injection matters far more for agents than for chat.

Assume every retrieved document, web page, email and tool result is untrusted, and design so that a hostile one cannot cause harm. Give the model the narrowest permissions the task allows, and keep credentials and capabilities out of reach of anything driven by external content.

Put a human in front of consequential actions rather than relying on the model to refuse. Validate output structurally instead of trusting it. Log tool calls so unexpected behaviour is visible after the fact. These are the same layered defences described under guardrails, applied to a threat that specifically targets the gap between data and instruction.

Treating it as solved by a filter. Detection helps and is not sufficient — an input filter is pattern-matching against an unbounded space of phrasings, and defences that depend on catching every variant are the ones that eventually fail.

The other misconception is that it only affects public-facing products. An internal assistant that reads shared documents, tickets or inboxes is reading content written by people outside the team more often than anyone expects.