Prompt Injection Defense Architecture: Beyond Input Filtering

Why input filtering cannot be the primary control

Prompt injection is not a fixed signature; it is any natural-language text that redirects a model’s behaviour. The input space is unbounded and the adversary is adaptive: paraphrase, translate into another language, encode in base64, split across turns, hide in a modality the classifier does not read, or embed in retrieved content the filter never sees. A classifier trained on today’s payloads is a speed bump, not a boundary. Treat input filtering as defence-in-depth — useful for cutting noise and raising cost — but architect as though it will be bypassed, because against a motivated attacker it will be.

Principle: separate the privileged plane from untrusted text

The durable insight is that the component reasoning over untrusted text must not be the component holding privilege. In most vulnerable designs a single LLM both reads attacker-influenced data and decides which privileged tools to call — so subverting its reasoning subverts your authorisation. The fix is architectural: keep the ability to act out of reach of the text that can be poisoned.

Pattern 1 — the dual-LLM / quarantine split

Run two logically separate models. A privileged model plans and issues tool calls but never sees raw untrusted content. A quarantined model processes untrusted data (documents, web pages, tool results) and can only return structured, typed values — never free-form instructions — back to the privileged planner. The planner treats the quarantined model’s output as opaque data: it can store it in a variable and pass it to a tool, but it cannot let that content redirect the plan. Modern refinements (capability-passing designs such as CaMeL) extend this by attaching provenance and a capability to each value, so a piece of data carries the authority to be used only in specific ways.

Pattern 2 — capability-based, least-privilege tools

Design tools so the damage a subverted agent can do is bounded by construction, not by the model’s good behaviour. Prefer narrow, typed tools (refund_order(order_id, amount≤cap)) over broad ones (run_sql(query)). Authorise every tool call against the end user’s entitlements, not the agent’s service identity, to close the confused-deputy path. Make irreversible actions require a capability the untrusted plane cannot mint. The question to ask of every tool is: “if the model calling this is fully attacker-controlled, what is the worst outcome?” — and then shrink that worst case.

Pattern 3 — deterministic guardrails on the action layer

Put non-model, deterministic checks between the agent’s decision and the real world. Allowlist the destinations a fetch tool may reach (defeats exfiltration to attacker servers). Enforce hard limits — spend caps, row-count limits, rate limits — in code, not prompt. Require structured, schema-validated arguments for every tool so the action layer can reason about what is being done without trusting model-authored prose. These controls do not care whether the model was injected; they constrain the outcome either way, which is exactly the property filtering lacks.

Pattern 4 — provenance, spotlighting, and output constraints

Where untrusted data must enter a model’s context, mark it: delimit it unambiguously, label its source, and use spotlighting so the model is explicitly told the span is data to analyse rather than instructions to obey. On the way out, constrain generation to the shape you expect (structured output, schema enforcement) so a hijacked model cannot smuggle instructions or secrets into a free-text field. None of these are sufficient alone, but layered on top of privilege separation they raise the cost of every remaining attack.

Assembling the layers

A defensible LLM application composes all of the above: input filtering to cut noise; a quarantine split so untrusted text never reaches the privileged planner; capability-based least-privilege tools authorised against the real user; deterministic guardrails (allowlists, caps, schemas) on the action layer; provenance and spotlighting on ingested data; and monitoring that alerts on anomalous tool sequences. Note the pattern — the controls that actually hold are the deterministic, architectural ones; the model-based controls are supporting cast. Test the result with the methodology in the agentic red-teaming module, and enforce it at the protocol boundary with the MCP server security module.