Improper Output Handling
Treating LLM output as trusted input to downstream systems. Render LLM output as HTML and you get XSS. Pass it to a shell and you get RCE. Insert it into SQL and you get injection. The LLM is an untrusted source; act accordingly.
01What it is
A category of vulnerabilities that arise when LLM output is consumed by a downstream component (browser, shell, database, API) without proper validation or escaping. This is not new — it is classic injection theory applied to a new untrusted source.
02Why it matters
Engineers anthropomorphise LLMs. They write code as if the model is a trusted collaborator. But the model has just processed attacker-controlled input via prompt injection; whatever it outputs may have been steered. Every output channel needs the same escaping discipline as user input.
03Attack vectors
- LLM-generated HTML rendered without sanitisation → XSS.
- LLM-generated SQL passed to a database without parameterisation → SQL injection.
- LLM-generated shell commands executed verbatim → RCE.
- LLM-generated URLs followed by an agent without allowlist → SSRF.
- LLM-generated Markdown with `` → XSS via rendered preview.
- LLM-generated JSON that downstream code `eval`s instead of `JSON.parse`s.
04Defence patterns
- Treat LLM output exactly like user input. Apply the same OWASP Top 10 controls.
- Use structured output (JSON schema, tool-calling) and validate against schema before use.
- Sanitise Markdown with a trusted library (DOMPurify, bleach) before rendering.
- Sandbox any code the model proposes — never execute in the main process.
- Output allowlists — for agents that call URLs, predefine the set.
- CSP — defence-in-depth against rendered XSS even when sanitisation fails.
05Detection
Signals to watch
CI tests that fuzz the LLM-output rendering path. WAF rules on responses, not just requests. Monitor for unusual control characters or HTML tags in LLM responses.
06India context
DPDP · RBI · CERT-In
For BFSI customer-facing AI, RBI directions on application security apply uniformly — LLM output is an output channel and inherits all the same controls. CERT-In incident classification covers XSS / SQLi / RCE without exception for "the LLM said so".