Browser-Use Agents — Risks When LLMs Browse the Web

Manish Garg
Manish Garg Associate of (ISC)² · RingSafe
Apr 29, 2026
9 min read
Read as
Anthropic computer-use Claude, OpenAI Operator, and frameworks like browser-use let agents control real browsers. They click, type, fill forms, log in. Every webpage is now an attack surface against the agent. This module covers the documented attacks (visual prompt injection, deceptive UI, exfiltration via form submission) and the patterns that make browser agents safe-ish to deploy.

Browser agents are the highest-capability LLM applications shipping in 2026. They can do anything a human can do in a browser. That includes visit attacker-controlled pages and follow malicious instructions. The defences are immature. Most teams should not deploy browser agents to production yet; understanding why is the first step.

How browser agents actually work

The agent receives a screenshot of the browser. Vision model interprets it. Reasoning step decides next action (click, type, scroll, navigate). Action is executed. New screenshot fed back. Loop until task complete. Variants: DOM-based (read HTML directly, less robust to layout but cheaper), accessibility-tree-based (Microsoft research), pure vision (Anthropic computer-use). Capabilities: log into sites with stored credentials, fill multi-step forms, extract structured data, automate workflows. Real value: tedious knowledge work (data entry, research, monitoring competitor pricing).

Need help with this?

Book a free 30-minute scoping call

Our senior consultants will review your stack and tell you honestly what to fix first. No slide deck. No obligation. Indian businesses only.

Book scoping call Replies in 4 working hrs · India-only · Senior consultants