Module 5 · AI Security & Red Teaming

Manish Garg
Manish Garg Associate of (ISC)² · RingSafe
Apr 25, 2026
5 min read
Read as

Last updated: April 29, 2026

Attack and defend AI systems — the field almost no one teaches. OWASP LLM Top 10, prompt injection, jailbreaks, guardrails, RAG poisoning, model extraction.

AI red teaming is one of the rarest skillsets in Indian cybersecurity — fewer than 200 practitioners can credibly do it. By the end of this module you’ll know the attack categories, the defences, and how to systematically test an LLM application. The capstone: build your own intentionally vulnerable LLM app and pwn it.

OWASP LLM Top 10 — your taxonomy

  1. LLM01 — Prompt Injection: Attacker overrides the system prompt with crafted user input.
  2. LLM02 — Insecure Output Handling: LLM output passed unsanitised to downstream systems (XSS, SSRF, code injection).
  3. LLM03 — Training Data Poisoning: Adversary inserts examples that bias the trained model.
  4. LLM04 — Model Denial of Service: Adversary causes resource exhaustion via expensive inputs.
  5. LLM05 — Supply Chain Vulnerabilities: Compromised pre-trained weights, backdoored fine-tuning datasets.
  6. LLM06 — Sensitive Information Disclosure: Model leaks training data, system prompts, or user PII.
  7. LLM07 — Insecure Plugin Design: Tool/function calls executed without sufficient input validation.
  8. LLM08 — Excessive Agency: Agent has too much permission to act on user behalf.
  9. LLM09 — Overreliance: Application trusts LLM output without verification (legal, medical, financial).
  10. LLM10 — Model Theft: Adversary clones or extracts the deployed model via API queries.

Memorise these — they’re the test categories every audit covers.

Want this for your team?

Custom team training + practitioner advisory

Beyond the free academy — we run private workshops, vCISO advisory, and red-team exercises tailored to your stack. For Indian SMBs scaling past their first hire.

Book team training call Replies in 4 working hrs · India-only · Senior consultants