Module 18 · Detection Engineering — Sigma, ATT&CK Coverage, Validation

What detection engineering is

• Design rules that fire on adversary behaviour, not noise.
• Test rules against historical data and red-team data.
• Tune to acceptable signal-to-noise.
• Deploy with documentation.
• Maintain — update when adversary techniques evolve.

The detection-engineering lifecycle

1. Source: hunt finding, TI report, red-team exercise, ATT&CK coverage gap.
2. Hypothesis: state what the rule should catch.
3. Logic: write the rule in your SIEM / EDR query language.
4. Test against historical: false-positive baseline; expected-positive validation.
5. Test against simulated TTPs: red-team or atomic-test-harness produces the technique; verify rule fires.
6. Tune: filter known-good; narrow scope.
7. Deploy: with severity, response runbook reference, owner.
8. Monitor: post-deploy false-positive rate; tune.
9. Retire: when technique is obsolete or detection moved elsewhere.

Sigma — the universal detection format

Sigma is a YAML-based generic format for SIEM detection rules. Translates to Splunk SPL, Sentinel KQL, Elasticsearch, etc.:

title: Suspicious WMI Lateral Movement
status: experimental
description: Detects WmiPrvSE.exe spawning unexpected child processes
references:
  - https://attack.mitre.org/techniques/T1047/
tags:
  - attack.lateral_movement
  - attack.t1047
logsource:
  category: process_creation
  product: windows
detection:
  selection:
    ParentImage|endswith: '\WmiPrvSE.exe'
    Image|endswith:
      - '\powershell.exe'
      - '\cmd.exe'
      - '\rundll32.exe'
  condition: selection
falsepositives:
  - Legitimate WMI-based administration
level: medium

SigmaHQ on GitHub maintains a community library — hundreds of rules already converted across products.

MITRE ATT&CK as coverage map

Each detection rule maps to ATT&CK techniques. The coverage matrix shows what your detections see:

• Initial Access: techniques you detect.
• Execution, Persistence, Privilege Escalation, Defense Evasion, … each column.
• Heat map: green = strong detection, yellow = partial, red = no detection.
• Tools: ATT&CK Navigator, DeTT&CT, custom dashboards.

Drive red coverage toward yellow; yellow toward green. The map is the strategic view of your detection programme.

The false-positive economics

Every false positive costs analyst time. A rule with 100 alerts/day, 5% true positive rate = 95 wasted analyst-investigations daily. The math:

• If average analyst investigates an alert in 10 minutes: 95 × 10 = 950 minutes/day = 16 analyst-hours.
• That is 2 full-time analysts dedicated to one noisy rule.
• Either fix the rule or accept the cost.

The atomic red team test

Atomic Red Team is a library of small TTP tests. Run a test → verify your detections fire → tune until they do:

# Test: WMI lateral movement via wmic process call create
Invoke-AtomicTest T1047 -TestNumbers 1

# Then check your SIEM — did the Sigma rule above fire?
# If no: rule is broken or telemetry is missing.
# If yes: detection is validated.

The validation cadence

• New rule: validated before deploy.
• Existing rules: monthly atomic-test validation for top-tier coverage.
• Quarterly: full red-team or purple-team validation against high-value techniques.

Common failure modes

• Rules written; never tested; doesn’t actually fire on the technique.
• False-positive rate not tracked per rule; noisy rules dominate alert queue.
• No coverage map; nobody knows what is or isn’t detected.
• Rules tied to one analyst’s understanding; nobody else can maintain.
• Detection logic uses indicators (IPs, hashes) that age out fast; rules degrade silently.

Indian context

• RBI / SEBI: SOC effectiveness is auditor-relevant; detection programme documentation is requested.
• NCIIPC for CII: detection coverage of nation-state-relevant TTPs expected.
• CERT-In threat advisories: each becomes a detection-engineering ticket.

Key takeaways

• Detection engineering: systematic creation, testing, tuning, maintenance of rules.
• Sigma is the universal detection format; SigmaHQ community library is the starting library.
• ATT&CK coverage map: strategic view of detection programme.
• False-positive cost is real; track per-rule.
• Atomic Red Team validates rules against actual TTPs.
• Cadence: validate new rules; quarterly full revalidation.