Module 2 · OSINT Collection for CTI

Last updated: April 29, 2026

Open-Source Intelligence (OSINT) is the practice of collecting information from publicly available sources — no hacking, no paid access required. For CTI analysts, it fills 60-80% of the picture at zero marginal cost. This module covers the tools, techniques, tradecraft, and operational security a researcher needs to do OSINT safely and effectively.

What OSINT covers

• Surface web — search engines, social media, company websites, press releases
• Technical data — DNS records, WHOIS, SSL certs, passive DNS
• Code / infrastructure — GitHub, Docker Hub, PyPI, cloud metadata
• Dark web / paste sites — forums, marketplaces, leaked data dumps
• Archive / historical — Wayback Machine, archive.today, cached versions
• Adjacent data — business records, court filings, regulatory disclosures

Core tool categories

Search engine operators

site:linkedin.com "security engineer" "target company"
intitle:"index of" "backup" site:target.com
filetype:pdf "confidential" site:target.com
-site:target.com "target company" "password"
cache:target.com/admin            # Google's cached version

Google dorks still work but are rate-limited. Rotate search engines (Bing, DuckDuckGo, Yandex, Baidu) — different indices surface different results.

Technical infrastructure

• Shodan — every internet-exposed service, indexed by banner/port/cert. Core tool
• Censys — similar to Shodan with stronger certificate/protocol search
• BinaryEdge — another competitor, strong leak data
• SecurityTrails — historical DNS, subdomain enumeration
• crt.sh — free certificate transparency search (subdomain discovery via issued certs)
• dnsdumpster.com / ViewDNS.info — DNS and reverse-DNS lookups
• Censys certs — search certificates by organization, SAN, fingerprint

Subdomain enumeration

# Free, passive
subfinder -d target.com
assetfinder target.com
amass enum -passive -d target.com

# Cert-based
curl -s "https://crt.sh/?q=%.target.com&output=json" | jq -r '.[].name_value' | sort -u

# Combine with active resolution
subfinder -d target.com | dnsx -resp -a -silent

Social / people

• LinkedIn — employee enumeration. Sales Navigator if budget; otherwise manual scraping
• Have I Been Pwned — email breach history
• Hunter.io / Snov.io — email pattern inference
• Maltego — graph-based OSINT pivot tool; commercial + community editions
• OSINT Framework (osintframework.com) — directory of tools organized by category

Code and leaks

• GitHub dorks: "target.com" password, "target.com" api_key, org:target-inc extension:env
• GitLeaks / TruffleHog — scan git history for leaked secrets (run against the target’s public repos)
• pastebin.com + DuckDuckGo — site:pastebin.com "target.com"
• Grep.app — fast code search across millions of repos

Historical / archive

• Wayback Machine — historical snapshots of web pages. Find deleted content, old API endpoints
• archive.today — similar; sometimes captures what Wayback misses
• Google cache — short-term but sometimes surfaces same-day deletions

Breach data

• Dehashed, IntelligenceX, BreachForums (commercial or careful browsing) — leaked credentials by domain
• DeHashed API is the commercial version — for research, often integrated into CTI workflows

OpSec for researchers

Researchers sometimes touch adversary infrastructure, dark web forums, or closed communities. OpSec:

• Dedicated research machine / VM — never mix with personal or employer assets
• VPN or Tor for sensitive browsing
• Sock puppet accounts — persistent, aged, believable identities. Do not reuse across investigations
• Browser fingerprint hygiene — anti-fingerprint extensions, separate profiles per investigation
• Payment — when paid services are needed, use research-budget cards not personal
• Leave no artifacts on research machine that identify you or employer

Professional CTI firms have formal tradecraft guidelines. Individual researchers benefit from reading them before starting.

Structured workflow — investigating a threat actor

1. Start from a single indicator — IP, domain, hash, handle
2. Passive DNS: what else has this IP hosted? What resolved to this domain in the past?
3. Certificate transparency: what other domains share this cert / cert issuer pattern?
4. Shodan: what services on this IP? Banners reveal infrastructure
5. Code search: does this infrastructure appear in public repos? Leak reports?
6. Pivot: identified adjacent infrastructure; repeat steps 2-5 for each
7. Enumerate adversary persona: look for matching handles on forums, GitHub, crypto wallets, Keybase
8. Document graph: Maltego or a simple diagram capturing the pivots
9. Report: what did you find, what’s your confidence, what are the gaps

Time discipline

OSINT is a tar pit. You can spend days going down a rabbit hole. Counter:

• Time-box investigations upfront (one week, two weeks)
• Define the exit criteria — what would constitute “answered enough”
• Daily standup of findings — force articulation of progress
• Document dead-ends as you hit them; save future time

Legal and ethical considerations

• Accessing dark-web forums may violate terms or — depending on jurisdiction — be restricted. Check local law
• Do not actively participate in criminal activity to gain access. Read-only observation is different from paying for stolen data
• Do not expose identities of researchers or informants in published reports
• Do not contact adversaries directly without legal counsel involvement
• India: CERT-In directives and IT Act provisions apply; work with legal before publishing anything naming Indian entities

Automating OSINT

Tools that orchestrate:

• SpiderFoot — open source, 200+ modules, produces a unified report from a seed
• Recon-ng — CLI modular OSINT framework
• theHarvester — email/subdomain enumeration wrapper
• Harpoon — Python toolkit for analysts
• OpenCTI — open-source threat intelligence platform with built-in OSINT connectors

Applying OSINT to CTI

The goal is not volume. It is answering specific questions:

• “Does this ransomware group target organisations of our size in our region?”
• “Is our executive team mentioned in dark-web chatter?”
• “What infrastructure is this threat group currently using?”
• “Has our source code been leaked publicly?”

Each question drives a specific collection plan, which drives specific tools. Avoid the “collect everything” trap — it generates noise and misses the signal.

Where this leads

With OSINT skill, you can characterize adversaries and infrastructure before they hit you. Module 3 covers the Pyramid of Pain and IOC lifecycle — the framework for prioritising which indicators to hunt for and when.