Model Theft and Extraction Attacks

Last updated: April 26, 2026

ML models are valuable IP. Models trained at significant compute cost can be approximately replicated — “stolen” — by querying the deployed API and using responses as labels for training a substitute model. Model-theft attacks have implications for proprietary AI products. This article covers the techniques and defences.

The mechanism

Attacker queries victim’s API with diverse inputs. Records (input, output) pairs. Trains a substitute model to mimic — copying the intellectual property without paying training cost.

# Example: image classifier
# Attacker generates 100K random images
# Queries victim API for labels
# Trains a CNN on (image, label) pairs
# Substitute model approximates victim's accuracy

Why it matters

• Training a frontier LLM costs $10M-$100M+ in compute. A substitute that captures 90% of capability via 100K queries costs much less.
• Once substitute is trained, attacker can use it without paying the API; can analyse it offline (find weaknesses, adversarial examples)
• For competitive AI products, theft directly threatens business model

Variants

Functional theft

Substitute approximates the original’s predictions. Sufficient for most commercial purposes.

Architecture / hyperparameter theft

Side-channel attacks (timing, memory) reveal model architecture details.

Membership inference

Different attack — given a sample, determine whether it was in the training set. Privacy implication when training data was sensitive (medical, personal).

Model inversion

Reconstruct training data from model outputs. Particularly for face recognition, can recover representative training-set images.

Defences

• Rate limiting — bound queries per user / IP / time
• Output perturbation — return slightly noisy outputs that preserve top-1 accuracy but degrade for substitute training
• Watermarking — embed hidden patterns in model behaviour that prove provenance if a substitute is found
• Behavioural analysis — detect query patterns indicative of extraction (high diversity, large volume from single user)
• Tiered access — high-volume access requires verified business identity
• Differential privacy in training — limits information leakage per query

The trade-off

Defences trade off accuracy / latency / cost. Heavy rate limiting hurts legitimate users. Output perturbation reduces accuracy. Watermarking is invisible until needed but doesn’t prevent theft, only prove it.

Most production deployments accept some extraction risk in exchange for service quality. Watermarking + rate limiting + behavioural detection is the typical stack.

The legal angle

• Model theft via API querying isn’t always illegal — depends on Terms of Service violation, copyright theory, trade-secret arguments
• OpenAI v. various lawsuits on this topic; case law evolving
• For commercial deployments, ToS clauses prohibiting model extraction provide some recourse

The takeaway

Model theft is a real risk for valuable AI products. Defences trade off user experience for protection. The combination of rate limiting + behavioural detection + watermarking provides reasonable protection without crippling legitimate users. For Indian AI startups developing proprietary models, these protections are part of go-to-market planning, not afterthought.