Model extraction (a.k.a. model stealing) lets you reconstruct a model from query access alone. You query the target API with carefully chosen inputs, collect outputs, train your own model on the dataset. The resulting student model approximates the teacher’s behaviour — for many tasks within 5% of teacher quality. Cost is genuinely low; legal status is murky.
Why extraction works
Knowledge distillation has been studied since 2015 (Hinton et al.). Train a student model to match teacher’s output distribution; the student often achieves near-teacher performance with a fraction of parameters. Originally used to compress large models into small ones for the same team. Becomes “extraction” when the student team is not the teacher team. For LLMs, the attack works because: (1) APIs return logits or top-K probabilities (richer signal than just argmax); (2) training datasets at extraction-feasible scale (millions of queries) cover the input distribution; (3) modern open-source base models (Llama-3-8B, Mistral-7B) are good starting points to fine-tune from, reducing data needed.
Book a free 30-minute scoping call
Our senior consultants will review your stack and tell you honestly what to fix first. No slide deck. No obligation. Indian businesses only.