ML Security
The model is an attack surface, not just a service
ML security extends classical security into the model's own attack surface: adversarial inputs that fool the learned decision, poisoning that corrupts data and weights, and extraction that steals the model. Protect the tract, the weights, the pipeline, and the platform.
What you will learn
- Map ML-specific threats onto the standard safety model
- Categorize adversarial, poisoning, and extraction attacks
- See where classical controls stop and ML controls begin
What is this?
ML security treats the model as attackable software with a twist: its 'logic' is learned, so attacks target the input distribution (adversarial perturbations), the training data (poisoning), the weights (theft/extraction), and the supply chain — requiring controls classical security doesn't cover.
Why do you need it?
- Adversarial inputs can flip real decisions — a stop sign misread as a yield sign, but in dollars and safety.
- Poisoning corrupts future versions even when the current one is clean.
- Extraction turns your expensive model into a competitor's free clone.
- The ML pipeline adds third-party code and data — a new supply-chain surface.
Mental model
A vault with open policy, plus a guard who believes forged IDs
ML security
The vault (platform) has deadbolts; but let an attacker forge one convincing ID (adversarial input) and the guard (model) opens the vault. And if the guard's training manual was written by an infiltrator (poisoned data), the guard is loyal to the wrong master. Secure both the walls and the guard's beliefs.
How it works
- Enumerate assets: data, weights, model service, pipeline, artifacts.
- Threat-model per asset: inputs (adversarial), training (poisoning), weights (extraction), chain (software/data).
- Apply classical controls: network, identity, secrets, quotas, monitoring.
- Apply ML controls: robust training, input sanitization, anomaly detection on queries, watermarking, integrity checks.
- Test continuously: playbooks, evasion suites, and audits keep posture current.
ML attack surface map.
In production
- A fraud model receives adversarially-shaped legitimate-looking requests probing threshold edges.
- A chatbot faces prompt injection and privacy probing alongside its content problem.
- A vision product ships robustness checks because evasion in the field flips real decisions.
Best practices
- Threat-model ML artifacts like any crown jewel — data access, weight access, serving access.
- Put input/anomaly checks in front of the model service, not just route authentication.
- Monitor query patterns for extraction signatures (batch-like similarity probing).
- Version-pin and sign the ML supply chain — base images, package locks, data snapshots.
Common mistakes
- Classical-only security: WAF and IAM, but nobody hardened the model's input surface.
- Open model endpoints 'behind the VPN' treating the trained weights as disposable.
- Ignoring data-level controls — a poisoned training run ships as 'improved accuracy'.
- No tamper detection on artifacts; a swapped model file goes unnoticed until behavior 'improves' oddly.
Debugging tips
- Accuracy 'improves' mysteriously after a retrain — inspect the training window; suspect poisoning/maintenance.
- Decisions shift for near-identical inputs — run an adversarial-minimal-perturbation probe.
- A surge of similar queries — that's extraction behavior; tighten rate limits and query anomaly detection.
Interview checkpoint
Quick quiz
An attacker tuning inputs to force a wrong label is…
ML security is fully covered by classical controls when the endpoint is firewalled.