Securing the ML Platform
The vault walls around the ML estate
Platform security controls who and what can touch data, weights, training runs, and endpoints — network isolation, IAM, secret hygiene, encryption, quotas, and audit. It's the spine every other ML control hangs from.
What you will learn
- Secure network, identity, and secrets across the ML platform
- Isolate tenants and training workloads
- Keep audit trails that survive incidents
What is this?
Securing the ML platform is applied infrastructure hardening: segmentation and network policy, least-privilege IAM for humans, services, and pods; secret management; encryption at rest/in transit; resource quotas; and machine-read audit log coverage across storage, compute, and serving.
Why do you need it?
- Every ML control above lives inside this spine — no gates, no claims.
- Untrusted workloads (training containers, notebooks) increase blast radius when unisolated.
- Secrets in artifacts are the fastest path to a full-model-compromise.
- Audit logs are the difference between 'we fixed it' and 'we know what happened'.
Mental model
A bank with vaults, badges, and cameras — and four locks on the boiler room
Platform security
The branch (UI) is open with good badges; but the vault room, the data hallways, and the training floor each have their own locks and cameras. Badges (IAM) stop at the wrong door, tapes (audit) survive the night, and the keys rotate. It's boring, and that's the point.
How it works
- Network: private subnets for data/storage; egress filtering for training/serving; no public dataset volumes.
- Identity: app-roles per service and per human, least-privilege; short-lived creds.
- Secrets: dedicated manager with rotation; no secrets in repos, notebooks, artifacts, or logs.
- Encryption: KMS-managed keys at rest and TLS everywhere in transit.
- Isolation: namespaces/tenants with quotas; resource limits stop neighbor blast radius.
- Audit: log access to data, weights, deployments, and serve-time traffic anomalies.
Layer model of platform security controls.
In production
- A bank's training cluster: training nodes on private network, GPU egress allowlisted, notebooks in isolated namespaces, weights encrypted with per-team keys.
- An ML platform teams-data segmented by namespace with resource quotas — a runaway job can't starve a neighbor.
- Serving endpoints sit behind a WAF-plus-model-level ACL; audit log joins serve-time anomaly signals.
Best practices
- Segment by data-class and workload-class before building anything pretty.
- Least-privilege + rotation everywhere: humans, services, pods.
- No-secrets-in-artifacts as a hard rule; secret scanner in merge gates.
- Encrypt sensitive storage and all traffic; log access machine-readably to enable replay analysis.
Common mistakes
- Shared 'convenience' bucket everyone writes to — model tampering via a data path.
- Training pods able to reach the model registry and production databases — a pivot dream.
- Secrets pasted into notebooks; audits show every AI-adjacent person can read S3+X.
- Resource limits absent — one tenant OOMs the shared cluster and eats production serving.
Debugging tips
- Unexplained data-access spikes — correlate with IAM and audit logs; a leaking token appears as a single identity.
- Cross-namespace blast radius — your network policy is too loose; test with a deny-by-default emulation.
- Rotation breaks a cron job for a week — that's the point; document token-aware deploys.
Interview checkpoint
Quick quiz
The fastest path from a compromised training container to the model registry is usually…
Network segmentation can contain a compromised training workload's blast radius.