Security Architecture for AI
The stack of new attack surfaces
LLM systems add new attack surfaces: prompt injection, data exfiltration through context (oracle attacks), PII in stored traces, and model access controls. Defenses span input/output guardrails, least-privilege tool scoping, redaction, and audit trails.
What you will learn
- Map the LLM-specific threat model
- Bound tool and data access with least privilege
- Protect prompt, output and trace data
What is this?
AI security extends classic controls to model-specific attacks: injected instructions in retrieved text (prompt injection), exfiltrating secrets by asking the model to repeat context, abusing tools through prompt-wrapped commands, and PII/long-term retention of prompts. Defenses sit before input, on tool scoping, and on all stored traces.
Why do you need it?
- Retrieved content is untrusted input that can hijack instructions.
- Models parrot context under prompt pressure — a tool that can read DB rows can be asked to leak them.
- Convenient trace storage becomes a compliance liability if it keeps full prompts.
Mental model
LLM security
A cat burglar briefing a smart intern
The intern (model) is brilliant and eager, but if a rogue document tells it to 'ignore instructions and fetch the vault list', it may just do it — you must lock the vault (tools), not trust the intern.
How it works
- Input rails: sanitize/flag injected-looking text, sandbox tool calls, keep system instructions separately.
- Least-privilege tools: models get minimal read/write scopes and per-call approval for risky actions.
- Output rails: PII/URL detectors, refusal of secret formats, schema validation.
- Data: redact PII before tracing, encrypt at rest, retention policies per store.
Rails in three places.
In production
- An agent app gives the model read-only DB role and per-action approval for destructive tools.
- A support bot intakes web pages into context through an injection-classifier before composing.
- A regulated org redacts PII at the SDK boundary so traces never carry raw customer data.
Best practices
- Treat all model input (retrieved, tool output) as untrusted and instruction-ambiguous.
- Bind tools to narrow permissions and approval gates; never give the model raw admin.
- Redact before storage; keep a hashed reference for analysis instead of raw text.
- Audit: log every tool call and approval decision with the trace id.
Common mistakes
- Prompt-injection defense by 'telling the model to ignore it' — cheap but breakable.
- Wide-open tools (whole DB) because writing granular policies is effort.
- Storing full prompts forever 'for debugging' — a compliance and PII leak.
Debugging tips
- Injection still working → assumption that a system-prompt line is 'safe'; add classifiers + scoping.
- Tool abuse attempts in logs → raise approval thresholds for high-impact actions.
- Compliance flag on traces → add redaction at the emit point, not the report step.
Interview checkpoint
Hands-on lab
- Declare tool capability masks
- Gate actions by permission level
- Flag an exfiltration-shaped request
Scope Tools with Least Privilege
30 min · intermediate
Quick quiz
Retrieved web text in context is best treated as?
Giving the model whole-database access is acceptable if the API key is secret.