System Prompts
The product's constitution, versioned
The system prompt is your product's operating manual: identity, rules, policies, output contract. Seen as code, it gets versioning, testing and rollback like any other artifact.
What you will learn
- Author a durable system prompt
- Version and rollback prompt changes
- Test system prompts across model upgrades
What is this?
The system prompt encodes the product's standing behavior: persona, tone, constraints, tool policies, refusal posture and output contract. Because it ships inside every call, changes are deployments — version them like code, gate with regression tests, and roll back when a bump to a model misbehaves with it.
Why do you need it?
- One consistent constitution beats ad-hoc per-call instructions scattered everywhere.
- Versioned prompts make model upgrades predictable (or roll back their regressions).
- Prompt drift is the quiet killer of product quality across releases.
- A well-tested system prompt minimizes the prompt-engineering debt operators pay later.
Mental model
An airline's in-flight manual
The system prompt
Every flight uses the same manual: how crew speaks (tone), what's banned (constraints), how to handle turbulence (refusal posture). The airline revises it with training, approval and a rollback path — not seat-of-the-pants edits during flights.
How it works
- Compose identity → rules → outputs → failure behaviors as compact sections.
- Store as a versioned file in the repo; config selects the active version.
- Run a prompt regression suite on every change and on model upgrades.
- Monitor behavior drift in prod; revert via config, not a scramble.
- Keep dynamic context OUT of the system prompt unless it's small and static.
Prompt as a release artifact.
In production
- Support copilots carry escalation + safety policy in system.
- Agent systems define tool permission and refusal rules there.
- Regulated products pin policy wording in the system prompt for audits.
Best practices
- Keep it structured and fairly short; put bulk data in retrieval, not system.
- Write rules as explicit conditionals ('if X then Y').
- Add a clear fail-safe clause: what to do when uncertain.
- Keep a changelog; tie changes to evals passed.
Common mistakes
- Stuffing megabytes of reference content into the system prompt.
- No versioning — behavior drifts and nobody can say what changed.
- Ignoring model upgrades as prompt-change events.
- Writing policies as vague prose when explicit conditionals would do.
Common issues
Brand-new model ignores the system prompt's old rulesRule phrasing incompatible with new model behavior
Re-run the regression suite and restructure; keep the old model for rollback.
User content overrides system rulesNo system-user hierarchy defense
Confirm the system lane and add explicit non-inheritance statements.
Debugging tips
- Rules wax and wane with user turns → ruler moves: re-assert in system + end anchor.
- One prompt 'works' on the old model only → run the suite before upgrading.
- Long system prompt + short user context = hierarchy tilt; watch proportions.
Interview checkpoint
Hands-on lab
- Estimate the token cost of system-prompt variants
- Keep a structural rubric the reply must pass
- Choose the cheapest variant that still conforms
Budget a System Prompt
25 min · beginner
Quick quiz
A system prompt change should ship like…
Model upgrades never affect how system prompts behave.