Generative AI
Beginner 14 min read Generative AI

Prompt Engineering

The cheapest lever: talk to the model deliberately

Prompts are the input interface to LLMs: system instructions, few-shot examples, and clear structure. Prompt engineering is systematically shaping that interface to get reliable outputs — a cheap, high-ROI skill before any fine-tuning.

promptsystem promptfew-shotchain-of-thoughttemperatureoutput formatin-context learning
Prerequisites: Generative Models Overview
Before we start

What you will learn

  • Structure prompts: system, context, task, output format
  • Use few-shot exemplars and chain-of-thought effectively
  • Tune temperature/top-p as part of prompt behavior, not a separate concern
Introduction

What is this?

Prompt engineering is designing the input so the model's learned distribution leans toward what you want: a clear system role, the user instruction, context/few-shot examples, and an explicit output format. Because LLMs 'learn in context', well-formatted prompts with demonstrations can steer behavior without touching weights — usually the fastest, cheapest quality lever in a GenAI product.

Motivation

Why do you need it?

  • Zero-cost: changes are text edits, deployed instantly, auditable.
  • Few-shot and chain-of-thought dramatically improve reasoning and format adherence.
  • The system prompt defines the product's persona, guardrails and contract.
  • Better prompts can reduce retries, parsing hacks and downstream bugs.
Think about it

Mental model

Imagine

Briefing a meticulous contractor

Think of

Prompting an LLM

Ambiguous instructions produce whatever the contractor assumes. A precise brief — role, context, constraints, 'deliver in this exact format' — plus one example of the finished job, produces the deliverable you intended on the first try.

Mechanics

How it works

  • Role: set persona and behavior in the system prompt ('You are a SQL expert answering from this schema').
  • Context: give the data and any constraints the model needs, placed before the request.
  • Task: be explicit about the action and the output format (JSON fields, no preamble).
  • Few-shot: include 2-5 input→output exemplars of the exact pattern you want.
  • Chain-of-thought: ask the model to reason step by step when the task involves multi-step logic.

Anatomy of an effective prompt.

System
Context
Task
Exemplars
Output spec
Real world

In production

  • Structured extraction: JSON schema in the prompt for stable parsing at scale.
  • Classification/decision routers that call specialized prompts.
  • RAG answers that cite sources: instruct the model to reference chunks by id.
Guidelines

Best practices

  • Put the instruction AFTER any data blocks the model must use — recency bias is real.
  • Specify negative constraints sparingly; the model follows 'do this' better than 'don't do that'.
  • Design output so it's machine-parseable: one JSON per response, validated against a schema.
  • Version prompts like code: store them, diff them, and test them against a regression suite.
Watch out

Common mistakes

  • Over-fitting the prompt to one lucky formulation instead of testing variants.
  • Hiding critical rules at the end of a huge context where attention is diluted.
  • Changing prompt and decoding parameters simultaneously — you can't attribute the win.
  • No regression tests: a new prompt can silently break a production parse.
Root cause analysis

Common issues

Error
Model ignores the requested JSON format
Root cause

Format stated only in prose with no example

Fix

Add a one-line few-shot exemplar of the exact JSON and enforce via response_format when supported.

Error
Inconsistent persona across turns
Root cause

System prompt diluted by user context

Fix

Repeat critical constraints in the user instruction; keep system prompt short and directive.

Error
Answer hallucinates company-specific facts
Root cause

No grounding context or instructions to use supplied data only

Fix

Fetch facts via RAG, instruct 'use ONLY the provided context', and lower temperature.

When things break

Debugging tips

  • A/B at least 3 prompt variants on 20+ fixed cases before committing.
  • Log exact prompts to replay failures — most 'model bugs' are prompt bugs.
  • If output format drifts, add one few-shot example in the exact format rather than restating rules.
  • Test temperature 0 for deterministic tasks; reserve high temperature for creative ones.
Interviews

Interview checkpoint

Test yourself

Quick quiz

0/2
1

Few-shot prompting means:

2

Output format rules work best when accompanied by a concrete example.

Go deeper

Official documentation

OpenAI prompting guide
docs