RAG
Intermediate 13 min read RAG

Grounding & Factuality

Every claim with a receipt

Grounding means every claim traces to a chunk. It's not just retrieval plus a prompt — it's explicit design: force citation, gate on support, and refuse when the corpus has no answer.

groundingfactualitycitationshallucinationno-answersupport
Prerequisites: Reranking for Precision
Before we start

What you will learn

  • Structure prompts that force grounded answers
  • Design no-answer behavior
  • Distinguish supported vs unsupported generation
Introduction

What is this?

Grounding is the discipline of making the model answer only from retrieved chunks. It combines prompt contracts ('answer only from context; if absent say unknown'), retrieval that actually covers the claim, and output-side checks (citations, support scoring, refusal). Grounded systems still hallucinate when context is thin — the answer is design, not hope.

Motivation

Why do you need it?

  • Unfettered fluency is the single biggest trust killer in AI products.
  • Refusal beats confident fabrication — a product decision, not an add-on.
  • Citations give humans a verification path and debuggability.
  • Grounding lets you catch failures in evaluation, not only in user complaints.
Think about it

Mental model

Imagine

A witness who can point to sources

Think of

Grounding

'Yes, the policy changed — here's clause 4.2, page 3' (grounded). 'Yes, the policy changed' with no source (ungrounded). A witness who refuses to testify on matters outside their materials is the trustworthy variant.

Mechanics

How it works

  • Prompt: 'Answer only from the context. If context lacks the answer, say you don't know.'
  • Retrieve enough, well-enough ranked chunks to actually cover the claim.
  • Force citations: ask the model to reference chunk ids per claim.
  • Validate: claims must (a) cite a chunk and (b) be contained therein.
  • Refuse: no satisfying context → the 'unable to answer' path.

Grounded answer contract.

Chunks in
Prompt rule
Cite
Verify
Refuse
Real world

In production

  • Legal support bots cite the clause id and refuse anything not in the contract.
  • Medical triage extras restrict claims to retrieved guidelines and state limitations.
  • Insurance responses auto-flag any span without a corresponding chunk for human review.
Guidelines

Best practices

  • Front-load the grounding rule in the system prompt; reinforce at the end.
  • Require chunk ids; verify citations point at real, containing chunks.
  • Ship a not-found answer: same tone, no fabrication.
  • Evaluate groundedness on a labeled claim set, not vibes.
Watch out

Common mistakes

  • Putting 'be grounded' in the prompt and calling it done — no verification.
  • Retrieving 100 chunks so context becomes noise and support breaks.
  • Answering 'the docs don't say' as 'yes, based on general knowledge'.
Root cause analysis

Common issues

Error
Model invents policy details
Root cause

Grounding rule weak + unsupported chunks

Fix

Reinforce prompt rule, reduce chunk noise, add support-check, verify citations.

Error
Refuses even when context has the answer
Root cause

Containment too strict or chunks misaligned

Fix

Loosen matching (semantic containment), improve retrieval, keep refusal for valid gaps.

When things break

Debugging tips

  • Confident wrong policy → citation check: is the cited chunk actually the source?
  • 'I can't find' when the answer exists → retrieval coverage; check top-k and rerank.
  • Citations point to irrelevant chunks → containment validation is broken or misattributed.
Interviews

Interview checkpoint

Test yourself

Quick quiz

0/2
1

A grounded answer is one that…

2

Refusing to answer is always a product failure.

Go deeper

Official documentation

RAG grounding (Anthropic)
docs