How to Generate Summaries with Source Citations
Standard summarization tells you what the source says. Cited summarization tells you what it says and exactly where it says it. The difference matters most when the stakes of an incorrect summary are high: a legal brief that misrepresents a ruling, a medical summary that overstates a drug's efficacy, a financial analysis that attributes a figure to the wrong quarter. In these domains, uncited summaries are professionally useless regardless of how well-written they are.
Step 1: Number Your Source Segments
Before sending the source to the LLM, pre-process it by splitting it into numbered segments. The numbering scheme gives the model a citation vocabulary, a set of reference identifiers it can use to ground each claim.
Paragraph numbering (most common): Split the source into paragraphs and prepend each with a sequential number: "[1] First paragraph text... [2] Second paragraph text..." This is simple, works for any document format, and produces fine-grained citations. Most paragraphs contain 2-5 sentences, so a paragraph-level citation narrows the verification scope to roughly 50-150 words.
Sentence numbering (maximum granularity): Number every sentence individually. This produces the most precise citations but doubles the token overhead from numbering prefixes and makes the source harder for the model to read naturally. Use sentence-level numbering only when precision is critical and the source is short enough that the numbering overhead does not push you past context window limits.
Section numbering (for structured documents): If the source has clear headings and sections, number the sections: "[Section 1: Introduction] Text... [Section 2: Methodology] Text..." This produces coarser citations but aligns with how the document was authored, making verification faster for readers familiar with the document's structure.
Page numbering (for PDFs and paginated content): When working with PDF sources where users will verify against the original paginated document, tag content with page numbers: "[Page 12] Text... [Page 13] Text..." This lets users jump directly to the cited page in the PDF viewer.
Token overhead: Paragraph numbering adds approximately 3-5 tokens per paragraph (the bracket, number, and bracket). For a 50-paragraph document, this is 150-250 extra tokens, under 1% of a typical document's total. The overhead is negligible and well worth the citation capability it enables.
Step 2: Prompt for Inline Citations
The summarization prompt must explicitly instruct the model to cite sources and define the citation format. Without explicit instructions, models occasionally produce citations spontaneously but inconsistently.
Effective citation prompt:
"Summarize the following numbered document in 300 words. For every factual claim, statistic, name, or conclusion in your summary, include an inline citation referencing the paragraph number(s) where that information appears. Use the format [N] or [N, M] for claims supported by multiple paragraphs. Every sentence in your summary must have at least one citation. Do not include any claim that cannot be traced to a specific paragraph in the source."
Citation density instruction matters: If you say "include citations where appropriate," the model will cite sporadically, maybe 30-40% of claims. If you say "every sentence must have at least one citation," citation coverage jumps to 85-95%. The explicit requirement forces the model to ground every statement, which also reduces hallucination because claims that cannot be cited get dropped from the summary.
Multiple citation formats:
Inline numbered: "Revenue grew 23% year-over-year [4], driven primarily by the services division [7, 8]." This is compact, familiar from academic writing, and easy to parse programmatically.
Inline quoted: "Revenue grew 23% year-over-year (source: 'Services revenue reached $4.2B, up from $3.4B in the prior year period'). Direct quotes provide immediate verification without checking the source but make the summary longer.
Footnote style: Claims are numbered with superscripts, and a footnote section at the end maps each superscript to the source paragraph. This keeps the summary clean but requires scrolling to verify.
For machine-consumed summaries (feeding into RAG systems, memory stores), use inline numbered citations because they are the easiest to parse and store as structured metadata. For user-facing summaries, use inline numbered with an option to expand citations to see the source text.
Step 3: Verify Citations Programmatically
LLMs sometimes produce incorrect citations: the claim is valid but attributed to the wrong paragraph, or the citation number does not correspond to any paragraph (a hallucinated reference). Post-processing verification catches these errors before they reach users.
String matching verification: For each citation [N] in the summary, extract the claim text and the cited paragraph text. Check whether key terms from the claim appear in the cited paragraph. If the summary says "revenue grew 23% [4]" and paragraph 4 contains "23%" and "revenue," the citation passes. If paragraph 4 discusses product launches with no mention of revenue, the citation fails.
Natural language inference (NLI) verification: Use an NLI model (or LLM-as-judge) to assess whether the cited paragraph entails the claim. This is more robust than string matching because it handles paraphrasing: if the summary says "costs decreased" and the source says "the company achieved significant cost reductions," NLI correctly identifies this as entailment despite the different wording. NLI verification catches 90-95% of incorrect citations compared to 60-70% for string matching alone.
Batch verification pipeline: Parse the summary to extract (claim, citation_number) pairs. For each pair, retrieve the cited paragraph. Run NLI or string matching. Report results as: verified (claim supported by cited paragraph), unsupported (claim not found in cited paragraph), or orphan (citation number does not exist in the source). Flag unsupported and orphan citations for correction.
Verification cost: Running an NLI check on each citation pair using a small model (DeBERTa-v3-large for NLI, or Claude Haiku as judge) costs under $0.001 per citation. A 300-word summary with 10 citations costs approximately $0.01 to verify fully. At scale (100,000 summaries per month), verification adds $1,000 monthly, a fraction of the summarization cost itself.
Step 4: Handle Citation Failures
When verification detects an incorrect citation, you have three options depending on your accuracy requirements and latency budget.
Option A: Regenerate the claim. Send the specific claim back to the model with the correctly identified source paragraph and ask it to rewrite the claim with the correct citation. This fixes the citation while preserving the summary's structure. Cost: one additional API call per failed citation.
Option B: Remove the claim. Drop the unsupported claim from the summary entirely. This guarantees every remaining claim is verifiable but may leave the summary incomplete if a key point had an incorrect citation. Use this approach when you need zero tolerance for unsupported claims (legal, medical contexts).
Option C: Flag for human review. Mark the claim with a warning indicator and present it to a human reviewer. This is the appropriate approach for semi-automated workflows where humans are expected to review summaries before they go to stakeholders.
Iterative correction: If more than 30% of citations fail verification, the entire summary is likely low quality. Regenerate it from scratch with a more explicit prompt rather than patching individual citations. High citation failure rates usually indicate the model is generating content from its training data rather than the source document, and patching cannot fix this fundamental problem.
Citation Quality Across Models
Citation accuracy varies significantly across models. In production benchmarks, Claude Sonnet and GPT-4o produce correct citations on 85-92% of claims when prompted explicitly. Claude Haiku and GPT-4o-mini drop to 70-80% accuracy, still usable but requiring more verification passes. Smaller open-source models (Llama 3 8B, Mistral 7B) struggle below 60% citation accuracy, making them unsuitable for cited summarization without extensive fine-tuning.
The key variable is not model size alone but instruction following. Models that reliably follow the citation format instructions produce more verifiable citations. Test your specific model/prompt combination on 20-30 representative documents and measure citation accuracy before deploying to production.
Storing Cited Summaries for Retrieval
When storing cited summaries in a memory or retrieval system, store the citations as structured metadata alongside the summary text. A memory record might contain: summary text (for embedding and display), citation map (claim index to source paragraph mapping), source document reference (document ID for retrieving the original), and verification status (whether each citation passed verification).
This structure lets downstream systems show users the summary with clickable citations that expand to show the source text, building trust in the AI system's outputs. For RAG applications, the citation map also serves as a built-in fact-checking mechanism: when the RAG system returns a cited summary in response to a query, the user can verify the answer against the original source without re-reading the entire document.
Cited summarization numbers source paragraphs before sending to the model, prompts for inline citations on every claim, then verifies each citation programmatically using NLI or string matching. This produces auditable summaries where every statement traces back to a specific source location. Expect 85-92% citation accuracy from frontier models with explicit prompting, and add verification to catch the remaining errors before they reach users.