Hierarchical Summarization for Books and Large Corpora

Step 1: Segment the Document into Structural Units

The quality of hierarchical summarization depends heavily on how you segment the source. Unlike map-reduce chunking (which uses arbitrary token-count boundaries), hierarchical segmentation respects the document's own structure.

For books: Use the table of contents as your segmentation guide. Chapters are your primary divisions. Within each chapter, identify sections (often marked by subheadings, scene breaks, or topic shifts). A typical nonfiction book with 15 chapters and 3-5 sections per chapter produces a three-level tree: 50-75 section summaries at the leaves, 15 chapter summaries at the middle level, and 1 book summary at the root.

For legal documents: Contract sections, case law opinions (holdings, reasoning, facts, procedural history), and regulatory filings have formal structural markers. Split at these markers. A 200-page SEC filing might segment into 40-60 sections: risk factors, financial statements, management discussion, legal proceedings, each with its own summary.

For technical manuals and documentation: Split at heading levels. H1 headings define major divisions, H2 defines sections within divisions, H3 defines subsections. This natural hierarchy maps directly to summary tree levels.

Automated segmentation when structure is absent: For unstructured text (transcripts, raw notes, scraped web content), use topic segmentation algorithms. TextTiling (Hearst 1997, still effective) detects topic boundaries based on vocabulary shifts. LLM-based segmentation asks the model to read a section and identify where the topic changes. Combine with paragraph boundary detection for clean segment breaks. Target segment sizes of 1,000-3,000 tokens for leaf-level granularity.

Step 2: Generate Leaf-Level Summaries

Process every smallest structural unit through a summarization prompt. These leaf summaries are the foundation of the entire tree, so their quality matters more than at any other level. Information lost at the leaf level cannot be recovered by higher-level reduction.

Target length: 200-500 words per section, depending on section length and complexity. Use a compression ratio of 15-25% for leaf summaries, more generous than the 10-15% typical for flat summarization. This preserves enough detail for the intermediate reduction layers to work with.

Consistent prompt across all leaves: "Summarize this section in 300 words. Preserve all named entities, specific numbers, conclusions, and claims. This summary will be combined with summaries of other sections to build a hierarchical understanding of the full document. Maintain the section's key relationships and arguments."

Parallelization: Leaf-level summaries are fully independent and can be processed in parallel. A 15-chapter book with 60 sections takes 60 parallel API calls, completing in the time of a single call (2-5 seconds) rather than 2-5 minutes sequentially. This is the same parallelization benefit as map-reduce, and it is critical for production throughput when processing hundreds or thousands of documents.

Quality check per leaf: After generating all leaf summaries, spot-check 5-10% by comparing them against the source sections. Verify that key entities appear in the summary. If a section discusses a specific court ruling (Smith v. Jones, 2024) and the summary mentions only "a recent ruling," the compression is too aggressive and the prompt needs entity preservation instructions strengthened.

Step 3: Build Intermediate Summary Layers

Group leaf summaries by their parent structural unit (chapter, part, or topic cluster) and reduce each group into a single intermediate summary. This is where the hierarchy takes shape.

Chapter-level reduction: Take all section summaries within a chapter (typically 3-8 summaries totaling 1,000-4,000 words) and produce a chapter summary of 300-600 words. The reduction prompt should emphasize synthesis: "Below are summaries of consecutive sections within a single chapter. Synthesize them into one coherent chapter summary. Identify the chapter's main argument or narrative arc. Preserve the most important entities, evidence, and conclusions from each section. Eliminate redundancy where sections overlap."

Multi-level intermediates: For very large documents (multi-volume works, complete legal case files with thousands of pages), you may need more than one intermediate level. Group chapters into parts, summarize parts, then combine part summaries into the final root. Each additional level compresses by 70-80%, so two intermediate levels handle corpora of millions of tokens.

Cross-reference preservation: Documents often reference earlier sections ("as discussed in Chapter 3," "consistent with the findings in Section 4.2"). The intermediate reduction step should resolve these references. Include this in the prompt: "If a section summary references content from another section, include the referenced content directly rather than preserving the cross-reference."

Narrative vs analytical structure: For narrative documents (novels, case studies), chapter summaries should follow chronological order and maintain character/event continuity. For analytical documents (research papers, reports), chapter summaries should maintain logical argument structure, showing how evidence builds toward conclusions. Match your reduction prompt to the document type.

Step 4: Produce the Root Summary

The root summary combines all chapter-level summaries into a single document overview. This is the summary a user sees when asking "what is this book about?" and it must be self-contained, coherent, and representative of the full work.

Target length: 500-1,500 words for the root summary, depending on document complexity. A 300-page business book can be adequately represented in 500-800 words. A 1,000-page legal case file with multiple parties and proceedings needs 1,000-1,500 words.

Root prompt strategy: "Below are chapter summaries of a complete [book/document/report], presented in order. Produce a comprehensive summary of [target length] words that: (1) States the document's central purpose, thesis, or narrative. (2) Covers each chapter's key contribution without simply listing chapters. (3) Identifies the most important conclusions, findings, or outcomes. (4) Preserves the names of key people, organizations, and specific results mentioned across chapters."

Avoid chapter-listing syndrome: The most common failure mode for root summaries is producing a list ("Chapter 1 discusses X. Chapter 2 covers Y. Chapter 3 examines Z."). This is a concatenation, not a synthesis. Explicitly instruct the model to organize information by theme rather than by chapter order, and to identify cross-chapter themes that unify the work.