What Is GraphRAG and How It Works

How Traditional RAG Works (and Where It Fails)

Traditional RAG follows a straightforward pipeline: the user asks a question, the question is embedded into a vector, the vector database returns the most similar document chunks, and those chunks are included in the LLM's prompt as context. The LLM then generates an answer grounded in the retrieved content. This works well when the answer exists in a single document that uses similar vocabulary to the question.

RAG fails in three common scenarios. First, multi-hop questions require combining information from multiple documents that are not individually similar to the query. "What is the backup strategy for the database that the payments service uses" requires finding the payments service, discovering it uses PostgreSQL, and then finding PostgreSQL's backup documentation. No single chunk answers the full question, and the intermediate chunks (about PostgreSQL) may not be similar enough to the query (about payments backup) to rank highly in vector search.

Second, broad summarization questions like "describe the architecture of our platform" require synthesizing information across dozens of documents. Vector search returns the 5 or 10 most similar chunks, which covers only a fraction of the architecture. Important components that are described in dissimilar language are missed entirely.

Third, entity-specific questions like "who maintains the authentication service" require finding a specific fact about a specific entity. Vector similarity is a blunt tool for this because it ranks by overall semantic similarity rather than by the presence of the specific entity-attribute pair.

What GraphRAG Adds

GraphRAG addresses these failures by adding a parallel retrieval path through a knowledge graph. The graph stores entities (services, people, technologies, concepts) and the typed relationships between them (depends_on, maintained_by, uses). When a query arrives, GraphRAG identifies the entities mentioned, looks them up in the graph, and traverses their relationships to find connected entities and their associated documents.

For the multi-hop question about payments backup, GraphRAG follows the chain: payments service (entity) -> uses (relationship) -> PostgreSQL (entity) -> documented_in (relationship) -> PostgreSQL backup guide (document). This path exists in the graph regardless of vocabulary overlap between "payments backup" and the PostgreSQL documentation.

For the broad summarization question, GraphRAG can use community detection to identify clusters of densely connected entities that represent architectural components, then retrieve the community summaries for each cluster. This provides comprehensive coverage that vector search cannot achieve.

For the entity-specific question, GraphRAG looks up "authentication service" as an entity, follows the "maintained_by" relationship, and returns the connected Person entity directly. No vector similarity needed.

Two Approaches to GraphRAG

Entity-Centric GraphRAG

The entity-centric approach extracts entities from the query, finds them in the graph, and retrieves content associated with their neighbors (entities connected by one or two relationship hops). This is the more common approach and the one that most implementations use. It works well for entity-specific and multi-hop queries because the traversal follows explicit relationships. It is fast (graph lookup plus a few hops) and easy to implement. The limitation is that it requires the query to mention at least one known entity as a starting point for traversal.

Community-Based GraphRAG

The community-based approach, introduced in Microsoft Research's 2024 GraphRAG paper, pre-computes community summaries from the knowledge graph. Community detection algorithms (like Leiden) identify clusters of densely connected entities, and an LLM generates a summary for each cluster. At query time, the system retrieves the community summaries most relevant to the query. This approach excels at broad, open-ended questions because it provides thematic overviews rather than individual entity details. The trade-off is higher preprocessing cost (summarizing every community) and the risk that pre-computed summaries become stale.

The GraphRAG Pipeline

A full GraphRAG pipeline has four stages:

Indexing stage: Process source documents to extract entities and relationships, build the knowledge graph, optionally compute community summaries, and create the vector embeddings that feed traditional RAG. This stage runs during ingestion, not at query time.

Query processing stage: When a query arrives, extract entities from it, link them to graph nodes, and determine the query type (entity-specific, multi-hop, broad). This stage adds 100 to 300 milliseconds of latency but enables the graph retrieval path.

Retrieval stage: Run vector search and graph traversal in parallel. Vector search returns semantically similar chunks. Graph traversal returns chunks associated with entities connected to the query entities. Fuse the results using weighted scoring or reciprocal rank fusion.

Generation stage: Pass the fused retrieval results to the LLM as context, along with the original query. The LLM generates an answer grounded in both vector-retrieved and graph-retrieved content. The graph-retrieved content provides the structural connections that fill in the gaps vector search leaves.

When GraphRAG Matters

GraphRAG provides the most value when your queries involve relationships between entities, when your knowledge base has dense interconnections, and when you need to answer multi-hop questions. In benchmarks, GraphRAG improves recall by 15 to 30% on multi-hop queries compared to standard RAG. On simple, single-topic queries, the improvement is minimal because vector search already handles those well.

The decision to adopt GraphRAG depends on the mix of query types your application handles. If 80% of queries are simple lookups, the infrastructure cost of a knowledge graph may not be justified. If 30% or more of queries involve entity relationships or require multi-step reasoning, GraphRAG will noticeably improve answer quality.

Adaptive Recall implements entity-centric GraphRAG as a core retrieval strategy. Entities are extracted automatically during memory storage and added to a knowledge graph. During recall, spreading activation traverses entity connections to boost structurally related memories alongside vector similarity results. This gives you the retrieval quality of GraphRAG without building or maintaining separate graph infrastructure.