When to Use Graph Memory vs Vector Memory

What Vector Memory Does Well

Vector memory stores each memory as an embedding, a high-dimensional numeric representation of its semantic content, and retrieves by finding the embeddings nearest to the query embedding. This is semantic search: the system finds memories that mean something similar to the query, regardless of exact vocabulary.

Vector memory excels at three types of queries. First, open-ended topical queries: "what do we know about authentication" finds memories about auth, login, SSO, OAuth, JWT, and access control, even though those memories may not share any exact keywords with the query. The embedding model captures semantic relationships that keyword search would miss. Second, fuzzy recall: "something about that meeting last week where we discussed the API" does not need to match exact terms; vector similarity finds memories that are conceptually close to this description. Third, cross-domain connections: a query about "improving system performance" might retrieve memories about caching strategies, database optimization, and code profiling, because the embedding model understands that all of these relate to performance.

Vector memory struggles with four types of queries. First, entity-specific lookups: "what do we know about the Acme Corp account" does not perform well with pure vector search because the embedding for "Acme Corp account" is semantically similar to many account-related memories, not specifically Acme Corp memories. Vector search conflates semantic similarity with entity relevance. Second, relationship queries: "what is connected to the authentication service" requires traversing connections, which vectors cannot represent. Vector similarity can find memories that mention authentication, but it cannot find memories that are connected through a chain of relationships (authentication connects to the user service, which connects to the billing service, which has a known issue). Third, exact-match requirements: when the user asks for a specific fact, vector search may return semantically similar but factually different results. Fourth, negation and constraint queries: "memories about APIs but not REST APIs" cannot be expressed as a single vector search because embeddings do not represent logical operators.

What Graph Memory Does Well

Graph memory stores memories as nodes and their relationships as edges, creating an explicit network of connections. Retrieval involves traversing this graph: starting from a known entity and following connections to discover related information.

Graph memory excels at four types of queries. First, entity-centric retrieval: "everything about Acme Corp" starts at the Acme Corp node and follows edges to find all connected memories, issues, contacts, and events. This is precise in a way that vector search is not, because it follows explicit connections rather than semantic similarity. Second, multi-hop reasoning: "what problems have affected customers who use the API integration" requires traversing from "API integration" to connected customers, then from those customers to connected problems. Each hop follows an explicit relationship, and the query can span any number of hops. Third, relationship discovery: "how is this error related to the deployment last Tuesday" can be answered by finding a path through the graph that connects the error node to the deployment node, revealing the chain of events or system connections. Fourth, aggregate queries: "which entities appear in the most memories" or "which topics are most connected to customer complaints" are graph analytics questions that are natural in graph databases but impossible in vector stores.

Graph memory struggles with three types of queries. First, open-ended semantic queries: "find memories about improving things" is too vague for graph traversal because there is no specific entity to start from and no specific relationship to follow. Graph traversal needs an entry point and a direction; semantic fuzziness is not its strength. Second, queries with no entity context: if the user's query does not mention any entity that exists in the graph, there is nowhere to begin traversal. Third, discovery of new topics: graph traversal finds memories connected to known entities, but it cannot discover memories about topics that are not yet in the graph. New information that has not been entity-extracted and connected cannot be found through graph queries.

The Complementary Pattern

Vector and graph memory complement each other because their strengths and weaknesses are nearly opposite. Vector search is strong where graph traversal is weak (open-ended semantic queries, discovery of new topics, fuzzy recall) and weak where graph traversal is strong (entity-specific lookups, relationship queries, multi-hop reasoning). This complementary relationship is why most production memory systems at scale use both.

The combined retrieval pattern works as follows. Query analysis determines whether the query is entity-focused (has a specific entity or relationship to explore), semantically-focused (asking about a topic or concept), or mixed (asking about a topic in the context of a specific entity). Entity-focused queries route primarily to graph traversal with vector search as a supplementary strategy. Semantically-focused queries route primarily to vector search with entity extraction on results to enable follow-up graph queries. Mixed queries run both strategies in parallel and fuse results using reciprocal rank fusion or similar techniques.

Adaptive Recall implements this combined pattern. Vector search finds semantically relevant memories while the knowledge graph provides entity traversal and relationship-aware retrieval. ACT-R cognitive scoring ranks the combined results using recency, frequency, and confidence alongside semantic similarity, producing results that are both semantically relevant and contextually connected.

Decision Criteria

Use the following criteria to decide which approach your application needs.

Choose vector-only if your queries are primarily semantic (find information about a topic), your memory content is mostly unstructured text without clear entities, your application does not need relationship tracking, and your memory volume is moderate (under 100,000 memories). Vector-only architectures are simpler to build and operate, and for many applications they are sufficient.

Choose graph-only if your queries are primarily entity-centric (find everything related to entity X), your memory content is highly structured with clear entities and relationships, your application needs multi-hop reasoning, and you have a reliable entity extraction pipeline. Graph-only architectures are rare in practice because most applications have at least some semantic query needs.

Choose combined vector and graph if your queries span both semantic and entity-centric patterns, your memory volume is large enough that vector search alone returns too many marginally relevant results, you need relationship-aware retrieval to surface connections that semantic similarity would miss, or you need to support multi-hop questions about how entities are related. The combined approach is more complex to build and operate, but it produces measurably better retrieval quality for applications with diverse query patterns.

Implementation Considerations

If you choose a combined architecture, the entity extraction pipeline becomes critical. Every memory must be processed to extract entities and relationships that feed the graph. The quality of entity extraction directly determines the quality of graph retrieval. Missed entities create blind spots in the graph; false entities create noise. Invest in entity extraction quality before investing in graph query optimization, because a perfect graph query against an incomplete graph returns incomplete results.

The graph also needs maintenance. As memories are consolidated, updated, or deleted, the corresponding graph nodes and edges must be updated too. Orphaned nodes (entities with no connected memories) and stale edges (relationships that no longer reflect current information) degrade graph query quality over time. Include graph maintenance in your lifecycle management processes alongside memory consolidation and archival.