The Limits of Vector Similarity for Retrieval

The Information Loss Problem

An embedding model compresses a piece of text (which might be a sentence, a paragraph, or an entire document) into a fixed-length vector, typically 768 or 1536 dimensions. This compression necessarily loses information. A 200-word paragraph contains thousands of bits of information about content, tone, specificity, authority, and structure. The embedding preserves the semantic gist but discards details that might matter for retrieval quality.

Consider two memories: "I think the timeout might be around 30 seconds" and "The request timeout is configured to 30000ms in production.conf at line 142, last verified on April 3." Both discuss the same topic with similar meaning, so their embeddings are close in vector space. But the second memory is far more useful: it is specific, precise, verifiable, and recent. The embedding model compresses away the differences in specificity and authority that would help a retrieval system rank the better answer first.

This information loss is not a bug in current models that future models will fix. It is an inherent consequence of fixed-dimensional compression. You can increase the embedding dimensions (from 768 to 1536 to 3072), and this preserves more nuance, but it can never preserve everything. The aspects that get preserved are optimized for semantic similarity, not for retrieval quality factors like authority, specificity, or temporal validity.

The Temporal Blindness Problem

Vector similarity has no concept of time. An embedding of "our API rate limit is 100 requests per minute" produced in January 2025 is identical to an embedding of the same text if it were somehow produced in May 2026. The embedding encodes what the text says, not when it was written or whether it is still true. A retrieval system using only vector similarity cannot prefer newer information over older information, even when the newer information explicitly supersedes the older version.

In practice, this means that every time a fact changes, the old and new versions compete equally in retrieval. If your product has changed its pricing three times, all three pricing memories score similarly for a "what is the pricing" query. The user might get any of them, and the system provides no signal about which is current. The problem is not that the old memories are retrieved (they are topically relevant), but that they are ranked as highly as the current memory.

Some teams work around this by manually deleting old memories when facts change, but this requires knowing which facts have changed and finding all related memories, a task that scales poorly. Others add metadata filters (only return memories from the last 30 days), but this throws out valid historical knowledge along with the stale data. Cognitive scoring handles temporal relevance through activation decay, which is more nuanced: recently accessed memories rank higher naturally, and unused memories fade gradually rather than being abruptly deleted or filtered.

The Contradiction Blindness Problem

Vector similarity cannot detect contradictions between results. "The API supports JSON and XML" and "The API only supports JSON, XML was deprecated in v3" are both relevant to a query about API data formats. They score similarly on cosine similarity. But returning both to an LLM for answer generation produces confusion: the model might hedge ("the API supports JSON and possibly XML") or pick one at random, potentially choosing the wrong one.

Contradiction detection requires comparing documents against each other, not just against the query. This is fundamentally outside the scope of vector similarity, which scores each document independently against the query vector. A retrieval system that handles contradictions needs an additional mechanism to identify conflicting statements and prefer the more reliable or more recent one.

Cognitive scoring addresses this through the consolidation process, which periodically reviews memories for contradictions. When a contradiction is detected, the older or less corroborated memory loses confidence, which reduces its ranking score in future retrievals. The system does not need to delete the old memory; it just needs to rank it lower so it does not compete with the correct information.

The Convergence Problem

As a vector index grows, the nearest neighbors to any query become increasingly similar in distance. In a collection of 1,000 documents, the top result might have a cosine similarity of 0.92 while the tenth result scores 0.78, a clear difference. In a collection of 100,000 documents, the top result might score 0.93 while the tenth scores 0.89. In a collection of 1,000,000 documents, the difference between the top 20 results might be less than 0.02.

This convergence means that at scale, vector similarity stops providing meaningful ranking signal among the top results. The first and twentieth results are effectively tied, and the ranking order within that cluster is essentially arbitrary. Any small perturbation, like re-embedding with a slightly updated model or adding a few words to the query, reshuffles the order within the cluster without improving quality.

Adding a second scoring dimension breaks this tie. Even a simple factor like access frequency provides enough signal to differentiate between candidates that vector similarity cannot distinguish. Cognitive scoring adds four additional dimensions, providing robust ranking differentiation even in very large collections where similarity scores have converged.

The Exact Match Problem

Semantic embedding models are trained to capture meaning, which sometimes works against retrieval when users search for exact terms. A search for "error code E_CONN_REFUSED" is looking for that specific string, not semantically similar text. An embedding model might rank a document about "connection errors and timeouts" higher than one that mentions E_CONN_REFUSED specifically, because the embedding captures the broad topic rather than the exact identifier.

This is why hybrid search (combining BM25 keyword matching with vector semantic search) outperforms either method alone. BM25 ensures exact matches on specific terms rank highly, while vector search ensures paraphrases and conceptually related documents are also found. Reciprocal rank fusion or weighted combination blends the two score types into a single ranking.

Cognitive scoring adds a third layer on top of hybrid search. After BM25 and vectors have identified the candidate set, cognitive scoring reranks by recency, confidence, and entity connections. This three-layer approach (keyword recall, semantic recall, cognitive ranking) produces the most robust retrieval quality across different query types.

The Single-Dimension Problem

The deepest limitation of vector similarity is that it provides a single dimension of scoring. No matter how good the embedding model is, cosine similarity produces one number per candidate. That one number must encode every aspect of relevance: topical match, answer quality, specificity, authority, currency, and reliability. It cannot do all of these well because they are independent dimensions that require independent measurements.

Human relevance judgment is inherently multi-dimensional. When an expert evaluates whether a search result is useful, they consider: Is it about the right topic? Does it actually answer my question? Is the information current? Is the source reliable? Is it specific enough to act on? These are separate assessments that often trade off against each other (a very current result might be less specific, a very specific result might be outdated).

Multi-factor scoring systems like cognitive scoring separate these dimensions and measure each independently. Vector similarity handles topical relevance. Base-level activation handles currency and usage validation. Confidence weighting handles reliability. Spreading activation handles contextual relationships. Combining them through weighted addition allows the system to balance these factors rather than collapsing them into a single score.