Can ACT-R Run in Real-Time Production Systems

Why the Concern About Performance

ACT-R was originally implemented as a research simulation in Common Lisp, running cognitive models that process one event at a time with precise timing predictions. This simulation-oriented design gives the impression that ACT-R is computationally expensive. When you read about ACT-R models that take seconds to process a single retrieval cycle, it is natural to wonder whether the approach can work in production systems that need sub-second response times.

The key distinction is between simulating the full cognitive architecture (which includes timing models, buffer management, and production matching) and extracting the retrieval scoring equations (which are simple mathematical functions). For AI memory systems, you need the scoring equations, not the full simulation. The equations themselves are computationally trivial.

Where the Time Goes

A cognitive scoring retrieval call has four computational phases:

The total cognitive scoring overhead (phases 2 through 4) is 6 to 24 milliseconds on top of the vector search. The vector search itself is the same operation you would do without cognitive scoring, so the net additional latency from the cognitive scoring layer is in the 10 to 25 millisecond range for typical configurations.

Optimization Strategies

Precompute Base-Level Activation

Instead of computing base-level activation from the full access history at retrieval time, precompute the value and store it alongside each memory. Update the precomputed value incrementally whenever the memory is accessed (add the new access event's contribution to the running sum). Run a background process periodically to apply decay (recompute activation values based on elapsed time since last computation). This reduces retrieval-time computation to a single database lookup per candidate.

Cache Entity Graph Lookups

Spreading activation requires traversing the entity graph, which involves looking up entity nodes, their connected memories, and their neighboring entities. Cache these lookups in memory (using a hash map or in-memory graph structure) rather than querying a database for each traversal. For graphs with fewer than 100,000 entities, the entire graph fits comfortably in application memory.

Limit Candidate Set Size

Cognitive scoring operates on the candidate set returned by vector search, not the full memory store. If vector search returns 50 candidates, cognitive scoring runs 50 activation calculations, 50 graph lookups (with caching), and 50 score blends. Reducing the candidate set to 20 items cuts the scoring time proportionally while still providing enough candidates for reranking to surface the best results.

Skip Spreading Activation for Ultra-Low Latency

Spreading activation is the most expensive component because it requires graph traversal. For applications with strict latency budgets (under 10 milliseconds total overhead), you can disable spreading activation and use only base-level activation and vector similarity. This reduces cognitive scoring to a simple weighted multiplication (similarity * activation_weight + precomputed_activation * base_weight), adding under 2 milliseconds. You lose the contextual connection benefits, but you retain the recency and frequency advantages.

Scaling Behavior

Cognitive scoring latency scales with the candidate set size, not the total memory store size. Whether your store has 1,000 or 1,000,000 memories, the cognitive scoring phase operates on the same number of candidates (determined by the top-N parameter of the vector search). This means scoring latency remains constant as the memory store grows, which is essential for production systems that accumulate data over time.

The entity graph does grow with the memory store, which can increase spreading activation lookup time. However, with in-memory caching, graph lookups are O(1) hash table operations regardless of graph size. The graph traversal at depth 2 involves a fixed number of lookups per query entity, independent of total graph size.

Production Deployment Considerations

Adaptive Recall runs cognitive scoring on every retrieval call in production. The architecture separates the scoring computation from the storage layer, so scoring latency does not depend on database query time. The precomputed activation values are stored alongside each memory record and updated asynchronously. The entity graph is cached in memory and refreshed when new memories are added.

For applications where retrieval is part of an LLM pipeline (the common pattern where retrieval feeds context into a prompt), the 15 to 40 millisecond scoring overhead is negligible compared to the LLM inference time (typically 500 to 3000 milliseconds). The cognitive scoring adds less than 2% to total response time while significantly improving the quality of the context that the LLM receives.