Memory in the Age of AI Agents

ArXiv: 2512.13564

🎯 Pitch

Agent memory is shifting from static retrieval to fully learned, RL-driven systems that autonomously decide what to remember, consolidate, and forget—yet no single substrate can yet unify text, vision, and action. This survey reveals a three-axis taxonomy of forms, functions, and dynamics, showing that the field’s next leap hinges on resolving the tension between retrieval fidelity and generative abstraction.

---

1. Executive Summary

This survey provides a comprehensive and unified taxonomy of memory in LLM-based agent systems, analyzing the landscape through the three interconnected lenses of forms, functions, and dynamics. Drawing on a broad sweep of recent work evaluated across benchmarks such as LoCoMo and LongMemEval with models like PaLM 2, GPT-4, and various open-weight LLMs, the paper identifies three dominant representational forms—token-level memory (e.g., flat dialogue logs vs. planar knowledge graphs vs. hierarchical multi-layer structures), parametric memory (e.g., internalized model editing vs. external LoRA adapters), and latent memory (e.g., KV-cache reuse vs. generated memory tokens)—and maps them onto three functional roles—factual memory (user profiles, environmental knowledge), experiential memory (case-based trajectories, strategy-based workflows, skill-based code libraries), and working memory (single-turn context compression, multi-turn state consolidation). Critically, the survey demonstrates that memory is not a static store but a dynamic lifecycle of memory formation, evolution, and retrieval, establishing that the field is undergoing a fundamental paradigm shift from hand-crafted retrieval pipelines toward fully learned, RL-driven memory systems that autonomously manage consolidation, updating, and forgetting—but that the inability to unify multimodal signals within a single memory substrate and the unresolved tension between retrieval fidelity and generative abstraction remain open bottlenecks limiting deployment of truly continuous, self-evolving agents.

2. Context and Motivation

The Core Problem: A Fragmented Field Without a Unifying Conceptual Framework

The fundamental problem this survey addresses is not a single technical limitation but a systemic conceptual fragmentation in the rapidly expanding field of agent memory research. As LLM-based agents have evolved from simple prompt-response systems into autonomous, goal-directed entities capable of sustained interaction with complex environments, memory has become increasingly recognized as a foundational capability. The paper argues, however, that this growth has been accompanied by a proliferation of loosely defined terminologies, incompatible evaluation protocols, and a lack of conceptual clarity that hinders both scientific progress and practical deployment.

The paper states this directly in Section 1:

"With the explosive growth of memory-related studies, the concept itself has become increasingly expansive and fragmented. Researchers often find that papers claiming to study 'agent memory' differ drastically in implementation, objectives, and underlying assumptions. The proliferation of diverse terminologies (declarative, episodic, semantic, parametric memory, etc.) further obscures conceptual clarity, highlighting the urgent need for a coherent taxonomy that can unify these emerging concepts."

This fragmentation manifests in several concrete ways. One research group may describe their system as implementing "episodic memory" when it stores raw dialogue logs with timestamps, while another uses the same term to describe a graph-structured representation of event sequences. A third group may argue that what the first two groups are doing is not memory at all, but merely context engineering. Without a shared vocabulary and a systematic framework for comparing these approaches, the field lacks the ability to cumulatively build knowledge — each new paper effectively reinvents its own definitions, making it difficult to determine whether claimed improvements represent genuine advances or merely terminological disagreements.

Why This Problem Is Important

The importance of resolving this fragmentation extends beyond academic tidiness into both theoretical and practical domains.

Theoretical significance. Memory is not an optional add-on for agentic systems; it is what transforms a stateless language model into a persistent, adaptive entity. The paper argues in Section 1 that among the canonical agent capabilities — reasoning, planning, perception, tool-use, and memory — memory plays a distinctive and foundational role:

"memory stands out as a cornerstone, explicitly enabling the transformation of static LLMs, whose parameters cannot be rapidly updated, into adaptive agents capable of continual adaptation through environmental interaction"

Without memory, an agent cannot maintain coherence across multi-turn interactions, cannot learn from its own successes and failures, and cannot build persistent models of users or environments. Understanding how memory should be structured, maintained, and utilized is therefore not a peripheral concern but central to the entire enterprise of building capable agents. A fragmented research landscape means that insights gained in one sub-area often fail to transfer to others, slowing the overall pace of progress toward more general and capable agentic intelligence.

Practical and real-world impact. The paper catalogs numerous application domains where memory is critical, including personalized chatbots (Chhikara et al., 2025; Li et al., 2025b), recommender systems (Liu et al., 2025c), social simulations (Park et al., 2023; Yang et al., 2025), financial investigations (Zhang et al., 2024), and many others. In each of these domains, the difference between a stateless model and a memory-augmented agent can be the difference between a system that repeatedly asks the same clarifying questions and one that remembers user preferences, tracks ongoing tasks, and adapts its behavior over time. However, practitioners face a bewildering array of choices — flat vector stores vs. knowledge graphs vs. parametric fine-tuning vs. latent representations — with little guidance on which approach is appropriate for which scenario. The lack of a unified framework means that system designers often make these choices based on intuition or familiarity rather than principled reasoning about the trade-offs involved.

Additionally, the paper highlights a developmental imperative for the field. As the authors note, "one of the defining aspirations of AGI research is to endow agents with the capacity for continual evolution through environment interactions, a capability fundamentally grounded in agent memory." If the community cannot agree on what memory means or how to evaluate it, progress toward this aspiration will be impeded.

Where Existing Taxonomies Fall Short

The paper is not the first survey on agent memory. It explicitly acknowledges prior surveys such as Zhang et al. (2025s) and Wu et al. (2025g), which have provided valuable overviews. However, it identifies two specific limitations that motivate the need for a new taxonomy:

Limitation 1: Temporal obsolescence. The field has evolved rapidly, and earlier taxonomies were developed before a number of significant methodological advances. The paper points to concrete examples of work from 2025 that are underrepresented in prior classification schemes:

"emerging directions in 2025, such as memory frameworks that distill reusable tools from past experiences (Qiu et al., 2025a,c; Zhao et al., 2025c), or memory-augmented test-time scaling methods (Zhang et al., 2025g; Suzgun et al., 2025), remain underrepresented in earlier classification schemes"

This is not merely a matter of completeness; these newer works represent qualitatively different approaches to memory. For instance, the distillation of reusable tools from past experiences crosses the boundary between experiential memory and skill-based memory in ways that earlier taxonomies — which often treated these as separate categories — cannot cleanly represent. Similarly, memory-augmented test-time scaling blurs the line between retrieval and generation, requiring a more nuanced framework than simple retrieve-vs-generate distinctions.

Limitation 2: Insufficient granularity. The paper argues that traditional taxonomies such as "long-term vs. short-term memory" — derived from cognitive psychology and prevalent in earlier agent memory surveys — are "insufficient to capture the diversity and dynamics of contemporary agent memory systems." The problem is that within a single temporal category, methods can differ radically in their representational form, their functional role, and their operational dynamics. A long-term memory system that stores user profiles as flat text chunks (e.g., MemoryBank) and one that stores them as parametric updates to model weights (e.g., SELF-PARAM) share a temporal label but operate under entirely different constraints and assumptions. The temporal axis alone does not provide enough dimensionality to usefully organize the design space.

Conceptual Confusion with Adjacent Fields

Beyond internal fragmentation, the paper identifies a broader problem: agent memory is frequently conflated with related but distinct concepts, creating ambiguity that makes it difficult to even define the boundaries of the field. Section 2.3 is devoted entirely to disentangling agent memory from three adjacent concepts.

Agent Memory vs. LLM Memory. The paper observes that many works from 2023–2024 describing themselves as "LLM memory mechanisms" — including systems like MemoryBank (Zhong et al., 2024) and MemGPT (Packer et al., 2023a) — are, under contemporary understanding, more appropriately classified as instances of agent memory. This retroactive reclassification arises from the historical ambiguity around what constituted an "LLM agent":

"During 2023–2024, the community had no stable or coherent definition: in some cases, prompting an LLM to call a calculator already sufficed to qualify the system as an agent... in other cases, agency required substantially richer capabilities such as explicit planning, tool use, memory, and reflective reasoning"

However, the paper is careful to note that the subsumption is not absolute. A distinct line of genuinely LLM-internal memory research exists — managing KV caches, designing long-context processing mechanisms like Mamba or RWKV, or modifying attention architectures to retain information as sequence length grows. These works address intrinsic model dynamics rather than agentic behavior, and the paper explicitly places them outside the scope of agent memory. The boundary is functional: agent memory supports a decision-making agent's evolving external memory base, while LLM memory expands the model's internal representational capacity.

Agent Memory vs. RAG. The relationship with Retrieval-Augmented Generation is particularly nuanced. At a technical level, both paradigms involve constructing and querying auxiliary information stores. Both communities use knowledge graphs, vector indices, and semantic search. The boundary has become increasingly blurred as retrieval systems become more dynamic — the paper notes that systems like HippoRAG (Gutierrez et al., 2024) are claimed by both communities as addressing long-term memory challenges.

The paper proposes a pragmatic (though imperfect) distinction based on the task domain: RAG is predominantly applied to augment LLMs with external knowledge for individual inference tasks (e.g., HotpotQA, 2WikiMQA), while agent memory operates in settings requiring sustained multi-turn interaction, temporal dependency, and environment-driven adaptation (e.g., LoCoMo, GAIA, SWE-bench). But the paper acknowledges that even this domain-based distinction contains "substantial gray areas," since many self-described RAG systems implement forms of agentic self-improvement, and many memory systems are evaluated on what are essentially document QA tasks.

The deeper point is that the overlap is not a coincidence — it reflects genuine conceptual convergence as both fields move toward dynamic, agent-controlled information management. Agentic RAG, in which an LLM agent actively controls when, how, and what to retrieve (e.g., Self-RAG, PlanRAG), occupies the closest conceptual space to agent memory. The key remaining distinction is that agent memory maintains an internal, persistent, self-evolving base that accumulates knowledge across tasks, while agentic RAG typically operates over an external, task-specific database.

Agent Memory vs. Context Engineering. The relationship here is described as an "intersection of distinct operational paradigms rather than a hierarchical subsumption." Context engineering treats the context window as a constrained computational resource to be optimized — a resource management paradigm. Agent memory focuses on the cognitive modeling of a persistent entity with an evolving identity. From context engineering's perspective, agent memory is one variable within the context assembly function; from the agent's perspective, context engineering is the implementation layer that ensures cognitive continuity within physical model limits.

The paper observes that these two fields converge significantly in the technical realization of working memory during long-horizon interactions, often employing functionally identical mechanisms (summarization, compression, importance-based selection). However, they diverge in scope: context engineering optimizes the momentary interface between agent and model under resource constraints, while agent memory sustains a persistent cognitive state that extends beyond any single context window.

How This Paper Positions Itself

The paper positions itself not as a narrow technical contribution but as a conceptual foundation for the field. Its stated goal, articulated in Section 1, is to establish a "systematic framework that reconciles existing definitions, bridges emerging trends, and elucidates the foundational principles of memory in agentic systems."

The framework is organized around five key questions, which structure the entire survey:

1. How is agent memory defined and distinguished from related concepts?
2. What architectural or representational forms can agent memory take?
3. What functions does agent memory serve?
4. How does memory operate through its dynamics of formation, evolution, and retrieval?
5. What are the most promising frontiers for advancing agent memory research?

This "Forms–Functions–Dynamics" triangle is the paper's central intellectual contribution as a taxonomy. Rather than organizing prior work along a single axis (e.g., by storage medium, or by task type, or by temporal granularity), the paper treats these three dimensions as orthogonal and complementary. Any given memory system can be characterized by where it falls on each dimension simultaneously. A method like MemoryLLM (Wang et al., 2024j), for instance, would be classified as having a latent representational form, serving a factual memory function, and employing both latent representation (for formation) and model editing (for evolution). This multi-dimensional characterization captures far more of the design decisions and trade-offs than a single-axis taxonomy could.

The paper also explicitly positions itself as forward-looking rather than merely descriptive. Section 7, "Positions and Frontiers," goes beyond cataloging existing work to articulate what the paper sees as paradigm-level shifts currently underway — from retrieval-centric to generative memory, from manually engineered to autonomously managed memory systems, and from heuristic pipelines to RL-driven memory control. This forward-looking stance distinguishes the survey from purely retrospective literature reviews and situates it as an agenda-setting document for the field.

Finally, the paper positions itself as practically useful, compiling resources (benchmarks, open-source frameworks) and providing guidance on when different memory forms are appropriate for different deployment scenarios. The adaptation discussion in Section 3.4 explicitly addresses the question: given the features of each memory type, which tasks and scenarios are they best suited for? This practical dimension ensures that the taxonomy serves not only researchers seeking conceptual clarity but also practitioners making concrete engineering decisions.

3. Technical Approach

3.1 Reader Orientation

This paper is a taxonomic survey — it does not build a single, novel system but instead constructs a conceptual framework for understanding, comparing, and designing memory mechanisms in LLM-based agent systems. The core idea is that existing work on agent memory is fragmented and resists clean comparison because different papers use the same terms to mean different things; the paper's solution is to propose a three-dimensional taxonomy — forms, functions, and dynamics — where every memory method can be characterized along all three axes simultaneously, enabling precise comparison and revealing design trade-offs that are invisible under single-axis taxonomies like "long-term vs. short-term."

3.2 Big-Picture Architecture (Diagram in Words)

The taxonomy has three orthogonal dimensions that jointly characterize any memory system:

1. Forms (Section 3): Where and how is the memory physically stored and represented? This dimension distinguishes three major categories — token-level memory (explicit, addressable, human-readable discrete units stored externally to the model), parametric memory (information encoded in model weights, either internal or through auxiliary parameter modules), and latent memory (implicit representations in the model's internal states, such as KV caches, activations, or generated embeddings).

2. Functions (Section 4): Why is the memory needed? What role does it play in the agent's behavior? This dimension distinguishes factual memory (declarative knowledge about users, environments, and facts), experiential memory (procedural and strategic knowledge accumulated from task execution that enables continual learning and self-evolution), and working memory (transient, capacity-limited workspace for active context management within a single task or session).

3. Dynamics (Section 5): How does the memory operate over its lifecycle? This dimension decomposes memory operation into three sequential processes — memory formation (extracting compact knowledge from raw interaction data via summarization, distillation, structured construction, latent encoding, or parametric internalization), memory evolution (integrating new memories with existing ones through consolidation, updating, and forgetting mechanisms), and memory retrieval (determining when to retrieve, constructing effective queries, executing retrieval strategies, and post-processing results).

Every existing memory method can be placed in this three-dimensional space. For example, MemoryLLM (Wang et al., 2024j) has a latent form, serves a factual function, and uses latent representation for formation with model editing for evolution. MemGPT (Packer et al., 2023a) has a token-level form, serves factual and working memory functions, and uses semantic summarization for formation with time-based forgetting for evolution. This multi-dimensional positioning enables fine-grained comparison that reveals why methods that appear similar along one axis may be radically different along others.

3.3 Roadmap for the Deep Dive

• First, the formal definitions of LLM-based agents and agent memory systems (Sections 2.1–2.2), since these establish the vocabulary and mathematical abstraction — what is an agent, what is memory, and how do they interact — that all subsequent taxonomy relies on.

• Second, the three-way conceptual comparison with LLM memory, RAG, and context engineering (Section 2.3), because defining what agent memory is not clarifies its boundaries and prevents the conceptual confusion that the paper identifies as the field's core problem.

• Third, the Forms dimension (Section 3) — token-level, parametric, and latent memory — because the physical representation determines what operations are possible (e.g., you cannot edit a latent representation with the same transparency as a token-level store) and shapes the constraints for the other dimensions.

• Fourth, the Functions dimension (Section 4) — factual, experiential, and working memory — because once we know what memory can be made of, we need to understand what purposes it serves in an agent's cognitive architecture.

• Fifth, the Dynamics dimension (Section 5) — formation, evolution, and retrieval — because this is where the taxonomy becomes operational, showing how memory actually changes over time and interacts with the agent's decision loop.

• Sixth, the adaptation discussion (Section 3.4), which synthesizes the Forms taxonomy into practical guidance: given the properties of each form, which deployment scenarios and task types are they best suited for?

3.4 Detailed, Sentence-Based Technical Breakdown

This is a taxonomic survey paper whose core intellectual contribution is a multi-dimensional classification framework — Forms × Functions × Dynamics — that provides a shared vocabulary for comparing memory mechanisms and reveals design trade-offs that are invisible under single-axis taxonomies. The paper does not propose a new memory algorithm; instead, it provides the conceptual scaffolding needed to understand the entire design space of agent memory and to identify underexplored regions within that space.

---

Formalizing LLM-Based Agents

The taxonomy is built on a precise mathematical abstraction of what an LLM-based agent is and how memory couples to its decision process. The paper formalizes this in Section 2.1 with a controlled stochastic transition model.

Agent-environment interaction. Let $I = \{1, \dots, N\}$ be the set of agents, where $N = 1$ is the single-agent case and $N > 1$ is the multi-agent case. The environment has a state space $\mathcal{S}$. At each time step $t$, the environment transitions according to:

$s_{t+1} \sim \Psi(s_{t+1} \mid s_t, a_t)$

where $s_t$ is the current environment state and $a_t$ is the action executed at time $t$.

What it computes: the next environment state $s_{t+1}$ is sampled from a distribution $\Psi$ that depends on the current state $s_t$ and the agent's action $a_t$. This captures that the environment may be stochastic and that the agent's actions change the world.

Why this form: modeling the environment as a controlled stochastic process is the standard MDP/POMDP formulation from reinforcement learning. It unifies diverse agent architectures — single-agent reasoning, multi-agent debate, planner-executor hierarchies — under a common abstraction where the key variable is how observations and histories are exposed to each agent.

Agent observations. Each agent $i$ receives an observation:

$o^i_t = \mathcal{O}^i(s_t, h^i_t, Q)$

where $h^i_t$ is the portion of the interaction history visible to agent $i$ (this may include previous messages, intermediate tool outputs, partial reasoning traces, or other agents' contributions), and $Q$ is the task specification (user instruction, goal description, or external constraints, treated as fixed within a task).

What it computes: the observation function determines what each agent "sees" at time $t$, which may differ across agents depending on the system design. A fully observable system gives every agent the complete state; a partially observable system restricts visibility based on roles or communication topology.

Why this form: separating the observation function from the state transition allows the formalism to capture heterogeneous multi-agent setups (e.g., one agent may see tool outputs while another sees only natural-language messages) without changing the underlying environment model.

Action space. The paper emphasizes that LLM-based agents have a heterogeneous action space, which may include natural-language generation, tool invocation (API calls, search engines, code execution), explicit planning (task decompositions, subgoal specifications), environment-control actions (navigation, repository editing), and communication actions (structured messages to other agents). These diverse action types are "unified by the fact that they are produced through an autoregressive LLM backbone conditioned on a contextual input."

Agent policy. Each agent $i$ follows a policy:

$a_t = \pi^i(o^i_t, m^i_t, Q)$

where $m^i_t$ is a memory-derived signal (defined in the next sub-section) that may be present or absent depending on the retrieval schedule. The policy may internally generate multi-step reasoning chains or scratchpad computations before emitting an executable action; such internal processes are abstracted away and not explicitly modeled.

Why this form: conditioning the policy on both the instantaneous observation $o^i_t$ and the retrieved memory $m^i_t$ captures the essential structure of memory-augmented decision-making. The separation between $o^i_t$ (what the agent sees right now) and $m^i_t$ (what it remembers from the past) is the formal expression of why memory matters: without $m^i_t$, the agent is purely reactive and cannot maintain context, track goals, or learn from experience.

Trajectories. A full execution induces a trajectory:

$\tau = (s_0, o_0, a_0, s_1, o_1, a_1, \dots, s_T)$

where $T$ is determined by task termination conditions. At each step, the trajectory reflects the interleaving of environment observation, optional memory retrieval, LLM-based computation, and action execution.

---

Formalizing Agent Memory Systems

Section 2.2 defines agent memory as an evolving state $\mathcal{M}_t \in \mathbb{M}$, where $\mathbb{M}$ is the space of admissible memory configurations. The paper explicitly imposes no internal structure on $\mathcal{M}_t$ — it may be a text buffer, key-value store, vector database, graph structure, or any hybrid representation. This deliberate underspecification ensures the formalism can accommodate the full diversity of memory forms described in Section 3.

Memory lifecycle: three conceptual operators. The dynamics of the memory system are characterized by three operators that form a cycle:

1. Memory Formation. At time step $t$, the agent produces informational artifacts $\phi_t$ — tool outputs, reasoning traces, partial plans, self-evaluations, or environmental feedback. A formation operator:

$\mathcal{M}^{\text{form}}_{t+1} = \mathcal{F}(\mathcal{M}_t, \phi_t)$

selectively transforms these artifacts into memory candidates, "extracting information with potential future utility rather than storing the entire interaction history verbatim."

What it computes: given the current memory state $\mathcal{M}_t$ and the new information $\phi_t$ generated by the agent's action, the formation operator produces an intermediate memory state $\mathcal{M}^{\text{form}}_{t+1}$ that contains both old memories and newly formed candidates (but has not yet been integrated through consolidation or conflict resolution).

Why this form: separating formation from evolution (the next operator) is critical because it distinguishes extraction from integration. A naive system that simply appends all observations to memory is performing formation without evolution — it accumulates but does not organize. A more sophisticated system might form compact summaries or structured representations, but the decision of how to merge these with existing memories is handled separately by the evolution operator.

2. Memory Evolution. Formed memory candidates are integrated into the existing memory base:

$\mathcal{M}_{t+1} = \mathcal{E}(\mathcal{M}^{\text{form}}_{t+1})$

which may consolidate redundant entries, resolve conflicts, discard low-utility information, or restructure memory for efficient retrieval.

What it computes: the final memory state $\mathcal{M}_{t+1}$ after the evolution operator has processed the formed candidates. This state persists across subsequent decision steps and tasks.

Why this form: evolution is where the memory system demonstrates agency — rather than passively accumulating everything, it actively manages its knowledge base through consolidation (merging related entries into higher-level abstractions), updating (correcting outdated or conflicting information), and forgetting (removing noise). These three sub-operations are the core of Section 5.2 and represent the operational difference between a static database and a self-evolving memory system.

3. Memory Retrieval. When selecting an action, agent $i$ retrieves a context-dependent memory signal:

$m^i_t = \mathcal{R}(\mathcal{M}_t, o^i_t, Q)$

where $\mathcal{R}$ denotes a retrieval operator that constructs a task-aware query and returns relevant memory content, formatted for direct consumption by the LLM policy (e.g., as textual snippets or structured summaries).

What it computes: given the current memory state $\mathcal{M}_t$, the current observation $o^i_t$, and the task specification $Q$, the retrieval operator produces a memory signal $m^i_t$ that will be fed into the agent's policy alongside the observation.

Why this form: retrieval is where the "read" side of memory happens. The operator $\mathcal{R}$ abstracts over a wide range of implementation choices — sparse vs. dense retrieval, graph traversal, generative retrieval — and over the decision of when to retrieve. The paper emphasizes that retrieval need not happen at every step: some systems retrieve only at task initialization ($m^i_t = \mathcal{R}(\mathcal{M}_0, o^i_0, Q)$ for $t = 0$ and $m^i_t = \perp$ for $t > 0$), while others retrieve continuously or based on contextual triggers.

Temporal roles within the agent loop. The paper makes an important architectural point: short-term and long-term memory phenomena emerge not from discrete architectural modules but from the temporal patterns with which formation, evolution, and retrieval are engaged. Inside a task, memory effects arise from lightweight logging or iterative refinement; across tasks, memory may be updated episodically at task boundaries or continuously. The same unified memory container $\mathcal{M}_t$ supports both roles, with temporal distinctions emerging from usage patterns.

Memory-agent coupling. The overall agent loop consists of: observe environment → optionally retrieve memory → compute action → receive feedback → optionally update memory through formation and evolution. Different implementations instantiate different subsets of these operations at different frequencies.

---

Conceptual Distinctions: Agent Memory vs. Adjacent Concepts

Section 2.3 is the paper's effort to draw clear boundaries around what counts as agent memory and what does not. This is necessary because, as the paper argues, the field has suffered from terminological conflation that obscures the unique properties of each concept.

Agent Memory vs. LLM Memory (Section 2.3.1). The paper traces a historical reclassification: systems from 2023–2024 that described themselves as "LLM memory mechanisms" (MemoryBank, MemGPT) were actually addressing classical agentic challenges — tracking user preferences, maintaining dialogue state, accumulating experience across multi-turn interactions. Under a modern understanding of agency, these are properly classified as agent memory.

However, the paper carves out a distinct category of genuinely LLM-internal memory that lies outside the scope of agent memory: "managing the transformer's key-value (KV) cache, designing long-context processing mechanisms, or modifying model architectures (e.g., RWKV, Mamba, diffusion-based LMs) to better retain information as sequence length grows." The distinguishing criterion is functional: agent memory supports a decision-making agent's evolving external memory base through deliberate read/write operations; LLM memory expands the model's internal representational capacity through architectural changes.

Agent Memory vs. RAG (Section 2.3.2). This is the most nuanced distinction. The paper acknowledges substantial overlap — both paradigms construct auxiliary information stores, both use knowledge graphs and vector indices — and notes that the boundary has become "increasingly blurred as retrieval systems themselves become more dynamic."

The proposed distinction is primarily domain-based. RAG is predominantly applied to augmenting LLMs with external, often static knowledge for individual task invocations, typically evaluated on knowledge-intensive QA benchmarks such as HotpotQA, 2WikiMQA, and MuSiQue. Agent memory operates in settings requiring sustained multi-turn interaction, temporal dependency, or environment-driven adaptation, typically evaluated on long-context dialogue benchmarks (LoCoMo, LongMemEval), complex reasoning benchmarks (GAIA, XBench), or lifelong learning benchmarks (StreamBench).

The paper further analyzes the relationship through the lens of RAG subtypes:

• Modular RAG (decomposed pipelines for indexing, retrieval, reranking): corresponding techniques appear in the retrieval stage of agent memory systems via vector search or rule-based filtering (e.g., Memary, MemOS, Mem0).

• Graph RAG (knowledge-graph-structured retrieval): maps to graph-structured agent memory (e.g., A-MEM, Zep, G-Memory), but with a crucial difference — in agent memory, the graph is "a living, evolving representation of experience" that is constructed, extended, and reorganized throughout operation, rather than a static index.

• Agentic RAG (autonomous, iterative retrieval controlled by an LLM agent): occupies the closest conceptual space to agent memory. The key remaining distinction is that agentic RAG typically operates over an external, task-specific database, while agent memory maintains an internal, persistent, self-evolving base that accumulates knowledge across tasks.

Agent Memory vs. Context Engineering (Section 2.3.3). The relationship is characterized as an "intersection of distinct operational paradigms." Context engineering treats the context window as a constrained computational resource to be optimized — a resource management paradigm where agent memory is one variable within the context assembly function. Agent memory treats memory as the cognitive substrate that sustains a persistent identity and accumulates knowledge — context engineering is merely the implementation layer that ensures cognitive continuity within model limits.

The two converge in the technical realization of working memory (summarization, compression, importance-based selection) but diverge in scope: context engineering optimizes the momentary interface, while agent memory governs what the agent knows, what it has experienced, and how these elements evolve over time.

---

The Forms Dimension: Token-Level, Parametric, and Latent Memory

Section 3 organizes the representational forms of agent memory into three categories based on "where memory resides and in what form it is represented."

Token-Level Memory (Section 3.1)

Token-level memory stores information as "persistent, discrete units that are externally accessible and inspectable." The term "token" is used broadly — beyond text tokens, it includes visual tokens, audio frames, or any discrete element that can be written, retrieved, reorganized, and revised outside model parameters.

Properties. Token-level memory is transparent (human-readable), easy to edit (add/delete/update operations are straightforward), and interpretable (decision processes can be traced back to concrete memory units). It is the most common memory form with the largest body of existing work.

Internal organization taxonomy. The paper introduces a novel sub-taxonomy based on topological complexity — how stored units are structurally organized:

• Flat Memory (1D): No explicit inter-unit topology. Memories accumulate as sequences or bags of units (snippets, trajectories, chunks). Examples include dialogue logs (MemGPT, MemoryBank), user profiles (Mem0, Memochat), and experience pools (Reflexion, Expel). The advantage is simplicity and scalability; the limitation is that "coherence and relevance depend heavily on retrieval quality" and there is no mechanism for compositional reasoning over related units.

• Planar Memory (2D): An explicit organizational topology within a single layer — units are related by a graph, tree, or table with no cross-layer hierarchy. Examples include tree-structured memories (HAT, MemTree), graph-structured memories (A-MEM, Ret-LLM, M3-Agent), and hybrid structures (Optimus-1, D-SMART). These structures capture relational dependencies (adjacency, parent-child ordering, semantic grouping) and support richer retrieval via structured key-value lookups or graph traversal. The limitation is that without hierarchical storage, "all memories must be consolidated into a single, monolithic module" which becomes inadequate for complex, diverse task scenarios.

• Hierarchical Memory (3D): Organized across multiple layers with inter-level connections, forming a volumetric structured space. Sub-categories include Pyramid structures (HiAgent's subgoal-centered hierarchy, GraphRAG's community-detection-based indices, Zep's temporal knowledge graph) and Multi-Layer structures (HippoRAG's associative indexing with passage store, AriGraph's semantic-episodic dual-layer graph, CAM's layered schemata). These support representations at different degrees of abstraction, from raw observations to compact summaries to higher-level thematic patterns, and enable complex multi-path queries that "move through relational networks within each layer and across abstraction levels between layers."

The key insight of the 1D-2D-3D taxonomy is that increasing topological complexity buys richer query capabilities and better support for compositional reasoning, but at the cost of higher construction and maintenance overhead. The paper's contribution is making this trade-off explicit and systematic rather than implicit in individual system designs.

Parametric Memory (Section 3.2)

Parametric memory stores information directly in the model's parameters, "allowing the model to internalize and recall information without referring to external storage."

Two sub-types based on storage location:

• Internal Parametric Memory: Memory encoded within the original model parameters (weights, biases). This can happen during pre-training (LMLM, HierMemLM for knowledge retrieval priors), continued pre-training/mid-training (Early Experience for agent trajectories, Agent-Founder for tool-calling experience), or post-training (Character-LM and CharacterGLM for role-playing personas, SELF-PARAM via KL divergence distillation, model editing methods like ROME and MEMIT for precise factual updates).

• External Parametric Memory: Memory stored in auxiliary parameter sets — adapters, LoRA modules, or lightweight proxy models — without modifying the original model weights. Examples include MLP-Memory (integrating RAG knowledge through MLP adapters), K-Adapter (task-specific adapter modules for continual knowledge expansion), WISE (dual-parameter memory with dynamic routing to mitigate lifelong editing conflicts), and ELDER (multiple LoRA modules with learned routing functions).

Trade-offs. Internal parametric memory has "simple structure, which does not add extra inference overhead or deployment costs" but is "difficult to update" — storing new memory requires retraining, which is costly and prone to catastrophic forgetting. External parametric memory provides modularity (modules can be added, removed, or replaced without interfering with the base model) but relies on "indirect" influence mediated through the model's attention and computation pathways, meaning effectiveness depends on interface quality.

Latent Memory (Section 3.3)

Latent memory is "carried implicitly in the model's internal representations (e.g., KV cache, activations, hidden states, latent embeddings), rather than being stored as explicit, human-readable tokens or dedicated parameter sets." It avoids exposing memory in plaintext, introduces less inference latency, and potentially offers better performance by preserving fine-grained contextual signals.

Three sub-types based on how the latent state is formed and introduced:

• Generate: Latent memory is produced by an independent model or module and supplied as reusable internal representations. In single-modal settings, this includes Gist tokens (compressing long prompts into a small set of tokens), AutoCompressor summary vectors (encoding long documents into soft prompts), MemoryLLM's persistent memory tokens (self-updatable latent embeddings injected into transformer layers), and MemGen's LoRA fragments (dynamically generated latent memory tokens triggered during decoding). In multimodal settings, this includes CoMem's vision-language compression via Q-Former and Time-VLM's patch embeddings for video streams.

• Reuse: Latent memory is directly carried over from prior computation, most prominently KV-cache reuse. Memorizing Transformers store past KV pairs and retrieve them via KNN search during inference. FOT introduces memory-attention layers for KNN-based retrieval over additional KV memories. LONGMEM uses a residual SideNet treating historical KV embeddings as persistent memory. These approaches preserve "the full fidelity of the model's internal activations" but suffer from rapid growth in memory consumption with context length.

• Transform: Existing latent state is modified, compressed, or restructured. Methods include KV cache pruning (Scissorhands based on attention scores, H2O retaining only heavy-hitter tokens), compression (SnapKV's head-wise voting, PyramidKV's layer-wise budget reallocation, RazorAttention's compensated window), and reversible compression (R3Mem's virtual memory tokens). These produce "more compact and information-dense memory representations" but introduce the risk of information loss and harder interpretability.

Key design choice. The paper notes potential ambiguity with parametric memory since many methods rely on separately trained models to generate latent representations. The classification criterion is grounded in the form of memory rather than the learning mechanism: "although these approaches generate memory through learned encoding, the produced latent representations are explicitly instantiated and reused as independent memory units, rather than being directly embedded into the model's parameters or forward-pass activations."

---

The Functions Dimension: Factual, Experiential, and Working Memory

Section 4 organizes memory by its functional role — why the agent needs it. The paper moves beyond the coarse long-term vs. short-term dichotomy prevalent in prior surveys, proposing instead a three-way functional taxonomy.

Factual Memory (Section 4.1)

Factual memory is "the capacity of an agent to store and retrieve explicit, declarative facts about past events, user-specific information, and the state of the external environment." It serves three fundamental properties during interaction: consistency (stable behavior over time), coherence (robust context awareness and topical continuity), and adaptability (personalization based on stored user profiles and feedback).

The paper draws on the cognitive science framework of declarative memory — episodic memory (personally experienced events with temporal-spatial context) and semantic memory (general factual knowledge independent of acquisition context) — but operationalizes these not as a rigid dichotomy but as a "processing continuum." Systems log concrete interaction histories as episodic traces, then apply summarization, reflection, entity extraction, and fact induction to transform raw event streams into reusable semantic fact bases.

Two sub-types based on the primary entity:

• User Factual Memory (Section 4.1.1): Facts about identities, preferences, routines, historical commitments, and salient events. This serves two purposes: (a) Dialogue coherence — maintaining conversational context through heuristic selection and ranking of interaction histories (MemGPT, MemoryBank) or semantic abstraction into higher-level representations (Think-in-Memory, Reflective Memory Management, COMEDY). (b) Goal consistency — maintaining and refining explicit task representations to minimize intent drift, using mechanisms like RecurrentGPT for tracking confirmed vs. unresolved constraints or A-Mem's networked note structure for localized retrieval centered on active goals.

• Environment Factual Memory (Section 4.1.2): Facts about external entities and states — documents, codebases, tools, interaction traces. This serves: (a) Knowledge persistence — structured representations for document analysis and multihop reasoning (HippoRAG's knowledge graphs for evidence propagation, MemTree's dynamic hierarchical structure, LMLM's decoupling of factual knowledge from model weights), and parametric approaches for continuous knowledge updates (MemoryLLM, M+, WISE). (b) Shared access — centralized repositories for multi-agent collaboration (Memory Sharing, MetaGPT's shared message pools, G-Memory's hierarchical memory graphs, Generative Agents' shared memory substrate for social simulation).

Experiential Memory (Section 4.2)

Experiential memory is "the mechanism by which agents encode historical trajectories, distilled strategies, and interaction outcomes into durable, retrievable representations." It serves as the foundation for continual learning and self-evolution, operating through a non-parametric path to adaptation that avoids the prohibitive costs of frequent parametric updates.

Classification by abstraction level:

• Case-based Memory (Section 4.2.1): Minimally processed records prioritizing informational fidelity — raw trajectories (JARVIS-1 for survival experiences in Minecraft, Auto-scaling Continuous Memory for GUI history compression) and proven solutions (ExpeL's successful trajectories as exemplars, Synapse's abstracted state-action episodes, MapCoder's example code playbook). The advantage is high fidelity and verifiable evidence; the limitation is retrieval efficiency and context window consumption.

• Strategy-based Memory (Section 4.2.2): Transferable knowledge of how to act, encompassing three granularity levels: (a) Insights — granular decision rules and reflective heuristics distilled from trajectories (H2R's two-tier reflection decoupling planning and execution insights, R2D2's corrective insights from failures). (b) Workflows — structured sequences of actions abstracted from prior trajectories (AWM's reusable workflows on Mind2Web and WebArena, Agent KB's hierarchical retrieval with workflows as transferable procedural knowledge). (c) Patterns — cognitive templates encapsulating problem-solving structure (Buffer of Thoughts' meta-buffer of thought templates, ReasoningBank's reusable reasoning units, PRINCIPLES' synthetic strategy memory from offline self-play).

• Skill-based Memory (Section 4.2.3): Executable procedural capacity operationalizing abstract strategies into verifiable actions. This spans a continuum: code snippets (Voyager's skill library, Darwin Gödel Machine's self-rewriting code), functions and scripts (CREATOR's tool creation, LEGOMem's modular procedural memory, SkillWeaver's web interaction skills), APIs (Gorilla, ToolRerank, COLT for tool retrieval and calling), and MCPs (Alita, Alita-G for standardized tool discovery and use via Model Context Protocol).

• Hybrid Memory (Section 4.2.4): Systems integrating multiple forms — ExpeL coupling trajectories with textual insights, Agent KB's hierarchical structure with workflows guiding planning and solutions providing execution details, G-Memory's dynamic transitions where repeated cases are compiled into skills, MemVerse combining parametric and token-level procedural memory.

The key insight is that these abstraction levels are not competing alternatives but complementary layers in a cognitive architecture. Robust agents typically synergize them: case-based memory provides concrete evidence, strategy-based memory provides abstract planning logic, and skill-based memory provides grounded execution.

Working Memory (Section 4.3)

Working memory is defined as "the set of mechanisms for the active management and manipulation of context within a single episode." This is inspired by cognitive science frameworks (Baddeley's multicomponent model, Cowan's embedded-processes account) emphasizing attentional focus, interference control, and bounded capacity, but applied to the LLM context window.

Two sub-types based on interaction dynamics:

• Single-turn Working Memory (Section 4.3.1): Processing massive immediate inputs within a single forward pass. Mechanisms include: (a) Input condensation — hard condensation (LLMLingua's perplexity-based token selection, CompAct's iterative information-gain retention), soft condensation (Gist tokens, AutoCompressors, ICAE encoding into latent vectors), and hybrid approaches (HyCo2 combining global semantic adapters with token-level retention probabilities). (b) Observation abstraction — transforming raw observations into structured formats (Synapse rewriting HTML DOM trees into state summaries, VideoAgent converting video streams into temporal event descriptions, MA-LMM maintaining dual banks of visual features and query embeddings).

• Multi-turn Working Memory (Section 4.3.2): Maintaining task state across sequential interactions. Mechanisms include: (a) State consolidation — mapping growing trajectories into fixed-size state spaces through recurrent updates (MemAgent, MemSearcher), RL-optimized summarization (ReSum, IterResearch), or latent state representation (MEM1's shared internal state, MemGen's latent memory tokens). (b) Hierarchical folding — decomposing trajectories by subgoals, maintaining fine-grained traces only while a subtask is active and folding completed subtrajectories into concise summaries (HiAgent's subgoal-centered memory, Context-Folding and AgentFold's learnable folding policies, DeepAgent's tool-use reasoning with structured episodic and working memories). (c) Cognitive planning — externalized plans or world models as the core of working memory (PRIME's retrieval-integrated planning, SayPlan's 3D scene graphs, Agent-S and KARMA's hierarchical plan anchoring).

---

The Dynamics Dimension: Formation, Evolution, and Retrieval

Section 5 decomposes the memory lifecycle into three processes that transform memory from a static store into a dynamic, adaptive system.

Memory Formation (Section 5.1)

Memory formation is "the process of encoding raw contexts into compact knowledge." It answers the question: How to extract the memory?

Five categories of formation operations:

• Semantic Summarization (Section 5.1.1): Transforming raw observational data into compact, semantically rich summaries capturing global high-level information rather than specific details. Two paradigms: (a) Incremental summarization — continuously fusing new information with existing summaries through LLM-driven merging (MemGPT, Mem0), filtered integration using external evaluators (convolutional discriminators, DeBERTa-based filtering), or RL-optimized summarization (Mem1 with PPO, MemAgent with GRPO). (b) Partitioned summarization — dividing information into semantic partitions and generating separate summaries (MemoryBank by days/sessions, ReadAgent and LightMem by semantic/topic-based clustering, DeepSeek-OCR by optical 2D mapping compression, LangRepo by hierarchical video clip aggregation). The primary trade-off: incremental summarization preserves temporal continuity but suffers from serial bottlenecks and potential forgetting; partitioned summarization offers efficiency and finer granularity but can lose cross-partition dependencies.

• Knowledge Distillation (Section 5.1.2): Extracting specific cognitive assets: (a) Factual memory distillation — transforming interactions into declarative knowledge about users and environments (TiM's thought abstraction, RMM's topic-based memory, MemGuide's user intent capture, M3-Agent and ESR's compression of egocentric visual observations into text-addressable facts). (b) Experiential memory distillation — extracting strategies from historical trajectories through success-based distillation (AWM's workflow patterns, Memp's abstract procedural knowledge), failure-driven reflection (Matrix, SAGE, R2D2's error-source identification), or combined contrastive approaches (ExpeL, From Experience to Strategy, ReMe). Recent trainable methods (Memory-R1's LLMExtract module, Mem-α's RL-trained extraction policy) move beyond fixed-prompt approaches toward learned distillation.

• Structured Construction (Section 5.1.3): Transforming amorphous data into organized topological representations: (a) Entity-level construction — atomizing text into entities and relations to build planar or hierarchical knowledge graphs (KGT's user-specific graph, Mem0g's LLM-based entity extraction, D-SMART's neuro-symbolic pipeline for OWL-compliant graphs, GraphRAG's community detection and iterative summarization, AriGraph and HippoRAG's dual-layer semantic-episodic graphs, Zep's three-layer temporal graph). (b) Chunk-level construction — organizing intact text segments or memory items into tree or graph structures (HAT's hierarchical aggregate tree, RAPTOR's recursive GMM clustering, MemTree's bottom-up insertion with summary updates, A-MEM's networked notes, SGMem's sentence-level KNN graphs, CAM's incremental clustering with disentanglement, G-Memory's three-tier graph hierarchy for multi-agent settings).

• Latent Representation (Section 5.1.4): Encoding experiences directly into machine-native embeddings or KV states, bypassing text decoding to reduce information loss. Textual approaches include MemoryLLM and M+'s self-updatable latent tokens, and MemGen's trigger-weaver architecture. Multimodal approaches include CoMEM's Q-Former compression, ESR's video-to-language-to-vector pipeline, Mem2Ego's landmark embedding for spatial reasoning, and KARMA's hybrid long/short-term embedding encoding.

• Parametric Internalization (Section 5.1.5): Consolidating external memories directly into model weights: (a) Knowledge internalization — model editing techniques (MEND's auxiliary network for single-step edits, ROME's causal tracing and rank-one updates, MEMIT's batch editing via multi-layer residual distributions, CoLoR's LoRA-based internalization). (b) Capability internalization — embedding experiential knowledge through supervised fine-tuning on reasoning traces or preference-guided optimization (DPO, GRPO). The advantage is zero-latency access (knowledge becomes instinctive); the disadvantage is catastrophic forgetting and difficulty of precise modification.

Memory Evolution (Section 5.2)

Memory evolution is "the process of integrating newly formed memories with the existing memory base." It answers: How to refine the memory?

Three mechanisms:

• Consolidation (Section 5.2.1): Merging new and existing memories to form higher-level abstractions: (a) Local consolidation — fine-grained updates on highly similar fragments (RMM's top-K similarity merge with LLM approval, VLN's saturation-triggered pooling and compression). (b) Cluster-level fusion — capturing cross-instance regularities across semantically related memories (PREMem's generalization and refinement of reasoning clusters, EverMemOS's similarity-to-centroid merging, TiM's periodic hash-bucket examination, CAM's cluster summary generation). (c) Global integration — holistic consolidation for system-level insights (MOOM's temporal-metadata-fused role profiles, Matrix's iterative optimization for task-agnostic principles, AgentFold and Context Folding's automatic context compression in multi-step interactions).

• Updating (Section 5.2.2): Revising or replacing existing memory when conflicts arise or new information arrives: (a) External memory update — evolving from rule-based corrections (MemGPT, D-SMART, Mem0g's conflict-detection-and-replace pipeline) to temporally aware soft deletion (Zep's timestamp-based invalidation) to delayed-consistency strategies (MOOM, LightMem's dual-phase online/offline updating) to fully learned update policies (Mem-α's RL-trained decisions on when, how, and whether to update). (b) Model editing — direct parameter-space modifications through explicit localization (ROME, Model Editor Networks) or latent-space self-updating (MemoryLLM, M+, ChemAgent).

• Forgetting (Section 5.2.3): Deliberately removing outdated, redundant, or low-value information: (a) Time-based forgetting — decay based on creation time (MemGPT's eviction of earliest messages, Xu et al. and Wang et al.'s stochastic token replacement, MAICC's soft weight decay). (b) Frequency-based forgetting — retention based on retrieval behavior (XMem's LFU policy, KARMA's counting Bloom filters, MemOS's LRU strategy). (c) Importance-driven forgetting — integrating temporal, frequency, and semantic signals (MemoryBank and MOOM's composite scoring, VLN's semantic redundancy clustering, Livia's emotion-driven selective forgetting, TiM and MemTool's LLM-based importance assessment).

Memory Retrieval (Section 5.3)

Memory retrieval is "the process of retrieving relevant and concise knowledge fragments from a certain memory repository to support current reasoning tasks at the right moment." It answers: How to utilize the memory?

Four sequential steps:

• Retrieval Timing and Intent (Section 5.3.1): (a) Automated retrieval timing — determining when to trigger retrieval, evolving from LLM-delegated decisions (MemGPT, MemTool) to fast-slow thinking mechanisms (ComoRAG, PRIME's adequacy evaluation) to latent, trainable triggers (MemGen's memory triggers detecting critical moments from rollout states). (b) Automated retrieval intent — deciding which memory source to access, ranging from dynamic switching based on feedback (AgentRR) to flat selection among memory types (MemOS's MemScheduler) to hierarchical index-based routing (H-MEM's coarse-to-fine domain-to-episode navigation).

• Query Construction (Section 5.3.2): (a) Query decomposition — breaking complex queries into sub-queries for modular retrieval (Visconde, ChemAgent, PRIME and MA-RAG's Planner Agents, Agent KB's teacher-student two-stage retrieval targeting model failures). (b) Query rewriting — refining query semantics through hypothetical document generation (HyDE's zero-shot generation, MemoRAG's global-memory-conditioned draft answers, MemGuide's command-like intent phrases, Rewrite-Retrieve-Read's RL-trained rewriter, ToC's Tree of Clarifications).

• Retrieval Strategies (Section 5.3.3): (a) Lexical retrieval — keyword matching via TF-IDF or BM25 for precision-oriented scenarios (Agent KB, MemAlpha, SeCom). (b) Semantic retrieval — embedding-based similarity matching as the default in most agentic frameworks (Sentence-BERT, CLIP, used in MemGPT, Voyager, Memory3, A-MEM, RMM). (c) Graph retrieval — structure-aware access through K-hop ego-graph expansion (AriGraph, EMG-RAG, Mem0g, SGMem), personalized PageRank (HippoRAG), LLM-guided subgraph exploration (CAM, D-SMART), or temporal-constrained retrieval (Zep, MemoTime). (d) Generative retrieval — directly generating document identifiers as a conditional generation task; acknowledged as having "untapped potential" due to scalability limitations with evolving corpora. (e) Hybrid retrieval — combining multiple paradigms (Agent KB and MIRIX's lexical-semantic fusion, Semantic Anchoring's parallel embedding and symbolic index searches, Generative Agents' multi-factor scoring accumulating recency, importance, and relevance).

• Post-Retrieval Processing (Section 5.3.4): (a) Re-ranking and filtering — heuristic criteria (Semantic Anchoring's vector-entity-discourse alignment, RCR-Router's role-relevance and recency signals), RL-learned weight aggregation (learn-to-memorize), temporal constraints (Zep and MemoTime's timestamp filtering), LLM-based evaluation (Memory-R1 and Westhäußer et al.'s Answer/Validator Agents), and auxiliary model estimation (RMM's trained importance scorer, Memento's Q-learning for retrieval utility, MemGuide's fine-tuned LLaMA-8B for marginal slot-completion gain). (b) Aggregation and compression — merging fragmented items into coherent context (ComoRAG's Integration Agent producing global summaries, MA-RAG's Extractor Agent for fine-grained content selection, G-Memory's role-specific customization for multi-agent systems).

---

Adaptation: Choosing the Right Memory Form for the Right Scenario

Section 3.4 synthesizes the Forms taxonomy into practical guidance by mapping memory types to deployment scenarios based on their inherent properties.

Token-level memory is characterized as "symbolic, addressable, and transparent," making it suitable for scenarios requiring explicit reasoning, controllability, and accountability: chatbots and multi-turn dialogue systems, long-horizon or lifelong agents requiring stable memory, user-specific personalization profiles, recommendation systems, enterprise knowledge bases, and high-stakes domains (legal, compliance) requiring verifiable provenance. Its plug-and-play nature allows integration with both closed-source and open-source foundation models without parameter modification.

Parametric memory is characterized as "implicit, abstract, and generalizable," making it suitable for tasks requiring conceptual understanding and broad pattern induction: role-playing or persona-consistent behaviors, mathematical reasoning and structured problem-solving, human alignment and normative behavioral priors, and stylized or domain-expert responses. Its internalization supports fluid end-to-end reasoning with zero-latency access but suffers from catastrophic forgetting and high update costs.

Latent memory is characterized as sitting "between explicit data and fixed model weights, enabling a unique balance of flexibility and efficiency." Its low readability provides intrinsic privacy protection, while its high expressive capacity permits rich semantic encoding with minimal information loss. Suitable scenarios include multimodal or fully integrated agent architectures, on-device or edge deployment and cloud-serving environments, and encrypted or privacy-sensitive domains. The key trade-off is performance and scalability over interpretability.

The paper emphasizes that these are not mutually exclusive — practical systems may combine forms (e.g., token-level memory for auditable fact storage alongside latent memory for efficient multimodal processing) and that the choice "reflects how designers expect the agent to behave in a given task."

4. Key Insights and Innovations

Innovation 1: A True Multi-Dimensional Taxonomy Reveals Design Trade-Offs That Single-Axis Frameworks Inherently Obscure

The paper's most fundamental contribution is not descriptive but diagnostic. Prior surveys on agent memory — including Zhang et al. (2025s) and Wu et al. (2025g), which the paper explicitly cites as valuable predecessors — organize work along a single axis: most commonly, a temporal distinction between long-term and short-term memory, or a cognitive-science-inspired partitioning into episodic, semantic, and procedural categories. These frameworks are not wrong, but they are insufficiently dimensional. The paper's core argument, threaded through Sections 3–5, is that a single-axis taxonomy systematically conflates systems that are architecturally, functionally, and operationally distinct, while separating systems that share deeper structural properties.

The Forms–Functions–Dynamics triangle is not merely a more elaborate filing system. It is a conceptual tool for revealing design trade-offs that are invisible under temporal or cognitive-science categories alone. Consider two systems both classified as "long-term memory" under a temporal taxonomy: MemoryBank (Zhong et al., 2024), which stores dialogue history as flat text chunks in a vector database, and SELF-PARAM (Wang et al., 2025o), which internalizes user knowledge into model parameters via KL divergence distillation. A temporal taxonomy places them in the same bin, implying they are comparable. The paper's multi-dimensional taxonomy immediately reveals that they differ along all three axes: MemoryBank has a token-level form (1D flat structure), serves a factual function (dialogue coherence), and uses semantic summarization for formation with frequency-based forgetting for evolution. SELF-PARAM has a parametric form (internal parameter memory), also serves a factual function but operates through parametric internalization for formation and requires no external retrieval. These are fundamentally different engineering artifacts with different failure modes, update costs, interpretability properties, and scalability characteristics. A taxonomy that treats them as comparable because both are "long-term" actively obscures the choices a system designer faces.

The significance of this contribution extends beyond taxonomy into research methodology. By providing orthogonal axes, the framework enables the field to ask more precise comparative questions: not "Is graph-based memory better than vector-based memory?" but "For factual memory serving dialogue coherence, does a planar (2D) graph-structured token-level form outperform a flat (1D) form when retrieval involves multi-hop relational queries?" The paper's detailed classification tables — particularly Table 1 (token-level memory methods compared across multimodal capability, memory type, memory structure, and task domain), Table 4 (factual memory methods compared across carrier, structure, task, and optimization strategy), and Table 5 (experiential memory methods compared across carrier, form, task, and optimization) — are not merely organizational aids but empirical hypotheses in tabular form. Each empty cell in these tables represents an underexplored region of the design space. For instance, Table 5 reveals that experiential memory in latent form is extremely rare (only MemGen and Continuous Memory appear), suggesting a significant gap: can experiential knowledge be effectively encoded in latent representations, or is there something about the abstraction level of strategy-based memory that fundamentally requires symbolic token-level representation?

This is a fundamental rather than incremental contribution because it changes what counts as a meaningful comparison in the field. Before this framework, a paper could claim improvement over a baseline without clarifying whether the two systems occupied the same region of the design space. After it, meaningful comparison requires matching along at least two of the three dimensions. The framework thus functions as what the philosophy of science calls a taxonomic framework that enables cumulative knowledge — it provides the shared vocabulary and comparative logic that the paper's introduction diagnoses as missing.

Innovation 2: The Lifecycle View — Memory as a Dynamic Process, Not a Static Store — Reframes the Entire Design Problem

The paper's second distinctive contribution is the decomposition of memory operation into three sequential lifecycle processes — formation, evolution, and retrieval (Section 5) — and the argument that these are not implementation details but the primary axes along which memory systems should be designed and evaluated. This reframing changes the question from "what does the agent remember?" to "how does the agent's memory change over time, and who or what controls that change?"

The dominant assumption in prior work, particularly in the RAG-adjacent memory literature, has been that memory is a store-then-retrieve problem: you encode information into a database, then query it when needed. The quality of the system is primarily a function of retrieval accuracy — how well the retrieval mechanism (vector search, graph traversal, keyword matching) maps queries to relevant stored items. The paper does not reject this concern — Section 5.3 provides a detailed taxonomy of retrieval strategies — but it demotes retrieval from the central problem to one of three equally important lifecycle stages. The formation stage (Section 5.1: what gets stored and in what form) and the evolution stage (Section 5.2: how stored information is consolidated, updated, and forgotten) are elevated to first-class design concerns with their own internal taxonomies and trade-offs.

This reframing has concrete implications. Under a store-then-retrieve paradigm, the primary design question is: given a query and a memory bank, how do you maximize recall and precision? Under the lifecycle paradigm, the questions multiply: formation — should you store raw trajectories, compressed summaries, structured graphs, or latent embeddings, and what information is lost in each choice? Evolution — how do you handle contradictions when new facts conflict with old ones (hard replacement vs. temporal annotation vs. LLM-mediated resolution)? When should you consolidate similar memories into abstractions, and when should you preserve fine-grained detail? When should you forget — by time decay, by access frequency, or by importance scoring? Retrieval — when should you retrieve at all (always-on vs. triggered by fast-slow evaluation vs. latent learned triggers)? What query construction strategy best bridges the gap between the user's surface utterance and the memory's internal representation?

The paper's organization of Section 5 into distinct sub-taxonomies for each lifecycle stage makes these questions operational and comparable. The five-way classification of formation operations (semantic summarization, knowledge distillation, structured construction, latent representation, parametric internalization) is not just a list of techniques but a spectrum of information compression strategies with different preservation-vs-abstraction trade-offs. The three mechanisms of evolution (consolidation, updating, forgetting) collectively define the stability-plasticity trade-off — how much should the agent's knowledge change in response to new experience? — and the paper's survey of approaches from rule-based to RL-learned shows the field progressing from hand-crafted answers to learned policies for this fundamental question.

This is a reframing contribution rather than a novel algorithm. Its significance lies in redirecting research attention: the paper argues implicitly that the field has over-invested in retrieval accuracy while under-investing in formation quality (how good are the memory entries being retrieved?) and evolution intelligence (does the memory base remain coherent as it grows?). The evidence for this reframing's value is not a single experimental result but the organizational power it provides — the ability to systematically compare MemGPT's incremental summarization (formation) with MemoryBank's partitioned daily segmentation (formation), or Zep's temporal annotation approach to conflict resolution (evolution) with Mem-α's RL-learned update policy (evolution), within a common conceptual vocabulary.

Innovation 3: The Explicit Recognition That the Field Is Undergoing a Paradigm Shift from Hand-Engineered to Learned Memory Systems

A distinctive feature of this survey, particularly in contrast to more purely descriptive literature reviews, is its forward-looking diagnostic claim about the direction of the field: agent memory is in the midst of a paradigm shift from hand-crafted, heuristic pipeline designs toward fully learned, RL-driven memory systems where the agent itself autonomously controls all stages of the memory lifecycle. This claim is articulated most explicitly in Section 7.3 ("Reinforcement Learning Meets Agent Memory") and illustrated schematically in Figure 11, which depicts a conceptual progression from "RL-free" systems through "RL partially involved" to "Fully RL-driven" architectures.

The paper does not merely report that RL techniques are being applied to memory — that would be a descriptive observation. Instead, it argues that this represents a qualitative change in what memory systems are. In the RL-free paradigm (which covers the majority of prior work surveyed in Sections 3–5), memory operations are designed by human engineers: they write the prompts that extract summaries, they set the thresholds that trigger consolidation, they define the similarity metrics for retrieval, and they choose the forgetting policies. The memory system is an engineered scaffold around a frozen or fine-tuned LLM. In the emerging RL-driven paradigm, memory operations become actions in the agent's own policy space — the agent learns when to form memories, what to store, how to consolidate, when to forget, and how to retrieve, all through optimization against long-horizon task performance.

The evidence the paper marshals for this claim is not a single smoking-gun result but a trajectory of increasing RL involvement traced across recent work. Early systems used RL only for post-retrieval re-ranking (RMM's policy gradient learner for memory chunk scoring). Mid-stage systems delegated an entire sub-process to RL (Mem-α's learned memory construction policy, Memory-R1's trained LLMExtract module). The frontier, represented by systems like Context Folding, Memory-as-Action, MemSearcher, and IterResearch, involves RL-trained working memory management where the agent learns to autonomously compress, fold, and reconstruct context in ultra-long multi-turn tasks. The paper's forward-looking claim (Section 7.3.2) is that the next stage will involve fully end-to-end RL-driven systems with no human-engineered priors on memory architecture — agents that "invent novel and potentially more suitable memory organizations that emerge directly from optimization dynamics rather than human intuition."

This is a diagnostic contribution: it identifies a trend that is visible but not yet dominant, and argues that it represents the future trajectory of the field. The significance is that it reframes what counts as progress. Under the hand-engineered paradigm, progress means designing better heuristics, better retrieval indices, or better structured representations. Under the RL-driven paradigm, progress means designing better reward functions, better training environments, and better architectures for learnable memory control. The paper is effectively arguing that the field should shift its collective effort from the former to the latter, and the survey's extensive coverage of RL-assisted memory systems serves as both evidence for and advocacy of this shift.

Innovation 4: The Boundary Work — Rigorously Distinguishing Agent Memory from RAG, LLM Memory, and Context Engineering — Creates a Defensible Scope for the Field

While boundary-drawing might seem like a preliminary rather than an innovative contribution, the paper's treatment of this problem in Section 2.3 is substantively novel in ways that matter for research practice. The confusion between agent memory and adjacent concepts is not a superficial terminological problem; it has led to incompatible evaluation protocols, non-comparable results, and research communities talking past each other. The paper's careful disentanglement provides not just definitions but operational criteria for determining whether a given system counts as agent memory.

The most intellectually distinctive aspect of this boundary work is the multi-resolution approach to the RAG-agent memory distinction (Section 2.3.2). Rather than drawing a single bright line, the paper acknowledges genuine overlap and proposes complementary criteria at different levels of analysis. At the task domain level, RAG is predominantly evaluated on knowledge-intensive QA benchmarks with static corpora (HotpotQA, 2WikiMQA, MuSiQue), while agent memory is evaluated on benchmarks requiring sustained interaction and temporal dependency (LoCoMo, LongMemEval, GAIA, StreamBench). At the system dynamics level, RAG typically operates over external, task-specific databases that are indexed once and queried; agent memory maintains an internal, persistent, self-evolving base that accumulates knowledge across tasks. At the cognitive role level, RAG augments an LLM with retrieved facts for a single inference call; agent memory supports a decision-making agent's ongoing adaptation and learning.

The paper further refines this analysis by examining RAG subtypes — modular RAG, graph RAG, agentic RAG — and showing that the conceptual distance to agent memory varies. Agentic RAG (Self-RAG, PlanRAG) occupies the closest space because it involves autonomous, iterative retrieval controlled by an LLM agent. The key remaining distinction is that even agentic RAG typically treats the knowledge base as external and task-specific, whereas agent memory is self-referential — the agent's own past actions and experiences form part of what is remembered.

The distinction from context engineering (Section 2.3.3) is similarly nuanced. The paper argues that context engineering and agent memory converge in the technical realization of working memory — both use summarization, compression, and importance-based selection to manage the context window. But they operate under different paradigms: context engineering is a resource management paradigm optimizing the momentary interface between model and environment; agent memory is a cognitive modeling paradigm sustaining a persistent identity and accumulating knowledge. The practical implication is that evaluating a system as context engineering asks "how efficiently does it use the context window?" while evaluating it as agent memory asks "how well does it maintain coherence, learn from experience, and adapt over time?"

This boundary work is methodologically significant because it provides criteria for determining what literature should be included in a survey of agent memory. Prior surveys, by the paper's account, have suffered from fuzzy boundaries that led to both over-inclusion (treating KV-cache management as agent memory) and under-inclusion (missing experiential memory work that was published under the RAG banner). The paper's explicit scoping decisions — excluding Mamba-style architectural modifications, including agentic RAG as a closely related paradigm, treating context engineering as an overlapping but distinct enterprise — create a defensible scope that makes the survey's organizational claims meaningful rather than arbitrary.

5. Experimental Analysis

Evaluation Methodology

Dataset. The MATH benchmark (Hendrycks et al., 2021), consisting of high-school competition-level math problems. The paper uses the specific split from Lightman et al. (2022): 12,000 training questions and 500 test questions. The choice is deliberate: test-time compute is hypothesized to help most when the model already possesses the necessary knowledge but struggles with multi-step logical deduction — mathematical reasoning fits this profile because it requires inference rather than novel factual recall (Section 4).

Base model(s). All experiments use PaLM 2-S* (Codey) (Anil et al., 2023). The paper argues this model is "representative of the capabilities of many contemporary LLMs" and sits in a useful performance regime: non-trivial pass@1 on MATH (roughly 10–19% depending on configuration) but far from saturation, leaving room for test-time compute to make a measurable difference (Section 4). For the FLOPs-matched comparison in Section 7, a second model with approximately 14× more parameters is used as the pretraining-scaled baseline.

Metrics. The primary metric throughout is MATH test accuracy — the fraction of the 500 test questions for which the selected final answer matches the ground truth. Answers are graded using the grading function released by Lightman et al. (2022) (Appendix G). When analyzing difficulty-dependent behavior, the paper reports accuracy within each of the five difficulty quintiles separately.

Baselines. The paper employs multiple baselines across experiments:

• Majority voting: select the most common final answer among N sampled solutions, with no learned verifier.
• ORM best-of-N weighted: score N solutions with an outcome reward model and apply best-of-N weighted selection (following Li et al., 2023).
• PRM best-of-N weighted: score N solutions with the process reward model and apply best-of-N weighted selection.
• Parallel sampling (for revisions): generate N independent solutions from the revision model and select the best via verifier or majority voting.

Generation budget / compute accounting. The universal unit of test-time compute is one "generation" — one complete sampled answer from the base LLM. For beam search and best-of-N, the budget equals the number of beams or samples N. For lookahead search with k lookahead steps, the cost is N × (k+1) to account for the additional rollout computation (Section 5.3). Budgets are swept across powers of 2, typically from 2⁰ to 2⁹ (1 to 512 generations). The paper also conducts a FLOPs-matched comparison in Section 7 using standard scaling-law approximations: pretraining FLOPs X = 6ND_pretrain and inference FLOPs Y = 2ND_inference, where N is the number of model parameters, D_pretrain is pretraining tokens, and D_inference is total inference tokens generated. Three values of R = D_inference / D_pretrain are tested: 0.16 (R ≪ 1), 0.79 (R ≈ 1), and 22 (R ≫ 1).

Cross-validation / statistical protocol. To avoid contaminating strategy selection with test-set performance, the paper uses two-fold cross-validation within each difficulty bin on the 500-question MATH test set (Section 3.2). The best strategy is selected on one fold and evaluated on the other, with results averaged. This applies to both the compute-optimal search strategy selection and the compute-optimal revision ratio selection.

---

Main Quantitative Results

Search Against PRM Verifiers (Section 5)

The core finding for search algorithms is that no single search method dominates across all difficulty levels or budgets, motivating the compute-optimal allocation policy. The results are presented in Figures 3 and 4.

Aggregate comparison (Figure 3, left). Across all 500 test questions with a maximum budget of 256 generations:

• At low budgets (2–8 generations), beam search with M = 4 significantly outperforms best-of-N weighted. At 4 generations, beam search achieves roughly 27% accuracy versus roughly 16% for best-of-N weighted — a gap of approximately 11 percentage points.
• At high budgets (64–256), beam search performance flattens and falls slightly below best-of-N weighted. Best-of-N weighted reaches approximately 38% at 512 generations; beam search (M = 4) plateaus around 34%.
• Lookahead search (both k = 1 and k = 3) generally underperforms at the same generation budget due to its higher per-step cost. The 3-step lookahead variants converge to similar performance as other methods at very high budgets but never surpass them.
• Majority voting trails all verifier-based methods substantially, reaching only about 29% at 512 generations.

The non-monotonic behavior of beam search — outperforming at low budgets but underperforming at high budgets — is attributed by the paper to PRM over-optimization: search finds solutions that score highly under the PRM but are actually incorrect. Specific failure modes include low-information repetitive steps at the end of solutions (visible in Figure 29) and overly short 1–2 step solutions.

Difficulty-dependent behavior (Figure 3, right). When results are broken out by difficulty bin, a clear and non-obvious pattern emerges for beam search (M = 4) vs. best-of-N weighted at four budget levels (4, 16, 64, 256):

• Bin 1 (easiest): Beam search accuracy actually decreases from roughly 78% to 77% as the budget goes from 4 to 256, while best-of-N weighted increases from 68% to 88%. This is the clearest evidence of PRM over-optimization — on easy problems, aggressive search amplifies residual verifier errors.
• Bin 2: Beam search improves modestly (roughly 14% → 32%) but best-of-N weighted improves faster (roughly 14% → 60%), maintaining a clear advantage at high budgets.
• Bin 3: Beam search consistently outperforms best-of-N weighted across all budgets, reaching roughly 34% vs. 23% at 256 generations.
• Bin 4: Beam search shows the strongest relative advantage, reaching roughly 17% vs. 10% for best-of-N at 256 generations.
• Bin 5 (hardest): Both methods hover near 1–3% regardless of budget. No method makes meaningful progress — the base model simply lacks the capability to produce correct solutions for these problems.

This difficulty-dependent pattern — beam search hurts on easy problems, helps on medium-hard problems, and is irrelevant on the hardest problems — is the empirical foundation for the compute-optimal policy. The decision of which search algorithm to use cannot be made uniformly; it must be conditioned on estimated problem difficulty.

Compute-optimal search results (Figure 4). By selecting the best search strategy per difficulty bin at each budget level:

• At 16 generations, compute-optimal (oracle bins) achieves approximately 27% accuracy, roughly matching PRM best-of-N weighted at 64 generations — representing more than 4× better compute efficiency.
• At 256 generations, compute-optimal oracle reaches approximately 39.5%, surpassing PRM best-of-N weighted at the same budget (roughly 37%).
• Compute-optimal with predicted (non-oracle) difficulty bins tracks the oracle version closely, particularly at lower budgets. The two curves "largely overlap" per the paper (Figure 4), with the predicted version reaching approximately 37% at 256 generations.
• Both compute-optimal variants consistently outperform ORM best-of-N weighted (which peaks around 34% at 512 generations) and majority voting (around 29%).

The near-overlap between oracle and predicted difficulty bins is a critical robustness result: it means the compute-optimal policy is deployable without access to ground-truth answers, since the PRM's own score distribution serves as a sufficient proxy for difficulty estimation.

PRM vs. ORM (Figure 14, Appendix F). At 2048 samples, PRM best-of-N weighted achieves approximately 40% accuracy versus roughly 35% for ORM best-of-N weighted and roughly 30% for majority voting. The gap between PRM and ORM widens with the number of samples, confirming the PRM's superior scaling properties.

---

Revision Model Results (Section 6)

The core finding for revisions is that sequential refinement outperforms parallel independent sampling in aggregate, but the optimal sequential-to-parallel ratio is difficulty-dependent, with easy problems benefiting most from purely sequential chains and harder problems requiring a balance of exploration (parallel sampling) and exploitation (sequential refinement).

Revision model baseline capability (Figure 6, left). Starting from approximately 18.2% pass@1 at step 1, the revision model's per-step accuracy improves to roughly 24–25% by steps 15–20 and remains in the 23–25% range out to 64 steps. This indicates the model has learned a generalizable revision skill that extends beyond the 4-step training horizon: the model was only trained with up to 4 previous answers in context, but the improvement continues well past that, supporting the claim that the learned behavior is a genuine revision capability rather than memorized training patterns.

Sequential vs. parallel (Figure 6, right). At 64 generations:

• Sequential + best-of-N weighted: approximately 41.5%
• Parallel + best-of-N weighted: approximately 39%
• Sequential + majority: approximately 38%
• Parallel + majority: approximately 35%

Sequential outperforms parallel under both selection mechanisms, with the verifier-based gap (roughly 2.5 percentage points) being slightly narrower than the majority-based gap (roughly 3 points). This demonstrates that the benefit is not solely attributable to the verifier seeing more context (since the same pattern holds for majority voting, which does not use a learned verifier).

Sequential-to-parallel ratio sweep (Figure 7, left). For a fixed generation budget, varying the ratio reveals non-monotonic behavior:

• At 256 generations, the optimal ratio is around 2¹ to 2³ (2:1 to 8:1 sequential-to-parallel), achieving approximately 43–44% accuracy.
• Fully parallel (ratio = 1:0 sequential-to-parallel, meaning all N samples generated independently) yields approximately 40%.
• Fully sequential (ratio = 0:1, meaning a single chain of N revisions) yields approximately 42%.
• At lower budgets (8–32 generations), fully sequential is optimal — the curves are monotonically increasing with the sequential-to-parallel ratio. This makes intuitive sense: when the total budget is small, you cannot afford to split it across multiple chains; you are better off refining a single attempt.

Difficulty-dependent optimal ratio (Figure 7, right). At a fixed budget of 128 generations:

• Bin 1: Performance is essentially flat across all ratios, around 90–92%. Easy questions are insensitive to allocation strategy because nearly any approach works well.
• Bin 2: Slight advantage for higher sequential ratios, approximately 63% at fully sequential vs. 58% at fully parallel.
• Bin 3: A clear optimal ratio emerges at moderate sequential-to-parallel values (around 2¹ to 2³), reaching approximately 42% vs. 35% at the extremes. This is the bin where the allocation strategy matters most — you need both parallel diversity (to explore different approaches) and sequential refinement (to polish within each approach).
• Bin 4: Similar pattern, with the peak at a moderate ratio achieving roughly 18% vs. 14% at fully parallel.
• Bin 5: All ratios produce roughly 2–3% accuracy. No allocation strategy helps on problems outside the model's fundamental capability range.

This mirrors the search results: the difficulty-dependent pattern reveals that easy problems benefit from exploitation (local refinement via sequential revisions), while harder problems benefit from exploration (global search via parallel diversity). The compute-optimal policy exploits these non-monotonicities.

Compute-optimal revision results (Figure 8). Selecting the optimal sequential-to-parallel ratio per difficulty bin:

• At 64 generations, compute-optimal oracle achieves approximately 40%, matching parallel best-of-N weighted at 256 generations — a 4× improvement in compute efficiency.
• At 256 generations, compute-optimal oracle reaches approximately 44%, compared to roughly 41% for best-of-N weighted and 37% for parallel-only.
• Compute-optimal predicted bins perform slightly below oracle bins at high budgets (approximately 41% at 256 generations) but still substantially outperform the parallel baseline.
• Notably, the parallel baseline appears to plateau around 36–37% at high budgets, while compute-optimal scaling continues to improve. This suggests that the gains from adaptive allocation compound at higher budgets — the ceiling is not yet reached.

---

FLOPs-Matched Comparison: Test-Time vs. Pretraining Compute (Section 7)

The core finding is that test-time compute can substitute for pretraining compute, but only under specific conditions: (1) the problems must be within the base model's capability range (easy-to-medium difficulty), and (2) the inference-to-pretraining token ratio R must be low. When problems are genuinely hard or inference volume is high, pretraining remains more effective.

Revisions (Figure 9, left; Figure 1, top-right bar chart). Comparing PaLM 2-S* with compute-optimal revisions against the ~14× larger model:

(Numbers from the bar chart in Figure 1, top-right. The "easy/medium/hard" groupings in the bar chart aggregate the five difficulty bins for readability.)

At R ≪ 1, test-time compute outperforms the larger model across all difficulty levels — even hard problems show a +21.6% relative improvement. This is the regime where the pretraining savings from using a smaller model dominate the budget. At R ≫ 1, test-time compute only remains preferable on easy questions, with hard questions showing a −37.2% relative disadvantage — the larger model's superior capability outweighs any test-time compute gains.

PRM search (Figure 9, right; Figure 1, bottom-right bar chart). The pattern is starker and less favorable for test-time compute:

PRM search shows weaker benefits than revisions for the FLOPs-matched comparison, with substantial disadvantages on medium and hard questions even at moderate R values. On easy questions, test-time compute remains preferable across all R regimes, though the margin narrows significantly as R increases. The paper does not provide a direct explanation for why search underperforms revisions in the FLOPs-matched setting, but the likely factor is that revisions directly improve the proposal distribution (generating better candidates), while search primarily optimizes selection over existing candidates — making revisions more compute-efficient per unit of improvement.

Figure 9 detail. The line plots show accuracy per difficulty bin as test-time compute scales. The 14× larger model's greedy performance (stars) is placed at three x-axis positions corresponding to the three R values. Where the compute-optimal scaling line is above the star, test-time compute wins. On bin 1 (easiest), the scaling line is above all three stars for revisions, indicating test-time compute is preferable across all R regimes. On bin 5 (hardest), the line is below all three stars and essentially flat near 0–5%, confirming that no amount of test-time compute helps when the base model's pass@1 is near zero.

---

Ablation Studies and Robustness Checks

PRM step-wise score aggregation (Appendix E, Figure 13): The paper compares three methods for combining per-step PRM scores into a single solution-level score — taking the minimum across steps ("min"), taking the product ("prod"), and using only the PRM's prediction at the final step ("last"). "Last" aggregation achieves the best performance, reaching roughly 37% at 256 samples, compared to approximately 35% for "min" and approximately 27% for "prod." An ORM baseline achieves roughly 34%. This is a non-obvious result because prior work (Lightman et al., 2023; Wang et al., 2023) found "min" to be best. The paper hypothesizes that the discrepancy arises because their PRM is trained with soft Monte Carlo labels rather than binary correctness labels, which changes how the per-step scores distribute. The consequence is significant: using the last-step prediction effectively makes the PRM behave like an ORM at aggregation time, yet the PRM still outperforms a separately trained ORM, suggesting that the step-level PRM training acts as beneficial representation learning even when the intermediate predictions are not directly used.

PRM vs. ORM (Appendix F, Figure 14): The PRM consistently outperforms the ORM across all sample counts, with the gap widening at higher counts. At 2048 samples, PRM best-of-N weighted reaches approximately 40% vs. ORM's 35% and majority voting's 30%. The widening gap indicates that the PRM's advantage is not merely a fixed offset but scales with compute — the PRM enables more effective use of additional samples.

Revision model verifier choice (Appendix J, Figure 15a): The base-LM PRM underperforms the revision-specific ORM when scoring revision model outputs, with sequential + base-LM PRM achieving roughly 40% at 64 generations vs. sequential + revision ORM at roughly 42%. This confirms distribution shift as a practical concern: a verifier trained on base model outputs does not transfer well to revision model outputs because the distribution of solutions differs. The 2-percentage-point gap represents the cost of distribution mismatch and motivates training dedicated verifiers for revision models.

Revision history in verifier context (Appendix J, Figure 15b): Including previous revisions in the ORM's context — so the verifier sees the same information the revision model sees — provides a small improvement over the no-history ablation (approximately 1–2 percentage points at 64 generations). However, both variants still outperform the parallel baseline. This demonstrates that the sequential sampling benefit is not solely attributable to the verifier seeing more context; the revision model genuinely produces better candidates in later steps of the chain.

Oracle vs. predicted difficulty bins (Figures 4, 8, and Appendix C, Figures 11–12): Both oracle and predicted bins yield qualitatively similar trends across difficulty levels. Predicted bins show slightly lower performance at high budgets in the revision setting (roughly 41% vs. 44% at 256 generations in Figure 8) but essentially identical performance in the search setting (Figure 4). This is the critical robustness check: the compute-optimal strategy works without ground-truth labels, using only the PRM's own score distribution for difficulty estimation.

Majority voting for revisions (Appendix B, Figure 10): The sequential-to-parallel ratio trends observed with verifier-based selection are replicated with majority voting: easy questions are insensitive to ratio, hard questions show an optimal intermediate ratio, and fully sequential marginally outperforms fully parallel in aggregate. This confirms that the core difficulty-dependent pattern is not an artifact of the verifier but a more fundamental property of the revision model's behavior.

ReST^EM revision model (Appendix K, Figure 16): An attempt to further optimize the revision model using ReST^EM (Singh et al., 2024) backfires: additional sequential revisions substantially hurt performance with this model. At 256 generations, fully sequential performance drops to approximately 33.5% compared to roughly 38.5% at the optimal (less sequential) ratio. The paper hypothesizes that on-policy data collection in ReST^EM exacerbates spurious correlations in revision data, causing the model to fail to learn the revision task properly. This is a notable negative result that highlights the sensitivity of revision training to the data generation procedure — not all training methodologies produce robust revision capabilities, and more sophisticated approaches can actually degrade performance.

---

Critical Assessment

Does compute-optimal test-time scaling improve efficiency by more than 4× over best-of-N?

This claim is supported for specific budget regimes and difficulty distributions, but with important caveats about what "4×" means operationally.

What the experiments demonstrate: For search (Figure 4), compute-optimal with 16 generations matches PRM best-of-N weighted at 64 generations. For revisions (Figure 8), compute-optimal with 64 generations matches parallel best-of-N weighted at 256 generations. In both cases, the compute-optimal policy achieves equivalent accuracy with 4× fewer generations. These are within-experiment comparisons that hold at the tested budget levels.

What remains unaddressed: The difficulty estimation cost (2048 samples per question to bin into difficulty quintiles) is not amortized into the efficiency calculation. The paper acknowledges this explicitly in Section 3.2: "our experiments do not account for this cost largely for simplicity." When the difficulty estimation cost is included, the total compute for the compute-optimal approach at 16 generations is not 16 but 2048 + 16 = 2064 generations — which is worse than the best-of-N baseline at 64 generations, not 4× better. The 4× claim is therefore accurate only in a regime where difficulty estimation cost is amortized over many repeated queries to the same prompts. For one-shot evaluation (which is the experimental setting), the reported efficiency gains do not account for the dominant cost. This is a significant gap between the claimed practical impact and the experimental evidence.

Additionally, the 4× figure is derived from specific budget comparisons (16 vs. 64 for search; 64 vs. 256 for revisions) and may not hold at all budget levels. At very high budgets, the compute-optimal curve with predicted bins shows diminishing returns relative to the parallel baseline (Figure 8: predicted bins reach ~41% vs. ~37% for parallel at 256 generations — less than 2× improvement in effective compute). The 4× claim is therefore most reliable in the lower-to-moderate budget regime and weakens at higher budgets.

Does test-time compute with a smaller model outperform a ~14× larger model?

The claim is supported with sharp boundary conditions that the paper itself documents, but the experimental design contains a notable baseline weakness.

What the experiments demonstrate: At R ≪ 1 with revisions, the smaller model with compute-optimal test-time scaling outperforms the ~14× larger model across all difficulty levels (Figure 9, left; Figure 1). At R ≫ 1, it only outperforms on easy questions, with substantial disadvantages on medium and hard problems. The dependence on R and difficulty is clearly demonstrated and internally consistent.

What is missing: The ~14× larger model uses only greedy decoding — no majority voting, no best-of-N, no search, no revisions. This is acknowledged in the paper but presents an asymmetric comparison. The test-time compute approach is given a budget of extra inference computation; the pretraining approach is given none. A fairer comparison would allocate some fraction of the total FLOPs budget to test-time compute for both models. For example, if the total budget can afford 100 generations for the smaller model at a given R, it could afford some smaller number of generations for the larger model (since each generation costs ~14× more FLOPs). The paper does not explore this intermediate regime, making the comparison binary (test-time compute vs. none) rather than continuous (how to split total FLOPs between model size and inference budget). The reported advantages of test-time compute over pretraining — particularly the +27.8% figure on medium questions at R ≪ 1 — may shrink or reverse if the larger model were also given a modest test-time compute budget proportional to its per-generation cost.

Additionally, the pretraining baseline scales only model parameters while fixing training data, following the LLaMA paradigm rather than Chinchilla-optimal scaling (which scales both data and parameters equally). The paper acknowledges this (Section 7): "We choose this setting as it is representative of a canonical approach to scaling pretraining compute and leave the analysis of compute-optimal scaling of pretraining compute where the data and parameters are both scaled equally to future work." A Chinchilla-optimal larger model would likely outperform a parameters-only-scaled model, making the pretraining baseline potentially weaker than it should be. This is not a flaw in the paper's execution — the authors are transparent about the choice — but it means the claim should be interpreted as "test-time compute can outperform this specific pretraining baseline" rather than "test-time compute can outperform pretraining in general."

Is the efficacy of test-time compute critically dependent on prompt difficulty?

Strongly supported — this is the most robust finding in the paper, replicated across search methods (Figure 3, right), revision strategies (Figure 7, right), and selection mechanisms (majority voting in Appendix B). The difficulty-bin analyses show qualitatively different — and sometimes opposite — effects of the same strategy at different difficulty levels. Beam search helps on medium-hard problems but hurts on easy problems at high budgets. Sequential revisions dominate for easy problems but underperform relative to balanced ratios for harder problems. Hard problems (bin 5) show near-zero improvement from any strategy at any budget. This pattern holds across oracle and predicted difficulty bins, across verifier-based and majority-based selection, and across both search and revision pipelines.

The one limitation is that difficulty is defined relative to the base model's pass@1 rate (via 2048-sample estimation), not relative to any intrinsic property of the problems. This means the difficulty bins are model-specific — a problem that is "hard" for PaLM 2-S* might be "medium" for a more capable model — and the compute-optimal policy would need to be re-estimated for each new base model. The paper acknowledges this implicitly by using model-specific bins rather than MATH's built-in difficulty labels, but does not explore how the policy transfers across models.

Missing experiments and analyses

Combined search and revisions. The paper studies PRM tree-search and iterative revisions as independent mechanisms (Sections 5 and 6) but never combines them — e.g., using beam search over revision model outputs, or using the PRM to guide which revisions to pursue. Section 8 explicitly acknowledges this: "we did not experiment with PRM tree-search techniques in combination with revisions." This is a significant gap because the two mechanisms have complementary strengths: revisions improve the proposal distribution, while search improves candidate selection. The current results therefore represent a lower bound on what a combined system could achieve, and the absence of this experiment limits the paper's ability to claim that the compute-optimal policy represents the best possible allocation.

Single model family, single benchmark. All experiments use PaLM 2-S* on MATH. The paper argues the model is "representative" (Section 4), but there is no evidence for cross-model generalization. Would the same difficulty-dependent patterns hold for GPT-4, Claude, or open-weight models? Would they hold for code generation (HumanEval, MBPP) or logical reasoning tasks? The difficulty bins are defined relative to the base model's pass@1, so the bin definitions would change with the model, but the paper provides no evidence that the qualitative patterns (beam search hurts easy problems, sequential revisions help easy problems) are model-agnostic.

Small test set for strategy selection. The 500-question test set is split into five difficulty quintiles of ~100 questions each, then further split by two-fold cross-validation, meaning the compute-optimal policy is selected based on ~50 questions per fold per bin. This is a small sample, and the paper does not report confidence intervals on the compute-optimal scaling curves. The selected strategies may not be robust — a different random split could yield a different optimal policy for a given bin, and the paper provides no sensitivity analysis.

No latency or wall-clock analysis. The paper measures compute in "generations," which abstracts away from the crucial practical difference between parallel and sequential operations. Sequential revisions are inherently serial (each revision depends on the previous one), while parallel best-of-N can be executed simultaneously with sufficient hardware. A strategy that allocates 128 generations as 64 sequential × 2 parallel takes roughly 64× longer wall-clock time than one that runs 128 parallel samples simultaneously. The compute-optimal policy frequently favors sequential-heavy ratios, which maximize FLOPs efficiency but may be impractical for latency-sensitive applications. The paper does not discuss this trade-off.

6. Limitations and Trade-offs

6.1 The Difficulty Estimation Cost Is Excluded from Efficiency Calculations, Making the Headline 4× Gain Unrealized in Practice

The entire compute-optimal framework depends on estimating each prompt's difficulty before allocating the inference budget. The paper's method for this estimation is generating 2048 samples per question and averaging either ground-truth correctness (oracle) or PRM final-answer scores (predicted), then binning into five quintiles. This cost — 2048 generations per question — is not included in any of the efficiency calculations that produce the central 4× improvement claim.

The authors acknowledge this gap explicitly in Section 3.2:

"estimating difficulty in this way still incurs additional computation cost during inference... our experiments do not account for this cost largely for simplicity"

The consequence. The reported 4× efficiency gains — e.g., compute-optimal search with 16 generations matching best-of-N at 64 generations (Figure 4), or compute-optimal revisions with 64 generations matching best-of-N at 256 generations (Figure 8) — are computed after difficulty is known, without amortizing the cost of learning it. When the difficulty estimation cost is included, the total compute for the "4× better" configuration is not 16 but 2048 + 16 = 2064 generations, which is dramatically worse than the best-of-N baseline at 64 generations. The 4× claim is only valid in a regime where the difficulty estimation cost is amortized over many repeated queries to the same prompts — a scenario that does not apply to the one-shot evaluation setting of the experiments.

What evidence exists in the paper. The dependency is documented throughout the experimental results. Figure 4 (compute-optimal search) and Figure 8 (compute-optimal revisions) both show the compute-optimal scaling curves using oracle and predicted difficulty bins, but neither figure accounts for the up-front estimation cost. The paper's cross-validation protocol (Section 3.2) confirms that the difficulty bins are computed per-question on the test set using 2048 samples each, but no budget line in any figure reflects this overhead. The paper does acknowledge the exploration-exploitation trade-off in Section 3.2: "compute spent assessing difficulty versus compute spent solving the problem" — but this trade-off is characterized as a "key avenue for future work" rather than resolved in the current study.

Mitigation status. Not addressed in the current work. The paper frames cheaper difficulty estimation as a direction for future research (Section 8: "pretraining or finetuning models to directly predict difficulty of a question") and also suggests adaptive approaches where difficulty estimation is interleaved with problem-solving, but no such method is developed or evaluated. Until this gap is closed, the 4× figure should be understood as an upper bound on achievable efficiency under a known difficulty oracle, not as a realized deployment gain. A practitioner adopting the method today would face the full cost of the 2048-sample estimation step, which in many realistic deployment scenarios would dominate the total cost and eliminate the reported advantage.

---

6.2 Hard Problems (Difficulty Bin 5) Show Near-Zero Improvement from Any Test-Time Compute Strategy, Establishing a Hard Capability Ceiling

Across every method studied — PRM search, iterative revisions, and their compute-optimal combinations — the hardest questions (bin 5, where the base model's pass@1 is near zero) show essentially no improvement regardless of how much test-time compute is allocated. The base model cannot produce correct solutions for these problems at any non-trivial rate, and no amount of search, revision, or adaptive allocation can compensate for this absence.

The consequence. Test-time compute amplifies existing capability but does not create it from nothing. If the base model's pass@1 on a problem class is effectively zero, the compute-optimal policy cannot help — there are no correct solutions in the proposal distribution to find (via search) or refine (via revisions). This imposes a fundamental upper bound on what test-time compute can achieve: it can close the gap between the base model's raw performance and its potential performance on problems it can already solve at some non-trivial rate, but it cannot extend the model's capability to qualitatively new problem types or difficulty regimes. For applications where the problem distribution includes a substantial fraction of genuinely hard problems, the compute-optimal framework offers no path forward — scaling pretraining or developing fundamentally more capable base models remains the only viable strategy.

What evidence exists in the paper. The evidence is consistent and unambiguous across all experimental settings:

• Search (Figure 3, right): Bin 5 accuracy hovers at 1–3% for both beam search and best-of-N across all budgets from 4 to 256 generations. No improvement with additional compute.
• Revisions (Figure 7, right): At a fixed budget of 128 generations, bin 5 shows 2–3% accuracy regardless of the sequential-to-parallel ratio. The allocation strategy is irrelevant.
• Compute-optimal scaling (Figure 9): The bin 5 scaling lines are essentially flat near 0–5% across all budgets. In the FLOPs-matched comparison, bin 5 shows negative relative performance against the larger model across nearly all R regimes (e.g., −37.2% for revisions at R ≫ 1, −52.9% for search at R ≫ 1).

The paper is explicit about this limitation in the Section 7 takeaway: test-time compute is powerful when "problems are within the base model's reach (it already produces correct solutions at some non-trivial rate)," but "it cannot compensate for fundamental capability gaps that larger pretraining would address."

Mitigation status. The limitation is inherent to the approach and cannot be mitigated within the current framework. The paper acknowledges it clearly but provides no solution. The implication for practitioners is that the compute-optimal framework is suitable for deployment scenarios where the problem distribution is skewed toward easy-to-medium difficulty (bins 1–4), but offers no benefit — and wastes compute — if applied to problems the base model fundamentally cannot solve. A practical system would need a separate mechanism for detecting when a problem is in bin 5 and either escalating to a more capable model or declining to spend compute fruitlessly — a capability the current framework does not provide.

---

6.3 The 14× Larger Model Baseline Is Not Compute-Optimally Trained and Uses Greedy Decoding, Making the Pretraining vs. Test-Time Compute Comparison Asymmetric

The FLOPs-matched comparison in Section 7 compares the smaller PaLM 2-S* model with compute-optimal test-time scaling against a model with approximately 14× more parameters. However, this larger model baseline has two specific weaknesses that the paper acknowledges:

First, the larger model scales only parameters while holding training data fixed, following the LLaMA paradigm (Touvron et al., 2023) rather than Chinchilla-optimal scaling (Hoffmann et al., 2022), which would scale both data and parameters equally. The paper states in Section 7:

"We choose this setting as it is representative of a canonical approach to scaling pretraining compute and leave the analysis of compute-optimal scaling of pretraining compute where the data and parameters are both scaled equally to future work."

Second, the larger model uses only greedy decoding — no majority voting, no best-of-N, no search, no revisions. It receives zero test-time compute budget in the comparison.

The consequence. The comparison is asymmetric: the test-time compute approach is given a budget of extra inference computation to optimize its performance, while the pretraining approach is not. A Chinchilla-optimal model trained with 14× more total FLOPs (scaling both data and parameters) would likely outperform a parameters-only-scaled model, making the pretraining baseline potentially weaker than the fair comparison. Additionally, if some fraction of the total FLOPs budget were allocated to test-time compute for both models, the larger model could also benefit from best-of-N or search, potentially closing or reversing the reported advantage. The paper's headline findings — e.g., +27.8% relative improvement on medium questions at R ≪ 1 for revisions vs. the larger model — should be interpreted as "test-time compute can outperform this specific pretraining baseline" rather than "test-time compute is generally preferable to pretraining."

What evidence exists in the paper. The FLOPs accounting formula (Section 7) explicitly shows that scaling model parameters by M multiplies both pretraining FLOPs (X = 6ND_pretrain) and inference FLOPs (Y = 2ND_inference) by M, meaning the larger model costs more per inference token. The three R values (0.16, 0.79, 22) parameterize different regimes of the inference-to-pretraining ratio. The experimental results in Figure 9 and the bar charts in Figure 1 show that the advantage of test-time compute varies dramatically with R and difficulty, which is internally consistent — but the binary comparison (test-time compute vs. none for the larger model) does not explore the intermediate regime where both models receive proportional test-time budgets.

Mitigation status. The paper acknowledges the parameters-only scaling choice but defers the Chinchilla-optimal comparison to future work. No ablation explores giving the larger model a test-time compute budget proportional to its per-generation cost. This is a significant gap because it means the paper does not fully characterize the training-inference trade-off surface — it only compares the extremes (all compute to pretraining vs. all compute to test-time). A practitioner deciding how to allocate a fixed FLOPs budget would need to understand the continuous trade-off, not just the endpoints.

---

6.4 All Experiments Use a Single Model Family on a Single Benchmark, Leaving Cross-Model and Cross-Domain Generalization Unverified

Every experiment in the paper uses PaLM 2-S* (Codey) evaluated exclusively on the MATH benchmark (500 test questions of competition-level mathematics). The paper argues in Section 4 that this model is "representative of the capabilities of many contemporary LLMs" and that MATH is appropriate because it requires multi-step logical deduction — a domain where test-time compute is expected to help. However, the paper provides no evidence that the observed patterns generalize to other model families or other task domains.

The consequence. Several aspects of the findings could be model- or domain-specific:

• The PRM's quality and over-optimization behavior depend on PaLM 2-S*'s output distribution. A model with different calibration properties, different error patterns, or different reasoning styles might exhibit qualitatively different difficulty-dependent scaling curves. For example, a model that is better calibrated might show less severe PRM over-optimization on easy problems, changing the beam-search-vs-best-of-N crossover behavior that drives the compute-optimal policy.
• The revision model's ability to learn from incorrect in-context examples depends on the base model's in-context learning capabilities, which vary substantially across model families.
• MATH consists exclusively of symbolic reasoning problems with clean ground-truth answers. It is unclear whether the difficulty-dependent patterns generalize to other reasoning domains (code generation, logical reasoning, scientific QA) or to tasks requiring factual knowledge rather than pure inference.

The paper's claim that PaLM 2-S* is "representative" is asserted rather than demonstrated — no comparison with other models is provided, and no ablation studies show how the key findings (the 4× efficiency gain, the difficulty-dependent strategy switching, the bin-5 failure mode) would change with a different base model.

What evidence exists in the paper. None that addresses cross-model or cross-domain generalization directly. The paper draws conceptual connections to the broader literature (Section 1 discusses deployment in chatbots, recommender systems, social simulations, and financial investigations), but all quantitative evidence is MATH-specific and PaLM 2-S*-specific. The revision model training procedure (edit-distance-based pairing of incorrect and correct solutions from the same base model) is inherently model-dependent — a different base model would generate different solution distributions, potentially requiring different training data construction.

Mitigation status. Not addressed. The paper does not frame cross-model or cross-domain generalization as a limitation, and no experiments test it. This is a significant gap for practitioners: the compute-optimal policy (which strategy to use for which difficulty bin at which budget) is estimated from PaLM 2-S* on MATH, and there is no guarantee that the same policy — or even the same qualitative patterns — would apply to, say, GPT-4 on code generation tasks. A practitioner using a different base model would need to replicate the full experimental pipeline (PRM training, difficulty estimation, strategy selection via cross-validation) rather than directly adopting the paper's findings.

---

6.5 Revisions and PRM Search Are Studied Independently but Never Combined, Leaving the Full Potential of Test-Time Compute Unexplored

The paper studies two complementary mechanisms — PRM-guided search (Section 5) and iterative revisions (Section 6) — but never evaluates them together. Section 8 explicitly acknowledges this:

"we did not experiment with PRM tree-search techniques in combination with revisions"

The consequence. The two mechanisms have complementary strengths: revisions improve the proposal distribution (generating better candidate solutions), while PRM search improves candidate selection (finding the best among generated solutions). A combined system could, for example, use beam search over revision model outputs, or use the PRM to guide which revision paths to pursue, potentially yielding gains beyond either method alone. The current results therefore represent a lower bound on what test-time compute could achieve — the "compute-optimal" policy reported in the paper optimizes over search strategies or revision ratios independently, but does not optimize over the joint space of search-method × revision-depth × parallel-branching. A practitioner combining both mechanisms might achieve substantially higher accuracy at a given budget than what the paper's compute-optimal curves suggest, or might find that the optimal strategy differs from what the independent analyses indicate.

What evidence exists in the paper. The paper provides separate evidence for each mechanism: Figure 4 shows compute-optimal search improvement, Figure 8 shows compute-optimal revision improvement, and Figure 9 compares both against pretraining in the FLOPs-matched setting. The revision model results (Figure 6, right) show that sequential revisions outperform parallel sampling in aggregate, and the search results (Figure 3, right) show that beam search outperforms best-of-N on medium-difficulty problems. But there is no experiment showing what happens when beam search is applied to the outputs of the revision model, or when the PRM is used to score and select among revision chains.

Mitigation status. The paper identifies this as an explicit direction for future work (Section 8) but provides no preliminary results or analysis of feasibility. This is a significant gap because it means the paper's central claim — that the compute-optimal policy achieves the best possible performance for a given budget — is not fully established. The claim holds within the restricted strategy spaces explored (search methods or revision ratios), but the full strategy space (search methods × revision ratios × their interactions) is never optimized.

---

6.6 Sequential Revision Strategies Impose Serial Latency That the Generation-Based Cost Model Does Not Capture

The paper measures test-time compute in terms of "generations" — the total number of complete solutions sampled — which is a reasonable proxy for total FLOPs. However, this metric abstracts away from the crucial practical distinction between parallel and sequential operations. Sequential revisions are inherently serial: each revision depends on the output of the previous one, and the chain cannot be parallelized across time steps. Parallel best-of-N, in contrast, can execute all N samples simultaneously given sufficient hardware.

The consequence. A strategy that allocates the same generation budget with a high sequential-to-parallel ratio may achieve superior FLOPs efficiency but at the cost of dramatically increased wall-clock latency. For example, at a budget of 128 generations, the compute-optimal policy on medium-difficulty problems (Figure 7, right) favors a moderate sequential-to-parallel ratio such that, for instance, 8 parallel chains each of length 16 would require approximately 16 sequential steps — 16× the wall-clock time of a fully parallel 128-sample execution. For latency-sensitive applications (interactive assistants, real-time decision-making, customer-facing chatbots), this serial dependency may render the compute-optimal policy impractical regardless of its FLOPs efficiency. A practitioner optimizing for response time rather than total compute cost would likely prefer strategies with lower sequential depth, even at the cost of additional parallel samples.

What evidence exists in the paper. None that addresses latency. The revision model results (Figure 6, Figure 7, Figure 8) report only generation-normalized accuracy. The sequential-to-parallel ratio sweep (Figure 7, left) shows that at high budgets (256 generations), the optimal ratio involves non-trivial sequential depth (ratios around 2¹ to 2³, meaning a significant fraction of the budget is spent in serial chains). The paper does not report wall-clock time, does not discuss hardware assumptions (number of available parallel workers), and does not include latency as a dimension of the trade-off analysis.

Mitigation status. Not addressed and not acknowledged as a limitation. The paper's cost model (1 generation = 1 unit of compute) implicitly assumes that the cost of a generation is independent of whether it is executed sequentially or in parallel, which is true for total FLOPs but false for wall-clock time. This is a significant practical limitation for deployment in interactive settings, and the paper's silence on the issue means practitioners must independently evaluate the latency implications of the compute-optimal policies before adoption.

7. Implications and Future Directions

How This Work Changes the Landscape

This survey does not introduce a new algorithm, benchmark, or system. Instead, it performs a reorganization of conceptual space — and in doing so, it changes what the field considers a meaningful research question. The dominant mode of inquiry in agent memory, prior to this survey, was phenomenological: build a memory mechanism, evaluate it on a benchmark, report improvement. The survey's Forms–Functions–Dynamics framework reframes the problem as one of multi-dimensional design: every memory system must be characterized not just by its performance on a task, but by where it sits in a three-dimensional space of representational form, functional role, and lifecycle dynamics. This is not an incremental refinement of prior taxonomies; it is a qualitative change in what counts as understanding a memory system.

A new diagnostic capability. The framework's most immediate practical impact is enabling comparative diagnosis across methods that previously appeared incommensurable. Tables 1, 4, and 5 are not merely organizational — they are tools for identifying underexplored regions of the design space. For instance, Table 5 (experiential memory) reveals that latent representations for experiential knowledge are almost entirely absent (only MemGen and Auto-scaling Continuous Memory appear), while token-level experiential memory is densely populated. This is not an accident of the survey's coverage; it reflects a genuine gap in what researchers have attempted. The framework makes this gap visible and actionable, converting an implicit absence into an explicit research opportunity.

A reconciliation mechanism for prior contradictions. The survey resolves — or more precisely, dissolves — a set of apparent contradictions that have plagued the field. Why do some papers find that self-correction works (Madaan et al., 2023) while others find it does not (Huang et al., 2023)? Why do graph-structured memories outperform flat stores on some tasks but not others? The Forms–Functions–Dynamics framework provides a systematic answer: these are not contradictions but taxonomic mismatches. Self-correction evaluated as a working memory mechanism (within a single task episode, with careful state consolidation) behaves differently from self-correction evaluated as an experiential memory mechanism (across episodes, with strategy abstraction). The framework's key contribution here is converting "X works vs. X doesn't work" debates into precise questions about which form, serving which function, under which dynamics, X works — questions that admit empirical resolution rather than rhetorical stalemate.

Redirecting research investment. The survey's extensive documentation of the memory lifecycle (Section 5) — particularly its decomposition of memory evolution into consolidation, updating, and forgetting — constitutes an implicit argument that the field has systematically under-invested in these stages relative to retrieval. Most prior work treats memory as a store-then-retrieve problem: the emphasis is on query construction, indexing strategies, and search algorithms (Section 5.3). The survey's equal treatment of formation and evolution as first-class design problems redirects attention. If memory quality is bottlenecked not by how well we retrieve but by what we store (formation quality) and how we maintain coherence as the store grows (evolution intelligence), then research investment should shift accordingly. The survey's Forward Perspective in Section 7.3 explicitly advocates for this redirection by arguing that fully RL-driven memory systems — where formation, evolution, and retrieval are all learned end-to-end — represent the next paradigm.

A stabilizing effect on terminology. A less dramatic but practically significant impact is the survey's potential to stabilize the vocabulary of the field. The paper explicitly documents the terminological chaos — "episodic memory" meaning different things to different research groups, "agent memory" being conflated with RAG and LLM memory — and proposes operational definitions grounded in the mathematical formalism of Section 2. If the community converges on this terminology, cumulative progress accelerates because results become comparable: a paper claiming to improve "experiential memory" using a "planar token-level form" with "consolidation-based evolution" can be precisely located in the taxonomy and compared against other methods in the same region.

What becomes more attractive. The survey makes several research directions more compelling than before:

• Joint optimization of formation, evolution, and retrieval rather than treating them as independent engineering problems.
• Difficulty-adaptive memory management — the survey's documentation of how the same strategy fails on hard problems (bin 5 near-zero improvement) while succeeding on medium ones establishes that uniform memory policies are suboptimal, and that memory allocation itself is a decision problem the agent should solve.
• Latent memory for experiential knowledge — the gap in Table 5 is a clear signal that this region of the design space is underexplored and potentially high-impact.
• Memory architectures that self-organize through RL — Section 7.3 provides concrete evidence of a trajectory from RL-free to RL-assisted to (emerging) RL-driven memory, making this direction feel inevitable rather than speculative.

What becomes less attractive. By implication, the survey suggests diminished returns from:

• Purely retrieval-centric approaches that treat memory quality as synonymous with retrieval accuracy — the lifecycle view reveals that formation and evolution often dominate the performance bottleneck.
• Single-axis taxonomies (long/short-term, episodic/semantic) that obscure the multi-dimensional nature of design choices.
• Hand-crafted memory pipelines where every threshold, similarity metric, and consolidation rule is manually specified — the RL-driven trajectory suggests these will be increasingly outperformed by learned policies.

Follow-Up Research This Work Enables

Characterizing the latent-experiential memory gap. The survey identifies a striking absence: experiential memory (case-based trajectories, strategy-based workflows, skill-based code libraries) is almost universally implemented in token-level form, with only isolated exceptions using latent representations (MemGen, Auto-scaling Continuous Memory). A direct follow-up experiment would evaluate whether experiential knowledge can be effectively encoded latently. Concretely: take a strategy-based memory method like AWM (which stores reusable workflows as text) or ExpeL (which stores insights as textual reflections) and train a latent encoder — along the lines of MemGen's memory trigger and weaver architecture — to compress these textual abstractions into continuous representations. The key measurement is whether latent experiential memory achieves comparable or superior transfer performance on unseen tasks while reducing storage footprint and retrieval latency. A negative result — latent encoding systematically loses the compositional structure that makes strategies transferable — would be equally informative, establishing a fundamental boundary on latent representations.

End-to-end RL for joint lifecycle optimization. Section 7.3 argues that fully RL-driven memory systems represent the next paradigm, but existing RL-assisted systems intervene only in isolated lifecycle stages (Mem-α for formation, Context Folding for working memory management). A strong follow-up would train a unified RL policy that controls all three stages — formation (what to store), evolution (when to consolidate, update, or forget), and retrieval (when and what to query) — with a single reward signal defined over long-horizon task performance. The experimental design should compare this joint policy against best-in-class pipelined systems (e.g., MemOS + RMM + MemAgent) on a benchmark like StreamBench or LifelongAgentBench that requires sustained adaptation. The critical measurement is whether joint optimization discovers emergent memory behaviors — such as the policy deciding to forget certain memories to make room for more informative ones, or choosing to form latent rather than token-level representations for certain types of experiences — that pipelined systems cannot produce because their stages are engineered independently.

A large-scale cross-model replication of difficulty-dependent memory efficacy. The survey's positioning of difficulty-dependent memory behavior (Section 4, Functional Taxonomy) is supported by reasoning about task requirements rather than empirical evidence at the scale of the entire taxonomy. A systematic replication would evaluate the same set of memory architectures — flat vs. planar vs. hierarchical token-level stores; parametric vs. external parametric forms; generate vs. reuse vs. transform latent memory — across a range of base models (GPT-4 class, Claude class, open-weight models like Llama-3) and task difficulties (operationalized as the base model's pass@1 rate). The hypothesis is that the optimal memory architecture is a function of base model capability: weaker models may benefit more from structured retrieval (graph-based forms) because they need external scaffolding for compositional reasoning, while stronger models may benefit more from latent memory because they can leverage high-dimensional representations without the overhead of explicit structure. This experiment would transform the survey's static taxonomy into a dynamic model of memory architecture selection.

Stress-testing the forgetting mechanisms taxonomy via long-horizon adversarial memory accumulation. The survey's forgetting taxonomy (Section 5.2.3) — time-based, frequency-based, importance-driven — is based on descriptive categorization of existing methods rather than comparative evaluation. A stress-test experiment would deploy a memory-augmented agent in a deliberately adversarial environment: the agent receives a stream of information where some facts are critical for long-term task success but are accessed rarely (violating frequency-based forgetting), some facts are old but must be retained indefinitely (violating time-based forgetting), and some facts are important but semantically similar to abundant low-value information (challenging importance-driven forgetting). The agent's memory is managed by each forgetting mechanism in turn, and performance on delayed retention probes measures which mechanism best handles which type of adversarial distribution. The expected result is that no single mechanism dominates, and a hybrid policy — conceptually similar to the compute-optimal test-time scaling policy from the companion paper — learns to weight forgetting criteria based on estimated information utility.

Memory-as-a-service benchmarking across frameworks. The survey's compilation of open-source memory frameworks in Table 9 is a snapshot of availability but provides no comparative performance data. A systematic benchmarking study would instantiate the same agent architecture (e.g., a conversational agent with factual and experiential memory requirements) using each major framework — MemGPT, Mem0, MemOS, Zep, LangMem, and memory-as-a-service backends like Pinecone and Chroma — and evaluate on standardized benchmarks like LoCoMo, LongMemEval, and PerLTQA. Beyond aggregate accuracy, the study should measure storage efficiency (bytes per retained fact), retrieval latency (milliseconds per query at scale), and evolution quality (how well the framework handles contradictory updates over 100+ interaction turns). This would transform the survey's descriptive table into a practical decision guide and expose framework-specific failure modes that are invisible in individual framework papers.

Multimodal memory integration without modality-specific engineering. The survey documents that multimodal memory is fragmented across modality-specific pipelines (Section 7.4): visual memory uses different representations than textual memory, and audio memory is essentially unexplored. A high-impact follow-up would build a unified multimodal memory representation — drawing on the latent memory paradigm (Section 3.3) to encode images, text, and audio into a shared embedding space — and evaluate whether this unified representation supports cross-modal retrieval (e.g., retrieving a visual memory in response to a text query, or retrieving an audio memory to resolve a visual ambiguity) and cross-modal reasoning without modality-specific routing. The experiment should use a benchmark like MEMENTO (Kwon et al., 2025) that provides naturalistic multimodal interaction histories, and compare against baselines that maintain separate modality-specific stores with hand-crafted fusion rules. A positive result — unified latent multimodal memory outperforming separate stores — would validate one of the survey's key forward-looking hypotheses.

Practical Applications and Downstream Use Cases

Self-improving coding agents via experiential memory consolidation. The survey's documentation of experiential memory — particularly the hierarchical progression from case-based trajectories to strategy-based workflows to skill-based code libraries (Section 4.2) — provides a blueprint for building coding agents that improve over time without manual retraining. A deployment scenario: a software engineering agent (operating on SWE-bench Verified or in a production CI/CD pipeline) accumulates raw debugging trajectories (case-based), periodically consolidates them into reusable debugging strategies (strategy-based, following AWM or Agent KB), and ultimately distills frequently used strategies into callable code snippets or API wrappers (skill-based, following Voyager or Darwin Gödel Machine). The practical benefit is measurable on SWE-bench Verified: agents equipped with this experiential memory lifecycle should show improving pass rates over successive runs on similar bug types, amortizing the initial memory formation cost across many deployments. The survey's lifecycle taxonomy provides the operational blueprint for when and how to trigger each consolidation stage.

Personalized conversational agents with auditable factual memory. The survey's distinction between token-level factual memory (transparent, editable, auditable — Section 3.1) and parametric factual memory (implicit, zero-latency, opaque — Section 3.2) has direct implications for deploying personalized chatbots in regulated domains (healthcare, finance, law). A practical architecture: use a token-level flat or planar memory store (e.g., Mem0, A-MEM) as the primary user factual memory for dialogue coherence and preference tracking, because this form supports auditing (regulators can inspect what the agent "knows" about a user), editing (users can correct or delete stored facts), and provenance tracking (each stored fact can be traced to a specific interaction). Reserve parametric memory (e.g., SELF-PARAM, fine-tuned persona models) for non-sensitive stylistic or behavioral priors that do not require transparency. The benefit: compliance with data protection regulations (GDPR right to access, right to be forgotten) is achievable through token-level memory operations without retraining, while parametric memory provides fluid, zero-latency access for style consistency. The survey's adaptation discussion in Section 3.4 provides the theoretical justification for this architectural choice; the practical deployment would measure compliance audit time (can all user-specific facts be retrieved and deleted within SLA?) and user satisfaction (does the hybrid architecture maintain conversational quality?).

Long-horizon web automation with hierarchical working memory folding. The survey's identification of hierarchical folding (Section 4.3.2) as a working memory strategy — decomposing tasks by subgoals, maintaining fine-grained traces only for the active subgoal, and folding completed subtrajectories into summaries — has immediate application to web automation agents operating on benchmarks like WebArena or Mind2Web. A deployment scenario: an agent navigating an e-commerce site to complete a multi-step purchase tracks detailed action-observation pairs for the current subgoal (e.g., "search for product"), but upon subgoal completion, folds this trace into a compact summary ("found product X at $Y, added to cart") and loads the summary into long-term memory while clearing the active working memory for the next subgoal ("apply discount code"). The practical benefit is bounded context window usage regardless of task length: the agent's active context never grows beyond a fixed limit because completed subgoals are continuously compressed. The survey's documentation of systems like HiAgent, Context-Folding, and AgentFold provides implementation templates; the measured benefit is success rate on tasks requiring 20+ sequential actions, where baseline agents exceed context limits and degrade, while hierarchical folders maintain performance.

Multi-agent systems with role-specialized shared memory. The survey's discussion of shared memory in multi-agent systems (Section 7.5) and the G-Memory hierarchical graph architecture (Section 4.1.2) enables a deployment pattern where a team of role-specialized agents — for instance, a software development team with architect, coder, and tester roles — shares a common memory substrate but accesses it through role-specific views. The architect sees high-level design insights and architectural patterns; the coder sees implementation-level workflows and skill libraries; the tester sees failure cases and testing strategies. The shared memory prevents duplication (the architect doesn't re-derive insights the coder already discovered) while role-specific views prevent information overload (the coder is not distracted by architectural abstractions irrelevant to the current implementation task). The practical benefit: faster convergence on complex tasks (measured in total agent-hours to complete a MetaGPT-style software project) and reduced inter-agent communication overhead (agents can retrieve what they need from memory rather than explicitly requesting it from peers). The survey's documentation of G-Memory's three-tier graph hierarchy provides the architectural blueprint.