When to Memorize and When to Stop: Gated Recurrent Memory for Long-Context Reasoning

ArXiv: 2602.10560

🎯 Pitch

Long-context reasoning in LLMs often means wading through thousands of irrelevant tokens, but GRU-Mem lets a model learn exactly when to jot something down and when to just stop reading altogether. By adding RL-trained update and exit gates to a recurrent memory loop, it slashes wasted computation while actually improving accuracy—up to 4× faster inference.

---

1. Executive Summary

This paper proposes GRU-Mem, a gated recurrent memory framework that augments the chunk-by-chunk recurrent memory paradigm of MemAgent with two text-controlled gating mechanisms—an update gate for selective memory writing (updating only on evidence-present chunks) and an exit gate for early termination (stopping the loop once sufficient evidence is collected)—trained end-to-end with reinforcement learning using dedicated reward signals for each gate behavior. Experiments on diverse long-context QA tasks (HotpotQA, SQuAD, NIAH variants from the RULER benchmark) with Qwen2.5-3B-Instruct and Qwen2.5-7B-Instruct show that GRU-Mem generally outperforms vanilla MemAgent while achieving up to 400% inference speed acceleration, with the update gate suppressing memory explosion by maintaining smaller memory sizes and the exit gate enabling accurate early stopping at the last evidence position in over 80% of cases. The RL training proves essential for gating accuracy, establishing that the learned update and exit behaviors—not merely the architectural prompt structure—are what yield both the stability and efficiency gains.

2. Context and Motivation

The Core Problem: Long-Context Reasoning Remains Brittle and Computationally Wasteful

Large language models (LLMs) exhibit a well-documented fragility when reasoning over long contexts. As input length grows, performance degrades sharply—a phenomenon confirmed across multiple benchmarks including LongBench and RULER (Bai et al., 2024; Hsieh et al., 2024). This degradation is not merely an artifact of exceeding context window limits; even when the model's architecture can theoretically accommodate the input, attention mechanisms struggle to locate and integrate evidence sparsely distributed across thousands or millions of tokens, giving rise to the well-known "lost in the middle" effect (Liu et al., 2024) where information positioned in the center of long inputs is disproportionately ignored.

This problem has direct practical consequences. Real-world applications increasingly demand reasoning over book-length documents, large-scale agent memory systems, or corpora that no single forward pass can ingest—for instance, maintaining coherent state across hours-long dialogue sessions (Chhikara et al., 2025; Packer et al., 2023) or answering questions that require synthesizing evidence from multiple documents scattered across a retrieval corpus. When LLMs fail to reliably extract and combine sparse evidence from long contexts, downstream systems accumulate errors that compound over time, particularly in agentic workflows where each decision depends on accurate recall of prior observations.

The challenge is compounded by a tension between two approaches to long-context processing, both of which carry significant drawbacks. The single-pass paradigm—feeding the entire context into the model in one forward pass using techniques like sparse attention (Child et al., 2019; Beltagy et al., 2020), linear attention (Katharopoulos et al., 2020), or positional embedding extrapolation (Su et al., 2024; Peng et al., 2025)—is architecturally elegant but fundamentally limited: computational cost scales quadratically (or linearly with approximations that sacrifice fidelity), and the model must attend to all tokens regardless of relevance, diluting attention on the few evidence spans that actually matter.

The retrieval-augmented paradigm—pre-filtering context to only relevant segments before feeding it to the LLM—introduces its own failure mode: retrieval errors are irreversible. If the retriever misses a key document, no amount of subsequent reasoning can recover. This brittleness is particularly acute for multi-hop questions where evidence dependencies span multiple documents and a single retrieval gap breaks the entire reasoning chain.

MemAgent: A Promising Third Path with Two Critical Flaws

The paper explicitly builds on MemAgent (Yu et al., 2025), which proposed a third paradigm: recurrent memory for long-context reasoning. Rather than ingesting the entire context at once or relying on a retriever to pre-filter, MemAgent reformulates long-context QA as a sequential, chunk-by-chunk memorization process in an RNN-like loop. The long context is divided into $T$ fixed-size chunks $\{C_1, \dots, C_T\}$. At each step $t$, a memory agent $\phi_\theta$ reads the current chunk $C_t$, the question $Q$, and the previous memory state $M_{t-1}$, then outputs an updated textual memory $M_t$ that distills the evidence seen so far into a compact, natural language summary. After processing all $T$ chunks, an answer agent $\psi_\theta$ conditions on the final memory $M_T$ to produce the answer $\hat{A}$. This workflow is trained end-to-end with reinforcement learning using the Multi-Conv DAPO algorithm (an extension of GRPO to multi-turn agent trajectories; Yu et al., 2025; Shao et al., 2024), which treats each agent turn (each memory update and the final answer generation) as an independent optimization target.

MemAgent demonstrated that this recurrent memory paradigm enables even small models (e.g., 7B parameters) to outperform much larger models that ingest the full context in a single pass—a finding that suggests the chunk-by-chunk approach genuinely mitigates the attention-dilution problem rather than merely circumventing context-window limits. However, despite this promise, the paper identifies two structural weaknesses inherited directly from the naive RNN-like update mechanism (Section 1, Figure 1):

Risk of Memory Explosion. At each step, MemAgent's memory agent $\phi_\theta$ generates an updated memory $M_t$ regardless of whether the current chunk $C_t$ contains any evidence relevant to the question. When a chunk is evidence-free (which is the common case—evidence is sparse by definition in needle-in-a-haystack tasks), the agent may still incorporate noise, redundant details, or hallucinated connections into the memory. Over many steps, this indiscriminate updating causes the textual memory to accumulate irrelevant content and progressively inflate. The paper observes that the memory size can grow until it hits the maximum allowed length (e.g., 1024 tokens), at which point the memory has exploded: it contains substantial noise that impedes the agent's ability to incorporate genuinely useful evidence from later chunks, and the cost of regenerating an already-bloated memory at each subsequent step compounds inference latency. This is a direct analog of the gradient explosion and vanishing problems that plagued classical RNNs before the introduction of gating mechanisms (Hochreiter and Schmidhuber, 1997; Bengio et al., 1994).

Lack of Exit Mechanism. MemAgent's workflow is hard-coded to process all $T$ chunks before answering, regardless of when the last piece of necessary evidence appears. In a typical needle-in-a-haystack scenario, the question-relevant evidence is concentrated in only a few chunks. Once those chunks have been processed, all subsequent computation is wasteful—the agent is reading and processing chunks it no longer needs. This inefficiency is amplified when the long context has been reordered (e.g., by reranking techniques that place key evidence early, as common in retrieval-augmented generation pipelines; Fan et al., 2024; Zhang et al., 2025), because the model is forced to continue scanning long after sufficient evidence has been collected. In the extreme, this means the compute cost is proportional to total context length rather than to the position of the last relevant evidence, imposing a fixed cost that cannot adapt to the evidence distribution.

Reconciling Conflicting Requirements: Stability vs. Flexibility

These two weaknesses reflect a deeper tension in the design of recurrent LLM agents. Stability demands that memory updates be conservative—only incorporating genuinely new and relevant information to avoid drift. Flexibility demands that the agent respond adaptively to the evidence it encounters—updating when needed, but also recognizing when the search can stop. The vanilla MemAgent optimizes for neither: it updates indiscriminately (sacrificing stability) and processes to the end unconditionally (sacrificing flexibility).

This tension is precisely the problem that gating mechanisms were designed to solve in classical recurrent neural networks. The Gated Recurrent Unit (GRU; Cho et al., 2014) introduced update and reset gates that let the network learn when to incorporate new input and when to retain existing state, directly addressing the exploding/vanishing gradient problems of plain RNNs. The Long Short-Term Memory (LSTM; Hochreiter and Schmidhuber, 1997) introduced input, forget, and output gates for the same purpose—enabling networks to learn selective memory operations over long sequences.

This paper draws a direct conceptual parallel: what gating did for vector-valued hidden states in RNNs, text-controlled gating can do for textual memory in recurrent LLM agents. The key insight is that the same policy model that generates memory updates can also be trained to generate decisions about whether to update and whether to continue—and that these decisions can be learned through appropriately designed reward signals within the same end-to-end RL framework that already trains the agent.

Where Prior Work Falls Short

Beyond MemAgent, several lines of work are relevant but fail to address the specific combination of problems this paper tackles:

Architectural modifications for long contexts (sparse attention, linear attention, state-space models like Mamba; Gu and Dao, 2023) focus on reducing the computational cost of processing long sequences at the architecture level. They do not address the semantic problem of knowing what to attend to or when to stop—these remain model-capability questions that architecture alone does not solve. A model with perfect linear scaling can still produce degraded outputs if its attention is diluted across irrelevant tokens.

Positional embedding extrapolation (RoPE scaling, YaRN) extends the maximum context length a model can technically process but does not improve the model's ability to effectively use that extended context. Longer context windows often reveal that degradation sets in well before the hard token limit.

Memory-augmented LLMs (MemGPT, MemOS, revisitable memory agents; Packer et al., 2023; Li et al., 2025; Shi et al., 2025) provide systems for managing memory in agentic loops but typically rely on heuristic update rules (e.g., update when the context window is full) rather than learned, evidence-conditional update decisions. They lack the tight integration of memory control into the end-to-end optimization objective that this paper pursues.

Multi-objective RL for LLMs (DeepSeek-R1, SPICE, Search Self-Play; DeepSeek-AI et al., 2025; Liu et al., 2025; Lu et al., 2025) has shown that LLMs can learn multiple distinct capabilities through appropriately designed reward signals—for instance, simultaneously acting as reasoner and data generator (He et al., 2025; Zhao et al., 2025) or learning to collaborate across roles for safety alignment (Zhang et al., 2025). This body of work provides the technical foundation for training a single policy model to exhibit both update-gating and exit-gating behaviors, but none of it has been applied to the specific problem of recurrent memory control for long-context reasoning.

How GRU-Mem Positions Itself

GRU-Mem positions itself as a minimal but targeted architectural extension to the MemAgent paradigm—not a replacement, but an augmentation that addresses the two identified failure modes while preserving the end-to-end RL training framework. The paper does not propose a new attention mechanism, a new context extension technique, or a fundamentally new memory architecture. Instead, it introduces two text-controlled gates into the existing recurrent loop:

• An update gate ($\mathcal{U}_t$) that decides whether the memory $M_t$ should be overwritten with a candidate memory $\hat{M}_t$ (when evidence is present) or retained as $M_{t-1}$ (when the chunk is evidence-free), directly addressing memory explosion.
• An exit gate ($\mathcal{E}_t$) that decides whether to terminate the loop early once the last necessary evidence has been collected, directly addressing the lack of an exit mechanism.

The gates are not implemented as separate neural modules. They are emergent behaviors of the same policy model, elicited through structured prompting (the memory agent outputs update decisions within <check> tags and exit decisions within <next> tags) and trained through dedicated reward signals ($r^{\text{update}}$ for correct update gating, $r^{\text{exit}}$ for correct exit gating) that are combined with the outcome reward in a trajectory-level advantage with a tunable balancing coefficient $\alpha$.

This design choice is deliberate and non-obvious. Rather than hard-coding rules (e.g., "update only when chunk contains the question keywords" or "exit when confidence exceeds a threshold"), which would be brittle and task-specific, the paper delegates gate control to the learned policy—enabling the model to develop its own internal heuristics for what constitutes "useful information" and "sufficient evidence" through RL optimization. This is analogous to how classical GRUs and LSTMs learned gating behavior through backpropagation rather than explicit programming, except here the gating operates at the semantic level of natural language decisions rather than continuous vector transformations.

The paper's framing as "GRU-Mem" is both a conceptual nod to the GRU inspiration and a precise description of what is new: MemAgent + gating mechanisms = GRU-Mem. The contribution is not a new training algorithm (Multi-Conv DAPO is inherited), not a new model architecture (the underlying LLM is off-the-shelf Qwen2.5-Instruct), and not a new task formulation. It is the specific integration of update and exit gating into the recurrent memory loop, trained through carefully designed reward signals, that enables simultaneous improvements in both reasoning quality (through more stable, noise-resistant memory) and computational efficiency (through early termination and smaller memory states).

Evidence Sparsity as the Enabling Assumption

A critical contextual point that makes the entire approach viable is the assumption of evidence sparsity—the idea that in long-context reasoning tasks, the information needed to answer a question is concentrated in only a few chunks, with the vast majority of chunks being evidence-free. This is explicitly not an assumption the paper invents; it is a structural property of needle-in-a-haystack benchmarks (Kamradt, 2023; Hsieh et al., 2024) and many real-world retrieval scenarios.

Evidence sparsity is what makes gating useful in the first place. If every chunk contained relevant evidence, update gating would have nothing to suppress, and exit gating could never trigger early (since evidence is distributed all the way to the end). Conversely, in the extreme sparsity regime—where only 1–3 chunks out of hundreds contain evidence—the update gate prevents 99%+ of memory writes from being unnecessary noise, and the exit gate enables stopping after processing perhaps 20% of the total context (if the last evidence happens to appear early). The paper exploits this property to the fullest: the larger the ratio of evidence-free to evidence-present chunks, the greater the potential efficiency gains from gating.

3. Technical Approach

3.1 Reader Orientation

GRU-Mem is a gated recurrent memory framework that augments the chunk-by-chunk text-processing loop of MemAgent with two learned binary decisions—should the model update its memory with this chunk's contents, and should it stop reading—implemented entirely within a single LLM policy trained end-to-end with reinforcement learning. The system solves the dual problem of memory explosion (indiscriminate accumulation of noise over hundreds of chunks) and wasted computation (processing chunks long after all necessary evidence has been collected) by giving the model explicit control over when to write and when to exit, trained through dedicated reward signals that separately reward correct gating behavior.

3.2 Big-Picture Architecture (Diagram in Words)

The GRU-Mem system has five major components arranged in a sequential pipeline:

1. Context Chunker: Takes the full long context $\mathcal{C}$ (which can be up to millions of tokens) and splits it into $T$ fixed-size chunks of 5,000 tokens each: $\{C_1, C_2, \dots, C_T\}$. This is a purely mechanical preprocessing step with no learned component—it simply partitions the raw text into equal-sized segments so the model can process them one at a time.

2. Memory Agent ($\phi_\theta$): The core recurrent component. At each step $t$, it receives three inputs: the original question $Q$, the current chunk $C_t$, and the previous memory state $M_{t-1}$. It produces three outputs: an update gate decision $\mathcal{U}_t$ (True/False, indicating whether the chunk contains useful evidence), a candidate memory $\hat{M}_t$ (a natural language summary of evidence seen so far, generated regardless of the gate decision), and an exit gate decision $\mathcal{E}_t$ (True/False, indicating whether all necessary evidence has been collected). The memory agent is instantiated by Qwen2.5-3B-Instruct or Qwen2.5-7B-Instruct with a structured output prompt that constrains its generation to a parseable format.

3. Gating Logic (Deterministic Controller): A hard-wired rule-based controller that interprets the memory agent's textual outputs and makes binary decisions. If $\mathcal{U}_t = \text{True}$, the current memory $M_t$ is overwritten with the candidate memory $\hat{M}_t$; if False, $M_t$ is set to $M_{t-1}$ (the candidate is discarded). If $\mathcal{E}_t = \text{True}$ and the exit gate is enabled at inference, the loop terminates immediately; otherwise processing continues to chunk $t+1$. This controller performs no learning—it simply parses the tagged output format and executes the indicated operations.

4. Answer Agent ($\psi_\theta$): After the loop terminates (either via exit gate or after processing all $T$ chunks), this agent receives the question $Q$ and the final memory $M_t$ and generates the predicted answer $\hat{\mathcal{A}}$, enclosed in \boxed{} notation. Critically, $\phi_\theta$ and $\psi_\theta$ are the same underlying LLM—they share parameters and differ only in their prompting format, meaning the model jointly learns memory management and question answering through a single optimization process.

5. RL Training Engine (Multi-Conv DAPO): The optimization framework that trains the shared policy model $\pi_\theta$ to produce correct answers and correct gating behavior. It computes three distinct reward signals—outcome reward $r^{\text{outcome}}$, update reward $r^{\text{update}}$, and exit reward $r^{\text{exit}}$—and combines them via a trajectory-level advantage with a tunable mixing coefficient $\alpha$. The training runs groups of $G = 16$ parallel trajectories per prompt, computes advantages per-turn and per-trajectory, and updates the policy using a clipped importance sampling objective with a KL penalty against a frozen reference model.

Information flow: Question $Q$ and context $\mathcal{C}$ enter the system → context is chunked → for each chunk $t = 1, 2, \dots$ (until exit or exhaustion): memory agent reads $(Q, C_t, M_{t-1})$ → outputs $(\mathcal{U}_t, \hat{M}_t, \mathcal{E}_t)$ → gating logic updates $M_t$ accordingly → if exit triggered, break → answer agent reads $(Q, M_{\text{final}})$ → produces $\hat{\mathcal{A}}$.

3.3 Roadmap for the Deep Dive

• First, the formal task decomposition—how long-context QA is framed as a sequential memorization problem with explicit evidence sparsity assumptions, since this framing determines what the gates need to accomplish.
• Second, the memory agent's structured output format and the deterministic gating logic that parses it, since this is the interface through which learned decisions become concrete memory operations.
• Third, the RL training framework—reward design (outcome, update, exit, format), advantage calculation (trajectory-level vs. turn-level, the $\alpha$ mixing coefficient), and the Multi-Conv DAPO loss—since this is how the model acquires gating behavior.
• Fourth, the inference-time configuration options (with vs. without exit gate), since the training always uses the exit gate but inference can optionally disable it for tasks where exiting early would be harmful.
• Fifth, the key hyperparameters and training configuration that make the system work.

3.4 Detailed, Sentence-Based Technical Breakdown

This is a systems-augmentation paper whose core idea is that adding two learned gating mechanisms—update gating (selective writing) and exit gating (early stopping)—to the chunk-by-chunk recurrent memory paradigm resolves long-standing instability and inefficiency problems, and that these gating behaviors can be trained through dedicated reward signals within the existing end-to-end RL framework.

---

Task Decomposition: Sequential Memorization Under Evidence Sparsity

The paper formalizes long-context question answering as operating over a triplet $(\mathcal{Q}, \mathcal{C}, \mathcal{A})$, where $\mathcal{Q}$ is the question, $\mathcal{C}$ is a long context (potentially millions of tokens), and $\mathcal{A}$ is the ground-truth answer (Section 2.1). The defining structural property exploited by the entire approach is evidence sparsity: only a few evidence pieces $\{e_k\}_{k=1}^K$ within $\mathcal{C}$ are relevant to answering $\mathcal{Q}$, and these pieces are sparsely distributed across the context. In the needle-in-a-haystack tasks that dominate the evaluation (SK, MK, MQ, MV variants from the RULER benchmark), $K$ is typically 1–3 evidence chunks out of hundreds of total chunks.

The context $\mathcal{C}$ is mechanically partitioned into $T = \lceil |\mathcal{C}| / s \rceil$ fixed-size chunks, where $s = 5000$ tokens is the chunk size used for all experiments (Appendix B). This chunking is deterministic and lossless—every token appears in exactly one chunk, and the original ordering is preserved. The rationale for the 5,000-token chunk size is not explicitly justified in the paper, but it represents a balance: small enough that individual chunks can be processed within the model's context window alongside the question and memory state, but large enough that evidence spans (which may be multi-sentence or multi-paragraph) are not fragmented across chunk boundaries arbitrarily.

Once chunked, the task reduces to a sequential memory accumulation problem: at each step $t$, the model must decide (a) whether chunk $C_t$ contains evidence that should be incorporated into its running summary $M_t$, and (b) whether the evidence accumulated so far in $M_t$ is sufficient to answer the question. This decomposition is what makes gating both possible and necessary—if evidence were uniformly distributed, gate (a) would always be True and gate (b) would always be False until the end, making gating useless. The paper thus depends critically on the empirical fact that long-context reasoning benchmarks exhibit extreme evidence sparsity.

Why this framing matters for what follows: The entire reward design for the update gate (Section 3.2.1) depends on having ground-truth labels for which chunks contain evidence—labels that are available because the benchmark construction process plants known evidence pieces at known positions. Without this supervised signal for update correctness, the $r^{\text{update}}$ reward could not be defined.

---

Memory Agent: Structured Output Format and Gating Logic

The memory agent $\phi_\theta$ is the workhorse of GRU-Mem. At each step $t$, it receives a prompt that concatenates the question $Q$, the previous memory $M_{t-1}$, and the current chunk $C_t$, and it must generate a response in a strictly structured format that the deterministic controller can parse (Section 3.1, Figure 3, and Appendix C, Figure 10a).

The structured output format consists of four tagged segments:

1. Reasoning section ( thinking ... response): Free-form natural language reasoning where the model analyzes whether the current chunk contains useful information. This section is not parsed for gate decisions—it exists purely as chain-of-thought to improve decision quality, following the principle that explicit reasoning before structured output improves accuracy.

2. Update gate decision (<check>yes</check> or <check>no</check>): The model declares whether the current chunk contains evidence relevant to the question. <check>yes</check> sets the update gate status $\mathcal{U}_t = \text{True}$; <check>no</check> sets $\mathcal{U}_t = \text{False}$. This is a binary forced choice—the format reward penalizes any other content between the <check> tags (Section 3.2.1).

3. Candidate memory (<update> ... </update>): A natural language summary generated regardless of the update gate decision. If the update gate is True, this candidate becomes the new memory $M_t$. If False, it is generated but immediately discarded ($M_t \leftarrow M_{t-1}$). This design—always generating the candidate even when it will be discarded—is significant: it means the model cannot "save computation" by generating a short placeholder when skipping an update; it must still produce a coherent update that will be scored by the gate reward, maintaining symmetry in the generation cost across chunks. This also ensures the model's per-step generation length doesn't leak information about its gate decision to an external observer.

4. Exit gate decision (<next>continue</next> or <next>end</next>): <next>continue</next> sets the exit gate status $\mathcal{E}_t = \text{False}$ (keep processing); <next>end</next> sets $\mathcal{E}_t = \text{True}$ (terminate immediately and proceed to answer agent). This is also a binary forced choice.

The deterministic gating logic (Algorithm 1 in Section 3.1) operates as follows at each step $t$:

1. Call the memory agent: $\mathcal{U}_t, \hat{M}_t, \mathcal{E}_t = \phi_\theta(Q, C_t, M_{t-1})$.
2. If $\mathcal{U}_t = \text{True}$, set $M_t \leftarrow \hat{M}_t$ (update). Otherwise, set $M_t \leftarrow M_{t-1}$ (retain previous).
3. If the exit gate is enabled at inference (use_exit_gate == True) and $\mathcal{E}_t = \text{True}$, break the loop immediately.
4. Otherwise, increment $t \leftarrow t + 1$ and continue.

A critical implementation detail: the exit condition is checked after the memory update at step $t$. This means that when the model correctly identifies chunk $t$ as the last evidence chunk, it (a) sets $\mathcal{U}_t = \text{True}$ to incorporate that evidence, (b) sets $\mathcal{E}_t = \text{True}$ to signal completion, and (c) the updated memory $M_t$ (now containing all evidence including the chunk-$t$ evidence) is passed to the answer agent. The exit does not skip the update on the exit-triggering chunk—it only skips subsequent chunks. This is the correct behavior because the last evidence chunk must still be processed before answering.

Design choice: why text-controlled gates rather than separate classifier heads? The paper embeds gate decisions within the language model's own text generation rather than adding separate binary classification heads on top of the LLM's hidden states. This has several implications: (a) no architectural modifications to the base model are needed—the same Qwen2.5-Instruct checkpoint is used as-is with only prompting changes; (b) the gate decisions can be informed by the full reasoning in the thinking block, which can reference semantic content, logical relationships, and prior memory state in ways that a fixed hidden-state classifier might struggle to represent; (c) the gates are trained through the same RL objective as memory content generation, ensuring that gate learning and content learning share the same optimization signal and representation space; (d) parsing failures (e.g., malformed tags) are penalized by the format reward, providing a direct training signal to maintain parseability.

The tradeoff is that text-controlled gates are less compute-efficient than classifier heads (generating tens of tokens for a binary decision vs. a single sigmoid output), but the paper argues this cost is negligible relative to the cost of generating the candidate memory itself (which can be hundreds of tokens).

---

Answer Agent: Terminal Question Answering

The answer agent $\psi_\theta$ is invoked exactly once per trajectory, after the memory loop terminates (either via exit gate or after processing all $T$ chunks). Its prompt (Appendix C, Figure 10b) provides the question $Q$ and the final memory $M_{\text{final}}$, and asks it to produce the answer enclosed in \boxed{} notation.

The answer agent and memory agent share the same parameterized policy model $\pi_\theta$—they are the same Qwen2.5-Instruct checkpoint, differentiated only by their prompt templates. This shared-parameter design means that improvements to the memory agent's gating behavior (learned during RL training) can potentially transfer to the answer agent's reasoning quality through shared representations, and vice versa. The Multi-Conv DAPO algorithm treats both agent roles within a single trajectory as separate conversation "turns" that are optimized jointly, meaning gradients from both the answer correctness reward and the gate behavior rewards flow through the same parameters.

The answer is evaluated against the ground-truth answer $\mathcal{A}$ using an equivalence check is_equiv(·, ·) that handles formatting variations (the specific equivalence function is not detailed in the paper but is inherited from the MemAgent evaluation protocol). The binary outcome of this check defines the outcome reward $r^{\text{outcome}}$ used in RL training (Equation 6).

---

RL Training Framework: Reward Design

The central technical contribution of GRU-Mem is the reward design that enables a single policy model to simultaneously learn memory content generation, update gating, and exit gating through end-to-end RL. Four distinct reward signals are defined (Section 3.2.1), covering answer quality, update accuracy, exit timing, and format compliance.

Outcome Reward ($r^{\text{outcome}}$): The simplest reward—a binary signal indicating whether the final answer matches the ground truth:

$r^{\text{outcome}} = \mathbb{I}(\text{is\_equiv}(\mathcal{A}, \hat{\mathcal{A}}))$

where $\mathbb{I}(\cdot)$ is the indicator function (returns 1 if the condition is True, 0 otherwise), $\mathcal{A}$ is the ground-truth answer, and $\hat{\mathcal{A}}$ is the model's predicted answer.

What it computes: a single scalar (0 or 1) for the entire trajectory, assigned to all conversation turns $(o_{g,1}, o_{g,2}, \dots, o_{g,T_g})$ within group $g$. Correct answer yields 1 for all turns; incorrect yields 0.

Why this form and assignment: the outcome reward is shared across all turns because answer correctness depends on the entire sequence of memory updates—a correct final answer implies (probabilistically) that the memory was managed well, even if individual update decisions were imperfect. Conversely, an incorrect answer might result from a single bad update decision that corrupted the memory, but the RL signal can't isolate which turn was responsible, so the penalty is distributed. This is standard in multi-turn RL for agent trajectories (inherited from MemAgent's Multi-Conv DAPO) and reflects the credit assignment challenge: the outcome is a function of the entire trajectory, not any single turn in isolation.

Update Reward ($r^{\text{update}}_t$): A per-turn reward that directly supervises the update gate decision. For each chunk $C_t$, the training data provides a ground-truth label of whether that chunk contains evidence relevant to $Q$ (available because benchmarks plant evidence at known positions). The reward is:

$r_t^{\text{update}} = \begin{cases} 1, & \mathcal{U}_t \text{ is correct} \\ -1, & \mathcal{U}_t \text{ is incorrect} \end{cases}$

where correctness means: $\mathcal{U}_t = \text{True}$ when $C_t$ contains evidence, and $\mathcal{U}_t = \text{False}$ when $C_t$ is evidence-free.

What it computes: a per-step scalar of +1 or −1 depending on whether the memory agent's <check> decision matches the ground-truth evidence-presence label for that chunk. This reward is calculated independently for each step $t$ and is not shared across turns.

Why +1/−1 (symmetric penalty) rather than 1/0 or only-positive: the symmetric penalty ensures that both types of errors (updating on evidence-free chunks, which causes memory explosion, and failing to update on evidence-present chunks, which causes missing evidence) are punished equally. A 1/0 reward (reward correct, no penalty for incorrect) would provide no disincentive for unnecessary updates, which is precisely the behavior GRU-Mem is designed to suppress. A +1/−0.5 or other asymmetric scheme would bias the model toward one type of error. The paper does not experiment with alternative reward magnitudes, but the symmetry is principled given that both error types are harmful (though through different mechanisms: unnecessary updates degrade memory quality over time, while missed updates cause information loss immediately).

Exit Reward ($r^{\text{exit}}$): A trajectory-level reward that penalizes incorrect exit timing. The ground-truth "correct" exit position is $t_{\text{last evidence}}$—the index of the chunk containing the last piece of evidence needed to answer the question. The reward is:

$r^{\text{exit}} = \begin{cases} -0.75, & t_{\text{exit}} < t_{\text{last evidence}} \\ 0, & t_{\text{exit}} = t_{\text{last evidence}} \\ -0.5, & t_{\text{exit}} > t_{\text{last evidence}} \end{cases}$

where $t_{\text{exit}}$ is the step at which the memory agent produced <next>end</next> (triggering the exit gate), with the convention that if the agent never exits (processes all $T$ chunks), $t_{\text{exit}} > t_{\text{last evidence}}$ applies.

What it computes: a trajectory-level scalar penalty that depends on when the agent chose to exit relative to the ground-truth last evidence position. Correct exit (exactly at $t_{\text{last evidence}}$) yields 0 (no penalty, but also no positive reward—correct exit behavior is expected, not bonused). Early exit (exiting before collecting all evidence) yields −0.75. Late exit (processing more chunks than necessary before exiting) yields −0.5. If the exit gate is disabled at inference, this reward is effectively not computed for those trajectories.

Why asymmetric penalties (−0.75 for early vs. −0.5 for late): the paper explicitly states (Equation 9) that "an early exit behavior gets more punishment than a late exit behavior due to the evidence insufficiency." The reasoning is that early exit is a hard error—it means the model will answer the question without having seen all necessary evidence, making correct answering impossible regardless of reasoning quality. Late exit is an efficiency error—it wastes computation but doesn't preclude correct answering (the model still has all necessary evidence). The −0.75 vs. −0.5 asymmetry encodes this different severity: losing information is worse than wasting computation.

Why 0 for correct exit rather than +1: the paper does not explicitly justify this choice, but it follows the principle that the exit gate's "default" reward should be neutral, with the outcome reward providing the positive signal for trajectories where good exit timing contributed to a correct answer. Adding a positive exit reward on top of the outcome reward could create perverse incentives where the model learns to exit correctly on trajectories where it can't answer correctly (getting the exit reward while sacrificing the outcome reward would still yield positive total reward). The neutral-correct / penalize-incorrect design makes the exit reward purely a regularizer that suppresses bad behavior without incentivizing exit for its own sake.

Format Reward ($r^{\text{format}}$): A trajectory-level strict format compliance check:

$r^{\text{format}} = \begin{cases} 1, & \text{The format of all turns is correct} \\ 0, & \text{Otherwise} \end{cases}$

Format correctness requires that every conversation output $o_{g,t}$ (for all $t = 1, \dots, T_g$) contains the required tagged sections ( thinking ... response, <check> ... </check>, <update> ... </update>, <next> ... </next>) in the correct order, with <check> content being exactly "yes" or "no" and <next> content being exactly "continue" or "end."

What it computes: a binary trajectory-level check: if every turn in the trajectory passes all format constraints, the reward is 1 (for all turns in the trajectory); if any turn fails any constraint, the reward is 0 for all turns.

Why this strict "all-or-nothing" design: the paper states (Section 3.2.1) that "we can not infer whether the incorrect format is caused by the previous erroneous parsing." If turn $t$ produces a malformed output, it's ambiguous whether the error originated in turn $t$'s generation or was caused by the controller incorrectly parsing turn $t-1$'s output and feeding corrupted context to turn $t$. The strict all-or-nothing format reward avoids this credit assignment problem by penalizing the entire trajectory uniformly, which is conservative but ensures the model learns to be format-robust in all contexts. In practice, this is not overly harsh because format compliance is quickly learned (Figure 21a shows near-100% format correctness within a few training steps).

The overall trajectory-level reward ($r_g^{\text{traj}}$): The three trajectory-level rewards (outcome, exit, format) are combined additively:

$r_g^{\text{traj}} = r_g^{\text{outcome}} + r_g^{\text{exit}} + r_g^{\text{format}}$

This sum is shared across all turns within trajectory $g$. The update reward $r_t^{\text{update}}$ is not included in this trajectory-level sum—it is handled separately at the turn level in the advantage calculation (see below). This separation is deliberate: the trajectory-level rewards capture global properties of the entire trajectory (did we answer correctly? did we exit at the right time? was the format valid?), while the update reward captures per-step decision quality that should not be contaminated by whether the final answer happened to be correct.

---

RL Training Framework: Advantage Calculation

The paper uses a disentangled advantage calculation scheme (Section 3.2.2, Figure 4) that separately computes advantages at the trajectory level (for outcome, exit, and format rewards) and at the turn level (for the update reward), then mixes them with a hyperparameter $\alpha$. This design is inspired by recent work on stabilizing multi-reward RL training (Shi et al., 2025; Liu et al., 2026) where different reward signals have different temporal granularities.

Trajectory-level advantage ($\hat{A}_{g,t,i}^{\text{traj}}$): Computed by comparing the total trajectory reward $r_g^{\text{traj}}$ for group $g$ against the mean trajectory reward across all $G$ groups in the batch:

$\hat{A}_{g,t,i}^{\text{traj}} = r_g^{\text{traj}} - \frac{1}{G} \sum_{g=1}^G r_g^{\text{traj}}$

where $G = 16$ is the number of trajectories (groups) generated per prompt during training (Appendix B).

What it computes: a baseline-subtracted advantage that measures how much better (or worse) trajectory $g$ performed compared to the average trajectory in the batch. All tokens $(t, i)$ within trajectory $g$ share the same trajectory-level advantage value. If trajectory $g$ had an above-average total reward, all its tokens receive a positive advantage signal; if below-average, a negative signal.

Why mean subtraction rather than a learned value baseline: the paper follows the GRPO-style advantage calculation (Shao et al., 2024), which uses the group mean as a baseline. This is simpler than training a separate value network and has been shown effective in recent LLM RL work (DeepSeek-R1, DAPO). The mean over $G = 16$ groups provides a reasonable estimate of expected reward for that prompt under the current policy.

Turn-level advantage ($\hat{A}_{g,t,i}^{\text{turn}}$): Computed separately for each step $t$, comparing the update reward for group $g$ at step $t$ against the mean update reward at the same step across groups:

$\hat{A}_{g,t,i}^{\text{turn}} = r_{g,t}^{\text{update}} - \frac{1}{G_t} \sum_{g=1}^{G_t} r_{g,t}^{\text{update}}$

where $G_t \leq G$ is the number of groups that reached step $t$ (some groups may have exited earlier due to the exit gate, reducing the effective group size at later steps).

What it computes: a per-step advantage that isolates the quality of the update gate decision at step $t$, independent of trajectory-level outcomes. A positive value means group $g$ made a better-than-average update decision at step $t$; negative means worse-than-average.

Why separate turn-level advantage calculation: the update reward (+1/−1) operates at a completely different scale from the trajectory-level rewards (which range from approximately −0.75 to 2.0 depending on outcome + exit + format). If the update reward were simply added into $r_g^{\text{traj}}$ and a single advantage computed, the small per-step update signal would be drowned out by the trajectory-level signals, and the model might never learn accurate gating. By computing advantages separately and then mixing them, the paper ensures that the update gate receives a clear, un-attenuated training signal at each step.

The mixing coefficient $\alpha$: The total per-token advantage $\hat{A}_{g,t,i}$ used in the policy gradient is a convex combination:

$\hat{A}_{g,t,i} = \alpha \hat{A}_{g,t,i}^{\text{traj}} + (1 - \alpha) \hat{A}_{g,t,i}^{\text{turn}}$

where $\alpha \in [0, 1]$ is a hyperparameter. The paper experiments with $\alpha \in \{1.0, 0.9, 0.5\}$ (Section 4.3, Figure 8) and selects $\alpha = 0.9$ as the default.

What this weighting does: $\alpha$ controls the relative influence of trajectory-level rewards (answer correctness, exit timing, format) vs. turn-level rewards (update accuracy). At $\alpha = 1.0$, the update reward has no direct influence on the policy gradient—the model learns gating only indirectly through how update decisions affect final outcomes. At $\alpha = 0.5$, the update and trajectory rewards are weighted equally. At $\alpha = 0.9$, trajectory rewards dominate but the update reward still provides a non-negligible per-step training signal.

Why $\alpha = 0.9$ is chosen: the ablation in Figure 8 shows that $\alpha = 1.0$ leads to a sharp drop in update accuracy on evidence-free chunks (the model updates indiscriminately, causing memory explosion), because without the per-step update reward, the model's only signal about gating quality comes from the final outcome—and that signal is noisy and delayed. $\alpha = 0.5$ provides strong update supervision but degrades trajectory-level performance (lower validation reward, Figure 8d), likely because the model over-optimizes for gate accuracy at the expense of answer quality. $\alpha = 0.9$ balances these effects: the 10% weight on the update advantage is sufficient to maintain high accuracy on both evidence-present and evidence-free chunks (Figures 8a, 8b), while the 90% weight on trajectory advantage keeps the model focused on the ultimate objective of correct answering.

---

RL Training Framework: The Multi-Conv DAPO Loss

The policy model $\pi_\theta$ is optimized using the Multi-Conv DAPO algorithm (Equation 3, Section 2.2), which extends the Group Relative Policy Optimization (GRPO) objective to multi-turn conversation trajectories. The loss for a batch of $G$ trajectories is:

$\mathcal{J}(\theta) = \mathbb{E}_{(\mathcal{Q}, \mathcal{A}) \sim \mathcal{D}, \{o_{g,t}\}_{g=1}^G \sim \pi_{\theta_{\text{old}}}(\cdot | \mathcal{Q}, o_{g,t-1})} \left[ \frac{1}{\sum_{g=1}^G \sum_{t=1}^{T_g} |o_{g,t}|} \sum_{g=1}^G \sum_{t=1}^{T_g} \sum_{i=1}^{|o_{g,t}|} \left( \ell_{g,t,i}^{\text{clip}} - \beta D_{\text{KL}}(\pi_{\theta} \parallel \pi_{\text{ref}}) \right) \right]$

where $T_g$ is the number of turns in trajectory $g$ (memory turns plus one answer turn), $|o_{g,t}|$ is the number of tokens in turn $t$, and $\ell_{g,t,i}^{\text{clip}}$ is the clipped surrogate objective:

$\ell_{g,t,i}^{\text{clip}} = \min \left( \rho_{g,t,i}(\theta) \hat{A}_{g,t,i}, \text{clip}(\rho_{g,t,i}(\theta), 1 - \varepsilon_{\text{low}}, 1 + \varepsilon_{\text{high}}) \hat{A}_{g,t,i} \right)$

where $\rho_{g,t,i}(\theta)$ is the importance sampling ratio:

$\rho_{g,t,i}(\theta) = \frac{\pi_{\theta}(o_{g,t,i} | \mathcal{Q}, o_{g,t,<i})}{\pi_{\theta_{\text{old}}}(o_{g,t,i} | \mathcal{Q}, o_{g,t,<i})}$

The clip ratio is set to $\varepsilon = 0.20$ (Appendix B, Table 3), with separate lower and upper clipping bounds $\varepsilon_{\text{low}}$ and $\varepsilon_{\text{high}}$ as introduced in DAPO (Yu et al., 2025)—though their specific values are not stated in the GRU-Mem paper, they are inherited from the DAPO defaults.

What this objective computes: for each token $i$ in each turn $t$ of each trajectory $g$, the loss compares the current policy's probability of generating that token (under the trajectory context so far) to the old policy's probability (from the rollout generation phase). If the advantage $\hat{A}_{g,t,i}$ is positive (the trajectory did better than average), the loss encourages increasing the token's probability, but only up to a factor of $1 + \varepsilon_{\text{high}}$ relative to the old policy. If the advantage is negative, the loss encourages decreasing the probability, but only down to $1 - \varepsilon_{\text{low}}$. The clipping prevents destructively large policy updates that could cause the model to forget previously learned behaviors—a standard technique inherited from PPO (Schulman et al., 2017).

The KL penalty term $-\beta D_{\text{KL}}(\pi_{\theta} \parallel \pi_{\text{ref}})$ further regularizes the policy to stay close to a frozen reference model $\pi_{\text{ref}}$ (the pre-trained Qwen2.5-Instruct checkpoint), preventing reward hacking where the model learns to produce high-reward outputs that diverge dramatically from coherent language. The coefficient $\beta$ is not explicitly stated in the paper but is inherited from the verl framework defaults.

Key normalization detail: the loss is normalized by the total number of tokens across all turns and all trajectories in the batch ($\sum_{g=1}^G \sum_{t=1}^{T_g} |o_{g,t}|$). This means each token contributes equally to the gradient, regardless of which trajectory or turn it belongs to. This is significant because trajectory lengths $T_g$ vary—some trajectories exit early (fewer turns), some produce longer memory outputs (more tokens per turn). Token-level averaging ensures that trajectories with more tokens don't dominate the gradient simply because they're longer.

The per-token advantage assignment: all tokens within turn $t$ of trajectory $g$ receive the same advantage $\hat{A}_{g,t,i}$ (the combined trajectory-turn advantage from Equation 13). This is a form of turn-level credit assignment—the model assumes that all tokens contributing to a given turn share responsibility for that turn's aggregate decision quality. This is a coarse approximation (some tokens in a turn may be more critical than others for the gate decision), but it avoids the complexity of token-level reward decomposition and has been shown to work in practice for multi-turn agent training (MemAgent, DAPO).

---

Training Configuration and Hyperparameters

The paper provides detailed training hyperparameters in Appendix B, Table 3. These are critical for reproducibility and understanding the scale of the training process.

Model and data:

• Base models: Qwen2.5-3B-Instruct and Qwen2.5-7B-Instruct (Yang et al., 2024), used off-the-shelf without architectural modification.
• Training data: the same dataset as MemAgent (Yu et al., 2025), which is constructed from HotpotQA-style multi-hop questions with planted evidence in long contexts of varying lengths. The exact dataset size is not stated in the GRU-Mem paper, but the MemAgent paper describes it as containing diverse multi-hop reasoning examples with context lengths from thousands to hundreds of thousands of tokens.
• Chunk size: 5,000 tokens (fixed for all experiments and all context lengths). Given that the evaluation contexts range from 7K to 896K tokens (Section 4), the number of chunks $T$ ranges from approximately 2 (for the shortest contexts) to 180 (for the longest).

RL-specific hyperparameters:

• Maximum prompt length: 8,192 tokens. This constrains the total input (question + previous memory + current chunk) to fit within the model's context window. With a 5,000-token chunk, approximately 3,192 tokens remain for the question and previous memory—this implicitly limits how large the memory can grow (consistent with the "memory explosion" problem where memory hits its maximum budget).
• Maximum response length: 2,048 tokens. This caps the length of the memory agent's output (reasoning + gate decisions + candidate memory) per turn. The memory explosion problem in MemAgent manifests when the candidate memory approaches this limit.
• Clip ratio: 0.20 (standard PPO value; the paper does not specify separate $\varepsilon_{\text{low}}$ and $\varepsilon_{\text{high}}$ values, but DAPO defaults likely apply).
• Learning rate: $1 \times 10^{-6}$ (relatively low, typical for fine-tuning large LMs with RL to avoid catastrophic forgetting).
• Sampling temperature (training): 1.0 (high temperature to encourage exploration of different gating and memory strategies).
• Top-p (training): 1.0 (no nucleus filtering during training; the model can sample from the full distribution).
• Sampling temperature (validation): 1.0.
• Top-p (validation): 0.7 (nucleus sampling with $p = 0.7$ during validation to reduce variance while maintaining some diversity).
• Training batch size: 128 (the number of distinct questions/prompts processed per gradient step).
• Rollout number ($N$): 16 (the number of trajectories $G$ generated per prompt for group-based advantage calculation).
• PPO mini-batch size: 128.
• Learning rate warmup: 20 steps (standard practice to stabilize early training).
• Early stopping criterion: "stop training until we observe the convergence of reward on the validation set" (Appendix B)—a manual inspection criterion rather than an automated threshold.

Hardware: all experiments run on 8-GPU nodes. The specific GPU type is not stated, but the verl framework supports common datacenter GPUs (A100, H100).

Training duration: not explicitly stated in terms of steps or wall-clock time. The paper says training continues until validation reward converges (Figure 8d suggests this takes on the order of tens to low hundreds of steps, but no exact number is given).

---

Inference Configuration: With and Without Exit Gate

A subtle but important design choice: the model is always trained with the exit gate enabled—during training, once the memory agent outputs <next>end</next>, the trajectory terminates immediately and proceeds to the answer agent. However, the paper provides two inference modes (Section 3.3):

• With exit gate (w EG): The trained model's exit decisions are honored. If the model outputs <next>end</next> at step $t$, the loop terminates and the answer agent is invoked immediately. The exit reward $r^{\text{exit}}$ during training ensures the model has learned to make this decision appropriately.

• Without exit gate (w/o EG): The model's exit decisions are ignored during inference—the loop processes all $T$ chunks regardless of what the model outputs in the <next> tags. The answer agent is only invoked after the final chunk.

Why provide two inference modes: the paper recognizes that some tasks require reading the entire context regardless of evidence sufficiency. The example given (Section 3.3) is the multi-values (MV) task from the RULER benchmark: "What are all the special magic numbers for xxx." In this task, the answer depends on collecting all instances of a pattern across the entire context, so early exit would be harmful—the model must process to the end to ensure completeness. The w/o EG mode allows GRU-Mem to handle such tasks correctly while still benefiting from the update gate (which suppresses unnecessary writes on evidence-free chunks, even if the loop must continue).

In the results (Table 1), the MV task only reports numbers for the w/o EG configuration—the w EG column shows a dash ("-") for MV, confirming that the exit gate is intentionally disabled for this task. For all other tasks (HQA, SQuAD, SK series, MK series, MQ), both modes are evaluated and compared.

Why this design doesn't require separate training: the model is trained to produce exit decisions (it always generates <next>continue</next> or <next>end</next> as part of its structured output), but whether those decisions are acted upon is a runtime configuration flag. The training process doesn't need to know which inference mode will be used—the model learns exit behavior from the exit reward, and at inference time, the user chooses whether to honor those learned decisions based on task characteristics. This is an elegant separation of concerns: the model learns when it could exit (a capability), and the deployment configuration decides whether to use that capability (a policy choice).

---

Summary of Design Decisions and Their Justifications

• Text-controlled gates rather than classifier heads: enables reasoning-informed gating decisions, requires no architectural modifications, and allows the same RL objective to optimize both content and gating.
• Symmetric +1/−1 update reward: penalizes both unnecessary updates (memory explosion) and missed updates (information loss) equally, reflecting that both errors are harmful.
• Asymmetric exit penalties (−0.75 early, −0.5 late): encodes the different severity of hard errors (missing evidence) vs. efficiency errors (wasted computation).
• Disentangled trajectory-level and turn-level advantages with $\alpha = 0.9$: ensures the per-step update gate signal isn't drowned out by trajectory-level rewards while keeping the model focused on answer correctness as the primary objective.
• All-or-nothing format reward: avoids ambiguous credit assignment when format errors could originate from either the current turn or corrupted context from previous turns.
• Always-generate-candidate-memory design: even when the update gate is False, the model generates $\hat{M}_t$, preventing the generation length from leaking gate decisions and ensuring symmetric per-step computation cost.
• Training with exit gate always enabled, inference optionally disabled: decouples capability learning (the model learns when to exit) from deployment policy (whether to use that capability), enabling task-specific configuration without retraining.
• Chunk size of 5,000 tokens: balances fitting $(Q, M_{t-1}, C_t)$ within the 8,192-token prompt limit while keeping chunks large enough that evidence spans aren't arbitrarily fragmented.
• Multi-Conv DAPO with token-level loss normalization: ensures all trajectories contribute equally per-token to the gradient, preventing longer trajectories from dominating training.

4. Key Insights and Innovations

Innovation 1: Gating as a Learned Semantic Decision, Not a Continuous-Valued Activation

The dominant paradigm for gating in neural sequence models—from LSTMs (Hochreiter and Schmidhuber, 1997) through GRUs (Cho et al., 2014) to modern state-space models (Gu and Dao, 2023)—treats gates as continuous-valued activations (sigmoid outputs between 0 and 1) that modulate the flow of vector-valued hidden states through differentiable element-wise multiplication. This is mathematically elegant (gradients flow smoothly through the gate) but fundamentally restricts gating to operate on distributed representations: the gate can partially attenuate each dimension of a hidden vector, but it cannot make a discrete semantic judgment like "this chunk is irrelevant, therefore the entire memory should be preserved as-is."

GRU-Mem makes a conceptual leap that is easy to overlook because it is implemented so simply. Instead of training a classifier head to output a scalar gate value, the paper delegates gating to the language model's own text generation, producing discrete binary decisions ("yes"/"no", "continue"/"end") expressed in natural language tags. This transforms gating from a continuous, sub-symbolic operation into a semantic reasoning act: the model must articulate why a chunk is relevant or irrelevant in its thinking block, then render an explicit binary judgment. The gate is no longer a learned linear transform of hidden states; it is the output of a reasoning process that can reference the question semantics, the prior memory content, and the chunk's relationship to both.

Why this is more than an implementation trick: it changes what the gate means. A continuous GRU gate can learn that certain input patterns should be attenuated, but it does so through statistical correlation—if a particular distribution of token embeddings correlates with irrelevance, the gate learns to output values near zero. It cannot condition its decision on the semantic content of a multi-paragraph chunk evaluated against a specific question. GRU-Mem's text-controlled gate can, because the gate decision is produced by the same autoregressive reasoning process that generates the memory content itself. The gate is informed by the same conceptual understanding that the model uses to summarize evidence—not by a separate, shallower signal path.

The field's prior approach to controlling LLM memory was either heuristic (update when the context window is full, as in MemGPT; Packer et al., 2023) or continuous-attention-based (learned scalar weights on retrieved documents, as in retrieval-augmented generation). Neither approach enables the model to make evidence-conditional decisions about memory preservation. GRU-Mem's discrete semantic gating is a fundamentally different capability—it requires the model to know whether something is relevant, not merely to have learned a statistical association that correlates with relevance during training. The evidence that this capability is genuinely learned (not just prompted) comes from the ablation in Figure 9: without RL training (i.e., using the base Qwen2.5-7B-Instruct model with the same structured prompt), performance drops sharply, confirming that the gating behavior is acquired through optimization, not merely elicited by the prompt format.

Innovation 2: Disentangling Credit Assignment for Heterogeneous Rewards in Multi-Turn Agent Training

The paper inherits MemAgent's Multi-Conv DAPO algorithm, which treats each conversation turn as an independent optimization target but assigns the same trajectory-level outcome reward to all turns. This is standard in multi-turn RL for language agents (Yu et al., 2025; Shao et al., 2024): all actions in a trajectory share responsibility for the final outcome. However, GRU-Mem introduces a per-step update reward ($r_t^{\text{update}}$, a +1/−1 signal at each chunk) that operates at a fundamentally different temporal granularity from the trajectory-level rewards (outcome, exit, format). Naively adding this per-step reward into the trajectory-level sum would create a credit assignment problem: the per-step signal would be diluted across all tokens in the trajectory, making it difficult for the model to associate the update gate decision at step $t$ with the reward it receives at step $t$.

The paper's solution—separately computing trajectory-level and turn-level advantages and mixing them with a coefficient $\alpha$ (Equation 12, Equation 13)—is significant not as a mathematical novelty (the individual advantage calculations are standard GRPO mean-subtraction) but as a diagnosis and resolution of a structural tension in multi-objective multi-turn RL. The tension is this: trajectory-level rewards provide a holistic signal about answer quality and overall behavior, but they are temporally diffuse (it's hard to tell which turn caused the outcome). Turn-level rewards provide precise per-step supervision, but they are myopic (a perfect update gate decision doesn't guarantee a correct answer). The mixing coefficient $\alpha$ explicitly trades off these competing signals.

What makes this an innovation rather than an obvious engineering choice is the counterintuitive direction of the optimal $\alpha$. One might expect that giving the turn-level update reward substantial weight (e.g., $\alpha = 0.5$, equal weighting) would produce the best update gating, since the update reward directly supervises that behavior. Instead, the paper finds (Figure 8) that $\alpha = 0.9$—where the turn-level advantage contributes only 10% of the total advantage signal—achieves the best balance. At $\alpha = 1.0$ (no direct update supervision), update accuracy on evidence-free chunks collapses because the model receives only noisy, delayed feedback about its gating decisions. At $\alpha = 0.5$ (equal weighting), update accuracy is high but validation reward degrades, likely because the model over-optimizes for gate accuracy at the expense of answer quality. The 10% "nudge" at $\alpha = 0.9$ is sufficient to maintain good gating without distorting the primary objective.

This finding is a diagnostic result with implications beyond this paper: it suggests that when adding auxiliary per-step rewards to multi-turn RL agents, the auxiliary signal should be present but deliberately weak relative to the outcome signal. A strong auxiliary reward can hijack the optimization, causing the model to maximize the auxiliary objective at the expense of the primary one. This principle—learned through empirical sweep rather than derived theoretically—is likely to transfer to other multi-objective agent training settings (e.g., safety-constrained RL, tool-use agents with per-call correctness signals).

Innovation 3: Memory Explosion as a Diagnosable, Measurable Failure Mode with a Targeted Intervention

The concept of "memory explosion" in recurrent LLM agents is, to the paper's credit, made concrete and measurable rather than left as a vague intuition. The paper tracks memory size dynamics over the course of long-context inference (Figure 6), showing that vanilla MemAgent's memory size grows approximately linearly with the number of processed chunks, eventually hitting the maximum memory budget (1024 tokens, implied by the prompt length constraints in Appendix B). Once the memory saturates, further updates become lossy (information must be discarded to make room for new content) and the accumulated noise from earlier indiscriminate updates impedes the model's ability to incorporate genuinely new evidence.

GRU-Mem's update gate reduces this growth dramatically: by only writing to memory on evidence-present chunks (which constitute a small fraction of total chunks in sparse-evidence tasks), the memory size remains compact and predominantly contains relevant information. This is not merely an efficiency gain (shorter memories are faster to generate) but a stability gain: a compact, clean memory makes it easier for the model to identify and incorporate new evidence when it does appear, because the model doesn't have to reason through accumulated noise.

What makes this an innovation rather than an obvious consequence of selective updating is the mechanism by which selectivity is achieved. Prior approaches to controlling memory growth in LLM agents used heuristics: truncate the memory to the last $k$ tokens, summarize when the context window is full, or use a separate retrieval step to decide what to keep (MemGPT; Packer et al., 2023). These are architectural workarounds that don't address the root cause—the model writes indiscriminately because it hasn't been trained to do otherwise. GRU-Mem's contribution is to train the same model that writes memory to also decide whether to write, making selectivity an intrinsic capability rather than an external mechanism. The memory doesn't stay small because a controller prunes it; it stays small because the model chooses not to write on most chunks.

The evidence for this claim is in Figure 6 (memory size dynamics, where GRU-Mem's curve is substantially flatter than MemAgent's) and in the per-task performance results (Table 1), where GRU-Mem's advantage over MemAgent is particularly pronounced on harder multi-key NIAH tasks (MK-1, MK-2, MK-3 at 3B scale: 91.52% vs. 79.46%, 67.08% vs. 35.05%, 91.41% vs. 44.42%). On these tasks, evidence is distributed across multiple sparse chunks, making memory quality critical—accumulated noise from early evidence-free chunks would directly interfere with incorporating later evidence chunks. GRU-Mem's update gate prevents this interference, while MemAgent suffers from it, producing the sharp performance drops visible in the MK series.

Innovation 4: Learned Early Stopping as a Capability Independent of Answer Correctness

The exit gate in GRU-Mem is trained to recognize when all necessary evidence has been collected, which is a subtly different capability from knowing the answer. A model could, in principle, have collected all evidence pieces but not yet know how to synthesize them into an answer—or conversely, it could feel confident in an answer before seeing all evidence (a premature conclusion). The exit gate must learn the former (evidence sufficiency) while avoiding the latter (premature confidence).

The paper's exit reward design (Equation 9) explicitly encodes this distinction. Correct exit (at $t_{\text{last evidence}}$) receives zero reward—no bonus, no penalty. The model is not incentivized to exit per se; it is only penalized for exiting at the wrong time. This means the exit gate is trained purely as a regularizer against bad behavior (early termination that loses evidence, late termination that wastes computation) rather than as a behavior to maximize. This is a deliberate choice that prevents the exit gate from competing with the outcome objective: if correct exit were positively rewarded, the model might learn to exit correctly on trajectories where it cannot answer the question (collecting the exit reward while sacrificing the outcome reward), creating a perverse local optimum.

The result is that the exit gate becomes an orthogonal capability to answer quality. Figure 7 shows that GRU-Mem achieves over 80% exact-stop accuracy (exiting exactly at $t_{\text{last evidence}}$) across all context lengths in the unbalanced evidence setting, while early-exit and late-exit rates are correspondingly low. Figure 8c shows that this exit accuracy is learned gradually but stably across training, and it is learned under all $\alpha$ settings tested (1.0, 0.9, 0.5)—suggesting that exit behavior is easier to learn than update behavior (which is sensitive to $\alpha$). This makes intuitive sense: the exit decision depends on a single binary judgment per trajectory (have I seen the last piece of evidence?), while the update decision depends on a per-chunk binary judgment that must be correct potentially hundreds of times per trajectory.

The practical significance of learned early stopping extends beyond the 400% speedup reported in the paper. It means that GRU-Mem's inference cost is proportional to the position of the last evidence, not to the total context length. In retrieval-augmented generation pipelines where rerankers place key evidence early (Fan et al., 2024; Zhang et al., 2025), this can reduce inference cost by an order of magnitude on long contexts. Conversely, for tasks where evidence is distributed throughout the context (like the multi-values MV task), the exit gate can be disabled at inference time (w/o EG mode), and the model still benefits from the update gate's memory stabilization. This decoupling of update and exit capabilities—train both, deploy selectively—is a design pattern that generalizes beyond this specific architecture.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The paper evaluates on a diverse suite of long-context QA tasks. The primary benchmark consists of: HotpotQA (HQA; Yang et al., 2018), a multi-hop QA task with in-distribution characteristics relative to the training data; SQuAD (Rajpurkar et al., 2016), a single-hop QA task; three single-key NIAH tasks (SK-1, SK-2, SK-3); three multi-key NIAH tasks (MK-1, MK-2, MK-3); one multi-queries NIAH task (MQ); and one multi-values NIAH task (MV). The NIAH tasks are drawn from the RULER benchmark (Hsieh et al., 2024), with the trailing numbers (1, 2, 3) indicating increasing difficulty levels—larger numbers correspond to more challenging needle-in-a-haystack variations such as more distractors or more complex evidence patterns. The paper evaluates across varying context lengths ranging from 7K to 896K tokens (Appendix D.1, Figures 11–20). All tasks except HQA are out-of-distribution relative to the training data, meaning the model must generalize its learned gating behaviors to unseen task formats. The benchmark preparation and evaluation protocols strictly follow MemAgent (Yu et al., 2025).

• Base model(s). Two open-source models from the Qwen2.5-Instruct family: Qwen2.5-3B-Instruct and Qwen2.5-7B-Instruct (Yang et al., 2024). These models are used off-the-shelf without architectural modification—gating behavior is elicited entirely through prompt design and RL training. The paper chooses these models because they "are representative of the capabilities of many contemporary LLMs" (stated implicitly through the choice to evaluate on two different scales) and because the MemAgent baseline was also demonstrated on similarly-sized models, enabling direct comparison. The 3B and 7B scales allow the paper to assess whether gating benefits transfer across model sizes.

• Metrics. Two primary metrics are reported for every method on every task: performance (accuracy, reported as Perf. % ↑) and inference time (wall-clock time in seconds, reported as Time s ↓). Performance is measured as the percentage of test questions for which the predicted answer $\hat{\mathcal{A}}$ matches the ground-truth answer $\mathcal{A}$ according to the equivalence check is_equiv(·, ·) inherited from MemAgent (the specific equivalence function handles formatting variations and is not detailed in the GRU-Mem paper). Inference time is measured as total wall-clock seconds to process all test instances at a given context length (or averaged across context lengths in Table 1). For tasks evaluated at multiple context lengths, per-length performance and timing are reported in Appendix D.1 (Figures 11–20), with "acceleration ratio" indicated by color shading rather than an explicit numeric metric. Efficiency gains are typically expressed as multiples (e.g., "400% times inference speed acceleration") or as reduction ratios relative to MemAgent at the same budget.

• Baselines. The primary baseline is MemAgent (Yu et al., 2025), the vanilla recurrent memory paradigm without any gating mechanisms, evaluated at the same model scales (3B and 7B) on the same tasks. All MemAgent results are reproduced by the authors under identical experimental conditions—chunk size of 5,000 tokens, same training data, same Multi-Conv DAPO RL framework. No other architectural baselines are compared (e.g., single-pass full-context models, retrieval-augmented generation baselines, or other memory architectures like MemGPT). The paper also reports an untrained baseline in the ablation study: "w/o RL" refers to using the base Qwen2.5-7B-Instruct model with the same structured GRU-Mem prompt format but without any RL training, to isolate the contribution of learned gating behavior from prompt-elicited behavior (Section 4.3, Figure 9).

• Generation budget / compute accounting. The paper measures compute in terms of wall-clock inference time (seconds), not in terms of FLOPs or number of generated tokens. This is a practical choice for efficiency comparisons but means the accounting conflates model size, generation length, and hardware utilization into a single metric. The chunk size is fixed at 5,000 tokens for all experiments, and the total context length determines the number of chunks $T$. For GRU-Mem, the effective number of processed chunks depends on whether the exit gate is enabled (w EG mode processes $t_{\text{exit}}$ chunks; w/o EG mode processes all $T$ chunks) and the update gate affects per-step generation length (since not updating memory produces shorter or equivalent-length outputs). The paper does not separately report token counts or FLOPs, making it difficult to decompose speedups into "fewer chunks processed" vs. "fewer tokens generated per chunk" components.

• Cross-validation / statistical protocol. The paper does not report cross-validation, statistical significance tests, confidence intervals, or error bars on any results. All reported numbers are point estimates from single evaluation runs (performance percentages on the full test set for each task, averaged across context lengths in Table 1). The training uses a fixed validation set to monitor reward convergence for early stopping (Appendix B: "stop training until we observe the convergence of reward on the validation set"), but validation accuracy is not reported as a final metric. The absence of variance estimates means that small performance differences between methods (e.g., 76.37% vs. 76.07% on HQA for 7B GRU-Mem w EG vs. MemAgent) cannot be assessed for statistical reliability.

---

Main Quantitative Results

The paper organizes experimental results around three research questions (RQs) stated in Section 4. Here we follow that organization.

Overall Performance and Efficiency Comparison (RQ1)

The headline results appear in Table 1, which reports performance (Perf. %) and inference time (Time s) averaged across context lengths for each task, comparing MemAgent, GRU-Mem w/o EG, and GRU-Mem w EG at both 3B and 7B scales.

Performance. Across both model scales and both inference modes, GRU-Mem generally outperforms MemAgent on the majority of tasks. At the 7B scale with the exit gate disabled (w/o EG), GRU-Mem achieves higher performance than MemAgent on 6 out of 10 tasks, with particularly large gains on MK-2 (84.15% vs. 75.78%) and MQ (84.12% vs. 88.37%—this one is actually lower, so let me be precise). Re-examining Table 1 row-by-row at 7B:

• HQA: MemAgent 76.07%, GRU-Mem w/o EG 75.59% (slightly lower), GRU-Mem w EG 76.37% (slightly higher)
• SQuAD: 79.56% vs. 80.73% vs. 80.47% (GRU-Mem better)
• SK-1: 99.78% vs. 100.00% vs. 100.00% (both at ceiling)
• SK-2: 95.54% vs. 95.43% vs. 96.65% (mixed)
• SK-3: 97.66% vs. 95.98% vs. 95.20% (MemAgent better)
• MK-1: 97.21% vs. 98.10% vs. 98.55% (GRU-Mem better)
• MK-2: 75.78% vs. 67.52% vs. 84.15% (GRU-Mem w/o EG worse, w EG better—a notable split)
• MK-3: 95.98% vs. 93.53% vs. 95.54% (mixed)
• MQ: 88.37% vs. 96.43% vs. 84.12% (GRU-Mem w/o EG substantially better, w EG worse)
• MV: 81.70% vs. 95.23% vs. — (w/o EG dramatically better; exit gate not evaluated on MV)

The pattern is not uniformly in GRU-Mem's favor—there are specific task-mode combinations where MemAgent outperforms (SK-3 across both GRU-Mem modes, MK-2 for w/o EG mode, MQ for w EG mode). However, the largest-magnitude differences favor GRU-Mem (e.g., +13.53 percentage points on MV at 7B, +23.99 points on MK-2 at 3B w/o EG vs. MemAgent). The paper emphasizes that GRU-Mem "excels at out-of-distribution tasks" (the NIAH variants), though the results show this claim is truer for some NIAH tasks (MK-1, MK-3, MQ, MV) than others (SK-3, MK-2 in w/o EG mode).

At the 3B scale, GRU-Mem's advantages are more consistent and often larger in magnitude:

• MK-1: 91.52% (GRU-Mem w/o EG) vs. 79.46% (MemAgent)—a +12.06 point gain
• MK-2: 67.08% vs. 35.05%—a +32.03 point gain
• MK-3: 91.41% vs. 44.42%—a +46.99 point gain
• MV: 59.46% vs. 36.27%—a +23.19 point gain

These are large effects, particularly on the multi-key NIAH tasks where MemAgent degrades sharply at the smaller model scale. The paper attributes this to "more stable memory updating with the introduced update gate."

Efficiency (inference time). At the 7B scale, GRU-Mem w/o EG reduces inference time by roughly 35–65% across tasks compared to MemAgent (e.g., HQA: 284s vs. 463s, ~39% reduction; SQuAD: 85s vs. 162s, ~48% reduction; MK-2: 258s vs. 350s, ~26% reduction). With the exit gate enabled, the reductions are larger: HQA 209s vs. 463s (~55% reduction), MK-1 102s vs. 413s (~75% reduction), MK-2 124s vs. 350s (~65% reduction). The "400% times inference speed acceleration" claim in the abstract corresponds to cases where GRU-Mem w EG is approximately 4× faster than MemAgent—Table 1 shows this for MK-1 at 7B (102s vs. 413s, approximately 4.05× faster) and SK-2 at 3B (84s vs. 176s, approximately 2.1× faster—not 4×; the 400% claim appears to refer to the maximum observed speedup across all context-length settings, visible in the per-length breakdowns in Appendix D rather than the averaged Table 1 numbers). At 3B, the absolute times are lower, and the relative speedups are comparable: MK-1 shows 74s vs. 147s (roughly 2× faster w EG), while the maximum speedup at specific context lengths in Figures 11–20 shows deeper color shading indicating higher acceleration ratios at longer contexts.

Context-length scaling (Appendix D.1, Figures 11–20). The paper provides detailed per-context-length breakdowns for each task, with performance numbers overlaid and color shading indicating acceleration ratio (darker = higher speedup). A consistent pattern emerges: GRU-Mem's efficiency advantage grows with context length. At short contexts (7K–28K tokens), the speedup is modest; at long contexts (448K–896K tokens), the speedup is dramatic. This is expected: the update gate's benefit (suppressing unnecessary writes) compounds over more chunks, and the exit gate's benefit (stopping early) scales with the ratio of total chunks to evidence-last-position. The acceleration is "more obvious as the context length increases" (Appendix D.1 text). On the MV task (Figure 5 in the main text, and Figure 20 in the appendix), GRU-Mem w/o EG shows both higher performance and lower inference time at all context lengths for both model sizes, with the performance gap widening at longer contexts (at 896K, 3B MemAgent drops to roughly 20% while GRU-Mem w/o EG maintains roughly 55–60%; 7B MemAgent drops to roughly 55% while GRU-Mem w/o EG maintains roughly 85%).

Gating Mechanism Analysis (RQ2)

Update gate: memory size dynamics. The paper tracks memory size (in tokens, presumably) over the course of inference on the MV task at 512K context length, comparing MemAgent and GRU-Mem (Figure 6). MemAgent's memory size grows approximately linearly with the number of chunks processed, eventually reaching the maximum memory budget (the paper does not state the exact cap, but it is implied by the 8,192-token maximum prompt length minus the 5,000-token chunk size and question length). GRU-Mem's memory size grows much more slowly and remains substantially lower throughout the trajectory. The figure shows MemAgent approaching the saturation point while GRU-Mem stays well below it, confirming that the update gate suppresses the indiscriminate accumulation that causes memory explosion. The paper states: "GRU-Mem only updates the memory on a few critical chunks which contain the evidence for answering, while MemAgent may indiscriminately update the memory."

Exit gate: early stopping accuracy under unbalanced evidence. To test the exit gate's behavior when evidence appears early, the paper manually constructs an "unbalanced evidence occurrence setting, where the last evidence must occur at the top 20% documents" (Table 2) and separately "at the top 10% position" (Appendix D.3, Table 4). Under the top-20% setting at 7B scale:

GRU-Mem maintains comparable performance (slightly higher at 224K and 448K, slightly lower at 112K and 896K) while reducing inference time to roughly 1/4 of MemAgent's time (60.81s vs. 171.65s at 112K; 454.72s vs. 1691.93s at 896K—the paper claims this as "1/4 of the vanilla MemAgent"). The top-10% setting (Table 4) shows the same pattern: comparable or slightly better performance with inference times reduced to approximately 1/3 to 1/4 of MemAgent's.

Figure 7 reports the ratio of early, exact, and late exit decisions made by GRU-Mem in the unbalanced evidence setting. The exact exit rate (exiting at $t_{\text{last evidence}}$) is above 80% in most cases, with early exit and late exit rates correspondingly low. In the top-10% setting (Appendix D.3, Figure 23), the exact exit ratio is also around 80%. This indicates that the exit gate is not merely triggering randomly or conservatively—it is correctly identifying the last evidence position in the large majority of cases, even when evidence is concentrated at the very beginning of a long context.

Ablation Study (RQ3)

Impact of the mixing coefficient α on training dynamics (Figure 8). The paper sweeps $\alpha \in \{1.0, 0.9, 0.5\}$ and tracks four metrics over the course of RL training:

• Update accuracy on evidence-present chunks (Figure 8a): All $\alpha$ settings achieve high accuracy (>0.9) relatively quickly. $\alpha = 0.9$ and $\alpha = 0.5$ converge to near-perfect accuracy; $\alpha = 1.0$ (no update reward) shows slightly lower and noisier accuracy.

• Update accuracy on evidence-free chunks (Figure 8b): This is where $\alpha$ matters dramatically. $\alpha = 1.0$ shows a sharp decline—accuracy on evidence-free chunks drops from near 1.0 to below 0.4 during training, meaning the model increasingly updates on chunks that contain no evidence (the very behavior that causes memory explosion). $\alpha = 0.5$ maintains high accuracy throughout. $\alpha = 0.9$ shows an intermediate pattern: accuracy dips slightly but recovers and stabilizes at a high level. This is the key empirical justification for $\alpha = 0.9$: without at least some weight on the turn-level update advantage, the model's gating on evidence-free chunks degrades substantially.

• Ratio of exactly exiting at the correct position (Figure 8c): All $\alpha$ settings achieve exit accuracy above 0.8, with $\alpha = 0.9$ and $\alpha = 0.5$ converging slightly faster than $\alpha = 1.0$. The exit gate behavior is robust to $\alpha$—even without any explicit per-step reward for update gating ($\alpha = 1.0$), the model learns to exit correctly, suggesting that the exit reward and outcome reward together provide sufficient signal.

• Validation reward (Figure 8d): $\alpha = 0.9$ shows the highest and most stable validation reward trajectory. $\alpha = 1.0$ is noisier and slightly lower. $\alpha = 0.5$ is intermediate but trends slightly downward in later steps (the paper says $\alpha = 0.9$ "outperforms experiments with $\alpha$ values of 1.0 and 0.5" and shows a "more stable trend").

The paper selects $\alpha = 0.9$ as the default based on "the relatively high reward and balanced update gate accuracy."

Additional training dynamics (Appendix D.2, Figures 21 and 22). The paper provides supplementary metrics tracking:

• Format correctness (Figure 21a): Rapidly reaches ~100% within a few steps for all $\alpha$ settings, confirming that the structured output format is easy for the model to learn.
• Average response length (Figure 21b): Lower $\alpha$ (stronger update reward) produces shorter average responses—the model learns to generate less content overall because it updates memory less frequently.
• Absolute exit deviation ($|t_{\text{exit}} - t_{\text{last evidence}}|$, Figure 21c): Decreases stably across all $\alpha$, confirming the exit behavior is consistently learned.
• Exit deviation ($t_{\text{exit}} - t_{\text{last evidence}}$, Figure 21d): Converges to near zero, with negative values indicating early exit and positive values late exit. All settings converge toward zero deviation.
• Ratio of early exit, exact exit, and late exit (Figure 22): Exact exit ratio increases while early and late exit ratios decrease, showing the model progressively learns the correct exit timing.

Effectiveness of RL training (Figure 9). The paper compares GRU-Mem with RL training against GRU-Mem without RL training (the base Qwen2.5-7B-Instruct model using the same structured prompt format but applied zero-shot). Across all tasks (HQA, SQuAD, SK-1 through SK-3, MK-1 through MK-3, MQ, MV), RL training provides substantial performance gains. The gains are larger on harder tasks—HQA, SQuAD, and the MK series show the biggest deltas—while simpler tasks like SK-1 (already near ceiling) show smaller improvements. This confirms that the gating behaviors and memory management strategies are acquired through RL optimization rather than being inherent capabilities of the base model that the prompt format simply elicits. For instance, on MK-3, the RL-trained model achieves roughly 95% while the untrained model achieves roughly 50%—a gap of approximately 45 percentage points that can only be attributed to learned gating behavior.

---

Ablation Studies and Robustness Checks

• Inference mode: with vs. without exit gate (Table 1, all tasks): The exit gate generally maintains or improves performance while reducing inference time, but there are notable exceptions. On MQ at 7B, w EG drops performance from 96.43% to 84.12% compared to w/o EG—a 12.31 point degradation—while still being faster (109s vs. 157s). On MK-2 at 7B, w EG improves performance to 84.15% vs. 67.52% for w/o EG, possibly because early exit prevents the model from accumulating noise after collecting all necessary evidence. The exit gate is intentionally disabled for the MV task because MV requires reading the entire context (the task asks for all special magic numbers, not a single one). This validates the paper's design choice of providing both inference modes: the exit gate is not universally beneficial and can be harmful on tasks where evidence is distributed throughout the context.

• Model scale robustness (3B vs. 7B, Table 1): GRU-Mem's benefits transfer across scales. At 3B, the performance gains over MemAgent are generally larger in absolute terms than at 7B—consistent with the intuition that smaller models benefit more from structured memory management because they have less intrinsic capacity to handle long contexts. The efficiency gains (inference time reduction) are present at both scales but are proportionally similar (both 3B and 7B see ~30–75% time reductions depending on the task and mode).

• Evidence distribution robustness (Tables 2 and 4, Figures 7 and 23): Under manually constructed unbalanced evidence distributions (top 20% and top 10% positions), GRU-Mem w EG maintains accuracy comparable to MemAgent while achieving 3–4× speedup. The exit accuracy remains above 80% exact-stopping in both settings, suggesting the learned exit behavior is not overfit to the (presumably uniform) evidence distribution in the training data.

• Format reward design justification (Figure 21a): The strict all-or-nothing format reward is empirically validated by the rapid convergence to ~100% format correctness across all $\alpha$ settings, confirming that the design choice (Section 3.2.1) does not impede training and achieves its goal of ensuring parseable outputs.

• Multi-Conv DAPO effectiveness (no dedicated ablation): The paper does not ablate the choice of RL algorithm (e.g., comparing Multi-Conv DAPO against standard GRPO, PPO, or supervised fine-tuning). The only training comparison is RL vs. no RL (Figure 9), which isolates the effect of training in general but not the specific algorithm choice.

• Negative result: ReST$^{EM}$ not applied: Unlike MemAgent, which explored further optimization with ReST$^{EM}$ (though that experiment is not reported in the GRU-Mem paper for the vanilla MemAgent baseline either), GRU-Mem does not attempt self-play or iterative refinement beyond the initial RL training. The paper notes in the limitations (Section 5) that "the extra rewards in GRU-Mem reduce training stability, requiring a smaller off-policy degree and longer convergence time," which is itself a negative result: the added gating rewards make the training process more delicate than the vanilla MemAgent setup.

---

Critical Assessment

Central Claim 1: "GRU-Mem generally outperforms the vanilla MemAgent across diverse tasks and among different model sizes"

What the experiments demonstrate: GRU-Mem achieves higher performance than MemAgent on most task-mode-model combinations in Table 1, with particularly large gains on multi-key NIAH tasks (MK-1, MK-2, MK-3 at 3B) and the multi-values task (MV at both scales). The gains are consistent across two model sizes and two inference modes.

What the experiments do NOT demonstrate:

• The claim is qualified by several counterexamples where MemAgent outperforms GRU-Mem: SK-3 at 7B (97.66% MemAgent vs. 95.98% w/o EG and 95.20% w EG), MK-2 at 7B w/o EG (75.78% MemAgent vs. 67.52% GRU-Mem w/o EG), and MQ at 7B w EG (88.37% MemAgent vs. 84.12% GRU-Mem w EG). The paper's framing as "generally outperforms" is accurate given the overall pattern, but readers should not interpret this as uniform superiority.
• There is no statistical significance testing. On tasks where the performance gap is small (e.g., HQA at 7B: 76.07% vs. 76.37%, a 0.30 point difference on a 500-question test set), the difference could easily be noise. Without confidence intervals or multiple evaluation runs, small differences cannot be reliably attributed to the method rather than sampling variance.
• The training data is the same as MemAgent's, but the paper does not report whether the data distribution favors GRU-Mem's gating mechanisms (e.g., whether the training data has particularly sparse evidence, which would make update gating easier to learn and more beneficial). The generalization to out-of-distribution NIAH tasks is encouraging but is tested only on synthetically constructed benchmarks with known evidence positions.

Central Claim 2: "GRU-Mem generally achieves up to 400% times inference speed acceleration"

What the experiments demonstrate: On specific task-context length combinations, GRU-Mem w EG is approximately 4× faster than MemAgent (e.g., MK-1 at 7B in Table 1: 102s vs. 413s; and the per-context-length breakdowns in Appendix D show deeper color shading at longer contexts indicating higher acceleration ratios). The speedup is robust across tasks and grows with context length (Figures 11–20), which is the expected direction if gating suppresses per-chunk computation and enables early stopping.

What the experiments do NOT demonstrate:

• The "400%" (4×) figure is a maximum observed speedup, not a typical or average speedup. In Table 1's averaged times, most speedups are in the 1.5–3× range. The headline number is the best case, selected from specific context-length settings visible in Appendix D but not enumerated in the main text.
• Inference time is measured in wall-clock seconds, which includes implementation-specific overhead (prompt construction, output parsing, memory management in the controller). The paper does not provide token counts or FLOPs, making it impossible to decompose the speedup into "fewer tokens generated" (due to update gating) vs. "fewer chunks processed" (due to exit gating) vs. "implementation efficiency differences." A different implementation of MemAgent might close some of the gap.
• The efficiency gains from the exit gate depend on evidence position. In the standard evaluation (Table 1), evidence distribution is not manipulated—it follows whatever distribution the benchmark construction process used (likely uniform or near-uniform). Under uniform evidence distribution, the exit gate provides minimal benefit (if evidence is spread evenly, the last evidence chunk is near the end). The large speedups in Table 1 for w EG mode suggest either (a) the benchmarks already have uneven evidence distributions, or (b) the update gate alone (which reduces per-step generation cost by suppressing unnecessary writes) accounts for most of the speedup even without early stopping. The paper does not disentangle these two effects for the standard benchmarks.

Central Claim 3: "The update gate reduces the risks of memory explosion"

What the experiments demonstrate: Figure 6 shows that GRU-Mem's memory size grows substantially more slowly than MemAgent's on the MV task at 512K context length, confirming that the update gate suppresses unnecessary memory writes. The performance advantages on multi-key NIAH tasks (where memory quality is critical for integrating evidence from multiple sparse chunks) provide indirect evidence that cleaner memory improves reasoning quality.

What the experiments do NOT demonstrate:

• Memory size dynamics are shown for only one task (MV) at one context length (512K). The paper does not report whether the same pattern holds across all tasks—though it is reasonable to expect it would, since the update gate mechanism is task-agnostic.
• The exact memory size metric is not specified (number of tokens? characters? some other unit?). The figure's y-axis is unlabeled in the paper text, making quantitative interpretation difficult.
• There is no direct causal evidence that memory size causes performance degradation in MemAgent—the relationship is correlational. It is possible that MemAgent's lower performance on MK tasks has other causes (e.g., the RL training converging to a different local optimum) and the memory size difference is a symptom rather than a cause.

Central Claim 4: "The exit gate provides a meaningful exit mechanism"

What the experiments demonstrate: Under manually constructed unbalanced evidence distributions (Tables 2 and 4), GRU-Mem w EG exits at the correct position in over 80% of cases (Figures 7 and 23) while maintaining accuracy comparable to MemAgent (which processes all chunks). The inference time is reduced proportionally to the evidence position (approximately 3–4× faster when evidence is in the first 10–20% of the context).

What the experiments do NOT demonstrate:

• The exit accuracy is tested only under artificial evidence distributions (top 10% and top 20%). The paper does not report exit accuracy on the standard (un-manipulated) benchmark evaluations, where evidence positions follow their natural distributions. Without this, we cannot assess how much of the "w EG" speedup in Table 1 comes from early stopping vs. the update gate's reduction in per-step generation cost.
• The exit gate is disabled for the MV task because it would be harmful. This is a legitimate design choice, but it means the system requires the user to know a priori whether a task requires reading the entire context. A more robust system would learn to recognize this property from the question itself.

Missing Experiments That Would Strengthen the Claims

• Statistical significance: The test sets appear to contain a few hundred questions per task (the exact sizes are not stated except for the 500-question MATH test set in MemAgent which GRU-Mem inherits). Reporting confidence intervals or running multiple evaluation seeds would help assess whether small performance differences are reliable.
• Token-level efficiency decomposition: Reporting the number of tokens generated per trajectory for GRU-Mem vs. MemAgent would clarify whether the speedup comes from (a) generating fewer tokens overall (update gate suppresses verbose memory writes), (b) processing fewer chunks (exit gate enables early stopping), or (c) both. The current wall-clock time metric conflates these mechanisms.
• Ablation of individual gate contributions in isolation: The paper compares GRU-Mem w/o EG (update gate only) and GRU-Mem w EG (both gates) against MemAgent (neither gate), but does not ablate an "exit gate only" configuration (update gate disabled). This makes it impossible to attribute the performance gains to the update gate vs. exit gate independently for the full benchmark suite.
• Larger model scale: Testing on a larger model (e.g., 14B or 32B parameters) would reveal whether the benefits of gating diminish as base model capacity increases (consistent with the pattern that 3B gains more than 7B), or whether they plateau.
• Non-QA tasks: The paper acknowledges in Section 5 that "it is limited to the QA domain, with other tasks (e.g., summarization) largely underexplored." Summarization, in particular, is a task where "evidence" is distributed throughout the context, making both the update gate (every chunk is relevant) and exit gate (cannot exit early) potentially unhelpful or harmful.
• Comparison against retrieval-augmented baselines: The paper positions GRU-Mem as an alternative to retrieval-based approaches (Section 2, discussion of brittleness from retrieval errors), but does not empirically compare against any retrieval-augmented generation pipeline. Such a comparison would contextualize the accuracy-efficiency tradeoff.
• Robustness to chunk size: All experiments use a fixed chunk size of 5,000 tokens. Varying the chunk size would test whether the gating behaviors transfer or whether they are sensitive to the granularity of evidence-chunk alignment.
• Analysis of error modes: The paper does not report what kinds of errors GRU-Mem makes compared to MemAgent (e.g., does the update gate sometimes suppress a chunk that actually contains evidence? Does the exit gate ever exit too early on the standard benchmarks, causing missed evidence?). Qualitative error analysis would strengthen the mechanistic claims about how the gates improve performance.

6. Limitations and Trade-offs

Scope Restricted to Question Answering with Known Evidence Positions

The assumption or constraint. The paper evaluates GRU-Mem exclusively on question-answering benchmarks (HotpotQA, SQuAD, and synthetic NIAH variants from the RULER suite). The authors explicitly acknowledge this in Section 5: "it is limited to the QA domain, with other tasks (e.g., summarization) largely underexplored." More fundamentally, the entire training pipeline depends on knowing which chunks contain evidence—this is what enables the update reward $r^{\text{update}}$ (Equation 8), which requires ground-truth labels for evidence-presence at each chunk position. These labels are available in synthetic NIAH benchmarks (where evidence is planted at known locations) and in multi-hop QA datasets (where evidence documents are annotated), but they do not exist for most real-world long-context tasks.

The consequence. Two distinct failure modes arise. First, on tasks where "evidence" is not a meaningful concept—such as summarization, where every chunk contributes to the output and there is no sparse evidence distribution—both the update gate and exit gate become counterproductive. The update gate would learn to write on every chunk (defeating the purpose of selective updating), and the exit gate could never trigger early (eliminating efficiency gains). The paper's design of providing a w/o EG inference mode partially addresses the exit gate problem, but the update gate would still be forced into a degenerate regime where it must update on every chunk—and it is unclear whether the RL training, which was optimized under evidence-sparse conditions, would transfer to this setting.

Second, and more perniciously, the method requires oracle evidence-position labels for training. The update reward (Equation 8) gives +1 when the model correctly identifies a chunk as evidence-present or evidence-free, and −1 otherwise. This is a fully supervised signal—the model is told, at each training step, whether the current chunk contains evidence. In real-world long-context tasks (legal document review, scientific literature synthesis, long-form dialogue analysis), such labels are rarely available. Constructing them would require annotating every chunk of every document for every question, which is precisely the label-intensive process that end-to-end RL is supposed to avoid. The paper does not discuss whether the gating behaviors could be learned purely from outcome rewards (without per-step update supervision), though the $\alpha = 1.0$ ablation in Figure 8b—where update accuracy on evidence-free chunks drops sharply when the update reward is disabled—suggests that the outcome signal alone is insufficient to learn selective updating.

What evidence exists in the paper. Figure 8b shows the dramatic dependence of update-gate accuracy on the update reward: at $\alpha = 1.0$ (no update reward), accuracy on evidence-free chunks decays from near 1.0 to below 0.4 during training, meaning the model increases its rate of unnecessary updates over time. This directly demonstrates that the update gate behavior does not emerge from the outcome reward alone—it requires the per-step supervised signal. Section 5 acknowledges the domain limitation explicitly. The absence of any non-QA evaluation (summarization, translation, code understanding) means there is no evidence about generalization.

Mitigation status. The paper does not attempt to address this limitation. It does not propose methods for learning evidence-position labels from weaker signals, adapting the update reward for tasks without clear evidence boundaries, or evaluating on tasks where evidence is uniformly distributed. The authors flag the domain limitation as future work in Section 5. A practitioner seeking to deploy GRU-Mem on a non-QA long-context task would need to either (a) construct per-chunk evidence labels for their training data, which may be prohibitively expensive, or (b) accept that the update gate behavior will not transfer and rely only on whatever gating behaviors the outcome reward can induce (which Figure 8b suggests will be poor).

---

Training Stability Degrades Relative to Vanilla MemAgent

The assumption or constraint. GRU-Mem introduces two additional reward signals (update and exit) on top of MemAgent's outcome reward, creating a multi-objective RL problem. The paper acknowledges in Section 5: "the extra rewards in GRU-Mem reduce training stability, requiring a smaller off-policy degree and longer convergence time." This is a significant practical concern: the system is more difficult to train than the baseline it improves upon.

The consequence. Reduced training stability manifests in several practical ways. First, the hyperparameter $\alpha$ that balances trajectory-level and turn-level advantages is critical—the difference between $\alpha = 0.9$ (selected default) and $\alpha = 1.0$ is the difference between learning effective gating and having update accuracy on evidence-free chunks collapse (Figure 8b). This sensitivity means practitioners cannot simply adopt default RL hyperparameters; they must tune $\alpha$ (and possibly other reward-weighting parameters not explored in the paper, such as the relative magnitudes of the exit vs. update vs. outcome rewards) for their specific task and model. The paper sweeps only three $\alpha$ values (1.0, 0.9, 0.5) on a single model scale, so the robustness of the $\alpha = 0.9$ finding to other model sizes, architectures, or task distributions is unknown.

Second, the paper notes that training requires a "smaller off-policy degree"—meaning the policy cannot deviate as far from the reference model between updates, which implies slower learning and potentially more wall-clock training time. The exact training duration is not reported, but the statement about "longer convergence time" in Section 5 suggests this is a non-trivial practical cost.

Third, the paper does not report whether training is sensitive to other hyperparameters: the reward magnitudes (−0.75 for early exit vs. −0.5 for late exit, +1/−1 for update correctness), the format reward's all-or-nothing strictness, the chunk size, or the number of RL training steps. If the exit reward penalties were changed (e.g., −1.0 for early exit, −0.25 for late exit), would the learned exit behavior change? There is no ablation probing this.

What evidence exists in the paper. Figure 8 provides the primary evidence: the validation reward curve (Figure 8d) for $\alpha = 0.9$ is higher and more stable than for $\alpha = 0.5$ or $\alpha = 1.0$, but even the best setting shows fluctuations over training steps. The sharp drop in evidence-free update accuracy for $\alpha = 1.0$ (Figure 8b) demonstrates that the multi-reward optimization is fragile—without careful balancing, the model's behavior on one objective can collapse even as others improve. Section 5 explicitly flags the stability issue as a limitation.

Mitigation status. The paper acknowledges the issue but does not propose solutions beyond the $\alpha$ balancing mechanism, which addresses the symptom (conflicting reward signals) but not the root cause (multi-objective RL instability). There is no exploration of alternative training paradigms that might be more stable—for instance, staged training (learn update gating first, then exit gating, then joint fine-tuning), reward normalization techniques beyond the advantage mixing, or architectural modifications that decouple the gate-learning pathways from the content-generation pathways. The limitation is presented as an inherent tradeoff ("the extra rewards… reduce training stability") with no proposed resolution.

---

Difficulty-Estimation Cost for Gating Is Not Accounted For in Efficiency Measurements

The assumption or constraint. GRU-Mem's efficiency gains—the 400% speedup headline—are measured as the wall-clock inference time of the recurrent loop, comparing how long GRU-Mem takes to process chunks (with gating) vs. how long MemAgent takes to process all chunks. However, this measurement starts after the model has been trained. It does not account for the training cost of learning the gating behaviors, which the paper shows is higher than vanilla MemAgent's training cost (longer convergence time, stability challenges). More subtly, it does not account for any per-task or per-deployment overhead of adapting the trained model.

The consequence. The efficiency comparison is an inference-time-only comparison between two trained models, not an end-to-end comparison of total compute (training + inference). If GRU-Mem requires, say, twice as many training steps as MemAgent to converge (and each training step has the same or higher cost due to the additional reward computation and gate-generation overhead), then the total compute budget (training + inference) might favor GRU-Mem only after many inference queries—the inference savings must amortize the training overhead. The paper provides no data to assess this tradeoff. For a practitioner deploying the method on a small number of inference queries, the training overhead could dominate, making the approach net-negative in total compute. Conversely, for high-volume production deployments (millions of queries), the training overhead amortizes to near-zero, and the inference savings dominate.

Additionally, the paper's reported inference time (wall-clock seconds in Table 1) includes the full recurrent loop but does not separate out parsing overhead (the deterministic controller that extracts gate decisions from the structured output), prompt construction overhead (assembling $(Q, M_{t-1}, C_t)$ at each step), or the cost of generating then discarding candidate memories when the update gate is False. In GRU-Mem, the model always generates a full candidate memory $\hat{M}_t$ even when $\mathcal{U}_t = \text{False}$ (the candidate is discarded). This means the per-chunk generation cost is not reduced when the update gate says "no"—the model still expends compute reasoning about the chunk and producing a (discarded) summary. The only savings from the update gate in w/o EG mode come from: (a) the memory text being shorter at subsequent steps (since it didn't grow with unnecessary content), which reduces the prompt length and slightly speeds up generation; and (b) the reduced risk of memory explosion, which prevents catastrophic slowdowns. The exit gate provides the chunk-skipping savings. The wall-clock time metric in Table 1 conflates all these effects, making it difficult to attribute speedups to specific mechanisms.

What evidence exists in the paper. The paper does not report training cost (total training steps, wall-clock training time, or total training FLOPs), so there is no evidence about the training-inference tradeoff. The always-generate-candidate design is described in Section 3.1 but its efficiency implications are not analyzed—there is no comparison of per-step generation length between GRU-Mem and MemAgent for evidence-free chunks. Figure 6 shows memory size dynamics but not per-step generation cost. Section 5 mentions "longer convergence time" as a limitation but does not quantify it.

Mitigation status. Not addressed. The paper treats training cost and inference cost as separate concerns and reports only inference cost. The "longer convergence time" mention in Section 5 is qualitative and unquantified. No suggestions are made for reducing training overhead (e.g., curriculum learning, reward scheduling, or pre-training the gating behavior with supervised data before RL fine-tuning).

---

Gate Behavior Depends on a Single Model Family; Transfer to Other Architectures Unknown

The assumption or constraint. All experiments use Qwen2.5-Instruct models (3B and 7B). This is a single model family with a specific training recipe, tokenizer, and instruction-tuning methodology. The paper claims (Section 4) that this model "is representative of the capabilities of many contemporary LLMs," but provides no evidence that the gating behaviors would transfer to models with different architectures (e.g., non-Chinese-aligned models, models with different context-window extensions, models trained primarily for code rather than text), different scales (beyond 3B–7B), or different training paradigms (base models vs. instruction-tuned models, RLHF-tuned vs. SFT-only models).

The consequence. Several aspects of GRU-Mem's performance could be model-specific:

• Structured output compliance: The ability to reliably generate <check>yes</check> / <check>no</check> and <next>continue</next> / <next>end</next> within a tagged format depends on the base model's instruction-following capability. Qwen2.5-Instruct models are specifically trained for structured output. A base model without instruction tuning, or a model from a different family with weaker formatting capabilities, might struggle to maintain the format consistency that the deterministic controller requires (and that the format reward enforces). Figure 21a shows format correctness reaching ~100% quickly for Qwen2.5, but this convergence rate could be dramatically slower for other model families.

• Gating as a semantic capability: The update gate must evaluate whether a chunk contains "useful information" relative to a question. This is a complex semantic judgment that depends on the model's reading comprehension and relevance-assessment capabilities. A model with weaker comprehension might learn a degenerate gating strategy (e.g., always update, or update randomly) if it cannot distinguish evidence-present from evidence-free chunks. The paper's $\alpha = 1.0$ result, where the model loses update accuracy on evidence-free chunks (Figure 8b), already shows that even within the Qwen2.5 family, gating behavior is fragile when not directly supervised. On a weaker model, the update reward might be insufficient to teach the distinction at all.

• Exit gate calibration: The exit gate's decision—"have I collected enough information?"—requires the model to maintain an implicit estimate of evidence sufficiency. This is a metacognitive capability that likely varies across model families and scales. The paper shows the exit gate achieving >80% exact-stop accuracy (Figure 8c), but this is on Qwen2.5 with explicit exit-reward training. A model with poorer calibration of its own knowledge state might exit prematurely or fail to exit when appropriate, even with the same reward structure.

• Scale trends: The paper shows that GRU-Mem's relative gains are larger at 3B than at 7B (e.g., on MK-3: +46.99 points at 3B vs. roughly neutral at 7B). This suggests that gating benefits diminish as base model capacity increases—the larger model has more intrinsic ability to handle long contexts without structured memory management. Whether this trend continues (e.g., at 14B or 32B, would GRU-Mem provide any benefit over MemAgent?) is untested.

What evidence exists in the paper. The paper evaluates only Qwen2.5-3B-Instruct and Qwen2.5-7B-Instruct. The two scales provide some evidence about within-family scaling, but no cross-family evidence. The diminishing-returns pattern from 3B to 7B is visible in Table 1 (the magnitude of GRU-Mem's advantage over MemAgent is generally smaller at 7B than at 3B) but is not explicitly analyzed or discussed by the authors. Section 5 does not flag model-family dependence as a limitation.

Mitigation status. Not addressed. The paper implicitly treats Qwen2.5 as representative through its experimental choices, but does not argue for or test this assumption. Replication on models from other families (e.g., Llama-3, Mistral, DeepSeek) and at larger scales would be necessary to establish that gating behaviors are a general property of the training signal rather than a capability that depends on Qwen2.5's specific instruction-tuning recipe.

---

The Exit Gate Requires a Priori Knowledge of Task Structure to Use Safely

The assumption or constraint. The paper provides two inference modes—with exit gate (w EG) and without exit gate (w/o EG)—and delegates to the user the decision of which mode to use for which task. This is not a learned distinction; the model was trained with the exit gate enabled and cannot autonomously determine whether early stopping is appropriate for a given question. The paper's example (Section 3.3) is instructive: on the multi-values (MV) task, which asks "What are all the special magic numbers for xxx?", the exit gate must be manually disabled because the model would otherwise exit after finding the first magic number, missing subsequent ones.

The consequence. This places a burden on the deployer to know, for each task or even each individual question, whether the exit gate should be enabled. The distinction is not always obvious from task metadata. Consider a slight variation on the MV task: "What is the special magic number for belligerent-councilperson?" This is a single-value question that looks superficially similar to MV but requires finding only one specific value, so early exit is appropriate. If a deployment pipeline handles both single-value and multi-value questions through the same interface, there is no automatic way to toggle the exit gate—it must be set uniformly, guaranteeing suboptimal behavior for one class of questions.

More subtly, even within a task where exit is generally appropriate (e.g., single-key NIAH), there may be edge cases where the model exits prematurely—for instance, if a distractor chunk contains text that the model incorrectly interprets as the final necessary evidence. The paper's exit accuracy is ~80% in the manipulated evidence-distribution experiments (Figures 7, 23), meaning ~20% of trajectories exit at the wrong position. In the w EG inference mode, these incorrect exits cause irrevocable evidence loss: any evidence appearing after the premature exit point is never seen by the model. In the w/o EG mode, these incorrect exit decisions are ignored (the loop continues), so the evidence is not lost—the cost is only wasted computation from the update gate's decisions, not evidence loss from the exit gate. The paper does not provide this comparison directly, but the existence of both modes reflects an implicit acknowledgment that the exit gate's errors in w EG mode can cause accuracy degradation that w/o EG mode avoids.

The MQ task at 7B provides a concrete example of this tradeoff (Table 1): GRU-Mem w/o EG achieves 96.43% accuracy vs. MemAgent's 88.37% (a substantial gain), while GRU-Mem w EG achieves only 84.12% (a degradation). This suggests that on MQ, the exit gate is making harmful early-stopping decisions—exiting before all necessary evidence is collected—and the update gate alone (w/o EG mode) is responsible for the performance gain. The paper does not analyze why the exit gate fails on MQ or whether this failure mode generalizes to other multi-query tasks.

What evidence exists in the paper. Table 1 provides direct evidence of the tradeoff: on MQ at 7B, w EG degrades performance by 12.31 points relative to w/o EG; on MK-2 at 7B, w EG improves performance by 16.63 points relative to w/o EG. These opposite-direction effects on different tasks confirm that the exit gate is not universally beneficial and its effects depend on task structure. Table 2 and Figure 7 show that exit accuracy is approximately 80% under the manipulated evidence distributions, implying a ~20% error rate. The decision to disable the exit gate on MV is stated in Section 3.3 and visible in Table 1 (the MV row shows a dash for w EG). However, the paper provides no guidance on how a practitioner should decide which mode to use beyond the extreme case of MV where the task explicitly asks for "all" values.

Mitigation status. The paper's mitigation is the two-mode inference design, which is a partial solution: it allows safe operation (w/o EG) at the cost of efficiency, but requires the user to know which mode is appropriate. The paper does not explore whether the model could learn to recognize task type from the question and adjust its exit behavior accordingly (e.g., generating <next>end</next> only for single-value questions and <next>continue</next> for multi-value questions, even when the exit gate is "enabled"). This would require training on a mixture of task types with different exit ground-truth labels, which the paper's training data (inherited from MemAgent) may not support. No suggestions are made for automatic exit-gate toggling.

7. Implications and Future Directions

How This Work Changes the Landscape

GRU-Mem does not introduce a new attention mechanism, a new context-extension technique, or a fundamentally new architecture for long-context processing. Its contribution is more targeted and, in some ways, more provocative: it demonstrates that control flow decisions in recurrent LLM agents—when to write to memory, when to stop reading—can and should be treated as learned behaviors optimized through end-to-end RL, rather than as hard-coded heuristics or architectural constraints. This is a methodological shift that repositions a class of design decisions from the system engineer to the optimization objective, with significant implications for how the field builds agentic language systems.

To understand why this matters, consider the alternative approaches the paper implicitly argues against. The dominant paradigm for memory management in LLM agents has been heuristic control: update memory when the context window is full (MemGPT; Packer et al., 2023), truncate to the last k tokens, or use a separate retrieval step with fixed similarity thresholds to decide what to keep (Li et al., 2025). These heuristics work in broad strokes but fail in the details—they update on irrelevant chunks because they cannot distinguish evidence from noise, and they process to the end because they have no mechanism for recognizing sufficiency. The alternative paradigm of architectural gating (continuous-valued gates in LSTMs, GRUs, or state-space models) operates at the sub-symbolic level, modulating vector-valued hidden states through differentiable element-wise multiplication—mathematically elegant, but incapable of making the semantic judgment that "this chunk about addiction and technology is irrelevant to the question about a 122nd SS-Standarte" (see Appendix E, Case 2).

GRU-Mem's core reframing is that gating decisions—update, exit—are themselves language generation acts that can be produced by the same autoregressive reasoning process that generates memory content. The model doesn't just decide what to remember; it decides whether to remember, and these two capabilities are learned jointly through a single RL objective with appropriately structured rewards. This unifies memory content generation and memory control under a single optimization framework, eliminating the brittle interface between a learned content generator and a heuristic controller that has characterized prior memory-augmented LLM systems.

The magnitude of this shift should not be overstated—this is not a paradigm revolution on the scale of the original transformer or the Chinchilla scaling laws. It is a targeted architectural augmentation to an existing framework (MemAgent) that addresses two specific, diagnosable failure modes. But the implications ripple outward. If gating decisions can be learned for memory control in long-context QA, then presumably they can be learned for other agentic control-flow decisions: when to call a tool vs. reason internally, when to ask for clarification vs. proceed with uncertainty, when to escalate to a larger model vs. answer with the current model. The paper provides a template—structured output format, per-decision reward signals, disentangled advantage calculation for multi-granularity objectives—that transfers to any setting where an LLM agent must make discrete control decisions as part of a sequential reasoning process.

The paper also reconciles a tension between the "lost in the middle" literature (Liu et al., 2024) and the recurrent memory literature. The former argues that LLMs struggle to attend to information in the middle of long contexts; the latter (MemAgent) argues that chunk-by-chunk processing mitigates this. GRU-Mem clarifies the mechanism: chunk-by-chunk processing helps if the memory remains clean (update gate prevents noise accumulation), but degrades if the memory accumulates irrelevant content that dilutes attention to later evidence. The update gate is the mechanism that keeps memory clean, and its absence explains why vanilla MemAgent still shows performance degradation on some tasks (e.g., MK-2, MK-3 at 3B in Table 1)—the recurrent loop helps, but indiscriminate updating eventually produces the same attention-dilution problem at the memory level that "lost in the middle" describes at the context level.

Perhaps the most consequential shift, if the findings generalize, is the implication that verifier quality is not the only bottleneck in test-time compute scaling; agentic control quality matters equally. Attention in the LLM community has focused heavily on reward model quality for RLHF and on verifier-guided search for reasoning tasks. GRU-Mem shows that for long-context reasoning, the ability to selectively engage with the input (update gate) and recognize when to stop (exit gate) can produce 4× efficiency improvements at matched or better accuracy—gains that no amount of verifier improvement could achieve if the underlying agent were still processing all chunks indiscriminately. This suggests that research investment in agentic control learning (how to train LLMs to make good decisions about what to process) may be underweighted relative to investment in verifier quality (how to train LLMs to evaluate what they've processed), and that the two are complementary rather than substitutable.

Follow-Up Research This Work Enables

Training update gating without oracle evidence-position labels through outcome-signal bootstrapping. The most immediate barrier to deploying GRU-Mem on real-world tasks is the dependence on per-chunk evidence-presence labels for the update reward (Equation 8). These labels are available in synthetic benchmarks but rarely in practice. A natural follow-up would replace the supervised update reward with a learned or bootstrapped signal: for example, using the outcome reward to infer which chunks were likely evidence-present (chunks that, when skipped in a counterfactual trajectory, cause the answer to change), then using these inferred labels as pseudo-ground-truth for the update reward in subsequent training iterations. The paper's $\alpha = 1.0$ result (Figure 8b) shows that pure outcome-signal training causes update accuracy to collapse, but a bootstrapping approach where the model initially learns from a small number of annotated examples and then self-labels could potentially overcome this. A strong follow-up would test this on a dataset where only 5–10% of chunks have human evidence annotations, measuring whether update accuracy on the unannotated chunks transfers.

Combining GRU-Mem with retrieval-augmented generation for hybrid evidence localization. GRU-Mem processes chunks sequentially in their original order, relying on the exit gate to stop early when evidence appears late. A natural extension would integrate a retriever or reranker that reorders chunks so that evidence-present chunks appear early, amplifying the exit gate's efficiency gains. This is the setting tested synthetically in Tables 2 and 4 (top 10% and top 20% evidence positions), where GRU-Mem achieves 3–4× speedup with no accuracy loss. A realistic evaluation would pair GRU-Mem with a standard dense retriever or cross-encoder reranker (e.g., Qwen3 embedding models; Zhang et al., 2025) on a multi-hop QA dataset like MuSiQue or 2WikiMultihopQA, measuring end-to-end accuracy and total latency (retrieval time + GRU-Mem time) against both pure retrieval-augmented generation and pure GRU-Mem baselines. The key question is whether retrieval errors (false negatives that miss evidence chunks) are partially recoverable by GRU-Mem's sequential processing of non-retrieved chunks, combining the complementary strengths identified in the paper's motivation (Section 2).

Stress-testing the exit gate on tasks where evidence sufficiency is ambiguous. The paper's exit gate is trained with a clean "last evidence position" label—the benchmark construction places evidence at known positions, so the correct exit point is well-defined. Real-world tasks rarely have this property. Consider a legal document review task where the question is "Was there a breach of contract?"—evidence is distributed across multiple clauses, and "sufficiency" is a matter of legal judgment, not a binary fact. A strong negative-result paper would evaluate GRU-Mem's exit gate on such ambiguously-evidenced tasks, measuring: (a) whether the model exits prematurely (before a human judge would consider the evidence sufficient), (b) whether the model continues processing long past the point where human annotators would stop, and (c) whether the exit accuracy metric (>80% in the paper) degrades to near-chance levels when "last evidence position" is defined by inter-annotator agreement rather than ground-truth planting. If the exit gate fails under ambiguity, this would establish a boundary condition for the approach and motivate research on confidence-calibrated or probabilistic exit mechanisms (e.g., the model outputting a probability of evidence sufficiency rather than a binary decision).

Extending GRU-Mem to tasks beyond question answering, starting with summarization as a hard case. The paper acknowledges the QA-only scope as a limitation (Section 5). Summarization is the most instructive counter-task because evidence (every chunk contributes to the summary output) is distributed uniformly rather than sparsely. In this setting, the update gate should learn to write on every chunk (no selectivity benefit), and the exit gate should never trigger early (no early-stopping benefit). The natural question is: does GRU-Mem degrade relative to vanilla MemAgent on summarization, or does it simply provide no benefit? If it degrades—perhaps because the update gate occasionally suppresses a chunk that actually contains summarizable content, or because the structured output format adds overhead without providing useful control—this would define a clear "do not use" regime. If performance is equivalent, the method is at least harmless even when its mechanisms provide no advantage. A concrete experiment would use the GovReport or SummScreenFD summarization benchmarks with context lengths scaled to 64K–256K tokens, comparing GRU-Mem (both w EG and w/o EG) against MemAgent and a long-context single-pass baseline on ROUGE and factuality metrics.

Scaling laws for gating: does gating benefit diminish with model size, and if so, at what scale? The paper shows that GRU-Mem's relative gains are larger at 3B than at 7B—compare the 3B multi-key NIAH results (MK-3: 91.41% GRU-Mem w/o EG vs. 44.42% MemAgent, +46.99 points) to the 7B results (MK-3: 93.53% vs. 95.98%, −2.45 points). This suggests a diminishing-returns pattern: as base model capacity increases, the model's intrinsic ability to handle long contexts improves, reducing the marginal benefit of explicit gating. A scaling study testing GRU-Mem on Qwen2.5 models at 0.5B, 1.5B, 3B, 7B, 14B, and 32B (if feasible) on a fixed task suite would establish whether there exists a crossover scale beyond which the gating overhead (training instability, structured output cost) outweighs the gating benefit. If such a crossover exists, it would refine the paper's implicit claim that gating is broadly useful, instead positioning it as a technique specifically valuable for smaller models where intrinsic long-context capability is weak.

Disentangling the causal mechanisms: what does the update gate actually prevent, and what does the exit gate actually enable? The paper compares GRU-Mem w/o EG (update gate only) and GRU-Mem w EG (both gates) against MemAgent (neither gate), but never evaluates an "exit gate only, no update gate" configuration. This makes it impossible to attribute performance effects to the update gate vs. exit gate independently. A clean ablation would train and evaluate all four configurations (no gates, UG only, EG only, both gates) on the full benchmark suite, measuring performance, inference time, memory size dynamics, and per-step generation length. This would answer: (a) does the update gate improve performance primarily by preventing memory explosion (cleaner memory → better final answers), or primarily by reducing per-step generation cost (shorter memories → faster inference)? (b) does the exit gate ever improve performance (e.g., by preventing the model from accumulating noise after collecting sufficient evidence, as the MK-2 at 7B result suggests—84.15% w EG vs. 67.52% w/o EG), or is it purely an efficiency mechanism? (c) do the two gates interact synergistically (e.g., does clean memory from the update gate make exit decisions more accurate)? The paper hints at such interactions but provides no mechanistic decomposition.

Practical Applications and Downstream Use Cases

Cost-efficient batch inference for long-document analysis at scale. Organizations that process large corpora of long documents—law firms reviewing discovery documents, pharmaceutical companies extracting drug-interaction evidence from research papers, financial institutions analyzing earnings call transcripts—face a cost structure where inference cost scales with total context length. GRU-Mem's exit gate, when paired with a reranker that places key evidence early, can reduce inference cost to a fraction of the total context length (3–4× reduction demonstrated in Tables 2 and 4). For a batch of 10,000 legal documents averaging 200K tokens each, the difference between processing 200K tokens per document and processing 40K tokens (if evidence appears in the first 20%) is the difference between 2B and 400M tokens of inference compute—roughly a 5× cost reduction that translates directly to API savings or throughput improvements. The update gate provides an additional efficiency margin by suppressing verbose memory writes on irrelevant sections, though this benefit is harder to quantify without per-task memory-size measurements.

On-device or edge deployment of small models for long-context tasks. The finding that GRU-Mem's benefits are largest at the 3B scale (Table 1) makes it particularly relevant for scenarios where larger models cannot be deployed—mobile devices, privacy-sensitive on-premise systems, or bandwidth-constrained edge environments. A 3B model with GRU-Mem achieving 91%+ accuracy on multi-key NIAH tasks (MK-1 through MK-3 at 3B w/o EG: 91.52%, 67.08%, 91.41%) approaches the performance of a 7B MemAgent model on the same tasks (97.21%, 75.78%, 95.98%) at roughly half the model size. If the inference speedup is factored in (GRU-Mem at 3B is faster than MemAgent at 7B both because of the smaller model and the gating efficiency), the effective cost-per-query can be dramatically lower. This enables long-context QA on devices where a 7B model would exceed memory or latency budgets.

Self-improving agent loops with dynamic context budgets. In agentic systems where an LLM accumulates context over many interaction turns—customer support dialogues, multi-step tool-use tasks, long-horizon planning—the context grows monotonically and inevitably exceeds what the model can process effectively. GRU-Mem's chunk-by-chunk architecture with update gating provides a natural mechanism for compressing accumulated context into a bounded memory that grows sublinearly with interaction length. Unlike heuristic summarization (which may discard critical details) or fixed-window truncation (which loses older context entirely), the update gate lets the model selectively preserve information that remains relevant to the ongoing task. The exit gate is less directly applicable here (agentic loops typically don't have a well-defined "last evidence" point), but the update gate alone—which the paper shows provides substantial efficiency and stability gains in w/o EG mode—could maintain agent coherence over much longer interaction horizons than naive context accumulation allows. The key deployment question is whether the update gate's evidence-to-noise discrimination, trained on QA tasks with planted evidence, transfers to settings where "relevance" evolves dynamically across turns.

When to Prefer This Method

The paper does not articulate an explicit decision rule comparing GRU-Mem against named alternatives (e.g., "prefer GRU-Mem over retrieval-augmented generation when X, prefer retrieval when Y"). It positions GRU-Mem as an augmentation to the MemAgent paradigm rather than as a replacement for retrieval-based or single-pass approaches, and does not empirically compare against those alternatives. The training-data requirement (evidence-position labels for the update reward) and the domain limitation (QA-only evaluation) are presented as scope constraints, not as a tradeoff matrix against other methods.

The closest the paper comes to a usage guideline is the inference mode selection (Section 3.3): use w EG mode when the task requires finding a specific piece of evidence and early stopping is safe; use w/o EG mode when the task requires collecting information from throughout the context (e.g., the MV task). This is a deployment configuration choice within GRU-Mem, not a choice between GRU-Mem and competing methods.