ArXiv: 2512.12967

🎯 Pitch

A 30B open-source model matches GPT-5 and Gemini-2.5-Pro on long-context reasoning by using entropy-controlled RL to prevent catastrophic collapse during training on progressively longer sequences. The same recipe enables a memory agent to handle over 4 million tokens with a 9.5-point gain over standard agents, while surprisingly improving general scientific reasoning and dialogue without forgetting.

1. Executive Summary

This paper introduces QwenLong-L1.5, a comprehensive post-training recipe that elevates a 30B-parameter MoE model to long-context reasoning performance comparable to flagship proprietary systems like GPT-5 and Gemini-2.5-Pro. The work builds on Qwen3-30B-A3B-Thinking and contributes three named mechanisms: a Long-Context Data Synthesis Pipeline (deconstructing documents into atomic facts and programmatically composing multi-hop, verifiable reasoning questions), Stabilized Reinforcement Learning for Long-Context Training (task-balanced sampling with task-specific advantage estimation to mitigate reward bias, plus Adaptive Entropy-Controlled Policy Optimization (AEPO) that dynamically masks negative-advantage rollouts based on policy entropy to prevent training collapse), and a Memory-Augmented Architecture (a multi-stage fusion RL paradigm that integrates single-pass 256K-context reasoning with iterative memory-based processing for sequences exceeding 4M tokens). On six long-context reasoning benchmarks, QwenLong-L1.5 surpasses its Qwen3-30B-A3B-Thinking baseline by 9.90 points on average, establishes state-of-the-art on MRCR (82.99), and achieves a 9.48-point gain over the agent baseline on 1M–4M token tasks, establishing that long-context reasoning skills acquired through this recipe generalize to out-of-domain scientific reasoning, memory tool use, and extended dialogue only when the multi-stage progressive length extension and entropy-controlled optimization prevent catastrophic forgetting and training instability.

2. Context and Motivation

The Core Problem: Long-Context Reasoning's Missing Post-Training Recipe

This paper addresses a specific, under-explored gap in the LLM development pipeline: the absence of a mature, end-to-end post-training system for long-context reasoning. The field has made substantial progress in two adjacent areas β€” extending context windows during pre- and mid-training (Yang et al., 2025a; Team et al., 2025a; Deepmind, 2025) and designing novel attention architectures for efficiency (Team et al., 2025b; Qiu et al., 2025) β€” but the paper argues that what happens after these stages is equally critical and poorly understood. Specifically, the paper identifies three missing pieces that together constitute the gap it aims to fill:

  1. A scalable pipeline for synthesizing challenging long-context reasoning data. Existing long-context benchmarks and training sets often default to simple retrieval tasks β€” "needle-in-a-haystack" (Kamradt, 2023; Hsieh et al., 2024) or single-hop RAG (KočiskΓ½ et al., 2018) β€” that test whether a model can find information rather than whether it can reason over it. The paper contends that genuine long-context reasoning requires multi-hop grounding over globally distributed evidence, and that no existing synthesis framework produces such data at the scale and quality needed for RL-based post-training.

  2. RL-based methods tailored to the nuances of long-context reasoning. Standard RL approaches for reasoning (GRPO, PPO) were developed and validated primarily on short-context tasks like mathematics and code generation. The paper argues β€” and demonstrates empirically in Sections 4.2–4.4 β€” that long-context RL introduces qualitatively different challenges: unstable mini-batch distributions due to heterogeneous task types, reward estimation bias from mixing dense-reward and sparse-reward tasks, and training collapse caused by the high similarity between correct and incorrect reasoning paths (which share substantial phrase-level overlap because both are grounded in the same long context). These challenges are not addressed by existing RL recipes.

  3. Agent architectures for information streams exceeding context capacity. Even extended context windows (the paper's base model supports 256K tokens) cannot accommodate arbitrarily long sequences. The paper argues that a practical long-context reasoning system needs a mechanism β€” specifically, a memory-augmented agent β€” that can process information streams exceeding the physical context window, and that integrating such a mechanism with single-pass reasoning capabilities requires a deliberate training strategy (the multi-stage fusion RL paradigm described in Section 4.1) rather than naive combination.

Why This Problem Matters

The paper's framing of importance operates on multiple levels, from practical deployment to theoretical understanding:

Practical deployment pressures. The authors position long-context reasoning as a critical capability for modern LLMs, driving two major application classes: single-pass reasoning over long documents (Bai et al., 2025; Zhao et al., 2024; Krishna et al., 2025; Wu et al., 2024) and multi-turn autonomous agent systems that must maintain coherent state over extended interactions (Mialon et al., 2023; Patil et al., 2025; Barres et al., 2025). Both classes require models to integrate key information across a global scope and execute complex multi-hop inference. The paper implicitly argues that without a principled post-training recipe, even models with large context windows underperform on these tasks because they haven't been trained to use their context effectively.

The capability-utilization gap. This points to a deeper issue that the paper surfaces but doesn't name explicitly: the gap between having a long context window (a pre-training/mid-training achievement) and reasoning effectively within it (a post-training achievement). The baseline model, Qwen3-30B-A3B-Thinking-2507, already supports 256K tokens at the architectural level. Yet on benchmarks like MRCR (which tests multi-needle retrieval in synthetic conversations), it scores only 51.27 β€” compared to QwenLong-L1.5's 82.99 after post-training. This ~32-point gap demonstrates that context window capacity alone is insufficient; the model must be taught to ground and reason over long sequences.

Economic and accessibility implications. The paper's focus on a 30B-A3B MoE model (substantially smaller than the ~235B and larger proprietary systems it competes with) carries an implicit argument about democratization. If post-training can elevate a midsize open-weight model to flagship-level long-context reasoning performance, this reduces the barrier to entry for organizations that cannot train or serve trillion-parameter models. The paper makes this point indirectly through its benchmark comparisons (Figure 1, Table 7), where QwenLong-L1.5-30B-A3B achieves scores competitive with GPT-5 and Gemini-2.5-Pro while being a fraction of their size.

Theoretical significance for RL in language models. Beyond the application-level motivation, the paper tackles a fundamental RL challenge that is under-studied in the LLM literature: how to maintain stable policy optimization when correct and incorrect trajectories share substantial overlap. This is the credit assignment problem (Sutton & Barto, 1998; Arumugam et al., 2021) made acute by long-context reasoning: because all responses must be grounded in the same long context, correct and incorrect reasoning paths can be nearly identical up to a critical error (see Figures 13 and 14 in the paper for concrete examples). Standard GRPO assigns a uniform advantage signal to the entire sequence, meaning tokens that are part of correct reasoning receive the same penalty as the erroneous token that caused the failure. The paper's proposed solutions β€” entropy-controlled gradient clipping and AEPO β€” represent a contribution to the RL-for-LLMs literature that extends beyond long-context specifically.

Where Prior Approaches Fall Short

The paper identifies deficiencies in existing work along four axes:

1. Data synthesis for long-context reasoning is underdeveloped. The dominant paradigm for long-context evaluation and training has been retrieval-centric. Needle-in-a-haystack tests (Kamradt, 2023) ask whether a model can locate a single fact within a long distractor text β€” a valuable sanity check, but fundamentally a search task, not a reasoning task. RULER (Hsieh et al., 2024) extends this to multiple needles but retains the retrieval framing. Single-hop QA datasets (KočiskΓ½ et al., 2018) require understanding a single passage within a long document but don't demand cross-passage integration. The paper argues that these tasks fail to develop the capabilities that matter for real-world long-context applications: "multi-hop grounding and reasoning over globally distributed evidence" (Section 1). Moreover, existing datasets are predominantly human-annotated, which the paper observes becomes impractical when contexts exceed ~32K tokens β€” human annotators "struggle to both formulate challenging questions requiring extensive reasoning and exhaustively verify answers" (Section 3). This scalability bottleneck motivates the paper's automated synthesis approach.

2. Standard RL recipes fail on long-context data. The paper identifies two specific failure modes of vanilla GRPO when applied to long-context training:

  • Distributional instability from heterogeneous data. As visualized in Figure 7, long-context training data exhibits a "multi-clustered distribution" β€” different task types (multiple-choice, multi-hop QA, NIAH, numerical calculation, dialogue memory) occupy distinct regions of embedding space. This contrasts sharply with math training data (e.g., DAPO-Math-17K, Open-R1 Codeforces), which is more homogeneous. A traditional random sampler can produce mini-batches dominated by a single task type, leading to unstable gradient updates. Figure 8 demonstrates this concretely: baseline GRPO shows erratic entropy dynamics and uncontrolled response length growth.

  • Biased advantage estimation from mixed reward densities. GRPO normalizes advantages using group-level reward standard deviation (Equation 3). The paper observes that this introduces bias when tasks have fundamentally different reward structures: NIAH tasks typically yield dense, continuous reward signals (e.g., 0–1 range), while QA and multiple-choice tasks yield sparse, discrete rewards (correct = 1, incorrect = 0, with little intermediate signal). Normalizing across these reward types in a batch-level computation distorts the advantage estimates, penalizing or rewarding sequences relative to an inappropriate baseline.

3. Training collapse from high similarity between correct and incorrect trajectories. This is the paper's most subtle β€” and arguably most interesting β€” critique of existing methods. As quantified in Table 3, the phrase-level overlap (ROUGE-L) between correct and incorrect responses on DocMath (a long-context numerical reasoning benchmark) is 45.37, compared to 27.71 on AIME24/25 (a short-context math benchmark). This ~64% higher similarity means that incorrect long-context responses contain many tokens that are part of a valid reasoning path β€” they go wrong at a specific step while sharing substantial structure with correct solutions. The paper argues this "exacerbates the reward credit assignment problem" because GRPO applies a uniform negative advantage to all tokens in an incorrect sequence, penalizing tokens that are actually correct and thereby destabilizing training. The symptoms of this instability include entropy collapse and, in the worst case, training divergence.

4. Memory-augmented architectures are not integrated with single-pass reasoning. The paper acknowledges prior work on memory agents for ultra-long contexts (specifically citing Yu et al., 2025a's MemAgent framework) but identifies a key limitation: "mixing memory management training data and single-pass full-context processing training data together causes considerable damage to the overall RL training infrastructure efficiency and model training stability" (Section 4.1). Prior approaches treated memory management and direct long-context reasoning as separate capabilities trained independently. The paper's contribution is demonstrating that they can be unified into a single model through a specific training paradigm (train specialized experts separately, then merge via SCE, then fine-tune jointly) without destructive interference.

How This Paper Positions Itself

The paper positions QwenLong-L1.5 not as a single novel method but as a comprehensive post-training system that unifies three components β€” data, training methodology, and architecture β€” into a coherent recipe. This systems-level framing is deliberate and distinguishes the work from papers that propose isolated techniques.

Relationship to QwenLong-L1. The paper explicitly builds on its predecessor, QwenLong-L1 (Wan et al., 2025), which introduced the multi-stage length extension paradigm and difficulty-aware retrospective sampling. QwenLong-L1.5 represents a substantial scale-up: training data expands from 1.6K to 14.1K samples (8.8Γ—), average input length increases from 11,441 to 34,231 tokens (3Γ—), and the data synthesis pipeline moves from primarily open-source curation to automated generation across three distinct methods (knowledge-graph-guided multi-hop QA, structural tabular data engine, and multi-agent self-evolution). Critically, QwenLong-L1 did not include the memory management framework or the stabilized RL innovations (task-balanced sampling, task-specific advantage estimation, AEPO), which are novel contributions of the L1.5 work.

Relationship to GRPO and DAPO. The paper adopts GRPO (Shao et al., 2024) as its base RL algorithm and incorporates one key technique from DAPO (Yu et al., 2025b): token-level policy gradient loss normalization, which prevents long responses from dominating the learning signal. However, the paper explicitly departs from standard GRPO in two ways: (1) it removes KL regularization entirely (Ξ² = 0), consistent with recent findings that KL penalties can impede exploration (Hu et al., 2025; Yu et al., 2025b; Wan et al., 2025), and (2) it operates in a strictly on-policy setting with a single gradient update per batch, which simplifies the objective by making the importance sampling ratio always 1 and rendering the clipping function inactive (Equation 4). The paper's RL contributions β€” task-balanced sampling, task-specific advantage estimation, and AEPO β€” are presented as additions to this simplified GRPO backbone that address long-context-specific challenges.

Relationship to memory agent literature. The paper adopts the Memory Agent paradigm from Yu et al. (2025a), which reframes reading comprehension as sequential decision-making with explicit memory updates and navigational planning. QwenLong-L1.5 extends this framework in two ways: (1) it incorporates user query decomposition into core question and formatting instructions, preventing format constraints from interfering with reasoning during memory updates, and (2) it introduces explicit planning alongside memory updates (Equation 5), where the agent generates both an updated memory state and a plan for processing the next chunk. The multi-stage fusion training paradigm (train full-context expert β†’ train memory expert β†’ merge via SCE β†’ joint fine-tune) is presented as a novel contribution for resolving the destructive interference that occurs when these capabilities are trained simultaneously.

The unifying thesis. The paper's overarching position is that long-context reasoning is a distinct capability that must be deliberately cultivated through post-training, not an automatic consequence of long-context pre-training. This thesis is supported by the 9.90-point gap between the baseline model (which already has a 256K context window from pre-training) and QwenLong-L1.5 (which adds targeted post-training). The paper further argues that the benefits of this cultivation are foundational rather than task-specific β€” the skills of information integration, multi-hop grounding, and maintaining coherence over extended sequences transfer to out-of-domain tasks (scientific reasoning, agentic memory, dialogue), as demonstrated in Section 5.3. This positions long-context post-training not as a narrow specialization but as a general capability enhancer, a claim that carries significant implications for how the field allocates post-training resources.

3. Technical Approach

This is primarily a systems-building paper whose core idea is that long-context reasoning is a distinct capability that must be deliberately cultivated through a specialized post-training recipe β€” not an automatic byproduct of long-context pre-training β€” and that this recipe requires co-designing the data synthesis pipeline, the RL training methodology, and the agent architecture to achieve stable, scalable improvements.

3.1 Reader orientation (approachable technical breakdown)

The paper builds a complete post-training system that takes a base language model with a 256K-token context window (Qwen3-30B-A3B-Thinking) and systematically teaches it to reason over long documents through a pipeline of data synthesis, reinforcement learning, and memory-augmented inference. The system solves the problem that simply having a long context window does not mean a model can use it effectively β€” the model must learn to ground information across globally distributed evidence, perform multi-hop reasoning, and manage contexts that exceed its physical window, and it must learn these skills in a stable, generalizable way without forgetting its existing capabilities.

3.2 Big-picture architecture (diagram in words)

The system has five major components arranged in a sequential pipeline, with a sixth component (memory agent) integrated through a specialized training stage and model merging:

  1. Document Corpus β€” A curated repository of 82,175 long documents (code repositories, academic literature, professional documents, general knowledge, dialogue data) totaling ~9.2 billion tokens, assembled from web crawling and open-source corpora and filtered through multi-stage quality checks. This is the raw material for all training data synthesis.

  2. Data Synthesis Engine β€” Three distinct methods that programmatically generate 14.1K question-answer pairs from the document corpus: a knowledge-graph-guided multi-hop reasoning pipeline, a structural tabular data engine for numerical reasoning, and a multi-agent self-evolution framework for general long-context tasks. The output is high-quality, challenging RL training data where questions require reasoning over globally distributed evidence.

  3. Multi-Stage RL Training Framework — Four sequential stages of full-context RL training with progressively increasing input/output lengths (32K→60K→120K tokens), plus a parallel memory-RL training stage. Each stage uses the GRPO algorithm augmented with three stabilization mechanisms: task-balanced sampling, task-specific advantage estimation, and Adaptive Entropy-Controlled Policy Optimization (AEPO).

  4. Memory Agent Architecture β€” A separate training pipeline that teaches the model to process ultra-long documents (beyond its 256K window) by chunking the context and iteratively updating a compressed memory state with explicit navigational planning, followed by a final answer generation step.

  5. Model Merging via SCE β€” The full-context reasoning expert (output of Stage 3) and the memory management expert are merged using the SCE algorithm (Wan et al., 2024), creating a single model that retains both capabilities. This merged model then undergoes a final Stage 4 full-context RL training to produce the unified QwenLong-L1.5 model.

Information flows as follows: documents enter the synthesis engine β†’ question-answer pairs undergo knowledge grounding and contextual robustness verification β†’ verified QA pairs enter the multi-stage RL pipeline β†’ each stage trains on progressively longer contexts with task-balanced sampling and AEPO entropy control β†’ in parallel, the memory agent is trained on chunked ultra-long documents β†’ the two expert models are merged via SCE and fine-tuned jointly β†’ the final model can handle both single-pass inference up to 128K tokens and memory-augmented processing beyond 4M tokens.

3.3 Roadmap for the deep dive

  • First, the mathematical formulation of long-context RL and the simplified GRPO objective (Section 2.2–2.3 of the paper), because all subsequent training components are built on this foundation and understanding the credit assignment problem requires knowing what GRPO does and does not optimize.

  • Second, the data synthesis pipeline (Section 3), which is the system's input and the primary driver of the model's multi-hop reasoning capabilities β€” understanding what makes the data "challenging" is essential for understanding why the RL innovations are necessary.

  • Third, the multi-task RL stabilization mechanisms (Sections 4.2–4.4): task-balanced sampling, task-specific advantage estimation, negative gradient clipping, and AEPO β€” these address the specific failure modes that emerge when GRPO is applied to long-context data.

  • Fourth, the progressive multi-stage training paradigm (Section 4.1), which sequences these RL stages with increasing context lengths and synchronously extends output lengths, including the model merging strategy for integrating memory management.

  • Fifth, the memory agent architecture (Section 2.3), which extends the model's operational range beyond its physical context window through sequential chunk processing, explicit memory updates, and navigational planning.

3.4 Detailed, sentence-based technical breakdown

Mathematical Formulation: Long-Context RL and Simplified GRPO

The paper formulates long-context reasoning as a reinforcement learning problem over documents and questions. Given a set of $n$ documents $\{c_i\}_{i=1}^n$ and a question $q$, the goal is to optimize a policy model $\pi_\theta$ to generate a response $y$ that maximizes a reward function $r_\phi(c, q, y)$. The standard objective maximizes the KL-regularized expected reward:

max⁑πθEc,q∼D,yβˆΌΟ€ΞΈ(β‹…βˆ£c,q)[rΟ•(c,q,y)]βˆ’Ξ²DKL[πθ(y∣c,q)Β βˆ£βˆ£Β Ο€ref(y∣c,q)]\max_{\pi_\theta} \mathbb{E}_{c,q \sim \mathcal{D}, y \sim \pi_\theta(\cdot|c,q)} [r_\phi(c, q, y)] - \beta D_{\text{KL}} [\pi_\theta(y | c, q) \ || \ \pi_{\text{ref}}(y | c, q)]

where $c = \text{Concat}(c_1, c_2, \ldots, c_n)$ is the concatenated long context, $\mathcal{D}$ is the training dataset, $\pi_{\text{ref}}$ denotes a reference policy (typically the pre-trained model before RL), and $\beta$ controls the strength of the KL regularization to prevent large deviations from the reference policy.

What it computes: a policy $\pi_\theta$ that generates high-reward responses while staying close to the reference distribution. The expectation samples a document-question pair from the dataset and a response from the current policy; the reward evaluates response quality; the KL penalty ensures the policy doesn't drift too far from its initial behavior, which can cause reward hacking or catastrophic forgetting.

Why this form: KL-regularized expected reward maximization is the standard formulation in RLHF (Schulman et al., 2017a; Wan et al., 2025). The KL term is crucial for maintaining output quality and preventing the policy from exploiting reward model imperfections. However, as the paper notes in Section 2.2, it makes the deliberate choice to set $\beta = 0$ β€” removing the KL penalty entirely β€” because recent work (Hu et al., 2025; Yu et al., 2025b; Wan et al., 2025) found that removing KL regularization can improve exploration and accelerate convergence. This is a significant design choice that increases the risk of reward hacking but is mitigated by the paper's other stabilization mechanisms.

GRPO and Its Simplification for Long-Context Training

The paper employs Group Relative Policy Optimization (GRPO) as its base RL algorithm. For long-context inputs, the quadratic complexity of attention makes PPO β€” which relies on generalized advantage estimation via a separate value network β€” computationally prohibitive (a value network would require processing the full long context for each advantage estimate, multiplying the already high per-example cost). GRPO avoids this by sampling a group of $G$ candidate responses and normalizing rewards within the group:

JGRPO(ΞΈ)=Ec,q∼D,{yi}i=1GβˆΌΟ€ΞΈold(β‹…βˆ£c,q)[1Gβˆ‘i=1G1∣yiβˆ£βˆ‘t=1∣yi∣[min⁑(ρi,t(ΞΈ)Ai,Β clip(ρi,t(ΞΈ),1βˆ’Ξ΅,1+Ξ΅)Ai)]βˆ’Ξ²DKL(Ο€ΞΈβˆ£βˆ£Ο€ref)]J_{\text{GRPO}}(\theta) = \mathbb{E}_{c,q \sim \mathcal{D}, \{y_i\}_{i=1}^G \sim \pi_{\theta_{\text{old}}}(\cdot|c,q)} \left[ \frac{1}{G} \sum_{i=1}^G \frac{1}{|y_i|} \sum_{t=1}^{|y_i|} \left[ \min\left( \rho_{i,t}(\theta) A_i, \ \text{clip}\left( \rho_{i,t}(\theta), 1-\varepsilon, 1+\varepsilon \right) A_i \right) \right] - \beta D_{\text{KL}}(\pi_\theta || \pi_{\text{ref}}) \right]

where $\rho_{i,t}(\theta) = \frac{\pi_\theta(y_{i,t}|c,q,y_{i,<t})}{\pi_{\theta_{\text{old}}}(y_{i,t}|c,q,y_{i,<t})}$ is the importance sampling ratio for token $t$ in sequence $i$, measuring how much the new policy's probability of that token differs from the old policy's.

What this computes: for each group of $G$ responses to the same input, GRPO estimates each response's advantage $A_i$ as its reward normalized by the group's mean and standard deviation, then uses PPO's clipped surrogate objective to update the policy β€” increasing the probability of tokens in high-advantage responses and decreasing it for low-advantage responses, with clipping to prevent destructively large updates.

Why this form: the group-based advantage normalization eliminates the need for a value network (the key computational bottleneck in PPO) because the baseline for each reward is computed from the other samples in the group rather than from a learned value function. This is crucial for long-context training where maintaining a separate value network would multiply the already high memory and compute costs.

The group-relative advantage $A_i$ is computed as:

Ai=riβˆ’mean({rk}k=1G)std({rk}k=1G)A_i = \frac{r_i - \text{mean}(\{r_k\}_{k=1}^G)}{\text{std}(\{r_k\}_{k=1}^G)}

where $r_i$ is the sequence-level reward for response $i$, mean and std are computed over all $G$ responses in the group.

What it computes: a z-score normalized advantage that centers and scales the group's rewards. Positive advantages mean the response performed above the group average; negative advantages mean below average. The normalization ensures the scale of advantages is independent of the absolute reward magnitudes, which can vary significantly across task types.

Why this form: z-score normalization provides a natural baseline without a learned value function and automatically adapts to the current policy's performance level. However, as the paper identifies in Section 4.2, this group-level normalization introduces bias when the group size is small or when the reward distribution within the group is skewed, motivating the paper's task-specific advantage estimation.

The Simplified On-Policy Objective

The paper makes three design choices that substantially simplify the GRPO objective, justified by the specific characteristics of long-context training:

1. No KL regularization ($\beta = 0$): Following recent findings that KL penalties can impede exploration (Hu et al., 2025; Yu et al., 2025b; Wan et al., 2025), the paper removes the KL term entirely. This means the policy is free to diverge from the reference model, which increases the risk of reward hacking but allows more aggressive learning β€” a tradeoff the paper manages through its entropy-control mechanisms.

2. Strictly on-policy with single gradient update: The paper performs only a single gradient update per batch of samples and then discards the data. This means that the policy being updated, $\pi_\theta$, is identical to the policy that generated the data, $\pi_{\theta_{\text{old}}}$. Consequently, the importance sampling ratio $\rho_{i,t}(\theta)$ is always exactly 1 (because $\pi_\theta(y_{i,t}) / \pi_{\theta_{\text{old}}}(y_{i,t}) = 1$ when $\pi_\theta = \pi_{\theta_{\text{old}}}$). This renders the clipping function inactive β€” there is no ratio to clip β€” and allows removing both the min and clip operations.

3. Token-level loss normalization from DAPO: The paper adopts the key technique from DAPO (Yu et al., 2025b): normalizing each token's contribution by the total number of tokens in the group ($\sum_{j=1}^G |y_j|$) rather than by the number of sequences ($G$). This ensures every token across all responses in the group contributes equally to the final objective, preventing long responses from dominating the learning signal and preventing short responses from having their individual tokens weighted too heavily.

With these simplifications, the GRPO objective reduces to:

JGRPO(ΞΈ)=Ec,q∼D,{yi}i=1GβˆΌΟ€ΞΈold[1βˆ‘j=1G∣yjβˆ£βˆ‘i=1GAiβˆ‘t=1∣yi∣ρi,t(ΞΈ)]J_{\text{GRPO}}(\theta) = \mathbb{E}_{c,q \sim \mathcal{D}, \{y_i\}_{i=1}^G \sim \pi_{\theta_{\text{old}}}} \left[ \frac{1}{\sum_{j=1}^G |y_j|} \sum_{i=1}^G A_i \sum_{t=1}^{|y_i|} \rho_{i,t}(\theta) \right]

where $|y_j|$ is the length in tokens of response $j$, $A_i$ is the group-relative advantage for response $i$, and $\rho_{i,t}(\theta) = 1$ (because on-policy). The outer expectation is over the training dataset and the sampled responses.

What it computes: for each group of $G$ responses, this objective computes a weighted sum of token-level policy gradients, where the weight for each token is the advantage of its response divided by the total number of tokens in all responses. High-advantage responses contribute positive gradients (increasing the probability of their tokens); low-advantage responses contribute negative gradients (decreasing token probabilities). The $1/\sum|y_j|$ normalization ensures the gradient magnitude is independent of group-level response length variations.

Why this form: it is the simplest possible policy gradient objective compatible with GRPO's group-based advantage estimation. By eliminating the clipping and KL terms, the objective reduces to an advantage-weighted likelihood maximization β€” essentially, the policy becomes more likely to generate tokens that appeared in high-advantage responses. The token-level normalization prevents the learning signal from being "diluted" in long responses (where many tokens share a single advantage) or concentrated in short responses. The paper argues this simplified objective is sufficient for long-context RL because the entropy-control mechanisms (Sections 4.3–4.4) provide the necessary regularization.

The Memory Agent Formulation

For sequences exceeding the model's physical context window (256K tokens), the paper adopts the Memory Agent paradigm from Yu et al. (2025a), which reframes reading comprehension as a sequential decision-making process. The formalization has two stages:

Sequential Memory Processing: The user query is decomposed into a core question $q_{\text{core}}$ and formatting instructions $q_{\text{inst}}$. This separation prevents format constraints (e.g., JSON schemas) from interfering with reasoning during iterative memory updates. The long document is partitioned into chunks $\{x_1, \ldots, x_K\}$. At each step $t$, the policy observes:

  • The current chunk $x_t$
  • The previous memory state $m_{t-1}$
  • The previous navigational plan $p_{t-1}$

and produces an updated memory $m_t$ and a plan $p_t$ for the next chunk:

(mt,pt)βˆΌΟ€ΞΈ(β‹…Β βˆ£Β mtβˆ’1,ptβˆ’1,xt,qcore)(m_t, p_t) \sim \pi_\theta(\cdot \ | \ m_{t-1}, p_{t-1}, x_t, q_{\text{core}})

What it computes: at each chunk, the model reads the new content, integrates relevant information into its compressed memory, and generates a plan for what to look for in the next chunk. The memory accumulates information across the entire document; the plan guides attention and extraction in subsequent chunks.

Why this form: this recurrent mechanism effectively "folds" the global context into a compact representation, replacing full-attention over all chunks with a sequential update that only requires attending to the current chunk and the memory state. This is what enables processing beyond the model's physical context window β€” the memory serves as a learned compression of all previously seen information. The explicit planning output $p_t$ is a novel addition beyond the basic MemAgent framework, designed to give the model a mechanism for actively directing its own information extraction strategy rather than passively accumulating facts.

Final Answer Generation: After processing all $K$ chunks, the model generates the answer $y$ by integrating the accumulated memory $m_K$ with the original formatting instructions:

yβˆΌΟ€ΞΈ(β‹…Β βˆ£Β mK,qcore,qinst)y \sim \pi_\theta(\cdot \ | \ m_K, q_{\text{core}}, q_{\text{inst}})

What it computes: the final answer is generated conditioned on the complete compressed memory (representing the entire document) and the formatting instructions (ensuring the output structure matches what was requested). The core question is included to maintain task focus.

Trajectory-Level RL for Memory: The policy is optimized using GRPO with trajectory-level rewards. For each question and document, $G$ distinct trajectories are sampled, where a trajectory $\tau_i$ consists of all memory updates, plans, and the final answer:

Ο„i={(mi,1,pi,1),…,(mi,K,pi,K),yi}\tau_i = \{(m_{i,1}, p_{i,1}), \ldots, (m_{i,K}, p_{i,K}), y_i\}

What it computes: a complete sequence of decisions across all chunks, from initial memory state to final answer. The trajectory-level reward $R(\tau_i)$ is based on answer correctness, and the trajectory-level advantage (computed via Equation 3) is broadcast as a consistent learning target to all actions in the trajectory β€” meaning every memory update and plan within a trajectory receives the same advantage signal derived from the final answer's correctness.

Why this form: broadcasting a single trajectory-level advantage to all actions treats the entire memory management process as a single decision β€” the model is rewarded or penalized for the complete chain of information extraction and reasoning that led to the final answer. This is simpler than per-step advantage estimation (which would require credit assignment across individual chunks) but relies on the assumption that all actions in a correct trajectory were helpful and all actions in an incorrect trajectory need improvement. The paper's AEPO mechanism (Section 4.4) partially addresses the credit assignment problem at this trajectory level.

The Long-Context Data Synthesis Pipeline

The data synthesis pipeline is the most architecturally complex component of the system, designed to produce training data that specifically targets multi-hop reasoning over globally distributed evidence β€” the capability the paper argues is missing from standard long-context training sets. The pipeline has three stages, with three distinct synthesis methods operating in parallel during the QA generation stage.

Stage 1: Corpus Collection and Preprocessing. The foundation is a multi-source, heterogeneous corpus of long documents gathered from five primary categories:

  • Code repositories: high-starred, high-quality open-source repositories, primarily in Python
  • Academic literature: scholarly papers and textbooks from STEM, medicine, law, social sciences, and recent AI research papers from arXiv
  • Professional documents: corporate annual reports, financial statements, product manuals, medical textbooks, and government publications
  • General knowledge and literature: classic novels, detective stories, and comprehensive Wikipedia pages
  • Dialogue data: a small collection of multi-turn dialogues simulated by large language models

All documents undergo "meticulous rule-based and LLM-as-a-judge filtering to ensure high quality, factual accuracy, and coherence" (Section 3.1). This yields a final repository of 82,175 high-quality documents totaling approximately 9.2 billion tokens. The diversity of document types is deliberate: different reasoning patterns (numerical calculation from financial reports, temporal reasoning from historical narratives, causal analysis from academic papers) require different document structures, and the model must learn to handle all of them.

Stage 2: Question-Answer Synthesis. The paper implements three distinct synthesis methods, each targeting a different class of reasoning challenges:

Method 1: Knowledge-Graph-Guided Multi-hop Reasoning QA. This is the most sophisticated synthesis method and the one most closely aligned with the paper's core thesis about multi-hop grounding. It operates in three sub-stages:

  1. KG Construction: Triple extraction from documents across diverse domains forms an initial knowledge graph. This is expanded into a complex cross-document KG via domain-level aggregation and refined using entity and relation clustering. The key design choice is cross-document aggregation β€” by merging entities and relations across different documents, the KG encodes connections that no single document contains, enabling the generation of questions that cannot be answered from any single source.

  2. Reasoning Path Sampling: Challenging multi-hop paths are generated by sampling relation-relevant subgraphs centered on target entities. Long-range paths are derived using Random Walk and BFS (Breadth-First Search). Three deliberate difficulty-amplification strategies are applied:

    • Path nodes are "deliberately distributed sparsely across multiple documents," forcing the model to gather evidence from different locations in the context
    • Path complexity is heightened via "information perturbation," including entity obfuscation β€” for example, replacing explicit entity names with descriptions like "the year ending with 5 in the late 20th century" (temporal obfuscation) or "a prestigious science university in Beijing" (institutional obfuscation)
    • Complexity is controlled by regulating path length (longer paths = more hops of reasoning required)

  3. Question Generation: Multi-hop QA pairs are synthesized from the extracted paths using a "multi-paradigm approach" spanning distinct reasoning types:

    • Multi-fact Reasoning: questions requiring integration of multiple facts across documents
    • Temporal Reasoning: questions involving chronological relationships and time-based inference
    • Causal Analysis: questions requiring identification of cause-effect chains spanning documents
    • Hypothetical Scenarios: counterfactual questions that test deep understanding of relationships
    • Quality is ensured through Blind Knowledge Screening (can the model answer without the documents? If so, discard) and Scarce Knowledge Validation (is the required information truly distributed and non-trivial to locate?)

The case studies in Appendix B (Section C of the paper) illustrate the resulting question types. For example, a Multi-fact Reasoning question asks: "Identify the character from a separate gaming franchise whose outfit is available as an unlockable cosmetic within the final Assassin's Creed installment for which one of the original game's creators served as creative director..." β€” this requires navigating a complex chain of relationships across documents about video games, creative directors, and game releases.

Method 2: Structural Tabular Data Engine for Corpus-level Numerical Reasoning QA. This method addresses a specific weakness: generating high-quality numerical reasoning questions across multiple documents typically requires manual authoring and validation. The engine automates this through a sequence of structured data transformations:

  1. Document Collection and Filtering: Unstructured documents are parsed and filtered to ensure sufficient token volume and the presence of rich statistical tables. The key insight is that documents containing tables (financial reports, statistical publications) have a natural structure that can be exploited for automated question generation.

  2. Schema Extraction: The underlying data structure and relational schema are rigorously formalized from the tables, converting unstructured table content into typed columns with known relationships.

  3. Data Table Aggregation: Disparate content across documents is transformed into a unified, structured, cross-document corpus table. This is the critical step that enables cross-document numerical reasoning β€” by joining tables from different documents, questions can require aggregating statistics that no single document contains.

  4. NL2SQL Execution: A diverse pool of natural language queries is generated through LLM-based expansion from initial templates. These queries are translated into executable SQL statements. By executing the SQL against the aggregated tables, complex calculation processes (statistical aggregation, numerical computation, temporal comparisons) are accurately simulated, and ground truth answers are derived programmatically. The final step concatenates the relevant source documents to form the long-context QA pair.

This method specifically targets "Statistical Aggregation, Numerical Calculation, and Temporal Reasoning" β€” problem types that are mathematically verifiable (the SQL execution provides exact ground truth) and that require "advanced capabilities for global information integration and complex numerical reasoning."

Method 3: Multi-Agent Self-Evolved (MASE) Framework for General Long-Context Reasoning. For other general long-context tasks β€” Viewpoint Analysis, Long In-context Learning, and others listed in Table 1 β€” the paper uses a multi-agent framework that automatically proposes, solves, and verifies questions through iterative collaboration:

The framework involves three distinct agents operating iteratively:

  • Proposer agent: Given a cluster of $n$ documents $C = \{c_i\}_{i=1}^n$ and a task type $\tau$, generates new question-answer pairs. In the first iteration, it is conditioned only on the raw documents. After each verified pair is created, it is appended to a history buffer $\mathcal{H}$. In subsequent iterations, the proposer is conditioned on both the raw documents AND the stored QA pairs, with explicit prompting to generate harder and more diverse questions than those already proposed β€” creating a difficulty curriculum.

  • Solver agent: Attempts to solve the generated question based on the documents. The paper uses a "rule-based method to extract the final prediction $y$ from the response."

  • Verifier agent: Estimates the semantic equivalence between the solver's prediction $y$ and the proposer's reference answer $a$. Validated tasks are stored to the RL dataset; their QA pairs are saved to the history buffer.

Why this multi-agent design: the separation of proposer, solver, and verifier creates a check-and-balance system. The proposer can generate ambitious, difficult questions without being constrained by whether it can solve them itself; the solver provides an independent assessment of answerability; the verifier ensures quality by confirming that the solver's answer matches the proposer's intended answer. The history buffer mechanism creates a natural curriculum β€” as more questions are generated for a document set, the proposer must find increasingly creative and difficult angles to avoid redundancy.

Stage 3: Data Verification. All synthesized samples undergo two critical validation checks:

  1. Knowledge Grounding Check: The source document is temporarily removed, and the model is tested on whether it can still answer the question. Samples that can be answered correctly from the model's internal knowledge are filtered out. This ensures the data specifically tests contextual reasoning β€” if the model already knows the answer, the question doesn't teach it to use the context.

  2. Contextual Robustness Check: The context is expanded with irrelevant documents, and the model's answer is verified. Any sample where the answer accuracy (pass@k) drops to zero is discarded. This ensures the question and its answer are "robust not brittle to context changes" β€” the model must be able to locate the relevant information even when the context is expanded with distractors.

Scale and characteristics of the final dataset: From an initial pool of 42.7K synthesized examples, after multi-stage difficulty filtering, deduplication, and test set decontamination, the final RL training set contains 14.1K high-quality samples. Compared to QwenLong-L1's 1.6K samples (primarily from open-source curation), this represents an 8.8Γ— scale-up. Key characteristics from Table 1:

  • Data sources expanded from 3 domains to 5 (adding Dialogue data and expanding General knowledge)
  • Question types expanded from 2 to 9+ (adding Hypothetical scenarios, Long in-context learning, Temporal reasoning, Causal analysis, Viewpoint analysis, Dialogue NIAH, and others)
  • Max input length increased from 59,563 to 119,932 tokens
  • Average input length increased from 11,441 to 34,231 tokens
  • Training data longer than 64K tokens substantially increased (visible in Figure 3's distribution comparison)

This shift toward longer, more complex training data is deliberate: the paper argues that to develop genuine long-context reasoning capabilities, the model must be trained on data that requires reasoning over extended contexts, not just retrieval from arbitrary-length documents.

Multi-Task RL Stabilization: Task-Balanced Sampling and Task-Specific Advantage Estimation

The paper identifies that long-context RL data is fundamentally different from the short-context reasoning data (mathematics, code) that GRPO was originally developed for. Figure 7 visualizes this through UMAP projections: the QwenLong-L1.5 training set forms distinct, separated clusters corresponding to different task types, while math datasets (DAPO-Math-17K, Open-R1 Codeforces) form more cohesive, overlapping distributions. This "multi-clustered distribution" causes two specific failure modes in vanilla GRPO:

Failure Mode 1: Unstable mini-batch distributions. A traditional random sampler can produce training batches dominated by a single task type, leading to gradient updates that overfit to one pattern and then overcorrect when the next batch is dominated by a different pattern β€” manifesting as erratic entropy dynamics (Figure 8b).

Failure Mode 2: Biased advantage estimation from mixed reward densities. Different task types have fundamentally different reward structures. NIAH tasks (like MRCR) typically yield dense rewards spanning the full [0, 1] range, while QA and multiple-choice tasks yield sparse, discrete rewards (typically just 0 or 1). When GRPO normalizes advantages using group-level standard deviation (Equation 3), it estimates the normalization denominator from only $G = 8$ samples. If those 8 samples happen to be mostly NIAH (with high variance rewards), the std estimate will be inflated, making QA rewards look less significant than they should. Conversely, a group of mostly QA samples (all with reward 1) will have std β‰ˆ 0, making even small reward differences appear enormous.

Solution 1: Task-Balanced Sampling. The paper implements a two-stage balancing strategy:

  • Pre-training stratification: Before RL training begins, the base model performs pre-inference on data from each source. The resulting pass@k scores are used to stratify the data into uniform bins. An equal number of training samples are then uniformly sampled from each bin. This ensures the training set has balanced difficulty representation β€” the model doesn't over-train on easy or hard instances.

  • Training-time task-balanced sampling: During RL training, the conventional random sampler is replaced with a task-balanced sampler. In each training batch, an equal number of samples are drawn from each of five designated task types: multiple choices, doc multi-hop reasoning, general reading comprehension, dialogue memory, and corpus-level numerical calculation.

Why this works: by enforcing task diversity in every batch, the sampler prevents any single task from dominating the gradient update. This stabilizes the entropy dynamics (Figure 8b shows much smoother entropy curves with task-balanced sampling) because the policy receives consistent, diverse learning signals at each step rather than oscillating between task-specific optima.

Solution 2: Task-Specific Advantage Estimation. Instead of normalizing advantages using the group-level or batch-level standard deviation, the paper computes the normalization denominator from all samples belonging to the same task within the current training batch $\mathcal{B}_{\text{task}}$:

Aitask=ritaskβˆ’mean({rktask}k=1G)std(rtask ∣ rtask∈Btask),task∈{mc,qa,niah,…}A_i^{\text{task}} = \frac{r_i^{\text{task}} - \text{mean}(\{r_k^{\text{task}}\}_{k=1}^G)}{\text{std}(r^{\text{task}} \ | \ r^{\text{task}} \in \mathcal{B}_{\text{task}})}, \quad \text{task} \in \{\text{mc}, \text{qa}, \text{niah}, \ldots\}

What it computes: for response $i$, the advantage is the z-score of its reward relative to the mean and standard deviation of all rewards from the same task type in the current batch (not just the 8-sample group). The task type is identified from metadata (mc = multiple choice, qa = multi-hop QA, niah = needle-in-a-haystack, etc.).

Why this form: it solves both failure modes simultaneously. By computing the standard deviation within each task type separately, tasks with different reward densities are not mixed β€” NIAH's dense 0–1 rewards don't distort the normalization for QA's sparse 0/1 rewards, and vice versa. Additionally, by using all samples of the same task in the batch (not just the 8-sample group), the standard deviation estimate is more stable because it's based on a larger sample. The paper notes that task-specific estimation "generally yields a larger normalization denominator, which prevents excessive gradient update" β€” this larger denominator reduces the magnitude of advantage signals, acting as an implicit learning rate reduction that further stabilizes training.

Empirical validation (Table 2, Figure 8): On Qwen3-4B-Thinking, task-balanced sampling alone provides a moderate improvement (+0.79 points over GRPO baseline). Combining task-balanced sampling with batch-level standard deviation yields +1.38 points. The full combination (task-balanced sampling + task-specific advantage estimation) yields +2.55 points over GRPO (from 56.07 to 58.62 average). Figure 8 confirms the mechanism: reward growth is similar across methods, but the full combination stabilizes entropy (panel b) and controls response length growth (panel c), preventing the uncontrolled length explosion that degrades training.

Negative Gradient Clipping

Section 4.3 addresses the credit assignment problem head-on. The paper quantifies a phenomenon that is unique to long-context RL: the high similarity between correct and incorrect responses. Table 3 shows that on DocMath (long-context numerical reasoning), the ROUGE-L between correct and incorrect model responses is 45.37, compared to 27.71 on AIME (short-context math). This means incorrect long-context responses share ~64% more phrase-level overlap with correct responses than short-context responses share.

Why this matters: in GRPO, all tokens in an incorrect response receive the same negative advantage $A_i < 0$. When incorrect responses contain substantial correct reasoning (as the high ROUGE-L indicates), the model is penalized for generating tokens that are actually part of a valid solution path β€” it's being punished for the right behavior embedded in the wrong overall answer. This "exacerbates the reward credit assignment problem" (as the paper phrases it) because the uniform advantage signal cannot distinguish between the one erroneous step that caused the failure and the many correct steps that preceded it.

The paper further observes (Figure 9) a strong correlation between token entropy and gradient norm in negative rollouts (Spearman's $\rho = 0.96, p < 0.0001$). High-entropy tokens β€” those where the model was uncertain β€” tend to produce large gradients, which "can increase the variance of parameter updates and destabilize optimization." Moreover, high-entropy tokens "often reflect exploratory behavior in the reasoning process" (Wang et al., 2025b), and "avoiding excessive penalization helps preserve the model's ability to explore and potentially correct originally erroneous paths."

The proposed solution: clip negative gradients based on entropy. The paper introduces an indicator function $\mathbb{I}(i, t)$ that selectively masks tokens or sequences from receiving negative gradient updates:

JGRPO(ΞΈ)=Ec,q∼D,{yi}i=1GβˆΌΟ€ΞΈold[1βˆ‘j=1G∣yjβˆ£βˆ‘i=1GAiβˆ‘t=1∣yi∣ρi,t(ΞΈ)I(t,i)]J_{\text{GRPO}}(\theta) = \mathbb{E}_{c,q \sim \mathcal{D}, \{y_i\}_{i=1}^G \sim \pi_{\theta_{\text{old}}}} \left[ \frac{1}{\sum_{j=1}^G |y_j|} \sum_{i=1}^G A_i \sum_{t=1}^{|y_i|} \rho_{i,t}(\theta) \mathbb{I}(t, i) \right]

where $\mathbb{I}(i, t)$ is defined as:

I(i,t)={0ifΒ Ai<0Β andΒ ((Ptoken_level∧H(t∣i)>Ο„token)∨(Β¬Ptoken_level∧HΛ‰(i)>Ο„sequence))1otherwise\mathbb{I}(i, t) = \begin{cases} 0 & \text{if } A_i < 0 \text{ and } \left( (P_{\text{token\_level}} \land H(t|i) > \tau_{\text{token}}) \lor (\lnot P_{\text{token\_level}} \land \bar{H}(i) > \tau_{\text{sequence}}) \right) \\ 1 & \text{otherwise} \end{cases}

where $H(t|i) = -\sum_{v \in \mathcal{V}} \pi_\theta(y_{i,t}|c, q, y_{i,<t}) \log \pi_\theta(y_{i,t}|c, q, y_{i,<t})$ is the token-level entropy at position $t$ in response $i$, and $\bar{H}(i) = \frac{1}{|y_i|}\sum_{t=1}^{|y_i|} H(t|i)$ is the sequence-level average entropy. $\tau_{\text{token}}$ and $\tau_{\text{sequence}}$ are threshold hyperparameters. $P_{\text{token\_level}}$ is a boolean selecting between token-level and sequence-level clipping.

What it computes: two variants of selective gradient masking:

  • Token-level clipping ($P_{\text{token\_level}} = \text{True}$): For responses with negative advantage, any individual token whose entropy $H(t|i)$ exceeds the threshold $\tau_{\text{token}}$ is excluded from the gradient update. Only low-entropy (high-confidence) tokens in incorrect responses are penalized. This preserves the model's ability to explore at uncertain decision points while still penalizing confident mistakes.

  • Sequence-level clipping ($P_{\text{token\_level}} = \text{False}$): For responses with negative advantage, if the entire sequence's average entropy $\bar{H}(i)$ exceeds $\tau_{\text{sequence}}$, the entire response is excluded from the gradient update. This prevents penalizing responses that were exploratory overall, even if they were incorrect.

Why this form: the key insight is that not all negative gradients are equally harmful. High-entropy tokens represent the model's uncertainty β€” places where it was "thinking" or exploring alternatives. Penalizing these tokens strongly (as vanilla GRPO does) discourages exploration and can cause the model to collapse to deterministic, low-entropy outputs that avoid difficult reasoning steps. By selectively masking high-entropy negative gradients, the model is still rewarded for correct responses (positive $A_i$ always pass through) and still penalized for confident mistakes (low-entropy tokens in incorrect responses), but it is not punished for uncertain exploration that happened to lead to an incorrect answer. This preserves the model's capacity for the type of extended reasoning that long-context tasks require.

Empirical validation (Table 4, Figure 10): On Qwen3-4B-Thinking, the results reveal important nuances:

  • Token-level clipping of high-entropy tokens yields the best performance (+0.95 over GRPO), while clipping low-entropy tokens is worse (+0.49) β€” confirming that penalizing high-confidence mistakes is important, and masking those penalties (by clipping low-entropy tokens) removes a necessary corrective signal
  • Sequence-level clipping of high-entropy sequences yields the best overall performance (+1.29 over GRPO at step 30), outperforming token-level approaches β€” suggesting that identifying and protecting entire exploratory trajectories is more effective than protecting individual tokens
  • However, sequence-level clipping can be taken too far: clipping low-entropy sequences (removing many more negative gradients) initially helps but later causes performance degradation (a drop between steps 30 and 40) β€” this is a clear case of "entropy collapse" where insufficient negative signal allows the policy to converge prematurely to suboptimal behavior
  • On MRCR specifically, clipping low-entropy tokens causes a 10-point drop compared to clipping high-entropy tokens (46.20 vs. 36.29), demonstrating that this benchmark is particularly sensitive to the exploration-exploitation tradeoff

Adaptive Entropy-Controlled Policy Optimization (AEPO)

Building on the findings from negative gradient clipping, the paper proposes AEPO as a dynamic, rather than static, entropy control mechanism. Instead of applying a fixed threshold, AEPO monitors the policy's entropy in real-time and switches between two modes:

The batch-level entropy is defined as:

H(πθ,B)=βˆ’1∣Bβˆ£βˆ‘i=1∣B∣1∣yiβˆ£βˆ‘t=1∣yiβˆ£βˆ‘v∈Vπθ(v∣c,q,yi,<t)log⁑πθ(v∣c,q,yi,<t)H(\pi_\theta, \mathcal{B}) = -\frac{1}{|\mathcal{B}|} \sum_{i=1}^{|\mathcal{B}|} \frac{1}{|y_i|} \sum_{t=1}^{|y_i|} \sum_{v \in \mathcal{V}} \pi_\theta(v|c, q, y_{i,<t}) \log \pi_\theta(v|c, q, y_{i,<t})

where $\mathcal{V}$ is the vocabulary space, $|\mathcal{B}|$ is the batch size, and the innermost sum is the standard entropy over the next-token distribution. This computes the average token-level entropy across all positions in all responses in the current batch.

What it computes: a single scalar measuring how "uncertain" or "exploratory" the policy is on the current batch. High entropy means the model is spreading probability mass across many tokens (exploring); low entropy means it's concentrating probability on few tokens (exploiting).

The AEPO switching rule: Two threshold values are established: a lower bound $H_{\text{low}}$ and an upper bound $H_{\text{high}}$. At each training step:

  • If $H(\pi_\theta, \mathcal{B}) > H_{\text{high}}$: The policy entropy is too high β€” it's exploring excessively and may be generating low-quality responses. AEPO masks ALL samples with negative advantages, updating the model exclusively using positive samples. This functions as "advantage-weighted online rejection sampling fine-tuning," effectively reducing the model's entropy by reinforcing only the successful behaviors.

  • If $H(\pi_\theta, \mathcal{B}) < H_{\text{low}}$: The policy entropy is too low β€” it's converging too quickly and may be trapped in a local optimum. Negative gradients are reintroduced to the optimization process, encouraging exploration and preventing entropy collapse.

  • If $H_{\text{low}} \leq H(\pi_\theta, \mathcal{B}) \leq H_{\text{high}}$: The policy is in the target entropy range. Both positive and negative gradients are used normally (standard GRPO objective).

What it computes, operationally: a dynamic gradient masking schedule that maintains the policy's entropy within a target band. When entropy drifts above the band, the policy is "cooled" by removing negative-advantage updates; when it dips below, it's "heated" by restoring them. The switching is automatic and data-dependent β€” no manual intervention is needed to tune the exploration-exploitation balance as training progresses.

Why this form: the key innovation over static negative gradient clipping is adaptivity. Static clipping uses fixed thresholds, which works well when the optimal entropy level is known and constant. But during long-context RL training, the optimal entropy level changes: early in training, higher entropy is desirable for exploration; later, lower entropy is needed for exploitation. AEPO automatically tracks these shifts by using the measured batch entropy as a real-time control signal. The paper frames this as maintaining "an optimal balance between exploration (with negative gradient) and exploitation (without negative gradient)" (Section 4.4), which is particularly important for "scaling RL training to a larger number of steps without degradation."

Empirical validation (Table 5, Figure 11): On Qwen3-4B-Thinking, AEPO yields +3.29 points over GRPO (from 56.07 to 59.36), a larger gain than any static clipping strategy. The improvements are broad: +1.25 on DocMath, +3.58 on LongBench-V2, +6.98 on MRCR, +5.94 on CorpusQA. Figure 11 demonstrates AEPO's behavior on the primary Qwen3-30B-A3B-Thinking model: the entropy oscillates in a controlled band as the algorithm switches between "with negative gradient" and "without negative gradient" phases, maintaining stability through 200 training steps without the divergence or collapse that would occur with vanilla GRPO.

Progressive Multi-Stage Training Paradigm

The paper's overall training pipeline is designed to gradually extend the model's reasoning capabilities to longer contexts without the instability that would result from abruptly switching from short-context to long-context training. The pipeline (Figure 6) proceeds through four full-context RL stages plus a parallel memory-RL stage, with increasing input and output lengths synchronized:

The need for progressive length extension: The paper observes that "abrupt transition from traditional short-input direct reasoning to the multi-hop grounding patterns required for long-context reasoning" causes training instability. This is because the reasoning patterns differ qualitatively β€” short-context reasoning can rely on local coherence and proximity-based attention, while long-context reasoning requires explicit search and aggregation strategies over widely separated evidence. Additionally, as input context length increases, "the reasoning content length exhibits a generally positive growth trend" β€” longer inputs require longer reasoning chains, so output length must also be extended.

Stage configuration:

  • Full-context RL Stage 1: Max input 32K tokens, max output 12K tokens
  • Full-context RL Stage 2: Max input 60K tokens, max output 20K tokens
  • Full-context RL Stage 3: Max input 120K tokens, max output 50K tokens

Between stages, the paper applies "difficulty-aware retrospective sampling" from QwenLong-L1: training data is filtered by difficulty using the input-output length settings of the subsequent stage. This means data that the model already handles well at the current stage is deprioritized, and data that becomes tractable (but challenging) at the next stage's length is emphasized.

Memory management integration via model merging: The paper found that "mixing memory management training data and single-pass full-context processing training data together causes considerable damage to the overall RL training infrastructure efficiency and model training stability." This destructive interference likely occurs because the memory agent requires learning sequential compression and planning behaviors (Equation 5) that are fundamentally different from single-pass attention patterns β€” training on both simultaneously creates conflicting gradient signals.

The solution is a "train specialized experts followed by model merging" paradigm:

  1. After Full-context RL Stage 3, a separate memory-RL training is conducted starting from the Stage 3 checkpoint, producing an expert model specialized in memory management. The memory-RL stage configuration: max input 128K tokens, chunk size 32K tokens, memory size 15K tokens.

  2. The SCE algorithm (Wan et al., 2024) is used to merge the full-context expert (Stage 3) and the memory expert into a single model. SCE (the paper doesn't detail the algorithm but cites Wan et al., 2024) presumably performs a weighted combination of model parameters that preserves capabilities from both experts.

  3. The merged model undergoes Full-context RL Stage 4 (max input 120K, max output 50K), which fine-tunes the unified model to maintain both capabilities without destructive interference.

The training details common to all stages: the RL framework is implemented using VeRL (Sheng et al., 2025). Generation uses sampling temperature 0.7 and top-p 0.95. The group size is $G = 8$. Training is strictly on-policy with batch size 128 and constant learning rate $2 \times 10^{-6}$. The reward mechanism combines rule-based verification (checking if ground truth is contained in the model output) with LLM-as-a-judge: gpt-oss-120b (OpenAI, 2025) is employed as an external judge for questions where rule-based verification fails.

Why multi-stage with model merging rather than joint training: the design reflects a practical constraint of RL training β€” when two capability types require fundamentally different optimization dynamics (full-attention vs. sequential memory compression), training them jointly can cause the gradients for one to interfere with the other. Training experts separately and merging them avoids this interference, and the final joint fine-tuning stage allows the model to learn to coordinate the two capabilities without the instability of learning both from scratch simultaneously. Table 10 validates this: after memory-RL, full-context performance drops from 71.59 to 68.53 (the cost of specialization), but model merging recovers it to 71.18 while improving memory agent performance from 20.34 to 21.68, and Stage 4 pushes both metrics higher (71.82 full-context, 22.53 memory agent).

Memory Agent Inference Architecture

While the mathematical formulation was covered earlier, the practical inference architecture has specific implementation details worth examining:

User query decomposition: Before any processing begins, the system separates the user's query into $q_{\text{core}}$ (the actual question to answer) and $q_{\text{inst}}$ (formatting instructions, e.g., "output as JSON" or "provide your answer in 3 paragraphs"). The formatting instructions are reserved for the final generation step; the core question guides all memory updates. This prevents format constraints from interfering with flexible reasoning during iterative memory updates β€” the model can think freely while reading documents, then apply format requirements only when generating the final answer.

Chunk processing with explicit planning: At each chunk $x_t$, the model receives four pieces of information (Section 2.3, Equation 5): the current chunk, the previous memory $m_{t-1}$, the previous plan $p_{t-1}$, and the core question $q_{\text{core}}$. It produces two outputs: an updated memory $m_t$ that integrates new information, and a navigational plan $p_t$ that guides attention for the next chunk. The plan is described as a mechanism to "guide the attention and information extraction for the subsequent chunk" β€” it is an explicit reasoning about what type of information to look for next, based on what has been seen so far and what the question requires.

Memory self-containment requirement: The prompt template (Appendix D) instructs the model: "Ensure the memory is self-contained – meaning the final memory alone, without additional context, should contain sufficient information to fully answer the problem." This forces the model to write complete, standalone facts into its memory rather than relying on pointers or references back to the original text. The memory must be sufficient for answering the question by itself after all chunks are processed.

Critical evaluation of prior memory: The prompt also states: "Recognize that both the previous memory and previous plan may contain inaccuracies or be suboptimal for the current task. Therefore, pay close attention to the problem, critically evaluate the existing memory and plan, and make appropriate updates and adjustments." This is important because it means the model can correct earlier mistakes β€” if it extracted incorrect information from an earlier chunk that is contradicted by new information, it can update the memory accordingly. This makes the memory agent robust to extraction errors.

One-way processing constraint: "This is a strictly one-way reading process. You cannot go back to re-read previous sections." This constraint is both a practical necessity (re-reading would defeat the purpose of the memory architecture) and a deliberate difficulty amplifier β€” the model must extract and compress information correctly on the first pass, just as a human reader processing a very long document without the ability to flip back to earlier sections.

Final answer with formatting: After the final chunk, the accumulated memory $m_K$, the core question, and the formatting instructions are combined to produce the answer (Equation 6). The formatting instructions are only applied at this point, ensuring the output structure is correct without constraining the reasoning process.

RL training of the memory agent: Each trajectory consists of all memory updates, plans, and the final answer. The trajectory-level reward is based on answer correctness (rule-based or LLM-as-judge). The trajectory-level advantage is broadcast uniformly to all actions β€” meaning every memory update and plan in a correct trajectory receives a positive advantage, and every action in an incorrect trajectory receives a negative advantage. The policy is updated end-to-end using the same simplified GRPO objective (Equation 4). This coarse credit assignment is a known limitation (discussed in Section 7.2), and the paper identifies developing "token-level credit assignment within the thinking trajectory" as a key future direction.

4. Key Insights and Innovations

Innovation 1: Long-Context Reasoning as a Distinct Capability Requiring Deliberate Post-Training Cultivation

The paper's most fundamental conceptual move is reframing long-context reasoning from an architectural property (does the model have a long context window?) into a learned capability (has the model been taught to use that window effectively?). This distinction may seem subtle, but it overturns a widespread implicit assumption in the field: that extending a model's context window during pre-training or mid-training is sufficient to enable long-context reasoning. The evidence that this assumption is wrong is stark: Qwen3-30B-A3B-Thinking-2507 already supports 256K tokens at the architectural level, yet scores 51.27 on MRCR and 61.92 average across six benchmarks β€” compared to QwenLong-L1.5's 82.99 and 71.82 after post-training (Table 7). The ~32-point gap on MRCR cannot be explained by model size, architecture, or context capacity; it reflects a genuine capability deficit in using the context that exists.

This reframing has implications that extend beyond this specific model. It suggests that the field's intense focus on context window extension (Yang et al., 2025a; Team et al., 2025a) has been necessary but insufficient β€” and that the next frontier is not pushing context lengths further but developing the training recipes that make existing context windows usable for reasoning. The paper demonstrates this by showing that the benefits of post-training are not uniform across task types: the largest gains come on benchmarks requiring multi-hop grounding and global information aggregation (MRCR: +31.72, CorpusQA: +9.69, LongBench-V2: +6.16), while simpler tasks like LBV1-QA show modest gains (+3.30). This pattern (Table 7) confirms that what the post-training teaches is not generic "long-context processing" but specifically the ability to integrate and reason over distributed evidence β€” precisely the capability that architectural context extension alone does not provide.

This is a fundamental reframing rather than an incremental improvement. Prior work treated long-context reasoning as a scaling problem β€” make the window bigger, and reasoning will follow. This paper argues it is a training problem β€” the model must be taught to reason over long contexts through carefully designed data and optimization strategies. The distinction is analogous to the difference between having a large memory (hardware) and knowing how to use memory effectively in problem-solving (skill). The paper's contribution is demonstrating that the skill can be systematically cultivated and that doing so produces gains that architectural scaling alone cannot.

Innovation 2: Diagnosing and Treating the Credit Assignment Crisis in Long-Context RL

The paper makes a specific diagnostic contribution that has broader significance for RL-based LLM training: it identifies, quantifies, and partially resolves a failure mode that is unique to long-context reasoning and that standard GRPO is structurally incapable of handling. This is the credit assignment crisis β€” the observation that when correct and incorrect reasoning paths share substantial overlap (because both are grounded in the same long context), GRPO's uniform per-sequence advantage signal becomes actively harmful, penalizing correct reasoning steps embedded in incorrect trajectories.

This is not merely a "GRPO has high variance" observation. It is a specific, measurable phenomenon with a clear mechanism: Table 3 quantifies that ROUGE-L between correct and incorrect responses is 45.37 on DocMath versus 27.71 on AIME β€” a 64% increase in overlap. The implication is that on long-context tasks, an incorrect response is not wrong throughout but rather mostly correct with a critical error. GRPO's uniform negative advantage treats every token in that response as equally blameworthy, which means tokens that are part of a valid reasoning path receive the same penalty as the erroneous token. This is a structural mismatch between the algorithm's credit assignment granularity (per-sequence) and the problem's error structure (per-step or per-token).

The paper's diagnostic sophistication lies in connecting this structural problem to observable training dynamics. Figure 9 demonstrates that high-entropy tokens β€” where the model was uncertain β€” produce the largest gradient norms in negative rollouts (Spearman's ρ = 0.96). The interpretation (Section 4.3) is that these high-entropy tokens represent exploratory reasoning β€” the model was trying something, it turned out to be wrong (or was part of a trajectory that ultimately failed), and GRPO penalizes this exploration heavily. Over time, this discourages the very exploration that long-context reasoning requires, leading to entropy collapse and training instability.

This diagnosis is conceptually novel in the RL-for-LLMs literature. Prior work on training instability in GRPO (Liu et al., 2025a; Hu, 2025) focused on statistical issues β€” biased advantage estimation from small group sizes, reward normalization across heterogeneous tasks. This paper adds a structural dimension: even with perfect advantage estimation, the per-sequence granularity of GRPO is fundamentally mismatched to the per-step error structure of long-context reasoning. This insight generalizes beyond this specific paper β€” any domain where correct and incorrect outputs share substantial structure (code generation with subtle bugs, multi-step planning with execution errors, legal reasoning with interpretive disagreements) may face the same credit assignment challenge.

The treatment β€” negative gradient clipping and AEPO β€” is pragmatic rather than fundamental (Section 7.2 acknowledges this explicitly: AEPO "serves as a robust stabilization mechanism rather than a fundamental solution to the credit assignment problem"). But the diagnostic framework itself is the innovation: it tells the field why long-context RL is harder than short-context RL in a precise, measurable way, and it points toward the type of solution that would be needed (token-level credit assignment, as the paper identifies for future work). This is a foundational diagnostic contribution that clarifies the problem even if the proposed solution is partial.

Innovation 3: Entropy as a Real-Time Control Signal for Policy Optimization Stability

The paper introduces Adaptive Entropy-Controlled Policy Optimization (AEPO), which is innovative not primarily as an algorithm but as a control philosophy for RL training. The standard paradigm in RL-for-LLMs is to set hyperparameters (learning rate, clipping thresholds, KL penalty coefficients) before training and let them remain fixed, with perhaps a learning rate schedule. AEPO instead treats the policy's entropy as a real-time diagnostic that determines which samples contribute gradients at each step, creating a feedback loop between the policy's current state and the optimization procedure.

This is conceptually distinct from both standard GRPO and from static gradient clipping approaches. Standard GRPO applies the same update rule uniformly regardless of the policy's current entropy. Static clipping (Section 4.3) uses fixed thresholds that are set before training. AEPO instead monitors the batch-level entropy (Equation 11) and switches between two qualitatively different optimization modes: when entropy is too high, it performs something akin to rejection sampling fine-tuning (updating only on positive examples); when entropy is too low, it restores full GRPO updates (including negative gradients) to encourage exploration. The switching is fully automatic and data-dependent β€” it requires no manual schedule or intervention.

The significance of this approach extends beyond long-context RL. It demonstrates that policy entropy β€” a quantity that is trivially computable during training (it is already calculated for the policy's forward pass) β€” can serve as a reliable signal for modulating the optimization procedure itself. This is a form of adaptive regularization that is more responsive than fixed KL penalties and more principled than ad hoc gradient clipping. The paper shows (Figure 11) that AEPO maintains entropy within a stable band through 200 training steps on the 30B model, where vanilla GRPO would typically show either divergence or collapse. The oscillation between "with negative gradient" and "without negative gradient" phases is not a bug but a feature β€” it reflects the algorithm actively maintaining the exploration-exploitation balance.

This is an incremental but conceptually important advance in RL training methodology. The individual components (entropy monitoring, gradient masking) are not individually novel, but the integration into a closed-loop control system for policy optimization represents a shift in how to think about training stability. Rather than trying to find the perfect static hyperparameters, AEPO suggests that training procedures should respond to the policy's state β€” and that entropy is a particularly informative state variable for reasoning tasks where the exploration-exploitation tradeoff shifts as capabilities develop.

Innovation 4: The Destructive Interference Between Full-Context Reasoning and Memory Management β€” and Its Resolution via Expert Merging

The paper makes an important negative finding with constructive implications: that training a model to perform both single-pass full-context reasoning and iterative memory-based processing simultaneously causes destructive interference, and that this interference can be resolved through a specific training architecture (train experts separately, merge via SCE, then jointly fine-tune). This finding is significant because it identifies a non-obvious tension between two capabilities that might naively be assumed to be complementary β€” both involve processing long documents, so why wouldn't training on both help?

The evidence for destructive interference is in Table 10. After Full-context RL Stage 3, the model achieves 71.59 average full-context performance and 12.66 on memory-agent MRCR (512K–1M). After Memory-RL training, memory-agent performance jumps to 20.34 (a substantial gain of 7.68 points) but full-context performance drops to 68.53 (a loss of 3.06 points). This is classic catastrophic interference: the memory-agent training, which teaches sequential chunk processing and compression, partially overwrites the full-attention reasoning patterns learned in previous stages. The fact that the interference is bidirectional (memory training hurts full-context performance, and the paper notes that "mixing memory management training data and single-pass full-context processing training data together causes considerable damage" in the other direction as well) confirms that these are genuinely competing optimization objectives, not simply different applications of the same underlying skill.

The resolution β€” expert merging via SCE followed by joint fine-tuning β€” is elegant not because SCE is novel (the algorithm is cited from Wan et al., 2024) but because it demonstrates that these capabilities can coexist in a single model when the training procedure respects their tension rather than ignoring it. After merging, full-context performance recovers to 71.18 (essentially back to Stage 3 levels) while memory-agent performance increases further to 21.68. Stage 4 joint fine-tuning pushes both higher: 71.82 full-context, 22.53 memory-agent. The merged-and-fine-tuned model outperforms any single-expert model on both dimensions, confirming that the capabilities are ultimately complementary β€” but only when the training architecture explicitly manages their interference.

This finding is fundamental for the design of multi-capability LLM training pipelines. It suggests that when different capabilities require qualitatively different information processing patterns (global attention vs. sequential compression, in this case), naive multi-task training is likely to produce interference rather than synergy. The expert-merging paradigm β€” train specialized models separately, then merge and fine-tune β€” provides a general architectural pattern for resolving such tensions. The paper demonstrates this pattern works for the specific case of full-context + memory, but the principle may generalize to other capability combinations that exhibit destructive interference when trained jointly.

5. Experimental Analysis

Evaluation Methodology

  • Dataset. The paper evaluates on six long-context reasoning benchmarks spanning three categories. For multiple-choice: LongBench-V2 (Bai et al., 2025), 503 questions assessing deep comprehension across single-document QA, multi-document QA, long in-context learning, long-dialogue history understanding, code repository understanding, and long structured data understanding. For needle-in-a-haystack (NIAH): MRCR (Vodrahalli et al., 2024), which tests the model's ability to find and disambiguate between multiple needles hidden in multi-turn synthetic conversations. For multi-hop QA: Frames (Krishna et al., 2025), 824 questions on diverse Wikipedia topics; five subsets from LongBench-V1-QA (Bai et al., 2024): 2WikiMultihopQA, HotpotQA, Musique, NarrativeQA, Qasper; DocMath (Zhao et al., 2024), numerical reasoning within financial reports, using the test-mini subset of 800 queries; and CorpusQA (Tongyi-Zhiwen Team, 2025), global reasoning over evidence dispersed throughout large document corpora. For ultra-long context evaluation beyond 128K tokens, subsets of MRCR and CorpusQA with contexts spanning 128K–512K, 512K–1M, and 1M–4M tokens are used.

  • Base model(s). The primary model is Qwen3-30B-A3B-Thinking-2507 (Yang et al., 2025a), a 30B-parameter Mixture-of-Experts architecture with a 256K-token context window. The paper argues this model is "representative of the capabilities of many contemporary LLMs" and sits in a useful regime: it already has architectural support for long contexts (from pre-training/mid-training) but has not been specifically post-trained for long-context reasoning, making it an ideal testbed for demonstrating that post-training adds capability beyond what architecture alone provides. For RL ablation experiments (Sections 4.2–4.4), a smaller Qwen3-4B-Thinking-2507 model is used to enable faster experimental iteration and more thorough hyperparameter exploration before scaling up to the 30B model.

  • Metrics. The primary metric is accuracy, computed differently per task type. For multiple-choice (LongBench-V2), standard exact-match accuracy is reported. For MRCR, the SequenceMatcher ratio from Python's difflib is used. For multi-hop QA tasks (Frames, LongBench-V1-QA, DocMath, CorpusQA), the maximum of Cover Exact Match (CEM) score (Song et al., 2025) and LLM-as-a-judge evaluation (Zheng et al., 2023) is reported, where DeepSeek-V3 (Liu et al., 2024) evaluates semantic equivalence between the model's prediction and the ground-truth answer using the prompt template in Table 6. All experiments use a sampling temperature of 0.7 and top-p of 0.95.

  • Baselines. The paper compares against multiple categories of models. Flagship reasoning models: Gemini-2.5-Pro, GPT-5, DeepSeek-R1-0528, Qwen3-235B-A22B-Thinking-2507, Qwen3-Max-Thinking-Preview. Lightweight reasoning models: Gemini-2.5-Flash-Thinking, GPT-5-Nano, GPT-OSS-120B, QwenLong-L1. The direct baseline is Qwen3-30B-A3B-Thinking-2507 β€” the model QwenLong-L1.5 is built upon. For the memory agent experiments, additional baselines include MemAgent-14B (Yu et al., 2025a) and full-context inference models like Qwen-Flash-Thinking-1M. For the RL ablation studies, the baseline is naive GRPO applied to the respective base model.

  • Generation budget / compute accounting. For full-context RL training, compute is measured by the maximum input and output token lengths at each stage: Stage 1 (32K input, 12K output), Stage 2 (60K input, 20K output), Stage 3 (120K input, 50K output), Stage 4 (120K input, 50K output). For memory-RL training: max input 128K tokens, chunk size 32K tokens, memory size 15K tokens. Training uses a group size of G = 8, batch size of 128, constant learning rate of 2 Γ— 10⁻⁢, sampling temperature 0.7, top-p 0.95, and a single gradient update per batch in a strictly on-policy setting. For evaluation, all models are evaluated with maximum input lengths of 128K tokens and maximum generation length of 50K tokens. For prompts exceeding the context window, middle truncation from Bai et al. (2024) is used to preserve the front and tail portions.

  • Cross-validation / statistical protocol. The paper does not employ formal cross-validation for its main results, as the evaluation is on fixed benchmark test sets rather than hyperparameter selection. The RL training itself uses an on-policy paradigm where each batch of samples is used for only a single gradient update, which naturally prevents overfitting to specific training batches. For the ablation studies on the 4B model, results are reported at specific training steps (e.g., step 30, step 40 in Table 4) to provide snapshots of training dynamics. The paper does not report confidence intervals or statistical significance tests for benchmark comparisons, which is a limitation β€” the 500-question test sets for some benchmarks mean that differences of a few points may not be statistically robust, though the magnitude of reported gains (9.90 points average) makes this concern less acute for the headline results.

Main Quantitative Results

Overall Long-Context Reasoning Performance

The headline result appears in Table 7 and Figure 1: QwenLong-L1.5-30B-A3B achieves an average score of 71.82 across six long-context reasoning benchmarks, representing a +9.90 point improvement over its direct baseline, Qwen3-30B-A3B-Thinking-2507 (61.92). This average score places QwenLong-L1.5 competitive with flagship proprietary models β€” it surpasses DeepSeek-R1-0528 (68.67), Gemini-2.5-Flash-Thinking (68.73), and Qwen3-Max-Thinking-Preview (69.43), and approaches Gemini-2.5-Pro (72.40) and GPT-5 (74.74).

The performance gains are not uniform across benchmarks; they concentrate on tasks requiring multi-hop reasoning and global information aggregation:

  • MRCR: The largest absolute gain, from 51.27 to 82.99 (+31.72 points). This is a needle-in-a-haystack benchmark with multiple needles in synthetic conversations, where the baseline model performs near chance while QwenLong-L1.5 achieves state-of-the-art performance (82.99 vs. 79.92 for Gemini-2.5-Pro and 77.29 for GPT-5). The paper notes that MRCR has an average input length of 36.5K tokens, and that this benchmark's dense reward structure made it particularly responsive to the task-specific advantage estimation method (Section 4.2).

  • CorpusQA: From 71.56 to 81.25 (+9.69 points), reaching performance competitive with GPT-5 (81.56). This benchmark requires aggregating scattered information from the context for calculation or synthesis, with an average input length of 92.8K tokens β€” the longest average context among the evaluation benchmarks. The gain demonstrates that the data synthesis pipeline's emphasis on cross-document information integration successfully transferred to evaluation performance.

  • LongBench-V2: From 49.11 to 55.27 (+6.16 points). The detailed breakdown in Table 11 reveals that the largest sub-category gain is on the Medium length subset (+10.23 points, corresponding to the 32K–128K token range where the training data was concentrated), while Short (+3.34) and Long (+2.87) subsets show more modest improvements. On difficulty, the gain is slightly larger on Easy (+7.30) than Hard (+5.49), suggesting the model improved more on problems within its reach than on fundamentally intractable ones.

  • Frames: From 70.27 to 74.76 (+4.49 points), approaching Gemini-2.5-Pro (74.51).

  • DocMath: From 62.26 to 66.26 (+4.00 points), a moderate improvement on numerical reasoning within financial reports.

  • LongBench-V1-QA: From 67.10 to 70.40 (+3.30 points), the smallest average gain. However, the subset breakdown in Table 12 reveals substantial heterogeneity: Musique (+7.00) and NarrativeQA (+9.00) show strong gains, while Qasper shows a decline (-3.00). The paper notes that NarrativeQA has one of the longest contexts in the LongBench-V1 suite, and Musique is specifically designed for multi-hop reasoning β€” both align with the training data's emphasis. The Qasper decline is not explained.

The correlation between performance gains and context length is explicitly noted: the largest improvements occur on benchmarks with the longest average contexts (MRCR: 36.5K tokens, +31.72; CorpusQA: 92.8K tokens, +9.69; LongBench-V2: 85.5K tokens, +6.16), while benchmarks with shorter or more varied contexts show smaller gains. This pattern supports the paper's claim that the methodology is "particularly effective at addressing the challenges inherent to reasoning over extended context lengths."

Comparison with QwenLong-L1 (the predecessor) shows the scale of improvement: QwenLong-L1 achieves 56.11 average, while QwenLong-L1.5 achieves 71.82 β€” a 15.71-point gain from the predecessor, compared to 9.90 from the base model. The largest gaps between L1 and L1.5 are on MRCR (47.86 vs. 82.99, +35.13) and CorpusQA (42.50 vs. 81.25, +38.75), reflecting the impact of the expanded data synthesis pipeline and stabilized RL training that were absent in the L1 release.

Generalization to Out-of-Domain Tasks

Table 8 reports performance on benchmarks outside the primary training distribution: general capabilities (MMLU-PRO, AIME24/25, GPQA-Diamond), agentic memory (BFCL-V4), and dialogue memory (LongMemEval).

No catastrophic forgetting: On MMLU-PRO, QwenLong-L1.5 scores 81.33 compared to the baseline's 81.03 (+0.30, essentially unchanged). On AIME24, it scores 90.0 vs. 90.31 (-0.31, within noise). This is significant because these benchmarks occupy a distinctly different semantic space from the long-context training data, as visualized in Figure 12's UMAP projection.

Positive transfer to reasoning-intensive tasks: On AIME25, performance improves from 82.81 to 86.46 (+3.65) β€” a meaningful gain on a challenging math competition benchmark. On GPQA-Diamond, from 75.88 to 76.78 (+0.90). The paper attributes this to transfer of "information integration skills from long-context training" to the model's own long-form reasoning outputs: AIME problems require maintaining coherence and integrating intermediate results across extended chains of reasoning, which is structurally similar to the multi-hop grounding patterns learned during long-context training. A case study in Appendix C.1 illustrates this concretely: on an AIME 2025 problem, the baseline model becomes "entrenched in a qualitative calculus approach" and resorts to heuristic approximation, while QwenLong-L1.5 identifies a logical contradiction, revises its hypothesis, and shifts to algebraic synthesis β€” demonstrating self-correction and strategic adaptation that the paper links to long-context reasoning skills.

Agentic memory improvement: On BFCL-V4, QwenLong-L1.5 shows gains across most sub-tasks, with the largest on Memory-KV (+5.80, from 10.97 to 16.77). This task involves maintaining and querying key-value structured memory over extended agent interactions β€” a direct application of the information-tracking skills developed through long-context training. Memory-Sum improves by +1.51, Memory-Vec by +0.64, while Memory-Rec_Sum shows a small decline (-1.94).

Dialogue memory: The most dramatic generalization result is on LongMemEval, where QwenLong-L1.5 scores 76.40 compared to the baseline's 60.80 (+15.60). This benchmark tests the ability to maintain state and recall information over extended conversations, which is a primary application domain for long-context models. The paper frames this as "directly confirming the success of our methodology in its intended domain" β€” dialogue memory was included as a training data category (Table 1 lists "Dialogue data" among sources and "Dialogue memory" among question types), so this result is both a validation of the training and evidence that the acquired skills transfer within-domain.

Ultra-Long Context Performance with Memory Agent

Table 9 reports performance on MRCR and CorpusQA subsets with contexts exceeding 128K tokens, where full-context inference becomes either computationally infeasible or architecturally impossible (the base model supports 256K tokens, but the paper evaluates memory-agent performance as a distinct capability).

On MRCR (128K–512K): QwenLong-L1.5 operating in the memory agent framework achieves 34.87, compared to 16.55 for the baseline Qwen3-30B-A3B-Thinking-2507 in the same framework β€” an +18.32 point gain. For context, the strongest full-context model (Gemini-2.5-Pro) achieves 53.83, and Gemini-2.5-Flash-Thinking achieves 53.98. The memory agent underperforms full-context inference at this scale, which is expected β€” the memory agent's advantage is in scaling to contexts where full-context is impossible, not in outperforming full-context on shorter sequences.

On MRCR (512K–1M): QwenLong-L1.5 achieves 22.53 vs. baseline 4.24 (+18.29), compared to Gemini-2.5-Pro's 39.51 (full-context) and Gemini-2.5-Flash-Thinking's 46.88 (full-context). The performance gap between QwenLong-L1.5's memory agent and full-context models narrows slightly at this scale, and the advantage over the baseline agent remains consistent (~18 points). Among agent-based methods, QwenLong-L1.5 substantially outperforms MemAgent-14B (which achieves 3.11 at this scale).

On CorpusQA (1M tokens): QwenLong-L1.5 achieves 20.72 vs. baseline 15.32 (+5.40). Gemini-2.5-Pro achieves 53.11 with full-context inference.

On CorpusQA (4M tokens): This is the regime where full-context inference is "intractable for current full-context methods" β€” the paper states that models like Gemini-2.5-Pro cannot process this length. QwenLong-L1.5 achieves 14.29 compared to baseline 9.52 (+4.77) and MemAgent-14B's 9.09. This demonstrates the memory agent framework's unique capability: it can perform reasoning at scales where full-attention models simply cannot operate.

The paper notes that the memory agent framework does not yet match the performance of the best full-context models on shorter ultra-long sequences, but emphasizes that "our approach has favorable context scaling properties" β€” performance degrades slowly as context length increases, and the framework extends farther than any full-context alternative. The paper positions this as a "robust foundation" for future optimization.

Performance Dynamics Across Training Stages

Table 10 provides a stage-by-stage ablation of the multi-stage training pipeline, revealing how different capabilities develop through the process.

Initial activation of long-context reasoning: After just Full-context RL Stage 1 (32K tokens max input), the average score jumps from 61.92 (baseline) to 69.59 β€” a +7.67 point gain that represents the majority of the total improvement. The paper interprets this as evidence that "the synthetic data used in Stage-1 activates the model's foundational ability to ground and reason over long documents." This is notable because Stage 1 uses the shortest context lengths; the large gain suggests that even moderate-length training with challenging multi-hop data can unlock capabilities that transfer to longer evaluation contexts.

Divergent development of different capabilities: The stage-by-stage patterns reveal that different benchmarks respond to different training stages:

  • Stable benchmarks (DocMath, Frames, LBV1-QA): Performance stabilizes after Stage 1 and remains relatively flat through subsequent stages. DocMath: 62.26 β†’ 65.88 β†’ 64.38 β†’ 66.25 β†’ slightly oscillating. Frames: 70.27 β†’ 73.79 β†’ 74.76 β†’ 74.27 β†’ 74.76. LBV1-QA: 67.10 β†’ 70.20 β†’ 70.90 β†’ 69.30 β†’ 70.40. These benchmarks have shorter average input lengths and lower information density; once the model learns basic multi-hop grounding (Stage 1), further extension of training context length doesn't substantially improve performance.

  • Progressive benchmarks (MRCR, CorpusQA): Performance continues to improve through Stage 3 as training context length increases. MRCR: 51.27 β†’ 76.35 β†’ 81.53 β†’ 82.69. CorpusQA: 71.56 β†’ 75.62 β†’ 77.50 β†’ 79.38. These benchmarks have longer contexts and require global information aggregation; the progressive length extension is essential for developing the full-context reasoning skills they demand. The paper emphasizes this distinction: "for tasks characterized by longer contexts and the need for global information aggregation, performance consistently improves as training progresses from Stage 1 to 3."

The memory specialization tradeoff: After Memory-RL training, full-context average performance drops from Stage 3's 71.59 to 68.53 β€” a -3.06 point degradation. Simultaneously, memory-agent performance on MRCR (512K–1M) jumps from 12.66 to 20.34 (+7.68). This is the clearest evidence of the destructive interference the paper describes in Section 4.1: specializing in memory management partially overwrites full-context reasoning capabilities.

Resolution through model merging: The SCE merging step recovers full-context performance to 71.18 (+2.65 from the post-memory-RL low) while further improving memory-agent performance to 21.68 (+1.34). Stage 4 full-context RL pushes both higher: 71.82 full-context (+0.64) and 22.53 memory-agent (+0.85). The fact that Stage 4 full-context training does not degrade memory-agent performance β€” it actually improves it β€” is described as "strong robustness" of the acquired memory skill. This is a non-obvious and important finding: once the memory capability is properly integrated via merging (as opposed to learned simultaneously, which causes interference), subsequent full-context training is complementary rather than destructive.

Comparison with naive GRPO: The baseline trained with naive GRPO achieves 67.24 average, compared to QwenLong-L1.5's 71.82 β€” a +4.58 point advantage for the full pipeline. This validates the paper's claim that the RL innovations (task-balanced sampling, task-specific advantage estimation, AEPO) contribute meaningfully beyond what simpler RL approaches achieve.

Ablation Studies and Robustness Checks

Task-Balanced Sampling and Task-Specific Advantage Estimation (Table 2, Figure 8): On Qwen3-4B-Thinking-2507, naive GRPO achieves 56.07 average. Adding task-balanced sampling alone yields 56.86 (+0.79), with improved entropy stability but uncontrolled response length growth (Figure 8c). Adding batch-level standard deviation normalization yields 57.45 (+1.38). The full combination β€” task-balanced sampling with task-specific (per-task-batch) advantage estimation β€” achieves 58.62 (+2.55), with stabilized entropy (Figure 8b) and controlled response length (Figure 8c). The improvement is concentrated on MRCR (+12.33 over GRPO baseline), which the paper attributes to MRCR's dense reward structure benefiting most from isolated task-specific normalization.

Negative Gradient Clipping Strategies (Table 4, Figure 10): On Qwen3-4B-Thinking-2507, comparing token-level vs. sequence-level clipping, and high-entropy vs. low-entropy targets:

  • Token-level, clip high-entropy tokens: 57.02 (+0.95 over GRPO), the best token-level variant
  • Token-level, clip low-entropy tokens: 55.56 (-0.51), worse than baseline β€” confirming that penalizing confident mistakes is necessary
  • Sequence-level, clip high-entropy sequences: 57.36 (+1.29), the best overall static clipping performance
  • Sequence-level, clip low-entropy sequences (step 30): 56.66 (+0.59), but degrades to 55.47 (-0.60) by step 40 β€” demonstrating entropy collapse from removing too many negative signals

The MRCR results are particularly revealing: token-level clipping of low-entropy tokens drops MRCR to 36.29 (-4.61 from GRPO), while clipping high-entropy tokens achieves 46.20 (+5.30). The 10-point gap between these strategies on a single benchmark highlights how sensitive long-context RL is to the exploration-exploitation tradeoff.

AEPO vs. Static Clipping (Table 5, Figure 11): On Qwen3-4B-Thinking-2507, AEPO achieves 59.36 (+3.29 over GRPO), outperforming all static clipping variants. The gains are distributed: DocMath +1.25, LongBench-V2 +3.58, MRCR +6.98, CorpusQA +5.94. Figure 11 demonstrates AEPO's dynamic behavior on the 30B model: entropy oscillates within a controlled band as training alternates between "with negative gradient" and "without negative gradient" phases, maintaining stability through 200 steps without manual intervention.

Multi-Stage Training Progression (Table 10): The full pipeline (71.82) outperforms naive GRPO (67.24) by +4.58 points. The majority of improvement comes from Stage 1 (+7.67 over baseline), with Stages 2 (+0.87) and 3 (+1.13) providing incremental gains concentrated on information-intensive benchmarks (MRCR, CorpusQA). The memory-RL β†’ SCE merging β†’ Stage 4 sequence demonstrates both the destructive interference problem (full-context drops to 68.53 after memory-RL) and its resolution (recovers to 71.18 after merging, improves to 71.82 after Stage 4). Memory-agent performance follows an upward trajectory throughout: 4.24 (baseline) β†’ 17.14 (Stage 1) β†’ 17.05 (Stage 2) β†’ 12.66 (Stage 3) β†’ 20.34 (Memory-RL) β†’ 21.68 (Merged) β†’ 22.53 (Stage 4). The dip at Stage 3 suggests that full-context training at 120K may partially interfere with memory-agent capabilities, though the paper doesn't discuss this reversal.

Generalization Robustness (Table 8): The model does not suffer catastrophic forgetting on out-of-distribution tasks: MMLU-PRO (+0.30), AIME24 (-0.31). It shows positive transfer to reasoning-intensive tasks (AIME25 +3.65, GPQA-Diamond +0.90) and to applied memory tasks (BFCL-V4 Memory-KV +5.80, LongMemEval +15.60). The negative result on BFCL-V4 Memory-Rec_Sum (-1.94) and LongBench-V1-QA Qasper subset (-3.00, Table 12) suggests not all long-context adjacent tasks benefit equally, though the paper doesn't analyze these failures in detail.

Full-Context vs. Memory Agent at Scale (Table 9): The memory agent framework's performance advantage over the baseline agent is consistent across context lengths: +18.32 at 128K–512K, +18.29 at 512K–1M, +5.40 at 1M, +4.77 at 4M. The decreasing absolute gap at extreme lengths (4M tokens: 14.29 vs. 9.52) suggests both models approach a performance floor, though QwenLong-L1.5 maintains a meaningful relative advantage. The framework cannot yet match full-context inference by proprietary models at scales where full-context is feasible (Gemini-2.5-Pro achieves 53.83 vs. QwenLong-L1.5's 34.87 on MRCR 128K–512K), but extends to scales where full-context is impossible (4M tokens).

Critical Assessment

Claim: QwenLong-L1.5 achieves performance comparable to flagship models like GPT-5 and Gemini-2.5-Pro.

The evidence for this claim is nuanced. On the average across six benchmarks, QwenLong-L1.5 (71.82) is indeed competitive with Gemini-2.5-Pro (72.40) and trails GPT-5 (74.74) by a modest margin. On individual benchmarks, it achieves state-of-the-art on MRCR (82.99 vs. 79.92 for Gemini-2.5-Pro) and competitive performance on CorpusQA (81.25 vs. 81.56 for GPT-5). However, this claim requires qualification on two fronts:

First, the benchmark suite is selective. It includes six benchmarks chosen by the authors, all of which align with the capabilities their training targets (multi-hop reasoning, information aggregation). A broader evaluation including benchmarks that test other aspects of long-context reasoning (e.g., temporal reasoning, causal analysis, long-form generation) might reveal different relative standings. The paper does evaluate some additional domains (Table 8: MMLU-PRO, AIME, GPQA-Diamond, BFCL-V4, LongMemEval), but these are presented as "generalization" rather than head-to-head comparison with flagship models β€” the flagship model scores on these benchmarks are not reported, so we cannot assess whether QwenLong-L1.5 is "comparable" on these axes.

Second, the comparison with flagship models is at evaluation time only. The flagship models (GPT-5, Gemini-2.5-Pro) were trained with substantially more compute, on more data, and likely with their own long-context post-training pipelines that are not public. QwenLong-L1.5 demonstrates that a 30B open-weight model can reach competitive performance, which is impressive, but the claim of "comparability" should be understood as "comparable on these specific benchmarks under these specific evaluation conditions," not as a general claim of equivalence across all long-context capabilities.

Claim: The post-training recipe yields a 9.90-point average gain over the baseline.

This claim is strongly supported by Table 7, which shows gains on every benchmark (range: +3.30 to +31.72). The gains are concentrated on benchmarks requiring the specific capabilities the training targets (multi-hop reasoning, global information aggregation), which is exactly what one would expect from a targeted intervention. The pattern of larger gains on longer-context benchmarks (MRCR +31.72, CorpusQA +9.69, LongBench-V2 +6.16) and smaller gains on shorter-context benchmarks (DocMath +4.00, LBV1-QA +3.30) provides internal consistency β€” the training is most effective where the training data's characteristics (long contexts, distributed evidence) most closely match the evaluation.

However, the baseline comparison has a limitation: the baseline model (Qwen3-30B-A3B-Thinking-2507) is evaluated with the same generation parameters (temperature 0.7, top-p 0.95, max 50K output tokens) but without any of the RL training. A fairer baseline might include the baseline model with best-of-N sampling or majority voting at equivalent compute budgets, which could close some of the gap without the specialized training. The paper does compare against naive GRPO (Table 10: 67.24 vs. 71.82), showing that the full pipeline adds +4.58 beyond what standard RL achieves, but this comparison is only reported in the stage-by-stage ablation and not in the main results table.

Claim: The data synthesis pipeline creates challenging training data that drives multi-hop reasoning improvements.

This claim is supported by two pieces of evidence. First, the performance pattern in Table 7 shows the largest gains on benchmarks requiring multi-hop grounding (MRCR, CorpusQA, LongBench-V2), which aligns with the data pipeline's stated objectives. Second, the stage-by-stage analysis in Table 10 shows that Stage 1 (which uses the synthetic data at 32K tokens) activates substantial long-context capabilities (+7.67 over baseline), suggesting the data is effective even at moderate lengths.

What is missing is a direct ablation comparing the synthetic data pipeline against alternative data sources. The paper does not report results using only open-source data (as QwenLong-L1 used), using only one of the three synthesis methods, or using simpler data generation strategies (e.g., random chunking without the KG-based multi-hop structure). Without such ablations, we cannot isolate the contribution of the data synthesis methodology from the contribution of simply having more data or longer data. The comparison with QwenLong-L1 (Table 7: 56.11 vs. 71.82) is suggestive β€” the L1 model used only 1.6K open-source samples compared to L1.5's 14.1K synthetic samples β€” but this conflates data quantity, data quality, data length, and RL methodology improvements, making it impossible to attribute the gain to synthesis specifically.

Claim: Task-balanced sampling and task-specific advantage estimation stabilize long-context RL training.

This claim is well-supported by the ablation study in Table 2 and Figure 8. The training dynamics (Figure 8) show clear stabilization: entropy becomes less volatile, and response length growth is controlled. The performance improvement (+2.55 over GRPO on the 4B model) is modest but consistent across benchmarks. The mechanism is clearly articulated and the empirical results align with the mechanism: the largest improvement is on MRCR (+12.33 over GRPO), which has dense rewards that would benefit most from task-specific normalization.

A limitation is that the ablation is conducted only on the 4B model, not the 30B model. The paper scales AEPO to the 30B model (Figure 11 shows entropy dynamics), but the full task-balanced + task-specific advantage combination is not separately ablated at 30B scale. It is possible that the benefits are smaller at larger scale where the base model is more capable and training is inherently more stable. The paper's choice to ablate on 4B for iteration speed is understandable but leaves open the question of whether these innovations are equally important at the target scale.

Claim: AEPO enables stable training at scale without degradation.

The evidence is in Figure 11, which shows controlled entropy oscillation through 200 steps on the 30B model, and Table 5, which shows +3.29 improvement over GRPO on the 4B model. The claim is about stability (not just performance), and Figure 11 is the primary evidence β€” it demonstrates that AEPO avoids both divergence and collapse. However, the paper doesn't show what happens to the 30B model with vanilla GRPO at the same scale for comparison. We see the 30B model's entropy dynamics with AEPO, but we don't see without β€” so we're asked to infer stability from the fact that AEPO training succeeds, without a counterfactual demonstrating that vanilla GRPO would have failed at this scale. The evidence from the 4B model (Figure 8, Figure 10) supports the claim at smaller scale, and Figure 11 shows that AEPO is functional at 30B, but the critical comparison (30B with vs. without AEPO) is not shown.

Claim: The memory management framework extends the model's operational range to 1M–4M tokens.

This claim is supported by Table 9, which shows QwenLong-L1.5 operating on CorpusQA at 4M tokens (score: 14.29) where full-context inference is intractable. The +4.77 gain over baseline at 4M tokens is meaningful, though the absolute performance is low β€” 14.29 means the model fails on >85% of questions at this scale. The paper is transparent about this limitation, noting that Gemini-2.5-Pro outperforms the memory agent on shorter ultra-long sequences (128K–512K) and that the memory agent framework is a "robust foundation" for future work rather than a solved problem.

A missing experiment is the comparison between QwenLong-L1.5 in memory-agent mode and QwenLong-L1.5 with full-context inference on the same sequences at lengths where full-context is possible (128K–256K). The paper reports full-context baseline performance on the standard MRCR and CorpusQA subsets (0–128K), but doesn't report full-context performance on the 128K–512K subsets where full-context might still be feasible for the 256K-window model. Without this comparison, we cannot quantify the performance cost of using memory-agent mode versus full-attention mode at the boundary of the model's context window.

Claim: Long-context reasoning skills generalize to out-of-domain tasks.

Table 8 provides evidence that generalization occurs β€” AIME25 +3.65, LongMemEval +15.60, BFCL-V4 Memory-KV +5.80 β€” without catastrophic forgetting on standard benchmarks (MMLU-PRO, AIME24). The UMAP visualization (Figure 12) shows that the training data occupies a distinct semantic space from these evaluation sets, ruling out the trivial explanation that the evaluation data is simply in-distribution. The case study in Appendix C.1 provides a qualitative illustration: on an AIME problem, QwenLong-L1.5 demonstrates hypothesis refinement and strategic adaptation that the baseline lacks, which the paper attributes to information integration skills transferred from long-context training.

However, the evidence for the mechanism of transfer is circumstantial. The paper argues that "enhancing a model's long-context reasoning capabilities provides a foundational boost to its ability to maintain coherence and reason over extended informational sequences," but it doesn't demonstrate that this specific mechanism β€” rather than general improvements in reasoning capability from additional RL training β€” drives the AIME25 gain. An ablation showing that models trained with equivalent RL compute on short-context data do not show the same AIME25 improvement would strengthen this claim, but such an ablation is not present.

Other limitations of the experimental design:

  • Single model family: All experiments use Qwen3 models (4B and 30B-A3B). The paper does not demonstrate that the post-training recipe transfers to other model families (e.g., LLaMA, DeepSeek). Given that the RL innovations address specific failure modes of GRPO that may be model-dependent, replication on other architectures would strengthen the claims of generality.

  • No compute-matched comparison with alternative approaches: The paper shows that QwenLong-L1.5 outperforms QwenLong-L1 and naive GRPO, but doesn't compare against alternative long-context post-training methods (e.g., continued pre-training with long-context data, instruction tuning on long-context tasks, or other RL formulations like PPO with a value network). Without these comparisons, we know the recipe works but not whether it's more efficient or effective than alternatives at equivalent compute.

  • Difficulty estimation cost not discussed: Unlike QwenLong-L1, which explicitly discussed the cost of difficulty estimation, QwenLong-L1.5 doesn't address this β€” possibly because the difficulty-aware retrospective sampling is inherited from L1. The data synthesis pipeline's cost (LLM API calls for generation and verification) is also not quantified, making it difficult to assess the practical scalability of the approach.

  • The 14.1K training samples are filtered from 42.7K: The paper states that "after a rigorous pipeline of multi-stage difficulty filtering, deduplication, and test set decontamination, we curated a final set of 14.1k high-quality training samples from an initial pool of 42.7k synthesized examples." This 67% filtering rate raises questions about synthesis quality β€” why are two-thirds of synthesized examples unsuitable? Are there systematic failure modes in the synthesis pipeline? Understanding this would help others attempting to replicate the approach.

  • Limited analysis of negative results: The paper reports performance declines on Qasper (-3.00 in Table 12) and BFCL-V4 Memory-Rec_Sum (-1.94 in Table 8) but doesn't analyze these failures. Understanding where and why the training degrades performance would be informative for practitioners considering adopting the recipe.

6. Limitations and Trade-offs

6.1 Data Synthesis Cost and Scalability Are Unquantified and Potentially Prohibitive

The paper's data synthesis pipeline β€” comprising knowledge-graph construction, multi-agent self-evolution, and structural tabular data generation β€” relies heavily on proprietary and large open-source LLMs for question generation, solving, and verification. Section 7.1 acknowledges this explicitly:

"Although our data synthesis pipeline is automated, its scalability is constrained by practical bottlenecks, namely the API quotas of proprietary models and the computational cost of serving large open-source models for generating long-context data."

The consequence: The paper does not report the computational or financial cost of generating the 42.7K initial synthesized examples (of which only 14.1K survived filtering). This makes it impossible for practitioners to estimate whether the pipeline is reproducible outside well-resourced industrial labs. The 67% filtering rate (42.7K β†’ 14.1K) implies that a substantial fraction of synthesis attempts fail verification, multiplying the effective cost per usable sample. For the knowledge-graph-guided method specifically, the pipeline involves entity/relation extraction across 82,175 documents, cross-document KG aggregation, cluster-based refinement, random walk and BFS path sampling, entity obfuscation, and multi-paradigm question generation β€” each step requiring LLM inference. The structural tabular engine requires schema extraction, cross-document table aggregation, NL2SQL translation, and SQL execution. The MASE framework requires iterative proposer-solver-verifier loops. None of these costs are reported.

What evidence exists: The paper does not provide any cost analysis. The only data point is the filtering rate (14.1K from 42.7K, Section 3), but this reports yield, not cost. The paper's comparison with QwenLong-L1 (1.6K open-source samples) provides an implicit baseline β€” L1.5 uses 8.8Γ— more training data, achieved through synthesis rather than curation β€” but the cost of achieving this scale-up is unknown.

Mitigation status: Section 7.1 proposes a "closed-loop data flywheel" as future work: using the trained QwenLong-L1.5 model itself as a data generator to reduce reliance on external proprietary models. This is a sensible direction but is entirely speculative β€” no experiments demonstrate that a model trained on synthetic data can subsequently generate equally effective training data for the next iteration. The proposal does not address the chicken-and-egg problem: the first iteration still requires external models at unknown cost.

6.2 The Hardest Long-Context Reasoning Problems Remain Largely Unsolved

The paper's evaluation reveals a sharp capability boundary: on benchmarks requiring the most complex reasoning or the longest contexts, performance remains low despite substantial training investment. This is most visible in two places.

First, on the LongBench-V2 "Hard" subset (Table 11): QwenLong-L1.5 scores 52.41, compared to 46.95 for the baseline (+5.49). While this is an improvement, it means the model fails on nearly half of hard questions. The gap to flagship models is substantial: Gemini-2.5-Pro achieves 61.64 and GPT-5's performance is not broken out. The "Long" subset (longest context lengths in LongBench-V2) shows an even smaller absolute gain: from 42.59 to 45.37 (+2.78), with Gemini-2.5-Pro at 64.71. On both dimensions, the post-training recipe improves performance but does not close the gap with proprietary systems on the most challenging instances.

Second, on ultra-long contexts (Table 9): At 4M tokens, QwenLong-L1.5's memory agent achieves 14.29 on CorpusQA β€” meaning it fails on more than 85% of questions. The baseline achieves 9.52, so the gain (+4.77) is real but modest. At 1M tokens, the score is 20.72, meaning roughly 80% failure rate. The absolute performance at these scales is low enough that practical deployment for tasks requiring reliable ultra-long-context reasoning is not viable without human oversight or additional verification.

The consequence: The paper's methodology amplifies existing capability β€” it teaches the model to use its context window more effectively β€” but it cannot create fundamentally new reasoning abilities. On problems that are genuinely beyond the base model's competence, post-training provides diminishing returns. This is not a failure of the method per se, but it establishes a hard boundary: organizations facing problem distributions heavy on very hard or very long-context instances should not expect this recipe alone to make a 30B model competitive with much larger proprietary systems. The paper is largely transparent about this β€” the gains are concentrated on easy-to-medium difficulty and moderate context lengths β€” but the headline framing of "comparable to GPT-5 and Gemini-2.5-Pro" (Section 1) can obscure that this comparability does not extend to the most challenging evaluation subsets.

What evidence exists: Table 11 breaks out LongBench-V2 by difficulty and length; Table 9 provides ultra-long-context results. Both show the capability boundary clearly. The paper does not directly discuss this limitation as a limitation β€” the pattern is presented as evidence of where gains concentrate, not where the method fails β€” but the numbers speak for themselves.

Mitigation status: Not addressed as a limitation. Section 7.1 discusses expanding data coverage to "long-input, long-output problems" and multi-modal data, suggesting the authors view this as a data coverage issue rather than a fundamental capability ceiling. Section 7.2 proposes "granular credit assignment" and "sophisticated reward models" as future RL improvements. Neither directly addresses whether a 30B model can ever close the gap with much larger models on the hardest problems, or whether fundamentally different training paradigms would be needed.

6.3 The Memory Agent Incurs a Large Performance Penalty Relative to Full-Context Inference

The memory agent framework extends the model's operational range beyond its physical context window, but at a significant performance cost that the paper quantifies implicitly but does not discuss as a tradeoff.

The consequence: On sequences where both full-context inference and the memory agent are feasible (128K–512K tokens), the memory agent underperforms full-context models dramatically. Table 9 shows QwenLong-L1.5's memory agent achieving 34.87 on MRCR (128K–512K), compared to Gemini-2.5-Pro's full-context score of 53.83 and Gemini-2.5-Flash-Thinking's 53.98. This ~19-point gap means that when full-context inference is possible, it should be preferred. The memory agent is only advantageous when full-context is impossible (beyond the model's window or computationally intractable). This creates a deployment dilemma: at what context length should a system switch from full-context to memory-agent mode? The paper provides no guidance on this transition point, and the performance discontinuity at the switch boundary could create unpredictable behavior for users submitting inputs near the model's context limit.

Furthermore, the memory agent's performance degrades substantially as context length increases: 34.87 at 128K–512K β†’ 22.53 at 512K–1M β†’ 20.72 at 1M β†’ 14.29 at 4M (Table 9). While this degradation is slower than what full-context models would experience (they simply cannot process 4M tokens at all), it means the memory agent's reliability decreases as the problem becomes harder β€” precisely when users most need reliable reasoning.

What evidence exists: Table 9 provides the direct comparison between QwenLong-L1.5's memory agent and full-context models (Gemini-2.5-Pro, Gemini-2.5-Flash-Thinking) on the same benchmark subsets. The paper notes that "a leading proprietary model like Gemini-2.5-Pro currently exhibits the strongest performance on these ultra-long context tasks," which is a diplomatic acknowledgment of the memory agent's performance deficit. However, the paper does not explicitly characterize this as a tradeoff that practitioners must navigate.

Mitigation status: The paper frames the memory agent as a "robust foundation" for future optimization (Section 5.4) and identifies "further optimization" as future work (Section 7), but proposes no concrete improvements to the memory agent architecture or training. The sequential chunk processing constraint ("you cannot go back to re-read previous sections," Appendix D) is a deliberate design choice that amplifies difficulty; relaxing this constraint (e.g., allowing the agent to revisit earlier chunks when later information contradicts earlier extraction) could improve performance but would require a fundamentally different architecture.

6.4 The Credit Assignment Problem Is Stabilized but Not Solved

Section 4.3 provides a sharp diagnosis: incorrect long-context responses share substantial phrase-level overlap with correct responses (ROUGE-L of 45.37 on DocMath vs. 27.71 on AIME), meaning GRPO's uniform per-sequence advantage penalizes correct reasoning steps embedded in incorrect trajectories. The paper's proposed treatments β€” negative gradient clipping (Section 4.3) and AEPO (Section 4.4) β€” are acknowledged as stabilization mechanisms, not solutions:

Section 7.2 states: "While our proposed AEPO method effectively mitigates this issue through techniques like gradient clipping, it serves as a robust stabilization mechanism rather than a fundamental solution to the credit assignment problem."

The consequence: The stabilization mechanisms prevent training collapse, but they do not provide fine-grained credit assignment. The model still receives a uniform advantage signal across all tokens in a sequence β€” AEPO simply decides whether to apply that signal (based on entropy) rather than how to distribute it across tokens. This means tokens that were part of correct reasoning in an ultimately incorrect trajectory are either penalized (when negative gradients are active) or ignored (when masked), but never positively reinforced. The model cannot learn that "this specific reasoning step was correct, even though the overall answer was wrong."

This limitation likely contributes to the performance plateau observed on the hardest problems (Section 6.2 above). If the model cannot distinguish which parts of a failed reasoning chain were valuable, it cannot systematically improve its multi-step reasoning β€” it can only avoid repeating entire trajectories that happened to fail. The paper's AEPO curves (Figure 11) show stable training, but Table 10 shows that most of the performance gain comes from Stage 1 (+7.67 over baseline), with diminishing returns in subsequent stages (Stage 2: +0.87, Stage 3: +1.13). This tapering improvement could reflect the credit assignment ceiling: once the model has learned to generate roughly correct trajectories, uniform advantage signals provide insufficient granularity to refine away the remaining subtle errors.

What evidence exists: Table 3 quantifies the ROUGE-L overlap that creates the credit assignment problem. Figure 9 demonstrates the entropy-gradient correlation that motivates the clipping approach. Table 4 shows that static clipping provides modest gains (+0.29 to +1.29 over GRPO) and can backfire (low-entropy clipping degrades over time). Table 5 shows AEPO provides larger gains (+3.29) but still leaves substantial room for improvement β€” the absolute score of 59.36 on the 4B model suggests the model is far from solving these benchmarks. The paper's own acknowledgment in Section 7.2 is the clearest evidence that this is an unsolved problem.

Mitigation status: Section 7.2 identifies "token-level credit assignment mechanism within the thinking trajectory" as a primary focus of future research. The paper suggests that this "would allow the model to differentiate the contribution of each token within a generated thought or plan, enabling more precise and efficient learning." This is the right direction, but the paper provides no preliminary experiments or architectural proposals β€” it is purely aspirational. The current approach treats the problem as one of preventing harm from coarse credit assignment (via gradient masking), not enabling benefit from fine-grained credit assignment.

6.5 Evaluation Is Limited to a Single Model Family and Narrow Task Distribution

All experiments β€” both the main results and all RL ablations β€” use Qwen3 models (4B and 30B-A3B-Thinking). The paper does not demonstrate that the post-training recipe transfers to other model families (e.g., LLaMA, DeepSeek, Gemma) or other architecture types (dense vs. MoE, different attention mechanisms).

The consequence: Several aspects of the paper's findings could be model-specific in ways that are not obvious from the reported results. The PRM-like behavior that the RL training cultivates may depend on specific properties of Qwen3's pre-training data mixture or attention patterns. The entropy dynamics that AEPO controls are influenced by the model's initial calibration, vocabulary size, and tokenization β€” models with different entropy characteristics might require different threshold settings or might not exhibit the same instability patterns at all. The task-balanced sampling strategy assumes five task categories with distinct reward distributions; models with different pre-training may have different reward characteristics for the same tasks, potentially requiring different balancing schemes.

The benchmark distribution is also narrow: all six primary benchmarks are English-language long-context reasoning tasks. The paper does not evaluate on non-English long-context tasks, on tasks requiring long-form generation (as opposed to question answering), or on tasks mixing modalities (images, tables, code within long documents). Section 7.1 acknowledges the text-only limitation: "our current data is exclusively text-based," and multi-modal extension is listed as future work. But the single-model-family limitation is not discussed β€” the paper treats Qwen3 as representative without evidence.

What evidence exists: None, by definition β€” the paper does not report results on non-Qwen models, so there is no evidence for or against transferability. The ablation studies on the 4B model partially address the concern that findings might not scale to larger models (the 30B results in Figure 11 and Table 10 confirm that AEPO and the multi-stage pipeline work at larger scale), but they don't address cross-architecture generalization. The paper's claim that Qwen3-30B-A3B is "representative of the capabilities of many contemporary LLMs" (Section 5.1) is an assertion, not an empirical finding.

Mitigation status: Not addressed. The paper does not claim cross-model generality, but the presentation β€” particularly the framing as a "comprehensive post-training recipe" (Section 1) β€” implies applicability beyond the specific model tested. Future work on other model families is not listed in Section 7.

6.6 The Revision Model Has No Mechanism to Preserve Correct Answers

The paper does not incorporate a revision model in the QwenLong-L1.5 system; this limitation is about the absence of a capability that would be valuable for long-context reasoning, rather than a flaw in what the paper does implement.

The consequence: The memory agent processes documents sequentially and can update its memory when new information contradicts earlier extraction (the prompt in Appendix D explicitly instructs: "critically evaluate the existing memory and plan, and make appropriate updates and adjustments"). However, there is no mechanism for the model to revise its own reasoning outputs. If the final answer generation produces an incorrect answer, there is no iterative refinement step β€” the system produces one answer and stops. This is in contrast to the revision model approach used in QwenLong-L1 (Wan et al., 2025), where the model could generate sequential revisions conditioned on previous attempts.

For long-context reasoning, this limitation is significant because errors in the final answer often stem from subtle misinterpretations or missed connections that a second pass β€” conditioned on the first attempt β€” could catch. The high ROUGE-L between correct and incorrect responses (Table 3: 45.37) indicates that incorrect answers are often "almost correct" β€” they share substantial structure with correct solutions but contain a specific error. A revision mechanism could potentially correct such near-misses without requiring the model to generate a completely different reasoning path from scratch.

What evidence exists: The paper does not discuss this limitation directly. The absence of a revision mechanism is a design choice, not an oversight β€” the paper focuses on single-pass reasoning quality and memory-augmented processing. QwenLong-L1 did incorporate revisions, and QwenLong-L1.5 explicitly builds on L1's multi-stage length extension paradigm while adding the memory agent and stabilized RL. The choice to omit revisions from L1.5 is not explained; it may reflect a judgment that the memory agent provides more value for ultra-long contexts than revisions do, or it may simply reflect scope constraints.

Mitigation status: Not addressed. Section 7 does not mention revisions or iterative refinement as future work. This is a notable omission given that revisions were a key component of the predecessor system and that the paper elsewhere emphasizes the value of multi-step reasoning (through the multi-hop data synthesis and the memory agent's sequential processing). Incorporating revisions β€” particularly a revision mechanism that leverages the memory agent's accumulated state β€” could potentially address some of the credit assignment and error correction challenges identified in Sections 6.4 and 6.2.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper effects a reframing of long-context reasoning from an architectural property to a learned capability, which is less than a paradigm shift but more than an incremental improvement. The field has largely treated context window extension as the primary lever for long-context performance β€” make the window bigger during pre-training, and reasoning will follow. This paper provides systematic evidence that architectural capacity is necessary but radically insufficient: Qwen3-30B-A3B-Thinking already supports 256K tokens, yet underperforms its post-trained counterpart by 9.90 points on average and by 31.72 points on MRCR specifically (Table 7). The implication is that post-training is not an optional refinement but a required stage for converting context capacity into reasoning capability.

This reframing has two concrete consequences for how the field allocates research effort. First, it redirects attention from context window extension toward post-training methodology. If a 30B model with a 256K window can achieve GPT-5-competitive long-context performance through systematic post-training (71.82 vs. 74.74 average, Table 7), then the marginal return on further window extension β€” which is computationally expensive and architecturally complex β€” may be lower than the return on improving data synthesis and RL stabilization for existing windows. Second, it establishes long-context RL as a distinct sub-problem with its own failure modes (multi-clustered data distributions, biased advantage estimation from mixed reward densities, credit assignment crisis from high correct-incorrect trajectory overlap) that are not addressed by RL recipes developed for short-context reasoning. This creates a new subfield with its own research agenda, rather than treating long-context RL as "just apply GRPO with longer sequences."

The paper also reconciles a latent tension in the literature between works demonstrating the effectiveness of GRPO for reasoning (Shao et al., 2024; Yu et al., 2025b) and reports of instability when scaling RL to diverse, long-context tasks. The diagnosis β€” that correct and incorrect long-context trajectories share 64% more phrase-level overlap than short-context trajectories (Table 3: ROUGE-L 45.37 vs. 27.71), causing GRPO's per-sequence advantage to penalize correct reasoning steps β€” explains why naive GRPO underperforms on long-context data without contradicting its success on math and code. This is a diagnostic contribution that converts an empirical puzzle ("GRPO works on math but not on long documents") into a mechanistic understanding, enabling future work to target the credit assignment problem directly rather than treating it as mysterious instability.

Methodologically, the paper elevates entropy from a monitoring metric to a control signal through AEPO. The standard practice in RL-for-LLMs is to track entropy as a diagnostic β€” if it collapses, training has failed; if it spikes, something is wrong β€” but not to use it as an active input to the optimization procedure. AEPO demonstrates that policy entropy can serve as a real-time feedback mechanism for modulating the exploration-exploitation tradeoff, switching between rejection-sampling-like fine-tuning (masking negative gradients when entropy is too high) and full GRPO updates (restoring negative gradients when entropy dips). This is a transferable technique that could benefit any RL training regime where the optimal exploration rate shifts over the course of training β€” which is essentially all RL training regimes. The paper demonstrates it for long-context reasoning, but the principle is general.

The expert-merging paradigm for resolving destructive interference between full-context reasoning and memory management is another pattern with broader applicability. The paper's finding that training these two capabilities simultaneously causes mutual degradation (full-context drops from 71.59 to 68.53 after memory-RL, Table 10) but that training them separately and merging via SCE yields a model that excels at both (71.82 full-context, 22.53 memory-agent) suggests that capability interference during RL is a first-order concern that should be managed through training architecture rather than ignored. This has implications for any multi-capability LLM training pipeline β€” if capabilities require qualitatively different information processing patterns, co-training may be counterproductive even when the capabilities seem complementary at the application level.

The paper also weakens the case for training ever-larger models as the primary path to long-context capability. QwenLong-L1.5-30B-A3B achieves performance competitive with Gemini-2.5-Pro and GPT-5 (Table 7) despite being a fraction of their size. This does not mean that scale is irrelevant β€” the hardest LongBench-V2 problems still show a gap (52.41 vs. 61.64 for Gemini-2.5-Pro, Table 11), and the memory agent underperforms full-context inference at scales where both are feasible (34.87 vs. 53.83 on MRCR 128K–512K, Table 9). But it does demonstrate that post-training methodology can close a substantial portion of the capability gap between midsize open-weight models and massive proprietary systems, which has significant implications for the economics and accessibility of long-context AI.

Follow-Up Research This Work Enables

Token-level credit assignment for long-context reasoning trajectories. The paper identifies the core limitation of its approach in Section 7.2: "our current approach assigns a uniform advantage signal to an entire reasoning step," and AEPO "serves as a robust stabilization mechanism rather than a fundamental solution to the credit assignment problem." The high ROUGE-L between correct and incorrect responses (45.37 on DocMath, Table 3) means that incorrect trajectories contain substantial correct reasoning that is being penalized (or at least not positively reinforced) under the current approach. A concrete follow-up would: (1) annotate a subset of the 14.1K training samples with per-step correctness labels (using the PRM-style Monte Carlo rollout approach or LLM-as-judge step evaluation), (2) modify the GRPO objective to compute per-token advantages by decomposing the sequence-level reward into step-level contributions using a learned credit assignment model, and (3) compare the resulting training dynamics against AEPO's uniform-advantage baseline on both stability metrics (entropy trajectories, response length control) and final benchmark performance. The key measurement would be whether per-token credit assignment reduces entropy volatility without needing AEPO's dynamic masking, and whether it enables continued performance improvement in Stages 2–4 of the training pipeline (where Table 10 shows diminishing returns from the current approach: +0.87 and +1.13 in Stages 2 and 3, respectively).

Verifier-guided search over long-context reasoning paths. The paper's memory agent processes documents sequentially and produces a single final answer, but there is no mechanism for the agent to backtrack or explore alternative interpretations when later chunks contradict earlier extraction. The prompt template (Appendix D) explicitly states "you cannot go back to re-read previous sections," which is a practical necessity for the current architecture but also a hard constraint that likely contributes to the 14.29 score on 4M-token CorpusQA (Table 9). A natural extension would: (1) train a step-level verifier (analogous to the PRMs used in math reasoning) that scores the quality of each memory update $m_t$ given the current chunk and question, (2) implement a beam search over the memory update process β€” at each chunk, generate multiple candidate memory updates, score them with the verifier, and keep the top-$k$ memory states, and (3) generate the final answer from each surviving memory state and select the best via the existing reward mechanism. This would test whether the memory agent's performance on ultra-long contexts (1M–4M tokens) can be improved through search without architectural changes to the underlying model. The expected benefit is largest at extreme lengths where extraction errors compound, and the cost is $k \times$ more inference compute per chunk (which may be acceptable for one-time processing of very long documents).

Cross-model-family replication of the RL stabilization findings. The paper's RL innovations β€” task-balanced sampling, task-specific advantage estimation, and AEPO β€” are validated only on Qwen3 models (4B and 30B-A3B). A critical stress-test would replicate the ablation sequence from Tables 2, 4, and 5 on a non-Qwen architecture, such as LLaMA-3-8B or DeepSeek-V2-Lite, using the same 14.1K training dataset. The key question is whether the specific instability patterns β€” multi-clustered data distributions (Figure 7), biased advantage estimation from mixed reward densities, and the entropy-gradient correlation in negative rollouts (Figure 9, Spearman's ρ = 0.96) β€” are properties of the task (long-context reasoning) or of the model (Qwen3's pre-training). If AEPO provides similar stabilization and performance gains across architectures, it strengthens the claim that long-context RL has universal failure modes requiring entropy-based control. If AEPO's benefit is model-specific (e.g., Qwen3 has unusual entropy characteristics that make the gradient-entropy correlation particularly strong), it would suggest that stabilization strategies need to be calibrated per model family β€” a finding that would be important for practitioners and would motivate research into model-adaptive RL regularization.

Direct comparison of data synthesis methods through ablation. The paper uses three synthesis methods (knowledge-graph-guided, structural tabular engine, MASE) but reports only the combined effect. A high-value follow-up would train three separate models, each using RL data from only one synthesis method (controlling for total sample count), and evaluate on benchmarks that differentially stress the targeted capabilities. Specifically: (1) the KG-guided model should show disproportionate gains on multi-hop reasoning benchmarks (Musique, HotpotQA, 2WikiMultihopQA) and on question types from the KG pipeline (Multi-fact Reasoning, Causal Analysis, Temporal Reasoning); (2) the tabular engine model should show disproportionate gains on DocMath and on CorpusQA subsets requiring numerical aggregation; (3) the MASE model should show gains on viewpoint analysis and long in-context learning. This ablation would answer a critical open question: is the data synthesis pipeline effective because of the specific structural constraints imposed by each method (KG paths enforce genuine multi-hop dependencies, SQL execution ensures numerical correctness), or because any diverse, challenging long-context data would work equally well? If the former, it justifies the substantial engineering investment in multi-method synthesis; if the latter, simpler and cheaper synthesis approaches might suffice. The paper reports a 67% filtering rate (14.1K from 42.7K synthesized examples); a per-method breakdown of filtering rates would also reveal which synthesis methods produce the highest-quality data per unit of generation cost.

Closed-loop data flywheel with the trained model as generator. Section 7.1 proposes using QwenLong-L1.5 itself to generate training data for subsequent iterations, but this is entirely speculative. A concrete experiment would: (1) use QwenLong-L1.5-30B-A3B as the proposer and solver in the MASE framework (replacing the proprietary models used in the initial synthesis), (2) generate a second generation of training data from the same document corpus, (3) train a new model (QwenLong-L1.5-Gen2) starting from Qwen3-30B-A3B-Thinking using this self-generated data with the same RL recipe, and (4) compare Gen2's performance against the original QwenLong-L1.5 on the same benchmarks. The critical measurement is whether Gen2 matches, exceeds, or degrades relative to Gen1. If Gen2 matches or exceeds Gen1, the flywheel is validated β€” the approach becomes self-sustaining and no longer dependent on proprietary model APIs for data generation. If Gen2 degrades, it would reveal that model-generated training data lacks the diversity or difficulty of externally generated data, potentially due to the model having learned the synthesis patterns from its own training and reproducing them rather than creating genuinely novel challenges. The paper's case studies (Appendix B) provide examples of the current data quality; comparing Gen2's generated questions against these would reveal whether self-generation produces qualitatively different (and likely worse) question distributions.

Long-context RL with process rewards for memory agent fine-tuning. The memory agent uses a single trajectory-level reward based on final answer correctness (Section 2.3), broadcasting the same advantage to all memory updates and plans in the trajectory regardless of their individual quality. Given the paper's diagnosis that credit assignment is the central challenge in long-context RL (Section 4.3), the memory agent is the component where this challenge is most acute β€” an incorrect final answer could result from a single poor memory update early in the sequence, but all updates are penalized equally. A targeted experiment would: (1) train a process reward model that scores individual memory updates $(m_t, p_t)$ based on whether they correctly extract and plan for information relevant to the question (using the same Monte Carlo rollout approach the paper uses for its QA verifier, but applied to the memory update step), (2) modify the memory agent's RL objective to use per-step advantages derived from the process rewards rather than broadcasting the trajectory-level advantage, and (3) measure whether per-step credit assignment improves memory agent performance on ultra-long contexts (128K–4M tokens) compared to the trajectory-level baseline. The expected benefit is largest for very long documents where one poor update can cascade; the cost is the additional training of the process reward model and the per-step reward computation during RL. This experiment directly addresses the "granular credit assignment for thinking trajectories" identified as future work in Section 7.2, applied to the memory agent rather than full-context reasoning.

Practical Applications and Downstream Use Cases

Cost-efficient batch processing of long-document QA at scale. Organizations that need to answer questions over large document collections β€” legal discovery, financial audit, academic literature review β€” currently face a tradeoff between using large proprietary models (expensive per query, but high accuracy) and smaller open-weight models (cheaper, but lower accuracy). QwenLong-L1.5-30B-A3B achieves 71.82 average accuracy, competitive with GPT-5 (74.74) and Gemini-2.5-Pro (72.40) across six benchmarks (Table 7), while being a 30B MoE model that can be self-hosted. For a legal discovery use case processing 100,000 documents with 1,000 queries each (100M total inferences), the cost difference between API calls to a proprietary model at commercial rates versus self-hosting QwenLong-L1.5 could be decisive. On tasks specifically requiring multi-hop aggregation β€” CorpusQA, where QwenLong-L1.5 scores 81.25 vs. GPT-5's 81.56 β€” the accuracy is essentially equivalent, meaning the cost savings come with no performance penalty. For deployments where documents are extremely long (1M–4M tokens), the memory agent provides a capability that some proprietary APIs may not offer at all β€” QwenLong-L1.5 achieves 14.29 on 4M-token CorpusQA (Table 9), which while low provides a baseline accuracy for tasks that are simply impossible with standard full-context inference.

On-device or edge deployment for long-context reasoning with latency constraints. The MoE architecture of Qwen3-30B-A3B-Thinking means that only a subset of parameters are active per token (the "A3B" designation likely indicates ~3B active parameters from a 30B total), making inference substantially cheaper than a dense 30B model. For applications requiring long-context reasoning with low latency β€” real-time dialogue systems that must maintain state over hour-long conversations, onboard document analysis in vehicles or field equipment, privacy-sensitive medical or legal applications where data cannot leave the device β€” this efficiency is critical. The LongMemEval result (+15.60 over baseline, from 60.80 to 76.40, Table 8) is directly relevant for dialogue systems: a model that can maintain and recall information over extended conversations without relying on cloud inference. The BFCL-V4 Memory-KV result (+5.80, from 10.97 to 16.77) is relevant for agentic systems that need to track structured state (key-value pairs) over multi-step operations β€” a common pattern in tool-using agents where function call parameters and results must be remembered across calls. Both use cases benefit from the generalization finding that long-context post-training transfers to applied memory tasks without catastrophic forgetting of general capabilities (MMLU-PRO: 81.33 vs. 81.03 baseline, Table 8).

Training data generation for self-improving long-context systems. The paper's data synthesis pipeline β€” particularly the MASE framework with proposer-solver-verifier agents β€” is a general-purpose system for generating challenging long-context QA pairs from any document corpus. Organizations with proprietary document collections (internal wikis, technical documentation, customer support logs) can use this pipeline to generate training data tailored to their domain, then fine-tune QwenLong-L1.5 (or another base model) on this domain-specific data using the paper's RL recipe. The expected benefit is that domain-specific long-context data would produce larger gains on in-domain tasks than the general-domain benchmarks reported in the paper β€” the +9.90 average gain over baseline (Table 7) was achieved with general-domain documents; domain-specific documents might yield larger improvements for the target domain. A concrete deployment scenario: a software company with millions of lines of internal code and documentation could use the code-repository-aware synthesis pipeline to generate QA pairs requiring multi-file reasoning (e.g., "if we change this function in module A, which tests in module B would break, and why?"), train a long-context model on this data, and deploy it as an internal developer assistant that can reason over the entire codebase.

Dialogue systems requiring extended conversational memory. The +15.60 gain on LongMemEval (Table 8) is the paper's largest generalization result and addresses a specific pain point in deployed chatbots and voice assistants: maintaining coherent state and recalling user-provided information over conversations that span hours or multiple sessions. Current production systems often rely on explicit memory architectures (RAG over conversation history, structured knowledge bases updated by extraction models) because standard LLMs exhibit substantial degradation in recall accuracy as conversation length increases. QwenLong-L1.5's dialogue memory training data (Table 1 lists "Dialogue data" among sources and "Dialogue memory" among question types) and the post-training recipe's emphasis on information retention over extended sequences directly target this failure mode. A practical deployment would replace or augment an existing RAG-based conversation memory system with QwenLong-L1.5 performing in-context recall over the full conversation history, with the 76.40 LongMemEval score (up from 60.80 baseline) representing substantially improved reliability for recalling user preferences, facts, and commitments made earlier in the conversation.

When to Prefer This Method

The paper does not articulate an explicit tradeoff against named alternative long-context post-training methods. It compares against its predecessor (QwenLong-L1) and against naive GRPO, but does not position QwenLong-L1.5 against, for example, continued pre-training with long-context data, supervised fine-tuning on long-context instruction datasets, or alternative RL formulations like PPO with a value network. The paper's contribution is positioned as a comprehensive recipe (data + RL + architecture) rather than as one method among interchangeable alternatives. The relevant decision is therefore whether to adopt this specific recipe versus relying on standard post-training approaches, and the paper does not provide the controlled comparisons against those alternatives that would support a formal decision rule. The implicit guidance from the results is: prefer this recipe when the target task requires multi-hop reasoning over globally distributed evidence in contexts exceeding 32K tokens, and when you have the infrastructure to run a multi-stage RL pipeline with on-policy GRPO β€” but this is a description of when the method is applicable, not a decision rule against named competitors.