Beyond Reasoning Gains: Mitigating General Capabilities Forgetting in Large Reasoning Models

ArXiv: 2510.21978

🎯 Pitch

RLVR post-training makes vision-language models better at reasoning but silently erodes their perception, knowledge, and robustness—sometimes dropping below the base model on 6 core benchmarks. The authors introduce RECAP, which automatically detects which skills are converged or flailing during multi-objective replay, and shifts training focus away from already-saturated rewards toward volatile ones, preserving general capabilities while also improving reasoning accuracy and cutting chain length by 60%.

1. Executive Summary

This paper investigates the problem of general capability regression during reinforcement learning with verifiable rewards (RLVR) post-training for vision-language models, and introduces RECAP—a replay strategy with dynamic objective reweighting for general knowledge preservation. The authors first empirically demonstrate that open-source reasoning models (e.g., VLAA-Thinker, SpaceThinker, vision-grpo variants derived from Qwen2.5-VL-3B and Qwen2.5-VL-7B) consistently underperform their base counterparts on non-reasoning benchmarks such as VisOnlyQA, A-OKVQA, and OCRBench, motivating the integration of general-capability data replay into RLVR training. RECAP addresses the challenge of heterogeneous convergence rates across objectives—format rewards saturate quickly while accuracy rewards remain volatile—by computing per-objective convergence rate and instability over sliding windows, then dynamically reweighting loss terms via a temperature-controlled softmax to shift capacity away from saturated signals toward underperforming or high-variance ones. Across two experimental settings (RLVR-only with Qwen2.5-VL-3B and a hybrid RLVR+SFT setting with Qwen2.5-VL-7B), RECAP not only preserves general capabilities but improves reasoning performance, achieving a 2% absolute gain on LISA segmentation while matching or exceeding specialized open-source models, and produces substantially shorter reasoning chains (~27 vs. ~67 words on average) without sacrificing accuracy, establishing that dynamic objective reweighting during replay can jointly optimize reasoning gains and general knowledge retention only when the scheduler correctly identifies and de-prioritizes objectives that have already converged.

2. Context and Motivation

The Core Problem: RLVR Post-Training Causes General Capability Regression

The fundamental tension this paper addresses is deceptively simple: making a model better at reasoning through reinforcement learning can make it worse at everything else. The paper's central concern is what happens to a vision-language model's broad, pretrained capabilities when we push it to excel at specific, rewardable tasks like mathematical reasoning or code generation. This is not merely a theoretical curiosity—it reflects a growing reality in how state-of-the-art models are trained in production pipelines, where RLVR has become a standard post-training step.

The authors frame this as a specific instance of catastrophic forgetting in the continual learning context, but with a twist that makes it both more subtle and more practically urgent than traditional formulations of the problem. In classical catastrophic forgetting, a model trained sequentially on task A then task B loses performance on task A—a problem well-studied since the late 1980s (McCloskey & Cohen, 1989; French, 1999). The RLVR setting is more complex because what is "forgotten" is not a single task but rather a diffuse set of general capabilities acquired during pretraining: perception, visual grounding, safety alignment, factual grounding, and robustness to distribution shifts. These capabilities are never explicitly rewarded during RLVR training, yet they constitute much of what makes a foundation model useful beyond its headline benchmark scores.

The paper opens with a concrete empirical demonstration of this regression. Figure 1 surveys multiple open-source reasoning models derived from the Qwen2.5-VL family (both 3B and 7B scales) and evaluates them on six non-reasoning benchmarks: A-OKVQA (knowledge-based visual question answering), AesBench (image aesthetics perception), VStar (spatio-temporal reasoning), VisOnlyQA (vision-only recognition), OCRBench (text recognition), and R-Bench-Dis (distribution-shift robustness). The pattern is consistent and sobering:

"Across both Qwen2.5-VL families, reasoning-finetuned models generally underperform their base models on perception and robustness tasks"

This means that models praised for their improved math and logic scores are simultaneously worse at recognizing objects, reading text in images, and maintaining factual accuracy than the base models from which they were derived. The one exception the paper flags is MiMo-VL-7B-RL, which—tellingly—employs mixed on-policy RL that tries to maximize model capacity on multiple axes beyond math and reasoning, though the paper notes that "the detailed framework and the sampling or reweighting are not disclosed" in MiMo's technical report. This observation serves as both a motivation and a hint: preserving general capabilities during RL training is possible, but current open methods do not achieve it.

The authors' own initial experiments (Figure 2) crystallize the problem in a controlled setting. Training Qwen2.5-VL-7B solely on math reasoning data for 500 steps causes a 7% absolute drop on LISA (a segmentation benchmark), from 65.1% to 57.6% IoU. Meanwhile, the reasoning-only model's performance on the target math task improves. This is the exact tradeoff the paper seeks to break.

Why This Problem Matters: The Practical and Theoretical Stakes

The significance of capability regression during RLVR extends well beyond academic interest in continual learning. The paper motivates the importance of this problem along several interconnected dimensions, each carrying distinct practical consequences.

The deployment paradox: better reasoning, poorer reliability. When a reasoning model is deployed in a real-world application—a tutoring system, a coding assistant, a scientific analysis tool—it is not evaluated solely on math competition problems. Users will ask it to look at diagrams, read charts, interpret photographs, and ground its reasoning in visual evidence. If the model's perception and OCR capabilities have eroded during RLVR training, its reasoning may be logically sound but empirically wrong because it misreads the input. The paper's evidence that OCRBench and VisOnlyQA scores regress (Figure 1) suggests this is not a hypothetical concern but a documented failure mode in current open-source reasoning models. The model becomes a better reasoner about things it cannot accurately perceive—a dangerous combination for any application where visual grounding matters.

Safety and hallucination amplification. The paper cites emerging evidence that RLVR-trained models exhibit increased hallucination (Jaech et al., 2024; Yao et al., 2025b) and heightened vulnerability to jailbreak attacks (Lou et al., 2025; Yao et al., 2025a). This is a particularly insidious form of regression because it is not immediately visible from benchmark scores. A model that scores higher on MATH while fabricating visual details or becoming easier to jailbreak represents a net negative for deployment safety, even if the headline numbers improve. The authors frame safety regression as part of the broader "general capability" degradation problem, arguing that capabilities like factual grounding and instruction-following honesty are among the pretrained skills most vulnerable to being overwritten by narrow reward optimization.

The economic inefficiency of single-purpose post-training. From an organizational perspective, producing specialized reasoning models through RLVR that then cannot perform basic perception tasks is fundamentally wasteful. It fragments the model portfolio: one model for reasoning, another for general visual understanding, a third for OCR, each derived from the same base model through separate, incompatible fine-tuning pipelines. This multiplies serving costs, complicates model routing, and prevents unified deployments. A post-training methodology that achieves reasoning gains without sacrificing general capabilities would enable a single model to serve both purposes, reducing infrastructure costs and simplifying deployment architectures. The paper's motivation is partly economic: if dynamic replay can preserve general capabilities at minimal additional training cost, it eliminates the need for separate specialized models.

Theoretical significance: verifier over-optimization meets catastrophic forgetting. At a conceptual level, the paper connects two well-known failure modes that are typically studied in isolation. The first is reward hacking and verifier over-optimization in RL—the phenomenon where models learn to exploit reward signals rather than truly achieving the intended behavior. The second is catastrophic forgetting in continual learning. The paper's argument is that these two phenomena interact in RLVR in a way that neither literature adequately addresses. Format rewards (e.g., "did the model put its answer in the right XML tags?") converge quickly and saturate early, after which continued optimization against them provides no genuine benefit while potentially crowding out gradient signal for capabilities that require sustained learning. This is a form of over-optimization, but its harm manifests as forgetting rather than as reward hacking in the traditional sense. The paper thus provides a bridge between these two research communities, showing that the dynamics of objective convergence in multi-reward RL directly cause forgetting of capabilities not represented in the reward set.

Where Existing Approaches Fall Short

The paper identifies several categories of prior approaches and explains why each is insufficient for the specific challenge of preserving general capabilities during RLVR post-training.

KL regularization toward the reference policy. The most widely deployed mitigation for policy drift in RLHF and RLVR is a KL divergence penalty that prevents the fine-tuned policy $\pi_\theta$ from deviating too far from the base model $\pi_{\text{ref}}$ . This is a cornerstone of the InstructGPT recipe (Ouyang et al., 2022) and is included by default in GRPO (the RL algorithm the paper uses) as a term $-\beta D_{KL}(\pi_\theta \parallel \pi_{\text{ref}})$ . The paper acknowledges this as a standard tool but diagnoses a fundamental limitation:

"this regularization term is imposed on the target domain data and does not guarantee retention of arbitrary non-target skills."

The KL penalty is computed on whatever data the model is currently being trained on—in this case, reasoning problems. It prevents the policy from changing too much on the reasoning distribution, but it provides no direct signal about behavior on perception, OCR, or safety tasks. The model can drift significantly on those distributions while maintaining the KL constraint on the reasoning distribution, because the KL is not a measure of capability retention on out-of-distribution inputs. The paper's experiments explicitly disable KL regularization in most runs to disentangle its effects from the replay mechanism, and the baseline labeled "LwF" (which uses KL with $\beta = 0.01$ ) is shown to underperform replay-based methods on reasoning benchmarks (Table 2: LwF achieves 29.59 on MMMU-PRO vs. 31.91 for Uniform and 34.15 for RECAP), consistent with the observation that KL constrains plasticity without providing targeted capability preservation.

Experience replay with uniform or proportional sampling. A natural extension is to simply mix general-capability data back into the RLVR training loop—replay old data alongside the new reasoning data. This approach has strong empirical support in continual learning (Rebuffi et al., 2017; Rolnick et al., 2019; Buzzega et al., 2020) and has been applied in LLM post-training settings (Zheng et al., 2023; Zhang et al., 2025). However, the paper identifies a critical and under-appreciated challenge: data mixture is not the same as objective mixture, and naively sampling data uniformly or proportionally leaves the relative weighting of different objectives unaddressed.

In the RLVR setting, different data domains come with fundamentally different reward structures. A reasoning problem may have both an accuracy reward (did the model get the right answer?) and a format reward (did the model follow the required output template?). A perception task might have an IoU reward (did the bounding box overlap sufficiently with the ground truth?), a format reward, and potentially a supervised next-token prediction loss. Even if data from different domains are sampled uniformly, these objectives will train at different speeds, have different noise characteristics, and reach saturation at different points in training. The paper's Figure 4 provides the key evidence: format rewards converge extremely quickly (reaching near-ceiling performance within ~50 steps), while accuracy rewards remain noisy and slowly improving throughout training. Uniform or proportional data sampling provides no mechanism to account for this heterogeneity—it merely ensures the model sees diverse data, not that it learns from it in a balanced way.

Static reward weight tuning. Practitioners commonly address multi-objective RL by manually setting reward coefficients—for example, doubling the weight of accuracy relative to format rewards, or assigning different weights to different domains based on intuition or small-scale sweeps. The paper references this practice explicitly in discussing MoDoMoDo (Liang et al., 2025), which the authors treat as an upper-bound baseline for static mixture approaches. MoDoMoDo trains proxy models to learn the mapping from data mixing ratios to test performance, which requires rerunning experiments when new tasks are introduced and is computationally expensive. Even after selecting an "optimal" mixture, the method still depends on hand-tuned reward weights, which the paper notes "are also set differently in prior work without clarification, which limits the generality." The fundamental limitation is that static weights cannot adapt to changing convergence dynamics during training—a weight that is appropriate in the first 50 steps (when format is underexplored) becomes suboptimal after step 200 (when format is saturated and further optimization offers no benefit).

Single-objective RLVR with no replay. At the extreme, many open-source reasoning models are trained only on reasoning data with reasoning rewards, making no attempt to preserve general capabilities during RLVR. Table 1 shows that multiple such models (e.g., Qwen2.5VL-3b-RLCS, vision-grpo models) achieve strong reasoning scores but struggle on broader benchmarks. This approach treats capability regression as an acceptable cost of reasoning improvement—a tradeoff the paper argues is unnecessary and harmful.

The MiMo approach: mixed on-policy RL with undisclosed reweighting. The paper identifies MiMo-VL-7B-RL as the one model that largely preserves general capabilities during reasoning training (Figure 1 shows it remaining competitive with its base model on non-reasoning tasks). The authors hypothesize this is due to "mixed on-policy reinforcement learning that tries to maximize the model capacity on multiple axes beyond math and reasoning." However, the MiMo technical report does not disclose its sampling or reweighting strategy, leaving the community without a replicable method. This observation motivates the paper's core contribution: a principled, transparent, and computationally lightweight mechanism for deciding how much to focus on each objective during multi-domain RLVR training.

How This Paper Positions Itself

The paper positions RECAP as filling a specific gap in the RLVR training toolkit: a magnitude-agnostic, online scheduler that dynamically reweights heterogeneous objectives based on their observed convergence behavior, requiring no auxiliary models, no per-objective normalization, and no manual coefficient tuning.

The intellectual lineage is clearly drawn from continual learning (where replay is a well-established technique) and multi-task learning (where loss reweighting based on convergence rates has been explored), but the paper argues that neither field's standard solutions transfer directly to the RLVR setting. Continual learning methods assume tasks are presented sequentially and focus on preventing interference; in RLVR, all objectives are present simultaneously, but they evolve at different rates and saturate at different times. Multi-task learning reweighting methods typically require per-objective gradient computation or assume stationary, low-variance loss signals—assumptions violated by the high-variance, non-stationary nature of on-policy RL rewards (Figure 11 in the Appendix shows the total reward standard deviation approaching 0.9 at step ~20, underlining just how noisy per-step RL signals are).

The paper's positioning is thus:

Against KL-only regularization: KL prevents drift on the training distribution but does nothing to preserve capabilities on unseen domains. It is a blunt instrument that constrains the policy uniformly rather than targeting specific capabilities at risk.
Against static replay with fixed weights: Mixing data uniformly or proportionally ensures exposure but does not account for differential convergence rates. A format reward that saturates in 50 steps continues to receive equal weight throughout training, crowding out gradient signal for objectives that still have room to improve.
Against manual or proxy-model-based weight tuning: These approaches are expensive, brittle to new task additions, and cannot adapt to changing dynamics during training.
For dynamic, convergence-aware reweighting: The key insight is that convergence rate and instability are sufficient signals to dynamically allocate training capacity. When a reward saturates (convergence rate $c \approx 1$ ) and stabilizes (instability $i \approx 0$ ), its weight should decrease. When a reward is still improving ( $c > 1$ ) or highly volatile ( $i$ large), its weight should increase. This is an online, local decision that requires no knowledge of the absolute magnitude of different rewards—the paper emphasizes that RECAP is "magnitude-agnostic" because it operates on ratios and normalized instability measures rather than raw reward values.

The paper draws an implicit parallel to the way adaptive learning rate methods (Adam, RMSprop) use gradient statistics to adjust per-parameter step sizes. RECAP does something analogous at the objective level: just as Adam gives larger updates to parameters with small, consistent gradients and smaller updates to parameters with large, noisy gradients, RECAP gives higher weight to objectives that are still improving (high $c$ ) or unstable (high $i$ ) and lower weight to objectives that have plateaued.

This parallel helps explain why the method works without expensive per-objective hyperparameter tuning. The scheduler's behavior is emergent from the observed dynamics of training itself—it does not need to know in advance which objectives will saturate when, because it discovers this during training and adjusts accordingly. The paper's Figure 4 provides the key empirical justification: format rewards show $c \to 1$ and $i \to 0$ after roughly 50 steps, at which point their priority naturally drops, while accuracy rewards maintain higher $c$ and $i$ values and thus retain higher weight. The system automatically reallocates capacity from easy-to-learn to hard-to-learn objectives, without any human specification of which is which.

The positioning also reflects a practical philosophy: RECAP is designed as a "plug-in scheduler" that can be dropped into existing RLVR pipelines without changing the underlying RL algorithm, without training additional models, and without heavy hyperparameter tuning. The paper explicitly states that the only new hyperparameter is the softmax temperature $T$ (set to 5 by default), and that the method reduces to uniform weighting as $T \to \infty$ , making it a conservative extension of standard practice rather than a radical departure. This design choice reflects the authors' intended audience—practitioners running large-scale RLVR training who need reliable capability preservation without extensive additional engineering.

Finally, the paper positions its contribution within a broader narrative about the future of post-training. As reasoning-specialized models become more common, the challenge of maintaining broad competence while pursuing narrow excellence will only grow. The paper's framing suggests that RECAP is not just a fix for current RLVR pipelines but a necessary architectural component for any multi-objective post-training regime, much as replay buffers are a necessary component of deep RL in non-stationary environments. The implicit claim is that dynamic objective reweighting should be as standard in multi-reward RLVR as KL regularization is in standard RLHF—not as a replacement, but as a complementary mechanism that addresses a failure mode (domain-specific capability regression) that KL alone cannot.

3. Technical Approach

3.1 Reader Orientation

RECAP is a plug-in training scheduler that sits on top of an existing reinforcement learning with verifiable rewards (RLVR) pipeline and dynamically adjusts how much attention the optimizer pays to each learning objective based on how quickly and how stably each objective is improving. The core problem it solves is that different rewards in multi-objective RLVR training converge at drastically different speeds—format-following rewards saturate in ~50 steps while reasoning accuracy remains noisy and slowly improving for hundreds of steps—and a static, uniform weighting of objectives wastes training capacity on already-mastered skills at the expense of harder ones, which manifests as forgetting of general capabilities not directly rewarded in the current training data.

3.2 Big-Picture Architecture (Diagram in Words)

The RECAP system has three major components layered on top of a standard GRPO-based RLVR training loop:

Multi-domain data replay: General-capability datasets (perception, OCR, referring expression comprehension) are sampled alongside the target reasoning dataset at each training step, ensuring the model continues to see diverse inputs rather than only reasoning problems. Data sampling is uniform across domains by default.
Per-objective convergence monitor: For each distinct learning objective (e.g., reasoning accuracy reward, format reward, IoU reward, next-token prediction loss), a sliding-window tracker maintains running estimates of the objective's recent average value, its average over the previous window, and its standard deviation. From these, two derived signals are computed: a convergence rate (ratio of current to previous window averages) and an instability (coefficient of variation over the combined window).
Dynamic objective reweighting scheduler: Each training step, the convergence rate and instability for every objective are combined into a scalar priority score, then passed through a temperature-controlled softmax to produce normalized weight coefficients. These coefficients multiply the corresponding loss terms in the total training objective, so objectives that have converged ( $c \approx 1$ ) and stabilized ( $i \approx 0$ ) receive lower weight, while objectives that are still improving ( $c > 1$ ) or fluctuating ( $i$ large) receive higher weight.

Information flows as follows: at each training step, a batch is assembled by uniformly sampling from all data domains → the base LLM (frozen rollout policy) generates responses → per-objective rewards and losses are computed → the convergence monitor updates its sliding-window statistics for each objective → the reweighting scheduler computes new $\lambda$ coefficients → the weighted sum of losses is formed → the optimizer updates model parameters → repeat.

3.3 Roadmap for the Deep Dive

First, the RLVR training formulation, including the GRPO algorithm definition and the multi-objective loss structure, because RECAP's reweighting operates directly on this loss—we need to understand what is being reweighted before we can understand how.
Second, the problem of heterogeneous convergence, with concrete evidence from the paper's training runs showing that format rewards saturate an order of magnitude faster than accuracy rewards, because this empirical observation is the direct motivation for the reweighting design.
Third, the per-objective convergence and instability signals, including the exact sliding-window formulas, because these are the raw measurements that feed the scheduler.
Fourth, the priority score and reweighting mechanism, including the temperature-controlled softmax and the design rationale for combining convergence rate and instability additively, because this is the core algorithmic contribution.
Fifth, the overall training objective and integration with GRPO, showing how the $\lambda$ coefficients are plugged into the existing RLVR loss, because this demonstrates the method's practical simplicity.
Sixth, domain-specific design choices, including the decision not to apply a thinking-format reward to perception tasks and the rationale for using uniform data sampling as the default replay strategy, because these implementation details materially affect performance.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a systems and methods paper whose core idea is that multi-objective RLVR training suffers from asynchronous convergence, and that a lightweight, online scheduler using only local window statistics can automatically reallocate training capacity from saturated objectives to under-performing ones, preserving general capabilities while improving reasoning.

GRPO and the Multi-Objective RLVR Formulation

RECAP is designed to integrate with Group Relative Policy Optimization (GRPO), the RL algorithm that has become standard for reasoning-focused post-training since DeepSeek-R1. Understanding RECAP requires understanding exactly what GRPO optimizes and how multiple objectives enter the loss.

Base GRPO formulation. For a single reward signal, GRPO operates as follows. Given a prompt $x$ , the frozen rollout policy $\pi_{\theta_{\text{old}}}$ generates a group of $G$ complete responses $\mathcal{O} = \{o_i\}_{i=1}^G$ . Each response receives a verifiable sequence-level reward $R_i$ (e.g., exact-match correctness, format adherence score). The advantage for response $i$ is computed by normalizing within the group:

$\hat{A}_i = \frac{R_i - \text{mean}(R)}{\text{std}(R)}$

where $\text{mean}(R)$ and $\text{std}(R)$ are the empirical mean and standard deviation of rewards across the $G$ responses in the group for this prompt.

The GRPO objective then uses a PPO-style clipped surrogate. Let $r_{i,t}(\theta) = \frac{\pi_\theta(o_{i,t} \mid x, o_{i,<t})}{\pi_{\theta_{\text{old}}}(o_{i,t} \mid x, o_{i,<t})}$ be the token-level importance ratio between the current policy and the rollout policy. The GRPO objective is:

$J_{\text{GRPO}}(\theta) = \mathbb{E}\left[\frac{1}{G}\sum_{i=1}^G \frac{1}{|o_i|}\sum_{t=1}^{|o_i|} \min\left(r_{i,t}(\theta) \hat{A}_i,\; \text{clip}(r_{i,t}(\theta), 1 - \epsilon, 1 + \epsilon) \hat{A}_i\right) - \beta D_{\text{KL}}(\pi_\theta \parallel \pi_{\text{ref}})\right]$

where $\epsilon$ is the clipping threshold and $\beta$ is the KL penalty coefficient.

What it computes: For each token in each generated response, the importance ratio $r_{i,t}$ measures how much more or less likely the current policy is to produce that token compared to the rollout policy. The clipped surrogate prevents the policy from changing too drastically for any single token. The group-normalized advantage $\hat{A}_i$ determines the direction and magnitude of the update: if response $i$ is better than the group average (positive advantage), the policy is encouraged to make its tokens more likely; if worse (negative advantage), less likely. The KL penalty prevents the policy from drifting too far from the reference model. The entire expectation is over prompts $x$ and the generated responses.

Why this form: Group normalization removes the need for a separately trained value function (critic), which is expensive to train and introduces its own instability. By normalizing advantages within each prompt's response group, GRPO automatically calibrates the scale of updates—responses are judged relative to their peers for the same prompt, not against some absolute reward scale. This is particularly important when reward magnitudes vary across prompts (some problems are inherently harder than others). The clipping mechanism is the standard PPO innovation that maintains a trust region without explicitly constraining the policy divergence.

Multi-objective extension. In RECAP's setting, the model does not optimize a single reward. Instead, there are $K$ distinct objectives $\{L_k\}_{k=1}^K$ , each computed over data from $N$ domains. Some objectives are RL reward surrogates (accuracy, format, IoU), while others are supervised learning losses (next-token prediction). The paper formalizes this as:

$L^{(t)}_k = \frac{1}{N}\sum_{n=1}^N \ell^{(t)}_{n,k}(\theta)$

where $\ell^{(t)}_{n,k}(\theta)$ is the mini-batch loss of objective $k$ on domain $n$ at iteration $t$ . Note that $K \geq N$ because some tasks use more than one reward or objective—for example, a reasoning domain might contribute both an accuracy reward and a format reward, while a perception domain might contribute an IoU reward, a format reward, and a next-token prediction loss.

The standard approach (which the paper calls "Uniform") minimizes the unweighted average:

$L^{(t)}(\theta) = \frac{1}{K}\sum_{k=1}^K L^{(t)}_k$

RECAP replaces the uniform $1/K$ coefficients with dynamically computed weights $\lambda^{(t)}_k$ , which are the paper's core contribution.

The Heterogeneous Convergence Problem: Empirical Evidence

Before presenting the reweighting mechanism, the paper establishes why dynamic reweighting is necessary through direct observation of reward trajectories during training. Figure 4 (and the more detailed Figure 11 in the Appendix) plots the per-step values of five distinct objectives during a multi-domain RLVR run on the hybrid setting: next-token prediction accuracy, answer format reward, thinking format reward, IoU reward, and reasoning accuracy reward.

Format rewards saturate extremely fast. The thinking format reward (whether the model correctly places its reasoning in thinking... tags and its final answer in <answer>...</answer> tags) rises from approximately 0.6 to above 0.95 within the first 50 steps of training and remains near ceiling thereafter. The answer format reward shows similarly rapid convergence, reaching approximately 0.98–0.99 early in training. This is unsurprising: learning to follow a simple XML-like output template requires minimal capacity and the reward signal is deterministic and low-variance.

Reasoning accuracy converges slowly and remains noisy. In contrast, the reasoning accuracy reward starts near 0.2–0.3 and improves gradually over 300+ steps, oscillating substantially throughout. The per-step values exhibit high variance, with frequent dips and spikes even late in training. This is also unsurprising: mathematical reasoning is genuinely difficult, requiring the model to learn complex multi-step deduction strategies, and the reward signal (binary correct/incorrect) provides sparse feedback.

IoU and next-token prediction occupy an intermediate regime. The IoU reward (Intersection-over-Union for bounding box predictions on perception tasks) and the next-token prediction accuracy both show steady but slow improvement over training, with moderate variance.

The consequence of uniform weighting. Under standard uniform weighting ( $\lambda_k = 1/K$ ), every objective contributes equally to the gradient at every step. After step 50, the format reward objectives are essentially solved—the model consistently produces correctly formatted outputs, so the loss is near zero and gradients from these objectives are negligible. However, the presence of these objectives in the loss sum does not harm training directly; the issue is more subtle. The paper argues that when objectives are reweighted uniformly, the optimization lacks a mechanism to actively shift capacity toward objectives that still have headroom for improvement. The format reward's rapid convergence means that the model very quickly allocates whatever parameters are needed for format-following, and then stops receiving meaningful gradient signal from that objective. Meanwhile, the accuracy reward continues to provide gradient signal but with high variance. The optimization becomes dominated by the noisy accuracy gradient, which can cause destructive interference with the more stable gradients from IoU and next-token prediction objectives that are also still improving.

The paper's key diagnostic: convergence rate and instability diverge across objectives. Figure 4 plots the two signals RECAP uses—convergence rate $c^{(t)}_k$ and inverse signal-to-noise ratio $i^{(t)}_k$ —over the course of training. The format reward shows $c \to 1$ (no further improvement) and $i \to 0$ (no fluctuation) after roughly 50 steps. The reasoning accuracy reward maintains $c$ slightly above 1 and $i \approx 0.3$ throughout training, indicating continued slow improvement with substantial noise. The IoU and next-token prediction signals fall between these extremes. This divergence is the empirical foundation for RECAP: it says that a well-designed scheduler should notice that format has converged and reduce its weight, while noticing that accuracy is still improving and maintain or increase its weight.

The Per-Objective Convergence and Instability Signals

RECAP's scheduler operates on two scalar signals computed for each objective $k$ at each training step $t$ , using a sliding window of the objective's recent loss values. The window length is a hyperparameter $W$ (the paper does not specify the exact value used, but the method is described generically).

Current window average. The average loss for objective $k$ over the most recent $W$ steps:

$\mu^{(t)}_k = \frac{1}{W}\sum_{s=t-W+1}^{t} L^{(s)}_k$

where $L^{(s)}_k$ is the mini-batch loss of objective $k$ at step $s$ . This is an estimate of the objective's current performance level—lower values mean better performance (assuming all losses are minimization objectives).

Previous window average. The average loss over the preceding $W$ steps:

$\tilde{\mu}^{(t)}_k = \frac{1}{W}\sum_{s=t-2W+1}^{t-W} L^{(s)}_k$

This is a baseline against which the current window is compared. Together, $\mu^{(t)}_k$ and $\tilde{\mu}^{(t)}_k$ span a total lookback of $2W$ steps.

Window standard deviation (instability). The standard deviation of the loss within the current window:

$\sigma^{(t)}_k = \sqrt{\frac{1}{W-1}\sum_{s=t-W+1}^{t} \left(L^{(s)}_k - \mu^{(t)}_k\right)^2}$

This captures how much the objective fluctuates around its recent mean, which reflects both genuine learning dynamics (exploration, partial credit assignment) and stochastic noise from the on-policy sampling process.

What these statistics compute: The sliding window transforms the raw, per-step loss values—which are extremely noisy in on-policy RL (Appendix Figure 11 shows the total reward standard deviation peaking near 0.9 at step ~20)—into lower-variance estimates of trend and volatility. The choice of a window rather than per-step values is critical: per-step RL signals are too noisy to reliably distinguish genuine improvement from random fluctuation. By averaging over $W$ steps, the scheduler obtains a more stable signal that reflects the underlying learning dynamics rather than sampling noise.

Why this form: The two-window design (current vs. previous) enables computing a rate of change—is the loss improving, stable, or worsening?—without requiring a parametric model of the loss curve or assumptions about its functional form. The ratio-based convergence signal (next section) is naturally normalized: it does not depend on the absolute magnitude of the loss, making it directly comparable across objectives measured in entirely different units (e.g., cross-entropy in nats vs. IoU in [0, 1] vs. binary accuracy). This magnitude-agnostic property is essential for a multi-objective scheduler that must compare apples to oranges.

The paper notes that the window approach is motivated by the high-variance, non-stationary nature of RL signals (Henderson et al., 2018). Computing per-objective gradients—as some multi-task learning reweighting methods do (e.g., GradNorm, Guo et al., 2025)—would be "prohibitively expensive at LLM scale, especially under reinforcement learning" because it requires separate backward passes for each objective. In contrast, the window statistics require only tracking scalar loss values, which adds negligible computation.

The Convergence Rate and Instability Signals

From the window statistics, RECAP derives two dimensionless, normalized signals that capture complementary aspects of an objective's learning state.

Convergence rate. The ratio of the current window average to the previous window average:

$c^{(t)}_k = \frac{\mu^{(t)}_k}{\tilde{\mu}^{(t)}_k}$

What it computes: This ratio measures how much the loss has changed relative to its recent past. When $c^{(t)}_k < 1$ , the loss is decreasing (improving)—the current window is better than the previous one. The smaller $c^{(t)}_k$ is, the faster the improvement. When $c^{(t)}_k \approx 1$ , the loss is flat—the objective has reached a plateau. When $c^{(t)}_k > 1$ , the loss is increasing (worsening), which could signal overfitting, reward hacking, or destructive interference.

Why this form: The ratio is dimensionless and self-normalizing. An objective with loss values in [0, 100] and one with loss values in [0, 0.01] will both produce $c^{(t)}_k \approx 1$ when they stop improving, making their convergence states directly comparable. This magnitude-agnostic property is what enables the scheduler to treat heterogeneous objectives (cross-entropy, IoU, binary accuracy) on equal footing without per-objective normalization or scaling. An alternative approach—using the absolute difference $\mu^{(t)}_k - \tilde{\mu}^{(t)}_k$ —would require knowing in advance what constitutes a "large" improvement for each objective, which is precisely the kind of manual tuning the paper aims to avoid.

Inverse signal-to-noise ratio (instability). The standard deviation normalized by the sum of the two window means:

$i^{(t)}_k = \frac{\sigma^{(t)}_k}{\mu^{(t)}_k + \tilde{\mu}^{(t)}_k}$

What it computes: This is a coefficient of variation—it measures how large the fluctuations are relative to the average magnitude of the loss. A high $i^{(t)}_k$ means the objective is volatile: its per-step values bounce around substantially relative to the overall level. A low $i^{(t)}_k$ means the objective is stable: its values cluster tightly around the mean. The denominator uses $\mu^{(t)}_k + \tilde{\mu}^{(t)}_k$ rather than $2\mu^{(t)}_k$ to be symmetric with respect to the two windows—this matters when the two window averages differ substantially, which is precisely the situation where the convergence rate is far from 1.

Why this form: The normalization by the sum of means (rather than just the current mean) makes $i^{(t)}_k$ a measure of relative instability—how noisy is this objective compared to its own typical magnitude? This is important because raw standard deviation $\sigma^{(t)}_k$ alone would be confounded by the objective's scale. An IoU reward with $\sigma = 0.05$ and mean 0.5 is more relatively stable than an accuracy reward with $\sigma = 0.1$ and mean 0.3, even though the accuracy reward has higher absolute variance. The normalized instability captures this.

The paper's Figure 4 demonstrates how these two signals evolve over training. The format reward shows $c^{(t)}_k \to 1$ and $i^{(t)}_k \to 0$ within roughly 50 steps, reflecting rapid convergence to a stable, near-perfect score. The accuracy reward maintains $c^{(t)}_k$ slightly above 1 (indicating slow, ongoing improvement) and $i^{(t)}_k \approx 0.3$ (indicating substantial relative instability) throughout training. The IoU and next-token prediction signals fall in intermediate ranges.

The complementarity of $c$ and $i$ . The paper notes that these two signals provide complementary information. The convergence rate $c$ captures the trend—is the objective still improving? The instability $i$ captures the noisiness—how much stochastic fluctuation remains? An objective can have $c \approx 1$ (not improving) while still having high $i$ (fluctuating around its plateau), or it can have $c < 1$ (clearly improving) with low $i$ (smooth convergence). The scheduler needs both because they have different implications for how to allocate training capacity:

High $c$ , low $i$ : The objective is improving rapidly and smoothly. This is ideal—the current training allocation is working well, and the scheduler should maintain or slightly increase weight to capture the available improvement.
High $c$ , high $i$ : The objective is improving but noisily. This might warrant increased weight to push through the noise, or caution to avoid overfitting to stochastic fluctuations.
Low $c$ , low $i$ : The objective has converged and stabilized. Weight should be reduced—further optimization offers little benefit and may cause overfitting.
Low $c$ , high $i$ : The objective is not improving but remains noisy. This could indicate a difficult optimization landscape, potential reward hacking, or destructive interference from other objectives. The scheduler's behavior here is nuanced.

The paper ultimately combines $c$ and $i$ additively into a single priority score (Section 3.4, Equation 1), arguing that this simple combination performs consistently well while avoiding the complexity of learning an optimal combination function.

The Priority Score and Temperature-Controlled Reweighting

RECAP converts the per-objective convergence rate and instability into normalized weights via a two-step process: a priority score followed by a temperature-controlled softmax.

Priority score. For each objective $k$ at step $t$ :

$s^{(t)}_k = c^{(t)}_k + i^{(t)}_k$

What it computes: This is the unweighted sum of the convergence rate and the instability. Since $c^{(t)}_k$ starts above 1 (when the objective is improving) and decays toward 1 (when saturated), and $i^{(t)}_k$ is always non-negative (it is a normalized standard deviation), the priority $s^{(t)}_k$ is larger for objectives that are either still improving ( $c > 1$ ) or noisy ( $i$ large). A fully saturated, stable objective (like the format reward after 50 steps) has $c \approx 1$ and $i \approx 0$ , giving $s \approx 1$ —the minimum possible priority. An objective that is improving ( $c > 1$ ) and noisy ( $i$ large) gets a high priority score.

Why this form (additive combination): The paper explicitly considers the tradeoff between these two terms and notes that "finetuning the trade-off between those two terms offers finer-grained control and potentially improves the performance." However, they choose the simple unweighted sum "for simplicity, which performs consistently well in our experiments." The additive form has the desirable property that an objective with $c = 1$ but high $i$ still gets elevated priority (from the instability term alone), while an objective with $c$ well below 1 but low $i$ also gets elevated priority (from the convergence term alone). Neither signal can completely zero out the other, so both aspects of learning dynamics are always represented.

An alternative approach—multiplying $c$ and $i$ —would have the problematic property that if either term goes to zero, the priority goes to zero, even if the other term indicates the objective deserves attention. The additive form avoids this, ensuring that an objective must be both saturated ( $c \approx 1$ ) and stable ( $i \approx 0$ ) to receive minimum priority.

Temperature-controlled softmax normalization. The priority scores are converted to weight coefficients via:

$\lambda^{(t)}_k = \frac{K \exp(s^{(t)}_k / T)}{\sum_{i=1}^K \exp(s^{(t)}_i / T)}$

where $T > 0$ is the temperature hyperparameter and $K$ is the number of objectives.

What it computes: This applies a softmax over the $K$ priority scores, with temperature $T$ controlling the sharpness of the distribution. The prefactor $K$ ensures that the average weight across objectives is 1: $\frac{1}{K}\sum_{k=1}^K \lambda^{(t)}_k = 1$ . This means that when all objectives have equal priority (i.e., all $s^{(t)}_k$ are equal), the weights reduce to $\lambda^{(t)}_k = 1$ for all $k$ , recovering the uniform baseline.

At low $T$ (e.g., $T \to 0$ ): The softmax approaches a hard argmax—the objective with the highest priority gets weight $K$ and all others get weight 0. This would be an extreme allocation that likely causes instability.
At high $T$ (e.g., $T \to \infty$ ): The softmax approaches a uniform distribution—all weights approach 1. This recovers the uniform baseline exactly.
At $T = 5$ (the paper's default): The softmax produces a moderate skew: objectives with higher priority get weight above 1, those with lower priority get weight below 1, but all objectives retain some non-negligible weight.

The paper sets $T = 5$ by default, which the authors describe as providing a balance between adapting to convergence dynamics and maintaining training stability. They do not report a sweep over $T$ values, so the sensitivity of results to this choice is not characterized.

Why this form: The softmax has several desirable properties for this application. First, it naturally produces a probability distribution over objectives (up to the $K$ scaling), ensuring weights are positive and sum to $K$ , which preserves the average gradient magnitude relative to uniform weighting—RECAP reallocates gradient budget across objectives rather than changing the total budget. Second, the temperature provides a single, interpretable knob for controlling how aggressively the scheduler responds to priority differences. Third, the softmax is differentiable and can in principle be optimized, though the paper does not learn $T$ or the priority scores during training. Fourth, the prefactor $K$ means that the scheduler reduces to uniform weighting exactly at $T \to \infty$ , making it a conservative extension of standard practice—if the dynamics signals prove unreliable, increasing $T$ gradually reverts to the safe baseline.

The Overall Training Objective and Integration with GRPO

At each training step $t$ , the RECAP-augmented training objective is:

$L^{(t)}(\theta) = \frac{1}{K}\sum_{k=1}^K \lambda^{(t)}_k L^{(t)}_k$

where $L^{(t)}_k$ is the mini-batch loss for objective $k$ (computed as the average over all $N$ domains that contribute to that objective, as defined in Section 3.4.1), and $\lambda^{(t)}_k$ is the dynamically computed weight from the scheduler.

What it computes: This is a weighted sum of all $K$ objectives, where the weights are determined by the scheduler based on each objective's recent convergence behavior. The factor $1/K$ ensures that when all $\lambda_k = 1$ (uniform case), the total loss is the simple average $L^{(t)} = \frac{1}{K}\sum_k L^{(t)}_k$ , matching the standard uniform baseline exactly. When the scheduler assigns higher weight to some objectives and lower to others, the $1/K$ factor preserves the overall scale of the gradient—the sum of the coefficients $\frac{1}{K}\sum_k \lambda^{(t)}_k = 1$ , so the optimizer sees a total loss of the same typical magnitude regardless of the allocation.

The optimizer then computes $\nabla_\theta L^{(t)}$ and applies the standard GRPO update rule (clipped surrogate, KL penalty) to the parameters. Crucially, RECAP does not modify the GRPO algorithm itself—it only modifies the coefficients in the weighted sum of per-objective losses. This means RECAP can be dropped into any existing GRPO training loop by replacing the uniform loss averaging with the dynamically weighted version, without changing the optimizer, the rollout collection, the advantage computation, or any other component of the training pipeline.

Integration mechanics. In practice, the integration works as follows at each step:

Rollout collection: For each prompt in the batch, the frozen rollout policy generates $G$ responses. This step is identical to standard GRPO and is not affected by RECAP.
Reward and loss computation: For each response, all $K$ per-objective rewards or losses are computed. For RL objectives, this means computing sequence-level rewards (accuracy, format, IoU) and forming the GRPO surrogate loss. For supervised objectives, this means computing the standard cross-entropy next-token prediction loss.
Per-objective loss aggregation: For each objective $k$ , the loss $L^{(t)}_k$ is computed by averaging over all domains that contribute to that objective and all responses within the batch.
Scheduler update: The scalar $L^{(t)}_k$ values are appended to the sliding window for objective $k$ . The window statistics $\mu^{(t)}_k$ , $\tilde{\mu}^{(t)}_k$ , $\sigma^{(t)}_k$ are recomputed, then $c^{(t)}_k$ , $i^{(t)}_k$ , $s^{(t)}_k$ , and $\lambda^{(t)}_k$ are derived.
Weighted loss formation: The total loss is formed as the weighted sum $L^{(t)}(\theta) = \frac{1}{K}\sum_k \lambda^{(t)}_k L^{(t)}_k$ .
Parameter update: The gradient $\nabla_\theta L^{(t)}$ is computed via backpropagation, and the optimizer (AdamW) updates $\theta$ .

Why this integration is practical: The only additional computation compared to standard uniform-weighted GRPO is the window update and softmax computation (steps 4–5), which involves $O(K)$ scalar operations per step. For typical values of $K \approx 5$ –10, this is negligible compared to the cost of forward and backward passes through a billion-parameter model. The scheduler does not require additional models, gradient computations, or hyperparameter sweeps—the only new hyperparameter is the temperature $T$ , and the paper shows that the default value of 5 works across both experimental settings.

The scheduler in the limit. The paper notes two limiting behaviors:

As $T \to \infty$ : $\lambda^{(t)}_k \to 1$ for all $k$ , recovering the uniform baseline. This means RECAP can be gradually "turned off" by annealing $T$ upward during training, providing a safety mechanism if the scheduler's reweighting proves harmful.
As $W \to \infty$ : The window averages converge to the global mean and the convergence rate $c^{(t)}_k \to 1$ for all objectives (because the windows span the entire training history, drowning out recent changes). This also recovers uniform weighting. The window length $W$ must be chosen to be long enough to smooth out per-step noise but short enough to capture genuine changes in convergence behavior. The paper does not specify the exact $W$ used, leaving it as an implementation detail.

Domain-Specific Design Choices

Beyond the core reweighting mechanism, the paper makes several design decisions about how data is replayed and how objectives are structured that materially affect RECAP's behavior.

Uniform data sampling as the replay strategy. RECAP samples data uniformly across all domains at each training step, regardless of domain sizes. The paper explicitly compares this to "PropMix" (proportional sampling, where domains are sampled in proportion to their dataset sizes) in Table 2, showing that Uniform generally outperforms PropMix. This is consistent with findings in continual learning: uniform sampling prevents large domains from dominating the training distribution, which is especially important when the general-capability domains vary dramatically in size (Table 3 shows RefCOCO has 321,327 samples while LISA-train has only 1,326). Under proportional sampling, the model would see RefCOCO data roughly 240 times more often than LISA data, causing the perception signal to be dominated by referring expression comprehension at the expense of segmentation.

The paper notes that the data sampling strategy and the loss reweighting strategy are orthogonal: "to isolate the effect of replay and dynamic reweighting and also for the ease of convenient implementation, we uniformly sample across data sources by default and reweight only the objectives of interest." Uniform sampling ensures each domain gets equal representation in the batch, while the scheduler then decides how much gradient signal each domain's objectives contribute to the parameter update.

No thinking reward for perception tasks. A critical design choice emerged from observing model behavior during training. The paper initially followed the approach of MoDoMoDo (Liang et al., 2025) by applying a uniform "thinking reward" across all domains—rewarding the model for producing chain-of-thought reasoning in thinking... tags regardless of the task type. However, the authors observed that on perception tasks (e.g., segmentation, object detection), the model rapidly learned that explicit reasoning was unnecessary and even counterproductive. Figure 6 shows the thinking length on a segmentation task during training: the average response length drops from several hundred characters at the start to tens of characters (often near zero) later in training. The paper notes:

"This behavior suggests that explicit reasoning is unnecessary for such perception tasks and that encouraging long rationales can even be detrimental."

Motivated by this observation, the paper adopts a domain-dependent format reward structure in the hybrid setting:

Reasoning tasks (ThinkLite-VL-70k, GeoQAV, SAT-train, ScienceQA-train): The model receives a thinking format reward for correctly structuring its output as thinking... followed by <answer>...</answer>, plus an accuracy reward for getting the correct final answer.
Perception tasks (RefCOCO, LISA-train, LLaVA-OneVision-OCR): The model receives only an answer format reward (no thinking required) for placing its output in <answer>...</answer> tags, plus an IoU reward for bounding box accuracy or a next-token prediction loss for OCR.

This decision has two benefits. First, it prevents the format reward from incentivizing the model to waste tokens on unnecessary reasoning chains for simple perceptual tasks, which would increase inference cost without improving accuracy. Second, it means the format reward on perception tasks is purely about output structure, not about encouraging a particular reasoning style, which is more aligned with the ground-truth expectations for those tasks.

Disabling the KL penalty. Unless otherwise noted, the paper disables the reference-KL penalty term in GRPO ( $\beta = 0$ ) for most experiments. The authors state they do this "to disentangle the effectiveness of regularization approaches and our replay mechanism." This is an important experimental design choice because it means RECAP's capability preservation cannot be attributed to the standard KL constraint preventing policy drift—the method must work through the data replay and reweighting mechanisms alone. The "LwF" baseline (which uses KL with $\beta = 0.01$ ) is included as a point of comparison to evaluate whether KL alone is sufficient, and the results (Table 2) show that LwF underperforms replay-based methods on reasoning benchmarks, consistent with the claim that KL constrains plasticity without providing targeted capability preservation.

Objective set used in the hybrid setting. Based on Table 3 and the experimental description, the objectives in the large hybrid setting (RECAP on Qwen2.5-VL-7B) are:

Reasoning accuracy reward: Binary reward for correct final answer on reasoning domains (ThinkLite-VL-70k, GeoQAV, SAT-train, ScienceQA-train). This is an RL objective computed via GRPO's clipped surrogate.
Thinking format reward: Reward for correctly structuring the output with thinking... tags on reasoning domains. This is also an RL objective.
Answer format reward: Reward for correctly placing the final answer in <answer>...</answer> tags, applied across all domains. This is an RL objective.
IoU reward: Intersection-over-Union between predicted and ground-truth bounding boxes on perception tasks (RefCOCO, LISA-train). This is an RL objective.
Next-token prediction loss: Standard supervised cross-entropy loss on OCR data (LLaVA-OneVision-OCR). This is a supervised objective, not an RL objective.

The total number of objectives $K = 5$ , though the paper notes that not all objectives apply to every domain—the IoU reward is only computed on perception domains, the thinking format reward only on reasoning domains, etc. The aggregation $L^{(t)}_k = \frac{1}{N}\sum_{n=1}^N \ell^{(t)}_{n,k}(\theta)$ handles this naturally by including only the domains where objective $k$ is defined.

Training hyperparameters. For the hybrid setting on Qwen2.5-VL-7B:

Optimizer: AdamW with $\beta_1 = 0.9$ , $\beta_2 = 0.999$ , $\epsilon = 10^{-8}$
Learning rate: linear schedule with 10% warmup to $\eta_{\max} = 1 \times 10^{-6}$ , then linear decay to 0
Precision: bfloat16 with FlashAttention kernels
Training steps: 500
Hardware: 8 GPUs with data parallelism
Per-device batch size: 1, gradient accumulation steps: 2 (effective batch size 16)
Rollouts per prompt: 4 (64 rollouts per optimizer step)
GRPO clipping: $\epsilon$ (not specified, but standard GRPO uses $\epsilon = 0.2$ )
KL penalty: $\beta = 0$ (disabled unless noted)

For the RLVR-only setting on Qwen2.5-VL-3B:

Per-device batch size: 2
Rollouts per prompt: 4
Training continues until data from a particular domain is exhausted
Other hyperparameters follow the same configuration as the hybrid setting

Evaluation protocol. The paper uses LMMS-Eval for standardized evaluation across all benchmarks. For reasoning tasks, the model is prompted to produce thinking traces (structured chain-of-thought) in the training data, but the evaluation metrics are based solely on final answer correctness, not on the quality of the reasoning chains. For perception tasks, the appropriate output format (bounding box coordinates, natural language answers) is enforced, and standard task-specific metrics (IoU for segmentation, accuracy for VQA, etc.) are used.

4. Key Insights and Innovations

Innovation 1: Reframing Catastrophic Forgetting in RLVR as a Multi-Objective Convergence Mismatch, Not Policy Drift

The dominant framing for capability loss during post-training—inherited from RLHF (Ouyang et al., 2022) and standard continual learning (Kirkpatrick et al., 2017)—casts the problem as policy drift: the fine-tuned model moves too far from the base model, and the fix is to constrain that movement, typically via KL divergence penalties. This framing implicitly assumes that if the policy is prevented from shifting too much on the training distribution, it will retain its behavior on held-out distributions. The paper's essential conceptual move is to reject this framing as insufficient and replace it with a different diagnosis: capability regression in multi-objective RLVR is not primarily about how far the policy moves, but about which objectives receive gradient signal and when.

The evidence for this reframing is in what KL regularization fails to do. The LwF baseline in Table 2 applies a standard KL penalty ( $\beta = 0.01$ ) during training but underperforms replay-based methods on reasoning benchmarks (29.59 vs. 31.91 on MMMU-PRO for Uniform, 34.15 for RECAP) while matching or slightly trailing on perception tasks. If policy drift were the primary mechanism, KL should be the most direct fix—yet it produces worse reasoning performance than simply replaying general data without any reweighting. The paper's interpretation, supported by Figure 4, is that KL constrains the total magnitude of parameter updates but does nothing to redirect gradient budget within that constraint toward objectives that still have headroom for improvement. The model can stay close to the reference in KL-divergence while still allocating nearly all its effective learning capacity to format rewards that saturate in 50 steps, starving the slower-converging accuracy and perception objectives of gradient signal.

This reframing matters because it redirects the research agenda. If forgetting is fundamentally about policy drift, the solution space is regularization: better KL penalties, Elastic Weight Consolidation (Kirkpatrick et al., 2017), Synaptic Intelligence (Zenke et al., 2017), or other parameter-importance methods that constrain movement on critical weights. These approaches have been extensively explored in continual learning and, the paper argues implicitly, are a limited fit for RLVR because they constrain uniformly rather than targeting specific capability gaps. If forgetting is instead about asynchronous objective convergence—some rewards saturate early and then crowd out others from the gradient—the solution space shifts toward dynamic resource allocation: online schedulers that monitor convergence rates and reallocate gradient budget in real time.

This is a fundamental conceptual shift, not an incremental refinement of existing replay methods. Prior work on replay in continual learning (Rebuffi et al., 2017; Rolnick et al., 2019) treats replay as a mechanism for exposing the model to old data, with the implicit assumption that uniform exposure is sufficient. RECAP's innovation is recognizing that exposure alone is not enough when objectives converge asynchronously—the model can see perception data at every step but still stop learning from it if the loss signals from that data are swamped by higher-magnitude or higher-variance gradients from other objectives. The reweighting scheduler is not an alternative to replay; it is a necessary complement that makes replay effective by ensuring that gradient signal from slowly-converging objectives is not crowded out.

The paper does not articulate this reframing in explicit philosophical terms, but the architecture of the method—the choice to disable KL, the focus on convergence rates rather than parameter distances, the design of the priority score as a function of improvement speed rather than absolute loss magnitude—embodies it. This is what separates RECAP from work like InstructGPT's PPO-ptx (Ouyang et al., 2022; Zheng et al., 2023), which interleaves pretraining gradients with RLHF updates as a regularization term: PPO-ptx adds a constant-magnitude supervised signal alongside RL, which helps but does not adapt to which capabilities are actually being forgotten. RECAP adds a signal that grows when capabilities are at risk of being lost and shrinks when they are secure, making the intervention proportional to the problem.

Innovation 2: Magnitude-Agnostic, Gradient-Free Objective Scheduling for Heterogeneous Reward Landscapes

Multi-objective optimization in deep learning typically requires solving a difficult meta-problem: how to combine losses measured in incompatible units (cross-entropy in nats, IoU in [0,1], binary accuracy, RL surrogate losses with unknown scale) into a single scalar that gradient descent can optimize. The standard solutions all have significant drawbacks in the RLVR context:

Manual coefficient tuning requires expensive sweeps and breaks when new objectives are added.
Gradient-based reweighting methods (e.g., GradNorm, uncertainty weighting, Guo et al., 2025) require computing per-objective gradients separately, which is "prohibitively expensive at LLM scale, especially under reinforcement learning" because each objective needs its own backward pass.
Loss-scale normalization (e.g., dividing each loss by its initial value or running average) is fragile to distribution shift during training and requires choosing a normalization baseline.

The paper's distinctive conceptual contribution is recognizing that for the specific problem of asynchronous convergence in RLVR, absolute loss magnitudes are irrelevant—what matters is the rate of change and the stability of each objective, both of which can be measured from scalar loss histories alone, without gradients, without normalization, and without knowing what units the losses are in.

The key design insight is the use of ratios rather than differences. The convergence rate $c^{(t)}_k = \mu^{(t)}_k / \tilde{\mu}^{(t)}_k$ compares the current window average to the previous window average via division, not subtraction. This means it is automatically dimensionless—an IoU loss improving from 0.5 to 0.4 and a cross-entropy loss improving from 2.3 to 1.8 both register as improvement, regardless of the numerical gap. Similarly, the instability $i^{(t)}_k = \sigma^{(t)}_k / (\mu^{(t)}_k + \tilde{\mu}^{(t)}_k)$ is a coefficient of variation that measures noise relative to signal amplitude. These ratio-based signals are what make the scheduler "magnitude-agnostic"—the paper explicitly claims this property, and it is what enables the method to work across RL and supervised objectives, binary and continuous rewards, without any per-objective calibration.

This is a genuinely novel contribution to the multi-task learning and continual learning literatures. Prior work on dynamic loss weighting—from uncertainty-based methods (Kendall et al., 2018) to gradient-magnitude balancing (Chen et al., 2018) to Pareto-optimal methods (Sener & Koltun, 2018)—relies on either absolute loss values, gradient norms, or both. These methods implicitly assume that losses can be compared in their raw units, which requires them to be similar in scale or normalized. RECAP sidesteps this entire problem by operating on relative change rather than absolute level, making it applicable to objective sets that standard multi-task weighting methods cannot handle without extensive preprocessing.

The practical significance of this magnitude-agnostic property extends beyond the paper's specific experiments. Modern post-training pipelines increasingly combine heterogeneous objectives: RL rewards for reasoning, DPO-style preference losses for safety, supervised next-token prediction for knowledge retention, auxiliary losses for specific capabilities. A scheduler that requires per-objective hyperparameter tuning for each new objective scales poorly as objective sets grow. RECAP's magnitude-agnostic design means adding a new objective requires no additional coefficient tuning—the scheduler automatically incorporates it based on its observed convergence dynamics. This is a qualitatively different scalability property than any prior method, and while the paper does not test it with more than ~5 objectives, the design principle is clear.

The decision to use sliding-window statistics rather than per-step signals is also a non-trivial design insight, though the paper treats it as practical engineering. On-policy RL rewards are notoriously high-variance and non-stationary (Henderson et al., 2018), with Appendix Figure 11 showing the total reward standard deviation peaking near 0.9 around step 20. Any scheduler operating on per-step values would be dominated by noise, reweighting objectives based on random fluctuations rather than genuine convergence trends. The two-window design ( $2W$ steps of history, split into current and previous windows) is a lightweight smoothing mechanism that filters out per-step noise while remaining responsive enough to detect genuine changes in convergence behavior within tens of steps—fast enough to catch format rewards saturating and slow enough to avoid chasing noise. This temporal scale is not obvious a priori and represents a design choice that the paper validates empirically rather than through ablation (no sweep over $W$ is reported).

Innovation 3: The Convergence-Instability Duality as a Sufficient Signal for Capability-Aware Resource Allocation

The paper's most theoretically interesting move is the claim—implicit in the priority score design $s^{(t)}_k = c^{(t)}_k + i^{(t)}_k$ —that convergence rate and instability together constitute a sufficient statistic for determining which objectives need gradient budget. This is not an obvious claim. Why should the speed at which a loss improves and the noise in that improvement tell us everything we need to know about whether to allocate more or less capacity to that objective?

The paper's empirical answer is Figure 4 and the attendant analysis. The format reward exhibits both convergence ( $c \to 1$ ) and stabilization ( $i \to 0$ ) after ~50 steps, and it is indeed the case that further optimization of the format reward after this point yields no benefit—the model already formats correctly on nearly 100% of outputs. The accuracy reward maintains $c$ slightly above 1 and $i \approx 0.3$ throughout training, and it is indeed the case that accuracy continues to benefit from sustained optimization. The IoU and next-token prediction objectives occupy intermediate regimes in both signals, and they benefit from intermediate levels of sustained attention. The two-dimensional signal space (convergence × instability) maps cleanly onto the one-dimensional decision of how much weight to assign, and the additive combination $c + i$ turns out to be sufficient for this mapping.

This is a conceptual contribution to the theory of multi-objective training dynamics. The field has long understood that different objectives in multi-task learning converge at different rates, but the standard response has been to either (a) tune weights to equalize convergence speeds, or (b) use gradient-based methods to balance the magnitude of updates. RECAP's innovation is to treat convergence rate not as a problem to be equalized but as a signal to be read—it is precisely because objectives converge at different rates that we can infer which ones need more attention. Fast convergence means the objective is easy and saturating; slow convergence or high noise means the objective is hard and needs sustained gradient budget. The scheduler does not try to make all objectives converge at the same rate; it tries to allocate gradient budget to the objectives that are currently benefiting from it, which is a fundamentally different optimization philosophy.

The paper provides suggestive evidence that this approach works better than trying to equalize convergence through static weights. The MoDoMoDo baseline in Table 1 represents the "equalize convergence" philosophy: it uses proxy models to learn optimal mixing ratios that presumably account for differential convergence. RECAP matches or exceeds MoDoMoDo while requiring no proxy model training and no manual reward coefficient tuning. This suggests that the adaptive nature of the reweighting—responding to convergence as it happens rather than predicting it in advance—provides a benefit beyond what even an optimized static mixture can achieve, because the optimal allocation changes during training in ways that a single mixing ratio cannot capture.

The additive combination of $c$ and $i$ is also a non-trivial design choice with interesting theoretical implications. The two signals capture complementary aspects of learning dynamics: $c$ captures the trend (is the objective still improving?), while $i$ captures the variability (how noisy is the improvement?). An objective can have $c \approx 1$ with high $i$ (plateaued but noisy—this might indicate reward hacking or optimization difficulties), or $c < 1$ with low $i$ (smoothly improving—ideal). The additive form means that both signals independently contribute to priority: a noisily plateaued objective still gets elevated weight from its instability, and a smoothly improving objective still gets elevated weight from its convergence. The paper notes that "finetuning the trade-off between those two terms offers finer-grained control and potentially improves the performance" but deliberately chooses the unweighted sum for simplicity. This is a design decision that privileges robustness over optimality—the scheduler works well across settings without requiring a tuned mixing coefficient between $c$ and $i$ .

Innovation 4: The Finding That Replaying General Data Reverses Reasoning Chain Inflation

Beyond the scheduler mechanism itself, the paper surfaces an empirical finding with practical significance: replaying general-capability data during RLVR training substantially reduces reasoning chain length without compromising accuracy. Figure 7 tracks the length of generated thinking traces on reasoning tasks, comparing a model trained solely on reasoning data against RECAP. The reasoning-only model maintains chains averaging ~67.3 words with high variability; RECAP reduces this to ~27.3 words—a roughly 60% reduction—while matching or exceeding the reasoning-only model's accuracy.

This finding is not the paper's main contribution, but it is conceptually significant because it challenges a tacit assumption in the reasoning-model literature: that long chain-of-thought traces are a natural and necessary byproduct of improved reasoning, to be accepted or even encouraged as evidence that the model is "thinking harder." The paper's evidence suggests an alternative interpretation: excessive chain-of-thought length is partially an artifact of training distribution narrowness. When the model sees only reasoning problems during RLVR, it learns that verbose reasoning is always appropriate, inflating its traces even on problems that require only modest deduction. When general-capability data is replayed—including perception tasks where the model learns that direct answers are better—this verbosity bias is counterbalanced, and the model learns to calibrate its reasoning length to problem difficulty.

This finding connects to broader concerns about reasoning model efficiency. Long chain-of-thought traces increase inference latency and compute cost linearly with token count. If the same accuracy can be achieved with 60% fewer tokens, that represents a direct inference cost reduction. Moreover, shorter traces tend to be more focused and less prone to the degeneracies (repetition, circular reasoning, hallucinated intermediate steps) that plague very long chain-of-thought generations. The paper does not extensively characterize the quality of the shortened traces, but the maintained or improved accuracy scores imply that the reduction is not achieved by truncating useful reasoning—the model is genuinely learning to be more concise without losing correctness.

The mechanism behind this effect is not fully explained by the paper, but a plausible interpretation emerges from the training dynamics: perception tasks with "no-thinking" format rewards teach the model that thinking... is inappropriate for some inputs. When these tasks are replayed alongside reasoning tasks, the model learns a conditional policy—use thinking for math, skip it for object detection—rather than a global policy of always thinking. This conditional policy then generalizes to produce shorter thinking traces on reasoning problems where the answer is relatively straightforward, because the model has learned that thinking length should be proportional to task difficulty. The unified objective space created by replay enables this calibration; a reasoning-only training distribution provides no signal that thinking can ever be too long.

This is a negative result about current practice—reasoning-only RLVR inflates chain length unnecessarily—masquerading as a positive result about RECAP. It suggests that the field's focus on reasoning benchmarks as the sole training signal for reasoning models is producing models that reason verbosely not because verbosity helps accuracy but because the training distribution provides no countervailing pressure toward concision. RECAP's general-capability replay provides that pressure organically, without an explicit brevity reward or length penalty.

5. Experimental Analysis

Evaluation Methodology

Dataset. The paper uses multiple datasets organized into reasoning and general-capability domains. In the RLVR-Only Setting (Qwen2.5-VL-3B), training data follows the MoDoMoDo configuration (Liang et al., 2025), with evaluation on six reasoning benchmarks: SAT (Ray et al., 2024a), ScienceQA (Lu et al., 2022a), MathVista-mini (Lu et al., 2023), ChartQA (Masry et al., 2022), InfoVQA (Mathew et al., 2022), and MMMU (Yue et al., 2023). In the Hybrid Setting (Qwen2.5-VL-7B), training uses ThinkLite-VL-70k (Wang et al., 2025c) for reasoning plus RefCOCO (Kazemzadeh et al., 2014), LLaVA-OneVision OCR (Li et al., 2024), LISA-train (Lai et al., 2023), GeoQAV (Li et al., 2025a), SAT-train (Ray et al., 2024b), and ScienceQA-train (Lu et al., 2022b) for general capabilities. Table 3 provides exact dataset sizes, ranging from 1,326 samples (LISA-train) to 321,327 samples (RefCOCO). Evaluation in the hybrid setting spans nine benchmarks: LISA (Lai et al., 2024), MMMU-Pro (Yue et al., 2024), AI2D (Kembhavi et al., 2016), MathVista, MathVision (Liu et al., 2024c), MathVerse (Zhang et al., 2024b), MMBench (Liu et al., 2024a), VizWiz (Gurari et al., 2018), and OCRBench v2 (Fu et al., 2024). Notably, in the RLVR-only setting, the paper uses a rule-based evaluator on MathVista rather than GPT-3.5-turbo to align with MoDoMoDo's protocol (Table 1 note).
Base model(s). Experiments use two model scales from the Qwen2.5-VL family: Qwen2.5-VL-3B for the smaller RLVR-only setting and Qwen2.5-VL-7B for the larger hybrid setting. The paper chooses these models because they are widely used open-source vision-language models that have been extensively reasoning-finetuned by the community (as shown in Figure 1), providing both a natural baseline comparison and a representative testbed for evaluating capability regression. The 7B scale additionally enables testing in a more realistic, large-scale post-training regime.
Metrics. The primary metric across all experiments is accuracy—the fraction of test questions where the model's final answer matches the ground truth, as determined by standard benchmark evaluators. For segmentation tasks (LISA), the metric is Intersection-over-Union (IoU) between predicted and ground-truth bounding boxes. For OCR and general VQA tasks, standard accuracy scoring is used. All evaluations are conducted through LMMS-Eval (Zhang et al., 2024a) to ensure consistency and reproducibility. The paper also tracks auxiliary training metrics: format reward values (both thinking and answer format), IoU reward, reasoning accuracy reward, and next-token prediction accuracy, reported as per-step curves or final values.
Baselines. The paper establishes five distinct baselines to isolate different aspects of the method:
1. Reasoning-only: Train solely on the target reasoning task with fixed reward weights, no general-capability data replay. This is the most aggressive baseline and shows the maximum forgetting.
2. Uniform: Sample data uniformly across all domains (reasoning + general capability) and compute losses with uniform weighting ( $\lambda_k = 1$ for all $k$ ). This isolates the effect of data replay alone, without dynamic reweighting.
3. PropMix: Sample data in proportion to each domain's dataset size. Losses are not reweighted. This tests whether proportional sampling (which heavily weights large domains) outperforms uniform sampling.
4. Coreset: Replay a size-limited subset of general data, set to half the reasoning-data volume, following standard coreset replay methods (Rebuffi et al., 2017; Chaudhry et al., 2019b). This tests whether a small amount of replay data is sufficient.
5. LwF (Learning without Forgetting): Sample data uniformly but add a KL regularization term with coefficient $\beta = 0.01$ toward the reference policy. This represents the standard RLHF stabilization approach (Li & Hoiem, 2017; Ouyang et al., 2022) and tests whether KL alone can prevent forgetting.
Additionally, in the RLVR-only setting (Table 1), MoDoMoDo (Liang et al., 2025) serves as an upper-bound static mixture baseline that trains proxy models to learn optimal data mixing ratios and uses hand-tuned reward weights. For broader context, the paper also reports scores for several open-source reasoning models (VLAA-Thinker, MM-R1-MGT-PerceReason, Ocean_R1_3B_Instruct, Qwen2.5VL-3b-RLCS, vision-grpo-qwen-2.5-vl-3b, Qwen2.5-VL-3B-Instruct-GRPO-deepmath for the 3B setting; VLAA-Thinker-7B, Vision-R1-7B, OpenVLThinker-7B for the 7B setting), noting that these are listed for benchmarking context rather than as a direct comparison target, since they often employ complex, multi-stage training pipelines.
Generation budget / compute accounting. Compute is measured in training steps, not inference FLOPs or generation budget, since the paper's focus is on training-time dynamics rather than test-time compute scaling. In the RLVR-only setting (Qwen2.5-VL-3B), training continues until data from a particular domain is exhausted, with 8 GPUs, per-device batch size 2, and 4 rollouts per prompt. In the hybrid setting (Qwen2.5-VL-7B), each model is trained for exactly 500 steps on 8 GPUs with per-device batch size 1, 2 gradient accumulation steps (effective batch size 16), and 4 rollouts per prompt (64 rollouts per optimizer step). This fixed-budget design ensures fair comparison across methods—all variants receive exactly the same number of optimizer updates, and any performance differences arise from how those updates are allocated across objectives, not from seeing more data. The GRPO algorithm's group-based advantage computation means each prompt produces $G=4$ complete responses per rollout, with rewards computed per response.
Cross-validation / statistical protocol. The paper does not report cross-validation or statistical significance testing. This is a notable methodological gap. The test sets are used as-is (standard benchmark splits), and results are reported as single-point accuracy scores without confidence intervals. For the RLVR-only setting with the smaller test sets (e.g., SAT, ScienceQA), score variation across runs is not characterized. The paper does use separate training and evaluation datasets following standard benchmark protocols, but the absence of error bars or multiple-seed results makes it difficult to assess whether observed differences (e.g., RECAP at 55.19 vs. MoDoMoDo at 49.95 on SAT in Table 1) are statistically reliable or within run-to-run variance.

Main Quantitative Results

RLVR-Only Setting: RECAP vs. Static Mixture and Open-Source Baselines (Table 1)

The RLVR-only setting tests whether dynamic reweighting improves over static mixture approaches when all objectives are RL-based. Table 1 reports accuracy on six reasoning benchmarks for Qwen2.5-VL-3B variants and open-source models.

Headline result: RECAP achieves the highest or competitive scores across all six benchmarks, substantially outperforming the Uniform baseline and the MoDoMoDo upper-bound static mixture.

On SAT, RECAP reaches 55.19%, compared to 44.55% for Uniform, 49.95% for MoDoMoDo, and 43.98% for the base model. This represents a 10.64 point absolute improvement over Uniform and a 5.24 point improvement over MoDoMoDo, the strongest static mixture baseline. Among open-source models, the best performer (Ocean_R1_3B_Instruct at 59.49%) scores higher, but RECAP exceeds all other open-source variants.

On ScienceQA, RECAP achieves 71.59%, compared to 64.85% for Uniform, 65.74% for MoDoMoDo, and only 6.20% for the base model. This is a dramatic 65-point improvement over the base model—the paper notes "RL lifts the performance of Qwen2.5-VL-3B from 6 to 60" on this benchmark—with RECAP pushing it substantially further. RECAP also surpasses all open-source models except Ocean_R1_3B_Instruct (68.72%).

On MathVista (mini), RECAP scores 33.2% vs. 32.4% (Uniform), 32.2% (MoDoMoDo), and 23.6% (base model). The margin over baselines is smaller here (0.8–1.0 points), but the consistent improvement over both Uniform and MoDoMoDo across all three methods with different reweighting philosophies (none, learned static, dynamic online) suggests the scheduler provides incremental benefit even when the primary gains come from RL training itself.

On ChartQA, RECAP achieves 70.40%, tied with MoDoMoDo and slightly ahead of Uniform (69.68%) and the base model (43.88%). On InfoVQA, RECAP scores 60.78% vs. 59.88% (MoDoMoDo), 58.30% (Uniform), and 32.02% (base). On MMMU, RECAP reaches 42.44% vs. 39.44% (Uniform), 39.11% (MoDoMoDo), and 38.67% (base).

Key pattern: RECAP's advantage over Uniform is largest on benchmarks where format reward saturation is most likely to crowd out accuracy signal. SAT and ScienceQA show the largest gaps (10.64 and 6.74 points, respectively), while MathVista and ChartQA show smaller gaps (0.8 and 0.72 points). This is consistent with the scheduler's mechanism: on benchmarks where the format reward saturates quickly and the accuracy reward remains difficult, RECAP's down-weighting of format and up-weighting of accuracy should produce the largest gains. The paper does not break out per-benchmark format and accuracy reward dynamics to confirm this interpretation, but the pattern is suggestive.

Comparison to open-source models. RECAP outperforms most open-source reasoning variants on most benchmarks. The notable exception is Ocean_R1_3B_Instruct, which leads on SAT (59.49% vs. 55.19%) and MathVista (38.7% vs. 33.2%) but trails substantially on ScienceQA (68.72% vs. 71.59%) and InfoVQA (38.02% vs. 60.78%). The paper appropriately notes that these open-source models "often undergo many complicated training pipelines" and are not direct comparisons—RECAP's value is in showing that a simple plug-in scheduler added to a basic RLVR pipeline can match or exceed specialized models on many benchmarks.

Critical observation: MoDoMoDo is treated as an upper bound, yet RECAP matches or exceeds it without proxy model training. MoDoMoDo requires training multiple proxy models of the same size as the baseline to learn the mapping from mixing ratio to test performance, plus hand-tuned reward weights (the paper notes MoDoMoDo "doubl[es] accuracy and IoU relative to formatting rewards"). RECAP achieves comparable or better performance with zero proxy model training, zero manual coefficient tuning, and zero per-benchmark optimization—the scheduler uses the same hyperparameters ( $T=5$ , unspecified $W$ ) across all benchmarks. This is a substantial practical advantage, though Table 1 only shows results for one run, making it unclear whether RECAP would maintain this parity across multiple seeds or hyperparameter configurations.

Hybrid Setting: General Capability Preservation and Reasoning Performance (Table 2)

The hybrid setting is the paper's primary testbed, evaluating whether RECAP can simultaneously preserve general capabilities (perception, OCR, VQA) and improve reasoning in a realistic post-training regime combining RLVR and SFT objectives. Table 2 reports accuracy on nine benchmarks for Qwen2.5-VL-7B variants.

Headline result: RECAP achieves best or second-best performance on 8 of 9 benchmarks among Qwen2.5-VL-7B variants, preserving general capabilities while matching or exceeding reasoning-only training on reasoning metrics.

On LISA (segmentation, the primary general-capability benchmark highlighted in Figure 2), RECAP achieves 67.24% IoU—the highest among all variants and a 2.11 point absolute improvement over the base model (65.13%). This directly addresses the paper's motivating observation in Figure 2, where reasoning-only training caused LISA performance to drop from 65.1% to 57.6%. All replay-based methods preserve LISA performance at roughly base-model levels (Uniform: 65.18%, PropMix: 66.80%, Coreset: 64.82%), confirming that replay alone prevents forgetting. But RECAP goes further, improving over the base model, which the scheduler's reweighting uniquely achieves—the uniform baseline merely maintains performance while RECAP actively improves it.

In contrast, Reasoning-only drops to 57.58% on LISA, confirming the catastrophic forgetting that motivates the paper—a 7.55-point absolute degradation from the base model. This is the clearest single-number demonstration of the problem RECAP solves.

On MMMU-Pro (multimodal reasoning), RECAP scores 34.15%—the highest among all Qwen2.5-VL-7B variants, ahead of Uniform (31.91%), Reasoning-only (33.87%), and LwF (29.59%). RECAP also exceeds all open-source baselines (best: VLAA-Thinker-7B at 26.30%). The gap between RECAP and Uniform (2.24 points) is notable because MMMU-Pro is a reasoning benchmark, not a general-capability benchmark—this demonstrates that capability preservation through replay does not come at the expense of reasoning improvement; it can actually enhance it.

On AI2D (diagram understanding), Coreset achieves the highest score (79.92%), with RECAP second at 78.21%. Uniform scores 76.43%, Reasoning-only scores 74.97%, and the base model scores 67.62%. RECAP's score represents a substantial improvement over the base model while maintaining competitiveness with the best static replay variant. The Coreset's strong performance here suggests that for certain visual understanding tasks, a smaller amount of diverse replay data may be sufficient or even preferable—though the paper does not analyze this result further.

On MathVista, RECAP achieves 66.70%, second to Coreset (66.90%) and ahead of Uniform (65.60%), Reasoning-only (65.50%), and PropMix (63.40%). On MathVision, RECAP scores 25.11% vs. 22.13% (Uniform), 24.87% (Reasoning-only), and 23.36% (Coreset). On MathVerse, RECAP achieves 40.83% vs. 36.07% (Uniform), 40.74% (Reasoning-only), and 37.58% (Coreset). Across all three math-in-vision benchmarks, RECAP either leads or is within one point of the leader, and consistently outperforms the Uniform baseline—the most direct comparison for isolating the effect of dynamic reweighting.

On MMBench (general multimodal understanding), RECAP scores 78.52%, second to Coreset (78.09%) and ahead of Uniform (75.34%). On VizWiz (accessibility-focused VQA), RECAP scores 61.97%, behind Coreset (63.76%) and Reasoning-only (62.45%) but substantially ahead of the base model (50.82%) and Uniform (54.05%). On OCRBench v2 (text recognition), RECAP achieves 39.72%, slightly above Uniform (38.06%) and the base model (39.49%).

The Uniform vs. RECAP comparison is the cleanest test of dynamic reweighting. Both methods use identical data sampling (uniform across domains), identical training steps, identical optimizer settings—the only difference is RECAP's scheduler for computing per-objective weights $\lambda_k$ vs. Uniform's fixed $\lambda_k = 1$ . Across all nine benchmarks, RECAP outperforms Uniform on 8 (all except VizWiz, where Uniform scores 54.05% vs. RECAP's 61.97%—wait, RECAP is actually higher here; both Coreset at 63.76% and RECAP at 61.97% exceed Uniform at 54.05%). The consistent improvement over Uniform—ranging from 0.28 points (ChartQA-style benchmarks) to 2.24 points (MMMU-Pro)—provides direct evidence that the scheduler's reweighting, not just data replay, drives the performance gains.

The LwF comparison isolates KL regularization vs. replay. LwF uses KL penalty ( $\beta = 0.01$ ) but no explicit general-capability replay. On MMMU-Pro, LwF scores 29.59% vs. RECAP's 34.15% (4.56-point gap). On AI2D, LwF scores 73.93% vs. RECAP's 78.21%. On MathVerse, LwF scores 33.98% vs. RECAP's 40.83%. These gaps support the paper's claim that KL regularization alone is insufficient—it prevents some policy drift but does not actively preserve or improve general capabilities the way replay + reweighting does. However, LwF does outperform the base model on most benchmarks, indicating that KL provides some benefit—just less than RECAP.

Comparison to open-source reasoning models. RECAP (Qwen2.5-VL-7B) outperforms VLAA-Thinker-7B, Vision-R1-7B, and OpenVLThinker-7B on most benchmarks, often by substantial margins. On MMMU-Pro, RECAP's 34.15% exceeds the best open-source model (Vision-R1-7B at 26.76%) by 7.39 points. On MathVerse, RECAP's 40.83% exceeds VLAA-Thinker-7B (29.87%) by 10.96 points. These comparisons must be interpreted cautiously—the open-source models use different base models and training pipelines—but they demonstrate that RECAP with a standard base model and minimal additional engineering is competitive with specialized, heavily engineered reasoning models.

Training Dynamics: Format vs. Accuracy Reward Allocation (Figure 5)

Figure 5 provides a window into how RECAP's reweighting affects training dynamics, comparing the evolution of format and accuracy rewards between RECAP and the Uniform baseline on the reasoning domain.

Format reward trajectory (Figure 5, left and middle): In early training (first ~40 steps), the Uniform baseline climbs faster on the format reward, reaching higher scores sooner. This is consistent with the Uniform baseline allocating equal weight to the format reward even as it saturates—the model optimizes the easy, low-variance format signal aggressively. RECAP initially lags because its scheduler has not yet accumulated enough window history to detect format saturation. Around step 40, a crossover occurs, and RECAP's format reward surpasses the Uniform baseline's, remaining higher for the rest of training. The paper interprets this as the scheduler successfully down-weighting the format reward once it detects convergence ( $c \to 1$ , $i \to 0$ ), which prevents over-optimization of format at the expense of other objectives. The Uniform baseline, by continuing to weight format equally, may overfit to format-specific patterns that actually reduce format adherence later—the slight decline in the Uniform baseline's format curve after step 200 is consistent with this interpretation.

Accuracy reward trajectory (Figure 5, right): RECAP maintains a growing lead in accuracy reward over the Uniform baseline throughout training. At step 500, the gap is visually substantial (the paper does not report final exact values in the text for Figure 5, but the curves show RECAP at approximately 0.4+ vs. Uniform at approximately 0.35). This is the key dynamic: by shifting weight away from saturated format and toward the still-improving accuracy objective, RECAP allocates more gradient budget to the harder task, producing better final accuracy.

Interpretation: These curves directly support the paper's central mechanism claim—that format rewards saturate quickly, that continuing to weight them equally wastes gradient budget, and that dynamic reweighting recovers this budget for the accuracy objective, improving final performance. The early-late crossover pattern (Uniform better early on format, RECAP better later on accuracy) is exactly what the scheduler is designed to produce: sacrifice some format-optimization speed early in exchange for sustained accuracy improvement later.

Response Length Analysis: Reasoning Chain Concision (Figures 6 and 7)

Figure 6 tracks thinking length on a segmentation task when a uniform "thinking reward" is applied to all domains (as in MoDoMoDo's approach). The model rapidly learns that long chain-of-thought is unnecessary for segmentation: response length drops from several hundred characters at the start of training to tens of characters later, often near zero. This observation is not a RECAP result per se—it is a motivating finding that led the paper to adopt domain-dependent format rewards (thinking for reasoning tasks, no-thinking for perception tasks). It demonstrates that applying the same format reward structure across heterogeneous tasks can produce unintended behaviors: the model discovers the reward can be achieved by simply not thinking, which defeats the purpose of encouraging reasoning on tasks that actually benefit from it.

Figure 7 is a RECAP result: it compares thinking length on reasoning tasks between a reasoning-only model and RECAP. The reasoning-only model maintains an average of approximately 67.3 words per thinking trace with high variability throughout training. RECAP progressively reduces thinking length to approximately 27.3 words—a ~60% reduction—while simultaneously improving reasoning accuracy (as shown in Tables 1 and 2). The paper notes this is a direct inference efficiency gain: fewer generated tokens reduce latency and compute cost. The mechanism is hypothesized as cross-task calibration: replaying perception tasks where thinking is unnecessary teaches the model to modulate chain-of-thought length based on problem difficulty rather than always defaulting to verbose reasoning.

Critical interpretation: This is an emergent benefit of multi-domain replay that the paper's scheduler likely amplifies but does not directly control—the scheduler reweights objectives, not response length. The reduction in thinking length is plausibly a consequence of the model learning a conditional policy (think for math, don't think for perception) that then generalizes to produce shorter chains on math problems where the answer is straightforward. The paper does not provide direct evidence for this generalization mechanism, but the observation is practically significant: if general-capability replay naturally produces more concise reasoning without a length penalty, it addresses the efficiency concerns around long chain-of-thought traces in production deployments.

Final Training Metrics: RECAP vs. Uniform (Figure 10)

Figure 10 in the Appendix reports final performance across five metrics: Think Format, Answer Format, Reasoning Accuracy, IoU, and Mean Token Accuracy. RECAP and Uniform achieve near-parity on format metrics (Think Format: 95.69 vs. 95.36; Answer Format: 95.76 vs. 95.79), confirming that RECAP's down-weighting of format rewards does not harm format adherence—the model still produces correctly structured outputs. However, RECAP shows consistent improvements on correctness-oriented metrics: Reasoning Accuracy (+2.01 points: 65.97 vs. 63.96), IoU (+1.11 points: 75.15 vs. 74.04), and Mean Token Accuracy (+1.40 points: 84.84 vs. 83.44). This pattern—format parity, correctness improvement—is the ideal outcome for the scheduler: it preserves what has already been learned while reallocating capacity to what still needs improvement.

Ablation Studies and Robustness Checks

Data sampling strategy (Uniform vs. PropMix vs. Coreset): Table 2 provides a three-way comparison of replay sampling strategies under the large hybrid setting, with all three using fixed objective weights (no scheduler). Uniform achieves best or competitive performance on most benchmarks, substantially outperforming PropMix—for example, on MathVerse: 36.07% (Uniform) vs. 34.75% (PropMix), on MMBench: 75.34% vs. 73.54%, on MathVision: 22.13% vs. 21.05%. This validates the paper's choice of uniform sampling as the default replay strategy and suggests that proportional sampling—which would heavily weight large domains like RefCOCO (321k samples) over small domains like LISA (1.3k samples)—provides an inferior training distribution for capability preservation. The large domain sizes in the general-capability datasets (Table 3) make proportional sampling particularly problematic. Coreset, which replays only half the reasoning-data volume from general domains, achieves surprisingly strong results, leading on AI2D (79.92%), MathVista (66.90%), and VizWiz (63.76%). This suggests that the quantity of replayed general data beyond a certain threshold may be less important than the diversity of domains represented, though the paper does not systematically vary the coreset size to test this hypothesis.

KL regularization vs. replay (LwF baseline): Table 2 includes LwF (uniform data sampling + KL penalty $\beta = 0.01$ ) as a direct test of whether standard policy regularization can substitute for replay. LwF underperforms Uniform (which uses replay without KL) on reasoning benchmarks: MMMU-Pro: 29.59% vs. 31.91%, MathVerse: 33.98% vs. 36.07%, MathVision: 18.42% vs. 22.13%. On LISA, LwF (65.08%) roughly matches Uniform (65.18%), confirming that KL alone provides some protection against perceptual capability regression but is insufficient for reasoning tasks. The paper interprets this as evidence that KL "constrains plasticity without providing targeted capability preservation"—the model stays close to the reference in KL divergence but still allocates its limited update budget to objectives that saturate quickly rather than to harder ones. This is a controlled comparison: LwF and Uniform both see the same data distribution; the only difference is the KL term vs. no KL term. The performance gap is therefore attributable specifically to how KL regularizes (or fails to regularize) the learning dynamics.

Dynamic reweighting on vs. off (RECAP vs. Uniform): The primary ablation is the scheduler itself. Across Tables 1 and 2, RECAP (with dynamic reweighting) consistently outperforms Uniform (without dynamic reweighting) under identical data sampling. In the RLVR-only setting (Table 1), RECAP leads Uniform on all six benchmarks, with gaps ranging from 0.72 points (ChartQA) to 10.64 points (SAT). In the hybrid setting (Table 2), RECAP leads on 7 of 9 benchmarks. The two exceptions in the hybrid setting—AI2D (78.21% vs. 76.43%) and MMBench (78.52% vs. 75.34%)—actually show RECAP winning; checking all nine benchmarks: RECAP scores higher than Uniform on LISA (67.24 vs. 65.18), MMMU-Pro (34.15 vs. 31.91), AI2D (78.21 vs. 76.43), MathVista (66.70 vs. 65.60), MathVision (25.11 vs. 22.13), MathVerse (40.83 vs. 36.07), MMBench (78.52 vs. 75.34), VizWiz (61.97 vs. 54.05), and OCRBench v2 (39.72 vs. 38.06). This is a clean sweep—RECAP outperforms Uniform on every single benchmark in both settings. The magnitude of improvement varies (2–3 points typically, up to 10+ on SAT), but the direction is consistent.

Domain-dependent format rewards (thinking vs. no-thinking): The paper does not report a formal ablation comparing uniform thinking rewards to domain-dependent format rewards, but the training dynamics in Figures 6 and 7 serve as qualitative evidence. Figure 6 shows that applying a thinking reward to segmentation causes the model to learn that thinking is unnecessary, rapidly shortening its responses. Figure 7 shows that combining domain-dependent format rewards with dynamic reweighting produces shorter, more calibrated reasoning chains. A direct ablation—e.g., RECAP with uniform thinking rewards vs. RECAP with domain-dependent thinking rewards—is not presented. This is a gap: we cannot quantify how much of RECAP's performance gain comes from the domain-dependent format structure vs. the reweighting scheduler, though the paper's description implies both are necessary.

Open-source reasoning model comparison: Tables 1 and 2 include multiple open-source reasoning models as contextual baselines. While these are not controlled comparisons (different base models, different training data, different pipelines), they demonstrate that RECAP with a simple plug-in scheduler is competitive with or superior to specialized models. In Table 1, RECAP outperforms VLAA-Thinker-3B, vision-grpo-qwen-2.5-vl-3b, Qwen2.5VL-3b-RLCS, and Qwen2.5-VL-3B-Instruct-GRPO-deepmath on most benchmarks. In Table 2, RECAP substantially outperforms VLAA-Thinker-7B, Vision-R1-7B, and OpenVLThinker-7B on nearly every benchmark, with particularly large margins on MMMU-Pro (34.15% vs. best open-source 26.76%) and MathVerse (40.83% vs. best open-source 29.87%). These comparisons support the paper's implicit claim that explicit capability preservation during RLVR training produces better overall models than specialized training that ignores forgetting.

Temperature sensitivity: The paper sets $T = 5$ by default and notes that "lower T sharpens priorities while higher T approaches uniform mixing." A sweep over $T$ values is not reported, which is a gap—we do not know how sensitive RECAP is to this hyperparameter. The paper's argument that T is an interpretable knob (lower = more aggressive reweighting, higher = more conservative) is plausible, but without empirical characterization, it remains a claim rather than a demonstrated property.

Window length sensitivity: The window length $W$ for computing $\mu^{(t)}_k$ , $\tilde{\mu}^{(t)}_k$ , and $\sigma^{(t)}_k$ is not specified in the paper, and no sweep over $W$ is reported. This is a significant gap because $W$ controls the fundamental tradeoff in the scheduler: too short, and the window statistics are dominated by per-step RL noise; too long, and the scheduler cannot detect rapid convergence. The paper's Figure 4 shows that format rewards saturate within ~50 steps, implying $W$ must be small enough to detect this change, but the exact value and its sensitivity are unknown. This makes replication difficult and leaves unclear whether $W$ requires task-specific tuning.

Multiple seeds / stability: The paper reports single-run results without confidence intervals, error bars, or multiple random seeds. Given the well-documented high variance of RL training (Henderson et al., 2018), and the paper's own evidence that per-step RL signals are extremely noisy (Appendix Figure 11 shows the total reward standard deviation near 0.9), score variation across runs could be substantial. The consistent pattern of RECAP outperforming Uniform across all benchmarks in both settings is suggestive, but without statistical characterization, we cannot rule out that some fraction of the reported gaps fall within run-to-run noise.

Scaling with number of objectives: The paper tests RECAP with $K=5$ objectives in the hybrid setting and an unspecified but similar number in the RLVR-only setting. Whether the scheduler remains effective with substantially more objectives (e.g., 20+) is untested. The softmax reweighting naturally handles any number of objectives, but the priority signal $s = c + i$ may become less discriminative as the number of objectives grows—more objectives means more competition for gradient budget, and fine distinctions in convergence rate may matter more. This is not tested.

Critical Assessment

The experiments in this paper provide genuine support for its central claims, but with important boundary conditions and methodological limitations that qualify the strength of the conclusions. I evaluate each major claim against the evidence presented.

Claim from Section 1: "RECAP not only preserves general capabilities but also improves reasoning by enabling more flexible trade-offs among in-task rewards."

The evidence for capability preservation is strong and direct. On LISA—the benchmark highlighted in Figure 2 as the primary example of forgetting—RECAP achieves 67.24% IoU compared to the base model's 65.13% and the reasoning-only model's 57.58% (Table 2). This is a clear demonstration that RECAP prevents the catastrophic forgetting observed under reasoning-only training and actually improves over the base model. On other general-capability benchmarks (MMBench, OCRBench v2, VizWiz), RECAP maintains or improves over the base model, while reasoning-only training shows mixed results (sometimes better, sometimes worse than the base).

The evidence for "improving reasoning" is also present but nuanced. RECAP outperforms Uniform—the fairest baseline—on all reasoning benchmarks in both settings, demonstrating that dynamic reweighting adds value beyond simple replay. However, the margin of improvement varies substantially: ~10 points on SAT (Table 1), ~2 points on MMMU-Pro (Table 2), ~1 point on MathVista (Table 2). The larger gains appear on benchmarks where the format reward is likely to dominate training dynamics (SAT, ScienceQA), while the smaller gains appear on benchmarks where accuracy may be the primary challenge regardless of format saturation. This pattern is consistent with the scheduler's mechanism but also implies that RECAP's benefit is largest when there is a clear, saturating objective to down-weight—on benchmarks where all objectives remain challenging, the scheduler's reallocation provides less leverage.

A qualification: "flexible trade-offs among in-task rewards" is demonstrated indirectly through the format vs. accuracy reward curves in Figure 5, not through a direct manipulation of the trade-off. The paper shows that the scheduler produces a different balance than uniform weighting, but does not demonstrate that this balance can be intentionally tuned—the scheduler's behavior is emergent from convergence dynamics, not configurable. "Flexible" implies controllability that is not actually tested.

Claim from Section 1: "Open-source reasoning models suffer performance degradation on core capabilities such as perception and faithfulness."

Figure 1 provides compelling visual evidence, with consistent patterns across both 3B and 7B model families. The VisOnlyQA benchmark shows particularly stark drops across almost all reasoning models. This claim is well-supported by the survey in Figure 1 and by the paper's own controlled experiment (reasoning-only training drops LISA from 65.13% to 57.58% in Table 2). However, the survey in Figure 1 is observational—it compares different models from different groups with different training pipelines, not a controlled experiment. Correlation is not causation, and some of these models may have been trained on different data mixtures, with different base model versions, or with different hyperparameters that affect their general capabilities independently of reasoning training. The controlled experiment in Table 2 (reasoning-only vs. base model) provides stronger causal evidence but only for the specific Qwen2.5-VL-7B model with the specific ThinkLite-VL-70k dataset. The paper does not demonstrate the same effect with other base models or other reasoning datasets, limiting the generalizability of the causal claim.

Claim from Section 2 and Section 4: "KL regularization alone is insufficient to prevent general capability forgetting because it operates on the training distribution, not on held-out capabilities."

This claim is supported by the LwF vs. Uniform comparison in Table 2. LwF (KL with β=0.01) trails Uniform (replay without KL) on almost every benchmark, particularly on reasoning: MMMU-Pro (29.59 vs. 31.91), MathVerse (33.98 vs. 36.07), MathVision (18.42 vs. 22.13). This demonstrates that replay provides capability preservation benefits that KL alone does not. However, the comparison is somewhat confounded: LwF uses both KL and data replay (uniform sampling), so the question is whether KL plus replay is better than replay alone. The data says no—replay alone (Uniform) is better than replay plus KL (LwF). This could be because KL constrains plasticity too much (as the paper suggests), or because β=0.01 is suboptimal for this task, or because the interaction between KL and the GRPO objective interferes with the replay gradients. A cleaner test would be LwF without any general-capability replay compared to Uniform (replay without KL), to isolate whether KL can substitute for replay entirely. This experiment is not run—LwF in Table 2 still uses uniform data sampling, so it cannot distinguish "KL replaces replay" from "KL plus replay is worse than replay alone."

Claim from Section 4: "Convergence rate and instability together constitute a sufficient signal for determining which objectives need gradient budget."

The consistency of RECAP's improvement over Uniform across 15 total benchmarks (6 in Table 1, 9 in Table 2) provides empirical support for this claim. The scheduler never makes things worse—RECAP matches or exceeds Uniform on every single benchmark. This is a strong result: a simple, magnitude-agnostic scheduler using only two derived signals from loss histories never harms performance and almost always helps.

However, "sufficient" is a strong claim. The paper does not test whether convergence rate alone, or instability alone, would work as well. The additive combination $s = c + i$ could be masking that one signal is doing all the work and the other is irrelevant. An ablation with $s = c$ only and $s = i$ only would be needed to establish sufficiency of both signals. The paper acknowledges this implicitly by noting that "finetuning the trade-off between those two terms offers finer-grained control," but does not run the simplest version of this test: zero out one term and see if performance degrades. This is a missed opportunity—if $s = c$ alone worked as well, the scheduler would be even simpler and the claim about instability's importance would be falsified.

Claim from Section 4: "RECAP is magnitude-agnostic, requires no auxiliary models, and drops into RLVR pipelines without overhead."

The "no auxiliary models" claim is clearly true—RECAP uses only scalar loss histories, no proxy models like MoDoMoDo. The "drops into RLVR pipelines without overhead" claim is plausible but not quantitatively demonstrated. The paper does not report wall-clock time or computational overhead of the scheduler compared to Uniform training. The sliding-window updates involve O(K) scalar operations per step, which is negligible compared to forward and backward passes through a 7B-parameter model, but this is reasoning from first principles rather than measurement. A small table showing training throughput (steps/second) with and without RECAP would strengthen this claim.

The "magnitude-agnostic" claim is well-supported by the scheduler's design (ratio-based convergence rate, coefficient-of-variation-based instability) and by its application across heterogeneous objectives (cross-entropy in nats, IoU in [0,1], binary accuracy, RL surrogate losses with unknown scale). No per-objective normalization or scaling was required. This is genuinely valuable and distinguishes RECAP from prior multi-task weighting methods.

Methodological weaknesses that qualify all results:

No error bars or multiple seeds. All results are single-point estimates. Given the high variance of on-policy RL training documented in Appendix Figure 11, it is plausible that some of the smaller gaps (e.g., RECAP at 66.70 vs. Coreset at 66.90 on MathVista in Table 2) are within run-to-run noise. The consistent direction of improvement (RECAP > Uniform on 15/15 benchmarks) is more robust evidence than any single benchmark gap, but the magnitude of improvement is uncertain.
Single model family. All experiments use Qwen2.5-VL (3B and 7B). The paper does not test with other base models (e.g., LLaVA, InternVL, DeepSeek-VL), other model scales (e.g., 13B, 72B), or pure language models (non-vision). The generalizability of the findings—both the forgetting problem and RECAP's effectiveness—to other architectures, scales, and modalities is unknown.
Single reasoning dataset (ThinkLite-VL-70k) for the primary hybrid experiments. The general-capability forgetting that RECAP prevents may depend on the specific reasoning dataset used. A different reasoning dataset with different difficulty, format requirements, or domain overlap with the general-capability data might produce different forgetting patterns or different optimal reweighting schedules. The paper does not test with alternative reasoning datasets.
No sensitivity analysis for the two most important hyperparameters. The window length $W$ and the softmax temperature $T$ are not swept. $W$ controls the scheduler's responsiveness to convergence changes; $T$ controls how aggressively it reweights. Without knowing sensitivity to these parameters, practitioners cannot assess how carefully they need to tune RECAP for new settings.
Difficulty estimation / convergence detection in the first $2W$ steps. The scheduler requires $2W$ steps of history before it can compute its first convergence rate and instability values. During this warm-up period, the behavior is undefined (the paper does not specify whether weights are uniform or some default is used). If $W$ is, say, 25 steps, then for the first 50 steps of training, the scheduler provides no dynamic reweighting benefit—yet format rewards can saturate within this window (Figure 4 shows format saturating by step 50). This means RECAP may miss the opportunity to down-weight format during its most rapid convergence phase. The paper does not address this warm-up limitation.
The RLVR-only setting (Table 1) is under-described. The training datasets, exact objective set, and training duration are not specified with the same detail as the hybrid setting. We know the configuration follows MoDoMoDo (Liang et al., 2025) and uses 8 GPUs with per-device batch size 2 and 4 rollouts per prompt, but the domains replayed and the number of objectives are not enumerated. This limits the reproducibility of the RLVR-only results.
No analysis of what RECAP's weights actually look like during training. The paper shows reward trajectories (Figures 4, 5, 11) but does not show the actual λ_k values the scheduler produces over time. Seeing that the format reward weight drops from ~1.0 to, say, ~0.3 after step 50 while the accuracy weight rises from ~1.0 to ~1.7 would provide direct evidence that the scheduler is behaving as designed. Without this, we infer the scheduler's behavior from the reward curves, which is indirect.
The claim that replay reduces thinking length (Figure 7) lacks causal evidence. The paper attributes shorter thinking traces to general-capability replay, but the reasoning-only baseline differs from RECAP in both data distribution (no general-capability data) and objective reweighting. Whether the length reduction is caused by (a) seeing perception tasks with no-thinking rewards, (b) the scheduler down-weighting thinking format rewards, or (c) some interaction effect is not isolated. An ablation comparing RECAP with thinking format reward on perception tasks vs. RECAP without thinking format reward on perception tasks would be needed to establish causality.

What would strengthen the paper:

Multi-seed runs (3–5 seeds) with mean and standard deviation for all main table results. This is the single highest-impact addition—it would transform the evidential value of every claim.
Sweep over $W$ and $T$ to characterize sensitivity.
A true LwF baseline without general-capability replay (KL only, reasoning data only) to cleanly test "does KL substitute for replay?"
Direct visualization of the scheduler's weight trajectories ( $\lambda_k$ over time) for a representative training run.
An ablation study with $s = c$ only and $s = i$ only to determine whether both signals are necessary.
Testing with at least one additional base model family and one additional reasoning dataset to establish generalizability.
Specification of and analysis of the warm-up period behavior before $2W$ steps of history are available.

Despite these limitations, the paper's core empirical contribution is robust: replaying general-capability data during RLVR training substantially mitigates forgetting, and dynamically reweighting objectives based on convergence rate and instability consistently improves over uniform weighting across a wide range of benchmarks. The consistency of the improvement—15 out of 15 benchmarks favoring RECAP over Uniform—is the most convincing evidence, even in the absence of error bars. The magnitude of the effect varies by benchmark and setting, but the direction is unambiguous, and the mechanism (down-weighting saturated format rewards, up-weighting slowly-converging accuracy rewards) is directly supported by the training dynamics in Figure 5.

6. Limitations and Trade-offs

The Difficulty Estimation Cost Is Not Accounted for in the Training Budget

The scheduler requires $2W$ steps of loss history before it can compute its first convergence rate and instability values, and the paper never specifies what policy RECAP follows during this warm-up period. Figure 4 shows that the format reward saturates within roughly the first 50 steps of training—convergence rate $c \to 1$ and instability $i \to 0$ by that point. If the window length $W$ is, say, 25 steps, then for the first 50 steps the scheduler provides no dynamic reweighting benefit, yet this window contains the format reward's most rapid convergence phase. The consequence is that RECAP may miss the optimal window for down-weighting the format reward—by the time the scheduler has enough history to detect saturation, the format reward is already saturated and the model has already allocated capacity to it that could have gone to accuracy or perception objectives. The paper does not report the value of $W$ used in experiments, does not analyze warm-up behavior, and does not specify whether weights are held uniform during warm-up or initialized to some default. This matters for two reasons: first, it means the scheduler's most important intervention (preventing format from consuming early training capacity) occurs after the format reward has already been largely learned, potentially reducing the scheduler's effectiveness compared to an oracle that knew format would saturate quickly; second, for very short training runs (e.g., fine-tuning for only 50–100 steps), RECAP may provide essentially no benefit because the warm-up period consumes the entire budget. A possible mitigation—using a shorter initial window or initializing weights based on prior knowledge—is not discussed. The paper also does not account for this warm-up period in any overhead calculation; while scalar window updates are computationally negligible, the effective loss of scheduler control during the most dynamic phase of training is a meaningful hidden cost.

No Statistical Characterization of Results Despite Documented High Variance in RL Training

The paper reports all benchmark results as single-point accuracy scores without error bars, confidence intervals, or multiple random seeds. Table 1 reports RECAP at 55.19% on SAT versus MoDoMoDo at 49.95%, and Table 2 reports RECAP at 67.24% versus Uniform at 65.18% on LISA—gaps of 5.24 and 2.06 points respectively. The paper's own Appendix Figure 11 demonstrates that on-policy RL reward signals are extremely noisy, with the total reward standard deviation peaking near 0.9 around step 20. This is not merely a theoretical concern—the reinforcement learning literature has extensively documented that identical algorithms run with different random seeds can produce substantially different final performance (Henderson et al., 2018, cited by the paper). The consequence is that we cannot distinguish genuine algorithmic improvements from run-to-run noise for the smaller-magnitude gaps. The consistent direction of improvement (RECAP outperforms Uniform on 15 of 15 benchmarks across Tables 1 and 2) is strong evidence that the scheduler provides real benefit, but the magnitude of that benefit—which determines whether RECAP is worth implementing in a production pipeline—is uncertain. A gap of 2 points on LISA that is actually 2 ± 1.5 points has very different practical implications than 2 ± 0.3 points. The paper does not attempt to characterize this uncertainty, making it difficult for practitioners to assess expected return on the engineering effort of implementing the scheduler. This is especially problematic for the comparisons against external baselines (MoDoMoDo, open-source reasoning models) where only one run of RECAP exists—we cannot determine whether RECAP's superiority to MoDoMoDo on SAT (55.19 vs. 49.95) is reliable or would reverse in a different random seed. The paper makes no mention of multi-seed experiments in the limitations section, effectively treating the single-run results as point estimates of true performance without acknowledging the variance documented in its own appendix.

Single Model Family and Single Reasoning Dataset Leave Generalizability Unestablished

All experiments use Qwen2.5-VL models at two scales (3B and 7B) and a single primary reasoning dataset (ThinkLite-VL-70k for the main hybrid setting, with the RLVR-only setting following MoDoMoDo's configuration). The paper's central empirical claim—that reasoning-focused RLVR causes general capability forgetting and that RECAP mitigates it—is demonstrated only within this narrow envelope. Several aspects of the findings could be model-specific or dataset-specific. The format reward's saturation speed (Figure 4, ~50 steps) depends on the specific format template used (e.g., thinking... followed by <answer>...</answer>) and the base model's instruction-following ability—a base model with stronger or weaker instruction following would show different saturation dynamics, potentially changing the optimal scheduler behavior. The general-capability regression patterns shown in Figure 1 are specific to the Qwen2.5-VL family and the specific reasoning models derived from it—models from other families (LLaVA, InternVL, DeepSeek-VL) or pure language models may exhibit different forgetting patterns or different sensitivity to the format vs. accuracy tradeoff. The paper acknowledges this limitation only indirectly, stating that it "believe[s] this model is representative of the capabilities of many contemporary LLMs"—a claim the paper's own evidence cannot verify. The consequence for practitioners is uncertainty: if you are training a LLaVA-based model with a different reasoning dataset and different format requirements, will RECAP's default $T=5$ and unspecified $W$ transfer directly? Will the convergence dynamics (fast format saturation, slow accuracy improvement) hold? The paper provides no evidence either way. This is particularly important because the scheduler's mechanism depends on the relative convergence speeds of different objectives—if a different model or dataset produces different relative speeds (e.g., format takes 200 steps to saturate, or accuracy converges quickly), the scheduler's reallocation behavior would differ, potentially reducing or eliminating its benefit. A sensitivity analysis across model families or reasoning datasets would address this but is not attempted. The paper does suggest future work on extending RECAP to preference- and alignment-based objectives, but does not flag the single-model-family scope as a limitation requiring empirical extension.

The Scheduler's Design Choices ( $W$ , $T$ , Additive Combination) Are Untested

RECAP introduces three design parameters that directly control its behavior—the sliding window length $W$ , the softmax temperature $T$ , and the additive combination $s = c + i$ —and the paper reports no sensitivity analysis for any of them. $W$ controls the fundamental speed-accuracy tradeoff: smaller $W$ makes the scheduler more responsive to rapid convergence changes (like format saturating in ~50 steps in Figure 4) but more vulnerable to per-step RL noise (which Appendix Figure 11 shows is substantial). Larger $W$ provides more stable estimates but may detect saturation too late to be useful. The paper does not specify what $W$ was used, making replication impossible and leaving practitioners to guess this critical parameter. $T$ controls how aggressively the scheduler reweights: the paper states "lower T sharpens priorities while higher T approaches uniform mixing" and sets $T=5$ by default, but provides no sweep showing what happens at $T=1$ , $T=10$ , or $T=20$ . A practitioner implementing RECAP for a new setting has no guidance on whether the default $T=5$ will work or whether task-specific tuning is needed. The additive combination $s = c + i$ is chosen "for simplicity" over a learned or tuned trade-off, but no ablation tests $s = c$ alone or $s = i$ alone to determine whether both signals are necessary. If $s = c$ alone worked as well, the scheduler would be simpler and the instability computation would be unnecessary overhead. The consequence of these untested design choices is that we cannot assess RECAP's robustness—does it work across a range of $W$ and $T$ values, or is performance brittle to these parameters? If the latter, the "plug-in" claim is misleading because substantial tuning may be required per task. The paper mentions that "finetuning the trade-off between those two terms offers finer-grained control and potentially improves the performance," implicitly acknowledging that the additive combination is not optimal, but does not explore this direction. These are not minor hyperparameters—they define the scheduler's core behavior—and the absence of any characterization means RECAP is presented as a finished method when it is closer to a promising design whose sensitivity is unknown.

Replay Data Must Be Curated Per-Domain with Domain-Appropriate Format Rewards, Not a Turnkey Solution

The paper discovers during experimentation that applying a uniform thinking-format reward across all domains causes the model to learn that chain-of-thought is unnecessary for perception tasks, rapidly shortening outputs to near-zero thinking length (Figure 6). This observation led the authors to adopt domain-dependent format rewards: thinking rewards for reasoning tasks, no-thinking answer-format rewards for perception tasks. This is a reasonable design choice, but it means RECAP is not a fully automated "drop-in" scheduler—it requires a human to decide which domains need thinking rewards and which do not, and to configure different reward structures for different data sources. The paper treats this as an implementation detail rather than a limitation, but it has two important consequences. First, the domain-dependent reward structure is a form of manual reward shaping that contributes to RECAP's performance independently of the reweighting scheduler. A practitioner applying RECAP to a new domain mixture would need to make similar decisions—does this new domain benefit from thinking rewards?—without clear guidance from the paper on how to decide. Second, the interaction between this manual reward structuring and the scheduler's dynamic reweighting is not isolated. When RECAP outperforms Uniform on reasoning benchmarks (e.g., Table 2: MathVerse 40.83 vs. 36.07), we cannot determine how much of the gain comes from the scheduler's reweighting and how much from the domain-dependent format structure enabling better cross-task calibration. The paper does not run an ablation with uniform thinking rewards plus RECAP's scheduler vs. domain-dependent rewards plus uniform weighting—either of which would isolate these two factors. The consequence for practitioners is that implementing RECAP requires not just plugging in the scheduler code but also manually designing per-domain reward structures, which may require iterative experimentation similar to the process the paper itself went through (observing Figure 6, then adjusting). This undercuts the paper's claim that RECAP "requires no auxiliary models, no per-objective normalization, and no manual coefficient tuning"—while the scheduler itself avoids these, the overall training setup still requires manual reward design decisions that materially affect performance.

The Observation That Reasoning Chain Length Declines with Replay Is Correlational, Not Causal, and the Mechanism Is Unexplained

Figure 7 shows a striking result: RECAP reduces average thinking length on reasoning tasks from ~67.3 words to ~27.3 words (a ~60% reduction) while maintaining or improving accuracy. The paper presents this as a benefit of replay: "mixing general-capability replay with dynamic objective reweighting progressively reduces thinking length and stabilizes variance, converging to concise rationales." However, the reasoning-only baseline differs from RECAP in two simultaneous ways: (1) it sees no general-capability data, and (2) it uses no dynamic reweighting. The paper cannot distinguish whether the length reduction is caused by exposure to perception tasks with no-thinking rewards (which teaches the model that thinking is not always necessary), by the scheduler down-weighting the thinking-format reward (which reduces the optimization pressure toward verbose chains), or by some interaction between the two. This matters for two reasons. First, if the length reduction is primarily driven by exposure to no-thinking perception data rather than by the scheduler, then simple uniform replay without RECAP should produce similar length reduction—but this comparison is not shown. Second, if the length reduction is primarily driven by the scheduler down-weighting the thinking format reward, then it is not an independent benefit of replay but a direct consequence of the core reweighting mechanism, which would change how we interpret the result. The paper's language suggests the former interpretation—"replaying general data yields shorter, more concise rationale"—but provides no causal isolation. Additionally, the mechanism by which shorter chains maintain accuracy is unexplained. Are the shortened chains genuinely more efficient reasoning, or are they simply truncating verbose but non-essential elaboration while preserving the core logical steps? The paper provides qualitative examples in Appendix Figure 9 showing that RECAP produces more concise answers on simple questions, but does not systematically analyze whether the eliminated tokens were genuinely redundant or contained useful intermediate reasoning. For a practitioner considering RECAP, the efficiency gain from shorter chains (reduced inference latency and cost) is potentially substantial, but without understanding the causal mechanism, they cannot predict whether this benefit will transfer to their own domain mixture—if their general-capability data does not include tasks with no-thinking rewards, the length reduction may not materialize. The paper does not acknowledge this as a limitation requiring further investigation.

7. Implications and Future Directions

How This Work Changes the Landscape

RECAP does not introduce a new RL algorithm, a new reward function, or a new model architecture. Its contribution is at the level of training dynamics governance—a lightweight scheduler that monitors per-objective convergence behavior and reallocates gradient budget in real time without auxiliary models, gradient computation, or manual coefficient tuning. This is not a paradigm shift in the sense of RLVR itself; it is a methodological reframing of how we think about multi-objective post-training. The field has implicitly treated the problem of capability regression as a policy-drift problem (prevent the model from moving too far) or a data-mixture problem (include enough diverse data). RECAP's central insight—that different objectives converge at different rates, and that a scheduler can read these rates from loss histories to decide where gradient budget is needed—shifts attention from what data the model sees to how effectively it learns from each signal it receives.

This reframing has two concrete consequences for how the community approaches post-training. First, it makes objective-level, not just data-level, monitoring a first-class concern. Prior work on replay and data mixing (Rebuffi et al., 2017; Zheng et al., 2023; MoDoMoDo's proxy-model-based mixing) treats data diversity as the primary lever. RECAP shows that two training runs seeing identical data distributions (Uniform vs. RECAP) can produce systematically different outcomes—across all 15 benchmarks in Tables 1 and 2, RECAP outperforms Uniform—purely through dynamic loss reweighting. This implies that future post-training pipelines should log and monitor per-objective convergence rates and instabilities as standard practice, much as learning rates and gradient norms are monitored today. The paper provides a concrete set of diagnostics (convergence rate $c$ , instability $i$ , weight trajectories) that practitioners can adopt immediately.

Second, RECAP changes the cost-benefit calculus for multi-objective RLVR training. Prior to this work, the strongest approach to preventing capability regression was MoDoMoDo's proxy-model training—requiring multiple full-size model training runs to learn optimal mixing ratios, plus hand-tuned reward weights, with the entire process needing to be rerun when new tasks are added. RECAP matches or exceeds MoDoMoDo's performance (Table 1) while requiring zero proxy model training, zero manual coefficient tuning, and zero per-benchmark optimization. The only additional computation is $O(K)$ scalar operations per training step—effectively free relative to forward and backward passes through a 7B-parameter model. This shifts the practical default: a team starting a new RLVR post-training project can begin with uniform data sampling plus RECAP's scheduler and expect competitive or superior results to what would previously have required expensive mixture optimization, freeing those resources for other priorities.

The paper also resolves a tension that has been building in the reasoning-model literature. On one side, models like DeepSeek-R1 and its derivatives demonstrate impressive reasoning gains from RLVR. On the other, a growing body of evidence shows these same models hallucinate more (Yao et al., 2025b), are more vulnerable to jailbreaks (Lou et al., 2025), and underperform base models on perception and robustness tasks (Figure 1). The tension has been whether these regressions are an unavoidable cost of reasoning improvement—a fundamental tradeoff—or an artifact of current training methodology. RECAP's results argue strongly for the latter: by replaying general-capability data with dynamic reweighting, the paper achieves reasoning improvements roughly matching or exceeding reasoning-only training (Table 2: RECAP 25.11% on MathVision vs. 24.87% for Reasoning-only, 40.83% vs. 40.74% on MathVerse, 66.70% vs. 65.50% on MathVista) while simultaneously preserving or improving perception and OCR capabilities (LISA: 67.24% RECAP vs. 57.58% Reasoning-only, a 9.66-point absolute gap). The implication is that capability regression during reasoning post-training is not necessary—it is a symptom of narrow training distributions and static objective weighting, both of which are fixable with relatively simple interventions.

The paper also redirects attention away from more complex mitigation strategies that have been proposed or could be pursued. The LwF baseline (KL regularization with $\beta = 0.01$ ) consistently underperforms replay-based methods on reasoning benchmarks (Table 2: MMMU-Pro 29.59 vs. 31.91 for Uniform, 34.15 for RECAP), suggesting that KL-based policy-constraint approaches—the dominant paradigm in RLHF—are not the most productive direction for capability preservation in RLVR. Similarly, MoDoMoDo's proxy-model-based mixing, while effective, is shown to provide marginal additional benefit over simply replaying data uniformly with RECAP's scheduler, suggesting that the expensive mixture-optimization step may not be worth its computational cost for many use cases. The paper thus makes several research directions—KL-only regularization, static mixture optimization—less attractive relative to dynamic, online approaches.

Perhaps most provocatively, the paper's observation that replaying general data substantially reduces reasoning chain length (~27 vs. ~67 words, Figure 7) without sacrificing accuracy challenges the tacit assumption that long chain-of-thought traces are a natural byproduct of improved reasoning. If excessive verbosity is partially an artifact of narrow training distributions—the model learns "always think verbosely" because it never sees tasks where thinking is inappropriate—then the field's focus on reasoning-only benchmarks as the sole training signal may be inadvertently producing models that are not only brittle in their general capabilities but also inefficient in their reasoning. RECAP's cross-task calibration (learning to modulate chain-of-thought length based on task type) suggests a path toward reasoning models that are both more capable and more efficient, achieved not through explicit length penalties but through a more balanced training diet.

Follow-Up Research This Work Enables

Characterize the warm-up behavior and determine whether early-format-saturation can be preempted. The paper's Figure 4 shows that format rewards saturate within roughly 50 steps, yet the scheduler requires $2W$ steps of loss history before producing its first reweighting. If the unspecified $W$ is substantial (e.g., 25–50 steps), RECAP may be effectively blind during the period when format reward saturation would most benefit from intervention. A follow-up study would sweep $W$ across a range (e.g., 5, 10, 25, 50, 100 steps) while tracking (a) the step at which the format reward weight $\lambda_{\text{format}}$ first drops below 0.5, (b) the final reasoning accuracy, and (c) the final general-capability score. This would establish whether smaller $W$ (faster detection) or larger $W$ (smoother estimates) dominates, and would provide the $W$ -sensitivity analysis missing from the current paper. An even stronger test would compare RECAP against an "oracle" scheduler that sets format weight to zero after step 50 based on prior knowledge, quantifying how much potential benefit is lost during the warm-up period. If the gap is large, it motivates research on warm-up strategies: initializing weights from prior runs, using a shorter initial window with gradual lengthening, or using domain-knowledge priors about which objectives are likely to saturate quickly.

Ablate $s = c + i$ into its components to determine whether both signals are necessary. The paper combines convergence rate $c$ and instability $i$ additively into a single priority score, arguing this is done "for simplicity, which performs consistently well in our experiments." But no experiment tests whether $c$ alone or $i$ alone would perform equivalently. A minimal but high-value follow-up would reproduce the hybrid-setting experiment (Qwen2.5-VL-7B, same datasets, same training budget) with three scheduler variants: RECAP-standard ( $s = c + i$ ), RECAP-convergence-only ( $s = c$ ), and RECAP-instability-only ( $s = i$ ). The null hypothesis is that convergence rate alone is sufficient—format rewards would be down-weighted when $c \to 1$ , and accuracy rewards would maintain weight when $c > 1$ —and that instability adds negligible additional discriminative power. If this null hypothesis holds, the scheduler simplifies further. If it fails—if both signals independently contribute to the 15/15 benchmark improvement over Uniform—then the paper's claim about their complementarity (Section 4, Innovation 3) is validated empirically rather than just conceptually. A positive variant of this experiment would also test a learned linear combination $s = \alpha c + (1-\alpha) i$ with $\alpha$ optimized on a validation fold, to determine whether the unweighted sum is genuinely optimal or a convenient approximation.

Test RECAP across model families and modalities to establish boundary conditions. The paper uses only Qwen2.5-VL at 3B and 7B scales. A direct replication using a different base model family—LLaVA-OneVision, InternVL2, or a pure language model like LLaMA-3 with a reasoning dataset like MetaMathQA or NuminaMath—would test whether RECAP's benefits generalize or are specific to Qwen2.5-VL's particular convergence dynamics. The key question is whether the relative convergence speeds of format vs. accuracy rewards are consistent across model families. A model with stronger instruction-following might saturate format even faster (making RECAP more beneficial), while a model with weaker instruction-following might show slower format convergence (making format and accuracy converge at similar rates, reducing RECAP's advantage over uniform weighting). A well-designed replication would measure format saturation speed (steps to reach 95% of maximum format reward), accuracy convergence speed, and the RECAP-vs-Uniform performance gap for each model family, then correlate these to determine whether RECAP's benefit is predictable from base model properties. If the benefit is unpredictable, RECAP's "plug-in without tuning" claim is weakened for new model families.

Evaluate whether RECAP's thinking-length reduction transfers to pure language reasoning tasks and characterize the mechanism. Figure 7 shows ~60% reduction in thinking length on Qwen2.5-VL-7B when general-capability replay is added, but the paper does not isolate whether this is caused by exposure to no-thinking perception tasks or by the scheduler down-weighting the thinking-format reward. A controlled experiment with three conditions would resolve this: (a) Uniform sampling with domain-dependent thinking rewards (the default setup), (b) Uniform sampling with thinking rewards applied to all domains (forcing the model to think on perception tasks, matching MoDoMoDo's approach), and (c) Uniform sampling with no-thinking rewards on all domains (forcing direct answers everywhere). Each condition would be run with and without RECAP's scheduler, producing a 3×2 factorial design. If condition (b) with RECAP still shows reduced thinking length compared to condition (b) without RECAP, the reduction is attributable to the scheduler's reweighting, not to the domain-dependent reward structure. If condition (a) without RECAP already shows reduced thinking length compared to a pure reasoning-only baseline, the reduction is attributable to task diversity, not the scheduler. For pure language models without a vision component, the relevant "no-thinking" tasks might include simple factual QA, translation, or summarization—tasks where chain-of-thought is unnecessary. This experiment would also establish whether RECAP's length-reduction benefit transfers to non-vision domains, which is essential for the method's claimed generality.

Stress-test RECAP under adversarial objective addition to determine whether the scheduler can be gamed. RECAP's magnitude-agnostic design means any objective with high convergence rate (still improving) or high instability (noisy) receives elevated weight. This raises a question: could a malicious or poorly-designed objective hijack the scheduler by maintaining artificially high $c$ or $i$ ? For example, an objective that oscillates wildly (high $i$ ) without genuine improvement would receive consistently high weight, starving other objectives of gradient budget. A stress-test would introduce synthetic "adversarial" objectives during RLVR training—objectives that either (a) oscillate with high amplitude around a fixed mean (high $i$ , $c \approx 1$ ), or (b) improve slowly but trivially (e.g., a reward for outputting a specific token that the model gradually learns, producing steady but meaningless $c < 1$ ). The test would measure whether these adversarial objectives capture disproportionate scheduler weight and, if so, whether they degrade reasoning and general-capability performance relative to a clean objective set. A positive result (RECAP is robust) would strengthen the claim that the scheduler can be safely deployed with diverse, potentially noisy objective sets. A negative result (RECAP can be distracted) would motivate research on robustness mechanisms: capping maximum weight per objective, incorporating prior knowledge about which objectives are trustworthy, or adding a regularization term that penalizes high-weight objectives with low absolute improvement.

Integrate RECAP into a self-improvement loop and measure whether capability preservation compounds over iterations. The paper studies a single post-training phase, but modern LLM pipelines increasingly use iterative self-improvement: generate rollouts, filter by reward, fine-tune, repeat (e.g., STaR, ReST $^{EM}$ ). Each iteration risks compounding capability regression, since the model that enters iteration $t+1$ is the model that potentially forgot general capabilities during iteration $t$ . A natural extension would run multiple iterations of RLVR training with and without RECAP, measuring general-capability scores (LISA, MMBench, OCRBench) and reasoning scores at each iteration. The key hypothesis is that RECAP's benefit compounds: if RECAP preserves 2 additional points of LISA performance per iteration compared to Uniform, then after 5 iterations the gap would be ~10 points rather than 2. Conversely, if RECAP's benefit is one-time (preventing the initial forgetting but not providing cumulative protection), the gap would remain constant across iterations. This experiment connects RECAP to the self-improvement literature and addresses a question the paper explicitly raises but does not test: whether capability preservation becomes more or less important as training extends.

Practical Applications and Downstream Use Cases

Production RLVR post-training pipelines for multimodal assistants. Organizations deploying vision-language models that must both reason about visual content (math problems, chart analysis, diagram understanding) and perform standard perception tasks (object detection, OCR, visual question answering) face the exact tradeoff documented in Table 2: reasoning-only training drops LISA segmentation from 65.13% to 57.58%, while RECAP improves it to 67.24%. For a multimodal assistant serving millions of users, this 9.66-point gap on segmentation represents a substantial fraction of queries where the model would produce incorrect bounding boxes or fail to ground visual references—directly degrading user experience. Implementing RECAP adds negligible computational overhead (scalar window updates per step) while eliminating the need for separate specialized models for reasoning vs. perception tasks, reducing serving infrastructure costs. The paper's domain-dependent format reward design (thinking rewards for reasoning tasks, direct-answer rewards for perception) also provides a template for production reward engineering: tasks should receive format rewards aligned with their natural output structure, not a one-size-fits-all thinking template.

Cost-efficient fine-tuning of open-source VLMs for domain-specific reasoning. A research lab or startup wanting to adapt a base VLM (e.g., Qwen2.5-VL-7B) for medical image reasoning or scientific diagram analysis can use RECAP to improve domain-specific accuracy while preserving the model's ability to handle general visual questions—a capability they likely want to retain for broader usability. Table 2 shows that Uniform replay (the simplest baseline) achieves 65.18% on LISA and 75.34% on MMBench while improving reasoning; RECAP pushes these higher. The practical workflow would be: assemble a small domain-specific reasoning dataset (analogous to the paper's ThinkLite-VL-70k), gather publicly available general-capability datasets for the target modality (RefCOCO, OCR data, or their domain equivalents), uniformly sample from all sources during RLVR training, and activate RECAP's scheduler with default settings. The paper's evidence that RECAP works with only 500 training steps on 8 GPUs suggests this is feasible with modest compute budgets, and the avoidance of proxy model training (unlike MoDoMoDo) makes it accessible to teams without large-scale experimentation infrastructure.

Reducing inference cost for deployed reasoning models through chain-of-thought length reduction. Figure 7 demonstrates that RECAP reduces average thinking length by ~60% (67.3 → 27.3 words) while maintaining or improving accuracy. For a deployed reasoning model serving user queries, thinking tokens are generated auto-regressively before the final answer, directly contributing to time-to-first-token latency and per-query compute cost. A 60% reduction in thinking tokens translates to roughly proportional savings in inference FLOPs and latency for the reasoning portion of generation. If a deployed model spends 70% of its generation time on chain-of-thought and 30% on the final answer, a 60% thinking reduction yields approximately 42% total latency reduction—directly improving user experience without any model architecture changes, quantization, or distillation. The paper does not measure wall-clock latency, but the token-count reduction is large enough that latency improvement is nearly guaranteed. The practical implementation is simply to include diverse general-capability data with no-thinking format rewards during RLVR training—a one-time training cost that yields persistent inference savings for the deployed model's lifetime.

When to Prefer This Method

The paper does not articulate an explicit decision framework comparing RECAP against named alternatives under specific conditions. The baselines are characterized (KL-only, Uniform replay, PropMix, Coreset, MoDoMoDo) and their relative performance is presented in Tables 1 and 2, but the paper does not state rules for when a practitioner should choose RECAP over, say, Coreset or MoDoMoDo. The implicit guidance from the results is:

Prefer RECAP over KL-only regularization (LwF) when capability preservation on out-of-distribution tasks matters more than minimizing policy drift on the training distribution. Table 2 shows RECAP outperforming LwF on 8 of 9 benchmarks, with reasoning gaps up to ~7 points (MathVerse 40.83 vs. 33.98).
Prefer RECAP over Uniform replay when the objective set includes rapidly-saturating signals (e.g., format rewards) alongside slowly-converging signals (e.g., accuracy). Figure 4 and 5 provide the diagnostic: if format saturates in under 50 steps while accuracy remains noisy, RECAP's reweighting provides benefit. RECAP outperforms Uniform on all 15 benchmarks tested.
Prefer RECAP over MoDoMoDo when avoiding proxy model training is a priority (e.g., limited compute budget, frequent new task additions). Table 1 shows RECAP roughly matching or exceeding MoDoMoDo performance, and RECAP requires no auxiliary model training or manual reward weight tuning.
Consider simple Uniform replay (without RECAP) if the objective set has similar convergence speeds or the training budget is too short for the scheduler to accumulate meaningful window statistics. The paper notes that RECAP requires $2W$ steps of warm-up; for very short fine-tuning runs (under ~100 steps), uniform weighting may perform equivalently, though this is not empirically tested.