Less Noise, More Voice: Reinforcement Learning for Reasoning via Instruction Purification

🎯 Pitch

A tiny fraction of prompt tokens often silently sabotages reasoning rollouts—removing just these few interference tokens can rescue over 20% of previously failed samples. Lens identifies and prunes these tokens during exploration, then trains the model to ignore them in the original noisy prompts, achieving a 3.88% average accuracy gain and 1.6× faster convergence than GRPO without discarding difficult prompts or scaling rollout budgets.

1. Executive Summary

The paper introduces the Less Noise Sampling Framework (Lens), a plug-and-play rollout mechanism for Reinforcement Learning with Verifiable Rewards that identifies and removes a small fraction of prompt tokens — termed interference tokens — whose large log-probability deviation from a reference model degrades exploration, and then transfers successful rollouts from the purified prompts to calibrate policy optimization on the original noisy prompts. Evaluated against GRPO across five model families (Llama-3.2-3B, Qwen2.5-3B/7B, Qwen3-4B/8B) on seven math reasoning benchmarks (MATH500, AMC23, AIME24/25, GaokaoEN-2023, Minerva, OlympiadBench), Lens achieves an average performance gain of 3.88% with over 1.6× faster convergence — reaching GRPO's peak accuracy in 1.67× fewer gradient steps on MATH500 — while outperforming both scaling-exploration baselines (GRPO with double rollouts) and filtering baselines (DAPO, GRESO) under a strictly lower computational budget, establishing that low-success prompts contain valuable training signals that pruning interference tokens unlocks without needing to discard difficult samples or inflate rollout counts.

2. Context and Motivation

The Core Problem: Exploration Collapse from a Handful of Tokens

The paper addresses a specific failure mode in Reinforcement Learning with Verifiable Rewards (RLVR) for LLM reasoning: many prompts that appear impossibly difficult during training are, in fact, only a few tokens away from producing successful rollouts. The authors demonstrate that this is not a minor observation but a structural problem — on the DeepMath dataset, simply removing a small set of interference tokens (< 5% of prompt tokens) improves rollout accuracy on previously failed samples by over 20% across all tested model families (Figure 2c). This means that what looks like a fundamental capability gap is often an artifact of token-level interference that distorts the model's exploration.

This matters because RLVR fundamentally depends on encountering correct rollouts to generate informative gradient signals. In complex reasoning tasks, rewards are sparse (binary correctness at the end of long multi-step generations) and the token-level action space is enormous. When interference tokens push the model away from correct solution trajectories, the result is zero-variance prompts — prompts where all sampled rollouts fail identically. Under GRPO's group-relative advantage computation, identical rewards within a group produce vanishing gradients, effectively halting learning on that prompt. The paper's core claim is that many of these zero-variance failures are not due to genuine problem difficulty but to a small number of token-level distractors.

The real-world impact is substantial because this exploration failure directly wastes compute. Recent work has shown that scaling rollout counts (Xu et al., 2025) or filtering out zero-variance prompts entirely (Yu et al., 2025; Zheng et al., 2025a) can maintain training stability, but both approaches are inefficient: the former burns computation on samples the model can't productively use, while the latter discards difficult prompts that contain genuine learning opportunities near the model's capability frontier. Lens claims to resolve this tension by extracting informative signals from those same challenging prompts through targeted token deletion.

Prior Approaches and Where They Fall Short

The paper situates itself relative to three existing strategies for handling the exploration bottleneck in RLVR, and argues each is fundamentally incomplete.

Scaling Exploration (More Rollouts). The most direct response to sparse rewards is to sample more — double the number of rollouts per prompt so that at least some will succeed. This is implemented as GRPO_extended in the paper's baselines. The problem is twofold. First, it incurs a linear increase in computational cost without improving per-sample efficiency: the model still samples most rollouts from the interference-distorted distribution, and the additional budget is spent on what are overwhelmingly failure trajectories. Figure 5 shows that even with double rollouts, the proportion of prompts with no successful samples (the "Failure" category) remains substantial across training. Second, increased sampling does not address the root cause — the interference tokens remain in the prompt, continuing to mislead the policy. The paper frames this as treating a symptom (low success rate) rather than the cause (token-level distractors).

Zero-Variance Prompt Filtering (DAPO, GRESO). DAPO (Yu et al., 2025) discards prompts after sampling when all rollouts receive identical zero rewards, preventing them from contributing zero-variance gradient updates that would destabilize training. GRESO (Zheng et al., 2025a) goes a step further by predicting which prompts will yield zero variance before sampling, and skipping them entirely. Both approaches stabilize training but at a cost: they sacrifice exploration on the most challenging samples. The paper explicitly argues that "aggressively discarding zero-variance prompts can limit capability expansion, particularly on challenging benchmarks" (Section 3.2), and their results support this — Lens shows larger gains over DAPO and GRESO on harder benchmarks like AMC23 and AIME24 where the discarded prompts would have contained the most learning signal at the capability frontier. This is a direct critique of the filtering philosophy: difficult prompts are not noise to be removed, but are precisely where the model needs to improve.

Credit Assignment Methods. The paper also references token-level credit assignment approaches (VinePPO, attention-based methods, entropy-based shaping) that aim to identify which output tokens contribute most to the final reward. These methods operate on the model's generated response — they try to assign credit within the rollout to determine which reasoning steps were good or bad. The paper points out that this overlooks a prior question: what if the instruction tokens themselves are actively misleading the model? The interference token concept shifts the analysis from the output side to the input side, asking not "which generated tokens deserve credit" but "which prompt tokens are causing the failure in the first place." This is a genuinely different analytical lens that prior work missed.

Reward Function Redesign. Some recent work (e.g., Le et al., 2025) tackles zero-variance prompts by modifying the advantage computation to prevent vanishing gradients even when all rewards are identical. The paper acknowledges this direction but argues it is a downstream fix — it helps the optimizer cope with poor exploration rather than improving exploration quality itself. Lens instead aims to make exploration better so that informative signals exist in the first place.

The Central Insight: A Token-Level Diagnostic for Exploration Failure

The paper's key discovery is empirical and surprising in its specificity: the authors compute a token-level Interference Score (Equation 1) that measures the absolute log-probability difference between the current policy $\pi_\theta$ and the reference model $\pi_{\text{ref}}$ for each token in the prompt. Tokens with large scores represent places where the policy has deviated substantially from its pre-training distribution — and these deviations, they argue, are not evidence of productive specialization but of over-optimization and noise-driven distortion. Figure 2b shows that interference scores follow a highly skewed distribution: only a small fraction (< 5%) of tokens exhibit high interference, yet removing these tokens dramatically improves rollout success rates.

The theoretical justification draws on work by Rafailov et al. (2024), which established that large KL divergences from the reference policy often signal reward over-optimization rather than genuine learning. The authors extend this logic to the token level: if a model has learned to place disproportionate probability mass on certain prompt tokens (relative to the reference), those tokens are likely acting as spurious features that trigger incorrect reasoning patterns. The reference model provides a stable anchor — it represents the distribution learned from pre-training data, before RL fine-tuning has introduced reward-driven distortions.

This is a concrete, testable hypothesis about why RLVR exploration fails: the model overfits to prompt-level distractors during RL training, learns to attend to tokens that are spuriously correlated with failure, and then cannot recover because those same tokens push every rollout toward the same wrong answer. The interference score mechanism provides a diagnostic tool to identify which tokens are causing this collapse, and the purification step (deleting the top- $k$ highest-scoring tokens) provides a causal intervention that validates the diagnosis when success rates improve.

How the Paper Positions Itself

The paper positions Lens not as a new RL algorithm but as a rollout framework — a plug-and-play mechanism that can wrap around existing RLVR methods (specifically GRPO in this work) to improve the quality of the samples used for gradient computation. This is a pragmatic framing: rather than proposing a fundamentally new optimization procedure, Lens augments the sampling stage of existing approaches.

The key conceptual move is to treat low-success prompts as salvageable rather than disposable. Where DAPO and GRESO see a zero-variance prompt and choose to ignore it, Lens sees the same prompt and asks: "which tokens are causing this, and can we get successful rollouts by removing them?" The second stage (CRPO) then transfers those successful rollouts back to the original prompt, creating a learning signal that teaches the model to be robust to the very interference tokens that caused the original failures. This is the "calibration" aspect — the model learns to produce correct reasoning despite the presence of distractors, rather than only performing well in a sanitized environment.

The paper also positions itself within the broader narrative of sample efficiency in RL for LLMs. The title phrase "Less Noise, More Voice" captures the intended contribution: by removing a small amount of prompt noise, the model's own capability ("voice") can be heard, rather than being drowned out by token-level interference. The efficiency gains — 1.6× faster convergence, 3.88% average accuracy improvement — are presented as downstream consequences of this improved signal quality, not as primary objectives in themselves.

3. Technical Approach

3.1 Reader Orientation

The paper describes a rollout sampling framework called Lens that wraps around an existing RL training loop (specifically GRPO) to improve the quality of the candidate solutions the model generates during training. The core problem it solves is that many prompts produce zero successful rollouts not because they are too hard, but because a small number of prompt tokens actively mislead the model — the solution is to identify and temporarily delete these tokens, generate better rollouts from the cleaned prompt, and then use those rollouts to teach the model to reason correctly even when the distracting tokens are present.

3.2 Big-Picture Architecture (Diagram in Words)

The Lens framework has three major components that operate as a pipeline within each training step:

Interference Token Detector — Given the current policy model and a frozen reference model, computes a per-token "Interference Score" for every token in the prompt, identifying which tokens have large log-probability deviations from the reference distribution. This produces a ranked list of candidate tokens to remove.
Rollout Purification Engine — For prompts with low sampling success (below a threshold $\tau$ ), deletes the top- $k$ highest-scoring interference tokens to create a denoised version of the prompt, samples new rollouts from this cleaned prompt, and collects the successful ones. This acts as an alternative exploration mechanism that bypasses the interference.
Calibrated Rollout Policy Optimization (CRPO) — Takes the successful rollouts from the denoised prompt and substitutes them into the rollout pool for the original prompt (replacing failed rollouts), then applies an importance-weighted, KL-regularized PPO-style update that trains the policy to produce correct reasoning on the original noisy prompt. This is the mechanism that transfers learning from the clean setting back to the real, noisy setting.

Information flows as follows: a batch of prompts enters → each prompt generates a group of rollouts under the current policy → the success rate per prompt is computed → low-success prompts trigger interference detection, purification, and resampling → successful purified rollouts are merged into the original rollout pool → the entire pool is reweighted and used for a calibrated gradient update.

3.3 Roadmap for the Deep Dive

First, the Interference Score (Equation 1): this is the analytical foundation — how the system identifies which tokens are causing harm. We need to understand it before the purification step, because purification depends on scoring.
Second, the purification mechanism itself: how tokens are selected, how many are removed, what threshold governs activation, and the guard condition $g_i$ that prevents unnecessary purification.
Third, the rollout replacement and reweighting logic: how successful denoised rollouts replace failed original rollouts, and how the weighting scheme (Equation 4) modulates their contribution based on the original prompt's difficulty.
Fourth, the CRPO objective function (Equations 5–7): how importance ratios correct for distribution mismatch between the original and denoised prompts, how group-relative advantages are computed with weighted normalization, and how the clipped surrogate objective with KL regularization produces the final gradient update.
Fifth, the algorithm as a whole: we walk through Algorithm 1 step by step to see how all components interlock in the training loop, and then discuss the key design choices and hyperparameter settings that make the framework work.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a methodological paper whose core idea is that exploration failures in RLVR can be traced to a small number of prompt-level interference tokens, and that a two-stage process of (1) temporarily removing those tokens to obtain successful rollouts, then (2) transferring those rollouts back to the original prompt via calibrated policy optimization, can unlock learning signals that standard GRPO misses.

Interference Token Identification: The Interference Score

The analytical core of Lens is a per-token diagnostic that measures how much the current policy's behavior on a given prompt token deviates from a stable reference distribution. The motivation, drawn from Rafailov et al. (2024), is that large deviations from the reference policy in RL fine-tuning often represent over-optimization or noise-driven distortion rather than productive specialisation — the reference model provides a learned prior from pretraining, and tokens where the fine-tuned policy has moved far from that prior are suspicious.

For a token prefix $s$ (the prompt tokens up to, but not including, the position being scored) and the token $a$ that occupies that position, the Interference Score is defined as:

$S_I(s, a) \triangleq \bigl| \log \pi_\theta(a \mid s) - \log \pi_{\text{ref}}(a \mid s) \bigr|$

where $\pi_\theta$ is the current policy model (being trained), $\pi_{\text{ref}}$ is the frozen reference model (the pretrained base model before RL fine-tuning), $a$ is a specific token in the prompt, and $s$ is the prompt prefix preceding that token.

What it computes: for a single token position in the prompt, this equation takes the logarithm of the probability that the current policy assigns to that token given its prefix, subtracts the logarithm of the probability that the reference model assigns to the same token given the same prefix, and takes the absolute value. The result is a non-negative scalar that measures absolute log-probability divergence at that position. A score of zero means the two models assign identical probability; a large score means the fine-tuned policy has substantially amplified or suppressed that token relative to the reference.

Why this form: the absolute log-difference captures both directions of deviation — the fine-tuned policy might assign more probability to a token (over-attending to a distractor) or less probability (suppressing a useful signal), and both are potentially harmful. The log-scale is chosen because it connects directly to KL divergence — the sum of these per-token log-differences over a sequence bounds the token-level KL divergence from the reference, which is the established measure of policy over-optimization in the RLHF literature (Gao et al., 2023). Using the absolute value rather than the signed difference means the score treats over-confidence and under-confidence symmetrically as signs of interference. An alternative would be to use the KL contribution directly ( $\pi_\theta \log(\pi_\theta / \pi_{\text{ref}})$ ), but that would be asymmetric — it would penalise tokens where the policy is over-confident more heavily than tokens where it is under-confident, which doesn't match the intuition that both are problematic for exploration.

The reference model $\pi_{\text{ref}}$ is the pretrained base model before any RL fine-tuning — it is frozen throughout training and never updated. This is critical because it provides a stable, unchanging baseline. If the reference model were itself being updated during RL (as the current policy is), the interference score would lose its meaning — both models would drift together, and the score would measure relative change rather than absolute deviation from a pretraining prior.

The practical operation is: for each token in the prompt, run a forward pass through both $\pi_\theta$ and $\pi_{\text{ref}}$ to obtain the log-probability of that token conditioned on its prefix, compute the absolute difference, and store the result. This produces a vector of scores, one per prompt token. Figure 2b in the paper shows that the distribution of these scores is highly skewed — the vast majority of tokens have near-zero scores, while only a small fraction (< 5%) exhibit large deviations. This skew is what makes pruning viable: removing only the top few percent of tokens by interference score eliminates the distorting signal while preserving the vast majority of the prompt's semantic content.

Interference Purification: Pruning and Resampling

Once interference scores are computed for all tokens in a prompt, the purification step selects which tokens to delete and generates new rollouts from the cleaned prompt. This is not applied uniformly — it is a conditional intervention triggered only when a prompt's empirical success rate falls below a threshold.

The deletion ratio $\gamma$ controls what fraction of prompt tokens are removed. For a prompt $x_i$ containing $|x_i|$ tokens after tokenization, the system selects the top $k$ tokens with the highest interference scores, where:

$k = \lceil \gamma \cdot |x_i| \rceil$

The paper sets $\gamma$ to a small value in the range 1% to 5%, with the default being explored through a sensitivity analysis in Section 4.3. The ceiling function $\lceil \cdot \rceil$ ensures at least one token is removed when $\gamma > 0$ and the prompt is non-empty, even for very short prompts. The set of selected tokens is denoted $I_i$ , and the denoised prompt is defined as:

$x'_i = x_i \setminus I_i$

which means: take the original prompt sequence and delete all tokens that appear in the interference set, preserving the order of the remaining tokens. This is a simple deletion operation — no tokens are reordered, substituted, or inserted. The paper explicitly claims this "preserves the original semantics of the prompt with minimal impact" because the deletion fraction is small (1–5%).

Why deletion and not masking or replacement? The paper does not provide an explicit ablation against alternative token-modification strategies (masking with a special token, replacing with a neutral token), but the implicit logic is that deletion is the most conservative intervention: it removes the suspected distractor entirely rather than inserting a new token that could itself introduce artifacts. Masking with a [MASK] token, for instance, would create a prompt that never occurred in pretraining (since pretraining masks are random, not targeted at high-interference positions), potentially causing the model to behave unpredictably. Deletion produces a sequence that — while slightly shorter — consists entirely of tokens that appeared in the original prompt, maintaining distributional proximity to natural language.

The success rate threshold $\tau$ is the gate that determines whether purification is activated for a given prompt. For each prompt $x_i$ , the system first samples $m$ rollouts (the group size, typically 8 in the paper's configuration) from the current policy $\pi_\theta(\cdot \mid x_i)$ and computes the initial empirical success rate:

$\bar{a}_i = \frac{|Y_i^+|}{|Y_i|}$

where $Y_i$ is the full set of $m$ rollouts, and $Y_i^+ \subseteq Y_i$ is the subset that received a positive (correct) reward. If $\bar{a}_i \geq \tau$ , the prompt is considered to have sufficient exploration quality and no purification is performed — the original rollouts are used as-is. If $\bar{a}_i < \tau$ , the prompt triggers the purification process.

The paper's sensitivity analysis (Appendix C) tests $\tau \in \{0.125, 0.25, 0.375, 0.5\}$ and finds that $\tau = 0.5$ provides the best aggregate performance, though lower values concentrate the calibration signal on the hardest prompts and can benefit high-difficulty benchmarks. With $m = 8$ rollouts and $\tau = 0.5$ , purification activates when fewer than 4 out of 8 rollouts succeed — meaning the prompt is producing mostly failures.

When purification activates, the system samples an additional $m$ rollouts from the denoised prompt:

$Y'_i \sim \pi_\theta(\cdot \mid x'_i)$

and computes the denoised accuracy $\text{acc}(x'_i) = |Y'_i{}^+| / m$ , where $Y'_i{}^+$ is the set of successful rollouts from the denoised prompt.

The Guard Condition: Verifying That Purification Actually Helped

A crucial design choice is that Lens does not blindly use the denoised rollouts — it verifies that purification produced a genuine improvement before incorporating them. This is encoded in the binary guard variable:

$g_i = \mathbb{I}\big[\text{acc}(x'_i) > \text{acc}(x_i)\big]$

where $\mathbb{I}[\cdot]$ is the indicator function (1 if the condition is true, 0 otherwise), $\text{acc}(x'_i)$ is the empirical accuracy from the denoised prompt, and $\text{acc}(x_i)$ is the empirical accuracy from the original prompt (which equals $\bar{a}_i$ ).

What it computes: a boolean flag that is 1 only when sampling from the cleaned prompt yields strictly more correct answers than sampling from the original prompt, and 0 otherwise (including the case where accuracy is equal or worse).

Why this form: the paper notes that only about 20% of prompts exhibit improvement after token removal (Figure 2c in Section 1). For the other 80%, purification either doesn't help or actively hurts — removing tokens can degrade semantics enough to reduce success rates. The guard condition prevents Lens from polluting the training data with rollouts from a denoised prompt that is no better (or worse) than the original. Without this check, the framework would sometimes replace failed original rollouts with equally bad or worse denoised rollouts, introducing noise rather than signal. The strict inequality (> rather than >=) biases toward conservatism: only activate the replacement when there is clear evidence of improvement.

The guard condition also serves as a validation of the interference hypothesis. When $g_i = 1$ , it confirms that the tokens identified by the interference score were genuinely causal in the exploration failure — removing them produced measurable improvement. When $g_i = 0$ , it suggests either that (a) the failure was not due to interference tokens but to genuine problem difficulty, or (b) the deletion removed semantically important tokens along with the interference, degrading the prompt rather than purifying it.

Rollout Replacement: Merging Purified Successes into the Original Pool

When $g_i = 1$ , the system takes the successful rollouts from the denoised prompt and uses them to replace failed rollouts from the original prompt. This creates a reconstructed rollout set $\mathcal{G}_i$ for the prompt.

Let $P_i$ be the set of successful rollouts from the denoised prompt (i.e., $P_i = Y'_i{}^+$ , the correct subset of $Y'_i$ ). The system randomly samples a subset $R_i$ from the original failed rollouts $Y_i^-$ such that:

$|R_i| = \min(|Y_i^-|, |P_i|)$

This means: we replace as many failed rollouts as we can, up to either the number of original failures or the number of purified successes, whichever is smaller. The min ensures we never try to replace more failures than we have purified successes to replace them with.

The reconstructed rollout set is then:

$\mathcal{G}_i = \begin{cases} Y_i^+ \cup (Y_i^- \setminus R_i) \cup P_i, & \text{if } g_i = 1 \\ Y_i, & \text{otherwise} \end{cases}$

What this builds: when purification succeeds ( $g_i = 1$ ), the new rollout pool contains three components: (1) all original successful rollouts ( $Y_i^+$ ), kept because they are already correct on the noisy prompt; (2) the original failed rollouts that were not selected for replacement ( $Y_i^- \setminus R_i$ ), kept to preserve some diversity of incorrect responses for the advantage computation; and (3) the purified successful rollouts ( $P_i$ ) from the denoised prompt. When purification fails or isn't triggered ( $g_i = 0$ ), the rollout pool is just the original set $Y_i$ , unchanged.

Why this mixed composition: the presence of both correct (from $Y_i^+$ and $P_i$ ) and incorrect (from $Y_i^- \setminus R_i$ ) rollouts in $\mathcal{G}_i$ is essential. GRPO's advantage computation requires within-group reward variance — if all rollouts in a group were correct, the advantages would be zero and no learning would occur. By retaining some original failures, the reconstructed set maintains the variance needed for meaningful gradient signals, while the injected purified successes provide positive examples that demonstrate correct reasoning trajectories.

The paper also maintains a prompt mapping function $x^{\text{roll}}(y)$ that records, for each rollout $y$ in the reconstructed set, which prompt variant was used to sample it:

$x^{\text{roll}}(y) = \begin{cases} x_i, & \text{for } y \in Y_i^+ \cup (Y_i^- \setminus R_i) \text{ (original prompt rollouts)} \\ x'_i, & \text{for } y \in P_i \text{ (denoised prompt rollouts)} \end{cases}$

This mapping is critical for the importance correction in the objective function (Equation 5) — rollouts from $x'_i$ were sampled under a different distribution than the one being optimized (which conditions on $x_i$ ), and the importance ratio must account for this.

Sample Reweighting: Scaling Contributions by Prompt Difficulty

Simply merging purified rollouts into the pool is not enough — the system needs to weight their contribution appropriately in the gradient update. The paper introduces a sample reweighting scheme that modulates each rollout's influence based on the original prompt's success rate $\bar{a}_i$ .

For each rollout $y$ in the reconstructed set $\mathcal{G}_i$ , the unnormalized weight is:

$\tilde{w}(y) = \begin{cases} \bar{a}_i, & y \in Y_i^+ \\ 1 - \bar{a}_i, & y \in P_i \cup (Y_i^- \setminus R_i) \end{cases}$

What it computes: two weight values, assigned based on whether the rollout was originally successful or not. Original successes ( $Y_i^+$ ) receive weight $\bar{a}_i$ ; everything else — purified successes and retained original failures — receives weight $1 - \bar{a}_i$ .

Why this form: the weighting scheme implements an adaptive emphasis on different types of rollouts based on how difficult the prompt is:

When $\bar{a}_i$ is very low (close to 0, highly challenging prompt): original successes are extremely rare and therefore highly informative — they represent the few trajectories where the model managed to reason correctly despite interference. They receive very low weight ( $\bar{a}_i \approx 0$ ), which might seem counterintuitive, but this serves to down-weight these rare successes relative to the much larger pool of failures and purified successes. The failures and purified successes receive weight close to 1 ( $1 - \bar{a}_i \approx 1$ ), meaning the majority of the learning signal comes from contrasting the purified correct trajectories against the original incorrect ones. This makes sense: on very hard prompts, the few original successes are not representative of the model's typical behavior, and over-weighting them could cause the policy to overfit to lucky samples.
When $\bar{a}_i$ is moderate (e.g., 0.5): original successes and the combined failure/purified set receive equal weight (0.5 each), giving balanced emphasis to both the model's existing capability and the improved trajectories from purification.
When $\bar{a}_i$ is high (close to 1, easy prompt): the prompt likely doesn't trigger purification at all (since $\bar{a}_i$ would exceed $\tau$ ), but if it did, original successes would receive high weight and failures would receive low weight — the model mostly gets it right already, so the few failures are probably random noise rather than systematic interference.

The weights are unnormalized at this stage — they will be normalized within each group during the advantage computation (Equation 6) by using weighted means and standard deviations. The key property is that the weighting is prompt-adaptive: the same rollout (e.g., a purified success) receives different weight depending on whether it came from a prompt where $\bar{a}_i = 0.1$ (very hard) or $\bar{a}_i = 0.4$ (moderately hard), reflecting the different informational value of that rollout in each context.

The CRPO Objective Function: Importance Correction

The objective function for policy optimization must handle a subtle distribution mismatch: rollouts in $\mathcal{G}_i$ were sampled from two different prompt distributions — some from the original prompt $x_i$ , some from the denoised prompt $x'_i$ — but the policy being optimized is $\pi_\theta(\cdot \mid x_i)$ , the policy conditioned on the original prompt. To use rollouts from $x'_i$ to update $\pi_\theta(\cdot \mid x_i)$ , we need importance sampling to correct for the difference in sampling distributions.

The importance ratio is defined as:

$\rho(y; \theta) = \frac{\pi_\theta(y \mid x_i)}{\pi_{\text{old}}(y \mid x^{\text{roll}}(y))}$

where $\pi_\theta$ is the current policy (being optimized), $\pi_{\text{old}}$ is the policy that was used to sample the rollouts (the "old" policy, frozen during this update), $y$ is a rollout, $x_i$ is the original prompt (always the conditioning variable in the numerator), and $x^{\text{roll}}(y)$ is the prompt variant that was actually used to sample $y$ — either $x_i$ or $x'_i$ , as recorded by the prompt mapping function.

What it computes: for each rollout, the ratio of two probabilities: (1) the probability that the current policy would generate this exact rollout conditioned on the original prompt, divided by (2) the probability that the old sampling policy assigned to this rollout conditioned on whatever prompt was actually used to generate it. This is a standard importance weight — it tells us how much more or less likely the current policy is to produce this rollout (under the original prompt) compared to how likely it was under the sampling distribution.

Why this form: this is the standard PPO importance ratio but with a crucial modification — the denominator conditions on $x^{\text{roll}}(y)$ rather than always on $x_i$ . When $y$ was sampled from the original prompt ( $x^{\text{roll}}(y) = x_i$ ), this reduces to the standard PPO ratio $\pi_\theta(y | x_i) / \pi_{\text{old}}(y | x_i)$ . When $y$ was sampled from the denoised prompt ( $x^{\text{roll}}(y) = x'_i$ ), the denominator is $\pi_{\text{old}}(y | x'_i)$ — the probability under the old policy conditioned on the cleaned prompt. This corrects for the fact that $y$ might be much more (or less) likely under $x'_i$ than under $x_i$ ; the importance ratio accounts for this difference so that the gradient update is unbiased.

Without this correction, simply treating denoised rollouts as if they came from the original prompt would produce biased gradient estimates — the policy would be updated as if it had generated those rollouts under the original prompt, when in reality they were generated under an easier (interference-free) condition. The importance ratio down-weights rollouts that were much easier to produce under the denoised prompt (preventing the policy from being overly rewarded for behavior it couldn't replicate under noisy conditions) and up-weights rollouts that were equally difficult under both prompts.

The CRPO Objective Function: Weighted Group-Relative Advantages

GRPO computes advantages within each prompt group by normalizing rewards relative to the group mean and standard deviation. Lens modifies this procedure to use the weighted statistics of the reconstructed rollout set $\mathcal{G}_i$ .

First, the unnormalized weights $\tilde{w}(y)$ from Equation 4 are normalized to sum to 1 within the group:

$w(y) = \frac{\tilde{w}(y)}{\sum_{y' \in \mathcal{G}_i} \tilde{w}(y')}$

Then, the weighted mean of rewards in $\mathcal{G}_i$ is:

$\mu_w(\mathcal{G}_i) = \sum_{y' \in \mathcal{G}_i} w(y') \cdot r(y')$

and the weighted standard deviation is:

$\sigma_w(\mathcal{G}_i) = \sqrt{\sum_{y' \in \mathcal{G}_i} w(y') \cdot \big(r(y') - \mu_w(\mathcal{G}_i)\big)^2}$

The calibrated advantage for rollout $y$ is then:

$\hat{A}(y) = \frac{r(y) - \mu_w(\mathcal{G}_i)}{\sigma_w(\mathcal{G}_i)}$

What it computes: a standardized advantage score for each rollout, measuring how much better (positive) or worse (negative) its reward is compared to the weighted average reward in its group, expressed in units of weighted standard deviation. A rollout with reward equal to the weighted mean gets advantage 0; a rollout one weighted standard deviation above the mean gets advantage +1; a rollout two standard deviations below gets advantage -2.

Why weighted normalization: the weights from Equation 4 are designed to modulate how much each rollout influences the group statistics. Consider a very hard prompt ( $\bar{a}_i \approx 0.1$ ) where the group contains 1 original success, 2 purified successes, and 5 retained failures. The original success gets weight $\bar{a}_i = 0.1$ , the purified successes and failures each get weight $1 - \bar{a}_i = 0.9$ . The weighted mean is therefore:

$\mu_w \approx \frac{0.1 \cdot 1 + 2 \cdot 0.9 \cdot 1 + 5 \cdot 0.9 \cdot 0}{0.1 + 2 \cdot 0.9 + 5 \cdot 0.9} = \frac{0.1 + 1.8}{0.1 + 1.8 + 4.5} = \frac{1.9}{6.4} \approx 0.297$

The original success is only 0.297 - 0 = 0.297 above the weighted mean — a relatively small advantage. If we used unweighted statistics, the mean would be $3/8 = 0.375$ and the original success would be $1 - 0.375 = 0.625$ above the mean — a much larger advantage. The weighting scheme deliberately compresses the advantage of the rare original success to prevent the policy from overfitting to it, while still providing a positive signal. Meanwhile, the purified successes receive advantages of $1 - 0.297 \approx 0.703$ in the weighted scheme versus $1 - 0.375 = 0.625$ unweighted — they are amplified relative to unweighted normalization because they carry the primary learning signal (they demonstrate correct reasoning that the model should learn to replicate on the noisy prompt).

The weighted standard deviation serves a similar purpose: it prevents the group variance from being dominated by the few original successes (which, being rare, would otherwise create artificially high variance), and instead distributes influence more evenly across the larger pool of purified successes and failures.

The CRPO Objective Function: Full Loss

The final training objective combines the importance-weighted, advantage-scaled policy gradient with a KL regularization term against the reference model:

$\mathcal{L}(\theta) = -\sum_{y \in \mathcal{G}_i} \tilde{w}(y) \min\Big(\rho(y; \theta) \hat{A}(y),\; \text{clip}(\rho(y; \theta), 1 - \epsilon, 1 + \epsilon) \hat{A}(y)\Big) + \beta \, \mathbb{D}_{\mathrm{KL}}\big(\pi_\theta(\cdot \mid x_i) \,\|\, \pi_{\text{ref}}(\cdot \mid x_i)\big)$

where $\rho(y; \theta)$ is the importance ratio from Equation 5, $\hat{A}(y)$ is the calibrated advantage from Equation 6, $\epsilon$ is the PPO clipping parameter, $\beta$ is the KL penalty coefficient, and $\mathbb{D}_{\mathrm{KL}}$ is the Kullback-Leibler divergence between the current policy and the reference model.

What it computes, term by term:

The summation runs over all rollouts in the reconstructed set $\mathcal{G}_i$ . Each rollout contributes a weighted, clipped surrogate loss.
The $\min$ operator implements PPO's pessimistic clipping: it takes the minimum of the unclipped importance-weighted advantage ( $\rho \hat{A}$ ) and the clipped version ( $\text{clip}(\rho, 1 - \epsilon, 1 + \epsilon) \hat{A}$ ). When the importance ratio exceeds $1 + \epsilon$ (policy became much more likely to produce this rollout) and the advantage is positive, the clip caps the update to prevent overly large policy changes. When the ratio drops below $1 - \epsilon$ and the advantage is negative, the clip similarly bounds the update.
The $\tilde{w}(y)$ multiplier applies the prompt-adaptive weight from Equation 4, scaling each rollout's contribution before summation. This is why the weights are "unnormalized" — the loss sums weighted contributions directly rather than averaging.
The negative sign makes this a minimization objective: minimizing $-\tilde{w} \rho \hat{A}$ corresponds to maximizing $\tilde{w} \rho \hat{A}$ , which increases the probability of rollouts with positive advantages and decreases the probability of rollouts with negative advantages.
The KL divergence term $\beta \mathbb{D}_{\mathrm{KL}}(\pi_\theta \| \pi_{\text{ref}})$ penalizes the policy for deviating too far from the reference model, preventing reward over-optimization and catastrophic forgetting. The paper sets $\beta = 0.001$ .

Why this form: this is structurally identical to the standard GRPO/PPO objective but with two Lens-specific modifications: (1) the rollout pool $\mathcal{G}_i$ includes purified rollouts from the denoised prompt, and (2) the per-rollout weights $\tilde{w}(y)$ modulate contributions based on the original prompt's difficulty. The clipping mechanism and KL penalty are standard and unchanged — Lens inherits their stabilization properties. The key insight is that Lens's improvements come entirely from what rollouts are in the pool and how they are weighted, not from changing the optimization algorithm itself. This makes it truly a plug-and-play sampling framework: it can be applied to any advantage-based RL algorithm that operates on grouped rollouts.

The importance ratio $\rho(y; \theta)$ in the objective always conditions the numerator on the original prompt $x_i$ , regardless of which prompt was used to sample $y$ . This is the mechanism by which learning transfers from the denoised setting to the noisy setting: when the policy update increases $\pi_\theta(y \mid x_i)$ for a purified success $y$ , it is teaching the model to produce that correct reasoning trajectory even when the interference tokens are present. The policy never learns to rely on the absence of interference tokens — it learns to produce correct answers despite them.

Algorithmic Walkthrough: How Lens Operates End-to-End

Algorithm 1 in the paper provides the complete procedure. Here is a step-by-step walkthrough of what happens in one training iteration:

Step 1: Sample a batch. Draw a batch of $B$ prompts from the training dataset $\mathcal{D}$ (line 3). In the paper's configuration, $B = 128$ .

Step 2: Initial rollout sampling. For each prompt $x_i$ , sample $m$ independent rollouts from the current policy $\pi_\theta(\cdot \mid x_i)$ (line 5). The paper uses $m = 8$ (group size 8). Each rollout is a complete generated solution, up to 4096 tokens in length, sampled at temperature 1.0 with top-p 1.0.

Step 3: Partition and assess. Split the $m$ rollouts into success set $Y_i^+$ (correct answers, reward 1) and failure set $Y_i^-$ (incorrect answers, reward 0) based on the verifiable reward signal (line 6). Compute the empirical success rate $\bar{a}_i = |Y_i^+| / m$ (line 7). Initialize the training group $\mathcal{G}_i$ to the original rollouts and the prompt mapping to all-original (line 8).

Step 4: Conditional purification. If $\bar{a}_i < \tau$ (line 9), the prompt has low exploration success and triggers the Lens pipeline:

4a. Score interference: compute the interference score $S_I(t)$ for every token $t$ in $x_i$ using Equation 1, which requires forward passes through both $\pi_\theta$ and the frozen $\pi_{\text{ref}}$ (line 11).
4b. Prune: sort tokens by descending interference score and delete the top- $k$ tokens to produce the denoised prompt $x'_i$ (line 12). The deletion count $k$ is precomputed from the ratio $\gamma$ and the prompt length.
4c. Resample: sample $m$ new rollouts $Y'_i$ from the denoised prompt $\pi_\theta(\cdot \mid x'_i)$ and compute the denoised accuracy $\text{acc}(x'_i)$ (line 13).
4d. Guard check: if $\text{acc}(x'_i) > \bar{a}_i$ $acc (x_{i}^{'}) > \overset{a}{ˉ}_{i}$ (line 14), purification produced genuine improvement:
- Extract the successful rollouts $P_i$ from $Y'_i$ (line 15).
- Randomly select a subset $R_i$ of original failures to replace, with size $\min(|Y_i^-|, |P_i|)$ (line 16).
- Update the training group: $\mathcal{G}_i \leftarrow (Y_i \setminus R_i) \cup P_i$ — remove the selected failures, add the purified successes (line 17).
- Update the prompt mapping: for each purified rollout $y \in P_i$ , record that it came from $x'_i$ (line 18).
4e. If guard fails ( $\text{acc}(x'_i) \leq \bar{a}_i$ ): do nothing — keep the original rollouts. This is the "only about 20% of prompts benefit" case from Figure 2c.

Step 5: Policy calibration. With $\mathcal{G}_i$ finalized for all prompts in the batch:

Compute importance ratios $\rho(y; \theta)$ for all rollouts using Equation 5 (line 22). This requires evaluating $\pi_\theta(y \mid x_i)$ for the current policy and $\pi_{\text{old}}(y \mid x^{\text{roll}}(y))$ for the old sampling policy.
Compute normalized weights $w(y)$ from the unnormalized $\tilde{w}(y)$ using Equation 4 (line 22).
Compute weighted group-relative advantages $\hat{A}(y)$ using Equation 6 (line 23).
Compute the full loss $\mathcal{L}(\theta)$ via Equation 7 and perform a gradient update on the policy parameters (line 23).

What happens to prompts that are not purified: for prompts where $\bar{a}_i \geq \tau$ , the training group $\mathcal{G}_i$ is just the original rollouts $Y_i$ , all rollouts have $x^{\text{roll}}(y) = x_i$ , the importance ratio collapses to the standard PPO ratio, and the weights $\tilde{w}(y)$ still apply (giving different emphasis to successes vs. failures based on $\bar{a}_i$ ) but there are no denoised rollouts to integrate. In this case, Lens reduces to a weighted version of standard GRPO — the prompt-adaptive weighting is always active, even when purification is not triggered.

Key Design Choices and Hyperparameter Settings

The paper reports several specific configurations:

KL coefficient $\beta = 0.001$ : controls the strength of regularization toward the reference model. This is small enough to allow the policy to move substantially from the base model but large enough to prevent extreme over-optimization.
Group size $m = 8$ : the number of rollouts per prompt. This is standard for GRPO and balances the need for within-group variance (more rollouts give better advantage estimates) against computational cost.
Batch size 128: both the rollout batch (number of prompts per iteration) and the update batch. All 128 prompts are processed, their rollouts are collected (up to 8 per prompt = 1024 total rollouts), and a single gradient step is taken on the aggregate loss.
Learning rate $1 \times 10^{-6}$ : a standard small learning rate for RL fine-tuning of LLMs.
Maximum response length 4096 tokens: the generation is truncated at this length.
Temperature 1.0 and top-p 1.0 during both training and evaluation: full sampling without temperature scaling or nucleus filtering, maximising exploration diversity.
Pruning ratio $\gamma \in [1\%, 5\%]$ : the default is explored through sensitivity analysis; optimal values depend on model capacity (Section 4.3).
Success rate threshold $\tau = 0.5$ (default): triggers purification when fewer than half of the rollouts succeed. Appendix C shows this provides the best aggregate performance.
PPO clipping $\epsilon$ : the standard PPO clipping parameter (value not explicitly stated in the main text, but typical values are 0.1–0.2).

The paper uses the verl framework on 8× NVIDIA A800 GPUs for all experiments. The computational overhead of Lens relative to GRPO is 1.27× to 1.62× per training step (Table 5 in Appendix D), primarily from the extra forward passes for interference scoring (requires running $\pi_{\text{ref}}$ on all prompt tokens) and the additional resampling from denoised prompts. The paper argues this overhead is justified by the 2–3% absolute accuracy gains, and notes that it is more efficient than the alternative of simply doubling the group size ( $m = 16$ ), which would cost 2× without the targeted signal improvement.

Why Lens Works: The Transfer Mechanism

The most subtle aspect of Lens is understanding why training on the original prompt with purified rollouts teaches robustness to interference. The model never sees the denoised prompt during the policy update — all conditioning in the objective (Equation 7) is on $x_i$ , the original noisy prompt. The purified rollouts serve only as target trajectories that the policy should learn to produce when conditioned on $x_i$ .

This works because the policy update increases $\pi_\theta(y \mid x_i)$ for purified successes $y$ (which have positive advantages) and decreases it for original failures (which have negative advantages). But $y$ was generated under $x'_i$ — a prompt missing the interference tokens. For the policy to increase $\pi_\theta(y \mid x_i)$ while the interference tokens are present in $x_i$ , it must learn to ignore those tokens when generating the response. The interference tokens are still in the input, but their influence on the policy's output distribution is reduced because the gradient signal pushes the policy toward producing the same output regardless of their presence.

This is fundamentally different from simply training on denoised prompts (which would create a model that only works well on clean inputs) or from filtering out difficult prompts (which avoids the problem entirely). Lens actively teaches the model to be robust to interference by providing positive examples of what correct reasoning looks like on the noisy prompt, even though those examples were discovered under easier conditions. The importance ratio corrects for the fact that the examples were easier to discover, preventing the policy from being over-rewarded, but the directional signal — "produce this kind of answer when you see this prompt" — is preserved.

The paper's training dynamics analysis (Appendix F, Figure 8) supports this interpretation: Lens-trained models show lower entropy (more decisive reasoning) and earlier emergence of long-form reasoning behaviors compared to standard GRPO, suggesting they have learned to focus on the signal-relevant aspects of the prompt while suppressing interference.

4. Key Insights and Innovations

Innovation 1: Reframing Exploration Failure as a Token-Level Interference Problem

The dominant assumption across prior RLVR work — implicit in both scaling exploration (Xu et al., 2025; Yang et al., 2025b) and prompt filtering (Yu et al., 2025; Zheng et al., 2025a) — is that when a prompt produces zero successful rollouts, it reflects a fundamental capability gap: the model simply cannot solve that problem. This framing treats the prompt as an atomic unit and exploration failure as a property of problem difficulty.

Lens advances a fundamentally different diagnosis: exploration failure is often not about the problem being too hard, but about a small number of prompt tokens actively misleading the model toward incorrect reasoning trajectories. The evidence is causal and striking — deleting fewer than 5% of tokens from previously failed prompts improves rollout accuracy by over 20% across all tested model families (Figure 2c). This is not an incremental improvement from better sampling; it demonstrates that the model already possesses the capability to solve many of these problems, but cannot express it when interference tokens are present in its input context.

What makes this idea intellectually distinctive is the diagnostic inversion it performs relative to prior work. Credit assignment methods (Kazemnejad et al., 2024; Li et al., 2025) ask which output tokens contributed most to the reward — they analyze the model's generated response to understand where reasoning succeeded or failed. Prompt filtering asks whether an entire prompt should be kept or discarded based on aggregate statistics. Lens instead decomposes the prompt itself into helpful and harmful components, asking: "which tokens in the instruction are causing the failure?" This shifts the analytical focus from the model's output distribution to its input processing — from "what the model wrote" to "what the model read that misled it." The Interference Score (Equation 1 in Section 3) is the measurement tool for this decomposition, but the conceptual contribution is recognizing that difficulty is not a monolithic property of problems but is localized to specific token-level distractors.

The highly skewed interference score distribution (Figure 2b) reinforces the practical significance of this reframing. It is not that prompts are diffusely noisy; rather, a tiny fraction of tokens carry disproportionate distorting influence. This explains two phenomena that prior work observed but could not explain mechanistically: (1) why increased sampling (GRPO_extended) is so inefficient — every rollout encounters the same token-level distractors, so more samples mostly produce more failures; and (2) why filtering sacrifices learning opportunities — the prompt as a whole is not the problem, only the specific interference tokens within it, and discarding the entire prompt throws away the semantic content that could support learning if the interference were removed.

This is a fundamental conceptual shift rather than an incremental refinement. It redefines the exploration bottleneck in RLVR from a capability problem (the model cannot solve hard problems) to an attention/distraction problem (the model cannot ignore misleading tokens), which has direct implications for what kinds of interventions will be effective at scale.

Innovation 2: Calibrated Transfer — Using Cleaned Prompts for Discovery, Not as Training Targets

The second conceptual innovation is the mechanism by which successful rollouts from denoised prompts are integrated into training. The standard approaches for leveraging easier or cleaner data in machine learning — curriculum learning (start with easy examples, gradually increase difficulty), data augmentation (train on cleaned examples as synthetic data), or distillation (train a student on teacher outputs from clean inputs) — all ultimately train the model to perform well under the cleaned conditions. The model learns $\pi_\theta(\cdot \mid x'_i)$ , a policy conditioned on interference-free inputs.

Lens does something fundamentally different: it uses the denoised prompt solely as a discovery mechanism to find correct rollouts that were inaccessible under the noisy prompt, then trains the model to produce those same rollouts when conditioned on the original noisy prompt $x_i$ . The gradient update in CRPO (Equation 7) always increases $\pi_\theta(y \mid x_i)$ for purified successes $y$ — it never reinforces $\pi_\theta(y \mid x'_i)$ . This means the interference tokens remain in the conditioning context throughout training; the model is explicitly taught to generate correct reasoning despite their presence. The result is learned robustness to distractors, acquired not by being told which tokens are problematic, but by being shown what correct behavior looks like and rewarded for replicating it under noisy conditions.

This inverts the usual clean-to-noisy data relationship in robust machine learning. Adversarial training and noise-augmentation approaches train on noisy data to improve clean-test performance — noise is added during training to force generalization. Lens trains on clean-discovered trajectories to improve noisy-input performance — interference is present during training, and the clean data serves to reveal what correct behavior should be. The importance ratio (Equation 5) is the mathematical device that makes this transfer principled, correcting for the distribution shift between the denoised prompt (under which rollouts were easier to discover) and the original prompt (under which the policy is being optimized). Without this correction, the policy would be over-rewarded for behavior it couldn't replicate under realistic conditions.

The guard condition $g_i$ (Equation 2) reinforces the selective, empirically-validated nature of this transfer. The paper's finding that only approximately 20% of low-success prompts actually benefit from token removal means that blind transfer — always replacing failures with denoised successes — would pollute training. Lens only activates the transfer when purification produces a measurable accuracy improvement, converting the framework from an indiscriminate augmentation scheme into a self-verifying mechanism that discovers during training which prompts suffer from token-level interference and which are genuinely beyond the model's current capability.

Innovation 3: Empirical Proof That Low-Success Prompts Are Underexploited Training Resources

Beyond the methodological contributions, the paper provides an important empirical finding that challenges a growing operational consensus in the RLVR community. The filtering approaches (DAPO, GRESO) rest on a pragmatic premise: zero-variance prompts contribute nothing to learning and risk destabilizing training through vanishing gradients, so discarding them is a net positive. This premise is intuitively appealing — if all rollouts fail identically, what gradient signal can exist? — and has been adopted as a default in several prominent systems.

Lens provides direct counter-evidence showing this premise is empirically false for a substantial fraction of prompts. The head-to-head comparison in Table 1 demonstrates that Lens, which actively extracts training signals from low-success prompts through purification and transfer, consistently outperforms both DAPO and GRESO — even when those baselines are given twice the training epochs (DAPO_extended, GRESO_extended). The performance gap is particularly pronounced on the hardest benchmarks (AMC23, AIME24), precisely where the filtering approaches would discard the largest number of prompts. This result cannot be explained by Lens simply having more training data; it has the same underlying prompt set. It reflects that the discarded prompts contained genuine learning opportunities that filtering threw away.

The training dynamics analysis (Figure 5, Section 4.1) provides mechanistic evidence for the claim. Lens fundamentally reshapes the exploration landscape during training, substantially reducing the proportion of prompts in the "Failure" category (zero successful rollouts) and shifting prompts into higher-success bins. The comparison with GRPO_extended in the same figure is particularly revealing: simply doubling the sample budget ( $n = 16$ vs. $n = 8$ ) does not achieve the same reduction in zero-reward prompts that Lens achieves with the same underlying group size. This confirms that the bottleneck is not sample quantity but sample quality — interference tokens systematically corrupt all rollouts regardless of count, while targeted removal of those tokens converts failure prompts into productive learning opportunities at the current sample budget.

This finding has implications for how the field should think about training data curation in RLVR. The emerging consensus that aggressive difficulty filtering is a necessary engineering compromise may be premature. Lens suggests that the optimization instability attributed to difficult prompts is often actually caused by token-level interference that can be addressed without discarding data. If this finding generalizes beyond the math reasoning domain tested, it could shift default practice from filtering toward interference-aware sampling as the primary mechanism for handling exploration difficulty in RLVR systems.

Innovation 4: Capacity-Dependent Interference Vulnerability as an Emergent Property

Section 4.3 reports a finding that the paper itself seems to treat as a sensitivity analysis result but that has deeper conceptual implications: the optimal pruning threshold for interference tokens is capacity-dependent. Smaller models (Qwen2.5-3B) achieve superior performance with higher deletion ratios, while larger models (Qwen2.5-7B) perform better with lower deletion ratios (the insets in Figure 7).

This is not simply a hyperparameter tuning observation. It reveals an emergent relationship between model scale and susceptibility to token-level interference that has not been previously documented in the RLVR literature. Smaller models, with fewer parameters and more constrained representational capacity, appear more vulnerable to having their reasoning derailed by specific prompt tokens — they cannot as easily learn to ignore distractors, so aggressive removal of those distractors during exploration provides greater benefit. Larger models, with greater capacity, can partially route around interference tokens even without explicit purification, so they benefit from milder intervention that preserves more of the original prompt semantics.

This finding connects to broader questions in the scaling literature about how model capabilities evolve with size. Prior work (e.g., Wei et al., 2022 on emergent abilities) has documented that certain reasoning capabilities appear only above specific scale thresholds. The interference vulnerability pattern suggests a complementary phenomenon: not only do capabilities emerge with scale, but robustness to input-level distractors also improves with scale. Smaller models are brittle in a specific, diagnosable way — they are easily misled by token-level interference — and this brittleness can be partially compensated for by targeted input cleaning during training. Larger models develop implicit robustness that reduces the need for such intervention.

The practical implication is that the relative value of Lens-like purification is not uniform across model scales — it provides greater marginal benefit for smaller models, potentially narrowing the performance gap between small and large models on reasoning tasks when interference is a significant factor. This aligns with the paper's efficiency narrative: Lens helps extract more capability from a given model size, which is most impactful when model capacity is constrained (deployment on edge devices, cost-limited settings).

5. Experimental Analysis

Evaluation Methodology

Dataset. The RL training phase uses Openr1-Math-46k (Yan et al., 2025), a large-scale dataset designed for mathematical reasoning. Evaluation is conducted across seven benchmarks covering a broad difficulty spectrum: MATH500 (Hendrycks et al., 2021), AMC23 (AI-MO, 2024), AIME24, AIME25 (Li et al., 2024), GaokaoEN-2023 (Zhang et al., 2023), Minerva (Lewkowycz et al., 2022), and OlympiadBench (He et al., 2024). The training set consists of 46,000 prompts; evaluation set sizes vary per benchmark but are standard public test splits.
Base model(s). Experiments span three model families and five specific models: Llama-3.2-3B-Instruct (Meta, 2024), Qwen2.5-3B and Qwen2.5-7B (Team et al., 2024), and Qwen3-4B-Base and Qwen3-8B-Base (Yang et al., 2025a). This selection covers instruction-tuned (Llama-3.2, Qwen2.5) and base (Qwen3) variants, and spans 3B to 8B parameters, enabling cross-architecture and cross-scale validation of the method's generality.
Metrics. The primary metric is Pass@1 accuracy — the fraction of evaluation prompts for which a single sampled generation produces the correct final answer, as determined by the verifiable reward function. For high-difficulty benchmarks (AMC23, AIME24, AIME25), results are averaged over 16 generation samples per prompt to reduce variance from the small test set sizes of these competition-level benchmarks. The paper also reports training dynamics metrics including sampling success rate distribution (proportion of prompts with 0, 1–3, 4–6, or 7–8 successful rollouts out of 8), training reward, policy entropy, and response length (Appendix F).
Baselines. The paper compares Lens against four categories of approaches:
- GRPO (Shao et al., 2024a): the vanilla Group Relative Policy Optimization algorithm with group size $m = 8$ , serving as the primary baseline and the foundation upon which Lens is built.
- GRPO_extended: GRPO with doubled rollout budget ( $m = 16$ ), representing the "scaling exploration" strategy of increasing sample count to combat sparse rewards (Xu et al., 2025; Yang et al., 2025b).
- DAPO (Yu et al., 2025): a post-rollout filtering approach that discards prompts where all sampled rollouts receive identical zero rewards to prevent zero-variance gradient updates. DAPO_extended trains DAPO for twice the number of epochs.
- GRESO (Zheng et al., 2025a): a pre-rollout filtering approach that predicts which prompts will yield zero-variance rollouts and skips them before sampling. GRESO_extended trains GRESO for twice the number of epochs.
The paper emphasises that Lens operates under a strictly lower computational budget than GRPO_extended (which doubles rollout counts) and DAPO_extended/GRESO_extended (which double training epochs), making the comparisons intentionally unfavorable to Lens in terms of raw compute allocation.
Generation budget / compute accounting. The unit of comparison is the number of rollouts per prompt per training step ( $m$ ). Standard GRPO and Lens both use $m = 8$ ; GRPO_extended uses $m = 16$ . For Lens, the additional computation from interference scoring (forward passes through $\pi_{\text{ref}}$ on all prompt tokens) and denoised-prompt resampling is reported as a 1.27× to 1.62× per-step overhead relative to GRPO (Appendix D, Table 5). This overhead is factored into the efficiency analysis: Lens claims faster convergence in terms of gradient steps (1.67× fewer steps to reach GRPO's peak accuracy on MATH500), and the paper argues this more than compensates for the per-step overhead. All training uses 8× NVIDIA A800 GPUs with the verl framework.
Cross-validation / statistical protocol. The paper does not employ cross-validation for strategy selection. Instead, it uses a fixed set of hyperparameters (success rate threshold $\tau = 0.5$ , pruning ratio $\gamma \in [1\%, 5\%]$ , group size $m = 8$ ) chosen through sensitivity analyses reported in Section 4.3 and Appendix C. These sensitivity analyses sweep $\gamma$ across 1%–5% and $\tau$ across $\{0.125, 0.25, 0.375, 0.5\}$ on the validation benchmarks, with the best aggregate setting ( $\tau = 0.5$ ) selected as the default. The paper reports results on the standard test splits of each benchmark without apparent contamination from hyperparameter tuning on test data. Training is conducted for 300 steps, with evaluation checkpoints at regular intervals.

Main Quantitative Results

Aggregate Performance Across Benchmarks

Table 1 presents the comprehensive evaluation across seven benchmarks for Qwen3-4B-Base, Qwen3-8B-Base, and Llama-3.2-3B-Instruct. The headline result: Lens achieves an average performance gain of 3.88% over GRPO across all configurations and benchmarks.

On Qwen3-4B-Base, Lens dominates convincingly. On MATH500, Lens reaches 72.2% versus GRPO's 68.4% (+3.8 points). On AMC23 — a significantly harder competition benchmark — Lens achieves 50.0% versus GRPO's 41.7% (+8.3 points, the largest single-benchmark gain on this model). On AIME24, Lens reaches 26.7% versus GRPO's 20.0% (+6.7 points). The gains are systematic: Lens outperforms GRPO on 7/7 benchmarks for this model, with an average improvement of approximately 5.0 percentage points. Even against GRPO_extended (which uses double the rollout budget, $m = 16$ ), Lens achieves superior performance on 6/7 benchmarks, with the sole exception being AIME25 where GRPO_extended scores 13.3% to Lens's 10.0%. The filtering baselines DAPO and GRESO trail Lens on all benchmarks, typically by 3–8 percentage points on the harder competition datasets.

On Qwen3-8B-Base, the pattern holds but with compressed margins, consistent with the paper's capacity-dependent interference hypothesis (Section 4.3). Lens achieves 78.4% on MATH500 versus GRPO's 75.6% (+2.8 points). On AMC23, Lens reaches 62.5% versus GRPO's 54.2% (+8.3 points — matching the 4B gain, suggesting interference is particularly acute on this benchmark regardless of scale). On AIME24, Lens reaches 33.3% versus GRPO's 30.0% (+3.3 points). The average gain across all seven benchmarks is approximately 2.5 percentage points — smaller than on the 4B model, which supports the claim that larger models are partially robust to interference but still benefit from purification. Lens outperforms GRPO_extended on 5/7 benchmarks and outperforms DAPO and GRESO on all benchmarks.

On Llama-3.2-3B-Instruct, Lens achieves 65.6% on MATH500 versus GRPO's 58.8% (+6.8 points). This is notable because Llama-3.2 is an instruction-tuned model from a different family than the Qwen models, and the large gain suggests interference vulnerability is not architecture-specific. On AMC23, Lens reaches 35.0% versus GRPO's 28.3% (+6.7 points). The average gain across seven benchmarks is approximately 4.2 percentage points. Lens outperforms all baselines — GRPO, GRPO_extended, DAPO, DAPO_extended, GRESO, and GRESO_extended — on 7/7 benchmarks for this model, a clean sweep.

Crucially, the extended baselines (DAPO_extended, GRESO_extended) are trained for 2× the number of epochs, meaning they process twice as many gradient updates as Lens. The paper reports that even in this "unfavorable setting where GRPO_extended, DAPO_extended and GRESO_extended are trained for 2× more rollouts, Lens still outperforms them on the majority of benchmarks" (Table 1 note). For instance, on Qwen3-4B-Base, DAPO_extended reaches 67.6% on MATH500 versus Lens's 72.2%; on AMC23, DAPO_extended reaches 43.3% versus Lens's 50.0%.

Appendix B (Table 3) extends these results to Qwen2.5-3B and Qwen2.5-7B, where Lens again achieves the best or second-best performance on the majority of benchmarks. On Qwen2.5-3B, Lens achieves 64.4% on MATH500 versus GRPO's 60.0% (+4.4 points) and DAPO_extended's 60.4%. On Qwen2.5-7B, Lens reaches 76.4% on MATH500 versus GRPO's 73.6% (+2.8 points).

Training Efficiency and Convergence Speed

Figure 4 shows learning curves for Qwen3-4B-Base and Qwen3-8B-Base on MATH500 (medium difficulty) and OlympiadBench (high difficulty). Four observations emerge:

Faster initial convergence. On Qwen3-4B-Base MATH500, Lens separates from GRPO within the first 50 training steps and maintains a consistent accuracy advantage throughout training. By step 100, Lens reaches approximately 66% while GRPO is at approximately 60% — a gap that persists to the end of training (300 steps).

Higher final accuracy. Across all four configurations (2 model sizes × 2 benchmarks), Lens's final accuracy equals or exceeds GRPO's. On Qwen3-4B-Base OlympiadBench, Lens reaches approximately 28% at step 300 versus GRPO's approximately 25%. On Qwen3-8B-Base OlympiadBench, the gap narrows but Lens maintains a small advantage (approximately 27% versus 26%).

Reduced training instability. On the more challenging OlympiadBench, GRPO exhibits visible fluctuations — accuracy oscillates by 3–5 percentage points across consecutive evaluation checkpoints, particularly in the middle training phase (steps 100–200). Lens shows smoother, more monotonic improvement, consistent with the claim that interference purification provides more stable gradient signals. The paper attributes this to two factors: improved exploration capacity from removing interference tokens, and contrastive learning between correct (purified) and incorrect (original) responses that sharpens the model's focus on key information.

Quantified speedup (Figure 6). The paper operationalises efficiency by measuring how many gradient steps each method requires to reach the peak average accuracy attained by GRPO over its entire training trajectory. This is a conservative benchmark: Lens is compared against GRPO's best-ever performance, not its performance at the same step count. On MATH500, Lens reaches GRPO's peak accuracy in approximately 1.67× fewer gradient steps — if GRPO peaks at step 250, Lens matches that accuracy by approximately step 150. On OlympiadBench, the speedup is 1.64×. The gray dashed lines in Figure 6 visually mark these crossover points.

The paper argues this efficiency gain more than compensates for the per-step computational overhead (1.27×–1.62×). A 1.67× reduction in total steps times a 1.5× per-step overhead yields approximately equal total wall-clock time to GRPO, but with the advantages of higher final accuracy and more stable training. The comparison with GRPO_extended is starker: GRPO_extended costs 2× per step (double rollouts) without achieving Lens's accuracy or convergence speed, making Lens the Pareto-dominant option in the performance-efficiency tradeoff.

Training Dynamics: Sampling Success Distributions

Figure 5 (Section 4.1) analyses the distribution of sampling success rates across prompts during three training phases: early (steps 1–100), middle (steps 101–200), and late (steps 201–300). Rollouts per prompt ( $m = 8$ ) are categorised into four bins: Failure (0/8 successful), Low (1–3/8), Mid (4–6/8), and High (7–8/8).

The key finding: Lens substantially reduces the proportion of Failure-category prompts compared to both GRPO and GRPO_extended across all training phases. In the early phase, GRPO shows approximately 40% of prompts in the Failure category; GRPO_extended reduces this to approximately 30% (the doubled sampling budget does help somewhat). Lens reduces it further to approximately 25%. In the middle phase, the gap widens: GRPO still shows approximately 30% Failure, GRPO_extended approximately 25%, and Lens approximately 18%. By the late phase, Lens achieves approximately 15% Failure versus GRPO's 22% and GRPO_extended's 20%.

Correspondingly, Lens shifts more prompts into the Mid and High categories. In the late phase, Lens shows approximately 45% of prompts in the Mid+High combined categories versus approximately 38% for GRPO and 40% for GRPO_extended. This redistribution is what drives the accuracy improvements: more prompts produce informative gradient signals (at least some successful rollouts), and fewer prompts produce zero-variance updates that contribute nothing to learning.

The comparison with GRPO_extended is particularly revealing for the paper's central argument. Doubling the sample budget ( $m = 16$ ) reduces the Failure proportion by only about 5–8 percentage points across phases. Lens achieves an additional 5–7 point reduction on top of that, using the same group size ( $m = 8$ ) but with interference purification. This demonstrates that the bottleneck is not the number of samples but their quality — interference tokens systematically corrupt rollouts regardless of count, and removing them is more effective than simply sampling more.

Model Capacity and Interference Sensitivity (Figure 7, Section 4.3)

Figure 7 shows validation accuracy on MATH500 for Qwen2.5-3B and Qwen2.5-7B across pruning ratios $\gamma \in \{1\%, 2\%, 3\%, 4\%, 5\%\}$ , compared against the GRPO baseline.

Two findings emerge:

Lens consistently matches or outperforms GRPO across all thresholds. For both model sizes and all five pruning ratios, the Lens accuracy curves lie above or overlap with the GRPO baseline throughout training. There is no threshold value at which Lens underperforms GRPO, indicating the approach is robust to the choice of $\gamma$ within the tested range.

The optimal pruning ratio is capacity-dependent. For Qwen2.5-3B (the smaller model), the highest accuracy is achieved with a higher pruning ratio (3–5%). The inset in Figure 7 highlights this: the $\gamma = 4\%$ and $\gamma = 5\%$ curves slightly outperform the lower thresholds in the later training stages. For Qwen2.5-7B (the larger model), the optimal ratio is lower (1–2%), with higher thresholds showing slightly degraded performance. The paper interprets this as evidence that smaller models are more strongly affected by interference tokens and benefit from more aggressive removal, while larger models have greater capacity to route around distractors and perform better with milder intervention that preserves more prompt semantics.

This capacity-dependent pattern has not been previously documented in the RLVR literature and represents an emergent finding with practical implications: the pruning ratio should be tuned based on model scale, with smaller models requiring more aggressive purification.

Success Rate Threshold Sensitivity (Appendix C, Table 4)

The paper sweeps the threshold $\tau$ that triggers purification across $\{0.125, 0.25, 0.375, 0.5\}$ . Lower thresholds restrict purification to prompts with very low success rates (0–1 successes out of 8 for $\tau = 0.125$ ; 0–2 for $\tau = 0.25$ ), while higher thresholds activate purification more broadly (0–3 for $\tau = 0.375$ ; 0–3 plus some prompts with 4+ successes for $\tau = 0.5$ , depending on rounding).

The results in Table 4 show a nuanced tradeoff. $\tau = 0.125$ concentrates the calibration signal on the hardest prompts — those where the model almost never succeeds — and achieves strong performance on high-difficulty benchmarks like Minerva and AMC23 where these frontier prompts matter most. $\tau = 0.5$ provides broader coverage, activating purification on a larger fraction of prompts (not just the hardest), and achieves the best aggregate performance across all seven benchmarks, with enhanced stability. The paper selects $\tau = 0.5$ as the default for all experiments, prioritising overall robustness over benchmark-specific optimisation.

Ablation Studies and Robustness Checks

Pruning strategy comparison (Appendix E, Table 6): Lens is compared against three alternative approaches for improving rollout quality on low-success prompts: (1) Resampling, which generates an additional $m$ rollouts from the original prompt and replaces failures with any new successes; (2) Random Pruning, which deletes the same fraction of tokens ( $\gamma$ ) uniformly at random per prompt rather than using interference scores; and (3) Gradient-based Pruning, which prunes tokens with the smallest gradient norm (a standard importance estimation method). Lens outperforms all three alternatives on all seven benchmarks. On MATH500 with Qwen3-4B-Base, Lens achieves 72.2% versus Resampling at 69.2%, Random Pruning at 68.8%, and Gradient-based Pruning at 69.6%. The gaps are larger on harder benchmarks: on AMC23, Lens reaches 50.0% versus Gradient-based Pruning at 43.3% (+6.7 points). This ablation validates that the interference score (Equation 1) is a meaningfully better token-importance metric for this task than random deletion or gradient-based selection, and that simply resampling more (without interference removal) is less effective.

Computational overhead analysis (Appendix D, Table 5): Lens incurs a 1.27× to 1.62× per-step wall-clock overhead compared to GRPO across model scales and configurations. For Qwen3-4B-Base with group size $G = 8$ , GRPO takes 287 seconds per step while Lens takes 464 seconds (1.62×). For Qwen3-8B-Base with $G = 8$ , the overhead is 1.45×. The paper notes this remains substantially more efficient than GRPO with $G = 16$ (which costs approximately 2× per step without the accuracy benefits), and that the faster convergence (1.67× fewer steps) partially or fully amortizes the per-step cost in total training time while delivering higher final accuracy. The overhead comes primarily from (a) inference through $\pi_{\text{ref}}$ to compute interference scores on all prompt tokens and (b) additional rollout sampling from denoised prompts for low-success prompts.

Training dynamics: reward, entropy, and response length (Appendix F, Figure 8): Lens-trained models exhibit distinct training dynamics compared to GRPO and GRPO_extended across both Qwen3-4B-Base and Qwen3-8B-Base. Training reward (Figures 8a, 8d) increases more steadily and reaches higher final values. Policy entropy (Figures 8b, 8e) is lower for Lens throughout training — the paper interprets this as reduced uncertainty and more decisive reasoning, consistent with the model learning to ignore interference tokens rather than exploring randomly. Response length (Figures 8c, 8f) grows more rapidly for Lens, with the insets highlighting an earlier emergence of long-form reasoning behaviours — the "aha moment" where the model spontaneously begins generating extended chain-of-thought reasoning. This earlier emergence is attributed to improved exploration quality: by reducing interference, Lens helps the model discover productive reasoning trajectories sooner in training.

Cross-model-family validation (Table 1, Table 3): The method is validated across three distinct model families (Llama, Qwen2.5, Qwen3) and five model sizes (3B, 4B, 7B, 8B), with consistent gains over GRPO in all configurations. This addresses the concern that interference purification might be specific to a particular architecture or training recipe. The Llama-3.2-3B-Instruct results are particularly informative because this is an instruction-tuned model (not a base model), and Lens demonstrates large gains (+6.8 points on MATH500), suggesting interference tokens are a problem across model types, not just in base models undergoing RL fine-tuning.

Benchmark difficulty spectrum: The seven evaluation benchmarks span from MATH500 (high-school competition, moderate difficulty) through AMC23/AIME24/AIME25 (elite competition, very high difficulty) to OlympiadBench (olympiad-level, extreme difficulty). Lens shows gains across this entire spectrum, with the largest absolute improvements on medium-to-hard benchmarks (AMC23, AIME24) and smaller but still positive gains on the easiest (MATH500) and hardest (OlympiadBench) benchmarks. This pattern supports the claim that interference tokens are most damaging on problems near the model's capability frontier — where small distractions can push reasoning over the edge from success to failure — and less impactful on problems that are either well within or far beyond the model's reach.

Critical Assessment

Claim: "Lens significantly outperforms GRPO, delivering higher performance and faster convergence, with a 3.88% average gain and over 1.6× speedup"

This claim is supported by the experimental evidence, but with important nuance about what "speedup" means.

The 3.88% average gain is computed across all model-benchmark pairs in Tables 1 and 3. It is a genuine performance improvement: Lens achieves higher Pass@1 accuracy than GRPO on the vast majority of individual benchmark-model combinations (the paper reports wins on the "majority" of configurations for the extended baselines, implying near-universal wins against vanilla GRPO). The gains are not uniform — they range from ~2 points on easy benchmarks with large models to ~8 points on hard benchmarks with small models — and the average is pulled up by larger gains where they matter most (harder tasks). This is a fair summary statistic, though reporting benchmark-specific breakdowns is more informative than the aggregate.

The 1.6× speedup claim requires careful interpretation. The paper measures speedup as "gradient steps to reach GRPO's peak accuracy" (Figure 6). This is a valid metric — it shows Lens reaches a given performance level faster — but it is not the same as "1.6× faster total training time." Because Lens incurs a 1.27×–1.62× per-step overhead (Appendix D), a 1.67× reduction in steps translates to approximately equal or slightly faster total wall-clock time. The paper is transparent about this in Appendix D but the main-text "1.6× speedup" language could be misinterpreted as end-to-end training time improvement. In practice, the primary efficiency benefit is that Lens achieves higher accuracy for the same or slightly less total compute, not dramatically less compute for the same accuracy. This is still a meaningful result — a Pareto improvement in the accuracy-efficiency tradeoff — but it is a more measured claim than "1.6× faster training" would suggest to a casual reader.

A genuine limitation of the speedup analysis is that it only compares to GRPO's peak, not to GRPO's performance at the same step count where Lens reaches that peak. If GRPO peaks at step 250 and Lens matches that at step 150, the relevant comparison is Lens at step 150 versus GRPO at step 150 — a much larger gap in Lens's favour at that point in training. The paper's choice of comparison point (GRPO's peak) is conservative in one sense (GRPO at its best versus Lens mid-training), but it obscures the fact that Lens at step 150 is substantially ahead of GRPO at step 150, which is arguably the more relevant metric for practitioners deciding when to stop training.

Claim: "Lens also exhibits better performance over both scaling exploration and prompt filtering baselines while using substantially fewer computational resources"

This claim has strong support, but "substantially fewer computational resources" deserves qualification.

Against GRPO_extended (scaling exploration), the claim is clear-cut. GRPO_extended uses double the rollout budget ( $m = 16$ vs. $m = 8$ ) and therefore approximately 2× the per-step computation, yet Lens ( $m = 8$ with 1.27–1.62× overhead) outperforms it on the majority of benchmarks. Lens uses less total computation per step (1.62× < 2×) and achieves higher accuracy. This is a genuine efficiency win.

Against DAPO_extended and GRESO_extended (filtering with double epochs), the comparison is less precisely quantified. "Twice the number of training epochs" means these baselines process twice as many gradient updates, consuming approximately 2× the total training compute. Lens outperforms them while training for the standard number of epochs. However, the paper does not report what accuracy DAPO and GRESO achieve at the standard epoch count before extension — it only reports their extended (2×) results. If DAPO at 1× epochs already underperforms Lens, the 2× comparison is a stronger claim, but if DAPO at 1× is competitive with Lens and the 2× extension provides minimal additional gain, the efficiency advantage is less dramatic. The missing data point is DAPO and GRESO performance at the same number of training steps as Lens, which would enable a like-for-like efficiency comparison.

The filtering baselines also have a structural advantage not fully accounted for: DAPO and GRESO skip low-success prompts entirely, meaning they perform fewer forward and backward passes per epoch than Lens or GRPO (since they don't process the skipped prompts). This makes their per-epoch computational cost lower than Lens's, partially offsetting Lens's efficiency advantage. The paper's claim of "substantially fewer computational resources" would be stronger with FLOPs-matched comparisons rather than step-count comparisons.

Claim: "Low-success, challenging prompts contain valuable training signals, highlighting the critical role of pruning interference tokens in improving rollout efficiency"

This is the paper's central conceptual claim, and the experimental evidence supports it convincingly, but with scope limitations.

The strongest evidence is the training dynamics analysis (Figure 5). Lens does not just achieve higher final accuracy — it fundamentally reshapes the exploration landscape during training, converting Failure-category prompts into Mid and High categories. This demonstrates that the same prompts that GRPO (and especially DAPO/GRESO) treat as unproductive are, in fact, capable of generating successful rollouts when interference tokens are removed. The fact that GRPO_extended (double rollouts) only partially achieves this conversion reinforces the claim: the issue is not sampling budget but token-level interference.

The ablation against pruning strategies (Appendix E) provides causal validation. Random pruning and gradient-based pruning — which delete tokens without targeting interference — produce weaker results than Lens's interference-score-based pruning. This confirms that the specific tokens identified by the interference score are genuinely causal in the exploration failures, and that the purification mechanism is not simply benefiting from any form of prompt perturbation.

However, the claim's generality is limited by the paper's scope:

Single domain (math reasoning). All experiments are on mathematical reasoning tasks with binary verifiable rewards. It is unknown whether interference tokens play a similar role in code generation, scientific reasoning, multi-turn dialogue, or tasks without clean verifiable rewards. Mathematical prompts may be unusually susceptible to token-level interference because small changes in problem specification (numbers, units, constraint wording) can drastically alter the solution. In less structured domains, interference tokens might be harder to identify or less impactful.

Single RL algorithm (GRPO). Lens is implemented and tested only within the GRPO framework. The paper positions Lens as "plug-and-play," but this is not empirically demonstrated. Whether the interference score and purification mechanism would provide similar benefits in conjunction with PPO, REINFORCE, or other RL algorithms is untested. The dependence on GRPO's group-relative advantage computation — where zero-variance prompts are particularly damaging — means Lens might provide different (possibly smaller) benefits in algorithms that handle sparse rewards differently.

Modest model scales (≤8B parameters). All experiments use models up to 8B parameters. The capacity-dependent interference sensitivity finding (Section 4.3) suggests that larger models are less affected by interference tokens. It is possible that at 32B, 70B, or larger scales, the benefit of interference purification diminishes to the point of irrelevance. Conversely, the interference score might identify different types of tokens in larger models (more subtle distractors that smaller models cannot represent). Without experiments at larger scales, the claim's generality across model sizes is speculative.

No comparison with combined approaches. The paper compares Lens against scaling exploration and filtering independently, but does not test Lens in combination with either. For instance, running Lens with $m = 16$ (double rollouts) might yield further gains, or applying GRESO's pre-rollout filtering to decide which prompts to purify (rather than purifying all low-success prompts) might reduce overhead. The standalone Lens results are strong, but the paper does not establish an upper bound on what combined approaches could achieve.

Missing Experiments That Would Strengthen the Paper

1. Zero-shot interference score transfer. The interference score requires running $\pi_{\text{ref}}$ (the frozen base model) alongside $\pi_\theta$ (the current policy). As training progresses, the gap between these models widens. Does the interference score remain calibrated over long training horizons, or do tokens that were initially identified as interference become less relevant as the policy evolves? A study showing interference score stability (or drift) over training would build confidence in the method's reliability.

2. Are interference tokens consistent across seeds? The paper does not report whether the same tokens are identified as interference when training is repeated with different random seeds. If interference tokens are consistent, they might represent genuine prompt-design flaws that could be fixed at the dataset curation stage. If they vary with random initialization or training trajectory, they reflect model-specific overfitting rather than general prompt properties. This distinction matters for whether the finding implies dataset improvement or training improvement.

3. Direct measurement of robustness to interference tokens. The core claim is that CRPO teaches the model to ignore interference tokens. A direct test would be: take a trained Lens model and a trained GRPO model, intentionally insert synthetic interference tokens into test prompts, and measure the accuracy drop. If Lens-trained models are genuinely more robust, they should show smaller degradation. The paper's training dynamics (lower entropy, longer responses) are indirect evidence of robustness; a direct interference-injection experiment would provide causal evidence for the transfer claim.

4. Comparison with data augmentation baselines. An alternative to Lens's purification approach would be to augment the training data by generating multiple paraphrased versions of each prompt (with different wording, removing potential distractors) and training on the augmented set. This is simpler than dynamic interference detection and would not require per-step reference model inference. The paper does not compare against this baseline, which would help establish whether dynamic, model-aware interference detection is necessary or whether static prompt diversification achieves similar benefits.

5. Ablation of the guard condition. The paper reports that only ~20% of low-success prompts benefit from purification (Figure 2c), and the guard condition $g_i$ (Equation 2) is designed to prevent harmful transfers. An ablation showing Lens performance without the guard condition — i.e., always replacing failures with denoised successes when $\bar{a}_i < \tau$ — would quantify the importance of this selective activation. If performance degrades significantly without the guard, it validates the ~20% finding; if performance is similar, the guard is unnecessary complexity.

6. Per-benchmark interference analysis. The paper reports average interference score distributions and average accuracy improvements from pruning. A benchmark-level breakdown of what fraction of prompts benefit from purification, and what types of tokens are identified as interference on different benchmarks (e.g., are they numerical values? units? constraint words?), would provide insight into when and why Lens works. This is particularly relevant for the claim that Lens helps most on hard benchmarks — understanding what makes AMC23 prompts more interference-prone than MATH500 prompts would strengthen the mechanistic story.

6. Limitations and Trade-offs

Scope Limited to Mathematical Reasoning with Binary Verifiable Rewards

The assumption or constraint. All experiments in the paper are conducted on mathematical reasoning tasks — specifically seven benchmarks (MATH500, AMC23, AIME24/25, GaokaoEN-2023, Minerva, OlympiadBench) all involving closed-form problems with verifiable correct answers. The reward signal is binary: a solution is either correct (reward 1) or incorrect (reward 0). The paper explicitly acknowledges this in the Limitations section:

"The effectiveness of our approach has been validated primarily in tasks with binary rewards. Its applicability to more complex environments, such as those with multi-dimensional scoring, requires further investigation."

The consequence. Several aspects of Lens's design depend on properties of math reasoning with binary rewards that may not generalise. The interference score (Equation 1) identifies tokens causing complete exploration failure — but in tasks with graded or multi-dimensional rewards, "failure" is not a clean binary. A prompt that produces rollouts scoring 0.3, 0.4, and 0.5 on a continuous metric still has within-group variance for GRPO to exploit; the entire concept of "zero-reward prompts" does not transfer directly. The guard condition $g_i$ (Equation 2), which triggers purification only when denoised accuracy strictly exceeds original accuracy, relies on comparing binary success rates — with continuous rewards, defining "improvement" requires a threshold or distributional comparison that the paper does not explore. The sample reweighting scheme (Equation 4) uses the original success rate $\bar{a}_i$ as the scaling factor, but with non-binary rewards, the analogue of $\bar{a}_i$ is unclear (mean reward? median? fraction above some threshold?). More fundamentally, the entire motivation — that interference tokens cause zero-variance exploration collapse — is specific to settings where reward sparsity creates the collapse condition. In dense-reward environments (e.g., RLHF with per-token preference signals, code generation with partial-credit unit tests), the exploration dynamics may be qualitatively different, and the fraction of prompts that benefit from purification (the ~20% figure from Figure 2c) could shift substantially.

What evidence exists in the paper. None — this is an acknowledged scope limitation. All experiments, ablations, and analyses are confined to binary-reward mathematical reasoning. The paper does not include any experiments on code generation, multi-step planning, dialogue, or any domain with non-binary rewards. The claim that Lens is "plug-and-play" and applicable to other RLVR settings is asserted but not tested.

Mitigation status. The paper flags this as future work in the Limitations section but does not propose any modifications to Lens that would extend it to continuous or multi-dimensional reward settings. The current formulation is tightly coupled to binary rewards, and adapting it would require revisiting several core components (the interference score's relationship to reward structure, the guard condition's comparison logic, the reweighting scheme's dependence on $\bar{a}_i$ ).

No Evidence on Larger Model Scales (≤8B Parameters)

The assumption or constraint. All experiments use models with at most 8 billion parameters (Llama-3.2-3B, Qwen2.5-3B/7B, Qwen3-4B/8B). The paper explicitly acknowledges this:

"Due to limited computational resources, our experiments were conducted on models with up to 8B parameters. Evaluating the performance and scalability of our method on larger-scale models (e.g., 32B or 70B) remains an avenue for future research."

The consequence. This is not merely a coverage gap — the paper's own findings suggest that larger models may benefit less from Lens, potentially to the point of irrelevance at frontier scales. Section 4.3 demonstrates that interference vulnerability is capacity-dependent: the Qwen2.5-3B model benefits from higher pruning ratios (3–5%), while the Qwen2.5-7B model performs best with lower ratios (1–2%). The magnitude of Lens's gains also shrinks with scale: on MATH500, Lens improves Qwen3-4B-Base by +3.8 points but Qwen3-8B-Base by only +2.8 points (Table 1). If this trend continues, Lens might provide negligible benefit at 32B or 70B parameters — the larger models may have sufficient representational capacity to route around interference tokens without explicit purification, making the 1.27×–1.62× per-step overhead unjustified. Alternatively, larger models might exhibit different interference patterns (more subtle distractors that smaller models cannot represent), and the interference score might identify different types of tokens at scale. Either way, the paper provides no evidence to distinguish between these scenarios.

A secondary consequence: the paper's FLOPs-efficiency claims depend on the magnitude of Lens's accuracy gains, which scale inversely with model size in the available data. If gains continue to shrink at larger scales, the efficiency advantage over GRPO narrows, and the overhead cost may not be amortised. The 1.67× convergence speedup on MATH500 was measured on Qwen3-4B-Base (Figure 6); the speedup at 8B is not separately reported, and the smaller accuracy gain at 8B suggests the speedup may also be smaller.

What evidence exists in the paper. The capacity-dependent interference sensitivity analysis (Section 4.3, Figure 7) provides the primary evidence. Table 1 shows smaller absolute gains on Qwen3-8B-Base than Qwen3-4B-Base across most benchmarks (e.g., MATH500: +2.8 vs. +3.8; AIME24: +3.3 vs. +6.7; OlympiadBench: ~+1 vs. ~+3). The paper's own Limitations section acknowledges the scale constraint. There are no experiments at 32B, 70B, or any scale above 8B.

Mitigation status. The paper identifies this as future work in the Limitations section. No extrapolation analysis or scaling trend projection is provided to estimate Lens's behaviour at larger scales. A practitioner deploying Lens on a 70B model would be operating entirely outside the paper's empirical coverage.

Per-Step Computational Overhead Is Omitted from Headline Efficiency Claims

The assumption or constraint. The paper's headline claims — "over 1.6× speedup" (Section 1, Section 4.2) and "1.67× fewer gradient steps" (Figure 6) — report convergence speed measured in training steps, not wall-clock time. However, Lens incurs a substantial per-step overhead relative to GRPO: Appendix D, Table 5 reports a 1.27× to 1.62× increase in per-step time across model configurations. The paper states:

"Lens incurs a higher wall-clock cost per update to prioritize signal quality."

The consequence. A 1.67× reduction in gradient steps combined with a 1.62× per-step overhead yields approximately equal total training time to GRPO — not a 1.6× speedup as the headline suggests. On Qwen3-4B-Base (Table 5), GRPO takes 287 seconds per step while Lens takes 464 seconds (1.62×). If Lens converges in 150 steps and GRPO in 250 steps (the 1.67× ratio), Lens requires 150 × 464 = 69,600 seconds of training while GRPO requires 250 × 287 = 71,750 seconds — a marginal 3% reduction, not 40%. The "1.6× speedup" is a step-count metric, not a wall-clock metric.

This matters for practitioners evaluating whether to adopt Lens. The primary benefit is higher final accuracy (the 3.88% average gain) and more stable training dynamics, not dramatically faster training. The efficiency claim should be understood as "Lens achieves a given accuracy level in fewer gradient updates, with total wall-clock time approximately equal to or slightly better than GRPO, while achieving higher final performance." The paper's abstract and main text use language that could be read as claiming an end-to-end training time reduction, which the computational overhead data does not fully support.

A secondary consequence: the overhead composition matters for cost analysis. The 1.62× overhead on Qwen3-4B-Base includes (a) forward passes through the frozen reference model $\pi_{\text{ref}}$ on all prompt tokens for interference scoring, (b) additional rollout sampling from denoised prompts for the ~65% of prompts below the $\tau = 0.5$ threshold (Figure 5 suggests 25–40% of prompts are in the Failure/Low categories that would trigger purification), and (c) the CRPO objective computation with importance ratios and weighted advantages. On models where Lens provides smaller accuracy gains (e.g., larger scales), this overhead may not be justified if accuracy is the sole criterion.

What evidence exists in the paper. Appendix D, Table 5 explicitly reports the overhead: 1.62× for Qwen3-4B-Base with group size 8, 1.45× for Qwen3-8B-Base with group size 8, 1.27× for Qwen2.5-7B with group size 8. The paper acknowledges the overhead but does not integrate it into the headline speedup numbers. Figure 6's speedup metric is defined as "gradient steps to reach GRPO's peak accuracy" — step-count, not wall-time.

Mitigation status. The paper provides the overhead data transparently in Appendix D and argues that the overhead is "significantly more efficient than the brute-force approach of doubling sample size (G=16)." This is true — GRPO_extended costs ~2× per step for smaller gains — but it frames the comparison against the worst baseline rather than against the primary baseline (vanilla GRPO with group size 8). The paper could strengthen its claims by reporting total training time (or FLOPs) to reach a given accuracy threshold for Lens vs. GRPO, explicitly incorporating the per-step overhead. No such analysis is provided.

Single RL Algorithm (GRPO) — Plug-and-Play Generality Is Asserted but Untested

The assumption or constraint. Lens is designed, implemented, and evaluated exclusively within the GRPO (Group Relative Policy Optimization) framework. The paper positions Lens as a "plug-and-play rollout framework" (Section 2) and states in the Limitations section:

"While we demonstrated the efficacy of our method within the GRPO framework, we have not yet explored its integration with other GRPO-based variants. Specifically, our method could be combined with algorithms that optimize rollout frequency or reward functions."

The consequence. Several of Lens's design choices are tailored to GRPO's specific mechanics in ways that may not transfer cleanly to other RL algorithms:

Group-relative advantage computation. GRPO normalises advantages within each prompt group using the group mean and standard deviation. Lens's sample reweighting (Equation 4) and weighted advantage computation (Equation 6) are designed to modulate influence within this group-relative framework. PPO-style algorithms that compute advantages using a learned value function (rather than group statistics) would need a different mechanism for incorporating the prompt-adaptive weights. REINFORCE-based methods that use a baseline for variance reduction would similarly require adaptation.
Zero-variance collapse is GRPO-specific. The entire motivation for Lens — that zero-variance prompts produce vanishing gradients that halt learning — is a property of GRPO's group-relative advantage formula. In PPO with a value function baseline, a prompt with all-zero rewards still produces non-zero advantages (the value function predicts some expected reward, and the actual reward of 0 creates a negative advantage). In REINFORCE with a baseline, all-zero rollouts produce negative but non-zero policy gradient contributions. The severity of the exploration bottleneck that Lens addresses may be substantially lower in algorithms that do not rely on within-group variance for advantage estimation, reducing Lens's relative benefit.
The KL penalty configuration. Lens inherits GRPO's KL penalty structure (Equation 7, $\beta \mathbb{D}_{\text{KL}}$ ) with $\beta = 0.001$ . If integrated with an algorithm that uses a different KL regularisation strategy (e.g., PPO's adaptive KL controller, or REINFORCE without explicit KL penalty), the interference score — which measures deviation from $\pi_{\text{ref}}$ — might interact differently with the optimisation dynamics.

The paper's claim of plug-and-play generality is a reasonable design aspiration, but it is empirically unverified. A practitioner using PPO, REINFORCE, or RLOO as their base algorithm has no evidence that Lens would provide similar benefits, or how the integration should be adapted.

What evidence exists in the paper. Zero — there are no experiments with any RL algorithm other than GRPO and its direct variants (GRPO_extended, DAPO, GRESO, which are all GRPO-based). The sole integration Lens performs is with GRPO, and all hyperparameters (KL coefficient, group size, clipping parameter) are set within GRPO's standard configuration. The ablation of pruning strategies (Appendix E) compares Lens against alternative pruning approaches, not alternative RL algorithms.

Mitigation status. The Limitations section acknowledges this as a direction for future work and suggests that Lens "could be combined with algorithms that optimize rollout frequency or reward functions, potentially enhancing exploration capabilities and training stability." This is speculative; no concrete integration strategy or preliminary experiments are provided. The paper does not identify which components of Lens would need modification for non-GRPO algorithms, nor does it discuss the theoretical compatibility of the CRPO objective with value-function-based advantage estimation.

Difficulty Estimation Cost Is Not Accounted for in Deployment

The assumption or constraint. Lens determines which prompts require purification by computing the empirical success rate $\bar{a}_i$ from $m = 8$ rollouts sampled from the current policy. For prompts with $\bar{a}_i < \tau = 0.5$ , the system performs interference scoring (forward passes through $\pi_{\text{ref}}$ on all prompt tokens) and denoised-prompt resampling (another $m$ rollouts). This means that for low-success prompts, Lens effectively doubles the sampling budget — $m$ rollouts for success rate estimation plus $m$ rollouts from the denoised prompt — and adds the cost of $\pi_{\text{ref}}$ inference on every prompt token. The paper accounts for this in its per-step overhead analysis (Appendix D) during training, but there is a subtler issue: the overhead varies with the fraction of low-success prompts in the batch. In early training, when most prompts have low success rates, Lens may trigger purification on nearly every prompt, making the per-step overhead closer to 2× rather than the average 1.27–1.62× reported. As training progresses and more prompts achieve $\bar{a}_i \geq \tau = 0.5$ , purification activates less frequently, and the overhead decreases.

The consequence. The paper's per-step overhead figures (Table 5) are averages over an unspecified period of training — likely the entire 300-step trajectory — and do not reveal the variation across training phases. In early training (steps 1–100), when the model's success rate is lowest, Lens's overhead could be substantially higher than 1.62×, potentially exceeding the cost of GRPO_extended (2×) during the period when both methods are least efficient. A practitioner allocating a fixed compute budget might find that Lens consumes more of that budget in the early, low-success phase than the average overhead suggests, potentially delaying the point at which Lens's accuracy gains manifest relative to the budget consumed.

More broadly, the paper does not analyse the marginal benefit per unit of overhead compute. For prompts where purification succeeds ( $g_i = 1$ ), the additional rollouts from the denoised prompt directly improve the training signal. For prompts where purification fails ( $g_i = 0$ , approximately 80% of low-success prompts per Figure 2c), the extra rollouts are wasted — the system samples $m$ additional rollouts, computes interference scores, and then discards everything because the guard condition prevents integration. The overhead from these failed purification attempts is pure cost with no training benefit. The paper does not report what fraction of the total overhead is spent on successful vs. failed purifications, making it difficult to assess whether the mechanism is cost-effective per-attempt or only in aggregate.

What evidence exists in the paper. Table 5 (Appendix D) reports average step times for Lens vs. GRPO across models, but does not break down the overhead by training phase or by purification success/failure. Figure 5 (Section 4.1) shows the distribution of prompt success rates across training phases, from which one could infer that early-phase overhead is higher (more prompts in Failure/Low categories), but the paper does not perform this analysis. Figure 2c reports that only ~20% of prompts benefit from purification on the DeepMath dataset, implying ~80% of purification attempts are wasted, but this is not connected to the computational overhead analysis.

Mitigation status. The paper does not address the variability of overhead across training phases or the cost of failed purification attempts. A dynamic scheduling mechanism — e.g., estimating success rate with fewer than $m$ initial rollouts to reduce the diagnostic cost, or caching interference scores across steps to avoid recomputation — could reduce this overhead but is not explored. The paper's decision to use a fixed $\tau = 0.5$ threshold for all training phases (rather than, say, annealing the threshold as training progresses) means the overhead structure is static and potentially suboptimal.

Robustness to Interference Is Inferred but Not Directly Measured

The assumption or constraint. The paper's central claim is that CRPO, by training the policy to produce purified-success trajectories when conditioned on the original noisy prompt $x_i$ , "equips the model with the ability to ignore interference and perform robust reasoning under noisy inputs" (Section 2.2). This robustness is the mechanism by which Lens's benefits persist beyond the training phase into evaluation — the model is supposed to learn to route around interference tokens even when they are present at test time.

The consequence. The paper provides only indirect evidence for this robustness claim. The training dynamics analysis (Figure 8, Appendix F) shows that Lens-trained models have lower entropy (suggesting more decisive reasoning) and longer responses (suggesting more thorough chain-of-thought), but these are correlates of robustness, not measurements of it. The primary evaluation (Table 1) tests models on standard benchmark prompts — there is no experiment that systematically varies the presence or intensity of interference tokens at test time to measure whether Lens-trained models are genuinely more robust than GRPO-trained models.

Without a direct robustness measurement, alternative explanations for Lens's accuracy gains cannot be ruled out. For instance, Lens might simply be a more effective exploration mechanism that helps the policy discover better reasoning strategies during training, with no special robustness property — the improved test accuracy could come from better optimisation (more effective gradient signals), not from learned interference immunity. The distinction matters for generalisation: if Lens primarily improves optimisation, its benefits might be limited to the training distribution; if it genuinely teaches robustness, the benefits should transfer to prompts with novel interference patterns not seen during training.

A related concern: the guard condition $g_i$ ensures that only prompts where purification measurably improves accuracy contribute purified rollouts to training. But this means the model is selectively trained to be robust to specific interference tokens that were causally verified to cause failures during training. It is unclear whether this robustness generalises to new interference tokens (different distractors in unseen prompts) or is specific to the tokens identified during training. The paper provides no evidence either way.

What evidence exists in the paper. Indirect: lower training entropy (Figure 8b, 8e), earlier emergence of long-form reasoning (Figure 8c, 8f insets), and higher final accuracy on standard test prompts (Table 1). None of these directly measure robustness to interference tokens. The paper does not include an experiment that inserts known interference tokens into test prompts and measures accuracy degradation, nor an experiment that ablates interference tokens from test prompts and measures the accuracy gap between Lens and GRPO models on cleaned vs. noisy prompts. A direct test would be: (a) identify interference tokens on training prompts, (b) at test time, create two versions of each prompt — original and interference-purified — and (c) measure whether Lens-trained models show a smaller accuracy drop between purified and original than GRPO-trained models. No such experiment is reported.

Mitigation status. The paper does not acknowledge this as a limitation. The robustness claim is stated as a conclusion ("ultimately enhancing the robustness of LLM reasoning through self-exploration," Section 1) but is supported only by indirect evidence. The CRPO mechanism is described as producing robustness by construction (the model is trained on $x_i$ with purified targets, so it must learn to ignore interference), but this is a theoretical argument, not an empirical demonstration. A direct robustness measurement would substantially strengthen the paper's central claim and is feasible within the existing experimental framework.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper introduces a new diagnostic category into the RLVR research landscape: the idea that exploration failure can be localized to specific prompt tokens rather than being a holistic property of problem difficulty. This is not a paradigm shift in the sense of overturning foundational assumptions — the paper works within the established GRPO framework and does not challenge the core RLVR methodology. But it is more than an incremental refinement, because it changes what researchers should look at when RLVR training stalls.

Prior to this work, the default response to unstable or collapsed training in RLVR was to either increase the sampling budget (more rollouts) or filter out problematic prompts (DAPO, GRESO). Both strategies treat the prompt as an atomic, indivisible unit of difficulty. Lens introduces a third option at a finer granularity: decompose the prompt, identify which specific tokens are causing the collapse, and surgically intervene. This reframes the exploration bottleneck from a capability problem (the model cannot solve hard problems) to an attention problem (the model cannot ignore specific distractors). The implication is that many prompts currently discarded as "too difficult" may actually be salvageable with targeted input cleaning — a finding with direct practical consequences for data curation and training pipeline design.

The paper also provides a reconciliation mechanism for conflicting intuitions in the field. The filtering camp (Yu et al., 2025; Zheng et al., 2025a) has argued that zero-variance prompts are harmful and should be removed. The scaling camp (Xu et al., 2025; Yang et al., 2025b) has argued that more samples can eventually break through the variance barrier. Lens suggests both perspectives are partially correct but incomplete: zero-variance prompts are harmful in their raw form (validating the filtering intuition), but they contain valuable learning signals when interference is removed (validating the scaling intuition that these prompts are worth engaging with). The synthesis is that interference removal is more efficient than scaling and more capability-preserving than filtering, offering a resolution that neither prior approach captured.

The finding that interference vulnerability is capacity-dependent (Section 4.3, Figure 7) also opens a new dimension in the scaling conversation. Prior work on emergent abilities (Wei et al., 2022) and scaling laws (Kaplan et al., 2020; Hoffmann et al., 2022) has focused on what capabilities appear at different scales. Lens suggests a complementary axis: robustness to input-level distractors also improves with scale, and this robustness can be partially compensated for by targeted input cleaning at smaller scales. This implies that the effective capability gap between small and large models may be narrower than raw accuracy numbers suggest, once interference effects are accounted for — a finding with implications for model compression, distillation, and edge deployment.

The paper also redirects research attention from output-side credit assignment to input-side interference detection. The past several years of RLVR research have focused heavily on better reward shaping (Le et al., 2025), token-level credit assignment (Kazemnejad et al., 2024; Li et al., 2025), and advantage computation improvements. Lens demonstrates that meaningful gains can be achieved by looking upstream — at the prompt tokens the model processes before generating any output — rather than downstream at the response tokens. This suggests that future work on RLVR efficiency should allocate at least some attention budget to prompt-level diagnostics, a direction that has been largely absent from the literature.

However, the paper's impact is bounded by its limited empirical scope. All experiments are on math reasoning with binary rewards and models ≤8B parameters. The interference token phenomenon may be domain-specific (math prompts may be unusually susceptible to token-level distractors because small wording changes can alter problem semantics) or scale-specific (larger models may not exhibit the same vulnerability). The paper's framing as a general RLVR contribution is aspirational; its demonstrated contribution is to math reasoning with modest-scale models. The field will need replication across domains and scales before the conceptual shift fully registers.

Follow-Up Research This Work Enables

Direct measurement of learned interference robustness through controlled token insertion. The paper claims that CRPO teaches the model to ignore interference tokens, but provides only indirect evidence (lower training entropy, longer responses). A direct test would: (a) train Lens and GRPO models on the same math dataset, (b) identify the top- $k$ interference tokens for a set of held-out test prompts using the interference score from the GRPO-trained model (to avoid contamination), (c) create perturbed test prompts by inserting these known interference tokens into clean prompts where they did not originally appear, and (d) measure the accuracy drop for Lens-trained vs. GRPO-trained models. If Lens genuinely teaches robustness, the Lens model should show a smaller degradation when known distractors are injected. This experiment would provide causal evidence for the paper's central mechanism — or would reveal that Lens's gains come from better optimization rather than learned robustness, in which case the transfer claim would need revision. The experiment requires no new infrastructure; it uses the same models and benchmarks already in the paper, with an additional prompt-perturbation step.

Interference score stability analysis across training and random seeds. The paper treats interference tokens as a property of the prompt-policy interaction at a given training step, but does not examine whether the same tokens are consistently identified as interference across training time or across independent training runs. A stability analysis would: (a) record the set of tokens identified as interference (top- $k$ by interference score) at regular intervals (e.g., every 50 steps) during a Lens training run, (b) compute the Jaccard similarity between interference sets at consecutive checkpoints to measure temporal stability, and (c) repeat Lens training with 3–5 different random seeds and compare interference sets at equivalent training steps across seeds. If interference tokens are highly stable across time and seeds, they represent genuine prompt-design flaws — tokens that reliably mislead the model regardless of training trajectory — and could be fixed at the dataset curation stage, potentially eliminating the need for dynamic detection during training. If interference tokens vary substantially across seeds, they reflect model-specific overfitting that is sensitive to initialization, and dynamic detection remains necessary. This experiment would clarify whether Lens is treating a dataset problem or a training-dynamics problem, with different implications for how the method should be deployed.

Extension to code generation with partial-credit rewards. The paper's scope is limited to math reasoning with binary rewards. Code generation is a natural next domain because it has structured reward signals (unit test pass rates) that can be binarized or used as continuous scores, and because prompt sensitivity to specific tokens is a known issue — a single word change in a coding problem description can shift the required algorithm. A replication on HumanEval or MBPP would: (a) train a base model with GRPO and Lens on a code generation dataset (e.g., APPS, CodeContests), (b) define success as passing all unit tests (binary) or as the fraction of tests passed (continuous), (c) adapt the guard condition $g_i$ to compare denoised vs. original accuracy using either binary pass rates or mean test coverage, and (d) measure whether Lens reduces the fraction of prompts with zero test-passing rollouts and improves final pass@1. This experiment tests both the domain generality of the interference phenomenon and the adaptability of Lens to non-binary reward structures. A negative result — Lens provides no benefit on code generation — would suggest interference tokens are specific to the formal reasoning structure of math problems, narrowing the method's claimed applicability. A positive result would substantially broaden the contribution.

Integration with non-GRPO RL algorithms (PPO, REINFORCE with baseline). The paper positions Lens as plug-and-play but tests only GRPO. The key adaptation question is whether the interference scoring and CRPO transfer mechanism provide benefits when the base RL algorithm does not suffer from GRPO's specific zero-variance collapse pathology. A PPO integration would: (a) implement Lens with a learned value function for advantage estimation instead of group-relative normalization, (b) adapt the sample reweighting (Equation 4) to modulate the PPO objective rather than group statistics — since PPO does not normalize advantages within prompt groups, the weighting would need to operate directly on the policy gradient contribution per rollout, (c) compare Lens-PPO against vanilla PPO on the same math benchmarks, and (d) measure whether the interference purification still provides gains when the base algorithm does not produce zero-gradient updates on uniform-reward prompts. A negative result — Lens provides no benefit over PPO — would suggest that Lens is primarily a workaround for GRPO's specific limitations rather than a general exploration improvement, limiting its significance. A positive result would validate the plug-and-play claim and extend Lens's applicability to a much wider range of RL training pipelines.

Combined Lens + filtering for resource-adaptive training. The paper shows Lens outperforming both scaling and filtering baselines, but does not test whether combining Lens with filtering would yield further gains or reduce overhead. A combined approach would: (a) use GRESO's pre-rollout prediction to identify which prompts are likely to be zero-variance before sampling, (b) apply Lens's interference purification only to the subset of prompts that are predicted to be zero-variance but above some difficulty threshold (indicating they are salvageable rather than genuinely impossible), (c) filter out (skip) prompts predicted to be zero-variance and below the difficulty threshold (genuinely beyond capability), and (d) process the remaining prompts normally. This would reduce Lens's overhead by avoiding purification on prompts where it is unlikely to succeed (the ~80% of low-success prompts where purification does not help, per Figure 2c), while still extracting signals from the ~20% where it does. The experiment would measure whether the combined approach achieves Lens-level accuracy with lower computational overhead. A positive result would provide a practical deployment recipe; a negative result would suggest the filtering and purification mechanisms interact adversarially in ways the current paper does not anticipate.

Scaling trend projection for interference benefit at larger model sizes. The paper's capacity-dependent interference finding (Section 4.3) shows that Lens's benefit shrinks from Qwen2.5-3B to Qwen2.5-7B, and from Qwen3-4B to Qwen3-8B. A scaling study at larger sizes would: (a) train Lens and GRPO on a consistent model family at 0.5B, 1.5B, 3B, 7B, and (if resources permit) 14B or 32B parameters, (b) measure the absolute accuracy gain from Lens at each scale on MATH500 and AMC23, (c) fit a scaling trend to estimate the model size at which Lens's benefit falls below, say, 1 percentage point (the point of practical irrelevance), and (d) analyze whether the interference score distribution changes qualitatively at larger scales (do large models have fewer high-interference tokens, or different types of high-interference tokens?). This experiment addresses the most significant limitation of the current paper — the unknown behavior at frontier model scales — and would determine whether Lens is a technique for the modest-scale regime or a general contribution to RLVR training regardless of model size. The paper's own data already provides two points on this curve (3B→7B and 4B→8B), and extending to more points and larger scales would be a natural follow-up that uses the same methodology.

Practical Applications and Downstream Use Cases

Cost-efficient RL fine-tuning for small-to-medium model deployments (3B–8B parameters). Organizations training reasoning models in the 3B–8B parameter range — a common scale for on-device or cost-constrained deployments — can adopt Lens as a drop-in replacement for standard GRPO with approximately equal total training time and a 2–8 percentage point accuracy improvement on math reasoning tasks (Table 1). The specific benefit depends on the difficulty profile of the target application: on competition-level benchmarks (AMC23, AIME24), Lens provides gains of 6–8 points on 4B models, while on standard benchmarks (MATH500) the gain is 3–4 points. The computational overhead (1.27–1.62× per step) is partially or fully amortized by faster convergence in terms of gradient steps (1.67× on MATH500), making the total training cost comparable to GRPO. For a team currently using GRPO with $m = 8$ rollouts, switching to Lens requires no additional hardware, no changes to the reward function or model architecture, and only the addition of the interference scoring and CRPO modules to the training loop. The primary practical consideration is that the benefit shrinks with model scale within the tested range — a 7B or 8B model sees smaller gains (2–3 points) than a 3B or 4B model (4–8 points) — so the cost-benefit calculation should account for model size.

Improving sample efficiency in self-improvement and iterative RL pipelines. In settings where an LLM is used to generate its own training data through repeated rounds of sampling and fine-tuning (e.g., ReST, STaR, or online RLVR loops), the quality of sampled rollouts determines the efficiency of each iteration. Lens's demonstrated ability to reduce the fraction of prompts with zero successful rollouts by 5–10 percentage points across training phases (Figure 5, Section 4.1) directly improves the yield of each sampling round. In a self-improvement pipeline where the model generates solutions, filters for correctness, and fine-tunes on the correct ones, Lens would increase the fraction of prompts that generate at least one correct solution — from approximately 60% to 75% in early training and from 78% to 85% in late training (approximate ranges from Figure 5). This means fewer wasted generation cycles and more training data per unit of inference compute. The effect is most pronounced in early iterations when the model's success rate is low and purification triggers most frequently. For a pipeline running 5–10 iterations of RL fine-tuning, the cumulative efficiency gain from Lens could substantially reduce the total inference budget required to reach a target accuracy.

Targeted prompt debugging for training dataset quality improvement. The interference score provides a per-token diagnostic that can be used outside the RL training loop to audit and improve training datasets. After training a model with Lens, practitioners can aggregate interference scores across the training corpus to identify which tokens most frequently appear in the top- $k$ interference set across prompts. If certain tokens — specific phrasings, numerical formatting conventions, ambiguous constraint words — consistently score as interference across many prompts, this indicates dataset-level issues where prompt design is systematically misleading models. The dataset can then be revised to rephrase or clarify these problematic tokens before the next training cycle. This is a lightweight auditing process: it requires storing the top- $k$ interference tokens per prompt during one training run (a small metadata cost), then aggregating with simple frequency counting. The benefit is improved training data quality that benefits all future training runs, not just Lens-based ones. The paper's finding that only ~20% of low-success prompts benefit from purification (Figure 2c) also implies that for ~80% of prompts, the interference tokens identified during one training run are not causal for failure — those prompts' failures are due to genuine difficulty. This provides a natural triage: focus dataset-revision efforts on the 20% of prompts where purification helped, as those are the ones where token-level prompt design issues are causally linked to model failure. This targeted approach is more efficient than blanket prompt rewriting and is grounded in empirical evidence from the model's own behavior.