ArXiv: 2504.20571

🎯 Pitch

A single carefully chosen math problem can replace thousands in RL-based reasoning training: 1-shot RLVR matches full-dataset performance, boosting Qwen2.5-Math-1.5B from 36% to 73.6% on MATH500. The improvement comes from the policy gradient signal itself—not dataset diversity—and test accuracy keeps climbing long after the model has memorized the one training example.

1. Executive Summary

This paper empirically demonstrates that reinforcement learning with verifiable reward using only a single training example—termed 1-shot RLVR—can dramatically improve LLM mathematical reasoning, applying GRPO to Qwen2.5-Math-1.5B and elevating MATH500 accuracy from 36.0% to 73.6% while matching the performance of full-dataset training with 1.2k examples. The work identifies several named phenomena driving this result, including post-saturation generalization (test accuracy continues improving long after training accuracy on the single example saturates near 100%), cross-category generalization (training on one math category—e.g., Geometry—improves performance across unrelated categories like Algebra and Number Theory), and increased frequency of self-reflection behaviors in downstream tasks as training progresses. Through ablation, the paper establishes that policy gradient loss—not weight decay or KL regularization—is the primary driver of improvement, distinguishing post-saturation generalization from grokking, while entropy loss further enhances exploration and yields nontrivial gains even when applied alone without any outcome reward.

2. Context and Motivation

The Core Puzzle: How Much Data Does RLVR Actually Need?

The paper addresses a remarkably straightforward yet provocative question: to what extent can we reduce the training dataset for reinforcement learning with verifiable rewards while maintaining comparable performance to full-dataset training? This question sits at the intersection of data efficiency and reasoning capability acquisition in LLMs, and the answer the paper provides—that a single example can suffice—challenges fundamental assumptions about how reinforcement learning signals propagate through language models during post-training.

The dominant narrative in the RLVR literature, shaped by landmark results from DeepSeek-R1, Kimi-1.5, and OpenAI's o1 series, is that scaling reinforcement learning on large mathematical reasoning datasets produces substantial gains in model capability. These works typically employ thousands of carefully curated problem-answer pairs. The implicit assumption is that diversity in the training data is necessary: the model must encounter many different problem types, solution strategies, and reasoning patterns to learn generalizable reasoning skills. This paper tests the limiting case of that assumption and finds it is largely wrong—at least for base models that already possess latent reasoning capabilities.

Why This Matters: Practical and Theoretical Significance

Understanding the data requirements of RLVR carries both immediate practical consequences and deeper scientific implications.

On the practical side, data curation is expensive. The DeepScaleR-Preview-Dataset, which the paper uses as its primary data pool, contains roughly 40,000 unique math problems extracted from competition sources (AIME, AMC, Omni-MATH) and processed through a pipeline involving Gemini-1.5-Pro-002 for answer extraction, RAG-based deduplication, and SymPy-based filterability checks. Constructing such datasets requires substantial engineering effort, API costs, and domain expertise. If comparable performance can be achieved with dramatically fewer examples—even a single one—the cost and complexity of building RLVR training sets collapses. This democratizes access to RL-based reasoning improvement for teams without the resources to curate large mathematical corpora.

Beyond cost, the finding forces a re-examination of what RLVR actually does to the model. If a single example provides essentially no new factual knowledge or problem-solving strategies (the model already knows how to solve it, as Section 3.2.1 demonstrates—the base model correctly computes all key steps for $\pi_1$ with high probability), then the improvement must arise from some other mechanism. The paper's evidence points toward RLVR acting as a capability activator: it teaches the model to reliably deploy reasoning skills it already possesses, to structure outputs in a verifier-friendly format, and to explore diverse solution paths rather than collapsing to shallow heuristics. This reframes RLVR from a knowledge-acquisition process to a behavioral-shaping one, with profound implications for how we think about post-training.

On the theoretical side, the paper engages with a emerging debate about whether base models already harbor significant reasoning capabilities that simply need to be "unlocked." Recent work—notably Dr. GRPO and the observation that carefully prompting base models without templates yields strong performance—suggests that much of what RLVR appears to teach is actually latent in pretrained weights. The 1-shot RLVR result provides perhaps the most striking evidence for this hypothesis: if a single algebra problem with a physics backdrop ($\pi_1$, shown in Table 2) can trigger 37.6 percentage point gains on MATH500, the base model clearly wasn't learning algebra from scratch. It was learning to express algebra it already understood.

Where Prior Approaches Fall Short

The landscape of data-efficient LLM post-training is well-studied for supervised fine-tuning but largely unexplored for RLVR. The paper identifies several gaps:

Data selection for SFT does not transfer to RLVR. Prior work on instruction tuning has developed sophisticated data selection methods—LLM-based quality scoring, gradient-based influence estimation, embedding-space nearest-neighbor retrieval, and active preference learning for RLHF. These methods exploit properties of supervised learning (e.g., gradient alignment between training and target distributions) that do not straightforwardly apply to reinforcement learning, where the training signal is a sparse binary reward mediated through advantage-weighted policy gradients. The paper notes that no principled data selection framework exists specifically for RLVR.

LIMR stopped too early. The most directly relevant prior work is LIMR, which introduced the Learning Impact Measurement (LIM) metric to score training examples and demonstrated that pruning the training set by a factor of six (from ~8.5k to ~1.4k examples) could maintain performance. However, LIMR did not ask the limiting question: how low can you go? By stopping at 1.4k examples—a substantial reduction but still three orders of magnitude above one—LIMR left open whether its metric identified a fundamental data efficiency floor or merely an incremental improvement. This paper shows the floor is far lower than anyone suspected, and that even LIMR's aggressive pruning may have been conservative.

The full-dataset assumption is untested. Works like DeepScaleR, DAPO, and VAPO focus on algorithmic improvements to RLVR (better credit assignment, more stable GRPO variants, value function optimization) while treating the dataset as a fixed, large-scale input. The possibility that algorithmic refinements and data curation might be substitutes rather than complements—that better algorithms could make large datasets unnecessary—was not systematically explored. This paper's demonstration that 1-shot RLVR with standard GRPO matches full-dataset performance suggests that many algorithmic improvements may be implicitly solving a data diversity problem that, in fact, barely exists for base models with sufficient pretraining.

Format correction vs. genuine reasoning improvement is confounded. A critical confound in evaluating RLVR is that some of the observed performance gains stem from format correction rather than improved reasoning. Qwen2.5-Math models, when prompted with their standard template, often fail to produce parseable final answers—the \boxed{} format that verifiers expect. Dr. GRPO explicitly documented that changing the template alone can significantly affect performance. Prior work reporting RLVR gains on Qwen models did not always disentangle format-fixing effects from reasoning improvements. This paper addresses this directly through format-reward baselines (assigning reward 1 if the answer is parseable, regardless of correctness), establishing that 1-shot RLVR with outcome reward provides nontrivial improvements beyond format correction (e.g., +7.4% on MATH500 and +5.8% average across six benchmarks beyond the format-reward baseline, per Table 14 and the discussion in Appendix C.2.3).

How This Paper Positions Itself

The paper explicitly casts itself not as proposing a new method—it does not claim that 1-shot RLVR is better than full-dataset RLVR in absolute terms—but as presenting a provocative empirical observation that demands explanation. This is an unusual framing: rather than arguing "our method beats baselines," the paper argues "our observation challenges your assumptions about what baselines need."

Several framing choices are notable:

The paper treats data as an independent variable to be minimized, not optimized. Unlike prior data selection work that asks "which data is best?", this paper asks "how little data can we use before things break?" This zeroes in on the necessity of data diversity rather than its sufficiency, a subtly different question that generates different experimental design (sweeping from 1 to thousands of examples rather than pruning within a large set).

The paper connects to the "base models already reason" narrative. By showing that $\pi_1$ is a problem the base model already largely solves (Section 3.2.1: the model correctly computes all key steps before the cube root calculation, with 57.8% of outputs producing "12.7" or "12.70"), the paper positions RLVR as activating latent capability rather than teaching new skills. This aligns with Dr. GRPO's template-analysis findings but extends them to the extreme of single-example training.

The paper distinguishes its phenomenon from grokking. Grokking—the sudden emergence of generalization long after training accuracy saturates on small algorithmic datasets—shares surface similarities with post-saturation generalization. The paper preemptively addresses this comparison through loss ablation (Section 4.1, Table 5), showing that removing weight decay (a known critical component for grokking) does not impair 1-shot RLVR performance, while removing policy gradient loss does. This establishes post-saturation generalization as a distinct phenomenon, likely driven by entropy-loss-encouraged exploration combined with policy-gradient-based verification of correctness on the single learned example.

The paper is explicit about its limitations as a practical method. It acknowledges (Appendix D.1) that "1-shot RLVR doesn't save (and maybe requires more) RL computation"—the number of RL update steps remains comparable to full-dataset training, and the single example must be duplicated to fill the training batch. The contribution is conceptual, not computational: understanding why a single example works illuminates what RLVR is doing in a way that training on thousands of examples obscures.

The paper positions data quality, not quantity, as the critical variable. The wide variance in 1-shot RLVR effectiveness across different individual examples (Table 3: $\pi_1$ yields 74.0% MATH500 while $\pi_{1207}$, which has a wrong label, yields only 54.0%) demonstrates that example choice matters enormously. This reframes the field's challenge: rather than collecting more data, the priority should be understanding what makes a single example effective at triggering latent reasoning capabilities. The historical variance score ($v_i$ in Equation 1) is proposed as one ranking criterion, but the paper is careful to note it is "not necessarily optimal" and that many examples with moderate or low variance still produce substantial gains—further evidence that the phenomenon is robust and not an artifact of a particular selection method.

3. Technical Approach

3.1 Reader Orientation

This paper is primarily an empirical investigation wrapped around a single training pipeline: applying GRPO-based reinforcement learning with verifiable rewards to a base LLM using only one or a few training examples, with the goal of understanding the minimal data requirements for activating mathematical reasoning capabilities. The core system is a standard RLVR training loop where the fundamental knob being turned is the size of the training dataset—reduced all the way to a single problem-answer pair—while keeping the RL algorithm, model architecture, and reward structure fixed, enabling the authors to isolate the role of data quantity and quality in RLVR-based reasoning improvement.

3.2 Big-Picture Architecture (Diagram in Words)

The system has five major components arranged in a standard RLVR pipeline:

  1. Base LLM (Qwen2.5-Math-1.5B by default, with variants at 7B, Llama-3.2-3B-Instruct, and DeepSeek-R1-Distill-Qwen-1.5B) — the pretrained language model that generates candidate mathematical solutions and is updated via policy gradient during training. It serves as both the policy being optimized and the generator of rollouts.

  2. Training Dataset (1 to 1209 examples, drawn from the DeepScaleR-Preview-Dataset or the MATH training set) — a collection of mathematical problem-answer pairs where each problem includes a ground truth answer that can be verified programmatically. When using 1-shot RLVR, the single example is duplicated to fill the training batch (128 copies to satisfy the dataloader's drop_last=True behavior in the verl framework).

  3. Verifiable Outcome Reward Function — a binary rule-based scorer that compares the model's final parsed answer against the ground truth label, outputting 1 for an exact match and 0 otherwise. No process reward model, no learned scorer, no partial credit. The reward is computed purely through answer matching using the Qwen2.5-Math evaluation pipeline's answer extraction and comparison logic.

  4. GRPO Loss Computation Engine (via the verl framework) — the module that takes model-generated responses, computes group-normalized advantages from the binary rewards, and constructs the full GRPO loss comprising policy gradient loss, KL divergence loss, and entropy loss. This module performs 8 gradient updates per rollout step (batch size 128, 8 responses per prompt).

  5. Data Selection Mechanism (the historical variance score) — a pre-processing step that ranks training examples by the variance of their training accuracy over multiple epochs of full-dataset RLVR, enabling the selection of individual examples (e.g., $\pi_1$, $\pi_{13}$) for 1-shot experiments. This step is executed once on the base model before any 1-shot RLVR training begins.

Information flows as follows: a training prompt (the single selected example, duplicated 128 times) enters the system → the old policy model $\pi_{\theta_{\text{old}}}$ samples 8 responses per prompt (1024 total responses per rollout step) at temperature 0.6 → the verifiable reward function parses each response and computes a binary 0/1 reward → the GRPO loss module normalizes rewards into group advantages and computes the policy gradient, KL divergence, and entropy loss components → the policy model parameters are updated via gradient descent (8 minibatch updates per rollout) → the updated policy becomes the new $\pi_{\theta_{\text{old}}}$ for the next rollout step → evaluation checkpoints are saved every 20 steps and tested on the 6-benchmark mathematical reasoning suite.

3.3 Roadmap for the Deep Dive

  • First, the GRPO loss function (Section B.1 of the paper), since it is the computational engine driving all RLVR experiments. Understanding how group-normalized advantages, KL divergence, and entropy loss combine is essential for interpreting the ablation results in Section 4.1.
  • Second, the verifiable reward mechanism, including how answers are extracted and matched, and the critical distinction between outcome reward and format reward—a distinction that underpins the paper's claim that 1-shot RLVR provides gains beyond format correction.
  • Third, the historical variance score data selection method (Equation 1, Equation 2), which determines which examples become $\pi_1$, $\pi_{13}$, etc. Understanding this mechanism is necessary to interpret the claim that 1-shot RLVR works for many examples, not just the highest-variance ones.
  • Fourth, the training/evaluation protocol—batch construction for 1-shot vs. full-dataset training, rollout sampling, checkpoint selection, and the two-benchmark evaluation procedure—since these design choices affect what the reported numbers mean and what conclusions they support.
  • Fifth, the specific example $\pi_1$ as a case study, including what makes it "not-so-difficult" for the base model and why this property may be connected to its effectiveness as a single training example.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily an empirical investigation paper whose core contribution is the observation that RLVR with verifiable outcome reward can produce dramatic reasoning improvements using only one training example, combined with analysis that decomposes the sources of this improvement and distinguishes the observed phenomena from superficially similar ones like grokking.

The GRPO Loss Function

The Group Relative Policy Optimization (GRPO) loss, as originally introduced by Shao et al. (2024) and adopted in DeepSeek-R1, replaces the value function critic of PPO with a group-based advantage normalization. The version used here follows the verl implementation, which includes three components: a clipped policy gradient loss, a KL divergence penalty from a reference model, and an entropy bonus to encourage exploration.

The overall GRPO objective for a single question $q$ sampled from the prompt distribution $P(Q)$ is:

LGRPO(θ)=Eq∼P(Q){oi}i=1G∼πθold(O∣q)[LPG-GRPO′(⋅,θ)+βLKL′(⋅,θ,θref)+αLEntropy′(⋅,θ)]\mathcal{L}_{\text{GRPO}}(\theta) = \mathbb{E}_{\substack{q \sim P(Q) \\ \{o_i\}_{i=1}^G \sim \pi_{\theta_{\text{old}}}(O|q)}} \left[ \mathcal{L}'_{\text{PG-GRPO}}(\cdot, \theta) + \beta \mathcal{L}'_{\text{KL}}(\cdot, \theta, \theta_{\text{ref}}) + \alpha \mathcal{L}'_{\text{Entropy}}(\cdot, \theta) \right]

where $G$ is the group size (8 responses per prompt), $\{o_i\}_{i=1}^G$ is the set of $G$ responses sampled from the old policy $\pi_{\theta_{\text{old}}}$, $\beta = 0.001$ is the KL penalty coefficient, $\alpha = -0.001$ is the entropy loss coefficient (negative because it is implemented as a bonus—lower entropy is penalized, higher entropy is rewarded), and $\theta_{\text{ref}}$ is the frozen reference model (the initial base model checkpoint).

What it computes: for each prompt, the model samples $G$ responses, scores them with the verifiable reward function, normalizes the rewards into group-relative advantages, and then updates the policy to increase the probability of higher-than-average responses while penalizing lower-than-average ones, all while staying close to the reference model's output distribution and being encouraged to maintain high per-token entropy.

Why this form: the group-based advantage normalization eliminates the need for a separately trained value function (which is complex to train and adds a second model to manage), while the clipping in the policy gradient (see below) prevents destructively large updates from single high-variance advantage estimates. The KL penalty prevents the model from diverging into degenerate outputs (a known failure mode in RL for language models), and the entropy bonus counteracts the natural tendency of policy gradient to collapse the output distribution toward a single high-reward response, which would kill exploration—particularly critical in 1-shot RLVR where the model sees only one problem and could otherwise converge to memorizing a single solution string.

Policy Gradient Loss (Clipped)

Inside the GRPO objective, the policy gradient term is a clipped surrogate objective following the PPO formulation:

LPG-GRPO′(q,{oi}i=1G,θ)=−1G∑i=1G[min⁡(πθ(oi∣q)πθold(oi∣q)Ai,clip(πθ(oi∣q)πθold(oi∣q),1−ε,1+ε)Ai)]\mathcal{L}'_{\text{PG-GRPO}}(q, \{o_i\}_{i=1}^G, \theta) = -\frac{1}{G} \sum_{i=1}^G \left[ \min\left( \frac{\pi_\theta(o_i|q)}{\pi_{\theta_{\text{old}}}(o_i|q)} A_i, \text{clip}\left( \frac{\pi_\theta(o_i|q)}{\pi_{\theta_{\text{old}}}(o_i|q)}, 1 - \varepsilon, 1 + \varepsilon \right) A_i \right) \right]

where $\frac{\pi_\theta(o_i|q)}{\pi_{\theta_{\text{old}}}(o_i|q)}$ is the importance sampling ratio (the probability of response $o_i$ under the current policy divided by its probability under the old policy), $A_i$ is the group-normalized advantage for response $i$ (defined below), and $\varepsilon$ is the clipping threshold (set to the standard PPO value of 0.2 in verl; not explicitly stated in the paper but standard).

What it computes: for each response in the group, compute the ratio of current-to-old probabilities (how much more or less likely the model now is to produce this response), multiply by the advantage (how much better or worse this response is than the group average), and then apply the PPO-style clipping to prevent the ratio from moving outside $[1-\varepsilon, 1+\varepsilon]$. The negative sign and the $\frac{1}{G}$ averaging make this a standard empirical risk minimization form: the loss is low when high-advantage responses become more probable and low-advantage responses become less probable.

Why this form: the clipping mechanism is the defining innovation of PPO over earlier policy gradient methods like REINFORCE. Without clipping, if a response receives a large positive advantage, the policy gradient would push its probability ratio toward infinity, potentially causing catastrophic forgetting of other capabilities. The $\min$ operation with the clipped objective ensures that the loss does not reward the policy for increasing the probability ratio beyond $1+\varepsilon$ (for positive advantages) or decreasing it below $1-\varepsilon$ (for negative advantages), creating a trust region that stabilizes training. This is particularly important for 1-shot RLVR where the model repeatedly sees the same prompt—without clipping, the model could rapidly overfit to producing a single high-reward response format.

Group-Normalized Advantage

The advantage $A_i$ for response $i$ is computed by standardizing the raw rewards within the group:

Ai=ri−mean({r1,r2,…,rG})std({r1,r2,…,rG})A_i = \frac{r_i - \text{mean}(\{r_1, r_2, \ldots, r_G\})}{\text{std}(\{r_1, r_2, \ldots, r_G\})}

where $r_i \in \{0, 1\}$ is the binary outcome reward for response $i$ (1 if the parsed answer matches the ground truth, 0 otherwise), $G = 8$ is the group size, and $\text{mean}$ and $\text{std}$ are the sample mean and standard deviation of the $G$ rewards within this group.

What it computes: subtract the group's mean reward from each individual reward and divide by the group's standard deviation. This transforms the raw binary rewards into zero-mean, unit-variance advantages: responses that are correct when most responses in the group are incorrect receive large positive advantages, responses that are incorrect when most are correct receive large negative advantages, and responses in homogeneous groups (all correct or all incorrect) receive zero advantage.

Why this form: the zero-mean property is critical—and the paper explicitly highlights it as part of RLVR's "anti-overfitting property" (Section 5). When training accuracy on the single example approaches 100% (all 8 responses in a group are correct), the standard deviation becomes zero or near-zero, and all advantages become zero or near-zero, meaning the policy gradient provides no update signal. This is exactly why post-saturation generalization is interesting: the model continues improving on test tasks even when the training advantage signal vanishes. The group normalization also eliminates the need for a learned baseline (value function), reducing the complexity of the RL system. A drawback is that the advantage magnitude depends on within-group variance—when the model sometimes succeeds and sometimes fails on the training example, advantages are large, but when it always succeeds, the signal disappears. The entropy loss (Section 3.4.4) mitigates this by ensuring some failures persist even after saturation.

KL Divergence Loss

The KL divergence term penalizes deviation from the reference model's output distribution:

LKL′(q,{oi}i=1G,θ,θref)=DKL(πθ∥πθref)=πθref(oi∣q)πθ(oi∣q)−log⁡πθref(oi∣q)πθ(oi∣q)−1\mathcal{L}'_{\text{KL}}(q, \{o_i\}_{i=1}^G, \theta, \theta_{\text{ref}}) = D_{\text{KL}}(\pi_\theta \| \pi_{\theta_{\text{ref}}}) = \frac{\pi_{\theta_{\text{ref}}}(o_i|q)}{\pi_\theta(o_i|q)} - \log \frac{\pi_{\theta_{\text{ref}}}(o_i|q)}{\pi_\theta(o_i|q)} - 1

where $\pi_\theta$ is the current policy, $\pi_{\theta_{\text{ref}}}$ is the frozen reference model, and the expression is the approximation form of KL divergence popularized by Schulman (2020) and widely used in prior RLVR work (DeepSeekMath, DeepSeek-R1).

What it computes: for each response $o_i$, compute the ratio $r = \pi_{\theta_{\text{ref}}}(o_i|q) / \pi_\theta(o_i|q)$ and then compute $r - \log r - 1$. This function is non-negative and equals zero when $r = 1$ (i.e., when the current and reference policy assign the same probability to $o_i$), and grows as the policies diverge. The result is averaged over tokens and responses and multiplied by $\beta = 0.001$ before being added to the total loss.

Why this form: the $r - \log r - 1$ form is a second-order Taylor approximation to the true KL divergence that is unbiased and lower-variance than the exact $\log(\pi_\theta/\pi_{\theta_{\text{ref}}})$ estimator when estimated from finite samples. It prevents the policy from drifting too far from the base model's language distribution, which is essential for maintaining general language quality and preventing reward hacking (e.g., the model outputting non-human-readable strings that happen to parse as correct answers). The coefficient $\beta = 0.001$ is relatively small, consistent with the finding in Table 5 (Row 4) that adding KL divergence has "no significant impact on model performance"—the policy gradient term dominates. This is different from RLHF settings where KL penalties are often larger to prevent the model from diverging into reward-maximizing but unhelpful outputs.

Entropy Loss

The entropy loss component encourages the model to maintain high per-token entropy in its output distribution:

LEntropy′(q,{oi}i=1G,θ)=∑b,sMb,s⋅Hb,s(X)∑b,sMb,s\mathcal{L}'_{\text{Entropy}}(q, \{o_i\}_{i=1}^G, \theta) = \frac{\sum_{b,s} M_{b,s} \cdot H_{b,s}(X)}{\sum_{b,s} M_{b,s}}

where $b$ indexes the batch dimension, $s$ indexes the sequence position, $M_{b,s} \in \{0, 1\}$ is the response mask (1 for tokens that are part of the generated response, 0 for padding or prompt tokens), and $H_{b,s}(X)$ is the per-token entropy computed from the model's output logits $X$:

Hb,s(X)=log⁡(∑veXb,s,v)−∑vpb,s,v⋅Xb,s,vH_{b,s}(X) = \log\left(\sum_v e^{X_{b,s,v}}\right) - \sum_v p_{b,s,v} \cdot X_{b,s,v}

where $v$ indexes over the vocabulary, $X_{b,s,v}$ is the unnormalized logit for token $v$ at position $(b,s)$, and $p_{b,s,v} = \text{softmax}(X_{b,s})_v$ is the predicted probability of token $v$ at that position.

What it computes: for each token position in the generated response (not the prompt), compute the Shannon entropy of the model's predicted token distribution: $H = -\sum_v p(v) \log p(v)$. This ranges from 0 (the model is certain about the next token—probability 1 on a single token) to $\log |V|$ (the model is maximally uncertain—uniform distribution over the vocabulary). The mask-weighted average over all generated tokens gives a single scalar representing the average per-token uncertainty. Since $\alpha = -0.001$ is negative, minimizing the total loss maximizes this entropy (the negative coefficient becomes a bonus for high entropy).

Why this form: Shannon entropy is the standard measure of uncertainty for a categorical distribution. The per-token averaging (masking out padding) ensures the metric reflects the model's uncertainty on tokens it actually generates. The negative coefficient transforms entropy from a regularizer that penalizes uncertainty into a bonus that rewards exploration. This design choice is crucial for 1-shot RLVR because, as the paper argues, when training accuracy approaches 100% and advantages vanish, the entropy loss is the remaining force driving the model to explore diverse solution strategies. Without it, the model would stop improving after saturation (as shown in Figure 5: without entropy loss, performance plateaus after step ~150; with entropy loss, it continues improving). However, the coefficient must be carefully tuned—Table 5 Row 6 shows that increasing $|\alpha|$ to 0.003 makes training "more unstable." The paper's ablation in Row 10 and Section 4.2 further shows that entropy loss alone (with no outcome reward) yields nontrivial performance gains (MATH500: 36.0% → 63.4% in 20 steps for Qwen2.5-Math-1.5B), which the authors attribute to format correction—entropy encourages exploration that happens to include properly formatted \boxed{} answers.

The Verifiable Outcome Reward

The reward function is the simplest possible: a binary 0/1 signal indicating whether the model's final answer matches the ground truth answer for the given problem. This is distinct from process-based reward models (PRMs) that score intermediate reasoning steps, and distinct from learned outcome reward models (ORMs) that predict correctness from the full response. The paper explicitly states (Section 2): "Since we focus on mathematical problems, the reward is defined as binary (0-1), where a reward of 1 is granted only when the outcome of the model's response correctly matches the ground truth."

How matching works: the Qwen2.5-Math evaluation pipeline (which is used for both reward computation during training and final evaluation) extracts the content inside the last \boxed{} in the model's response and compares it against the ground truth answer using a rule-based grading function. This grading function (inherited from the MATH benchmark's original release) handles various formatting differences: equivalent fractions, different radical simplifications, unit variations, and other common mathematical equivalences. If no \boxed{} is present, the answer is treated as unparseable and receives reward 0.

Format reward baseline: the paper introduces a format-only reward variant (Appendix C.2.3) that assigns reward 1 if any answer can be parsed from the response (regardless of correctness) and 0 otherwise. This serves as a control to isolate format-correction effects from genuine reasoning improvements. The format reward is computed identically to the outcome reward except that the match check against ground truth is skipped—the mere presence of a parseable \boxed{} content suffices. When used for training, this teaches the model to structure outputs in a verifier-friendly way without providing any correctness signal.

Why binary and outcome-based: the paper justifies this choice implicitly through its relationship to prior work. Binary outcome rewards are the standard in RLVR for mathematical reasoning (DeepSeek-R1, Kimi-1.5, DAPO) because they are cheap to compute (no learned model needed), immune to reward hacking of the "convince the reward model" variety (there is no reward model to convince), and well-defined (an answer either matches or it doesn't). The tradeoff is sparsity—the model receives no signal about where in the reasoning chain it went wrong—but the GRPO group-normalized advantage partially mitigates this by comparing responses within a group and reinforcing ones that happen to succeed. The paper does not experiment with process rewards or dense rewards, keeping the focus on data quantity rather than reward design.

Historical Variance Score for Data Selection

The paper proposes a simple ranking criterion for selecting which individual examples to use in 1-shot RLVR experiments. The process has two stages:

Stage 1: Full-dataset RLVR for data profiling. The base model is first trained for $E = 500$ steps (the paper says "E epochs" in Equation 1 notation but "500 steps" in Section 3.1) on the full 1,209-example DSR-sub dataset using standard GRPO. For each example $i \in \{1, \ldots, N\}$ where $N = 1209$, the average training accuracy during each epoch is recorded, producing a vector $L_i = [s_{i,1}, \ldots, s_{i,E}]$ where $s_{i,e} \in [0, 1]$ is the fraction of responses to example $i$ that were correct during epoch $e$.

Stage 2: Variance-based ranking. The historical variance score $v_i$ for example $i$ is computed as:

vi:=var(si,1,…,si,E)v_i := \text{var}(s_{i,1}, \ldots, s_{i,E})

where $\text{var}$ is the sample variance over the $E$ training accuracy values.

What it computes: for each training example, take the list of per-epoch training accuracies, compute their variance (a measure of how much the accuracy fluctuated across training), and use this variance as a quality score. High-variance examples are those where the model sometimes succeeded and sometimes failed during full-dataset RLVR—they were "at the edge" of the model's capability and produced varying reward signals. Low-variance examples are those where the model consistently succeeded (variance near 0, always correct) or consistently failed (variance near 0, always incorrect).

The examples are then sorted by descending variance to produce a permutation $\pi$:

π(j)=arg⁡sortj{vl:l∈[N]}\pi(j) = \arg\text{sort}_j\{v_l : l \in [N]\}

where $\pi(1)$ is the index of the highest-variance example, $\pi(2)$ the second-highest, and so on. The shorthand $\pi_j = \pi(j)$ denotes the example with the $j$-th largest variance. Thus, $\pi_1$ is the highest-variance example, $\pi_{13}$ the thirteenth-highest, and $\pi_{1209}$ the lowest-variance.

Why this form: the justification, while not rigorously defended with ablations comparing variance to alternatives, is grounded in the observation from prior work (Razin et al., 2025, cited in the paper) that "the variance of the reward signal is critical for RL training." High-variance examples produce advantages with larger magnitude (since advantage normalization divides by standard deviation, but higher raw variance means the rewards are not all identical, so the numerator $r_i - \text{mean}$ is non-zero more often), providing stronger and more diverse policy gradient signals. An example that the model always gets right (low variance) provides no negative signal (the model never sees what a wrong answer looks like for that problem), while an example it always gets wrong (also low variance) provides no positive signal. Examples with intermediate difficulty—where the model sometimes succeeds and sometimes fails—provide both positive and negative signals, which may be more informative for shaping the policy.

The paper explicitly notes that this criterion is "not necessarily optimal for selecting single examples for 1-shot RLVR," and Table 3 confirms this: examples with moderate or low variance (e.g., $\pi_{605}$ at rank 605, $\pi_{1209}$ at rank 1209) still produce substantial improvements when used as single training examples. The variance score is thus a heuristic that happens to surface strong examples at the top ranks (both $\pi_1$ and $\pi_{13}$ perform well) but does not fully capture what makes an example effective.

Importantly, the $\pi_i$ correspondence is fixed once computed on Qwen2.5-Math-1.5B and reused for all other models in the paper (Qwen2.5-Math-7B, Llama-3.2-3B-Instruct, DeepSeek-R1-Distill-Qwen-1.5B). This means the same examples are used across models even though their variance profiles might differ on other base models—a deliberate choice to maintain consistency in the experimental protocol and avoid circularity.

Training Protocol and Batch Construction for 1-Shot RLVR

The standard RLVR training loop using the verl framework expects a dataset with at least as many examples as the training batch size, because the dataloader is configured with drop_last=True (Section 3.1, footnote 3). The training batch size is 128, and the dataloader drops any incomplete final batch. This creates a problem for 1-shot RLVR: a single example cannot fill a batch of 128 without modification.

Solution: duplication to fill the batch. The selected single example is duplicated until the stored dataset contains 128 copies of the identical prompt. These 128 copies are treated as independent entries by the dataloader, meaning that in each training step, the model receives 128 instances of the same problem. For each instance, the model generates $G = 8$ responses (the group size in GRPO), producing $128 \times 8 = 1024$ responses per rollout step, all to the same prompt. The 8 responses within each group are used to compute group-normalized advantages, and the policy gradient is averaged over the 128 groups.

Implications for the RL signal: when training accuracy is below 100%, each group of 8 responses contains a mixture of correct and incorrect answers (since the model samples stochastically at temperature 0.6), producing non-zero advantages. As training accuracy approaches 100%, groups become increasingly homogeneous (all correct), advantages approach zero, and the policy gradient signal vanishes. The entropy loss then becomes the primary remaining training signal, which is why Section 4.1 finds it critical for post-saturation generalization. The duplication strategy also means the model sees the identical prompt millions of times over the course of training (1024 responses per step × 2000 steps = over 2 million exposures to the single example), yet—as Section 3.2.2 shows—overfitting in the sense of producing degenerate outputs only occurs after approximately 1.4k steps for $\pi_1$ (and 1.8k for $\pi_{13}$), and even then, test-set reasoning outputs remain coherent.

Training hyperparameters (Section 3.1, Appendix B.4):

  • Rollout temperature: 0.6 (controls the stochasticity of response generation; higher temperature means more diverse outputs and more variance in the reward signal)
  • Learning rate: 1e-6 (for DeepSeek-R1-Distill-Qwen-1.5B; the paper does not explicitly state the learning rate for Qwen2.5-Math models, but following the verl defaults, it is likely 1e-6 as well)
  • Weight decay: 0.01 (standard L2 regularization coefficient)
  • KL coefficient $\beta$: 0.001
  • Entropy coefficient $\alpha$: -0.001 (negative, as discussed)
  • PPO clipping $\varepsilon$: 0.2 (standard; not explicitly stated but inherited from verl/DeepSeek-R1)
  • Maximum prompt length: 1024 tokens
  • Maximum response length: 3072 tokens (for Qwen2.5-Math-1.5B/7B, whose context window is 4096; DeepSeek-R1-Distill-Qwen-1.5B uses 8192 following the DeepScaleR stage-1 setup)
  • Gradient updates per rollout: 8 (since minibatch size is the same as batch size, 128, and there are 1024 total responses organized into 8-minibatch groups)
  • Checkpoint frequency: every 20 steps
  • Total training steps: 2000 for Qwen2.5-Math-1.5B, 1000 for Qwen2.5-Math-7B and Llama-3.2-3B-Instruct, 1200 for DeepSeek-R1-Distill-Qwen-1.5B (training is stopped early if performance significantly degrades)
  • GPU usage: 8 A100 GPUs per experiment

Evaluation Protocol and Benchmark Suite

The paper evaluates on six mathematical reasoning benchmarks using the official Qwen2.5-Math evaluation pipeline (Appendix B.3, B.5):

Benchmarks with temperature 0 (greedy decoding):

  • MATH500 (500 problems): a curated subset of the MATH test set, developed by OpenAI for efficient evaluation. Reports $\text{pass@1}$ (accuracy with a single greedy sample).
  • Minerva Math (272 problems): undergraduate-level STEM problems from MIT OpenCourseWare covering solid-state chemistry, information theory, differential equations, and special relativity. Reports $\text{pass@1}$.
  • OlympiadBench (675 problems): Olympiad-level math competition problems, open-ended text-only math subset in English. Reports $\text{pass@1}$.

Benchmarks with temperature 0.6 and avg@8 (for stability due to small test sets):

  • AIME 2024 (30 problems): American Invitational Mathematics Examination 2024 (both I and II). The 30-question set is repeated 8 times, the model generates 8 responses per repetition with temperature 0.6, and $\text{avg@8}$ reports the average pass@1 across all 240 evaluations. This repeated evaluation design addresses the high variance that would arise from evaluating on only 30 questions.
  • AIME 2025 (30 problems): same design as AIME 2024 but for the 2025 examination.
  • AMC 2023 (40 problems): from AMC 12A and 12B, covering secondary-school mathematics (arithmetic, algebra, combinatorics, geometry, number theory, probability). Uses the same repeated-evaluation protocol as AIME.

Non-mathematical reasoning tasks (for cross-domain generalization analysis, Table 1):

  • ARC-Easy (2,376 test questions): science reasoning questions solvable through simpler approaches, multiple-choice, English-language.
  • ARC-Challenge (1,172 test questions): the subset of ARC questions that information retrieval and word co-occurrence methods fail on, designed to require genuine reasoning.

Evaluation-specific settings:

  • Prompt template: Qwen-based models use the qwen25-math-cot template; Llama and DeepSeek-distilled models use their original chat templates.
  • Maximum generated tokens: 3072 (matching the training response length constraint).
  • Seed: 0 for reproducibility.
  • Top-p: 1.0 (no nucleus sampling truncation; all probability mass is considered during greedy/temperature sampling).
  • DeepSeek-R1-Distill-Qwen-1.5B special settings: following the evaluation protocol from DeepScaleR and DeepSeek-R1, this model uses temperature 0.6, top-p 0.95, and reports $\text{avg@16}$ for MATH500/MinervaMath/OlympiadBench and $\text{avg@64}$ for AIME24/AIME25/AMC23 (higher $k$ values because this model generates longer chain-of-thought and benefits from more sampling). Evaluation is performed at both 8k and 32k context lengths, with the 32k results reported as the primary numbers in Table 4.

Checkpoint selection: by default, the paper reports results from the checkpoint achieving the best average performance across all 6 mathematical benchmarks. However, in Sections 3.2.3 (the per-category MATH500 analysis in Table 3) and 4.1 (the ablation study in Table 5), results are reported using the best checkpoint for each benchmark independently—i.e., the best MATH500 checkpoint may differ from the best AIME2024 checkpoint. This choice is explicitly noted ("we report the best model performance on each benchmark separately" in Appendix B.5) and allows estimating the upper bound of what each training run can achieve, at the cost of not representing a single deployable model. The paper includes both reporting styles (Tables 8/10 for best-average checkpoints, Tables 9/11 for best-per-benchmark) to provide a complete picture.

Case Study: $\pi_1$ — Anatomy of an Effective Single Example

The paper provides a detailed analysis of $\pi_1$, the highest-variance example from the historical variance ranking, to understand what makes a single example effective for 1-shot RLVR. The example (Table 2) is:

"The pressure P exerted by wind on a sail varies jointly as the area A of the sail and the cube of the wind's velocity V. When the velocity is 8 miles per hour, the pressure on a sail of 2 square feet is 4 pounds. Find the wind velocity when the pressure on 4 square feet of sail is 32 pounds. Let's think step by step and output the final answer within \boxed{}."

Ground truth: 12.8 (in the DSR-sub dataset).

Key structural properties of this problem:

  1. It requires multi-step reasoning. The solution involves (a) establishing the relationship $P = kAV^3$ from the "varies jointly" description, (b) solving for the constant $k$ using the first scenario: $4 = k \cdot 2 \cdot 8^3 \implies k = 4 / (2 \cdot 512) = 1/256$, (c) applying the constant to the second scenario: $32 = (1/256) \cdot 4 \cdot V^3 \implies V^3 = 2048$, (d) computing the cube root: $V = \sqrt[3]{2048} \approx 12.8$. This multi-step structure means the model's output contains a chain of reasoning with intermediate computations—precisely the kind of chain-of-thought that RLVR is thought to incentivize.

  2. The base model already largely solves it. From 128 samples of the untrained base model (Section 3.2.1, Figure 3), 57.8% of outputs produce "12.7" or "12.70," 6.3% produce "12.8," and 6.3% produce "13." The model correctly performs the algebraic manipulation to set up $V^3 = 2048$ with high probability; the primary source of variance is the final cube root computation, where the model samples diverse approximations (12.7, 12.8, 12.88, 13, etc.). This means the base model's pass@1 is non-trivially above zero (the model can solve it) but well below 100% (it often fails on the final numerical step), creating the intermediate-difficulty profile that produces high training accuracy variance—and thus strong policy gradient signals—during RLVR.

  3. The ground truth label is slightly inaccurate but not catastrophically so. The mathematically precise answer is $12.6992...$, which rounds to 12.7, not 12.8. The DSR-sub label is 12.8, likely due to dataset construction artifacts (the answer was extracted by Gemini-1.5-Pro-002, which may have rounded). The paper tests this explicitly in Table 5 (Row 11): 1-shot RLVR with the correct label "12.7" performs essentially identically to RLVR with the original label "12.8" (MATH500: 73.4% vs. 74.8%; AIME24: 17.9% vs. 17.5%). This demonstrates that 1-shot RLVR is robust to small label errors—the model overfits to whatever label is provided, but the reasoning process learned generalizes correctly regardless. The contrast with a wildly incorrect label ("4", Row 12) is instructive: the model can still overfit (training accuracy saturates), but test performance drops to 57.0% MATH500, far below the 63.4% achieved by entropy-loss-only training (Row 10). This suggests that overfitting to a wrong-but-learnable label is worse than receiving no correctness signal at all, because the policy gradient actively reinforces an incorrect answer pattern.

  4. The problem has a clear "correct format" that generalizes. The final answer is a single number that fits naturally inside \boxed{}. The CoT structure (step-by-step algebraic manipulation followed by numerical computation) is a template that transfers to other algebra problems. This aligns with the cross-category generalization finding in Table 3: $\pi_1$ improves performance not just on Algebra problems (its own category) but on Geometry, Number Theory, and Prealgebra as well. The CoT pattern—set up the equation, solve for constants, compute the final value—is broadly applicable.

  5. Complexity matters beyond the "tricky step." Appendix C.2.5 (Table 18) tests a simplified variant $\pi'_1$ whose prompt is simply "Calculate $\sqrt[3]{2048}$"—only the final computational step of $\pi_1$, stripped of the physics context and the multi-step setup. This simplified example performs substantially worse: MATH500 drops from 72.8% (full $\pi_1$) to 65.4%, and the gain over the format-reward baseline (65.0%) is only 0.4 percentage points on MATH500 and 1.3% on average across benchmarks. This demonstrates that it is not merely the "difficult step" that makes $\pi_1$ effective; rather, the full multi-step reasoning chain—including the problem interpretation ("varies jointly"), the constant-solving step, and the algebraic manipulation—provides a richer CoT template that better incentivizes generalizable reasoning patterns. The simplified version essentially teaches the model to compute a specific cube root, which provides minimal transferable benefit.

The paper's interpretation: the fact that the base model already "almost solves" $\pi_1$ is a feature, not a bug. The RLVR process on this example does not teach the model algebra—it already knows algebra. Instead, it (a) teaches the model to reliably produce the correct final answer format (the \boxed{} structure) through policy-gradient reinforcement of properly-formatted outputs, (b) encourages the model to explore and commit to coherent multi-step reasoning chains (since sloppy or truncated chains are less likely to produce the correct answer and thus receive lower advantage), and (c) through the entropy bonus, pushes the model to try diverse solution strategies for the same problem, some of which generalize to other problem types. The example functions as a "calibration target": the model learns that certain reasoning patterns (setting up proportional relationships, solving for constants, computing with exponents and roots) tend to produce answers that match the ground truth, and this meta-knowledge transfers even though the specific content (pressure on sails) does not.

Distinction Between 1-Shot RLVR and Format-Reward RLVR

A crucial technical distinction in the paper is between outcome-reward RLVR (the primary 1-shot experiments) and format-reward RLVR (used as a baseline to isolate format-correction effects). Both use the identical GRPO training pipeline and the identical dataset (the single example, duplicated), but they differ in the reward function:

  • Outcome reward: $r = 1$ if and only if the parsed final answer matches the ground truth label. The model receives 0 reward for incorrect answers even if they are perfectly formatted.
  • Format reward: $r = 1$ if and only if any answer can be parsed from the output (i.e., the response contains a \boxed{} with some content), regardless of correctness. The model receives 1 reward for producing a parseable \boxed{} even if the answer inside is wrong, and 0 only if the parser cannot extract any answer.

What format-reward RLVR teaches: the model learns to consistently produce outputs that end with \boxed{<something>}. It does not learn to produce correct answers. The improvement from format-reward RLVR (Table 14: MATH500 36.0% → 65.0%, average 17.6% → 28.7%) represents the pure format-correction gain: the base model already knows some correct answers but fails to put them in a parseable format, and format reward solves this. The 7.4% gap on MATH500 between outcome-reward 1-shot RLVR (72.8%) and format-reward 1-shot RLVR (65.4%) represents the additional gain attributable to correctness-guided reasoning improvement beyond mere format fixing.

Why this distinction matters: it resolves a potential criticism that 1-shot RLVR is "just" teaching the model to format its answers correctly, with the single example serving only as a format template. The nontrivial gap between outcome and format reward—and the fact that this gap varies by benchmark (larger on AIME24, smaller on OlympiadBench)—demonstrates that the outcome reward provides a correctness signal that shapes the model's reasoning process, not just its output structure. This is further supported by the increased frequency of self-reflection words (Figure 4) and the observation that test accuracy continues improving after \boxed{} ratios saturate (Figure 15): the model is learning something about how to reason, not just how to format.

4. Key Insights and Innovations

Innovation 1: Reframing RLVR as Capability Activation, Not Knowledge Acquisition

The most fundamental conceptual move in this paper is the implicit reframing of what RLVR actually does to a language model. Prior to this work, the dominant mental model—inherited implicitly from the RLHF and instruction-tuning literature—treated RLVR primarily as a knowledge acquisition process: the model encounters diverse training problems, receives reward signals indicating which solution strategies work, and gradually learns generalizable reasoning patterns by aggregating information across many examples. Under this view, data diversity is essential because each example contributes a small piece of the overall reasoning competency; reduce the dataset and you should expect proportional degradation in performance. LIMR (Li et al., 2025), the most directly relevant prior work, operated entirely within this framework—their goal was to identify which subset of examples contained the most "learning impact," implicitly assuming that learning impact accumulates across examples.

This paper shatters that framing by demonstrating that a single example produces gains indistinguishable from a 1,209-example dataset. If data diversity were the mechanism, this should be impossible. The finding forces a fundamentally different interpretation: for base models that already possess substantial latent reasoning capabilities from pretraining (which recent work like Dr. GRPO and the pass@k analyses have argued is the case for Qwen2.5-Math models), RLVR functions as a capability activator rather than a knowledge teacher. What the model needs is not exposure to many problem types—it already understands algebra, geometry, and number theory at a deep level from its pretraining corpus—but rather a mechanism that surfaces this latent capability, shapes it into a reliably deployable form, and teaches the model that producing coherent, well-formatted, multi-step reasoning chains is the path to reward.

The evidence for this reframing is multilayered. First, the paper explicitly shows that $\pi_1$ is a problem the base model already largely solves (Section 3.2.1): 57.8% of base model outputs produce the numerically correct answer. The model knows how to set up proportional relationships, solve for constants, and manipulate equations—it just sometimes fails on the final cube root computation and sometimes fails to format the answer properly. RLVR on this single example does not teach it algebra; it teaches it that the kind of output that gets rewarded is one where all steps are coherently chained and the final answer is clearly boxed. Second, the cross-category generalization results (Table 3) demonstrate that training on a single algebra problem improves performance on geometry, number theory, and counting problems—categories with entirely different mathematical content but similar structural requirements (multi-step reasoning, intermediate computation, boxed final answers). If RLVR were teaching algebra-specific skills, cross-category transfer would be minimal. The fact that it is substantial confirms that what is being learned is something more abstract: a behavioral template for mathematical reasoning that generalizes across content. Third, the comparison with format-reward RLVR (Appendix C.2.3, Table 14) provides the cleanest decomposition: format reward alone (which teaches only output structure, not correctness) recovers a large fraction of the gain (MATH500: 36.0% → 65.0%), while outcome reward adds an additional correctness-guided component (→ 72.8%) that pushes beyond pure formatting. This two-phase picture—first teach the model to produce parseable reasoning chains, then teach it that chains ending in correct answers are better—is a far more nuanced account of what RLVR does than the monolithic "train on diverse data" narrative.

This reframing is not merely semantic. It has direct practical implications: if RLVR activates rather than teaches, then the quality of the activation signal matters far more than the quantity or diversity of the training data. This explains why different single examples produce dramatically different outcomes in Table 3 ($\pi_1$: 74.0% MATH500; $\pi_{11}$: 64.0%; $\pi_{1208}$: 45.0%)—not because some examples "teach more" than others, but because some examples provide a better template for the kind of reasoning behavior the model should adopt. It also explains why the simplified $\pi'_1$ (just computing $\sqrt[3]{2048}$) dramatically underperforms the full $\pi_1$ (Table 18): a one-step computation provides a poor template for multi-step chain-of-thought reasoning, even though it targets the specific step the base model finds hardest. The model needs to see an exemplar of the reasoning process, not just practice on a difficult sub-step.

This insight connects directly to the growing body of evidence that base models are far more capable than their naive evaluation scores suggest. Dr. GRPO showed that template choice dramatically affects Qwen2.5-Math performance; recent pass@k analyses show that base models often outperform RLVR-trained models at high k; and work on "cognitive behaviors" (Gandhi et al., 2025) has identified self-reflection patterns in base model outputs that RLVR amplifies. This paper provides the limiting case: if a single example can trigger these latent capabilities, then most of what appears to be "learning" during RLVR is actually "uncovering" capabilities the model already acquired during pretraining.

Innovation 2: Post-Saturation Generalization as a Distinct Phenomenon from Grokking

The second conceptual contribution is the identification and partial characterization of post-saturation generalization—the phenomenon where test accuracy continues improving long after training accuracy on the single example has saturated near 100%—and its careful differentiation from the superficially similar but mechanistically distinct phenomenon of grokking.

Grokking, as described by Power et al. (2022) and extensively studied since, refers to the sudden emergence of generalization on a held-out validation set after a prolonged period of near-perfect training accuracy and near-chance validation accuracy on small algorithmic datasets (e.g., modular arithmetic). The key mechanistic insight from the grokking literature is that it depends heavily on regularization—particularly weight decay—which slowly drives the network toward a simpler, more generalizable solution even after the training loss has reached zero. Without weight decay, networks may memorize the training data and never generalize. The phenomenon has been studied primarily in small transformer models on synthetic tasks, and its relevance to large-scale language model training has been unclear.

This paper's post-saturation generalization shares the surface pattern—training accuracy saturates, test accuracy keeps improving—but the underlying mechanism appears to be fundamentally different. The critical evidence is the ablation study in Table 5 (Section 4.1). Row 2 shows that policy gradient loss alone (without weight decay, KL divergence, or entropy loss) produces results close to the full GRPO loss (71.8% vs. 74.8% MATH500). Row 3 adds weight decay—the component essential to grokking—and finds "no significant impact on model performance" (71.4%). Row 8 removes policy gradient and keeps only weight decay and KL divergence, producing essentially no improvement (39.0% MATH500, barely above the base model's 36.0%). Row 7 removes weight decay and KL divergence, keeping only policy gradient and entropy loss, and matches full GRPO performance (75.6% MATH500). The pattern is unambiguous: policy gradient loss, not weight decay, drives post-saturation generalization in 1-shot RLVR. Whatever is happening here is not grokking, or at least not the version of grokking that the literature has characterized.

The paper proposes a different mechanism (though it is careful to note this is speculative, Section D.3): as training accuracy approaches 100%, the GRPO advantage signal vanishes for correct answers (since all responses in a group are correct, advantages are zero), but the entropy loss continues to encourage diverse outputs. This diversity occasionally produces errors—the model explores a weird reasoning path and gets the answer wrong—which creates non-zero advantages (since now some responses are correct and some are not, the group variance is non-zero). The policy gradient then corrects these errors, reinforcing the model's ability to recover from exploration-induced mistakes. This creates a self-sustaining cycle: entropy loss encourages exploration, exploration causes occasional failures, failures create policy gradient signals, policy gradient corrects the failures, and the model learns more robust reasoning strategies that transfer to test tasks. Figure 5 provides supporting evidence: without entropy loss, test performance plateaus shortly after training saturation; with entropy loss, it continues improving.

This is a conceptually novel contribution because it identifies a phenomenon that is both empirically similar to grokking (generalization after training saturation) and mechanistically distinct from it (driven by policy gradient rather than weight decay), suggesting that post-saturation generalization in RLVR is a new phenomenon that requires its own theoretical framework. The connection to entropy loss further suggests that exploration—not regularization—is the key enabler, which is a distinctly reinforcement-learning-flavored mechanism rather than a supervised-learning one. If this characterization holds up under further study, it would position post-saturation generalization as the RL analog of grokking, with its own set of necessary conditions (entropy bonus or other exploration incentives, verifiable reward signal, intermediate-difficulty training examples) and its own theoretical underpinnings (exploration-correct cycles rather than weight-decay-induced simplicity bias).

Innovation 3: The Functional Decomposition of RLVR Gains into Format, Entropy, and Correctness Components

A third conceptual contribution is the paper's systematic decomposition of where RLVR's performance gains actually come from, separating contributions that prior work had conflated. This is less a single dramatic finding and more a diagnostic framework that reinterprets and clarifies a confused literature.

Prior to this work, papers reporting RLVR results on mathematical reasoning generally reported a single number: the accuracy improvement from base model to RLVR-trained model on a benchmark suite. The implicit assumption was that this improvement reflected improved reasoning capability—the model learned to think better. Dr. GRPO challenged this by showing that changing the Qwen2.5-Math evaluation template could dramatically affect base model performance, suggesting that some fraction of reported RLVR gains might be attributable to template-mismatch artifacts rather than genuine reasoning improvement. But Dr. GRPO's analysis was qualitative: it showed the problem existed without quantifying it or providing a systematic way to measure and control for it.

This paper provides that quantification by introducing two clean baselines—format-reward RLVR and entropy-loss-only training—that decompose the total RLVR gain into three separable components:

  1. Format correction gain: the improvement from teaching the model to produce parseable answers (the \boxed{} format that verifiers require). Measured by the gap between the base model and format-reward RLVR. On Qwen2.5-Math-1.5B with DSR-sub, this accounts for MATH500 improvement from 36.0% to 65.0% (Table 14)—that is, roughly 76% of the total 39.6 percentage point gain comes from format fixing alone.

  2. Entropy-driven exploration gain: the improvement from simply encouraging the model to produce more diverse, less deterministic outputs, even without any correctness signal. Measured by entropy-loss-only training (Table 13: MATH500 36.0% → 63.4% in 20 steps). This gain partly overlaps with format correction (more diverse outputs are more likely to include a correctly formatted answer by chance) but the paper does not fully disentangle their interaction—an acknowledged limitation.

  3. Correctness-guided reasoning gain: the additional improvement beyond format correction that comes from reinforcing correct answers specifically. Measured by the gap between format-reward and outcome-reward RLVR. On Qwen2.5-Math-1.5B with $\pi_1$, this gap is 7.4% on MATH500 and 5.8% on the 6-benchmark average (Table 14). This is the component that genuinely reflects improved reasoning—the model is not just learning to put things in boxes, but learning what to put in the box.

This decomposition matters for several reasons. First, it provides a calibration tool for future RLVR research: any paper claiming that a new RLVR method "improves reasoning" should report format-reward baselines to establish that the improvement is not merely format correction. Second, it explains why different Qwen2.5-Math variants show dramatically different RLVR gains: Qwen2.5-Math-1.5B, which the paper notes has a ~40% repetition rate at the end of its outputs causing frequent parse failures, benefits enormously from format correction (~76% of total gain), while instruction-tuned variants like Qwen2.5-Math-1.5B-Instruct, which already produce properly formatted outputs, show much smaller absolute gains from RLVR because the format-correction component is already saturated (Table 10: base 73.4% MATH500 → RLVR 75.6%, only +2.2%). The differences in "RLVR effectiveness" across models that prior work might have attributed to model-specific properties are revealed to be largely differences in how broken the base model's output format was.

Third, this decomposition clarifies the paper's own central finding. When the paper claims 1-shot RLVR "matches" full-dataset RLVR, it is not claiming that the correctness-guided reasoning gain is identical in both cases—it is claiming that the total gain (format + entropy + correctness) is comparable. The deep implication is that most of the reasoning improvement obtainable through RLVR on these models comes from components that do not require data diversity: format correction needs any data with proper formatting, entropy-driven exploration needs basically no data at all, and correctness-guided reasoning needs only enough examples to provide a reliable reward signal for the behavioral template the model should adopt. Data diversity plays a surprisingly minor role—at least for the Qwen2.5-Math family, at least on these benchmarks, and at least for the GRPO algorithm with these hyperparameters.

Innovation 4: Data Quality Over Data Quantity as the Central Challenge for RLVR

The paper's fourth conceptual contribution is less a finding and more a reframing of the field's priorities. By demonstrating that a single example can match the performance of thousands, the paper argues—implicitly through its results and explicitly in its discussion (Section D.4)—that the critical bottleneck for RLVR is not the quantity of training data but its quality, where "quality" means the example's effectiveness at activating and shaping the model's latent reasoning capabilities.

This reframing is significant because the current trajectory of RLVR research heavily emphasizes data scale. The DeepScaleR-Preview-Dataset contains ~40,000 problems; the MATH training set has 7,500; DAPO and VAPO use similarly large corpora. The implicit assumption is that more data → more diverse training signals → better generalization. This paper's Table 3 provides a stark counterpoint: $\pi_1$ (a single algebra problem) achieves 74.0% MATH500, while $\pi_{1208}$ (an extremely difficult combinatorics problem that the base model cannot solve) achieves only 45.0%. This is a 29 percentage point gap attributable purely to example choice—larger than the gap between 1-shot and full-dataset RLVR in most comparisons. The choice of which single example to use matters more than the choice of whether to use one example or a thousand.

The paper's historical variance score (Equation 1, Equation 2) is a first attempt at operationalizing "data quality" for RLVR, but the paper is notably self-aware about its limitations. Table 3 shows that $\pi_{605}$ (rank 605 by variance) achieves 71.8% MATH500—within 2.2 percentage points of $\pi_1$ (rank 1)—while $\pi_{11}$ (rank 11) achieves only 64.0%. The variance score surfaces good examples at the top ranks but does not perfectly order them. The paper explicitly states the criterion is "not necessarily optimal," and the results suggest that data quality for RLVR is a multidimensional property that variance captures imperfectly.

What might those dimensions be? The paper's analyses suggest several candidates. Solvability by the base model: $\pi_1$ has high base-model pass@1 (~64% including all answers near 12.7-12.8), $\pi_{1208}$ has near-zero base-model pass@1, and the latter performs dramatically worse. This suggests that effective examples must be within the base model's capability range—confirming the "capability activation" framing from Innovation 1. Reasoning chain complexity: the simplified $\pi'_1$ (a one-step cube root computation) substantially underperforms the full $\pi_1$ (multi-step algebra with physics context), suggesting that examples with rich, multi-step reasoning chains provide better behavioral templates. Correctness of the label: $\pi_{1207}$ has a wrong label (the correct answer is 2/3, but the DSR-sub label is 4/5) and performs substantially worse (54.0% MATH500), while small label errors (12.8 vs. 12.7 for $\pi_1$) are essentially harmless (Table 5, Rows 5 vs. 11). This suggests that catastrophic label errors degrade performance by teaching incorrect answer patterns, while minor numerical imprecisions are absorbed by the model's existing knowledge.

This reframing from data quantity to data quality has direct practical consequences. It implies that effort currently spent on curating large RLVR datasets might be better spent on identifying or constructing a small number of highly effective "template examples" that provide clean demonstrations of the reasoning behaviors the model should adopt. It also suggests that the variance-based ranking procedure—running full-dataset RLVR once to profile examples, then selecting the best ones—is unnecessarily expensive. If the key property is base-model solvability and chain complexity, these could potentially be estimated from the base model's own outputs without any RLVR training, dramatically reducing the cost of example selection. The paper's observation that the same $\pi_1$ example works across multiple model families (Qwen2.5-Math-1.5B, Qwen2.5-Math-7B, Llama-3.2-3B-Instruct) and even for in-context learning (Table 16: $\pi_1$ as a 1-shot prompt improves Qwen2.5-Math-7B MATH500 from 51.0% to 75.4%) further suggests that "good examples" may be transferable across models, making the one-time cost of identifying them amortizable.

Innovation 5: Entropy Loss as an Independent Driver of Reasoning Improvement

The final conceptual contribution is the discovery that entropy loss—typically viewed as a minor auxiliary regularizer in policy gradient methods—can independently drive substantial reasoning improvements even in the complete absence of outcome rewards. This finding (Section 4.2, Table 13, Table 5 Row 10) has both practical and theoretical implications that reshape how we think about the components of RLVR training.

The standard view of entropy bonuses in RL is that they serve a purely auxiliary role: they prevent premature policy collapse by maintaining a minimum level of stochasticity, enabling continued exploration. Without an entropy bonus, policy gradient methods can converge to deterministic policies that exploit a locally optimal strategy, missing better solutions that require exploration to discover. The bonus is a means to an end—it enables the policy gradient to do its job of optimizing the reward, but is not itself a source of improvement.

This paper's entropy-loss-only experiments challenge that view. On Qwen2.5-Math-1.5B, training for 20 steps with entropy loss alone (no outcome reward, no policy gradient, no KL divergence) improves MATH500 from 36.0% to 63.4% (Table 13). This is a +27.4 percentage point gain—not far from the +29.0 point gain of the format-reward baseline (65.0%) and more than half the +36.8 point gain of full outcome-reward 1-shot RLVR with $\pi_1$ (72.8%). Similar patterns hold for Qwen2.5-Math-7B (+6.2% MATH500 in 4 steps) and Llama-3.2-3B-Instruct (+7.0% in 10 steps). Something about simply encouraging the model to be more uncertain about its token predictions causes it to produce better mathematical answers, even without any correctness signal.

The paper's interpretation (Appendix C.2.2) is that entropy loss drives format correction: encouraging higher per-token entropy produces more diverse outputs, and a more diverse output distribution is more likely to include properly formatted \boxed{} answers by chance. Since the base model sometimes knows the correct answer, and entropy loss makes it more likely to output that answer in a parseable format, accuracy improves. Under this interpretation, entropy loss is essentially doing unsupervised format correction—it doesn't teach correctness, but it removes the "format barrier" that prevents the base model's existing knowledge from being expressed in a verifier-compatible way.

But this interpretation leaves open a deeper question: if entropy loss primarily fixes formatting, why does it provide only partial gains compared to format-reward RLVR (which explicitly teaches formatting)? The format-reward baseline achieves 65.0% MATH500 vs. 63.4% for entropy loss—close but not identical. The gap suggests that format-reward RLVR provides a more targeted formatting signal than the diffuse exploration encouraged by entropy loss. Conversely, the fact that entropy loss can recover most of the format-correction gain without any explicit formatting signal suggests that a substantial fraction of the "format problem" is not about the model not knowing how to format—it presumably saw millions of \boxed{} examples during pretraining—but about the model's output distribution being too peaked (low entropy) and thus rarely sampling the well-formatted outputs it knows how to produce. Entropy loss "unlocks" these outputs by flattening the distribution.

The theoretical implication is significant: entropy regularization in RLVR is not merely an auxiliary component but an active contributor to downstream performance, operating through a mechanism (format barrier removal) that is complementary to and partially independent of the policy gradient's correctness signal. This explains why the entropy coefficient must be carefully tuned (Table 5: $\alpha = -0.001$ works, $\alpha = -0.003$ causes instability)—too little entropy doesn't unlock enough diversity, too much entropy produces random outputs. It also explains the synergy with post-saturation generalization: the entropy loss does double duty, providing both initial format correction and sustained exploration after training saturation. This dual role makes entropy loss a far more important hyperparameter in RLVR than the field has typically recognized, and suggests that "entropy scheduling" (starting with high entropy to unlock format, then annealing to stabilize correctness-guided improvement) could be a fruitful direction for future work.

5. Experimental Analysis

Evaluation Methodology

  • Dataset. The paper uses the DeepScaleR-Preview-Dataset (Luo et al., 2025), specifically a randomly selected subset of 1,209 examples referred to as "DSR-sub", as the primary training instance pool (Section 3.1). This dataset consists of approximately 40,000 unique mathematics problem-answer pairs from AIME (1984-2023), AMC (pre-2023), and other sources including Omni-MATH and Still, processed through a pipeline involving answer extraction with Gemini-1.5-Pro-002, RAG-based deduplication using Sentence-Transformers embeddings, and SymPy-based filterability checks (Appendix B.2). The full MATH training set (7,500 examples from Hendrycks et al., 2021) is used as a comparison for full-dataset RLVR. The historical variance score for ranking examples is computed once on the 1,209-example DSR-sub using Qwen2.5-Math-1.5B, and this ranking is kept fixed for all subsequent experiments across all models to avoid circularity (Section 3.1).

  • Base models. The primary experiments use Qwen2.5-Math-1.5B (Yang et al., 2024), with secondary experiments on Qwen2.5-Math-7B, Llama-3.2-3B-Instruct (Grattafiori et al., 2024), and DeepSeek-R1-Distill-Qwen-1.5B (Guo et al., 2025). The Qwen2.5-Math-1.5B is chosen as the main model because the authors argue it sits in a "sweet spot" for studying data efficiency: its base performance on MATH500 is 36.0%, leaving substantial room for improvement, and it is representative of commonly used model scales in RLVR research. The paper additionally includes results for Qwen2.5-1.5B (the non-math-specific base model) and Qwen2.5-Math-1.5B-Instruct (the instruction-tuned variant) in Appendix C.1.2 to test generalization across model families. The Qwen2.5-Math-7B tests scaling to a larger model size, Llama-3.2-3B-Instruct tests a non-Qwen architecture, and DeepSeek-R1-Distill-Qwen-1.5B tests a model already distilled from long chain-of-thought data.

  • Metrics. The primary metric throughout is mathematical reasoning benchmark accuracy, computed as pass@1 for most benchmarks and avg@k for small benchmarks to reduce variance. For MATH500, Minerva Math, and OlympiadBench (which have 272-675 questions), accuracy is computed using greedy decoding (temperature 0) with a single sample per question. For AIME2024 (30 questions), AIME2025 (30 questions), and AMC2023 (40 questions), the test sets are repeated 8 times and the model generates responses at temperature 0.6, with the reported metric being avg@8—the average pass@1 across all 8 repetitions. The Qwen2.5-Math evaluation pipeline (Yang et al., 2024) handles answer extraction (parsing the content inside \boxed{}) and correctness checking via rule-based grading that accounts for equivalent mathematical expressions. For non-mathematical reasoning, ARC-Easy and ARC-Challenge accuracy is reported using the standard multiple-choice evaluation protocol (Appendix B.3). The paper reports two versions of results: checkpoints achieving best average across 6 benchmarks (the primary metric used throughout Sections 3.2 and 3.3) and checkpoints achieving best performance on each benchmark independently (reported in Tables 9 and 11 for estimating upper bounds). Training accuracy is reported as the fraction of the 1,024 responses per rollout step (128 batches × 8 responses) that produce the correct answer for the training example.

  • Baselines. The paper employs multiple baselines arranged in increasing order of sophistication. The base model (no training, denoted "NA") provides the floor. Format-reward RLVR (Appendix C.2.3, Table 14) trains with GRPO using a reward of 1 if the model's output contains any parseable \boxed{} content regardless of correctness—this isolates format-correction effects from reasoning improvement. Full-dataset outcome-reward RLVR using the 1,209-example DSR-sub and the 7,500-example MATH training set establishes the ceiling that 1-shot RLVR aims to match. In-context learning with 1 or 4 examples (Table 16) tests whether simply providing the example as a prompt prefix (without any RL training) can achieve similar gains—this distinguishes RLVR's training dynamics from few-shot prompting effects. For the Qwen models specifically, the paper uses the "qwen25-math-cot" evaluation template consistently, but does not compare against alternative templates, which recent work (Dr. GRPO, Liu et al., 2025) has shown can substantially affect base model performance. The four in-context learning examples from the official Qwen evaluation repository (Yang et al., 2024) provide one template comparison point.

  • Generation budget / compute accounting. The paper does not measure compute in terms of FLOPs or wall-clock time. The controlled resource is the training dataset size (1, 2, 4, 16, or 1,209 examples), not the number of training steps or gradient updates. Critically, the paper explicitly acknowledges (Appendix D.1): "1-shot RLVR doesn't save (and maybe requires more) RL computation." The number of training steps is held constant or nearly constant across dataset sizes: 2,000 steps for Qwen2.5-Math-1.5B with both 1-shot and full-dataset RLVR, 1,000 for Qwen2.5-Math-7B and Llama-3.2-3B-Instruct, and 1,200 for DeepSeek-R1-Distill-Qwen-1.5B. Each step processes 128 batches × 8 responses = 1,024 model generations, regardless of whether the 128 batches contain 128 copies of the same example or 128 different examples. Thus, total inference FLOPs are equal across dataset sizes; what differs is the diversity of prompts generating those FLOPs. This makes the paper's efficiency claim about data requirements (how many distinct examples must be collected and curated), not compute requirements. The historical variance score profiling step (500 steps of full-dataset RLVR to compute variance rankings) is an additional one-time cost not amortized into any efficiency calculation.

  • Cross-validation / statistical protocol. There is no cross-validation for checkpoint selection or strategy comparison. The paper evaluates on fixed test benchmarks (MATH500, AIME2024, AMC2023, Minerva Math, OlympiadBench, AIME2025) with no held-out validation split used for early stopping or hyperparameter selection. The best checkpoint is selected by retrospectively identifying the step that achieved the highest average test accuracy—a form of test-set optimization that may overestimate true generalization performance, particularly for benchmarks with only 30-40 questions where test accuracy is noisy. For the AIME2024/2025 and AMC2023 benchmarks, the paper uses repeated evaluation (8 passes with temperature 0.6) to reduce variance from the small test sets. For larger benchmarks (MATH500, Minerva Math, OlympiadBench), a single greedy decoding pass is used. No confidence intervals, standard errors, or statistical significance tests are reported anywhere in the paper. The paper also notes a precision inconsistency between Hugging Face bfloat16 models and the float32 checkpoints saved by the verl/deepscaler pipeline (Appendix B.6, Table 7), with base model performance differing slightly depending on the checkpoint format used (e.g., Qwen2.5-Math-1.5B MATH500: 36.0% from Hugging Face vs. 39.6% from the stored initial checkpoint). The paper uses the Hugging Face downloaded model for recording initial performance, which is a conservative choice (making RLVR gains appear slightly larger than if the stored-initial-checkpoint baseline were used).

Main Quantitative Results

1-Shot RLVR Matches Full-Dataset RLVR Performance

The paper's central empirical result appears in Figure 1 and Tables 8-9. Applying RLVR with a single training example ($\pi_1$) to Qwen2.5-Math-1.5B improves MATH500 accuracy from 36.0% (base model) to 72.8%—a gain of 36.8 percentage points—at the checkpoint achieving best average across 6 benchmarks (Table 8). This is within 0.8 percentage points of the 1,209-example DSR-sub dataset (73.6%) and within 1.6 points of the 7,500-example MATH training set (74.4%). On the 6-benchmark average, 1-shot RLVR with $\pi_1$ achieves 35.0% versus 35.9% for DSR-sub and 36.7% for MATH—a gap of 0.9 and 1.7 percentage points respectively. With a second example ($\pi_{13}$), 2-shot RLVR achieves 74.8% MATH500 (exceeding DSR-sub's 73.6%) and 36.6% average (exceeding DSR-sub's 35.9%). The format-reward baseline (which isolates format-correction gains) reaches 65.0% MATH500 and 28.7% average, establishing that 1-shot RLVR with outcome reward provides an additional +7.8% MATH500 and +6.3% average beyond pure format fixing. The improvement curves in Figure 1 show that 1-shot RLVR with $\pi_{13}$ (green line) tracks the DSR-sub curve (blue) closely, with $\pi_1$ slightly below and the 2-shot variant {$\pi_1$, $\pi_{13}$} slightly above. All 1/2-shot variants dramatically outperform the format-reward baseline (gold line) and the base model (starting point of all curves at step 0).

When reporting best per-benchmark results independently (Table 9), the gaps narrow further: 1-shot RLVR with $\pi_1$ achieves 74.0% MATH500 (vs. 75.2% DSR-sub), 16.7% AIME24 (vs. 18.8%), 54.4% AMC23 (vs. 52.5%), 30.2% Minerva Math (vs. 34.9%), 35.3% OlympiadBench (vs. 35.1%), and 9.2% AIME25 (vs. 11.3%). On four of six benchmarks (MATH500, AMC23, OlympiadBench, and arguably AIME24 within sampling noise), the 1-shot performance is within striking distance of full-dataset RLVR. Two-shot RLVR with $\pi_1$ and $\pi_{13}$ achieves the best individual MATH500 result of any configuration (76.0%), exceeding both DSR-sub (75.2%) and the MATH training set (75.4%).

Different Single Examples Produce Dramatically Different Outcomes

Table 3 (Section 3.2.3) evaluates 15 different single examples used in 1-shot RLVR, spanning high-variance ($\pi_1$, $\pi_{13}$, $\pi_{17}$), medium-variance ($\pi_{605}$, $\pi_{606}$), and low-variance ($\pi_{1201}$, $\pi_{1209}$) examples according to the historical variance ranking. The MATH500 performance varies from 74.0% ($\pi_1$, step 1860) and 74.4% ($\pi_{13}$, step 1940) at the top to 45.0% ($\pi_{1208}$, step 240) and 54.0% ($\pi_{1207}$, step 100) at the bottom—a range of 29.4 percentage points attributable purely to example choice. The two worst-performing examples have identifiable flaws: $\pi_{1207}$ has a wrong label in the DSR-sub dataset (the correct answer should be 2/3, but the dataset label is 4/5), and $\pi_{1208}$ is an extremely difficult problem that the base model essentially never solves correctly, producing near-zero training accuracy throughout RLVR. Excluding these two flawed examples, the range narrows to 64.0% ($\pi_7$, $\pi_{11}$) to 74.4% ($\pi_{13}$)—still a 10.4 percentage point spread, demonstrating substantial variability even among "functional" examples.

The category-specific breakdown within Table 3 reveals that performance improvements are broadly distributed across all MATH500 subjects regardless of the training example's category. For instance, $\pi_1$ (an Algebra problem) improves Geometry from 39.0% (base) to 56.1%, Number Theory from 24.2% to 79.0%, and Precalculus from 33.9% to 64.3%. $\pi_{13}$ (a Geometry problem) improves Algebra from 37.1% to 89.5%, Number Theory from 24.2% to 83.9%, and Intermediate Algebra from 43.3% to 55.7%. There is no clear pattern of in-category advantage: a Number Theory example ($\pi_{11}$) achieves only 66.1% on Number Theory (the lowest among the top-ranked examples), while a Precalculus example ($\pi_{605}$) achieves 82.3% on Number Theory (among the highest). The paper notes this counterintuitive result in Section 3.2.3: "test data belonging to the same category as the single training example does not necessarily exhibit better improvement."

The best checkpoint step also varies dramatically across examples (Table 3, "Step" column): $\pi_{11}$ peaks at step 20, $\pi_4$ at step 80, $\pi_2$ at step 220, $\pi_{606}$ at step 460, $\pi_{1209}$ at step 1140, $\pi_1$ at step 1860, and $\pi_{13}$ at step 1940. This variation in optimal stopping time is substantial—two orders of magnitude—and correlates roughly with eventual performance: examples that peak early tend to achieve lower terminal accuracy (e.g., $\pi_{11}$: 64.0% at step 20; $\pi_4$: 65.6% at step 80), while examples that benefit from extended post-saturation training achieve the highest scores ($\pi_1$, $\pi_{13}$, $\pi_{1209}$). This suggests a connection between post-saturation generalization duration and eventual performance that the paper does not fully explore quantitatively.

Post-Saturation Generalization: Test Improvement After Training Saturation

Figure 2 (Section 3.2.2) documents the core dynamic of 1-shot RLVR. For $\pi_1$ (left panel), training accuracy (red curve) rises rapidly to near 100% before step 100 and remains saturated. Yet test accuracy (blue curve, 6-benchmark average) continues improving from approximately 31.5% at step 100 to 35.0% at step 1540—a gain of 3.5 percentage points occurring entirely after training saturation. For $\pi_{13}$ (middle panel), the effect is even more pronounced: training accuracy saturates by step 200, but test accuracy climbs from approximately 24% at step 500 to 35.7% at step 2000—a 9.9 percentage point post-saturation gain. In contrast, the full-dataset DSR-sub training (right panel) shows no post-saturation generalization within the 2,000 steps observed: training accuracy has not saturated by step 2000 (it continues rising), and test accuracy plateaus around step 1000 and begins declining after step 1500.

The paper provides qualitative evidence of what happens during post-saturation training via Figure 3, which shows model responses to both the training example $\pi_1$ and a selected MATH500 test problem at four checkpoints (steps 0, 500, 1300, 1860). At step 500 (post-saturation, when training accuracy is ~100%), the model produces a clean, correct solution to $\pi_1$ and a partially correct but incomplete solution to the test problem (finding x=5 but missing x=3,7). At step 1300 (mid-post-saturation), the model's $\pi_1$ solution becomes more elaborate—it still arrives at the correct answer but uses more verbose reasoning with conversational phrasing ("For a moment press onward investing concrete data to pinpoint...")—while its test solution improves substantially, correctly finding all three roots using a systematic approach. At step 1860 (terminal checkpoint), the model has overfit $\pi_1$: its output mixes the correct calculation with long stretches of unintelligible multilingual gibberish (Chinese characters, Korean, random Unicode). Yet the test response remains perfectly coherent, correctly solving the cubic equation and even trying a different strategy (Rational Root Theorem) than at step 1300. The paper quantifies this overfitting threshold: "overfitting in RLVR occurs quite late (π1 after 1400 steps and π13 after 1800 steps)" and notes that "considering that each example is sampled 1024 times per step, the single training example is not overfitted until after millions of rollouts."

Supporting this qualitative picture, Figure 4 (left) shows that average response length on the training data remains stable through step ~1300 for $\pi_1$ and step ~1700 for $\pi_{13}$, then increases dramatically—from ~500 tokens to ~2500-3000 tokens—coinciding with the onset of overfitting gibberish outputs. Figure 4 (middle) confirms that entropy loss increases in parallel with response length, indicating the model is becoming more uncertain/exploratory in its token predictions. Figure 4 (right) tracks the frequency of self-reflection keywords ("rethink", "recheck", "recalculate") in evaluation task outputs: for both $\pi_1$ and $\pi_{13}$, the number of responses containing these words increases significantly after step ~1250, from fewer than 10 occurrences to 30-50 occurrences per benchmark set. In contrast, full-dataset DSR-sub training shows a gradual decrease in reflection words as training progresses, from ~25 at step 0 to ~10 at step 1500. This divergence—1-shot RLVR increasing reflection while full-dataset RLVR decreases it—is one of the paper's more striking qualitative findings.

Cross-Model and Cross-Algorithm Generalization

Table 4 (Section 3.3) extends 1-shot RLVR to three additional model families and one additional algorithm. On Qwen2.5-Math-7B, 1-shot RLVR with $\pi_1$ achieves 40.2% average across 6 benchmarks, compared to 42.8% for DSR-sub (1,209 examples) and 34.3% for format-reward baseline. This represents a 17.8 percentage point improvement over the base model's 22.4%—with 5.9 points of that attributable to non-format gains (40.2% - 34.3%). Individual benchmark results: MATH500 79.2% (vs. 78.6% DSR-sub, 51.0% base), AIME24 23.8% (vs. 25.8% DSR-sub), OlympiadBench 39.1% (vs. 41.6% DSR-sub). Four-shot RLVR with {$\pi_1$, $\pi_2$, $\pi_{13}$, $\pi_{1209}$} achieves 42.5% average, matching DSR-sub's 42.8%. Importantly, the set of 16 top-variance examples {$\pi_1$, ..., $\pi_{16}$} achieves 42.5% average, outperforming 16 randomly sampled examples (40.2%), providing the paper's only direct evidence that the historical variance score provides a better-than-random ranking—though the gap (2.3 percentage points) is modest relative to the variance across individual examples in Table 3.

On Llama-3.2-3B-Instruct, the absolute gains are smaller across all configurations. The base model achieves 17.5% average; DSR-sub full-dataset RLVR achieves 19.8%; 1-shot RLVR with $\pi_1$ achieves 19.0%; 2-shot RLVR with {$\pi_1$, $\pi_{13}$} achieves 21.0% (exceeding DSR-sub). However, the paper notes (Appendix C.1.5) that "Llama3.2-3B-Instruct is more unstable during training, as almost all setups start having performance degradation before 200 steps." The detailed curves in Figure 9 confirm this: all variants peak between steps 60-200 and then decline, with 4-shot RLVR performing worse than 2-shot by step 1000. This instability limits the reliability of the comparison and suggests that 1-shot RLVR's viability may depend on model-specific training dynamics.

On DeepSeek-R1-Distill-Qwen-1.5B (a model already fine-tuned on long chain-of-thought reasoning data), the gap between 1-shot and full-dataset RLVR is larger. At 32k evaluation length (Table 4, bottom section), DSR-sub achieves 48.6% average while 1-shot RLVR with $\pi_1$ achieves 46.3%—a gap of 2.3 points. Four-shot RLVR achieves 46.9% and 16-shot achieves 48.3%, nearly closing the gap. The paper suggests that for models already strong at reasoning, more examples provide diminishing but non-zero returns, and "using few-shot RLVR may be more stable for training" (Appendix D.1).

On PPO (rather than GRPO), trained on Qwen2.5-Math-1.5B (Table 4, third group): DSR-sub achieves 35.4% average; 1-shot RLVR with $\pi_1$ achieves 33.8%—a gap of 1.6 points. This demonstrates that the 1-shot phenomenon is not specific to GRPO's group-normalized advantage structure, though the paper does not explore whether PPO exhibits post-saturation generalization or the same difficulty-dependent patterns.

Entropy-Loss-Only and Format-Reward Baselines

Table 14 and Figure 13 (Appendix C.2.3) compare outcome-reward RLVR against format-reward RLVR for both full-dataset and 1-shot training on Qwen2.5-Math-1.5B. The headline finding: format-reward RLVR with DSR-sub (1,209 examples) achieves 65.0% MATH500 and 28.7% average, while outcome-reward RLVR with DSR-sub achieves 73.6% MATH500 and 36.0% average—a gap of 8.6 points MATH500 and 7.3 points average attributable to correctness-guided reasoning improvement beyond format fixing. For 1-shot RLVR with $\pi_1$, format-reward achieves 65.4% MATH500 and 29.2% average; outcome-reward achieves 72.8% MATH500 and 35.0% average—nearly identical gaps (7.4 points MATH500, 5.8 points average). Strikingly, format-reward RLVR performs almost identically regardless of dataset size: DSR-sub (1,209 examples) and $\pi_1$ (1 example) achieve 65.0% vs. 65.4% MATH500 and 28.7% vs. 29.2% average. This suggests that format correction requires essentially zero data diversity—any set of properly formatted examples suffices to teach the model to produce parseable \boxed{} answers.

Table 5 (Rows 9-10) and Table 13 (Section 4.2, Appendix C.2.2) evaluate entropy-loss-only training, where the GRPO policy gradient and KL divergence terms are removed, leaving only the entropy bonus (with $\alpha = -0.001$). On Qwen2.5-Math-1.5B, training for 20 steps with entropy loss alone using $\pi_1$ (which serves only as a prompt to generate outputs; the reward is not used) improves MATH500 from 36.0% to 63.4% and the 6-benchmark average from 17.6% to 25.0%. This is remarkably close to the format-reward baseline's 65.0% MATH500 and 28.7% average, despite providing no explicit formatting signal. On Qwen2.5-Math-7B, entropy-loss-only training for 4 steps improves MATH500 from 51.0% to 57.2% (vs. format-reward's 65.8%). On Llama-3.2-3B-Instruct, 10 steps of entropy-loss-only training improves MATH500 from 40.8% to 47.8%. The paper interprets this as evidence that entropy loss drives format correction indirectly: by encouraging the model to be less deterministic, it increases the probability of sampling well-formatted outputs that the model already knows how to produce but was not generating under its default peaked output distribution.

In-Context Learning Comparison

Table 16 (Appendix C.2.3) compares 1-shot RLVR against in-context learning—providing the training example as a prompt prefix without any gradient updates. On Qwen2.5-Math-1.5B, 1-shot RLVR with $\pi_1$ achieves 72.8% MATH500 and 35.0% average; in-context learning with $\pi_1$ (including the example's question and a ground-truth solution in the prompt) achieves 59.0% MATH500 and 25.5% average—a gap of 13.8 points MATH500. In-context learning with the official Qwen evaluation repository's 4 examples achieves 49.8% MATH500 and 18.0% average (barely above the base model's 17.6%). On Qwen2.5-Math-7B, $\pi_1$ as a 1-shot in-context prompt achieves 75.4% MATH500 and 37.4% average (vs. 79.2% and 40.2% for 1-shot RLVR), while the official 4 examples achieve 59.2% and 21.7%—actually below the base model's 22.4% on some benchmarks. This demonstrates that $\pi_1$ is an unusually effective in-context example independent of RL training, substantially outperforming Qwen's hand-picked examples, but that RLVR still provides additional gains beyond what in-context learning achieves.

Non-Mathematical Generalization

Table 1 (Section 2) shows that 1-shot RLVR with math examples improves performance on non-mathematical reasoning tasks. On ARC-Challenge, the base Qwen2.5-Math-1.5B achieves 30.2%; full-dataset DSR-sub RLVR achieves 29.9% (no improvement); 1-shot RLVR with $\pi_{13}$ achieves 33.4% (a +3.2 point gain). On ARC-Easy, the base model achieves 48.0%; DSR-sub achieves 42.2% (a degradation); 1-shot RLVR with $\pi_{13}$ achieves 55.8% (+7.8 points). Counterintuitively, full-dataset RLVR worsens ARC performance while 1-shot RLVR improves it—a finding the paper reports but does not explain. The paper notes this as evidence that "1(few)-shot RLVR with these math examples enable better generalization on non-mathematical reasoning tasks" (Section 3.2), though the mechanism remains unclear and the result is reported on only two benchmarks.

Ablation Studies and Robustness Checks

Policy gradient loss is the primary driver of 1-shot RLVR improvement (Table 5, Rows 1-8). Removing policy gradient while keeping KL divergence, weight decay, and entropy loss (Row 9) drops MATH500 to 65.4% and AIME24 to 7.1%—essentially the entropy-loss-only baseline. Removing entropy loss from the full GRPO loss (Row 2) yields 71.8% MATH500 and 15.4% AIME24—only slightly below the full-loss 74.8%/17.5% (Row 5), confirming that entropy loss provides a modest but real boost. Removing weight decay (Row 7: policy gradient + entropy only) yields 75.6% MATH500—slightly better than the full loss, suggesting weight decay may mildly impair 1-shot RLVR performance. Removing weight decay and keeping only policy gradient (Row 2) yields 71.8%, already close to full loss. Removing all loss components (Row 1) yields 39.8% MATH500—barely above the 36.0% base model, confirming that simply generating outputs and checkpointing (without any training signal) provides negligible improvement. The key pattern: policy gradient alone accounts for most of the gain (71.8% vs. 74.8% full loss), entropy loss adds a smaller but consistent boost (71.8% → 74.8%), and weight decay and KL divergence are inessential (adding them to policy gradient yields 71.4% and 70.8%, slightly below policy gradient alone).

Entropy coefficient sensitivity (Table 5, Rows 5-6). Increasing the entropy loss coefficient from $\alpha = -0.001$ (Row 5) to $\alpha = -0.003$ (Row 6) degrades MATH500 from 74.8% to 73.6% and AIME24 from 17.5% to 15.4%. The paper notes this makes "training more unstable" (Section 4.1). This sensitivity aligns with prior observations from Skywork (He et al., 2025, cited in Appendix C.2.2) that entropy regularization is effective but coefficient-dependent in full-dataset RLVR as well.

Label correctness robustness (Table 5, Rows 5, 11-13). Using the mathematically correct label for $\pi_1$ ("12.7", Row 11) instead of the DSR-sub label ("12.8", Row 5) yields nearly identical performance: MATH500 73.4% vs. 74.8%, AIME24 17.9% vs. 17.5%. Using a wrong but learnable label ("4", Row 12) causes training accuracy to still saturate (the model can learn to output "4" for this problem) but degrades MATH500 to 57.0% and AIME24 to 9.2%—worse than the 63.4%/8.8% achieved by entropy-loss-only training (Row 10), suggesting that actively reinforcing a wrong answer is worse than providing no correctness signal. Using a wrong and unlearnable label ("929725", Row 13) causes training accuracy to never converge (the model never receives reward), yielding 64.4% MATH500 and 9.6% AIME24—close to the entropy-loss-only baseline. The paper interprets this as: "if the incorrect label deviates substantially while remaining guessable and overfittable, the resulting performance can be even worse than using a completely incorrect and unguessable label, which behaves similarly to training with entropy loss alone" (Section 4.2).

Prompt complexity matters beyond the "tricky step" (Table 18, Appendix C.2.5). Training on a simplified version of $\pi_1$ whose prompt is simply "Calculate $\sqrt[3]{2048}$" ($\pi'_1$, which isolates the specific step the base model finds hardest) achieves only 65.4% MATH500 and 30.0% average—essentially matching the format-reward baseline (65.0%, 28.7%) and representing only a 1.3 percentage point average improvement beyond format correction. This is dramatically worse than the full $\pi_1$ (72.8%, 35.0%). The paper concludes that "although solving $\sqrt[3]{2048}$ is one of the most difficult parts of π1, π1 still needs other key steps to solve... that may generate different patterns of CoT (rather than just calculating), which may allow more exploration space at the post-saturation generalization stage."

Random label noise in full-dataset RLVR (Table 17, Appendix C.2.4). When 60% of the DSR-sub dataset's labels are randomly corrupted (randomly adding/subtracting within 10, changing signs, or modifying fraction numerators/denominators), full-dataset RLVR (GRPO) achieves 34.6% average—only 1.3 points below the clean-label 35.9%. When 90% of labels are corrupted (leaving only ~120 correct labels among 1,209 examples), performance drops to 31.2% average. Critically, 1-shot RLVR with $\pi_1$ (which has a single correct label) achieves 35.0% average—outperforming the 90%-corrupted full dataset by 3.8 points. Under PPO, the same pattern holds: 90% corrupted labels achieve 32.9% vs. 1-shot RLVR's 33.8%. This demonstrates a striking form of label-noise robustness in RLVR: a single clean example can outperform a dataset where most examples have wrong answers, because wrong-answer examples provide actively misleading policy gradient signals that interfere with the correct signals from the few clean examples.

Entropy loss and temperature interaction for post-saturation generalization (Figure 5). Without entropy loss, test performance (6-benchmark average) plateaus around step 150 at ~30% and shows limited further improvement through step 2000. With entropy loss ($\alpha = -0.001$) and rollout temperature 0.6, performance reaches ~32.3% by step 2000, a 2.3 percentage point gain over the no-entropy variant. Increasing the rollout temperature to 1.0 (while keeping entropy loss) further boosts performance to ~33.1%, an additional 0.8 point gain. The paper interprets this as evidence that "encouraging greater diversity in model outputs... can enhance post-saturation generalization" (Section 4.1), with the mechanism being that higher entropy/temperature creates occasional errors that generate non-zero advantages, sustaining the policy gradient signal after training saturation.

Pass@8 analysis (Table 20, Appendix C.4). On three small benchmarks evaluated with pass@8 (AIME24, AIME25, AMC23), 1-shot RLVR with $\pi_1$ achieves its highest pass@8 at step 980 (51.7% average across the three tasks), matching DSR-sub's best pass@8 at step 160 (50.3%). However, DSR-sub's pass@8 declines thereafter—from 50.3% at step 160 to 48.6% at step 500, 42.8% at step 1000, and 41.3% at step 1500—a pattern consistent with recent findings (Yue et al., 2025) that RLVR can degrade pass@k performance even as pass@1 improves. 1-shot RLVR's pass@8 at step 500 is 42.2%, lower than DSR-sub's 48.6% at the same step, suggesting that 1-shot RLVR may not match full-dataset RLVR's pass@k performance at intermediate training stages. The paper only briefly mentions this point (Appendix C.4), noting it "is consistent with recent findings that RLVR may sometimes degrade the pass@n performance."

Response length dynamics (Table 19, Appendix C.3). Evaluation-set response lengths for 1-shot RLVR with $\pi_1$ show a modest increase from step 100 to step 1500 (MATH500: 611 → 740 tokens; AIME24: 1123 → 1352; AMC23: 939 → 986), while full-dataset DSR-sub RLVR shows a decrease over the same period (MATH500: 636 → 562; AIME24: 1268 → 949; AMC23: 874 → 762). The 1-shot length increase aligns with the rising entropy loss and reflection-word frequency in Figures 4, suggesting deeper/more exploratory reasoning. The full-dataset length decrease is consistent with the known tendency of RLVR to produce more concise solutions over time.

Critical Assessment

Do the experiments demonstrate that 1-shot RLVR "matches" full-dataset performance?

The central claim is that "1-shot RLVR can perform as well as using datasets with thousands of examples." The evidence partially supports this but with important caveats that the paper itself acknowledges in various places.

What the experiments actually show: On Qwen2.5-Math-1.5B, the best-performing 1-shot example ($\pi_{13}$) achieves a 6-benchmark average of 35.7% vs. 35.9% for DSR-sub (Table 8), and 2-shot RLVR achieves 36.6%, slightly exceeding DSR-sub. On Qwen2.5-Math-7B, 1-shot achieves 40.2% vs. 42.8% for DSR-sub—a gap of 2.6 points (Table 10). On Llama-3.2-3B-Instruct, 1-shot achieves 19.0% vs. 19.8% for DSR-sub—a gap of 0.8 points (Table 4). On DeepSeek-R1-Distill-Qwen-1.5B, 1-shot achieves 46.3% vs. 48.6%—a gap of 2.3 points. So "matches" is accurate for Qwen2.5-Math-1.5B and Llama-3.2-3B-Instruct (where the gap is within 1 point), approximately correct but with a small gap for Qwen2.5-Math-7B, and somewhat less accurate for DeepSeek-R1-Distill-Qwen-1.5B. The claim is most strongly supported for the model that the historical variance ranking was computed on (Qwen2.5-Math-1.5B), which is a form of indirect data leakage: the $\pi_1$/$\pi_{13}$ examples were selected based on full-dataset RLVR behavior on this exact model, so it is perhaps unsurprising that they perform best on this model.

What weakens the claim:

  1. The claim is about a specific, carefully selected example—not about 1-shot RLVR in general. The wide variance in Table 3 (45.0% to 74.4% MATH500) means that "1-shot RLVR" is not a well-defined method without specifying which example is used. The paper's title and abstract emphasize that RLVR with "one training example" is effective, but the body reveals that this is true only for certain examples, and identifying those examples required 500 steps of full-dataset RLVR. This circularity—needing full-dataset training to find the one example that makes full-dataset training unnecessary—is not a flaw in the empirical observation but limits its practical interpretation.

  2. "Matches" is measured at the best checkpoint, not at a fixed step. The comparison in Tables 8-9 selects the checkpoint achieving the best average or best per-benchmark performance. Given that different dataset sizes cause different training dynamics (Figure 2: DSR-sub peaks around step 1000-1500 and then degrades; $\pi_1$ peaks at step 1540-1860; $\pi_{13}$ peaks at step 1940-2000), comparing best-checkpoint-to-best-checkpoint is reasonable for establishing an upper bound but does not reflect a realistic deployment where the optimal stopping point is unknown a priori. The paper does not evaluate a held-out validation set for early stopping, so the best-checkpoint selection is inherently a form of test-set optimization, particularly for benchmarks as small as AIME2024 (30 questions) where best-checkpoint variance could be substantial.

  3. The comparison treats dataset size as the only cost, ignoring the cost of finding good examples. The historical variance score requires running full-dataset RLVR first. If this profiling cost is amortized over many subsequent 1-shot training runs, it might be negligible, but the paper does not make this argument or quantify the cost. Moreover, the $\pi_1$ example transfers reasonably well to other models (it was selected on Qwen2.5-Math-1.5B and works on 7B Qwen and Llama), but not perfectly—the gap between 1-shot and full-dataset is larger for DeepSeek-R1-Distill-Qwen-1.5B (2.3 points) and Qwen2.5-1.5B (7 points; Table 10 in Appendix C.1.2). The transferability of "good examples" across model families is asserted but not systematically studied.

Do the experiments demonstrate post-saturation generalization as a distinct phenomenon from grokking?

What the experiments actually show: Table 5 clearly demonstrates that weight decay—the component essential for grokking in prior work—is unnecessary for post-saturation generalization in 1-shot RLVR. Removing weight decay (Row 7) yields identical or slightly better performance than the full loss (Row 5). Removing policy gradient (Row 8) collapses performance. Removing entropy loss (Figure 5) eliminates or reduces the post-saturation improvement. This pattern is internally consistent and supports the paper's distinction.

What weakens this finding: The paper does not run a positive control—it does not demonstrate that grokking does occur in some configuration with weight decay and does not occur without it, establishing a baseline for what grokking looks like in this setting. The claim that post-saturation generalization is "not grokking" rests entirely on the negative result that removing weight decay does not impair performance. This is logically valid (if grokking requires weight decay, and this phenomenon does not, then this is not grokking) but empirically thin—it is a single ablation on one model with one hyperparameter configuration. The paper also does not test other known contributors to grokking (e.g., small dataset size relative to model capacity, which is certainly true here; specific weight decay schedules; the sudden-vs-gradual onset of generalization). The test accuracy curves in Figure 2 show gradual improvement, not the sudden phase transition characteristic of grokking, but this distinction is qualitative.

The more fundamental limitation is that the paper does not provide a mechanistic explanation for why post-saturation generalization occurs. The proposed mechanism (entropy loss encourages exploration, exploration causes occasional failures, failures create non-zero advantages, policy gradient corrects failures) is plausible but untested. Direct evidence for this cycle—showing that advantage magnitudes increase after exploration events, or that specific exploration-induced errors are subsequently corrected—is absent. Figure 16 shows that policy gradient loss remains non-zero after saturation, which is consistent with the proposed mechanism but does not causally link it to test improvement.

Do the experiments support the claim that RLVR "activates latent capability" rather than teaching new skills?

What supports this claim: The finding that $\pi_1$ is a problem the base model already largely solves (64% pass@1 across all near-correct answers), combined with the observation that a single example provides essentially no new mathematical knowledge, makes the capability-activation interpretation the most parsimonious one. The cross-category generalization results in Table 3 (training on Algebra improves Geometry) further support this: if RLVR were teaching mathematical content, content-specific transfer would be expected, not broad cross-category improvement. The format-reward baseline showing that ~76% of the total improvement on Qwen2.5-Math-1.5B is pure format correction (MATH500: 36.0% → 65.0% with format reward alone) also supports this interpretation—the model already knew the answers; it just wasn't expressing them properly.

What weakens this claim: The paper's evidence is entirely behavioral (input-output patterns), not mechanistic. The claim that "the base model already has strong reasoning capability" is inferred from the effectiveness of 1-shot RLVR, but no direct measurement of base model capability is provided beyond the pass@1 numbers on MATH500 (which is confounded by format issues). Recent work that the paper cites (Yue et al., 2025) suggests that pass@k metrics can reveal base model capabilities that pass@1 obscures. The paper does report limited pass@k results (Table 20, three benchmarks only), showing that the base model achieves 39.7% average pass@8 on AIME24/AIME25/AMC23—substantially higher than its 17.6% avg@1 across all six benchmarks—but does not systematically compare base model pass@k against 1-shot RLVR pass@k to determine whether RLVR is genuinely improving capability or merely redistributing probability mass toward already-existing correct solutions. A complete "activation" story would need to show that correct solutions exist in the base model's output distribution at non-trivial rates (high pass@k) and that RLVR increases pass@1 primarily by making those solutions more probable. The current evidence is suggestive but incomplete.

The claim is also challenged by the observation that 1-shot RLVR's pass@8 performance (Table 20) eventually exceeds the base model's pass@8 (51.7% for 1-shot RLVR at step 980 vs. 39.7% for the base model on the three-task subset). If RLVR were purely a probability-mass-redistribution mechanism, pass@k at high k should remain constant or decrease (since correct solutions are being concentrated rather than created). The fact that pass@8 increases over the base model suggests that RLVR is doing more than just "unlocking" existing capability—it is enabling the model to find correct solutions that were not present at any probability in the base distribution. This complicates the clean "activation" narrative and suggests a hybrid picture: RLVR partly unlocks existing capability (format correction) and partly creates new capability (improved reasoning strategies that produce correct answers the base model could not generate even with many attempts).

Are there missing experiments that would substantially strengthen or challenge the paper's conclusions?

Missing: systematic study of what makes an example "good" for 1-shot RLVR. The paper identifies several candidate properties (base-model solvability, reasoning chain complexity, label correctness) but does not systematically vary them. A factorial experiment crossing these properties (2 × 2 × 2: easy vs. hard for base model, simple vs. complex reasoning chain, correct vs. slightly-incorrect label) would transform the descriptive Table 3 into a predictive framework. As it stands, the paper's advice for selecting examples reduces to "run full-dataset RLVR, compute variance, pick high-variance examples"—which is not practically useful for avoiding full-dataset RLVR.

Missing: comparison against training on a single example with supervised fine-tuning (SFT) instead of RLVR. If the claim is that RLVR specifically is effective with one example, a natural baseline is SFT on that same example (i.e., behavior cloning on the ground-truth solution). If SFT on $\pi_1$ achieved similar performance to 1-shot RLVR, the finding would be about data efficiency in general, not RLVR specifically. If SFT on $\pi_1$ failed, it would strengthen the claim that RL's exploration and advantage-weighted updates are doing something SFT cannot.

Missing: systematic evaluation of base model pass@k across all benchmarks at multiple k values. The paper reports pass@8 on only three benchmarks (Table 20) and does not report pass@1, pass@4, pass@16 systematically. Given recent findings that RLVR can degrade pass@k while improving pass@1 (Yue et al., 2025), understanding whether 1-shot RLVR's gains hold at higher k—or are purely a pass@1 phenomenon—is important for interpreting the capability-activation claim.

Missing: evaluation on held-out validation data for checkpoint selection. The practice of selecting the best checkpoint based on test-set performance is standard in much RLVR work but is conceptually problematic when the test sets are small (30 questions for AIME) and the effect sizes being claimed are modest (2-5 percentage points). A proper validation split would provide a more conservative estimate of generalization.

Missing: multiple random seeds. The paper uses a single random seed (seed 0 for evaluation) and does not report variance across training runs with different random initializations or data orders. Given the instability noted for Llama-3.2-3B-Instruct (Figure 9) and the sensitivity to entropy coefficient (Table 5), seed variance could be substantial. A finding that 1-shot RLVR with $\pi_1$ achieves 72.8% ± 2% MATH500 across 5 seeds would be far more convincing than a single run achieving 72.8%.

Missing: evaluation on mathematical reasoning benchmarks beyond the six used. The paper uses MATH500, AIME2024/2025, AMC2023, Minerva Math, and OlympiadBench—a strong and diverse set. But it does not test on widely used alternatives like GSM8K (grade-school math, which might show ceiling effects) or newer benchmarks like Omni-MATH. The two non-mathematical benchmarks (ARC-Easy, ARC-Challenge) are a start but are relatively small and multiple-choice, limiting the strength of the cross-domain generalization claim.

Missing: investigation of whether the $\pi_i$ ranking transfers to identifying good examples for other model families without re-running profiling on each family. The paper runs historical variance profiling once on Qwen2.5-Math-1.5B and reuses the ranking for all other models. This is a practical choice but leaves open whether a profiled ranking on the target model would yield even better 1-shot examples (especially for DeepSeek-R1-Distill-Qwen-1.5B, where the gap between 1-shot and full-dataset is largest). A comparison of 1-shot performance using $\pi_1$ (selected from Qwen2.5-Math-1.5B profiling) versus the top-variance example from profiling on the target model itself would quantify the transferability cost.

Missing: direct comparison against LIMR's data selection approach. As the most directly relevant prior work (reducing RLVR datasets from 8.5k to 1.4k examples), a head-to-head comparison would situate this paper's contribution. Does LIMR's top-ranked example perform as well as $\pi_1$? Does LIMR's metric identify the same examples? The paper cites LIMR but does not use it as a baseline data selection method.

Summary assessment

The paper's core empirical claim—that a carefully selected single example can produce RLVR performance close to that of a 1,209-example dataset—is robustly supported for Qwen2.5-Math-1.5B with some caveats about checkpoint selection, seed variance, and benchmarking scope. The claim weakens somewhat for other model families (Qwen2.5-Math-7B, DeepSeek-R1-Distill-Qwen-1.5B) where small but consistent gaps remain. The conceptual claims about post-saturation generalization being distinct from grokking, RLVR as capability activation, and the decomposition of RLVR gains into format/entropy/correctness components are well-supported by the ablation structure but rest on single-configuration results that would benefit from broader replication (more seeds, more models, more benchmarks). The paper's most valuable contribution may be less the specific 1-shot result and more the diagnostic framework it introduces—format-reward baselines, entropy-loss-only training, per-example evaluation, and post-saturation tracking—that future RLVR research should adopt as standard practice regardless of dataset size.

6. Limitations and Trade-offs

The Cost of Finding an Effective Single Example is Unaccounted for and Potentially Dominates Training Cost

The paper's headline claim—that 1-shot RLVR matches full-dataset performance—depends on using a specific, carefully selected training example (e.g., $\pi_1$ or $\pi_{13}$). The selection procedure is not free. The paper's historical variance score (Equation 1) requires first running 500 steps of full-dataset RLVR on the 1,209-example DSR-sub to profile each example's per-epoch training accuracy, then computing the variance over those accuracies to rank examples. The paper acknowledges this explicitly (Section 3.1):

"we first train Qwen2.5-Math-1.5B for 500 steps, and then obtain its historical variance score ... and the corresponding ranking"

The consequence is that the cost of finding $\pi_1$ includes the cost of full-dataset RLVR itself: 500 steps × 128 batches × 8 responses = 512,000 model generations, plus gradient computation, across 8 GPUs. This is not amortized into any of the paper's efficiency calculations. A practitioner starting from scratch cannot simply "use 1-shot RLVR"—they must first run a substantial RLVR training run to identify which single example to use. The paper does not compare the total cost of (profiling + 1-shot RLVR) against simply running full-dataset RLVR for 2,000 steps and stopping.

The practical severity depends on how well the profiled ranking transfers across models. The paper tests this implicitly by reusing the Qwen2.5-Math-1.5B $\pi_i$ ranking for Qwen2.5-Math-7B, Llama-3.2-3B-Instruct, and DeepSeek-R1-Distill-Qwen-1.5B (Section 3.3, Table 4). The results suggest reasonable transfer: $\pi_1$ performs well on Qwen2.5-Math-7B (40.2% average vs. 42.8% DSR-sub) and Llama-3.2-3B-Instruct (19.0% vs. 19.8%). However, the gap is larger for DeepSeek-R1-Distill-Qwen-1.5B (46.3% vs. 48.6%) and Qwen2.5-1.5B (15.0% vs. 22.0%; Table 10), suggesting transfer is imperfect. More critically, the paper does not test whether profiling on a different model would surface different top-ranked examples that perform even better on the target model—a direct comparison that would quantify the transferability cost.

The paper does not attempt to mitigate this limitation through cheaper difficulty estimation methods. Section D.4 suggests future work on "better selection and collection of data for RLVR" but provides no concrete path toward eliminating the profiling overhead. The observation that many examples with moderate or low variance still produce substantial gains (Table 3: $\pi_{605}$ at rank 605 achieves 71.8% MATH500) hints that a simpler, cheaper selection criterion might exist—perhaps based directly on base model pass@1 on the candidate example, which can be estimated from a few hundred samples without any RLVR training—but this is not explored.

The Method Does Not Improve—and May Not Be Applicable to—Problems Outside the Base Model's Capability Range

The paper documents that 1-shot RLVR fails to produce meaningful improvement when the single training example is too difficult for the base model to solve. The clearest evidence comes from Table 3 (Section 3.2.3): $\pi_{1208}$, described as an "extremely difficult example," achieves only 45.0% MATH500 and 8.8% AIME24—far below other single examples and only modestly above the base model's 36.0%/6.7%. The paper notes that for this example and $\pi_{1207}$ (which has a wrong label), the model "(almost) never successfully samples the ground truth in training." The consequence is stark: if the base model's pass@1 on the training example is near zero, 1-shot RLVR provides essentially no improvement, because the policy gradient receives no positive reward signal to reinforce.

More broadly, Figure 2 (right panel) and the discussion in Section 3.2.2 show that even for full-dataset RLVR with 1,209 examples, test accuracy on the hardest problems (difficulty bin 5 in the MATH taxonomy, though the paper does not explicitly use this binning for all analyses) shows minimal improvement. The paper does not break down 1-shot RLVR performance by problem difficulty, so it is unknown whether the $\pi_1$-based improvement is concentrated on easy/medium problems (where the base model already had non-trivial pass@1) and negligible on the hardest problems. If so, 1-shot RLVR would inherit the same capability ceiling as full-dataset RLVR—it cannot create reasoning ability that was absent from pretraining—but would lack even the potential benefit that diverse training data might provide for covering a wider range of difficulty levels. The cross-category generalization results in Table 3 provide indirect evidence: all examples improve all categories, but the absolute scores remain lower on inherently harder categories (e.g., Intermediate Algebra and Geometry consistently score below Algebra and Prealgebra across all single examples).

The paper acknowledges this limitation implicitly through its framing of RLVR as "capability activation" rather than "capability creation" (Section 5, Appendix D.2), but does not explicitly test the boundary. A direct experiment—evaluating 1-shot RLVR on a held-out set of problems binned by base model pass@1—would clarify whether the gains are restricted to problems the base model could already sometimes solve. This is a critical practical consideration: if a deployment scenario involves genuinely novel or out-of-distribution reasoning tasks, 1-shot RLVR provides no path to solving them, and pretraining (or at minimum, RLVR on a diverse dataset covering the target distribution) remains necessary.

No mitigation is proposed. The paper's discussion of this limitation is confined to the general observation that "the reasoning capability of the model is already buried in some base models" (Section 5), implying that models lacking such buried capability would not benefit.

All Experiments Use a Single Mathematical Reasoning Benchmark Suite; Transfer to Code, Science, or General Reasoning is Unverified

Every quantitative result in the paper is evaluated on six mathematical reasoning benchmarks (MATH500, AIME2024, AIME2025, AMC2023, Minerva Math, OlympiadBench) plus two non-mathematical benchmarks (ARC-Easy, ARC-Challenge) used only in Table 1. The training data is drawn from the DeepScaleR-Preview-Dataset, which consists entirely of competition mathematics problems. The finding that $\pi_1$—an algebra problem set in a physics context—improves MATH500 by 37.6 percentage points establishes that 1-shot RLVR works for mathematical reasoning with verifiable rewards, but provides no evidence about its effectiveness for code generation (where verifiable rewards exist via unit tests), scientific reasoning (where rewards may be harder to define), or open-ended generation tasks (where binary outcome rewards are inapplicable).

The paper is transparent about this scope limitation. Appendix D.1 states:

"we mainly focus on the math domain, but haven't tried 1(few)-shot RLVR on other verifiable domains like coding. But we also emphasize that all math-related experiments and conclusions in our paper are logically self-contained and clearly recorded, to ensure clarity and avoid confusion for readers."

The consequence is that a practitioner interested in applying 1-shot RLVR to code generation, theorem proving, or multi-step planning cannot rely on this paper for guidance about whether the phenomenon transfers. Mathematics has specific properties—answers are typically short strings with unambiguous correctness, the reasoning structure is highly stereotyped (step-by-step algebraic manipulation), and the base models tested (Qwen2.5-Math variants) were explicitly pretrained on mathematical data—that may make it an unusually favorable domain for single-example RLVR. Code generation, for instance, involves longer outputs, more diverse reasoning patterns (algorithm design, API usage, debugging), and correctness signals that may be sparser (a program can be almost entirely correct but fail one edge case).

The paper provides one piece of cross-domain evidence: Table 1 shows that 1-shot RLVR with math examples improves ARC-Easy (48.0% → 55.8%) and ARC-Challenge (30.2% → 33.4%). These are multiple-choice science reasoning tasks, distinct from the math benchmarks used for training. However, the gains are modest (+7.8 and +3.2 percentage points respectively), and the paper notes that full-dataset DSR-sub RLVR actually degrades ARC performance (ARC-Easy drops from 48.0% to 42.2%, ARC-Challenge from 30.2% to 29.9%). This anomalous result—1-shot outperforms full-dataset on a non-math task—is reported without explanation and raises questions about whether the ARC improvement is a robust effect or a statistical artifact from a single training run. The paper evaluates only one checkpoint (the one with best average math performance) on ARC, which may not be optimal for non-math tasks.

No mitigation is attempted beyond the ARC evaluation, which is too limited (two benchmarks, one checkpoint, no cross-validation) to establish cross-domain generalization. The paper suggests future work on "other verifiable domains like coding" (Section D.4) but does not provide even preliminary results.

Checkpoint Selection Relies on Test-Set Optimization; No Held-Out Validation is Used, Risking Overstated Generalization

Throughout the paper, results are reported for the checkpoint that achieves the best performance on the test benchmarks themselves—either the best average across all six math benchmarks (Tables 8, 10) or the best performance on each benchmark independently (Tables 9, 11). The paper describes this explicitly (Appendix B.5):

"by default, we report the performance of the checkpoint that obtains the best average performance on 6 benchmarks."

There is no held-out validation set used for early stopping, hyperparameter selection, or checkpoint selection. The model is trained for a fixed number of steps (2,000 for Qwen2.5-Math-1.5B, 1,000 for Qwen2.5-Math-7B and Llama-3.2-3B-Instruct), checkpoints are saved every 20 steps, and after training completes, the checkpoint with the highest test accuracy is retrospectively identified and reported.

For large benchmarks like MATH500 (500 questions), this practice is unlikely to produce dramatically inflated estimates because the test set is large enough that the best-of-N-checkpoints selection bias is modest. However, for AIME2024, AIME2025, and AMC2023—with only 30, 30, and 40 questions respectively—test accuracy is substantially noisier. The paper attempts to mitigate this noise through repeated evaluation (8 passes with temperature 0.6), but this reduces variance in the estimate of a given checkpoint's performance, not the variance in which checkpoint is selected as best. With 100 checkpoints evaluated (2,000 steps / 20) and 30-40 question test sets, the best-observed AIME performance could easily be 2-4 percentage points above the true expected performance due to selection bias alone.

The consequence is that the reported numbers—particularly on AIME2024, AIME2025, and the 6-benchmark average—may overstate the true generalization performance of 1-shot RLVR. The gap between 1-shot and full-dataset RLVR is often small (0.9 points average on Qwen2.5-Math-1.5B in Table 8; 2.6 points on Qwen2.5-Math-7B in Table 10), and selection bias of even 1-2 points could change whether 1-shot "matches" full-dataset or falls measurably short. The paper's practice of reporting both best-average and best-per-benchmark results mitigates this concern partially—the best-average metric is less susceptible to benchmark-specific noise than best-per-benchmark—but does not eliminate the fundamental issue that the test set is used for model selection.

The paper does not address this limitation. There is no discussion of potential selection bias, no proposal to use a separate validation split, and no reporting of confidence intervals that would quantify the uncertainty in the best-checkpoint estimates. This is a methodological weakness shared with much of the RLVR literature (DeepScaleR, DAPO, and others report best-checkpoint results similarly), but it is particularly consequential for a paper whose central claim involves small performance differences between experimental conditions.

Training Dynamics and Final Performance Are Sensitive to Entropy Coefficient and Example Choice Without Clear Selection Principles

The paper demonstrates that 1-shot RLVR effectiveness varies dramatically with two factors that lack principled selection criteria: the choice of training example and the entropy loss coefficient.

Example choice sensitivity: Table 3 shows MATH500 performance ranging from 45.0% ($\pi_{1208}$) to 74.4% ($\pi_{13}$) across 15 single examples—a 29.4 percentage point spread. The worst-performing examples are identifiable as flawed in retrospect ($\pi_{1207}$ has a wrong label; $\pi_{1208}$ is too difficult), but even among "clean" examples, the range is 64.0% ($\pi_7$) to 74.4% ($\pi_{13}$), a 10.4 point gap. The historical variance ranking surfaces strong examples at the top ranks ($\pi_1$ at rank 1, $\pi_{13}$ at rank 13 both perform well) but does not perfectly order them: $\pi_{11}$ at rank 11 achieves only 64.0%, underperforming $\pi_{605}$ at rank 605 (71.8%) and $\pi_{1209}$ at rank 1209 (72.2%). The paper acknowledges this explicitly: "we emphasize that this criterion is not necessarily optimal for selecting single examples for 1-shot RLVR" (Section 2).

Entropy coefficient sensitivity: Table 5 (Rows 5-6) shows that increasing the entropy coefficient from $\alpha = -0.001$ to $\alpha = -0.003$ degrades MATH500 from 74.8% to 73.6% and AIME24 from 17.5% to 15.4%, with the paper noting the larger coefficient "might make the training more unstable." Figure 5 further shows that removing entropy loss entirely causes post-saturation performance to plateau, while the combination of entropy loss and temperature tuning (t=0.6 vs. t=1.0) yields an additional 0.8 percentage point gain. The optimal entropy coefficient likely interacts with the choice of training example, model architecture, and other hyperparameters, but this interaction space is not explored.

The consequence is that a practitioner cannot simply "apply 1-shot RLVR" and expect the paper's reported results. They must either (a) run the expensive historical variance profiling procedure and hope the top-ranked example works well, or (b) try multiple candidate examples and entropy coefficients, which multiplies the total compute cost by the number of candidates tested. Since the paper provides no predictive framework for what makes an example effective—beyond the post-hoc observation that the base model should be able to solve it with non-trivial probability and it should contain multi-step reasoning—example selection remains a trial-and-error process.

The paper partially mitigates the example choice limitation by demonstrating that many examples work reasonably well (most achieve 64-74% MATH500 in Table 3) and that the top-ranked examples by historical variance are reliably strong across multiple model families (Table 4). However, the entropy sensitivity is not mitigated—only two coefficient values are tested, the default $\alpha = -0.001$ is inherited from the verl framework without justification, and no scheduling or adaptive scheme is explored. The paper's suggestion that "encouraging the model to explore more diverse outputs within the solution space is critical" (Section D.4) points toward future work on better exploration strategies but does not resolve the coefficient sensitivity for current practitioners.

The Paper Does Not Establish Whether 1-Shot RLVR Genuinely Improves Reasoning Capability or Primarily Redistributes Probability Mass Toward Correct Answers the Base Model Already Generated

The paper frames its findings as evidence that RLVR "activates latent reasoning capability" in base models (Section 5, Appendix D.2). This interpretation rests on the observation that a single example provides essentially no new mathematical knowledge, yet produces dramatic performance gains. However, the paper does not provide the direct evidence needed to distinguish between two distinct mechanisms: (a) RLVR improves the model's reasoning process, enabling it to solve problems it previously could not, versus (b) RLVR redistributes probability mass toward correct solutions that already existed in the base model's output distribution at low probability, primarily by fixing formatting issues and reinforcing coherent chain-of-thought structure.

The paper provides partial evidence on both sides without resolving the question. Supporting mechanism (b)—redistribution rather than improvement—is the finding that format-reward RLVR alone accounts for approximately 76% of the total MATH500 gain on Qwen2.5-Math-1.5B (36.0% → 65.0% with format reward, vs. → 73.6% with outcome reward; Table 14). This means the majority of the "reasoning improvement" is actually format correction: the base model already knew many correct answers but failed to output them in a parseable format, and RLVR fixes this. The entropy-loss-only results (Table 13: 36.0% → 63.4% in 20 steps) further suggest that simply encouraging more diverse outputs—without any correctness signal—recovers most of the format-correction gain, because the model's peaked default distribution was suppressing well-formatted answers it could produce.

Supporting mechanism (a)—genuine improvement—is the finding that outcome-reward RLVR provides an additional +7.4% MATH500 beyond format reward (65.0% → 72.8%, Table 14), and that test accuracy continues improving after format ratios saturate (Figure 15: \boxed{} ratios plateau around step 200 but MATH500 accuracy climbs from ~70% to ~74% by step 1860). The increased frequency of self-reflection words during 1-shot RLVR (Figure 4, right panel) and the qualitative evidence of improved reasoning strategies in Figure 3 (the test problem solution evolves from incomplete to fully correct with a different approach) also suggest genuine improvement. The pass@8 evidence in Table 20 is the most direct test: if RLVR only redistributes probability mass, pass@8 should remain constant or decrease relative to the base model. The paper finds that 1-shot RLVR's pass@8 on three benchmarks (51.7%) exceeds the base model's (39.7%), suggesting new correct solutions are being generated.

However, this pass@8 evidence has significant limitations. It is reported on only three of six benchmarks (AIME24, AIME25, AMC23), uses the best checkpoint selected by test-set optimization (which may inflate the estimate), and is not compared against the format-reward baseline's pass@8 (which would establish how much of the pass@8 gain is format correction vs. reasoning improvement). The paper acknowledges the relevance of pass@k analysis by citing recent work (Yue et al., 2025) showing that RLVR can degrade pass@k while improving pass@1, but does not systematically evaluate this for 1-shot RLVR across all benchmarks and k values.

The consequence of this unresolved question is that the paper's central interpretive claim—that RLVR activates latent capability—remains a hypothesis rather than an established finding. A practitioner deciding whether to invest in 1-shot RLVR needs to know whether the gains reflect genuine capability improvement (which would transfer to new, unseen problem types) or primarily format correction + probability sharpening (which might not transfer beyond the types of problems the base model could already sometimes solve). The paper's ARC results (Table 1) provide a hint: 1-shot RLVR with math examples improves non-math reasoning, which is hard to explain purely through format correction (since ARC is multiple-choice and doesn't use \boxed{} formatting). But two benchmarks with a single checkpoint are insufficient to establish this as a robust phenomenon.

The paper does not attempt to mitigate this limitation through more comprehensive pass@k evaluation, probing of reasoning quality beyond accuracy, or analysis of whether 1-shot RLVR changes the model's representations versus its output distribution. The question is flagged as future work only indirectly, through the paper's call for "understanding 1-shot RLVR and post-saturation generalization" (Section D.4).

7. Implications and Future Directions

How This Work Changes the Landscape

This paper does not propose a new algorithm, a new architecture, or a new training objective. It proposes something more destabilizing: a single empirical observation that invalidates a widely held and rarely questioned assumption about what RLVR requires. The assumption—that diverse, large-scale training datasets are necessary for RLVR to produce meaningful reasoning improvements—underpins the data collection efforts behind DeepScaleR (40,000 problems), the MATH training set (7,500 problems), and the datasets used in DAPO, VAPO, and similar works. The paper demonstrates, with careful ablations and across multiple model families, that this assumption is false for models that already possess latent reasoning capabilities from pretraining. A single algebra problem with a physics backdrop (\pi_1, Table 2) can trigger MATH500 improvements from 36.0% to 73.6%—matching the 1,209-example DSR-sub dataset within 0.8 percentage points (Table 8).

This is not an incremental refinement of data selection methodology. It is a reframing of what RLVR does to a language model. The dominant mental model in the RLVR literature has treated RLVR as a knowledge-acquisition process: diverse training data teaches the model diverse reasoning patterns, and more data yields more learning. The 1-shot result forces a different interpretation: for base models with sufficient pretraining, RLVR functions primarily as a capability activator—it teaches the model to reliably deploy reasoning skills it already possesses, to structure outputs in verifier-friendly formats, and to commit to coherent multi-step reasoning chains rather than collapsing to shallow heuristics. The knowledge is already there; what RLVR adds is the behavioral commitment to use it.

The paper's decomposition of RLVR gains into three separable components—format correction, entropy-driven exploration, and correctness-guided reasoning improvement—provides a diagnostic framework that clarifies a confused literature. Prior work reporting RLVR gains on Qwen2.5-Math models (DeepScaleR, Dr. GRPO, and others) did not systematically separate these components. Dr. GRPO identified the template-sensitivity problem but did not quantify it. This paper quantifies it: on Qwen2.5-Math-1.5B, format correction alone accounts for approximately 76% of the total MATH500 gain (36.0% → 65.0% with format reward, vs. → 73.6% with outcome reward; Table 14). Entropy-loss-only training recovers most of this format-correction gain without any correctness signal (36.0% → 63.4% in 20 steps; Table 13). The remaining correctness-guided component (+7.4% MATH500 beyond format reward) is real but substantially smaller than the total gain would suggest if all improvement were attributed to "learning to reason better." This decomposition resolves the apparent contradiction between papers claiming large RLVR gains (which are real but largely format-correction) and papers questioning whether RLVR genuinely improves reasoning (which correctly identify that the reasoning-specific component is more modest). Both perspectives are partially right; the disagreement arose from conflating distinct sources of improvement.

The identification of post-saturation generalization as a phenomenon distinct from grokking shifts attention from regularization-based explanations of late-phase generalization toward exploration-based ones. Grokking, as characterized in the literature (Power et al., 2022; Nanda et al., 2023), depends critically on weight decay to slowly drive networks toward simpler solutions. This paper's ablation (Table 5, Rows 2-8) demonstrates that weight decay is inessential for 1-shot RLVR's post-saturation improvement, while policy gradient loss and entropy loss are essential. This is not merely a taxonomic distinction—it suggests that the mechanism of post-saturation generalization in RLVR is fundamentally different from the mechanism of grokking in supervised learning, and that future theoretical work on RLVR generalization should focus on the interaction between exploration incentives (entropy bonuses, temperature) and advantage-weighted policy gradients, rather than importing grokking's weight-decay-centric framework.

The paper also redirects research attention from data quantity to data quality in a way that is practically actionable. Table 3 shows that the choice of which single example to use produces a 29.4 percentage point spread in MATH500 performance (45.0% for \pi_{1208} vs. 74.4% for \pi_{13})—larger than the gap between 1-shot and full-dataset RLVR in most comparisons. This means that effort currently spent curating ever-larger RLVR datasets might be better spent identifying or constructing a small number of highly effective "template examples." The historical variance score provides one such selection criterion, but the paper's own results show it is imperfect (e.g., \pi_{605} at rank 605 outperforms \pi_{11} at rank 11). This opens a new research frontier: what properties make a single example maximally effective at activating latent reasoning? The paper provides preliminary candidates (base-model solvability, reasoning chain complexity, correctness of the label) but does not systematically test them, leaving the field with a clear set of hypotheses to evaluate.

The finding that 1-shot RLVR with math examples improves non-mathematical reasoning (ARC-Easy: 48.0% → 55.8%, ARC-Challenge: 30.2% → 33.4%; Table 1) while full-dataset RLVR degrades ARC performance—is a surprising cross-domain result that challenges assumptions about RLVR's specificity. If mathematical RLVR training transfers to science reasoning, then the behavioral template being learned (coherent multi-step reasoning, verification of intermediate claims) may be more domain-general than the mathematics-specific framing of most RLVR work suggests. The anomalous result that full-dataset RLVR worsens ARC performance while 1-shot RLVR improves it is reported without explanation, making it both a limitation of current understanding and a tantalizing clue about the conditions under which RLVR generalizes versus specializes.

Follow-Up Research This Work Enables

Cheap example quality estimation without full RLVR profiling. The paper's historical variance score requires 500 steps of full-dataset RLVR to compute—a profiling cost that rivals or exceeds the cost of the 1-shot RLVR training itself. A natural follow-up would develop a method to predict an example's 1-shot RLVR effectiveness directly from the base model's behavior, without any RLVR training. The paper's analysis of \pi_1 suggests candidate features: base-model pass@1 on the candidate example (estimated from 128-256 samples), the length and multi-step structure of the example's solution chain, and the diversity of the base model's outputs on that example (e.g., entropy of the final answer distribution). A strong follow-up would evaluate 50-100 candidate examples, compute these features from base-model samples alone, then run 1-shot RLVR on all of them and measure the correlation between each feature and eventual MATH500 performance. If base-model pass@1 correlates at r > 0.8 with 1-shot RLVR performance, the profiling step becomes unnecessary—a practitioner could select examples by sampling from the base model and picking one with intermediate pass@1 (say, 0.3-0.7) and a multi-step structure. This would make 1-shot RLVR genuinely practical by eliminating the circular dependency on full-dataset training.

Direct manipulation of example properties to isolate causal factors. The paper identifies several properties that correlate with 1-shot RLVR effectiveness—base-model solvability, reasoning chain length and complexity, label correctness—but the evidence is observational (Table 3 shows a range of examples with different properties). A factorial experiment would transform these correlations into causal claims. Design: construct synthetic training examples that independently vary (a) difficulty for the base model (measured by pass@1: easy ~0.8, medium ~0.5, hard ~0.1), (b) reasoning chain complexity (number of distinct algebraic steps: 1, 3, 5), and (c) label accuracy (correct, slightly wrong as in 12.7 vs. 12.8, substantially wrong as in "4," unachievable as in "929725"). A 3 × 3 × 3 design with 2 seeds per cell would require 54 training runs—feasible on 1.5B models—and would produce a quantitative model of how each property affects 1-shot RLVR outcomes. The paper's existing results predict that difficulty and label correctness will interact (wrong labels hurt more when the base model can overfit to them, as in Table 5, Rows 11-13), and that chain complexity matters beyond mere problem difficulty (as suggested by the \pi'_1 vs. \pi_1 comparison in Table 18). A factorial study would test these predictions directly.

Systematic pass@k evaluation across all benchmarks at multiple k values. The paper reports pass@8 on only three benchmarks (Table 20) and finds that 1-shot RLVR's pass@8 (51.7% at step 980) exceeds the base model's (39.7%), suggesting that RLVR generates solutions that were not present in the base distribution. This is critical for the "capability activation vs. capability creation" debate but is woefully under-sampled. A comprehensive follow-up would evaluate pass@{1, 4, 8, 16, 32, 64} for base model, format-reward RLVR, 1-shot outcome-reward RLVR, and full-dataset outcome-reward RLVR on all six benchmarks. If pass@k curves for 1-shot RLVR are consistently above those for the base model at high k (k ≥ 32), the "activation" narrative is incomplete—some genuine capability improvement is occurring. If pass@k curves converge at high k (1-shot RLVR and base model asymptote to the same accuracy as k → ∞), the gains are purely probability-mass redistribution. Recent work (Yue et al., 2025) suggests the latter may hold for full-dataset RLVR; whether 1-shot RLVR shows the same pattern is unknown. This experiment would also test whether the anomalous ARC result (full-dataset RLVR degrades ARC while 1-shot improves it) is robust or a single-seed artifact.

1-shot RLVR in code generation, theorem proving, and other verifiable domains. The paper's results are confined to mathematical reasoning, and the authors explicitly flag this as a limitation (Appendix D.1). A direct replication in code generation would use the same GRPO pipeline, replace the MATH benchmarks with HumanEval, MBPP, and LiveCodeBench, replace the training data pool with programming problems from APPS or CodeContests, and run the identical 1-shot vs. full-dataset comparison. Code generation shares key properties with math (verifiable outcomes via unit tests, multi-step reasoning, the existence of format-like effects with proper function signatures and output formatting) but differs in output length, reasoning structure, and the granularity of the reward signal (a program can be 90% correct and fail one test). A strong follow-up would report not just final accuracy but also the decomposition into format-correction (teaching the model to produce compilable code with proper function signatures) versus algorithmic improvement (teaching correct algorithm design). The paper's entropy-loss-only and format-reward baselines provide a ready-made template for this decomposition.

Combining 1-shot RLVR with better exploration strategies beyond entropy loss. The paper identifies entropy loss as critical for post-saturation generalization (Figure 5, Table 5) but also notes its coefficient sensitivity (\alpha = -0.001 works, \alpha = -0.003 destabilizes training). This suggests that the standard per-token entropy bonus is a crude tool for the exploration problem—it encourages diversity indiscriminately rather than targeting the specific exploration behaviors that generate useful training signal. Recent work on RLVR has explored more sophisticated exploration mechanisms, including dynamic temperature scheduling, curiosity-driven intrinsic rewards, and entropy-based data filtering (DAPO, VAPO). Testing these within the 1-shot RLVR framework would be informative: does a more targeted exploration mechanism (e.g., rewarding responses that are semantically different from previous attempts on the same problem, rather than merely having high per-token entropy) extend the post-saturation generalization phase or increase its slope? If so, it would both improve 1-shot RLVR's absolute performance and provide evidence about the mechanism—specifically, whether the benefit of exploration comes from generating any diverse outputs (which a flat entropy bonus provides) or from generating outputs that explore semantically distinct reasoning strategies (which a targeted bonus would better incentivize).

Identifying the mechanism of post-saturation generalization through interventional experiments. The paper proposes a specific mechanism for post-saturation generalization (Section D.3): entropy loss causes occasional errors → errors create non-zero advantages → policy gradient corrects errors → model learns more robust strategies. This mechanism is plausible but untested. A direct intervention would manipulate the error rate during post-saturation training: after training accuracy saturates (step 100 for \pi_1), artificially inject errors by forcing the model to occasionally produce incorrect answers (e.g., by temporarily increasing the rollout temperature to 2.0 on 20% of batches, or by adversarially perturbing the model's hidden states to increase the probability of known failure modes). If the proposed mechanism is correct, artificially increasing the error rate should accelerate post-saturation generalization (by providing more policy gradient signal). If it degrades performance instead, the mechanism is something else—perhaps the benefit comes from prolonged exposure to the single example's reasoning template rather than from error-correction dynamics. Conversely, one could suppress errors by using rejection sampling (only accepting correct responses for training), which should eliminate post-saturation generalization if the error-correction mechanism is correct. These interventional experiments would transform the paper's correlational evidence (entropy loss correlates with post-saturation improvement) into causal evidence, which is necessary for designing better training protocols.

Practical Applications and Downstream Use Cases

Rapid capability assessment for base models. The 1-shot RLVR framework provides a lightweight diagnostic for evaluating a base model's latent reasoning potential. Rather than running full-scale RLVR on thousands of examples to determine whether a model benefits from reinforcement learning, a practitioner could run 1-shot RLVR with 3-5 candidate examples (selected based on base-model pass@1 as described above) for 500-1000 steps and measure the resulting improvement. If none of the examples produce substantial gains, the base model likely lacks the latent reasoning capabilities that RLVR can activate, and resources should be directed toward pretraining or supervised fine-tuning instead. If even one example produces a large gain (e.g., +30 points on MATH500), the model is a good candidate for full RLVR, and the effective example identifies what kind of reasoning template the model needs. This use case is supported by the paper's finding that the same \pi_1 example transfers across Qwen2.5-Math-1.5B, Qwen2.5-Math-7B, and Llama-3.2-3B-Instruct (Table 4), suggesting that a small set of "probe examples" can serve as a reusable diagnostic across model families. The cost reduction is substantial: a 500-step 1-shot diagnostic run on a 1.5B model with 8 GPUs takes hours rather than days, compared to full-dataset RLVR.

Data curation prioritization for RLVR pipelines. The paper's demonstration that different examples produce dramatically different outcomes (Table 3: 45.0% to 74.4% MATH500) provides a concrete framework for evaluating data quality before committing to large-scale RLVR. Organizations building RLVR datasets (e.g., for internal reasoning models or domain-specific mathematical assistants) can adopt the following workflow: (1) curate a candidate pool of problems, (2) run 1-shot RLVR on a random sample of 20-30 candidates to estimate the distribution of per-example effectiveness, (3) prioritize examples that produce strong 1-shot results for inclusion in the final dataset, and (4) discard or downweight examples that perform poorly. This turns data curation from a "collect everything and hope" process into an empirically driven selection procedure. The paper's observation that many examples with moderate or low historical variance still produce substantial gains (Table 3: \pi_{605} at rank 605 achieves 71.8% MATH500) suggests that a large fraction of typical RLVR datasets may be "good enough"—but identifying and eliminating the genuinely harmful examples (those with wrong labels like \pi_{1207}, or those too difficult for the base model like \pi_{1208}) could improve dataset quality at minimal curation cost. The paper's label-noise experiment (Table 17: 90% wrong labels degrades performance below 1-shot with a single correct label) further motivates this approach: a dataset with many wrong answers is worse than a dataset with one right answer, so identifying and removing label errors is higher-value than adding more correctly labeled data.

Lightweight post-training for domain-specific reasoning models. For organizations deploying reasoning models in specialized domains (e.g., pharmaceutical dosage calculation, engineering stress analysis, financial risk modeling), the 1-shot RLVR framework offers a path to domain adaptation that does not require curating thousands of domain-specific problems. A practitioner could: (1) identify 5-10 high-quality, multi-step reasoning problems from the target domain that the base model can already sometimes solve, (2) run 1-shot RLVR on each individually for 500-1000 steps, (3) evaluate on a held-out domain benchmark, and (4) either deploy the best-performing single-example model or combine the top 2-3 examples for 2/3-shot RLVR. The paper's results suggest that 2-shot RLVR ({\pi_1, \pi_{13}}) can exceed single-example performance (Table 8: 36.6% average vs. 35.0% for \pi_1 alone) and even surpass full-dataset training on some metrics (MATH500: 76.0% for 2-shot vs. 75.2% for DSR-sub; Table 9). The total training cost for 2-shot RLVR is 2,000 steps—identical to the full-dataset training cost in terms of GPU-hours, but with the data curation cost reduced from thousands of examples to two. For a specialized domain where curating thousands of verified problem-answer pairs is expensive (requiring domain experts), this is a dramatic cost reduction. The caveat is that the examples must be well-chosen, which requires some domain expertise or the cheap diagnostic procedure described above.

When to Prefer This Method

The paper does not advocate for 1-shot RLVR as a generally superior alternative to full-dataset RLVR. It explicitly states that "1-shot RLVR doesn't save (and maybe requires more) RL computation" (Appendix D.1) and that "using few-shot RLVR may be more stable for training" (Appendix D.1). The contribution is an empirical observation and diagnostic framework, not a prescription. However, the results imply several conditions where extremely data-efficient RLVR is particularly attractive, and several where it is not:

  • Prefer 1-shot or few-shot RLVR when the base model already demonstrates non-trivial performance on the target task (pass@1 well above zero on a representative sample), since the paper shows that examples the base model cannot solve (\pi_{1208}) produce near-zero improvement. The base model must have latent reasoning capability to activate.
  • Prefer 1-shot or few-shot RLVR when data curation is the primary bottleneck—domains where verified problem-answer pairs are expensive to create (requiring expert annotators), scarce (rare languages, specialized scientific domains), or noisy (user-generated content with label errors). The paper's label-noise experiment (Table 17) shows that a single clean example can outperform a large dataset with many corrupted labels.
  • Prefer 1-shot or few-shot RLVR when the goal is rapid capability assessment or diagnostic evaluation rather than maximum absolute performance. For screening base models, testing data curation hypotheses, or evaluating RL algorithm variants, 1-shot RLVR provides a fast feedback loop that full-dataset training does not.
  • Prefer full-dataset RLVR when the base model's pass@1 on the target task distribution is near zero for a substantial fraction of problems, since 1-shot RLVR cannot create capability where none exists. The paper shows this for \pi_{1208} but does not evaluate whether full-dataset RLVR succeeds where 1-shot fails on the hardest problems—this is unknown and should be tested.
  • Prefer full-dataset RLVR when the highest possible absolute performance matters and small differences (2-5 percentage points) are consequential. The paper's best 1-shot and 2-shot results match or slightly exceed DSR-sub on Qwen2.5-Math-1.5B, but show small consistent gaps on Qwen2.5-Math-7B (40.2% vs. 42.8% average; Table 10) and larger gaps on Qwen2.5-1.5B (15.0% vs. 22.0%; Table 10). Whether these gaps are worth the data curation cost is a domain-specific engineering decision that the paper provides evidence for but does not resolve.