Training Methods2025~114 minarXiv:2511.06307

DRIVE: Data Curation Best Practices for Reinforcement Learning with Verifiable Reward in Competitive Code Generation

score 8Speed Zhu, Jianwei Cai, Guang Chen +3OpenAI↑ 53★ 7Nov 9, 2025cs.LGhttps://github.com/Tencent-Hunyuan/DRIVE-RLVR

🎯 Pitch

Training code models with reinforcement learning on uniformly distributed problem difficulties causes performance to stagnate on the hardest competitive programming tasks, even with continued scaling. This paper shows that a two-stage approach—first expanding exploration entropy on a broad set of problems, then applying a hard-focus curriculum with 64 rollouts per prompt on the most challenging instances—yields a 58.3% relative Codeforces improvement over standard RL, enabling a 32B model to rival systems 20× its size.

1. Executive Summary

This paper studies how to construct RL prompts and design curricula for reinforcement learning with verifiable rewards (RLVR) in competitive-programming code generation, using Qwen2.5-32B as the base model and evaluating on LeetCode and Codeforces weekly contests. The core contribution is a two-stage RL framework comprising an entropy expansion stage (training on a large, uniformly distributed set of ~9k problems with moderate rollout budgets to diversify outputs and reduce repetition) followed by a hard-focus curriculum called Pre-GRPO (training on a small, high-quality set of challenging problems with large rollout budgets — 64 rollouts per prompt — while continuously retaining the lowest-pass-rate instances). The method achieves state-of-the-art performance among 32B-parameter models, with a 58.3% relative improvement over the SFT baseline on Codeforces and results competitive with much larger systems such as DeepSeek V3.1, establishing that targeted data curation and difficulty-aware curriculum design are as critical as RL algorithm design — but only when hard problems receive sufficient rollout budgets and entropy expansion precedes the hard-focus phase, since standard RL training on uniformly distributed difficulties stagnates on the most challenging cases.

2. Context and Motivation

The Core Problem: RLVR Has Ignored Data Curation for Competitive Programming

The field of reinforcement learning with verifiable rewards (RLVR) has seen explosive growth since DeepSeek R1 (Guo et al., 2025) demonstrated that LLMs can develop sophisticated reasoning capabilities through RL training on tasks with automatically checkable correctness. However, this progress has been overwhelmingly concentrated in a single domain: mathematics, particularly the AIME benchmark. The paper's opening paragraph identifies this directly:

"Although a large body of research centers on math benchmarks such as AIME... applying these models to competitive programming—a domain demanding both deep algorithmic insight and precise implementation—remains relatively underexplored."

This is not merely a gap in coverage. Competitive programming poses qualitatively different challenges from mathematics:

Executability: Solutions must compile and run without errors. A logically sound algorithm expressed in broken code scores zero.
Efficiency constraints: Problems impose time and memory limits that require not just correct algorithms but efficient implementations — asymptotic complexity analysis matters in practice.
Edge case handling: Real test cases include corner cases that mathematical proofs often gloss over (empty inputs, boundary values, integer overflow).
Implementation precision: Off-by-one errors, incorrect data structure choices, and subtle language-specific behaviors all cause failures that have no analog in pure mathematical reasoning.

These differences mean that techniques optimized for AIME do not necessarily transfer. The paper argues this gap is "particularly notable" — the very capabilities that make reasoning models impressive (systematic, multi-step inference) are precisely what competitive programming demands, yet the field lacks a systematic understanding of how to apply RLVR to this domain.

Beyond the domain gap, the paper identifies a deeper problem in how RLVR research is conducted: the near-exclusive focus on RL algorithm design at the expense of data curation. The related work section (Section 2) catalogs a series of algorithmic innovations — DAPO (Yu et al., 2025), VAPO (Yue et al., 2025), ProRL (Liu et al., 2025), GSPO (Zheng et al., 2025) — all of which modify the RL optimization procedure. However:

"despite these algorithmic advances, remarkably little attention has been paid to the critical aspect of constructing suitable RL prompts to enhance RLVR performance—a gap our work addresses."

This is a significant observation because it points to an asymmetry in the field's attention. In supervised fine-tuning, data curation (filtering, difficulty weighting, diversity sampling) is recognized as central to performance. In RLVR, the default assumption has been that the RL algorithm itself will handle difficulty through its reward signal — hard problems provide informative gradients, easy problems provide stability, and the optimization process naturally balances them. This paper challenges that assumption directly.

Why This Problem Matters: Practical and Theoretical Stakes

Practical significance. Competitive programming is not an academic curiosity. The skills it tests — algorithmic reasoning, efficient implementation, systematic debugging — are directly transferable to software engineering, particularly in domains like systems programming, quantitative finance, and technical interview assessment. A model that excels at competitive programming can serve as an AI coding assistant capable of tackling genuinely novel algorithmic problems, not just retrieving memorized solutions from training data.

Moreover, competitive programming provides a uniquely clean testbed for RLVR. Problems have unambiguous correctness criteria (passes all hidden test cases or does not), making the reward signal perfectly reliable — no reward model approximation, no human preference ambiguity, no RLHF-style reward hacking. This makes it an ideal domain for isolating the effects of data curation and curriculum design from the confounding variable of reward quality. Insights from this domain could inform RLVR practice in noisier settings where reward signals are imperfect.

Theoretical significance. The paper's central finding — that standard RL training stagnates on hard problems while improving rapidly on easy and medium ones — touches on a fundamental question in reinforcement learning: does the optimization process naturally allocate its "learning budget" across problem difficulties in a desirable way? The evidence in Figure 5 suggests it does not. When the training set mixes difficulties uniformly, the model's accuracy on the hardest problems barely moves, while medium-difficulty problems surge ahead. This is not an artifact of insufficient data (the hard problems are present in the training set) or insufficient steps (training continues for many iterations). It is a property of how the GRPO algorithm distributes its updates across the problem distribution.

This finding has theoretical implications beyond competitive programming. If RLVR naturally underinvests in hard cases, then any RLVR system operating on a heterogeneous problem distribution will develop a capability ceiling — a level of difficulty beyond which the model cannot improve regardless of training duration. The paper's solution (a hard-focus curriculum that concentrates computational resources specifically on the most challenging instances) suggests a general principle: in RLVR, data scheduling may be as important as data content.

Where Prior Approaches Fall Short

The paper identifies specific limitations in existing work along three axes:

1. Uniform or naive difficulty treatment in RLVR training.

Most existing RLVR pipelines apply uniform training strategies across all problem difficulties. They sample problems from a fixed distribution, run RL for a fixed number of steps, and report aggregate metrics. The paper argues this is suboptimal for two reasons:

Easy problems dilute the training signal. When the model already achieves high accuracy on easy problems, further training on them provides minimal learning signal — the advantages (correct rollouts) dominate, so the policy gradient provides little information about which aspects of the solution were correct. The ablation in Table 4 (row 2: "RL with all 9k data") shows that training on a uniform mix of all problems yields substantially worse results than the two-stage approach, particularly on hard external benchmarks like Codeforces (0.102 vs. 0.182 for the full method).
Hard problems are systematically neglected. Figure 5 shows this directly: when training on the full LiveCodeV6 dataset (175 problems), cases with low initial accuracy (the hardest problems) show "stagnant progress" while medium-accuracy cases improve rapidly. The optimization process naturally gravitates toward problems where it can make quick progress, leaving the hard cases — precisely the ones that define competitive programming excellence — essentially untouched.

2. Insufficient rollout budgets for challenging problems.

A key empirical finding (Section 4.5, Appendix A) is that large rollout budgets are necessary for learning hard cases. When training on a single hard case (Figure 6), performance plateaus at low rollout counts but improves substantially when the number of rollouts per prompt increases. This matters because in standard GRPO training, the rollout budget is a fixed hyperparameter (often 8–16 rollouts) applied uniformly to all prompts. The paper shows this uniform allocation is wasteful: easy problems need few rollouts because the model's policy already covers the correct solution, while hard problems need many rollouts to have any chance of sampling a correct trajectory and receiving a positive reward signal. Without sufficiently many rollouts, the model never "sees" what a correct solution to a hard problem looks like during training, and therefore cannot learn to produce one.

This finding contradicts the implicit assumption in much RLVR work that the algorithm-design innovations (better advantage estimates, clipping strategies, KL penalties) are the primary levers for improvement. The paper argues that for hard problems, the bottleneck is not algorithmic sophistication but sampling budget — you simply need enough shots at the problem to occasionally succeed and propagate that signal.

3. The SFT→RL transition creates entropy collapse and repetition.

The paper identifies a specific pathology that arises when transitioning from supervised fine-tuning to RL training. After extensive SFT (Section 3.2, Figure 3), the model exhibits:

Low output entropy: The model converges to a narrow set of solution modes, repeatedly generating structurally similar responses regardless of the prompt.
Repetitive generation patterns: The model produces redundant code structures, falls into loops, and generates sequences so long they get truncated before completing.
Mode collapse: The distribution over solutions collapses around a few high-probability patterns, limiting the diversity the RL process can explore.

The authors document these patterns concretely in Appendix B, showing a case study where the model's reasoning process manifests extensive repetition: identical permutation-verification code blocks repeated dozens of times, formulaic edge-by-edge evaluation patterns, and cyclical "try permutation → calculate cost → express confusion → try next permutation" loops. These are not bugs — they are consequences of the SFT distribution being too narrow, leaving the RL process with insufficient exploration to discover novel solutions.

This is a distinct problem from the difficulty-imbalance issue. Even on easy problems, if the model's output distribution has collapsed, RL cannot improve because it cannot sample diverse candidate solutions to compare and learn from. The entropy expansion stage directly targets this problem.

4. Existing data curation work operates in different paradigms.

The paper acknowledges that data curation has been studied in related contexts, but argues these approaches don't address the specific challenges of RLVR for code generation:

RLHF data selection (Li et al., 2025a; Shen et al., 2025): These methods select which prompts to use for preference-based RLHF training. The objective is identifying prompts where the reward model signal is informative. This is fundamentally different from RLVR, where the reward is a deterministic function of correctness (pass/fail on test cases) — the challenge is not reward quality but problem difficulty scheduling.
DPO data filtering (Gao et al., 2025): This work shows that overly difficult examples hinder alignment in DPO training. The finding is superficially opposite to this paper's (hard examples are harmful vs. hard examples are essential), but the resolution is that the mechanism differs: in DPO with a learned reward model, hard examples produce noisy preference signals; in RLVR with executable verification, hard examples provide clean but rare reward signals that require large rollout budgets to capture.
Difficulty-aware rejection tuning (Tong et al., 2024): This approach for mathematical problem-solving uses model sampling to identify hard problems and then applies rejection sampling fine-tuning (RFT). The paper adopts the spirit of difficulty awareness but explicitly notes they "do not directly adopt their RFT method, as it requires extensive model sampling which is computationally expensive." This highlights a practical constraint: in competitive programming with 32B models, generating and evaluating thousands of samples per problem during data curation is prohibitively expensive, motivating more efficient alternatives.

How This Paper Positions Itself

The paper's framing is that data curation and curriculum design are first-class research problems in RLVR, not afterthoughts. This is a shift from the dominant narrative where algorithmic innovation (new PPO variants, better KL control, more stable training) is the primary source of progress. The paper does not claim that algorithms don't matter — they use GRPO, an existing algorithm — but rather that at the current state of the field, the data choices (which problems, in what order, with what sampling budget) dominate the algorithmic choices in determining final performance.

This positioning is supported by the ablation structure in Table 4 and Figure 4:

Training with all 175 LiveCodeV6 problems directly (uniform difficulty, standard setup) causes performance to collapse on LeetCode Weekly (0.296 vs. 0.578 for the SFT baseline).
Training with all 9k problems (Stage 1 only) improves over SFT but substantially underperforms the full two-stage method, particularly on hard benchmarks.
Skipping the entropy expansion stage and directly applying the hard-focus curriculum yields mixed results — improvements on the training distribution but degradation on out-of-distribution benchmarks.

These results collectively argue that the sequence of training stages, the difficulty distribution of training data, and the rollout budget per prompt are not hyperparameters to be tuned within a fixed paradigm but are themselves the architecture of effective RLVR training.

The paper also positions itself as providing scaling evidence beyond small-scale experiments. Section 4.6 reports results on an internal large-scale MoE model, showing that the two-stage strategy transfers to models with substantially more capacity. This addresses a common criticism of RLVR research: that techniques demonstrated on 7–32B dense models may not scale. The authors explicitly note that "most RLVR evaluations focus on dense 7–32B models, leaving open questions about scaling to much larger models" (Section 2), and position their MoE results as preliminary evidence that the principles (entropy expansion + hard-focus curriculum) are architecture-agnostic.

Finally, the paper implicitly positions itself against the common practice of evaluating on contaminated benchmarks. They construct evaluation sets from recent LeetCode and Codeforces weekly contests released after their training data collection cutoff, explicitly "to avoid data leakage" (Section 1, Section 4.1). This is a methodological choice that strengthens the credibility of their results: improvements on these held-out contests cannot be explained by memorization of training data. The consistent gains across both LiveCode benchmarks (some of which overlap with training) and the temporally separated contest benchmarks provide evidence that the learned capabilities are genuine rather than artifacts of data contamination.

3. Technical Approach

3.1 Reader Orientation

This paper presents a two-stage reinforcement learning training pipeline for competitive programming code generation, built on top of a supervised fine-tuned Qwen2.5-32B model. The system solves the problem that standard RLVR training — which treats all problems uniformly — stagnates on hard algorithmic challenges while over-investing compute in easy cases, by first diversifying the model's output distribution through entropy expansion on a broad problem set, then concentrating computational resources on the hardest problems through a curriculum that continuously retains low-pass-rate instances and allocates large per-prompt rollout budgets.

3.2 Big-Picture Architecture (Diagram in Words)

The system is a sequential pipeline with three major stages, each feeding into the next:

Supervised Fine-Tuning (SFT) with Hard-Example Oversampling: Starting from Qwen2.5-32B-Instruct, the model is trained on 470K distilled prompts from open-source models, with hard problems duplicated twice to allocate more learning budget to challenging cases. This produces a strong base policy but one with low output entropy and repetitive generation patterns.
RL Stage 1: Entropy Expansion: The SFT model is trained via GRPO on approximately 9,000 uniformly distributed competitive programming problems with 8 rollouts per prompt and a 24k-token context window. The goal is to diversify the model's output distribution, reduce mode collapse, and eliminate the repetition and truncation pathologies inherited from SFT. This stage uses a relatively short response-generation window to encourage concise reasoning.
RL Stage 2: Hard-Focus Curriculum (Pre-GRPO): The entropy-expanded model is trained on a small, high-quality set of challenging problems from LiveCode V6, filtered to retain only the hardest cases. Training proceeds through three phases of decreasing size but increasing difficulty (72 → 50 → 25 problems), with 64 rollouts per prompt and a 32k-token context window. A key mechanism called Pre-GRPO carries forward the lowest-pass-rate cases from each phase to the next, ensuring the model continuously faces its hardest remaining challenges.

During inference, the final RL model generates solutions with a 32k-token context window and is evaluated on held-out LeetCode and Codeforces weekly contests to prevent data leakage.

3.3 Roadmap for the Deep Dive

First, the SFT data construction and the Twice Hard Learning strategy — how the 1.27M raw prompts are distilled, filtered, and difficulty-weighted to produce the 470K-prompt training set, and why oversampling hard problems matters.
Second, the entropy expansion stage (RL Stage 1) — what GRPO is, how it is configured for entropy expansion, why 24k context and 8 rollouts are chosen, and what concrete problems this stage solves (low entropy, repetition, truncation).
Third, the Pre-GRPO mechanism and hard-focus curriculum (RL Stage 2) — how problems are filtered by pass rate, how the three-phase curriculum is structured, why 64 rollouts are necessary for hard problems, and how Pre-GRPO differs from standard training on all data.
Fourth, the RL reward signal and verifier design — how executable test cases provide deterministic rewards, what constitutes a correct solution, and how this clean reward signal interacts with the difficulty curriculum.
Fifth, the two-stage synergy — why entropy expansion must precede hard-focus curriculum, what happens when the order is reversed or a stage is omitted, and how the stages complement each other.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily an empirical systems paper whose core idea is that the optimal way to apply RLVR to competitive programming is to sequence training into an entropy expansion phase (broad, moderate-budget training to diversify outputs) followed by a hard-focus curriculum phase (narrow, high-budget training to master challenging problems), with careful attention to rollout budgets and difficulty filtering at each stage.

SFT Stage: Data Construction and the Twice Hard Learning Strategy

The pipeline begins with supervised fine-tuning on Qwen2.5-32B-Instruct, which serves as the base model. The SFT stage is not just about imparting coding knowledge — it establishes the initial policy distribution that RL will later reshape, making the data construction choices critical for downstream RL performance.

Raw data collection and distillation. The authors start by collecting 1.27 million open-source competitive programming prompts. For each prompt, they generate a corresponding response through distillation from DeepSeekR1-0528, a strong open-source reasoning model. This means the SFT targets are not human-written solutions but model-generated ones — the 32B model is learning to imitate a more capable system's output distribution. This is a standard distillation approach, but it introduces a distribution shift risk: the 32B model may not have the same reasoning capacity as the teacher, so simply memorizing the teacher's outputs does not guarantee the 32B model can generalize to novel problems.

Arena Learning for data refinement. To improve data quality beyond simple distillation, the authors apply a 5-round arena learning procedure (Luo et al., 2024). The mechanism works as follows:

Split the 1.27M prompts into five equal folds (roughly 254K prompts each).
Train an initial SFT model on the first fold.
Use this model to generate predictions on the second fold's prompts.
Retain only the prompts where the model failed — these are identified as "hard samples" for the current model.
Train the next model iteration on the combined hard samples from the previous round plus the new fold.
Repeat through all five folds.

The output is a condensed dataset of 470K prompts — roughly 37% of the original 1.27M — all of which were "hard" for at least one intermediate model in the arena learning sequence. This is a form of curriculum data filtering: rather than training on all available data, the process identifies which prompts actually provide a learning signal (i.e., the model cannot already solve them) and focuses training there.

The paper reports (Table 3) that the Arena Learning Strategy achieves comparable performance to the Basic SFT Strategy (training on all 1.27M prompts) despite using only 37% of the data. Specifically, on LiveCode 08-11, Arena Learning achieves 0.600 pass@1 versus 0.582 for Basic SFT — actually slightly better. This is a significant efficiency gain: removing easy, already-mastered prompts does not hurt performance because they provide negligible learning signal.

The Twice Hard Learning Strategy. The authors identify a limitation in the Arena Learning approach: although it successfully identifies hard prompts, it reduces the total training token budget because it discards 63% of the data. The model may not have enough exposure to the hard cases to fully learn them. The Twice Hard Learning Strategy addresses this by:

Identifying hard samples using the same arena learning procedure.
Duplicating each hard sample so it appears twice in the training data.
Keeping the overall training token budget comparable to the Basic SFT Strategy by filling the remaining capacity with the other (non-hard) samples from the 470K set.

This is conceptually simple but intentionally designed: it reallocates the fixed training compute budget away from easy samples (which the model already handles) and toward hard samples (which need more exposure). The paper explicitly frames this as ensuring "sufficient computational resources for the model to learn from them effectively" (Section 4.3).

The results in Table 3 validate this approach: Twice Hard Learning achieves 0.602 on LiveCode 08-11, 0.594 on LiveCode V5, and 0.549 on LiveCode V6 — the best or tied-for-best across all SFT variants. On Codeforces OJ, it reaches 0.115 versus 0.111 for Arena Learning and 0.112 for Basic SFT. The gains are modest but consistent.

Why difficulty weighting matters for downstream RL. A subtle but important point: the SFT stage establishes the initial policy that RL will optimize. If the SFT model has never seen enough hard problems during training, its initial policy on those problems will be essentially random — producing solutions that are almost always incorrect. When RL training begins, these hard problems will generate zero or near-zero reward signals because none of the rollouts are correct, and the policy gradient will provide no useful learning signal. By oversampling hard problems during SFT, the Twice Hard Learning Strategy increases the probability that the SFT model produces at least occasionally correct solutions on challenging cases, giving RL training a non-zero reward signal to work with.

The paper does not explicitly make this connection, but it follows directly from the finding in Section 4.5 that hard problems with near-zero initial accuracy show "stagnant progress" during RL (Figure 5). If SFT can push initial accuracy from 0% to even 5% on hard problems, that small foothold may be sufficient for RL to begin improving.

Additional data sources. Beyond competitive programming prompts, the authors augment the SFT data with "general-purpose coding data and reasoning-intensive problems" from other code generation tasks. They report that this cross-domain data "significantly improves performance on competitive programming tasks, likely due to enhanced reasoning and code comprehension abilities" (Section 3.1). The specific datasets and proportions are not enumerated, but the principle is clear: competitive programming requires general coding competence (syntax, standard libraries, common patterns) plus specific algorithmic reasoning — training on both types of data is better than training on either alone.

SFT training hyperparameters. The SFT model is trained for 3 epochs on the 470K prompts using 256 GPUs with a global batch size of 512 and a learning rate of 1 × 10⁻⁵. The base model is Qwen2.5-32B-Instruct. The 3-epoch training duration, combined with the difficulty-weighted data, aims to achieve sufficient exposure to hard cases while avoiding overfitting on the distilled responses.

The SFT model's limitations that motivate RL. After SFT, the model exhibits three specific pathologies that the RL stages are designed to address:

Low output entropy: The model converges to a narrow set of solution modes. Figure 3 (referenced in Section 3.2) shows an entropy comparison between "24k-style training" and "32k-style training" — the SFT model, having been trained on relatively compact solutions, produces low-diversity outputs. In RL terms, this means the policy distribution has low variance, which limits exploration — the model cannot discover better solutions because it rarely ventures far from its current high-probability outputs.
Repetitive generation patterns: Appendix B documents extensive repetition in the model's reasoning: identical code blocks repeated dozens of times, formulaic edge-by-edge analysis loops, and cyclical "try → fail → express confusion → try again" patterns. These are not simple loops (the model isn't stuck in an infinite loop) but rather inefficient exploration strategies — the model repeatedly tries minor variations of the same approach rather than considering qualitatively different algorithms.
Disproportionate weakness on hard problems: Table 1 shows that the SFT model's pass@10 on LeetCode Weekly (96.88%) is comparable to DeepSeek V3.1 and Seed1.6-0715, but on the harder Codeforces OJ, its pass@10 (24.24%) substantially lags both baselines (33.33% and 39.39%). This gap indicates that SFT alone, even with difficulty weighting, cannot close the performance gap on the most challenging algorithmic problems — these require the deeper reasoning capabilities that RL is expected to develop.

RL Stage 1: Entropy Expansion via GRPO on Broad Problem Distribution

The first RL stage addresses the low-entropy, repetitive-output pathology of the SFT model. The core insight is that training on a large, diverse set of problems with moderate compute per problem forces the model to explore a wider range of solution strategies, breaking the mode collapse inherited from SFT.

Algorithm: Group Relative Policy Optimization (GRPO). The paper uses GRPO (Guo et al., 2025; Yu et al., 2025) as the RL algorithm for both stages. GRPO is a variant of policy gradient methods designed for language model fine-tuning with verifiable rewards. While the paper does not provide the full GRPO mathematical formulation, the key operational characteristics relevant to this work are:

Group-based advantage estimation: For each prompt, the model generates multiple rollouts (complete solutions). The reward (pass/fail from test cases) is computed for each rollout. The advantage of each rollout is computed relative to the group mean reward for that prompt, rather than using a learned value function. This is analogous to REINFORCE with a baseline, where the baseline is the average performance on the same prompt, not a separately estimated value.
No critic model: Unlike PPO, GRPO does not train a separate value network. This reduces memory and compute requirements, which is important when working with 32B-parameter models.
Policy gradient with clipping: Similar to PPO, GRPO uses a clipped surrogate objective to prevent destructively large policy updates. The exact clipping parameters are not specified in the paper.

Why GRPO over alternatives? The paper does not explicitly justify choosing GRPO over PPO, DPO, or other RL algorithms. However, the implicit rationale is clear from the RLVR context: with executable test cases providing deterministic, noise-free rewards, there is no need for a learned reward model or a learned value function. GRPO's group-based advantage estimation is particularly well-suited because for any given prompt, the rewards across rollouts are directly comparable — a correct solution always scores higher than an incorrect one, with no ambiguity. The baseline (group mean) is unbiased and does not require training, eliminating a source of potential instability.

Training data: 9K uniformly distributed competitive programming problems. The first RL stage uses approximately 9,000 prompts from "open-source repositories." The paper describes these as "a mixed dataset" of competitive programming problems. The distribution is "uniformly distributed" — problems of all difficulty levels are represented roughly equally, without the difficulty filtering that characterizes Stage 2.

This is a deliberate contrast to the hard-focus curriculum: Stage 1 intentionally includes easy and medium problems alongside hard ones. The rationale is that the model needs diversity of experience to break out of its narrow SFT-induced output modes. Training only on hard problems at this stage would be counterproductive because (a) the model lacks the exploration capacity to solve them, producing only incorrect rollouts and thus zero reward signal, and (b) easy and medium problems provide "scaffolding" success experiences that maintain training stability.

Configuration: 8 rollouts per prompt, 24k context window, 32 training steps. The specific hyperparameters for Stage 1 are:

Rollouts per prompt: 8. This is a moderate budget — enough to sample several distinct candidate solutions per problem, but not so many as to be computationally prohibitive across 9,000 prompts. Eight rollouts provide sufficient within-group variance for GRPO's advantage estimation to be meaningful while keeping the total generation cost manageable.
Context window: 24k tokens total (prompt + response). This is notably shorter than the 32k context used in SFT and Stage 2 RL. The paper frames this as intentional: the shorter window encourages the model to produce more concise reasoning, which (a) increases effective throughput since more rollouts fit in the same total token budget, and (b) reduces the probability of the model wandering into repetitive, truncated outputs by limiting how far it can drift.
Training steps: 32 steps. The training duration is relatively short — enough to observe meaningful entropy expansion and performance improvement, but not so long as to overfit to the 9K problem distribution. The paper's ablation shows that training longer on this stage without transitioning to the hard-focus curriculum yields diminishing returns, particularly on hard benchmarks.

What entropy expansion accomplishes concretely. The paper identifies three specific benefits of Stage 1 (Section 3.2):

Entropy Enhancement: Training across a broad distribution of problems forces the policy to maintain diversity. Because different problems require different algorithmic approaches (dynamic programming, graph algorithms, number theory, greedy strategies), the model cannot succeed by specializing to a single reasoning template. The GRPO objective, which rewards correct solutions regardless of their structure, encourages the policy to cover multiple solution modes rather than collapsing to a single high-probability pattern.
Pattern Reduction: The paper reports that Stage 1 "significantly decreas[es] truncated errors" by reducing repetitive generation. The mechanism is straightforward: repetitive outputs that lead to truncation are inherently low-quality solutions that fail test cases. The RL process penalizes these outputs (they receive zero reward), while rewarding concise, correct solutions. Over 32 steps, the policy shifts probability mass away from the SFT model's repetitive patterns toward more diverse, effective reasoning strategies.
Overall Performance Improvement: The model's aggregate competitive programming capabilities improve during Stage 1, providing a stronger base policy for the hard-focus curriculum in Stage 2. Table 2 shows that the RL Stage 1 model (24k) improves over the SFT model on LiveCode V5 (0.627 vs. 0.594) and LiveCode V6 (0.634 vs. 0.549), though interestingly not on Codeforces OJ (0.112 vs. 0.115 — essentially flat).

The 24k vs. 32k tradeoff. Figure 3 (referenced in Section 3.2) compares the entropy of models trained with 24k and 32k context windows. The paper's choice of 24k for Stage 1 reflects a deliberate efficiency-diversity tradeoff. Longer contexts allow more elaborate reasoning chains, which can be necessary for solving very hard problems, but they also provide more "room" for the model to wander into repetitive patterns. The shorter 24k window acts as an implicit regularizer: by limiting how many tokens the model can generate, it forces the model to be more concise and reduces the probability of degenerate long-form outputs. In Stage 2, where the hardest problems genuinely require long reasoning chains, the context window is expanded to 32k — but only after the entropy expansion stage has already broken the model's tendency toward repetition.

The relationship to the SFT→RL transition problem. The entropy expansion stage directly addresses the common RLVR failure mode where the initial policy (from SFT) is too narrow to support effective exploration. In standard reinforcement learning, if the policy is deterministic or near-deterministic, the RL process has limited capacity to discover improved behaviors because it never samples them. By training on a broad problem set with moderate rollouts, Stage 1 intentionally "broadens" the policy distribution before the hard-focus curriculum begins, ensuring that when the model faces very challenging problems in Stage 2, it has a diverse enough output distribution to occasionally sample correct solutions and receive learning signals.

RL Stage 2: Hard-Focus Curriculum with Pre-GRPO

The second RL stage is the paper's primary technical contribution and the source of most performance gains. It addresses the finding from Section 4.5 that standard RL training "struggles with hard problems" — when all problems are trained uniformly, the optimization process allocates most of its learning budget to medium-difficulty cases where progress is rapid, leaving the hardest cases essentially untouched.

Core mechanism: Pre-GRPO (Pre-filtered GRPO). Pre-GRPO is a data selection procedure applied before each GRPO training phase. The name combines "Pre" (pre-filtering before training) with "GRPO" (the RL algorithm used). The procedure works as follows:

Before each training phase, evaluate the current model on all candidate problems by generating multiple rollouts and computing the pass rate (fraction of rollouts that are correct).
Sort problems by pass rate in ascending order (lowest pass rate = hardest problems).
Select the top-K hardest problems (where K depends on the phase) to form the training set for that phase.
Train the model via GRPO on only these selected problems for a fixed number of steps.
Before the next phase, re-evaluate pass rates on all problems (including those not selected in the previous phase) and select the new top-K hardest.
Repeat through all phases.

This is fundamentally different from standard RL training, which either uses a fixed training set throughout or samples problems uniformly from a pool. Pre-GRPO continuously re-evaluates difficulty and re-selects the hardest cases, ensuring that the model always trains on the problems at its current capability frontier.

The three-phase curriculum structure. Stage 2 uses a progressive curriculum with three phases, each with fewer but harder problems (Section 3.2):

Phase 1: 72 hardest cases, 64-step budget. The model trains on 72 problems for an unspecified number of steps (the paper reports total Stage 2 is 32k steps across all three phases, but the per-phase breakdown is not given). The large problem count at this stage ensures that the model is exposed to a diversity of hard algorithmic challenges, preventing overfitting to a narrow set of problem types.
Phase 2: 50 hardest cases, 32-step budget. After Phase 1, the model re-evaluates all problems and selects the 50 with the lowest pass rates. Some of these will be problems that were in Phase 1 but the model still struggles with; others may be new problems that became relatively harder as the model improved on other cases. The reduced problem count allows more focused training.
Phase 3: 25 hardest cases, 32-step budget. The final phase narrows to the 25 most difficult problems, concentrating the entire training budget on the hardest remaining challenges.

The decreasing problem count across phases reflects a focusing strategy: as training progresses and the model masters more problems, the training set is intentionally narrowed to concentrate computational resources on the stubbornly difficult cases that resist improvement. This is the opposite of a standard curriculum that gradually increases difficulty — here, the difficulty of the average training problem increases as the model improves, because the easy and medium cases are filtered out as they become mastered.

Why 64 rollouts per prompt? Stage 2 uses 64 rollouts per prompt, compared to 8 rollouts in Stage 1. This 8× increase in sampling budget per problem is a critical design choice that the paper justifies through the single-case training experiments in Section 4.5 and Appendix A. The logic is:

Hard problems have low probability of correct solutions under the current policy. If a problem has, say, a 2% pass rate under the current model, then with 8 rollouts the expected number of correct solutions is 0.16 — the model will almost never generate a correct answer, and the RL process will receive no positive reward signal. With 64 rollouts, the expected number of correct solutions is 1.28 — the model will occasionally (in expectation, at least once per training step) generate a correct answer and receive a positive reward.
The RL signal requires positive examples. GRPO's advantage estimation depends on within-group variance in rewards. If all 8 rollouts for a prompt are incorrect (reward = 0), the advantage for every rollout is 0 (since the group mean is 0), and the policy gradient is zero — no learning occurs. With 64 rollouts, even if only 1–2 are correct, the rewards have variance, and the advantages for correct rollouts are positive while incorrect rollouts are negative, providing a meaningful gradient.
Empirical evidence from Figure 6. The single-case training experiments show that higher rollout counts accelerate learning on hard cases. Even when training on just one problem in isolation, increasing the rollout count substantially improves learning speed. This is direct evidence that the bottleneck is sampling budget, not optimization algorithm design.

The computational cost of 64 rollouts is substantial: for a single prompt, generating 64 complete solutions at 32k context length consumes roughly 64 × 32k = 2 million tokens of generation, which at scale across 25–72 problems requires significant GPU resources. The paper acknowledges this cost implicitly by limiting Stage 2 to a small number of problems (25–72, not thousands) and a finite number of steps (32k total across all three phases). The design principle is concentrated compute on hard cases, trading breadth for depth.

Training hyperparameters for Stage 2. The paper specifies:

Total training steps: 32k steps across all three phases. The per-phase breakdown is not provided, but Phase 1 likely receives the majority since it has the most problems and the model needs more exposure early in the curriculum.
Context window: 32k tokens (prompt + response). This is longer than Stage 1's 24k, reflecting that hard problems require longer reasoning chains to solve. The risk of repetitive generation is lower at this stage because the entropy expansion in Stage 1 has already diversified the model's output patterns.
Rollouts per prompt: 64. As discussed, this large budget is essential for generating positive reward signals on hard problems.

Data source: LiveCode V6 with hard-case filtering. Stage 2 uses the LiveCode V6 dataset as its candidate pool. LiveCode V6 contains "175 high-quality examples with comprehensive test cases" (Section 4.1). The key property is that each problem comes with executable test cases, enabling the deterministic verifiable reward signal that RLVR requires.

Not all 175 problems are used in any single training phase. The Pre-GRPO filtering mechanism selects only the hardest subset in each phase. This means that easier LiveCode V6 problems (ones the model quickly masters) may never be used in Stage 2 training at all — they are filtered out because they no longer provide useful training signal.

How Pre-GRPO differs from standard difficulty-based sampling. A naive approach to difficulty-aware training would be to sample problems with probability proportional to some difficulty metric (e.g., inverse pass rate). Pre-GRPO differs in two important ways:

Hard cutoff, not soft weighting: Pre-GRPO uses a hard threshold — only the K hardest problems are trained on; all others are excluded entirely. This is more aggressive than soft weighting and reflects the paper's finding that easy problems can actively harm training when mixed with hard ones (as shown by the "RL with all LiveCodeV6 dataset" ablation in Table 4, where using all 175 problems caused performance to collapse to 0.296 on LeetCode Weekly, well below the SFT baseline of 0.578).
Dynamic re-evaluation between phases: Difficulty is not static — as the model improves on some hard problems, their pass rates increase, and they may be replaced in the next phase by previously easier problems that are now relatively harder. This dynamic re-ranking ensures the model is always training on its current capability frontier, not a historical one.

The interaction between context length and difficulty. The paper notes that "harder problems tend to require longer responses, which are harder to learn and generalize during SFT due to long-sequence dependency challenges." This observation connects the 24k → 32k context window increase to the difficulty escalation across stages: Stage 1 uses 24k context on a broad distribution of mostly medium-difficulty problems where long reasoning chains are less necessary; Stage 2 uses 32k context specifically because the hardest problems demand extended algorithmic reasoning that would be truncated at 24k. By delaying the longer context until after entropy expansion, the paper avoids the trap of giving the model more tokens to fill with repetitive content before it has learned to use those tokens efficiently.

Including additional hard cases beyond LiveCode V6. The ablation in Section 4.5 (Table 4, last row) shows that further augmenting Stage 2 with 109 hard cases curated from an internal dataset of 633 candidates yields additional improvements: LiveCode V6 rises from 0.703 to 0.743, LeetCode Weekly from 0.653 to 0.678, and Codeforces from 0.182 to 0.188. This suggests that the Pre-GRPO mechanism benefits from having a larger pool of candidate hard problems to select from — the more genuinely challenging problems available, the more effectively the curriculum can target the model's weaknesses.

The RL Reward Signal: Executable Test-Case Verification

Both RL stages rely on the same reward mechanism: executable test-case verification. This is what makes the framework "RLVR" (Reinforcement Learning with Verifiable Rewards) rather than RLHF (Reinforcement Learning from Human Feedback). The reward is not learned or approximated — it is a deterministic function of whether the generated code passes all hidden test cases.

What constitutes a correct solution? For competitive programming, a solution is correct if and only if:

The generated code compiles and runs without errors (for the relevant programming language, presumably Python given the model base but not explicitly stated).
The code produces the correct output for every test case — both the visible examples provided in the problem statement and the hidden test cases used for evaluation.
The code completes execution within the problem's time and memory limits.

The reward signal is therefore binary: 1 for a completely correct solution, 0 for anything else (compilation error, runtime error, wrong answer on any test case, time limit exceeded, memory limit exceeded). There are no partial rewards for "almost correct" solutions — an off-by-one error that causes failure on a single edge case produces the same zero reward as a completely nonsensical output.

Why this reward structure matters for training dynamics. The binary, all-or-nothing reward has important implications:

No reward shaping: Unlike many RL applications where the reward function can be shaped to provide incremental feedback (e.g., "you're getting closer"), test-case verification provides no gradient of partial correctness. The model either passes all tests or it fails. This means the RL process cannot gradually improve a solution from "mostly wrong" to "slightly wrong" to "correct" — it can only learn from the binary distinction between complete success and anything else.
Sparse rewards on hard problems: As discussed above, hard problems have low pass rates, meaning positive rewards are rare. This is the fundamental reason why large rollout budgets are necessary: without enough rollouts, the model may never encounter a positive reward on a hard problem, receiving zero learning signal regardless of how many training steps are taken.
Clean signal, no reward hacking: A major advantage of executable verification is that it is impossible to game. Unlike learned reward models (which can be exploited by generating high-scoring but incorrect outputs), test cases are ground truth — there is no difference between "the code passes the tests" and "the code is correct" because passing the tests is the definition of correctness in competitive programming. This eliminates the reward over-optimization concerns that plague RLHF and learned-verifier approaches.

The relationship between test cases and training data selection. The LiveCode V6 dataset used in Stage 2 is specifically chosen because it has "comprehensive test cases." This is critical: if the test cases are insufficient (e.g., only testing a few easy cases), the model can learn to produce solutions that pass the visible tests but fail on hidden edge cases — a form of "teaching to the test" that produces models that look good during training but fail in evaluation. The paper's use of datasets with comprehensive test suites, combined with evaluation on temporally separated contest benchmarks with independent test cases, mitigates this concern.

The Two-Stage Synergy: Why Order Matters

The paper's ablation studies provide strong evidence that the two stages are not merely additive — they are synergistic, and the ordering is critical.

What happens when Stage 1 is omitted. The "RL without First Stage (Second Stage Only)" ablation in Table 4 applies the Stage 2 hard-focus curriculum directly to the SFT model, skipping entropy expansion entirely. The results are mixed and revealing:

Improvement on training distribution: LiveCode V6 (the training data source for Stage 2) improves from 0.549 (SFT) to 0.691 — a 25.9% relative improvement. This shows that the hard-focus curriculum is effective even without entropy expansion.
Degradation on out-of-distribution benchmarks: LeetCode Weekly OJ drops from 0.578 (SFT) to 0.550 — a 4.8% decrease. The model becomes worse on problems it wasn't directly trained on.

This pattern — improvement on training data, degradation on held-out data — is a classic signature of overfitting. The model specializes to the specific hard problems in the Stage 2 curriculum but loses the general problem-solving capabilities that allowed it to handle novel problems. The entropy expansion stage prevents this by forcing the model to maintain a broad, diverse policy distribution before the hard-focus narrowing begins. Metaphorically: Stage 1 teaches the model to think flexibly about many types of problems; Stage 2 teaches it to apply that flexibility to the hardest cases. Without Stage 1, Stage 2 teaches the model to solve specific hard problems at the expense of flexibility.

What happens when only Stage 1 is used. The "RL with all 9k data" ablation applies only the entropy expansion stage (or its equivalent with 32k context), without transitioning to the hard-focus curriculum. The results show:

Modest improvements over SFT on LiveCode benchmarks: 0.676 (08-11), 0.688 (V5), 0.675 (V6) — versus 0.602, 0.594, 0.549 for SFT.
Very limited improvement on hard benchmarks: Codeforces OJ drops from 0.115 (SFT) to 0.102 — actually worse.

This confirms that Stage 1 alone cannot solve hard problems. The entropy expansion diversifies outputs and improves general capabilities, but the hard-focus curriculum is necessary to push the model's problem-solving frontier to genuinely challenging algorithmic problems.

What happens when all data is trained together (no curriculum). The "RL with all LiveCodeV6 dataset" ablation trains on all 175 LiveCode V6 problems uniformly, without difficulty filtering or staging. The results are catastrophic:

LiveCode V6: 0.549 (SFT) → 0.522 — degradation even on the training set.
LeetCode Weekly: 0.578 → 0.296 — a 48.8% collapse.
Codeforces OJ: 0.115 → 0.105 — slight decline.

This is the paper's most striking negative result: training on a uniform mix of easy and hard problems with a fixed rollout budget (8 rollouts, 24k context) is not just suboptimal — it is actively destructive. The paper attributes this to easy problems "diluting" the training signal and destabilizing optimization. Easy problems generate high reward rates (most rollouts are correct), which means the GRPO advantage estimates have low variance and the policy updates are dominated by noise rather than signal. When mixed with hard problems (which generate near-zero reward rates), the optimization process receives conflicting gradients — it is simultaneously being told that its current policy is good (on easy problems) and bad (on hard problems) — leading to training instability and eventual collapse.

The full two-stage method resolves these failure modes. The combination of entropy expansion (Stage 1) and hard-focus curriculum (Stage 2) achieves:

LiveCode V6: 0.549 → 0.703 (+28.1%)
LeetCode Weekly: 0.578 → 0.653 (+13.0%)
Codeforces OJ: 0.115 → 0.182 (+58.3%)

The improvements are consistent across all benchmarks, including the hardest held-out contests. The two stages work together because they solve different problems: Stage 1 creates a diverse, flexible policy that generalizes well; Stage 2 pushes that policy's frontier to handle increasingly difficult algorithmic challenges without sacrificing generality.

The Pre-GRPO mechanism as curriculum design. Pre-GRPO can be understood as an instance of competence-based curriculum learning: the model's current competence (measured by pass rate) determines which problems it trains on next. The specific design choices — hard cutoff rather than soft weighting, dynamic re-evaluation between phases, decreasing problem count across phases — reflect empirical optimization for the competitive programming domain. The paper does not claim these are universally optimal; rather, they are demonstrated to work well for this specific task and model scale.

Summary of Key Design Choices and Their Justifications

Two-stage RL over single-stage: A single stage cannot simultaneously address entropy collapse (which requires broad, moderate-budget training) and hard-problem mastery (which requires focused, high-budget training). The sequential design decouples these objectives.
GRPO over PPO or DPO: GRPO's group-based advantage estimation is well-suited to RLVR with deterministic rewards — no learned value function or reward model is needed, reducing complexity and potential instability.
8 rollouts (Stage 1) vs. 64 rollouts (Stage 2): Moderate rollouts suffice for entropy expansion on medium-difficulty problems; large rollouts are necessary for generating positive reward signals on hard problems where the pass rate under the current policy is very low.
24k context (Stage 1) vs. 32k context (Stage 2): Shorter context in Stage 1 acts as an implicit regularizer against repetitive generation; longer context in Stage 2 accommodates the extended reasoning chains required by hard algorithmic problems.
Pre-GRPO hard cutoff over soft weighting: Easy problems actively harm training when mixed with hard ones (as shown by the full-dataset ablation); hard exclusion is necessary, not just down-weighting.
Dynamic difficulty re-ranking over static assignment: Problem difficulty is relative to the current model; re-evaluating between phases ensures the curriculum tracks the model's evolving capability frontier.
Twice Hard Learning (SFT) over uniform sampling: Allocating more training tokens to hard problems during SFT provides a stronger initial policy for RL, increasing the probability that the model can occasionally generate correct solutions on challenging cases at the start of Stage 2.
Arena Learning for SFT data filtering over manual curation: Automated difficulty identification via model failure is more scalable and more tightly aligned with the specific model's weaknesses than human-labeled difficulty categories.
Temporally separated evaluation benchmarks over random splits: Using recent contest problems released after training data collection prevents data leakage from inflating performance estimates, providing more credible evidence of genuine capability improvements.

4. Key Insights and Innovations

Innovation 1: Difficulty-Aware Data Curation Is a First-Class Design Axis in RLVR, Not an Afterthought

The dominant paradigm in RLVR research has been algorithm-centric: the field's attention has focused on developing novel policy optimization procedures (DAPO, VAPO, ProRL, GSPO), stabilizing training dynamics, and improving advantage estimation. The implicit assumption has been that what you train on matters less than how you train — that the RL algorithm itself will handle difficulty through its reward signal, naturally allocating more learning to problems where the policy has room to improve.

This paper challenges that assumption at a fundamental level. The core intellectual move is to treat data curation and curriculum design as co-equal with algorithm design in determining RLVR outcomes. This is not a minor methodological preference — it is a reframing of where progress in RLVR comes from. The evidence for this reframing is stark: training on a uniform mix of easy and hard problems (the default approach in much prior work) not only underperforms the two-stage curriculum but actively causes training collapse (Table 4: "RL with all LiveCodeV6 dataset" drops LeetCode Weekly performance by 48.8% below the SFT baseline). This is not a modest efficiency gain — it is evidence that data distribution is a stability condition for RLVR training, not merely a performance knob.

What makes this contribution intellectually distinctive is its diagnostic character. The paper does not simply propose a better data mixing strategy and report improved numbers. It first diagnoses why uniform training fails: easy problems generate low-variance, uninformative advantage signals (most rollouts are correct, so the group mean is near 1, and advantages are near zero), while hard problems generate zero-variance, zero-signal updates (no rollouts are correct, so all advantages are zero). When these two regimes are mixed, the optimizer receives conflicting gradients that destabilize training. This diagnostic framework — identifying the mechanism of failure, not just its existence — enables principled solutions rather than ad-hoc tuning.

The contrast with prior work is instructive. Shen et al. (2025) and Li et al. (2025a) studied data selection for RLHF, where the challenge is identifying prompts that produce informative reward model signals. Gao et al. (2025) found that overly difficult examples hinder DPO alignment. These works operate in paradigms where the reward signal is learned and noisy — the data curation challenge is about signal quality. In RLVR with executable verification, the reward signal is perfectly clean (test cases are ground truth), yet the paper shows that data curation remains critical — but for entirely different reasons (difficulty scheduling and rollout budget sufficiency, not reward noise). This is a genuinely novel insight: even with noise-free rewards, what you train on and in what order dominates algorithmic choices.

The significance extends beyond competitive programming. Any RLVR task with a heterogeneous difficulty distribution — mathematical reasoning, theorem proving, code generation, planning — faces the same structural challenge. The paper's framework (diagnose the failure mode → design a staged curriculum targeting that failure → validate through ablation that both stages are necessary) provides a template for approaching these domains. It suggests that the field's algorithm-centric research program may be systematically missing a major source of variance in RLVR outcomes, and that future work should report data curation choices with the same rigor as optimizer hyperparameters.

This is a fundamental shift rather than an incremental refinement. It redefines what constitutes a complete RLVR method: not just an algorithm and a reward function, but a data curriculum with explicit difficulty scheduling and per-stage rollout budget allocation.

Innovation 2: The Entropy Expansion Stage as a Solution to the SFT→RL Mode Collapse Problem

A significant but under-discussed challenge in RLVR fine-tuning is that supervised fine-tuned models exhibit low output entropy and repetitive generation patterns that cripple RL's ability to explore. The paper provides concrete evidence of this pathology (Appendix B documents extensive repetition: identical code blocks repeated dozens of times, formulaic edge-by-edge analysis, cyclical "try-fail-confusion-retry" loops) and, more importantly, identifies it as a structural barrier to RLVR, not merely an annoyance.

Prior work has largely treated the SFT→RL transition as unproblematic — you train SFT first to establish a reasonable policy, then apply RL to improve it. The implicit assumption is that RL can always explore effectively from any reasonable starting distribution. This paper shows that assumption is false: when the SFT policy is too narrow, RL cannot discover improved behaviors because it never samples them. The mode collapse creates a self-reinforcing failure cycle: the model only generates a narrow range of (mostly incorrect) solutions → RL receives zero or low-variance rewards → the policy gradient provides minimal update signal → the model remains stuck in its narrow distribution.

The intellectual contribution is naming and characterizing this failure mode and then designing a training stage whose explicit purpose is to resolve it. The entropy expansion stage (Stage 1) is not about improving benchmark performance directly — the paper shows that Stage 1 alone yields only modest gains and can even hurt on hard benchmarks (Codeforces drops from 0.115 to 0.102). Its function is to broaden the policy distribution before the hard-focus curriculum begins, ensuring that when the model faces genuinely challenging problems in Stage 2, it has a diverse enough output distribution to occasionally sample correct solutions and receive learning signals.

This is conceptually analogous to exploration bonuses or entropy regularization in classical RL, but implemented through a deliberate training stage rather than an auxiliary loss term. The key insight is that on-policy entropy regularization (encouraging diversity during training) is insufficient to escape a collapsed distribution — you need a separate, dedicated phase operating on a different data distribution to restructure the policy's output modes before fine-grained optimization can begin.

The evidence for this innovation's necessity comes from the ablation where Stage 1 is omitted (Table 4, "RL without First Stage"). Without entropy expansion, the hard-focus curriculum improves training-set performance but degrades out-of-distribution performance — a signature of overfitting. The model learns to solve specific hard problems but loses the flexible reasoning capabilities needed for novel problems. Stage 1 prevents this by establishing a broad, generalizable policy foundation that Stage 2 can then sharpen.

This is a fundamental contribution to RLVR methodology. It identifies a previously unrecognized failure mode in the standard SFT→RL pipeline and provides a principled, empirically validated solution. The finding has implications beyond competitive programming: any RLVR application where the base policy (from SFT or pretraining) might be overly narrow should consider an explicit entropy expansion phase before targeted optimization.

Innovation 3: The Pre-GRPO Mechanism — Continuous Hard-Problem Retention as a Curriculum Design Principle

The paper's most operationally novel contribution is Pre-GRPO, a curriculum mechanism that continuously retains the hardest problems throughout training by dynamically re-ranking the candidate pool based on current pass rates and training only on the lowest-pass-rate instances. This is distinct from prior curriculum learning approaches in several fundamental ways.

First, Pre-GRPO uses a hard cutoff, not soft weighting. Most difficulty-aware training methods (including the paper's own SFT-stage Twice Hard Learning) sample hard problems more frequently or weight them more heavily in the loss. Pre-GRPO excludes easy and medium problems entirely from Stage 2 training. This is a stronger claim: it asserts not just that hard problems should receive more attention, but that easy problems are actively harmful when mixed with hard ones during RL. The empirical justification is the catastrophic collapse when training on all 175 LiveCode V6 problems uniformly (LeetCode Weekly drops from 0.578 to 0.296, a 48.8% decline).

Second, difficulty is continuously re-evaluated rather than statically assigned. In most curriculum learning work, problem difficulty is determined once (e.g., by human labels, by initial model performance) and fixed throughout training. Pre-GRPO re-evaluates pass rates between each phase, meaning that a problem can enter the training set if the model has mastered other problems and this one is now relatively harder, or leave the training set if the model has improved on it. This creates a moving capability frontier — the model always trains on exactly the problems at the boundary of its current competence, never wasting compute on problems it has already mastered or problems so far beyond its reach that no amount of training helps.

Third, Pre-GRPO operates with large rollout budgets per prompt (64 rollouts). This is not incidental — it is integral to the mechanism's logic. The high rollout budget ensures that even problems with very low pass rates (~1-2%) occasionally generate correct solutions, providing the positive reward signals that drive learning. Without sufficient rollouts, Pre-GRPO would simply identify hard problems but fail to improve on them, since the RL process would never receive a correct example to learn from. The single-case training experiments (Figure 6) provide direct evidence: learning speed on hard cases increases monotonically with rollout count.

The intellectual significance is that Pre-GRPO operationalizes a principle of concentrated computational resource allocation that runs counter to the field's default of uniform training. The standard approach — train on all available data with equal weight, let the optimizer figure out what to learn — implicitly assumes that gradient-based optimization naturally allocates learning across problem difficulties in a desirable way. The paper's Figure 5 shows this assumption is false: when trained uniformly, the model's accuracy on hard problems stagnates while medium problems surge ahead. Pre-GRPO is a mechanism for correcting this natural imbalance by forcibly redirecting the optimizer's attention to the hardest remaining cases.

This is a conceptual advance rather than a purely empirical one. Pre-GRPO is not just "a curriculum that works for competitive programming" — it embodies a general design principle: in RLVR with heterogeneous problem difficulty, data selection should be dynamic, exclusionary, and budget-aware. The specific parameters (three phases, 72→50→25 problems, 64 rollouts) are tuned to the domain, but the principle should transfer to other RLVR tasks where problem difficulty varies widely.

Innovation 4: The Rollout Budget as a Primary Bottleneck for Hard-Problem Learning in RLVR

A recurring theme in RLVR research is the search for algorithmic improvements that stabilize training or improve sample efficiency. This paper makes a counterintuitive empirical claim: for hard problems, the primary bottleneck is not algorithmic sophistication but sampling budget. Specifically, the number of rollouts per prompt during training must be large enough that the model occasionally generates correct solutions, or no learning can occur regardless of the optimizer design.

This finding is significant because it redirects attention from algorithm design to resource allocation. Prior work on RLVR for reasoning has largely used fixed rollout budgets (typically 8–16) across all prompts, focusing innovation on how to use those rollouts (better advantage estimates, clipping strategies, KL penalties). The paper shows that for hard problems, these algorithmic innovations are largely irrelevant if the rollout budget is insufficient — you simply cannot learn from problems you never solve. The single-case training experiments (Figure 6) are particularly compelling: even when training on a single hard problem in isolation, increasing the rollout count improves learning dramatically, with no change to the algorithm.

The intellectual contribution is identifying the sparsity-of-positive-reward problem as the fundamental mechanism limiting hard-problem progress in RLVR. This connects to well-known challenges in sparse-reward reinforcement learning, but with a domain-specific twist: in competitive programming, the reward function (executable test-case verification) is deterministic and noise-free, so the sparsity is not due to stochasticity or reward approximation error — it is purely due to the policy's low probability of generating correct solutions. This means the solution is straightforward in principle (sample more), but expensive in practice (64× more generation per prompt). The paper's willingness to pay this computational cost — by limiting Stage 2 to only 25–72 problems — represents a deliberate tradeoff of breadth for depth that is unusual in the RLVR literature.

The practical implication is that reporting rollout budgets per problem difficulty tier should become standard practice in RLVR research. A method that performs well with 8 rollouts on AIME-level math problems may fail completely on harder problems not because the algorithm is wrong but because the sampling budget is insufficient to generate any correct solutions. Without this reporting, cross-study comparisons conflate algorithmic quality with resource allocation choices.

This is an empirical insight with significant methodological implications rather than a theoretical advance. It does not propose a new algorithm or a new theory of RLVR training — it provides evidence that a specific, often-overlooked hyperparameter (rollout count) dominates other design choices for a critical subset of the problem distribution. The field's tendency to fix rollout counts and vary algorithms may systematically underestimate the importance of sampling budget in determining RLVR outcomes.

5. Experimental Analysis

Evaluation Methodology

Dataset. The paper uses three primary evaluation benchmarks: LiveCode08-11 (166 problems), LiveCodeV5 (167 problems), and LiveCodeV6 (175 problems), all derived from the LiveCodeBench competitive programming benchmark suite. Additionally, the authors construct evaluation sets from 32 recent LeetCode weekly contest problems and 33 recent Codeforces weekly contest problems, deliberately selected from contests released after the training data collection cutoff date "to ensure the integrity of our experimental results and avoid data leakage" (Section 4.1). LiveCodeV6 was exposed during the second RL stage, while LiveCode08-11 and LiveCodeV5 serve as uncontaminated validation benchmarks. The LeetCode and Codeforces contests provide the strongest test of generalization since they are temporally separated from all training data.
Base model(s). All main experiments use Qwen2.5-32B-Instruct as the foundation model. The paper states this model was chosen as representative of the capabilities available at this scale, and the resulting system achieves state-of-the-art performance among similarly sized models. For scaling analysis, the paper also reports results on an internal large-scale Mixture-of-Experts (MoE) model (Section 4.6), though specific parameter counts and architectural details for this model are not disclosed. The SFT stage distills from DeepSeekR1-0528, a strong open-source reasoning model, to generate training targets.
Metrics. The primary metric is pass@1 accuracy, computed as the fraction of problems for which the model's single generated solution passes all test cases. For the initial SFT model evaluation (Table 1), the paper additionally reports pass@10 (fraction of problems solved in at least one of 10 attempts) and avg@1 (the pass@1 averaged across problems). The paper does not explicitly define avg@1 versus pass@1 — based on Table 1, avg@1 appears to be the standard pass@1 metric, while pass@10 measures coverage across multiple attempts. All metrics are reported as decimals (e.g., 0.699) or percentages (e.g., 69.9%). For training dynamics analysis, the paper uses first-appearance accuracy — the model's accuracy on each problem during the first RL training step when that problem is evaluated — as a proxy for initial problem difficulty (Section 4.5, Figure 5).
Baselines. The paper compares against several external models of varying scales:
- DeepSeek V3.1 (Ta¸syürek et al., 2025): a large-scale model with 64k context length, serving as a strong general-purpose baseline.
- Seed1.6-0715 (ByteDance Seed et al., 2025): a reasoning-focused model with 64k context.
- Qwen3-235B-2507 (Yang et al., 2025): a 235B-parameter model with 64k context.
- QwQ-32B: a 32B-parameter reasoning model with 32k context, representing the prior 32B-scale state-of-the-art.
- OpenReasoning-Nemotron-32B: another 32B reasoning model with 32k context.
Internal baselines include the SFT model (post-supervised-fine-tuning, before any RL) and the RL Stage 1 model (after entropy expansion but before the hard-focus curriculum). These ablations allow the paper to attribute gains specifically to each RL stage.
Generation budget / compute accounting. The paper measures computational cost implicitly through several mechanisms. For RL training, the primary axes are rollout count per prompt (8 in Stage 1, 64 in Stage 2), context window length (24k tokens in Stage 1, 32k tokens in Stage 2), and number of training problems (9k in Stage 1, 25–72 in Stage 2). For evaluation, pass@1 uses a single generation per problem, while pass@10 uses 10 generations. The paper does not report total FLOPs or GPU-hours, and does not provide a unified cost metric that would enable direct efficiency comparisons between methods with different rollout budgets and problem counts. The training hyperparameters (256 GPUs for SFT, unspecified GPU count for RL; batch sizes of 512 for SFT, 512 for RL Stage 1, 128 for RL Stage 2) provide a partial picture of computational requirements but are not structured as a controlled cost-ablation.
Cross-validation / statistical protocol. The paper does not employ cross-validation or report confidence intervals. The evaluation benchmarks are fixed test sets (32 LeetCode problems, 33 Codeforces problems, 166–175 LiveCode problems). All comparisons are point estimates without statistical significance testing. The temporally separated contest benchmarks serve as a form of out-of-distribution evaluation but do not constitute cross-validation in the statistical sense. For the scaling experiments (Section 4.6), the authors note that "training was not continued to full convergence" due to computational constraints, indicating that the reported MoE results represent intermediate checkpoints rather than converged performance.

Main Quantitative Results

SFT Stage Results: Twice Hard Learning Improves Over Uniform Training

The paper evaluates three SFT strategies (Table 3) on the Qwen2.5-32B-Instruct base model, using the five evaluation benchmarks described above.

Twice Hard Learning achieves the best overall SFT performance. Across the three LiveCode benchmarks, Twice Hard Learning attains 0.602 (08-11), 0.594 (V5), and 0.549 (V6), compared to 0.582/0.603/0.545 for Basic SFT and 0.600/0.598/0.542 for Arena Learning. On the external contest benchmarks, it reaches 0.578 on LeetCode Weekly and 0.115 on Codeforces OJ, versus 0.558/0.112 for Basic SFT and 0.553/0.111 for Arena Learning. The gains over Basic SFT are modest but consistent: +3.4% relative on 08-11, +3.6% on LeetCode Weekly, +2.7% on Codeforces.

Arena Learning maintains performance with 63% less data. Despite reducing the training set from 1.27M prompts to 470K (a 63% reduction), Arena Learning achieves comparable or slightly better performance than Basic SFT on most benchmarks — 0.600 vs. 0.582 on LiveCode 08-11, and 0.598 vs. 0.603 on LiveCode V5. However, it slightly underperforms Basic SFT on LeetCode Weekly (0.553 vs. 0.558) and Codeforces (0.111 vs. 0.112). The paper attributes this to "an insufficient training token budget, which leads to incomplete learning during the SFT stage" (Section 4.3), motivating the Twice Hard Learning strategy that restores the token budget by oversampling hard examples.

The SFT model shows a sharp difficulty gradient. Table 1 reveals a striking pattern: the SFT model's pass@10 on LeetCode Weekly (96.88%) is competitive with DeepSeek V3.1 (96.88%) and Seed1.6-0715 (96.88%), but on the harder Codeforces OJ, its pass@10 (24.24%) substantially lags both DeepSeek V3.1 (33.33%) and Seed1.6-0715 (39.39%). This gap — 9–15 percentage points on Codeforces despite parity on LeetCode — demonstrates that SFT alone, even with the Twice Hard Learning strategy, cannot close the performance gap on the most challenging algorithmic problems. This observation directly motivates the two-stage RL design.

RL Stage Results: Two-Stage RL Delivers Substantial Gains Over SFT

Table 2 presents the paper's headline results, comparing the full RL model against the SFT baseline, the RL Stage 1 intermediate, and external models.

The full two-stage RL model achieves +13.0% to +58.3% relative improvement over SFT. On the five benchmarks, the RL model reaches:

LiveCode 08-11: 0.699 vs. 0.602 (SFT) → +16.1% relative improvement
LiveCode V5: 0.697 vs. 0.594 (SFT) → +17.3%
LiveCode V6: 0.703 vs. 0.549 (SFT) → +28.1%
LeetCode Weekly: 0.653 vs. 0.578 (SFT) → +13.0%
Codeforces OJ: 0.182 vs. 0.115 (SFT) → +58.3%

The largest relative gain occurs on Codeforces, the hardest benchmark, where the RL model nearly doubles the SFT model's accuracy. This supports the paper's central claim that the hard-focus curriculum specifically improves performance on challenging problems.

Stage 1 alone provides mixed results. The RL Stage 1 model (after entropy expansion, before hard-focus curriculum) achieves 0.625 on LiveCode 08-11, 0.627 on LiveCode V5, and 0.634 on LiveCode V6 — all improvements over SFT (0.602, 0.594, 0.549). However, on Codeforces OJ, Stage 1 performance (0.112) is essentially flat compared to SFT (0.115), and on LeetCode Weekly (0.603 vs. 0.578 SFT), the gain is modest. This pattern confirms that entropy expansion alone diversifies outputs and improves general capabilities but does not push the model's frontier to the hardest problems.

The full RL model is competitive with much larger models. Despite using only 32B parameters and a 32k context window, the RL model achieves 0.699 on LiveCode 08-11, comparable to DeepSeek V3.1 (0.692, 64k context), and 0.703 on LiveCode V6, exceeding DeepSeek V3.1 (0.693). On LiveCode V5, the RL model (0.697) is competitive with DeepSeek V3.1 (0.713) despite the latter's substantially larger scale. The RL model substantially outperforms Qwen3-235B-2507 (a 235B model) on LiveCode 08-11 (0.699 vs. 0.681) and LiveCode V6 (0.703 vs. 0.646), though it trails on Codeforces (0.182 vs. 0.200). Among 32B-scale models, the RL model dominates: +0.076 over OpenReasoning-Nemotron-32B on LiveCode 08-11, +0.079 on LiveCode V6, and +0.050 on Codeforces OJ.

LeetCode Weekly and Codeforces results from Table 1 reinforce the competitive positioning. The RL model achieves pass@10 of 96.88% and avg@1 of 65.31% on LeetCode Weekly (32 problems), compared to DeepSeek V3.1's 96.88%/68.75% and Seed1.6-0715's 96.88%/74.38%. On Codeforces OJ (33 problems), the RL model reaches pass@10 of 33.33% and avg@1 of 18.18%, versus DeepSeek V3.1's 33.33%/16.06% and Seed1.6-0715's 39.39%/18.79%. The RL model's pass@10 on Codeforces (33.33%) matches DeepSeek V3.1 — a model with substantially more parameters — and its avg@1 (18.18%) exceeds DeepSeek V3.1 (16.06%), though both trail Seed1.6-0715.

Ablation of RL Training Strategies: Both Stages Are Necessary

Table 4 presents the paper's primary ablation study, comparing four RL configurations against the SFT baseline and the full two-stage method.

Training on all LiveCode V6 without difficulty filtering causes catastrophic degradation. The "RL with all LiveCodeV6 dataset" configuration applies 24k-style RLVR using all 175 LiveCode V6 problems directly on the SFT model, without entropy expansion or difficulty filtering. The results are uniformly negative:

LiveCode 08-11: 0.602 (SFT) → 0.506 (−15.9%)
LiveCode V5: 0.594 → 0.512 (−13.8%)
LiveCode V6: 0.549 → 0.522 (−4.9%)
LeetCode Weekly: 0.578 → 0.296 (−48.8%)
Codeforces OJ: 0.115 → 0.105 (−8.7%)

The 48.8% collapse on LeetCode Weekly is the paper's most striking negative result and directly supports the claim that mixing easy and hard problems in uniform RL training is actively harmful, not merely suboptimal.

Training on all 9k data (Stage 1 only) improves on LiveCode but not on hard external benchmarks. The "RL with all 9k data" configuration trains on the full Stage 1 problem set without transitioning to the hard-focus curriculum. Results show:

LiveCode 08-11: 0.676 vs. 0.699 (full method) — 2.3 percentage points lower
LiveCode V5: 0.688 vs. 0.697 — 0.9 points lower
LiveCode V6: 0.675 vs. 0.703 — 2.8 points lower
LeetCode Weekly: 0.592 vs. 0.653 — 6.1 points lower
Codeforces OJ: 0.102 vs. 0.182 — 8.0 points lower

The gap is most pronounced on the hardest benchmarks (LeetCode Weekly and Codeforces), confirming that Stage 1 alone cannot master challenging problems — the hard-focus curriculum is essential for pushing the capability frontier.

Omitting Stage 1 and directly applying the hard-focus curriculum produces mixed results. The "RL without First Stage (Second Stage Only)" configuration bypasses entropy expansion entirely and applies the Stage 2 Pre-GRPO curriculum directly to the SFT model:

LiveCode V6: 0.549 (SFT) → 0.691 (+25.9%) — substantial improvement on the training distribution
LiveCode 08-11: 0.602 → 0.636 (+5.6%) — moderate improvement
LiveCode V5: 0.594 → 0.626 (+5.4%) — moderate improvement
LeetCode Weekly: 0.578 → 0.550 (−4.8%) — degradation
Codeforces OJ: 0.115 → 0.142 (+23.5%) — improvement, but far below the full method's 0.182

The degradation on LeetCode Weekly (−4.8%) while improving on the training distribution (LiveCode V6, +25.9%) is a clear signature of overfitting. Without the entropy expansion stage, the hard-focus curriculum causes the model to specialize to the specific hard problems in the training set at the expense of general problem-solving capabilities needed for novel problems.

The full two-stage method achieves the best results across all benchmarks. With improvements of +13.0% to +58.3% over SFT (as detailed above), the complete pipeline substantially outperforms every ablation configuration. The largest gaps between the full method and the next-best ablation are on LeetCode Weekly (0.653 vs. 0.592 for Stage 1 only) and Codeforces (0.182 vs. 0.142 for Stage 2 only), indicating that both stages contribute independently and synergistically to hard-problem performance.

Adding more hard cases to Stage 2 yields further improvements. The "Our Method with More Harder Samples in Stage 2" row in Table 4 shows that incorporating 109 additional hard cases from an internal dataset into Stage 2 produces:

LiveCode 08-11: 0.699 → 0.712 (+1.9%)
LiveCode V5: 0.697 → 0.707 (+1.4%)
LiveCode V6: 0.703 → 0.743 (+5.7%)
LeetCode Weekly: 0.653 → 0.678 (+3.8%)
Codeforces OJ: 0.182 → 0.188 (+3.3%)

These gains are modest but consistent, suggesting the Pre-GRPO mechanism benefits from a larger candidate pool of hard problems — more challenging instances provide more opportunities for the curriculum to target the model's specific weaknesses. The largest gain on LiveCode V6 (+5.7%) may partially reflect that the additional hard cases are drawn from a similar distribution to the LiveCode benchmarks.

Training Dynamics: Hard Problems Stagnate Under Uniform RL

Figure 5 (Section 4.5) analyzes training dynamics by clustering the 175 LiveCode V6 cases based on their initial rollout accuracy (evaluated with rollout=8) and tracking accuracy evolution for each cluster. The paper reports that "cases with medium initial accuracy demonstrate the most rapid improvement, substantially outperforming both low- and high-accuracy clusters." High-accuracy clusters show modest gains because "these problems are already largely solved." The critical finding concerns low-accuracy clusters: "the stagnant progress in low-accuracy clusters raises concerns about the model's ability to master challenging problems."

The paper argues that "standard RL training struggles with difficult cases, potentially creating a capability ceiling that limits the model's performance on complex problem-solving tasks." This ceiling is not a property of the model architecture or the RL algorithm per se, but rather an emergent property of the interaction between difficulty distribution and learning dynamics: the optimization process naturally gravitates toward problems where it can make rapid progress (medium difficulty), allocating less effective learning to problems where initial accuracy is very low because positive reward signals are too sparse. This finding directly motivates the hard-focus curriculum, which forcibly redirects attention to low-accuracy problems.

Single-Case and Few-Case Training: Large Rollouts Are Necessary for Hard Problems

The paper conducts additional experiments (Section 4.5, Appendix A) training on drastically reduced datasets — four cases and single cases — to isolate per-problem learning dynamics.

Hard cases resist learning even in isolation. When trained on four randomly selected cases from LiveCode V6 using GRPO with rollout=8, two cases demonstrate rapid convergence (accuracy surging from ~15-22% to ~80% within 60 steps), while the remaining two cases show minimal improvement: one rises from 12% to only 36.7%, the other "remains virtually frozen at 12.5% throughout training" (Appendix A). Even when a single hard case is trained in isolation for 60 steps, the model "struggled to reach satisfactory performance" (Section 4.5). This demonstrates that the learning difficulty is intrinsic to the problems, not an artifact of competition with easier problems for optimizer attention.

Positive transfer between problems exists but is limited. The paper observes that Case 1's learning trajectory is "noticeably slower" when trained in isolation compared to the four-case setting, providing "evidence for positive transfer effects — the simultaneous training on Cases 2, 3, and 4 appears to facilitate Case 1's learning" (Appendix A). However, even with this transfer benefit, Case 1 plateaus at 36.7% — far from mastery. Positive transfer helps but does not solve the fundamental challenge of learning hard problems.

Higher rollout counts accelerate learning on hard cases. Figure 6 (Section 4.5) shows the effect of varying rollout numbers on single-case training performance. The paper reports that "higher rollout counts accelerate learning" and makes two additional observations: (1) training on a single case has "negligible impact on the performance of other cases, including easy ones" — the learning is localized rather than disruptive; (2) "learning a hard case can improve generalization to other hard cases" — mastering one challenging problem transfers partially to other challenging problems. These findings support the Stage 2 design: large rollout budgets (64) are necessary for hard-problem learning, and focusing on hard cases does not degrade performance on easier problems.

Ablation Studies and Robustness Checks

SFT data strategy (Table 3): Twice Hard Learning outperforms both Basic SFT (full 1.27M dataset, uniform sampling) and Arena Learning (470K filtered hard samples) across all benchmarks. The key comparison is Twice Hard Learning versus Arena Learning: the former achieves 0.602 vs. 0.600 on LiveCode 08-11, 0.594 vs. 0.598 on LiveCode V5, 0.549 vs. 0.542 on LiveCode V6, 0.578 vs. 0.553 on LeetCode Weekly, and 0.115 vs. 0.111 on Codeforces OJ. The consistent advantage — particularly on external benchmarks (LeetCode +4.5%, Codeforces +3.6%) — validates the hypothesis that hard examples require sufficient token budget for effective learning, not just identification. Arena Learning identifies the right data but allocates insufficient learning budget; Twice Hard Learning corrects this.

RL training data difficulty distribution (Table 4, Figure 4): The comparison between "RL with all LiveCodeV6 dataset" (uniform difficulty, 175 problems), "RL with all 9k data" (Stage 1 only, moderate difficulty), "RL without First Stage" (Stage 2 only, hard problems), and the full method (Stage 1 + Stage 2) provides a systematic ablation of difficulty scheduling. The key non-obvious finding is the interaction between difficulty filtering and training stability: the uniform-difficulty configuration (all 175 problems) causes catastrophic collapse (LeetCode Weekly: −48.8%), while both difficulty-filtered configurations (9k only, Stage 2 only) improve over SFT without collapsing. This indicates that the mere presence of easy problems in RL training — not the specific difficulty distribution or the total problem count — is the destabilizing factor. The full method's superiority over both partial configurations demonstrates that entropy expansion and hard-focus curriculum provide complementary benefits that neither achieves alone.

Stage ordering (Table 4, "RL without First Stage" vs. full method): The comparison between Stage 2 only and the full two-stage method directly tests whether the ordering matters. Stage 2 only improves LiveCode V6 by 25.9% but degrades LeetCode Weekly by 4.8%, while the full method improves both (+28.1% and +13.0%, respectively). This asymmetry — training-distribution improvement with out-of-distribution degradation — is a strong signature of overfitting that the entropy expansion stage prevents. The paper does not test the reverse ordering (Stage 2 before Stage 1), which would be informative but is not reported.

Number of hard cases in Stage 2 (Table 4, last row): Adding 109 more hard cases to Stage 2 improves performance across all benchmarks (+1.4% to +5.7% relative), with the largest gain on LiveCode V6 (+5.7%). This suggests the Pre-GRPO mechanism's effectiveness scales with the diversity of available hard problems — a larger candidate pool allows more precise targeting of the model's weaknesses. However, the experiment conflates two changes (more hard problems AND those problems are from a different internal dataset), so the improvement cannot be attributed solely to increased problem count.

Rollout budget in single-case training (Figure 6): Higher rollout counts monotonically improve learning speed on hard cases. This is a critical robustness check because it isolates the effect of rollout budget from all other variables (problem difficulty, curriculum design, dataset composition). The finding that learning acceleration with rollout count is smooth and monotonic strengthens the paper's argument that sampling budget, not algorithmic design, is the primary bottleneck for hard-problem learning.

Scaling to MoE architecture (Table 5): The two-stage strategy transfers to a large-scale internal MoE model. After Stage 1, the MoE model shows +1.32% on LiveCode 08-11 and +6.94% on LiveCode V5, but −2.55% on LeetCode Weekly and −20.65% on Codeforces OJ — a pattern of regressing on hard benchmarks that mirrors the dense model's Stage-1-only results. After Stage 2, the MoE model achieves +3.96% (08-11), +7.51% (V5), +12.35% (V6), +15.17% (LeetCode Weekly), and +25.16% (Codeforces) relative to its SFT baseline. The large gain on Codeforces (+25.16%) and the recovery from Stage 1's regression on LeetCode Weekly (−2.55% → +15.17%) demonstrate that the hard-focus curriculum's value is robust across model architectures and scales. However, the authors note training was "constrained by computational resources, and training was not continued to full convergence," meaning these results represent lower bounds on what the approach could achieve with sufficient compute at MoE scale.

Generalization from training to held-out benchmarks: The paper evaluates on three categories of benchmarks with varying degrees of overlap with training data. LiveCodeV6 was used in Stage 2 training and shows the largest RL gains (+28.1%), which could partially reflect training-set overfitting. LiveCode08-11 and LiveCodeV5 were not used in any RL stage but are from the same LiveCodeBench distribution; they show substantial but smaller gains (+16.1% and +17.3%). The LeetCode and Codeforces weekly contests are temporally separated from all training data and show gains of +13.0% and +58.3% respectively. The consistent improvement across all three categories, including the fully held-out contest benchmarks, is the strongest evidence that the learned capabilities are genuine rather than memorized. The magnitude gap between training-distribution gains (+28.1%) and held-out gains (+13.0% to +17.3% on LiveCode, +13.0% on LeetCode) is expected but modest, suggesting limited overfitting to the Stage 2 training problems.

Critical Assessment

Claim 1: "A two-stage RL framework that systematically addresses the limitations of normal RLVR methods through entropy expansion followed by hard-focus curriculum learning."

The experiments provide strong evidence that the two stages serve distinct and complementary functions. The ablation where Stage 1 is omitted (Table 4, "RL without First Stage") demonstrates that the hard-focus curriculum alone causes overfitting (LeetCode Weekly degrades 4.8% below SFT), which is resolved by adding the entropy expansion stage. The ablation where Stage 2 is omitted ("RL with all 9k data") demonstrates that entropy expansion alone cannot improve hard-problem performance (Codeforces drops from 0.115 to 0.102). The full method achieves the best results across all benchmarks.

However, there are important limitations to what these experiments demonstrate about "systematically addressing limitations." First, the paper does not directly measure entropy or output diversity — it infers entropy expansion from performance patterns and the qualitative case study in Appendix B. A direct measurement of policy entropy before and after Stage 1 would strengthen this claim considerably. Second, the paper does not isolate which aspect of Stage 1 (the 9k problem distribution, the 24k context length, the 8 rollouts, or the 32 training steps) is most responsible for the entropy expansion effect. If the critical factor is simply training on a diverse problem set, then other configurations might achieve the same effect more efficiently. Third, the paper does not test whether the entropy expansion could be achieved through other means — for example, adding an entropy bonus to the RL objective during a single-stage curriculum, or using a different SFT data mixture that produces higher initial entropy. Without these comparisons, the claim that a separate, dedicated stage is necessary for entropy expansion is supported by the ablation but not by direct alternatives.

Claim 2: "Large rollout budgets are crucial for learning challenging problems, while moderate budgets suffice for entropy expansion."

The single-case training experiments (Figure 6) provide direct causal evidence that higher rollout counts accelerate learning on hard problems. The comparison between Stage 1 (8 rollouts on 9k problems) and Stage 2 (64 rollouts on 25–72 problems) demonstrates the practical implementation of this principle. However, the evidence for the second half of the claim — that "moderate budgets suffice for entropy expansion" — is indirect. The paper does not run an ablation where Stage 1 uses different rollout counts (e.g., 4, 8, 16, 32) and measure the resulting entropy or downstream Stage 2 performance. The choice of 8 rollouts for Stage 1 is stated as part of the configuration but never justified through comparison with alternatives. It is possible that 4 rollouts would suffice, or that 16 rollouts would yield meaningfully better entropy expansion and stronger downstream results. The paper's finding that rollout budget matters for hard problems is well-supported; the specific claim that "moderate budgets suffice" for entropy expansion is asserted rather than demonstrated.

Additionally, the single-case training experiments that establish the value of large rollouts (Figure 6, Appendix A) are conducted on a very small number of cases. The paper trains on four cases and single cases — findings from such limited experiments may not generalize to the full 25–72 problem Stage 2 setting. The per-problem dynamics (particularly the plateau at 36.7% for Case 1 after 60 steps) suggest that even large rollouts have limits for certain problems.

Claim 3: "Standard RL struggles with difficult cases, motivating our curriculum-based approach."

Figure 5 provides direct evidence for this claim: when training uniformly on all 175 LiveCode V6 problems, cases with low initial accuracy show "stagnant progress" while medium-accuracy cases improve rapidly. The catastrophic collapse when training on all LiveCode V6 problems uniformly (Table 4, row 2) further demonstrates that standard RL training not only fails on hard problems but can actively degrade overall performance when easy and hard problems are mixed.

However, this evidence is limited to a specific configuration: 8 rollouts, 24k context, GRPO algorithm, LiveCode V6 dataset. The paper does not investigate whether other "standard RL" configurations — for example, using more rollouts, a different algorithm (PPO, REINFORCE), or a different reward normalization scheme — would also exhibit this failure mode. The finding that 64 rollouts are necessary for hard-problem learning (Figure 6) suggests an alternative interpretation: "standard RL" may not inherently struggle with difficult cases, but rather the standard rollout budget (8) is insufficient for hard problems. If the all-LiveCodeV6 experiment were run with 64 rollouts instead of 8, would the performance collapse still occur? The paper does not test this. The claim that difficulty-based curriculum is necessary (rather than simply increasing rollout budget uniformly) is therefore supported against the specific baseline of uniform 8-rollout training, but not against the alternative of uniform 64-rollout training.

Claim 4: "Our model achieves state-of-the-art performance among 32B parameter models."

This claim is well-supported by the comparisons in Table 2. The DRIVE model (0.699 on LiveCode 08-11, 0.697 on V5, 0.703 on V6) substantially outperforms QwQ-32B (0.578, 0.569, 0.537) and OpenReasoning-Nemotron-32B (0.623, 0.618, 0.600). On Codeforces, the DRIVE model's 0.182 vs. 0.124 (QwQ-32B) and 0.132 (OpenReasoning-Nemotron-32B) represents a 37.9% to 47.6% relative advantage. The claim is accurate for the models compared.

However, the claim applies specifically to the benchmarks reported. The paper does not evaluate on standard competitive programming benchmarks like CodeContests, APPS, or USACO, which would provide additional evidence of state-of-the-art status. The LiveCode and contest benchmarks are reasonable choices, but "state-of-the-art" is benchmark-dependent. Additionally, the paper does not compare against all 32B-scale models that might exist at the time of writing — the comparison set is limited to QwQ-32B and OpenReasoning-Nemotron-32B.

Claim 5: "Our model achieves performance comparable to much larger systems such as DeepSeek V3.1."

This claim requires careful qualification. On LiveCode benchmarks, the DRIVE model is indeed comparable: 0.699 vs. 0.692 (08-11), 0.697 vs. 0.713 (V5), 0.703 vs. 0.693 (V6). These are genuinely impressive results for a 32B model with 32k context competing against a much larger model with 64k context. On LeetCode Weekly, the DRIVE model's 0.653 trails DeepSeek V3.1's 0.688 by 3.5 percentage points — still in the same performance tier. On Codeforces, the DRIVE model's 0.182 exceeds DeepSeek V3.1's 0.161 by 2.1 percentage points.

However, the claim as stated ("comparable to much larger systems") oversimplifies a more nuanced picture. The DRIVE model trails Seed1.6-0715 on every benchmark: 0.699 vs. 0.803 (08-11), 0.697 vs. 0.824 (V5), 0.703 vs. 0.770 (V6), 0.653 vs. 0.743 (LeetCode), 0.182 vs. 0.188 (Codeforces). The gaps range from 0.6 to 12.7 percentage points. Since Seed1.6-0715 is also a "much larger system," the claim of comparability is valid for DeepSeek V3.1 but not for all larger systems. Furthermore, both DeepSeek V3.1 and Seed1.6-0715 are general-purpose models evaluated zero-shot on competitive programming; the DRIVE model is specifically trained for this task. A fairer comparison would be against larger models that have also undergone competitive-programming-specific RLVR training.

Missing experiments that would strengthen the paper:

Varying the Stage 1 rollout budget. The paper claims 8 rollouts suffice for entropy expansion but never tests 4, 16, or 32. This matters because Stage 1's cost is proportional to rollout count × problem count (8 × 9k = 72k effective generations per step × 32 steps = ~2.3M generations total), which is substantial. If 4 rollouts achieved similar entropy expansion, significant compute could be saved.
Direct measurement of output entropy. The paper diagnoses low entropy as the SFT model's key limitation but the only evidence is qualitative (Appendix B case study) and indirect (Figure 3, which is referenced but not fully described). Measuring the policy's token-level or semantic-level entropy before and after Stage 1 would make the diagnosis-test-treat logic much stronger.
Uniform 64-rollout training on all LiveCode V6. The catastrophic collapse when training uniformly with 8 rollouts (Table 4) is the paper's strongest evidence that difficulty filtering is necessary. But if the same experiment with 64 rollouts did not collapse, it would suggest the problem is insufficient rollout budget, not difficulty mixing. Without this ablation, the paper cannot distinguish between "easy problems cause instability" and "insufficient rollouts cause instability on the hard subset of a mixed-difficulty dataset."
Comparison against a strong 32B baseline trained with a different RLVR recipe. All external baselines (QwQ-32B, OpenReasoning-Nemotron-32B) use different base models, different SFT data, and different RL procedures. A controlled comparison where the same Qwen2.5-32B-Instruct base model undergoes an alternative RLVR regimen (e.g., uniform training with higher rollout budget, or a different curriculum strategy from prior work) would isolate the contribution of the paper's specific two-stage design.
Statistical significance or confidence intervals. All results are point estimates on fixed test sets of 32–175 problems. The Codeforces evaluation uses only 33 problems — a 3-percentage-point difference represents a single problem. Without confidence intervals, it is unclear whether the 0.182 vs. 0.161 comparison with DeepSeek V3.1 on Codeforces (a 2.1-point gap on 33 problems) is statistically meaningful or noise.

Generalizability concerns:

The experiments are conducted entirely within competitive programming code generation using a single base model architecture (Qwen2.5) and a single RL algorithm (GRPO). The paper's claims about data curation and curriculum design are framed as general principles for RLVR, but the evidence is domain-specific and model-specific. The scaling experiment on the internal MoE model (Table 5) provides some evidence of generalizability across model scales and architectures, but this model and its training procedure are not described in sufficient detail to evaluate the transfer. Key questions remain: Would the same two-stage curriculum work for mathematical reasoning (AIME) where the paper identifies the current RLVR focus? Would it work with PPO instead of GRPO? Would it work with a different base model family (e.g., Llama, DeepSeek)? The paper cannot answer these questions from the reported experiments alone.

Evaluation validity:

The paper's use of temporally separated LeetCode and Codeforces contests as evaluation benchmarks is a methodological strength that mitigates data contamination concerns. However, the test sets are small (32 and 33 problems respectively), and the paper does not report how many times each model was evaluated or whether any form of test-set optimization occurred. The consistent improvements across all benchmarks suggest genuine capability gains, but the magnitude on the smallest benchmark (Codeforces, +58.3% relative) should be interpreted cautiously given the small sample size — this could reflect improvement on a handful of problems rather than broad capability advancement.

Additionally, the LiveCodeV6 benchmark was used in Stage 2 training. The paper acknowledges this and reports LiveCode08-11 and LiveCodeV5 as uncontaminated validation sets. However, these are from the same LiveCodeBench distribution, so improvements could partially reflect distribution-level overfitting (the model learns patterns specific to LiveCodeBench problem styles) rather than general competitive programming capability. The contest benchmarks address this concern to some degree, but the gap between LiveCode gains (+28.1% on V6, +16.1% on 08-11) and external contest gains (+13.0% on LeetCode, but +58.3% on Codeforces) is inconsistent and hard to interpret without larger test sets.

6. Limitations and Trade-offs

1. Difficulty Estimation Is Not Addressed, Making the Compute-Optimal Allocation Impractical at Deployment

The assumption or constraint. The entire two-stage RL framework depends on the ability to identify which problems are "hard" — both for the SFT-stage Twice Hard Learning strategy and for the RL-stage Pre-GRPO curriculum that continuously filters training to the lowest-pass-rate cases. The paper uses pass rate under the current model as the difficulty metric, which requires generating and evaluating multiple rollouts per candidate problem to determine whether it should be included in training. The authors acknowledge that they evaluate the SFT model on all 1.27M prompts during Arena Learning to identify hard samples, and that Pre-GRPO re-evaluates pass rates on all candidate problems between each curriculum phase. However, the paper never accounts for the computational cost of this difficulty estimation in any budget or efficiency calculation.

The consequence. In a realistic deployment where a practitioner wants to apply this method to a new problem distribution, they must first run the model on all candidate problems to measure pass rates, then iteratively re-measure throughout training as the curriculum progresses. For the Stage 2 curriculum with its three phases and 175 LiveCode V6 candidates, this means evaluating the 32B model on all 175 problems at least 4 times (once before each phase, plus initial filtering). With 64 rollouts per evaluation (to get reliable pass-rate estimates on hard problems), this costs 175 × 64 × 4 = 44,800 generations just for difficulty estimation, before any training updates occur. This cost is comparable to or exceeds the training budget itself (Stage 2 trains on 25–72 problems with 64 rollouts each). The headline performance numbers — the 58.3% relative improvement on Codeforces — are reported net of this cost. If a practitioner had to pay the full difficulty-estimation cost plus the training cost, the effective "efficiency" of the method would be substantially lower than the paper implies. The paper briefly touches on this problem when describing Arena Learning for SFT data, noting that the Twice Hard Learning strategy is computationally simpler than prior difficulty-aware methods, but says nothing about the RL-stage cost.

What evidence exists in the paper. The paper provides no ablation, measurement, or even estimate of the difficulty-estimation cost. Section 4.1 describes the data construction process in general terms ("we initially collected 1.27M open-source prompts... we then refined this dataset to 470K high-quality prompts using a 5-round arena learning method") but does not report the number of model generations required for this filtering. Section 3.2 describes Pre-GRPO as evaluating pass rates and selecting the K hardest cases, but does not specify how many rollouts are used for pass-rate estimation or what fraction of total compute this represents. Figure 5, which clusters cases by initial rollout accuracy, implicitly reveals that 8-rollout pass-rate estimates are used for difficulty assessment, but the cost of obtaining these estimates across all 175 problems is not discussed.

Mitigation status. The paper does not address this limitation directly. There is no proposal for cheap difficulty estimation (e.g., a lightweight classifier, a heuristic based on problem metadata, or adaptive difficulty assessment that amortizes estimation into training). The Arena Learning and Pre-GRPO mechanisms are presented as effective but their implementation cost is externalized from all reported results. This is a significant gap because it means the method, as described, is not deployable as a turnkey system — a practitioner would need to independently solve the difficulty-estimation problem before the curriculum design could be applied. The paper's contributions are therefore best understood as demonstrating what is possible with perfect difficulty information, leaving the practical challenge of how to obtain that information efficiently as entirely future work.

2. Single Benchmark Domain, Single Model Family, and a Single RL Algorithm

The assumption or constraint. All experiments are conducted on competitive programming code generation using the LiveCodeBench suite and LeetCode/Codeforces contests, with Qwen2.5-32B-Instruct as the sole base model, and GRPO as the sole RL algorithm. The paper states that competitive programming is "relatively underexplored" in RLVR and that prior work has focused on mathematics (AIME), positioning this as a contribution to broadening RLVR's domain coverage. However, the paper does not test whether its core methodological claims — that entropy expansion must precede hard-focus curriculum, that large rollout budgets are necessary for hard problems, that mixing easy and hard problems in uniform RL causes training collapse — generalize beyond this single domain, model architecture, and algorithm.

The consequence. A practitioner working in a different RLVR domain (mathematical reasoning, theorem proving, code generation for non-competitive tasks, planning) or with a different model family (Llama, DeepSeek, Gemini) or a different RL algorithm (PPO, REINFORCE, DPO) cannot determine from this paper alone whether the two-stage curriculum would help, hurt, or be irrelevant in their setting. Several aspects of the findings could be domain-specific or model-specific:

The entropy collapse problem (Figure 3, Appendix B) is attributed to the SFT model's narrow output distribution, but this may depend on the specific SFT data mixture (470K distilled prompts from DeepSeekR1-0528) and the Qwen2.5 architecture's tendency toward mode collapse. A different base model with different pretraining data or architectural inductive biases might not exhibit the same low-entropy pathology, making Stage 1 unnecessary.
The training collapse when mixing easy and hard problems (Table 4, "RL with all LiveCodeV6 dataset") occurs under a specific configuration (8 rollouts, 24k context, GRPO, LiveCode V6). The paper does not test whether this collapse is specific to competitive programming (where reward sparsity is extreme for hard problems), to GRPO's advantage estimation, or to the 8-rollout budget. In mathematical reasoning, where partial-credit rewards or step-level verification might provide denser signals, the same collapse might not occur.
The necessity of 64 rollouts for hard problems (Figure 6) is demonstrated on a small number of competitive programming cases. Whether the same rollout budget threshold applies to hard AIME problems (which require different reasoning patterns and have different solution-length distributions) is unknown.
GRPO-specific dynamics may not transfer to PPO. GRPO's group-based advantage estimation computes advantages relative to the prompt-specific mean reward. When all rollouts for a hard problem are incorrect (reward = 0), the advantage is identically zero for all rollouts, producing no gradient. PPO with a learned value function might produce non-zero advantages even when all rollouts fail (if the value function predicts non-zero expected reward), potentially enabling learning on hard problems even with low rollout budgets. The paper's central finding that large rollout budgets are necessary might be an artifact of GRPO's advantage estimation, not a general property of RLVR.

What evidence exists in the paper. The scaling experiment on the internal MoE model (Section 4.6, Table 5) provides some evidence of generalizability across model scales and architectures. The MoE model shows qualitatively similar patterns: Stage 1 yields mixed results with regression on hard benchmarks (Codeforces drops 20.65%), Stage 2 recovers and improves substantially (+25.16% relative to SFT on Codeforces). This suggests the two-stage curriculum's value is not purely an artifact of the Qwen2.5-32B architecture. However, the MoE model is still trained on competitive programming, still uses GRPO, and its specific architecture, size, and training details are not disclosed ("internal large-scale Mixture-of-Experts model"). This provides limited evidence of generalizability — it shows the approach scales to a larger model in the same domain with the same algorithm, but does not address cross-domain or cross-algorithm transfer.

Mitigation status. The paper does not claim generalizability beyond competitive programming and GRPO. The contributions are framed specifically in the context of "competitive-programming code generation" using "GRPO" (Section 1). The authors do not assert that their findings apply to mathematical reasoning, other code generation tasks, or other RL algorithms. However, the paper's positioning in the broader RLVR literature (Section 2) — critiquing the field's focus on mathematics and calling for more attention to data curation — implies that the methodological insights should be relevant beyond the specific domain studied. The tension between this implied generality and the narrow experimental scope is not addressed. The paper would benefit from an explicit discussion of which findings are likely domain-specific and which are hypothesized to be general, but the current text makes no such distinction.

This limitation is partially mitigated by the existence of the MoE scaling experiment, which at least demonstrates that the approach is not exclusively tied to the Qwen2.5-32B architecture. But the absence of cross-domain or cross-algorithm evidence means a practitioner in a different RLVR setting must treat the paper's prescriptions as hypotheses to be tested rather than established principles.

3. No Accounting for Inference Latency or Wall-Clock Time — Only Generation Count

The assumption or constraint. The paper measures computational cost exclusively in terms of generation count — the number of complete solutions the model produces. Training budgets are specified as "8 rollouts per prompt," "64 rollouts per prompt," "32 steps," and "32k context length." Evaluation metrics are pass@1 and pass@10. At no point does the paper report or discuss wall-clock time, inference latency, or the parallelizability of the training or inference procedures.

The consequence. The two-stage RL framework makes design choices that trade generation efficiency for latency in ways that may be unacceptable for real-world deployment. Specifically:

Stage 2's 64-rollout training per prompt is inherently serial for each prompt. While different prompts can be processed in parallel across a GPU cluster, within a single training step the 64 rollouts for each prompt must be generated, evaluated, and used for GRPO advantage computation before the next step begins. With 32k-token context windows, generating 64 complete solutions per prompt represents a substantial wall-clock time cost per training step — far more than the 8-rollout Stage 1 or standard uniform-RL approaches. The paper does not report training wall-clock time for either stage, making it impossible for a practitioner to estimate whether the accuracy gains justify the latency cost.
During inference, the model generates a single solution (pass@1). This is latency-efficient, but the paper's own results show that pass@10 (10 independent generations, selecting the best) yields substantially higher accuracy: the RL model achieves 33.33% pass@10 on Codeforces vs. 18.18% avg@1 (Table 1). The paper does not explore whether the model's pass@10 performance could be further improved by test-time techniques (majority voting, verifier-based selection, or the same Pre-GRPO-style allocation of inference budget to harder problems), or how such techniques would interact with the latency requirements of competitive programming contests (which typically impose time limits of 1–2 hours for human participants, but no hard latency constraints for automated systems).
The entropy expansion stage's short context window (24k) deliberately limits generation length to reduce repetition and truncation. But at inference time (32k context), the model generates longer solutions for hard problems. The paper does not report average inference latency per problem or discuss the tradeoff between solution length, accuracy, and time-to-solution. In a real competitive programming setting, a model that takes 10 minutes to solve a problem at 90% accuracy may be less useful than a model that takes 30 seconds at 70% accuracy, depending on the use case (contest participation vs. batch evaluation vs. interactive assistance).

What evidence exists in the paper. The paper provides no latency measurements, no wall-clock time estimates, and no discussion of the throughput-latency tradeoff. The only cost-related metrics are generation counts and context lengths, which are reasonable proxies for total FLOPs but ignore the serial dependencies that dominate wall-clock time in practice. The paper's evaluation protocol (pass@1, pass@10) measures accuracy as a function of generation budget but not as a function of time.

Mitigation status. The paper does not acknowledge this limitation. The entire experimental methodology treats generations as the only resource constraint, ignoring the time dimension entirely. This is standard practice in much of the RLVR literature, but it is particularly consequential for this paper because the central methodological claim — that large rollout budgets are necessary for hard problems — has direct and potentially prohibitive latency implications that go unexamined. A practitioner reading this paper would have no way to estimate whether the 64-rollout Stage 2 training is feasible within their compute budget and timeline, or whether the resulting model's inference speed is acceptable for their use case.

The only indirect acknowledgment of computational constraints appears in Section 4.6, where the authors note that MoE-scale experiments "were constrained by computational resources, and training was not continued to full convergence." This admission suggests that the method is computationally intensive, but it is framed as a resource constraint on the experimenters rather than a methodological limitation of the approach.

4. The Unverified Assumption That Difficulty Filtering (Not Increased Rollout Budget) Prevents Training Collapse

The assumption or constraint. The paper's most striking negative result is that training GRPO uniformly on all 175 LiveCode V6 problems with 8 rollouts and 24k context causes catastrophic performance degradation — LeetCode Weekly drops from 0.578 (SFT) to 0.296, a 48.8% collapse (Table 4, "RL with all LiveCodeV6 dataset"). The paper attributes this collapse to the mixing of easy and hard problems in uniform RL training, arguing that "easy problems dilute the training signal and even destabilize optimization" (Section 4.4). This interpretation motivates the entire Pre-GRPO mechanism: hard problems must be isolated and trained on exclusively, with easy problems excluded entirely.

The consequence. There is a competing interpretation that the paper does not test: the collapse may be caused by insufficient rollout budget (8), not by difficulty mixing. In the all-LiveCodeV6 experiment, hard problems with very low pass rates generate zero or near-zero reward signals because 8 rollouts rarely produce a correct solution. These problems contribute noisy, zero-variance gradients that may destabilize training. If the same experiment were run with 64 rollouts per prompt (matching Stage 2's budget), the hard problems would occasionally generate correct solutions, providing a meaningful learning signal. Under this alternative hypothesis, the problem is not that easy and hard problems are mixed — it is that the rollout budget is uniformly too low for the hard subset.

If this alternative hypothesis is correct, then a simpler solution exists: uniformly increase the rollout budget for all problems rather than designing a staged curriculum with difficulty filtering. This simpler approach would avoid the complexity and computational overhead of Pre-GRPO (difficulty estimation, dynamic re-ranking, phase management) while potentially achieving similar or better results. The paper provides no evidence to distinguish between these two interpretations.

What evidence exists in the paper. The single-case training experiments (Figure 6, Appendix A) demonstrate that higher rollout counts improve learning on hard cases in isolation, but these experiments do not involve mixing easy and hard cases — they cannot test whether difficulty mixing per se causes instability or whether insufficient rollouts cause instability that manifests when hard problems are included. The paper's observation that easy problems "dilute the training signal" is a plausible mechanism but is never isolated experimentally. Specifically, the paper does not report:

Uniform training on all LiveCode V6 problems with 64 rollouts. This is the critical missing ablation. If this configuration also collapsed, it would support the paper's claim that difficulty mixing is inherently destabilizing. If it performed well (comparable to or better than the two-stage method), it would undermine the necessity of Pre-GRPO and suggest that the primary finding is about rollout budgets, not curriculum design.
Stage 2 training with 8 rollouts instead of 64. This would test whether isolating hard problems is sufficient to enable learning even with low rollout budgets. If Stage 2 with 8 rollouts performed poorly (as the single-case experiments suggest), it would indicate that both isolation and large budgets are necessary. But this ablation is not reported.
Any experiment that varies rollout budget independently of difficulty filtering. The paper conflates these two variables throughout: Stage 1 uses moderate budgets with mixed difficulty; Stage 2 uses large budgets with hard-only filtering. Without independent manipulation, the effects of each variable cannot be separated.

Mitigation status. The paper does not acknowledge this confound. The interpretation of the collapse as evidence for difficulty filtering is presented as a direct conclusion rather than one of several possible explanations requiring additional experiments to adjudicate. The single-case experiments (showing that large rollouts help hard problems) and the full-dataset experiment (showing that uniform 8-rollout training collapses) are presented in different sections and never integrated into a unified causal model. A reader who notices this confound would be uncertain whether to implement the full Pre-GRPO curriculum or simply increase their rollout budget — two very different engineering choices with different complexity and cost profiles.

This limitation is particularly important because it affects the paper's central methodological claim: that data curation and curriculum design are first-class design axes in RLVR. If the primary finding is actually about rollout budgets (a resource allocation choice, not a data curation choice), then the paper's framing as a contribution to "data curation best practices" overstates the specificity of the insight. The paper's valid contribution would remain — large rollout budgets are necessary for hard problems — but the more complex and domain-specific Pre-GRPO mechanism might be an over-engineered solution to a simpler problem.

5. Small Evaluation Test Sets with No Statistical Significance Testing

The assumption or constraint. The paper evaluates on test sets of limited size: 32 LeetCode Weekly problems, 33 Codeforces OJ problems, and 166–175 LiveCode problems (Table 1, Table 2, Table 4). The paper reports point estimates of accuracy (e.g., 0.182 on Codeforces, 0.653 on LeetCode Weekly) without confidence intervals, standard errors, or any form of statistical significance testing. All comparisons between methods — the 58.3% relative improvement on Codeforces, the claim of state-of-the-art performance among 32B models, the comparison with DeepSeek V3.1 — are based on these point estimates alone.

The consequence. On the smallest test set (Codeforces OJ, 33 problems), a single-problem difference corresponds to approximately 3 percentage points of accuracy. The paper's headline result — 0.182 (6 correct out of 33) versus the SFT baseline of 0.115 (approximately 4 correct) — represents an improvement of roughly 2 problems. The comparison with DeepSeek V3.1 on Codeforces (0.182 vs. 0.161, approximately 6 vs. 5 correct) is a 1-problem difference. Whether these differences reflect genuine capability improvements or sampling noise is impossible to determine from the reported data.

This uncertainty cascades to the paper's broader claims. The 58.3% relative improvement on Codeforces sounds impressive but represents a small absolute change (0.115 → 0.182, or roughly +2 problems out of 33). On a different sample of 33 Codeforces problems, the magnitude of improvement could be substantially different. The paper's claim that the RL model achieves "performance comparable to much larger systems" is supported on the LiveCode benchmarks (166–175 problems, where differences of 1–2 percentage points on 0.7-scale accuracies represent ~2–3 problems) but is more fragile on the contest benchmarks.

Additionally, the paper evaluates the SFT model, RL Stage 1 model, and final RL model on the same fixed test sets. All design decisions — the choice of 8 vs. 64 rollouts, the 24k vs. 32k context length, the three-phase Pre-GRPO structure with 72→50→25 problems — were presumably informed by performance on these test sets (or on the LiveCodeV6 training set, which shares distribution with the test benchmarks). Without a held-out validation set used exclusively for hyperparameter selection, the reported numbers may be optimistically biased relative to true out-of-sample performance. The paper does use temporally separated contest benchmarks to avoid data leakage, but these same benchmarks appear to have been used for both model selection and final evaluation — there is no indication of a separate held-out set.

What evidence exists in the paper. The paper reports only point estimates. Section 4.1 describes the evaluation sets and the number of problems in each (166, 167, 175, 32, 33) but does not discuss statistical methodology. There are no error bars in any figure or table. The paper does not report how many evaluation runs were performed per model, whether results are averaged across multiple seeds, or whether any form of statistical comparison was conducted. The training dynamics figures (Figure 4, Figure 5) show smooth curves that might be interpreted as evidence of stable trends, but these are training-set metrics, not evaluation-set metrics, and do not address test-set variance.

Mitigation status. The paper does not acknowledge this limitation. The use of small test sets without statistical testing is common in the competitive programming evaluation literature (where problem sets are expensive to construct and evaluate), but it is a meaningful weakness when making comparative claims between methods with small absolute performance differences. The paper would benefit from at minimum reporting binomial confidence intervals for the accuracy estimates, or from using a larger set of contest problems (e.g., aggregating across multiple weeks of LeetCode and Codeforces contests). The current sample sizes — particularly 33 problems for Codeforces — are insufficient to support precise claims about relative model quality, especially when the claimed improvements are on the order of 2–3 problems.

This limitation interacts with the generalizability concern (Limitation 2): not only are the results specific to competitive programming, but even within that domain, the evidence for the method's superiority is based on small samples that may not be representative of the broader distribution of competitive programming problems. A practitioner deciding whether to invest in implementing this training pipeline would want to know whether the 58.3% relative improvement on Codeforces is a robust finding or a consequence of the specific 33 problems chosen for evaluation.

6. The Hard-Focus Curriculum Cannot Help When the Base Model's Capability Is Insufficient

The assumption or constraint. The entire RLVR training paradigm — and the two-stage curriculum specifically — assumes that the base model (after SFT) has some non-zero probability of generating correct solutions on the target problems. The Pre-GRPO mechanism selects the "hardest" problems based on pass rate, but this is always relative to the model's current capability. There is an implicit assumption that with enough rollouts and enough focused training, the model can eventually learn to solve these problems — that the pass rate is low because the model hasn't been trained on similar problems, not because the problems require capabilities the model fundamentally lacks.

The consequence. There exists a class of problems for which the base model's pass rate is functionally zero — not 2% or 5%, but truly zero even with 64 or 128 or 256 rollouts. These might be problems requiring algorithmic insights the model's pretraining never covered, or problems with solution patterns that are genuinely out-of-distribution for the SFT policy. For such problems, Pre-GRPO with 64 rollouts would generate zero correct solutions in expectation, producing zero reward signal and zero gradient — the model would never learn, regardless of how long or how exclusively it trained on these problems. The hard-focus curriculum would waste computational resources on problems that are beyond the model's fundamental capability frontier.

The paper's evidence on this limitation is mixed and not fully explored. The single-case training experiments (Appendix A) show that Case 4 "remains virtually frozen at 12.5% throughout training" even when trained in isolation — this is not zero, suggesting some initial capability, but it also shows no improvement despite focused training. The paper does not report whether any problems in the LiveCode V6 set had zero pass rate under the SFT model even with 64 rollouts, or whether Pre-GRPO ever selected such problems and, if so, whether they improved. The "stagnant progress in low-accuracy clusters" (Figure 5, Section 4.5) shows that the hardest problems improve little under uniform training, but does not reveal whether they are fundamentally unlearnable or merely under-resourced.

What evidence exists in the paper. The paper's own difficulty-bin analysis in Figure 5 shows that low-accuracy clusters make "stagnant progress" — accuracy barely moves over the course of training. This is presented as evidence that standard RL struggles with hard problems and motivates the Pre-GRPO curriculum. But the paper does not demonstrate that Pre-GRPO solves this problem. The Stage 2 results are reported only in aggregate (Table 2, Table 4) — there is no breakdown showing whether the specific hard problems that stagnated under uniform training (Figure 5) actually improved under the hard-focus curriculum. The paper claims that Stage 2 improves performance on "challenging problems" and that Codeforces gains (+58.3%) demonstrate this, but Codeforces is a separate evaluation benchmark — there is no per-problem tracking showing that the hard cases in the Stage 2 training set were successfully learned.

The additional-hard-cases experiment (Table 4, last row) shows that adding 109 more hard cases improves aggregate performance, but this could be driven by improvements on the easier cases among those 109, with the genuinely hardest cases still showing no progress. Without per-problem or per-difficulty-cluster reporting for Stage 2, it is impossible to determine whether Pre-GRPO successfully pushes the model's capability frontier or merely improves problems that were already marginally solvable.

Mitigation status. The paper does not explicitly discuss a fundamental capability ceiling or acknowledge that some problems may be unlearnable regardless of curriculum design. The concept of a "capability ceiling" is mentioned in Section 4.5 — "standard RL training struggles with difficult cases, potentially creating a capability ceiling that limits the model's performance" — but this is presented as a problem that the two-stage curriculum solves, not as a limitation that the curriculum inherits. The authors do not report whether any problems in their training set remained unsolved after Stage 2, or what fraction of the initial low-pass-rate problems improved under the hard-focus curriculum. Without this analysis, a practitioner cannot determine whether Pre-GRPO is likely to help with their specific hard problems (which might be of a different character than LiveCode V6 problems) or whether those problems are simply beyond their model's reach.

This limitation is particularly important for practitioners considering applying the method to genuinely novel or out-of-distribution problems — the very "hard" problems that competitive programming contests are designed to test. If a new Codeforces problem requires a technique the model has never seen, no amount of focused RL training with 64 rollouts will enable the model to solve it from scratch, because RLVR can only amplify existing capability, not create it from nothing. The paper's framing implies that the hard-focus curriculum extends the capability frontier, but the evidence cannot distinguish between "extending" (teaching the model to solve problems it could occasionally solve) and "revealing" (making more reliable a capability the model already possessed but expressed unreliably). This is a fundamental open question that the paper's experimental design does not address.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper does not propose a new RL algorithm, a new model architecture, or a new reward function. Its contribution is methodological and diagnostic: it demonstrates that data curation and curriculum design are not auxiliary concerns in RLVR — they are stability conditions. This is a reframing of what constitutes a complete RLVR method, not an incremental improvement. Prior work treated the RLVR pipeline as: pick a base model → do SFT → apply an RL algorithm (GRPO, PPO, DAPO variants) with a fixed rollout budget on a static dataset. This paper shows that this pipeline, applied naively to competitive programming, can be actively destructive — the "RL with all LiveCodeV6 dataset" configuration in Table 4 degrades LeetCode Weekly performance by 48.8% below the SFT baseline. That is not underperformance; it is collapse.

The conceptual shift is from asking "which algorithm?" to asking "which problems, in what order, with what sampling budget?" The paper provides a diagnostic framework for this shift: identify the specific pathologies of the SFT→RL transition (entropy collapse, repetitive generation, sparse rewards on hard problems), design training stages that target each pathology, and validate through ablation that each stage is necessary. This framework is transferable even if the specific two-stage recipe (GRPO on 9k problems with 8 rollouts followed by Pre-GRPO on 25–72 hard problems with 64 rollouts) is competitive-programming-specific.

The paper also reconciles a latent tension in the RLVR literature. Prior work on data selection for RLHF (Shen et al., 2025; Li et al., 2025a) and on difficulty filtering for DPO (Gao et al., 2025) treated data curation as a problem of reward signal quality — identify prompts where the learned reward model produces informative, low-noise signals. This paper operates in RLVR where the reward is executable verification (noise-free, deterministic), yet shows that data curation remains critical — but for entirely different reasons: difficulty scheduling (not reward noise) and rollout budget sufficiency (not reward model calibration). This implies that data curation is a first-class concern in RLVR regardless of reward type, and that the specific curation strategy must be matched to the reward structure: with noisy learned rewards, filter for signal quality; with clean verifiable rewards, schedule for difficulty and budget allocation.

Two research directions become more attractive as a result of this paper. First, difficulty-aware training dynamics analysis — the paper's Figure 5 (clustering problems by initial accuracy and tracking learning trajectories) is a diagnostic tool that every RLVR practitioner should adopt. It reveals whether the optimization process is naturally allocating learning across difficulty tiers or concentrating on medium-difficulty cases while neglecting hard ones. Second, rollout budget as a first-class hyperparameter — the finding that 64 rollouts are necessary for hard-problem learning (Figure 6, Appendix A) suggests that reporting rollout budgets per difficulty tier should become standard practice, analogous to how pretraining scaling laws report model size and data quantity separately rather than a single aggregate FLOPs number.

Conversely, one research direction becomes less attractive: the pursuit of ever-more-sophisticated RL algorithms (new PPO variants, better KL control, novel advantage estimators) as the primary path to improved RLVR performance. The paper's results suggest that, at least for competitive programming with clean rewards, the bottleneck is not algorithmic design but data scheduling and sampling budget. A practitioner could achieve larger gains by reallocating their engineering effort from implementing a novel policy gradient variant to carefully curating their training problem set and increasing their rollout budget on hard cases. This does not mean algorithmic innovation is irrelevant — there may be domains or reward structures where it dominates — but it shifts the default assumption about where gains come from.

Finally, the paper introduces Pre-GRPO as a specific curriculum mechanism — continuous hard-problem retention through dynamic pass-rate re-ranking — that operationalizes a broader principle: in RLVR with heterogeneous difficulty, the training set should evolve to track the model's moving capability frontier. This principle is likely more important than the specific implementation details (three phases, 72→50→25 problems, 32k total steps). Future work will need to determine whether other instantiations of this principle (soft weighting, adaptive thresholds, continuous difficulty estimation) achieve similar benefits with lower computational overhead.

Follow-Up Research This Work Enables

Disentangling difficulty filtering from rollout budget in the training collapse. The paper's most consequential negative result — the 48.8% collapse on LeetCode Weekly when training uniformly on all 175 LiveCode V6 problems (Table 4) — is attributed to mixing easy and hard problems, but it is confounded with rollout budget: the experiment used 8 rollouts per prompt, which the paper's own analysis shows is insufficient for hard problems (Figure 6). A critical follow-up experiment would train GRPO uniformly on all 175 LiveCode V6 problems with 64 rollouts per prompt, matching the Stage 2 budget but without difficulty filtering. If this configuration does NOT collapse, the causal attribution shifts: the problem is not difficulty mixing per se, but insufficient rollout budget for the hard subset within a mixed distribution. If it DOES collapse, the paper's claim that easy problems actively destabilize RL training is strengthened. Either outcome would substantially refine our understanding of RLVR training dynamics. A second arm of this experiment would apply Pre-GRPO's hard-only filtering but with 8 rollouts per prompt — this would test whether isolating hard problems is sufficient for learning even with low budgets, or whether both isolation and high budgets are necessary. Together, these two experiments would resolve the central confound in the paper's causal model.

Cheap difficulty estimation to make Pre-GRPO practical for deployment. The paper's Pre-GRPO mechanism requires evaluating pass rates on all candidate problems between curriculum phases, which costs 175 problems × 64 rollouts × 4 evaluations ≈ 44,800 generations for difficulty estimation alone — comparable to the training budget itself. The paper does not account for this cost. A practical follow-up would train a lightweight difficulty classifier that takes only the problem statement (and optionally a small number of model-generated solution sketches) as input and predicts whether the problem will be in the lowest pass-rate quintile under the current policy. The classifier could be trained on pass-rate data collected during the first phase of a single run, then deployed to filter problems for subsequent runs or for new problem sets without re-running full pass-rate evaluations. A strong result would demonstrate that classifier-guided filtering achieves within 2–3 percentage points of the accuracy of full Pre-GRPO with pass-rate evaluations, while reducing the difficulty-estimation cost by 10× or more. This would transform Pre-GRPO from a research technique that assumes oracle difficulty information into a deployable component of an RLVR pipeline.

Cross-domain replication on mathematical reasoning (AIME) with the same base model. The paper frames data curation and curriculum design as general principles for RLVR but evaluates exclusively on competitive programming. A natural test of generality would replicate the two-stage curriculum — entropy expansion on a broad problem set (e.g., MATH training split) followed by Pre-GRPO on the hardest AIME problems — using the same Qwen2.5-32B-Instruct base model and GRPO algorithm. The key questions: (1) Does the SFT→RL entropy collapse problem (Figure 3, Appendix B) manifest in mathematical reasoning, or is it specific to code generation where SFT solutions tend to converge on verbose, repetitive debugging traces? (2) Does uniform RL training on mixed-difficulty math problems cause the same catastrophic degradation observed in Table 4, or does the denser reward signal in math (step-level verification vs. all-or-nothing test cases) mitigate this? (3) Does the 64-rollout budget threshold for hard problems (Figure 6) transfer, or do hard math problems require different budgets because solution lengths and pass rates differ from code? A replication that found qualitatively similar patterns would establish the paper's framework as domain-general; a replication that found different patterns would precisely delineate the boundary conditions and prevent over-generalization.

Testing whether PPO with a learned value function can learn hard problems with low rollout budgets. The paper's finding that large rollout budgets are necessary for hard problems is demonstrated exclusively with GRPO, whose group-based advantage estimation produces zero gradient when all rollouts for a prompt are incorrect (since group mean reward = 0, every advantage = 0). PPO with a learned value function could theoretically produce non-zero advantages even when all rollouts fail, if the value function predicts non-zero expected reward — the policy could receive a signal to increase the probability of actions that the value function believes are promising, even though those actions haven't yet produced a correct solution in the current batch. A direct comparison would train PPO (with identical rollout budget, problem set, and base model) on the same hard cases that stagnate under GRPO in Figure 5. If PPO with 8 rollouts achieves learning progress on cases where GRPO with 8 rollouts shows zero progress, the paper's claim about rollout budgets would be algorithm-specific rather than fundamental to RLVR. If PPO also requires 64 rollouts, the finding generalizes and strengthens the paper's argument that sampling budget, not algorithmic design, is the bottleneck.

Single-case learning with controlled difficulty to measure the "unlearnable problem" threshold. The paper's single-case training experiments (Appendix A) show that some hard cases plateau at low accuracy (12.5% for Case 4) even after 60 steps of isolated training. But these experiments use a fixed rollout budget (8) and fixed training duration (60 steps). A systematic follow-up would take a set of problems spanning a range of initial pass rates (0%, 2%, 5%, 10%, 20%, 50%, 80%) and train on each in isolation with exponentially increasing rollout budgets (8, 16, 32, 64, 128, 256) until convergence or until a compute budget is exhausted. The output would be a learnability curve: for a given initial pass rate and rollout budget, what is the asymptotic accuracy the model can reach through focused RL training? This would identify whether there exists a hard floor — problems with initial pass rate below some threshold that are fundamentally unlearnable regardless of budget — or whether any problem with non-zero initial pass rate can be mastered given sufficient rollouts. The existence (or absence) of such a floor has direct implications for when Pre-GRPO is worth applying versus when problems should be routed to a larger model or human intervention.

Adaptive difficulty assessment during training to replace static Pre-GRPO phases. The paper's Pre-GRPO uses discrete phases with fixed problem counts (72→50→25), requiring explicit re-evaluation between phases. A more efficient approach would be continuous difficulty tracking: after each training step, update a running estimate of each problem's pass rate using an exponential moving average of the rewards from that step's rollouts. Problems with estimated pass rates above a threshold are dynamically removed from the training pool; problems below the threshold are retained. New problems from a candidate pool are added whenever the active training set falls below a target size. This would eliminate the need for separate evaluation phases, reduce the total difficulty-estimation cost (since pass-rate estimates are a byproduct of training rollouts rather than requiring dedicated evaluation rollouts), and allow the curriculum to adapt more smoothly to the model's evolving capabilities. A strong result would match or exceed the full Pre-GRPO's performance while reducing total generation cost by 30–50% by eliminating dedicated evaluation rollouts.

Practical Applications and Downstream Use Cases

Cost-efficient training of specialized coding models for competitive programming platforms. Organizations that operate competitive programming platforms (Codeforces, AtCoder, LeetCode, HackerRank) or build AI systems for automated problem-solving can use the two-stage curriculum to train models that achieve strong performance at modest parameter scales. The paper shows that a 32B model trained with this method reaches 0.182 pass@1 on Codeforces, exceeding DeepSeek V3.1 (0.161) — a model with likely 10× or more parameters. For a platform that needs to evaluate thousands of contest problems daily for automated problem routing, difficulty tagging, or solution validation, deploying a 32B model instead of a 200B+ model represents a 5–10× reduction in inference cost per query, with no accuracy penalty on the hardest benchmark. The practical recipe: collect problem sets from the platform's contest history, perform SFT with Twice Hard Learning on distilled solutions from a strong teacher model, run entropy-expansion GRPO on the full problem set with 8 rollouts, then deploy Pre-GRPO with 64 rollouts on the hardest contest problems identified by pass rate. The resulting model can serve as an always-available, low-latency coding assistant for contest participants or an automated baseline for problem difficulty calibration.

Data generation pipelines for self-improving code models. When using LLMs to generate training data for themselves — through rejection sampling, STaR-style self-training, or RLVR fine-tuning — the quality and coverage of generated solutions determine the ceiling of downstream performance. The paper's finding that uniform RL training on mixed-difficulty problems causes collapse (Table 4) has direct implications for how such data generation pipelines should be structured: do not train uniformly on all generated data. Instead, apply difficulty filtering to identify hard cases (low pass rate under the current model), and allocate a disproportionate generation budget to these cases. For a self-improvement loop targeting competitive programming, the practical implementation would be: (1) generate 8 solutions per problem across the full training set to estimate per-problem pass rates; (2) for problems with pass rates above 80%, stop generating — the model already handles these; (3) for problems with pass rates below 20%, generate 64 solutions and use any correct ones as positive training examples; (4) fine-tune on the combined dataset of correct solutions, weighted by problem difficulty. This targeted allocation could substantially improve the sample efficiency of self-improvement pipelines compared to uniform data generation, concentrating compute where it produces novel correct solutions rather than redundant correct solutions on already-mastered problems.

Benchmark construction with difficulty-calibrated problem sets. The operator of a competitive programming benchmark (or an organization building internal evaluation suites for coding models) can use the paper's pass-rate-based difficulty estimation to construct difficulty-calibrated test sets. Rather than relying on human-assigned difficulty labels (which may not correlate with model performance) or contest problem ratings (which are noisy and contest-specific), the operator can run their target model(s) on a large pool of candidate problems with a fixed budget (e.g., 64 rollouts per problem), compute per-problem pass rates, and select test sets stratified by pass-rate quintile. The paper shows that model-relative difficulty (pass rate under the current model) is more predictive of RL training dynamics than dataset-level difficulty labels (Section 3.2's use of quintiles, Figure 5's clustering by initial accuracy). A benchmark constructed this way would provide more informative evaluation: it would reveal not just aggregate accuracy but the shape of the model's capability frontier — which difficulty tier is the primary source of errors, and whether test-time or training-time interventions help on that tier. This is directly actionable for model developers deciding where to invest their next unit of compute.

When to Prefer This Method

The paper does not explicitly position its two-stage RL curriculum against named alternatives (e.g., "use our method instead of DAPO" or "prefer our curriculum when data is scarce"). The comparisons in Table 2 and Table 4 are against the SFT baseline, Stage-1-only, Stage-2-only, and uniform RL ablations — internal variants of the same pipeline, not external competing methods. The external baselines (DeepSeek V3.1, Seed1.6-0715, Qwen3-235B) are presented as performance references rather than method alternatives, and the paper does not claim superiority over them (it claims "comparable" or "competitive" performance at smaller scale). Since the paper does not articulate a clear decision rule for when to choose its specific two-stage curriculum design over, say, a single-stage GRPO with higher uniform rollout budget or a different curriculum strategy from prior work, a forced "Prefer A when X; Prefer B when Y" matrix would be speculative rather than paper-grounded.

The one actionable tradeoff the paper does support — implicitly through its ablation structure — is between staged curriculum training and uniform training on all available data. The evidence in Table 4 provides clear conditions: if your problem set contains a mix of difficulties and your initial RL training on the full mix shows stagnant progress on the hardest subset (diagnosable via the Figure 5 clustering analysis), you should consider a curriculum that isolates hard problems and increases per-problem rollout budget, rather than continuing uniform training with the existing configuration. But this is a diagnostic heuristic, not a comparative claim against a named alternative method, and the paper does not test whether alternative curricula (soft weighting, priority sampling, different phase structures) would perform equivalently or better.

Dependency tree

Papers this one builds on, walking backwards through the same domain and shared topics.

Competitive Programming with Large Reasoning Models2025· rl-training← LLM-judged similar
- Process Reinforcement through Implicit Rewards2025· rl-training← in-domain predecessor
- o3-mini vs DeepSeek-R1: Which One is Safer?2025· safety← topic thread
Reinforcement Learning Improves Traversal of Hierarchical Knowledge in LLMs2025· training-methods← in-domain predecessor
- SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training2025· alignment← LLM-judged similar
- FP8-Flow-MoE: A Casting-Free FP8 Recipe without Double Quantization Error2025· training-methods← in-domain predecessor
- TruthRL: Incentivizing Truthful LLMs via Reinforcement Learning2025· training-methods← similar topic
The Path Not Taken: RLVR Provably Learns Off the Principals2025· rl-training← topic thread
- TimeSearch-R: Adaptive Temporal Search for Long-Form Video Understanding via Self-Verification Reinforcement Learning2025· rl-training← in-domain predecessor
- Fine-Grained GRPO for Precise Preference Alignment in Flow Models2025· training-methods← topic thread

What it influenced

Later papers in the same chain. Direction reversed.

Black-Box On-Policy Distillation of Large Language Models2025· training-methods→ in-domain predecessor
- Distribution-Aligned Sequence Distillation for Superior Long-CoT Reasoning2026· reasoning→ LLM-judged similar
- Learning beyond Teacher: Generalized On-Policy Distillation with Reward Extrapolation2026· training-methods→ LLM-judged similar
- Online Experiential Learning for Language Models2026· training-methods→ LLM-judged similar
- Beat the long tail: Distribution-Aware Speculative Decoding for RL Training2025· training-methods→ in-domain predecessor
- Few-Step Distillation for Text-to-Image Generation: A Practical Guide2025· diffusion→ topic thread
- Generative Adversarial Nets2014· training-methods→ similar topic
- On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes2023· training-methods→ similar topic
Evaluating Parameter Efficient Methods for RLVR2025· rl-training→ topic thread
- ThinkRL-Edit: Thinking in Reinforcement Learning for Reasoning-Centric Image Editing2026· rl-training→ in-domain predecessor
- Exploration vs Exploitation: Rethinking RLVR through Clipping, Entropy, and Spurious Reward2025· training-methods→ topic thread