Nemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy Distillation

ArXiv: 2603.19220

🎯 Pitch

A 3B-active-parameter model matches the olympiad gold-medal performance of a model 20× its size. This breakthrough comes from sequentially chaining reinforcement learning across domains while distilling from the best intermediate checkpoint for each—stabilizing capability growth without the usual forgetting or regression. The recipe delivers frontier reasoning and agentic results at a fraction of the scale previously thought necessary.

1. Executive Summary

This paper introduces Nemotron-Cascade 2, an open 30B MoE model with 3B activated parameters post-trained using Cascade RL (sequential, domain-wise reinforcement learning stages spanning instruction-following, multi-domain reasoning, RLHF, long-context, code, and software engineering) and Multi-Domain On-Policy Distillation (MOPD) (distilling knowledge from the strongest intermediate teacher checkpoints within each domain during the Cascade RL process to recover benchmark regressions). Evaluated across mathematical reasoning, competitive coding, alignment, and agentic benchmarks, Nemotron-Cascade-2-30B-A3B achieves gold-medal performance on the 2025 International Mathematical Olympiad (35/42 points), the 2025 International Olympiad in Informatics (439.28/600), and the ICPC World Finals 2025 (10/12 problems solved), establishing that olympiad-level reasoning can emerge from a model with ~20× fewer parameters than the prior open-weight gold-medal achiever only when post-training orchestrates domain-specialized RL with on-policy distillation to stabilize capability retention across diverse environments.

2. Context and Motivation

The Core Problem: Scaling RL Post-Training Across Many Diverse Domains Without Destabilizing the Model

The fundamental problem this paper tackles is deceptively practical: how do you apply reinforcement learning to a language model across a wide array of increasingly complex reasoning and agentic domains without causing the model to forget earlier capabilities or collapse from incompatible training signals? This matters because the state-of-the-art paradigm for building capable language models has shifted decisively toward RL-based post-training (Guo et al., 2025; Ouyang et al., 2022), but the engineering challenge of running RL across instruction-following, mathematical reasoning, competitive coding, human preference alignment, long-context understanding, software engineering, and terminal-based agentic tasks — each with different reward types (verifiable binary signals, generative model judgments, environment feedback), different response lengths (from short answers to 256K-token agent trajectories), and different verification costs — has not been solved by prior work in a way that scales gracefully.

The consequences of this gap are substantial. If each new domain requires a separate training run (or causes degradation in previously learned skills), the cost and engineering complexity of building truly general reasoning models becomes prohibitive. Conversely, if all domains are naively mixed together in a single RL stage, incompatible reward signals can create conflicting objectives — for example, instruction-following RL optimizing for strict format adherence may directly compete with RLHF optimizing for conversational fluency, or code RL encouraging extremely long reasoning traces may distort the model's behavior on short-answer STEM questions. The paper's Cascade RL framework directly addresses this by providing a principled approach to ordering RL stages to minimize interference while maximizing per-domain gains.

The Catastrophic Forgetting Problem in Multi-Domain RL

The paper builds directly on its predecessor, Nemotron-Cascade 1 (Wang et al., 2025). That work demonstrated that sequential, domain-wise RL — where each stage targets a specific capability before moving to the next — significantly outperforms both pure single-domain RL (which leaves other capabilities untouched) and naive multi-domain blending (which can cause interference). The three key advantages of the Cascade approach, as recapitulated in Section 1 of the current paper, are:

Resistance to catastrophic forgetting: Domain-specific RL stages rarely degrade benchmarks from earlier domains and may even improve them through positive transfer.
Tailored optimization: Hyperparameters, curricula, and reward functions can be customized for each domain's specific characteristics (e.g., binary execution feedback for code vs. generative reward model scores for creative writing).
Compute efficiency: Task homogeneity within each stage keeps response lengths and verification wall-clock times uniform, avoiding the inefficiency of mixing short-answer and long-trajectory tasks in the same batch.

However, Nemotron-Cascade 1 left open a critical question that Nemotron-Cascade 2 directly confronts: as the number of RL domains scales up and the environments become significantly harder, the sequential nature of Cascade RL still exhibits measurable benchmark regressions, even if much smaller than in arbitrary-order or blended approaches. The paper observes in Section 4.4:

"While well-designed Cascade RL substantially reduces catastrophic forgetting compared with vanilla sequential RL in an arbitrary order, it does not fully eliminate capability drift as the number of training environments increases."

This is the gap that motivates the introduction of Multi-Domain On-Policy Distillation (MOPD) as a new component within the Cascade pipeline. MOPD serves as a stabilization stage that re-balances capabilities by distilling from the strongest intermediate teacher checkpoints for each domain, effectively recovering ground lost during specialized RL stages.

Why Prior Approaches Fall Short

The paper identifies several specific limitations in existing approaches, both from the broader literature and from the prior Cascade RL framework:

Naive multi-domain RL causes interference. Training multiple domains simultaneously in a single RL stage can create conflicting reward signals. The paper provides concrete evidence for this when discussing their ordering decisions: for example, "extended IF-RL training can lead to excessive token usage" (Section 4.2.2), and "IF-RL can negatively impact human alignment capabilities (e.g., ArenaHard)" (Section 4.2.2). Similarly, "certain RLVR training often reduces model entropy and shortens reasoning traces, thus can negatively impact mathematical reasoning performance" (Section 4.4). These domain-specific side effects mean that simply mixing all datasets together would produce a model pulled in incompatible directions.

Pure sequential RL without stabilization still drifts. The Cascade 1 approach already demonstrated that ordering matters enormously for reducing forgetting, but the authors of Cascade 2 found that as the number of stages grew (6 major RL stages in this work), even careful ordering couldn't prevent some capability slippage. This is visible in the benchmark regressions they report and in the observation that MOPD is "effective at recovering benchmark performance that may have regressed during earlier, more specialized stages of the cascade RL, leading to a more balanced and robust final policy model" (Section 4.1.1).

Single-teacher distillation is insufficient. Prior on-policy distillation work (Agarwal et al., 2024; Gu et al., 2024; Xiao et al., 2026; Zeng et al., 2026) typically uses a single teacher model to guide the student. But a single teacher checkpoint that is strong at math may be weak at instruction-following; one that excels at human preference alignment may have lost code reasoning capability. The paper's key insight is that the Cascade RL process itself naturally produces a collection of intermediate checkpoints, each of which represents peak performance on a specific capability domain. By selecting these checkpoints as domain-specific teachers for MOPD — rather than using a single external teacher or the final model — the distillation can provide dense, token-level supervision that recovers multi-domain performance simultaneously.

Standard GRPO with sparse rewards is sample-inefficient. The paper explicitly contrasts MOPD with standard GRPO training, noting that "MOPD provides a dense token-level distillation advantage, whereas GRPO relies on a sparse sequence-level outcome reward that is shared across all generated tokens" (Section 4.4). This makes MOPD "substantially more sample- and step-efficient in practice," as demonstrated by the convergence comparison in Figure 3(c) and Table 3: MOPD reaches 92.0 on AIME25 in 30 steps vs. GRPO reaching 91.0 in 25 steps but not matching teacher-level performance, and MOPD achieves ArenaHard Hard Prompt 85.5 in 52 steps vs. RLHF requiring 160 steps to reach 80.7.

Prior work treated test-time compute and post-training RL as separate. While not explicitly named as a gap, the paper's integration of competitive-level test-time scaling strategies (multi-round generate-select-submit pipelines for IOI, generate-verify-refine for IMO proofs) with the post-trained model demonstrates that scaling RL post-training and scaling inference compute are complementary. The IMO and IOI results were not achievable through post-training alone — they required test-time strategies that would only be effective because the post-trained model had learned sufficiently strong reasoning and self-verification behaviors during RL.

How This Paper Positions Itself

The paper positions itself as an evolution and substantial scaling of the Cascade RL paradigm, not a replacement. The three novel contributions relative to Cascade 1 are:

1. Multi-Domain On-Policy Distillation as an integration mechanism. MOPD is the most significant architectural addition. It leverages the fact that Cascade RL naturally produces a collection of capability-diverse checkpoints, all derived from the same SFT initialization, sharing the same tokenizer and vocabulary. This eliminates the distribution shift and alignment problems that arise when distilling from external teacher models while providing dense token-level advantages that converge faster than sparse GRPO rewards. The paper demonstrates this concretely in Section 4.4, showing both the training dynamics (reverse KL converging toward zero within 40-50 steps, gradient norms dropping rapidly after warm-up) and the downstream benefits (Figure 3(c) showing MOPD recovering and even exceeding teacher performance on AIME25, Table 3 showing superior efficiency on ArenaHard v2).

2. Substantially expanded Cascade RL domains and environments. Compared to Cascade 1, Cascade 2 covers a much broader spectrum: instruction-following RL, multi-domain RL (STEM MCQA + agentic tool calling + structured output), RLHF with a large generative reward model, long-context RL, competitive code RL (with 118K-token maximum response lengths and 2,048 code executions per step), agentless SWE RL (with a code repair reward model), and execution-based agentic SWE RL (with 256K context and 200-turn interaction budgets). This represents roughly a doubling of the RL stages and a significant increase in environment complexity.

3. A revised ordering principle informed by scaling experience. The ordering of stages in Cascade 2 differs from Cascade 1 because "advancements in SFT data quality and substantially scaling the complexity of the RL environments and tasks have fundamentally altered the model's behavioral dynamics" (Section 4.1.1). The paper describes a "rule of thumb" for ordering: identify which tasks serve as foundational priors and which act as specialized refinements, then place foundational tasks earlier. IF-RL is placed first because it produces a model with superior instruction-following that serves as a strong teacher for subsequent MOPD stages. RLHF is placed after MOPD because the GenRM-based optimization can trade off against instruction-following behavior, but early IF-RL already maximizes that capability. Long-context RL is placed late because it benefits from the model already having strong reasoning but can be destabilizing if introduced too early. SWE RL is placed last because it represents the most complex, scaffold-integrated agentic behavior.

The paper also explicitly situates itself relative to the training-inference tradeoff literature, though more implicitly. The IMO, IOI, and ICPC results demonstrate that a relatively compact model (30B MoE, 3B activated), when post-trained with careful Cascade RL + MOPD and equipped with test-time scaling strategies, can achieve performance previously thought to require either frontier proprietary models (Gemini Deep Think) or the largest open models (DeepSeek-V3.2-Speciale-671B-A37B). This establishes that post-training quality can substitute for pretraining scale in specific high-difficulty reasoning domains, a finding that complements prior work on compute-optimal test-time scaling (though Cascade 2's contribution is about post-training compute allocation, not inference-time compute allocation).

tag after a</think>token indicates non-thinking mode, while a single response` behavior. The trade-off is that thinking-mode IF-RL generates longer responses (the model writes out its reasoning before answering), but since the later RLHF stage will recover any human-preference regressions anyway, the paper prioritizes maximizing IF-Bench accuracy over keeping responses short. Additionally, eliminating the auxiliary reward model reduces computational overhead.

Training details. The configuration is: batch size 128 (128 distinct prompts per gradient step), 16 rollouts per prompt, temperature 1.0, top-p 1.0, AdamW optimizer with learning rate $3 \times 10^{-6}$ , entropy loss coefficient 0, KL loss coefficient 0, maximum response length 49K tokens, and the stage runs for approximately 180 steps. The learning rate of $3 \times 10^{-6}$ is the standard across almost all Cascade RL stages (only MOPD uses $2 \times 10^{-6}$ ), suggesting this was found to be a stable value through empirical tuning.

Why IF-RL produces a strong teacher for MOPD. The paper explicitly states that the early IF-RL stage "produces a model with superior instruction-following capabilities, which serves as a strong teacher for subsequent multi-domain on-policy distillation." This is a key architectural insight: by placing IF-RL first, its checkpoint achieves peak instruction-following performance (83.13% on IFBench) before any other RL optimization has occurred. This checkpoint then becomes one of the three domain-specific teachers in the MOPD stage, providing a dense token-level supervision signal specifically for instruction adherence that the student model can absorb alongside math and RLHF capabilities from the other teachers.

3.4.4 Multi-Domain RL — Stage 2

Following IF-RL, a second RL stage trains on three domains simultaneously (Section 4.3). The decision to group these specific domains — rather than keeping them separate — is based on empirical observations about non-interference and computational efficiency.

Domain composition. The data mixture consists of approximately 55% multi-choice question answering (MCQA) in the STEM domain, 30% agentic tool calling using the "Workplace Assistant" setup from NVIDIA Nano-v3, and 15% structured output for instruction following. All data is drawn from the NVIDIA Nano-v3 RL training blend.

Why these three domains are grouped. The paper provides two specific justifications. First, they "do not observe performance degradation across evaluation benchmarks when training on the blended domains," meaning there is no measurable negative interference between MCQA, tool calling, and structured output. Instead, "the model exhibits consistent improvements on benchmarks including MMLU-Pro, $\tau$ 2-Bench, and IF-Bench." This is the critical condition for grouping: if there were cross-domain interference, each domain would need its own sequential Cascade RL stage; the fact that they coexist without conflict means they can be combined, reducing the total number of stages. Second, "the response lengths and verification times of these datasets are similar, which minimizes training inefficiencies caused by waiting for longer generations or slower environment verification." This is a practical consideration: MCQA answers are short, tool calls are structured and bounded, and structured output is constrained by format requirements. If one domain generated 100K-token responses while another generated 1K tokens, the batch would be limited by the slowest generation, wasting compute. Homogeneous response lengths mean the 128-prompt batch can be processed efficiently.

Training details. The configuration mirrors IF-RL: batch size 128, 16 rollouts per prompt, temperature 1.0, top-p 1.0, AdamW with learning rate $3 \times 10^{-6}$ , entropy loss coefficient 0, KL loss coefficient 0, maximum response length 49K. The stage runs for approximately 70 steps — significantly shorter than IF-RL's 180 steps, suggesting that these domains converge faster or that the model already has strong foundational capabilities from SFT and IF-RL for these tasks.

The foundational-to-specialized progression. The placement of this stage after IF-RL but before MOPD reflects the Cascade RL ordering principle. IF-RL establishes the most fundamental behavior (strict adherence to any instruction), and then Multi-domain RL expands into specific application domains (STEM knowledge, tool use, structured output) that build on that foundation but do not conflict with it. The checkpoint after this stage becomes one of the MOPD teachers, specifically "selected from the checkpoints after previous IF-RL + Multi-domain RL stages" (Section 4.4) — meaning it captures peak multi-domain capability before any subsequent RL optimization might partially regress it.

3.4.5 Multi-Domain On-Policy Distillation (MOPD) — The Stabilization Mechanism

This is the paper's primary methodological innovation and the most technically detailed component (Section 4.4). MOPD is a dense token-level training stage that distills knowledge from multiple domain-specific teacher checkpoints back into a single student policy, positioned at the midpoint of the Cascade RL pipeline to recover any benchmark regressions from earlier specialized stages and produce a balanced model for subsequent refinement.

The problem MOPD solves. The paper identifies a specific failure mode in Cascade RL scaling: as the number of RL environments increases, even careful sequential ordering cannot fully prevent capability drift. They observe "noticeable fluctuations across different benchmark categories tracked throughout training," and the dominant trade-offs differ by stage. Two concrete examples are given: "certain RLVR training often reduces model entropy and shortens reasoning traces, thus can negatively impact mathematical reasoning performance," and "RLHF-oriented optimization can partially trade off against instruction-following behavior." These are not catastrophic forgetting (the model still performs the task), but rather partial regressions where earlier gains are diluted by subsequent training objectives. Standard GRPO with sparse sequence-level rewards is too coarse to recover these regressions efficiently because a single binary or continuous reward per response provides no guidance about which tokens contributed to the regression or how to restore the lost capability.

The teacher selection strategy. The key insight is that the Cascade RL process itself naturally produces a collection of capability-diverse checkpoints without requiring external model families. The paper selects three domain-specific teachers:

Math teacher: the initial SFT checkpoint — "which already exhibits strong mathematical reasoning capabilities thanks to the meticulously curated SFT dataset." This teacher has not undergone any RL optimization, so its math capability represents the pure SFT baseline.
RLHF teacher: "a checkpoint optimized through RLHF from the initial SFT checkpoint." This is NOT the checkpoint after MOPD or after the main RLHF stage; it is a separate RLHF run performed directly on the SFT model, specifically to create a teacher that excels at human preference alignment without any of the intermediate Cascade RL stages having been applied.
Multi-domain teacher: "selected from the checkpoints after previous IF-RL + Multi-domain RL stages." This captures peak instruction-following and multi-domain capability achieved during the first two Cascade RL stages.

The paper emphasizes three advantages of using internal Cascade checkpoints as teachers rather than external model families: "(1) teacher checkpoints can be selected directly from the Cascade RL pipeline by choosing the strongest validation checkpoint for each benchmark category," meaning no external model training is needed; "(2) these teachers are derived from the same SFT initialization, they share the same tokenizer and vocabulary as the student, reducing distribution shift and avoiding additional alignment issues," meaning the distillation signal is perfectly aligned with the student's output space; and "(3) MOPD provides a dense token-level training advantage, which is especially useful compared with sparse outcome rewards."

The MOPD objective. Let $\pi_{\text{inf}}$ denote the student policy used for response generation in the inference engine, and $\pi_{\text{train}}$ denote the student policy optimized by the training engine. The two are separated because generation typically uses a faster inference-optimized deployment while training updates a separate copy. For each prompt $x$ , a response $y = (y_1, \ldots, y_T) \sim \pi_{\text{inf}}(\cdot | x)$ is sampled. A domain teacher $\pi_{\text{domain}_i}$ is selected for that example, where $\text{domain}_i$ indicates the capability domain associated with the chosen teacher (math, RLHF, or multi-domain). Writing $s_t = (x, y_{<t})$ for the decoding state at step $t$ , the token-level distillation advantage is:

$a^{\text{MOPD}}_t = \log \pi_{\text{domain}_i}(y_t | s_t) - \log \pi_{\text{train}}(y_t | s_t)$

where:

$\log \pi_{\text{domain}_i}(y_t | s_t)$ is the log-probability the domain teacher assigns to the sampled token $y_t$ given the prompt and previously generated tokens,
$\log \pi_{\text{train}}(y_t | s_t)$ is the log-probability the current training policy assigns to the same token.

What it computes. For each token in the generated response, this computes the difference in log-probability between the domain teacher and the current training policy. If the teacher assigns higher probability to the token than the training policy, $a^{\text{MOPD}}_t > 0$ , meaning the training policy is under-confident on this token relative to the teacher and should increase its probability. If the teacher assigns lower probability, $a^{\text{MOPD}}_t < 0$ , meaning the training policy is over-confident and should decrease its probability. The advantage is computed only on the student-sampled token (not over the full vocabulary), making it computationally efficient — no full softmax over the vocabulary is needed for the teacher's distribution.

Why this form. This is reverse-KL distillation: the advantage term $\log \pi_{\text{teacher}} - \log \pi_{\text{student}}$ is the gradient of the reverse KL divergence between the teacher and student distributions at the sampled token. Reverse KL is mode-seeking (it penalizes the student for placing probability mass where the teacher does not, encouraging the student to concentrate on the teacher's high-probability regions), which is the right objective for capability distillation because we want the student to recover the teacher's strong behaviors, not to match the teacher's full distribution including its low-probability mistakes. Forward KL ( $\log \pi_{\text{student}} - \log \pi_{\text{teacher}}$ ) would be mean-seeking, trying to cover the teacher's entire distribution including tail behaviors, which would dilute the capability signal.

Truncated importance weighting. Because responses are sampled under $\pi_{\text{inf}}$ but optimized under $\pi_{\text{train}}$ , there is a potential distribution mismatch if the inference engine and training engine use slightly different policies (e.g., due to quantization, different decoding parameters, or asynchronous updates). To account for this, importance weights are computed:

$r_t = \frac{\pi_{\text{train}}(y_t | s_t)}{\pi_{\text{inf}}(y_t | s_t)}, \quad w_t = \text{sg}[r_t] \; \mathbf{1}[\epsilon_{\text{low}} \leq r_t \leq \epsilon_{\text{high}}]$

where:

$r_t$ is the per-token ratio of training-policy probability to inference-policy probability for the sampled token,
$\text{sg}[\cdot]$ denotes stop-gradient — the ratio is computed but not differentiated through,
$\mathbf{1}[\epsilon_{\text{low}} \leq r_t \leq \epsilon_{\text{high}}]$ is an indicator that clips the weight to 0 if the ratio is outside the range $[\epsilon_{\text{low}}, \epsilon_{\text{high}}]$ ,
$\epsilon_{\text{low}} = 0.5$ and $\epsilon_{\text{high}} = 2.0$ .

What it computes. For each token, the importance weight measures how much more or less likely the token is under the training policy compared to the inference policy. If $r_t \approx 1$ , the two policies agree. If $r_t \gg 1$ , the training policy assigns much higher probability to this token than the inference policy did when generating it — this is a warning sign that the advantage signal might be unreliable because the token wouldn't have been sampled under the training policy. If $r_t \ll 1$ , the training policy considers this token much less likely. The truncation to $[0.5, 2.0]$ removes tokens where the mismatch is extreme, preventing those tokens from contributing to the gradient. The stop-gradient on $r_t$ ensures the importance weight doesn't influence the gradient of the training policy — it only gates whether the distillation advantage is applied.

The surrogate loss. The final MOPD objective is:

$\mathcal{L}_{\text{MOPD}} = -\mathbb{E}_{x \sim \mathcal{D}, y \sim \pi_{\text{inf}}(\cdot|x)} \left[ \frac{1}{|\mathcal{V}(y)|} \sum_{t \in \mathcal{V}(y)} w_t \; \text{sg}\!\left[a^{\text{MOPD}}_t\right] \log \pi_{\text{train}}(y_t | s_t) \right]$

where:

$\mathcal{V}(y)$ is the set of valid response tokens retained by a token mask (excluding padding, special tokens, and tokens from the prompt),
$|\mathcal{V}(y)|$ is the number of valid tokens in the response,
$\text{sg}[a^{\text{MOPD}}_t]$ means the distillation advantage is treated as a fixed weight (stop-gradient) — the loss only backpropagates through $\log \pi_{\text{train}}(y_t | s_t)$ .

What it computes. For each valid token in the generated response, the training policy's log-probability $\log \pi_{\text{train}}(y_t | s_t)$ is weighted by (a) the importance weight $w_t$ (gating whether the token is reliable) and (b) the stopped-gradient distillation advantage $a^{\text{MOPD}}_t$ (how much the teacher prefers this token relative to the current training policy). The minus sign makes this a minimization of negative weighted log-probability, equivalent to maximizing the probability of tokens the teacher favors. The normalization by $|\mathcal{V}(y)|$ ensures responses of different lengths contribute equally to the batch loss.

Why this form. The stop-gradient on the distillation advantage is a critical design choice. If the advantage were differentiated through, the training objective would try to simultaneously (a) increase $\log \pi_{\text{train}}(y_t|s_t)$ to match the teacher and (b) change $\pi_{\text{train}}$ in ways that change $a^{\text{MOPD}}_t$ , creating a moving-target optimization. By stopping the gradient on $a^{\text{MOPD}}_t$ , MOPD becomes a simple weighted maximum-likelihood problem at each step: given that we know the teacher prefers this token with strength $a^{\text{MOPD}}_t$ (fixed), adjust $\pi_{\text{train}}$ to increase its probability. This is analogous to policy gradient with a learned advantage function, but the advantage is provided by the teacher rather than estimated from rewards.

Hyperparameters and training dynamics. The main MOPD configuration uses a rollout size of 4 (4 responses per prompt) and 128 prompts per update, giving an effective batch size of 512 responses. The paper notes: "In later experiments, we find that using 512 prompts with rollout size 1 yields slightly more stable optimization while producing similar final results." The learning rate is $2 \times 10^{-6}$ with linear warm-up over the first 30 optimization steps, starting from $2 \times 10^{-7}$ . Training converges within 40–50 steps.

The warm-up is specifically motivated by gradient norm observations: "gradient norms are substantially larger at the beginning of training and decrease rapidly after the warm-up phase (Fig. 3(b))." This suggests that when MOPD begins, the student policy is far from the teacher in some domains, creating large distillation advantages and correspondingly large gradients. The warm-up prevents these large initial gradients from destabilizing training.

Training prompt sampling. The prompts for MOPD are "sampled accordingly from the RL training data pools (RLHF, IF-RL, and Multi-domain), as well as from AceReason-Math for math." When a prompt from the math pool is sampled, the math teacher ( $\pi_{\text{domain}_i} = \text{SFT checkpoint}$ ) is used; when an RLHF prompt is sampled, the RLHF teacher is used; when an IF-RL or Multi-domain prompt is sampled, the multi-domain teacher is used. This per-prompt teacher assignment means the student model receives interleaved distillation signals from all three teachers within the same training run, learning to balance their preferences.

Training efficiency advantage. The paper provides specific comparisons in Figure 3 and Table 3 to demonstrate MOPD's advantage over GRPO. Starting from the same initial checkpoint on math-only training (Figure 3(c)): GRPO improves AIME25 from 89.9 to 91.0 after 25 steps, while MOPD reaches 92.0 within 30 steps and "recovers teacher-level performance." On ArenaHard v2 (Table 3): starting from an initial model with Hard Prompt 71.5 and Creative Writing 40.6, MOPD achieves Hard Prompt 85.5 and Creative Writing 71.0 in 52 steps, while RLHF training requires 160 steps to reach only 80.7 and 71.2. This 3× faster convergence is attributed to the dense token-level advantage providing per-token guidance rather than the sparse sequence-level reward of GRPO.

3.4.6 Reinforcement Learning from Human Feedback (RLHF) — Stage 4

After MOPD has consolidated capabilities from the domain teachers, the first specialized post-MOPD stage focuses on human preference learning (Section 4.5). This stage improves creative writing and non-verifiable problem-solving (where ground-truth answers don't exist), as measured by ArenaHard v2, while maintaining performance across other domains.

Generative reward model (GenRM). The RLHF stage uses Qwen3-235B-A22B-Thinking-2507 as a generative reward model, "trained via the HelpSteer3 framework." The GenRM works as follows: given a conversation history, a user request, and two candidate responses (A and B), the GenRM "first reasons through the strengths and weaknesses of each response before producing individual helpfulness scores and a final comparative ranking." This means the reward signal is not just a scalar preference (A > B or B > A), but includes an explicit reasoning trace that provides richer supervision — though in GRPO, only the final ranking matters for the reward. The GenRM is a separate 235B MoE model with 22B activated parameters, substantially larger than the 30B/3B student model, providing a strong supervisory signal.

Pair-wise comparison reward aggregation. Unlike RLVR stages where each rollout gets an independent verifiable reward, RLHF uses pair-wise comparisons: "we adopt pair-wise comparisons for all pairs of rollouts per prompt." For a prompt with $G = 16$ rollouts, this means $\binom{16}{2} = 120$ pair-wise comparisons, each producing a win/loss/tie judgement from the GenRM. The paper then "aggregates the reward scores in the same way as NVIDIA Nano-v3 RLHF training," though the exact aggregation formula is not specified in the paper — it likely involves computing an Elo-like rating or win-rate for each rollout from the pairwise comparisons, which becomes the $r_i$ in the GRPO advantage formula.

Length-normalized reward and quality-gated conciseness bonus. A specific failure mode in RLHF is that models learn to produce longer, more verbose responses because GenRMs tend to prefer them (longer responses appear more thorough, even if the extra length is padding). The paper applies two mechanisms to counteract this:

Length-normalized reward adjustment: the raw reward from the GenRM is adjusted to penalize excessive length. The exact formula is not given, but the mechanism "encourages shorter responses without sacrificing quality."
Quality-gated conciseness bonus: a bonus reward is added for concise responses, but only if the response quality is already high. This prevents the model from learning to produce trivially short (and low-quality) responses to exploit the conciseness bonus.

Thinking mode exclusivity with KL penalty. Unlike Cascade 1 (which trained RLHF in both thinking and non-thinking modes), Cascade 2 trains RLHF "exclusively in the thinking mode." The paper reports: "while incorporating both thinking and non-thinking modes can improve training convergence and yield slight gains on evaluation benchmarks, we observe a significant degradation in instruction-following performance. The resulting drop is substantial enough that the gains obtained in the earlier RLVR stage cannot be fully recovered." This is a key empirical finding that justifies the Cascade ordering: thinking-mode RLHF preserves IF-RL gains, while mixed-mode RLHF destroys them. The KL loss coefficient is set to 0.03 during RLHF (unlike the 0.0 used in other stages), specifically to "keep the model capabilities on other domains." This small KL penalty acts as a regularizer that prevents the policy from diverging too far from the pre-RLHF checkpoint (which already has strong multi-domain capability from MOPD), ensuring that RLHF's preference optimization doesn't undo the distillation gains.

Training details. Batch size 128, 16 rollouts per prompt, temperature 1.0, top-p 1.0, maximum response length 16K tokens (significantly shorter than the 49K+ used in other stages, reflecting that RLHF focuses on shorter-form human preference tasks), no overlong filtering applied, AdamW with learning rate $3 \times 10^{-6}$ , entropy loss coefficient 0, KL loss coefficient 0.03, and training runs for approximately 30 steps.

3.4.7 Long-Context RL — Stage 5

Following RLHF, a domain-specific stage targets long-context understanding and reasoning (Section 4.6). This stage is notable for what is explicitly excluded: the paper "restricts this phase to long-context datasets only" because "incorporating other domains during long-context RL negatively affects performance on unrelated benchmarks."

LLM judge for question answering. Unlike code RL (which uses binary execution feedback) or math RL (which uses answer verification), long-context QA tasks have no automatic ground truth. The paper uses Qwen3-235B-A22B-Instruct-2507 as an LLM judge: for each question about a long document, the judge evaluates whether the model's answer is correct based on the document content. This is a form of RLVR where the reward is a binary correctness judgement from the judge model (or possibly a continuous score; the paper only says "LLM judge to evaluate model rollouts for question answering tasks").

Input/output length constraints. Input sequences are limited to 32K tokens, and the maximum sequence length is set to 49K tokens. This means the model sees the first 32K tokens of the long document (truncated) and has 17K tokens for reasoning and answer generation. This is a pragmatic choice for training efficiency — the full documents in benchmarks like AA-LCR average 100K tokens, so training on full documents would be prohibitively slow. The model presumably generalizes to longer contexts at inference time through the long-context SFT data and the representations learned during this stage.

Training duration sensitivity. The paper reports that training "runs for approximately 30 steps, as we observe a rapid increase in generated tokens beyond that point." This is an important empirical observation: long-context RL initially improves reasoning quality, but extended training causes the model to produce excessively long responses, likely because the model learns to "think longer" to improve its judge score, even when the extra tokens don't improve answer quality. The 30-step cutoff is an early stopping decision based on monitoring response length, not just reward.

Training details. Batch size 128, 16 rollouts per prompt, temperature 1.0, top-p 1.0, AdamW with learning rate $3 \times 10^{-6}$ , entropy loss coefficient 0, KL loss coefficient 0. The absence of a KL penalty (unlike RLHF's 0.03) suggests that long-context RL is less likely to cause distribution shift that harms other domains, consistent with the earlier finding that this stage is naturally isolated from other capabilities.

3.4.8 Code RL — Stage 6

The competitive coding RL stage (Section 4.7) represents a significant scale increase over Cascade 1's code RL, with longer responses, more rollouts, and more aggressive filtering.

Data curation and filtering. The training data draws from the Nemotron-Cascade coding corpus, containing prompts from AtCoder, Codeforces, and AIZU with robust test cases. The key filtering decision is: "we aggressively filter out prompts that GPT-OSS-120B solves correctly in all 8 of 8 rollouts, yielding a compact final set of only 3.5K samples." This means only prompts that are hard enough that even a strong 120B model sometimes fails are retained. The rationale is that "high-difficulty prompts paired with strong test cases are critical for further boosting model performance" — easy prompts provide no learning signal because the model already solves them, and prompts without strong test cases risk reward hacking (the model learning to produce code that passes weak tests but is actually incorrect).

Extended generation budget. The maximum response length during Code RL is increased to 118K tokens, and the number of rollouts per sample is increased to 16. This contrasts with typical RL stages that use 16K–49K tokens, and is specifically motivated by the need to "better capture sparse reward signals on extremely difficult problems that require long reasoning traces." In competitive programming, hard problems may require the model to explore multiple algorithmic approaches, analyze edge cases, and write complex code — all of which consume tokens. The 118K budget gives the model space to do this without truncation.

Binary reward function and on-policy constraint. The paper "adopts the strict binary reward function to avoid potential reward hacking." This means the reward is 1.0 if the generated code passes all test cases, and 0.0 otherwise — no partial credit for passing some tests. This is important because continuous rewards (e.g., fraction of tests passed) can lead to reward hacking: the model might learn to produce code that passes the easy public tests while failing the hard hidden tests. The binary reward eliminates this by requiring all-or-nothing correctness. Training is "fully on-policy for stability," meaning rollouts are always generated from the current policy before each gradient step.

Asynchronous verification infrastructure. The batch configuration (128 prompts × 16 rollouts = 2,048 code executions per RL step) requires non-trivial infrastructure: "to support the resulting verification throughput, we deploy an asynchronous reward verification server that completes each batch in 427.2 seconds across 384 CPU cores." This is a practical engineering detail that highlights the computational demands of code RL: each of the 2,048 generated code samples must be executed against test cases in a sandboxed environment, and the results (pass/fail) must be returned before the gradient update. The asynchronous server parallelizes this across 384 cores, achieving approximately 4.8 executions per second per core.

Training details. Batch size 128, 16 rollouts per prompt, AdamW with learning rate $3 \times 10^{-6}$ , temperature 1.0, top-p 0.95 (slightly lower than the 1.0 used in other stages, introducing a small amount of nucleus sampling to encourage diversity), and training runs for 22 steps. The short training duration (22 steps) with a small dataset (3.5K prompts) suggests that Code RL provides a concentrated, high-quality signal that rapidly improves competitive programming performance without overfitting.

3.4.9 Software Engineering Reinforcement Learning (SWE RL) — Stage 7

The final Cascade RL stage targets software engineering capabilities through two complementary approaches: agentless code repair RL and execution-based agentic RL within interactive scaffolds (Section 4.8).

Agentless RL — code repair as a standalone skill. The first SWE RL component trains the model to repair buggy code without an interactive environment. The data follows the same source as Cascade 1: for each SWE instance, prompts are constructed using both the "golden localization" (the known location of the bug) and the top-5 retrieved localizations from a code embedding model, and relatively easy samples are filtered out. The reward model is GPT-OSS-120B, which evaluates the quality of code repairs by comparing the model's patch to the correct fix.

A specific training detail matters: "During training, we mask the loss for prompts for which none of the rollouts receives a reward greater than 0.5." This means if a prompt is so hard that all 16 generated patches are poor (reward ≤ 0.5), that prompt is excluded from the gradient update. The rationale is that "these difficult prompts degrade the stability and effectiveness of agentless SWE RL training" — when all rollouts are bad, the group-normalized advantage is approximately zero and the gradient is pure noise, potentially destabilizing the optimizer.

The paper demonstrates an important finding in Table 4: agentless RL training (which focuses purely on code repair, not on interactive debugging) transfers to agentic scaffolds. After agentless RL, the model improves from avg@4 41.9% to 44.3% on Agentless Mini evaluation, but also from avg@4 49.8% to 50.8% on OpenHands evaluation. This suggests that "improving models' code repair capability alone can generalize across different scaffolds," because the fundamental skill of localizing and fixing bugs is scaffold-independent — the agentic scaffold provides tools and environment interaction, but the core repair reasoning is the same.

Training details for agentless RL. Batch size 128 prompts × 16 rollouts = 2,048, maximum sequence length 98,304 tokens, learning rate $3 \times 10^{-6}$ with AdamW, temperature 1.0, top-p 1.0, and training runs for 40–50 steps.

Execution-based agentic SWE RL. The second SWE RL component trains the model to operate within the OpenHands scaffold framework, which coordinates repository interaction, tool calling, code editing, and test execution. Unlike agentless RL (where the model only produces a patch), agentic RL optimizes the entire multi-turn problem-solving trajectory — the model must interact with the environment, explore the codebase, formulate hypotheses, and iteratively debug.

Environment and reward. The training environments integrate OpenHands frameworks that "provide structured tool usage, repository interaction, and iterative patch generation." Each episode corresponds to resolving a software issue from SWE-Gym or R2E-Subset. The agent operates inside an instrumented repository with tools for file inspection, search, code editing, and test execution. Candidate patches are executed within the environment, which returns "verifiable signals from compilation results and unit test outcomes, enabling automatic reward computation without human annotation." The reward is deterministic: the model either produces a patch that passes all tests (reward = 1) or it doesn't (reward = 0), reflecting the "strict binary reward" philosophy from Code RL.

Scale and interaction budget. The configuration is: batch size 1,024 (16 prompts × 64 rollouts — notably more rollouts per prompt than any other stage, reflecting the high variance of agentic trajectories), maximum context length 256K tokens (the largest in the pipeline, necessary to hold the full conversation history, tool outputs, and code context across up to 200 turns), and up to 200 interaction turns (allowing the agent to explore, fail, and retry multiple times within a single episode).

Data filtering by difficulty. Similar to Code RL, the paper applies aggressive filtering: "instances for which all rollouts pass verification (100% accuracy), indicating overly simple problems, are removed from the dataset. For instances where none of the rollouts pass verification (0% accuracy), indicating extremely difficult problems, we randomly discard 90% of such cases to reduce their proportion in the training data." The filtering serves two purposes: removing trivially easy problems (no learning signal) and down-weighting impossibly hard problems (pure noise, destabilizing). The 90% random discarding of unsolved instances — rather than 100% removal — ensures some very hard problems remain in the training distribution, preventing the model from losing the ability to attempt them.

Training details for execution-based agentic RL. Table 10 specifies: 16 prompts per step, 64 rollouts, temperature 0.8 (lower than other stages, likely to reduce variance in agent trajectories), maximum sequence length 256K tokens, maximum 200 turns, learning rate $3 \times 10^{-6}$ with 10-step warm-up and cosine decay to 0, AdamW optimizer. The lower temperature (0.8 vs. 1.0) is a deliberate choice for agentic RL: at temperature 1.0, the 64 rollouts per prompt would explore wildly different strategies, making it difficult to identify which actions lead to success; temperature 0.8 provides enough exploration to try variations while keeping trajectories somewhat focused.

3.4.10 Summary of Design Choices and Their Justifications

Cascade ordering. The specific sequence (IF-RL → Multi-domain → MOPD → RLHF → Long-context → Code → SWE) is determined by a "dynamic function of the model's underlying behaviors and learning trajectories," not a fixed recipe. The paper provides specific empirical justifications for each placement: IF-RL first because it creates a strong teacher for MOPD and later stages can recover alignment regressions; RLHF after MOPD because GenRM-based optimization can trade off against instruction-following, but MOPD has already consolidated IF-RL gains; Code and SWE last because they require extreme context lengths and complex environment interactions that could destabilize earlier training.

MOPD at the midpoint. The strategic placement of MOPD after the first two RL stages but before the later specialized stages reflects the observation that capability drift accumulates gradually across stages and is best addressed by a periodic re-balancing step. Placing it at the exact midpoint (after 2 stages, before 4 subsequent stages) is likely an empirical choice — the paper doesn't ablate alternative placements, but the logic is that the first two stages establish broad foundational capabilities (instruction-following, STEM, tool use) whose peak performance is worth preserving, while the later stages are specialized refinements that benefit from starting from a balanced foundation.

GRPO with no KL penalty in most stages. The removal of the KL divergence term from all stages except RLHF (where KL = 0.03) reflects the paper's finding that strict on-policy training (single gradient step per batch of fresh rollouts) is sufficient regularization for RLVR stages. RLHF is the exception because the GenRM provides a learned reward signal that can be exploited (the model could learn to produce outputs that maximize the GenRM's score rather than genuinely improving), so a small KL penalty anchors the policy to the pre-RLHF checkpoint.

Thinking-mode exclusivity. Across all RL stages, the model is trained exclusively in thinking mode (generating explicit reasoning before answering). The paper reports specific failures when incorporating non-thinking mode: in IF-RL, thinking mode "yields higher accuracy on instruction-following benchmarks"; in RLHF, mixing modes "causes significant degradation in instruction-following performance [...] substantial enough that the gains obtained in the earlier RLVR stage cannot be fully recovered." This suggests that the thinking-mode format (explicit chain-of-thought before the answer) is not just a stylistic choice — it fundamentally changes the learning dynamics, likely because the model learns to use the reasoning section for self-verification and error correction during RL.

Binary rewards for verifiable domains, GenRM for subjective domains. Code RL, agentless SWE RL, and execution-based agentic SWE RL all use strict binary rewards (passes all tests = 1, otherwise = 0). Math RLVR uses correctness verification against ground-truth answers. These are unambiguous and cannot be gamed. RLHF and long-context RL use learned reward models (GenRM and LLM judge), which are necessary for subjective tasks but carry risk of reward hacking — the model could learn to produce outputs that please the judge rather than genuinely improve. The KL penalty in RLHF and the early stopping (30 steps) in long-context RL are defenses against this risk.

Aggressive filtering of easy and impossible prompts. Code RL retains only prompts that GPT-OSS-120B fails at least once in 8 attempts (3.5K out of a much larger set). Agentic SWE RL removes instances where all or none of the 16 rollouts pass. Agentless SWE RL masks loss for prompts where all rollouts score ≤ 0.5. Dynamic filtering in IF-RL removes prompts where all rollouts are all-correct or all-incorrect. This consistent filtering philosophy across stages reflects a key insight: RL training signal quality depends on the model being uncertain — prompts that are trivially easy or impossibly hard provide near-zero advantage and contribute only noise to the gradient.

4. Key Insights and Innovations

Innovation 1: Multi-Domain On-Policy Distillation as a Stabilization Mechanism Within Sequential RL

The paper's most intellectually distinctive contribution is reframing knowledge distillation — typically a post-hoc compression technique (distill a large model into a small one) or a single-teacher guidance signal — as a proactive stabilization mechanism integrated into the sequential RL pipeline. This is not just "distillation applied to RL." It is a specific recognition that sequential domain-wise RL naturally produces capability-diverse checkpoints whose peak performances are transient, and that a dense token-level distillation stage placed at the midpoint of the cascade can simultaneously recover these peaks across domains.

The field's default assumption, seen in prior on-policy distillation work (Agarwal et al., 2024; Gu et al., 2024; Xiao et al., 2026; Zeng et al., 2026), has been that distillation uses a single external teacher — typically a larger, more capable model — to lift a weaker student. The conceptual limitation is obvious: a single teacher cannot be simultaneously optimal at mathematics, instruction-following, and human preference alignment. A math-specialized teacher may produce poor conversational responses; a general-purpose teacher may have weaker coding capability. Cascade 2's framework dissolves this problem by observing that the training process itself generates the best teacher for each domain — the SFT checkpoint for math, a separately RLHF-optimized checkpoint for human preference, the post-IF-RL + Multi-domain checkpoint for instruction-following — all sharing the same tokenizer and vocabulary, all derived from the same initialization, and all naturally arising from the very pipeline that would otherwise progressively overwrite their capabilities.

This reframes distillation not as a way to borrow external knowledge, but as a checkpoint-preservation mechanism: the MOPD stage in effect says "the model at step 250 of IF-RL was better at instruction-following than the model at step 500 will be; let's use that checkpoint to inject its peak capability back into the later model." The dense per-token reverse-KL advantage provides precision that sparse GRPO rewards cannot: when a response should be formatted according to a specific constraint, the IF-RL teacher assigns high probability to constraint-conforming tokens and low probability to violations, and MOPD propagates this token-level signal back into the student policy irrespective of the current task's reward function.

Significance beyond performance. MOPD is not fundamentally about achieving higher numbers — though Figure 3(c) shows it recovers and exceeds teacher performance on AIME25, and Table 3 shows it reaches ArenaHard Hard Prompt 85.5 in 52 steps versus RLHF requiring 160 steps to reach 80.7. The deeper significance is that it eliminates the zero-sum tension inherent in multi-objective RL post-training. Prior to this work, the implicit assumption was that training a model to excel at code would necessarily trade off against instruction-following, or that RLHF would dilute mathematical reasoning. MOPD demonstrates that these trade-offs are artifacts of inadequate knowledge preservation, not fundamental capacity limitations of the model. By periodically re-injecting domain-specific peak capabilities, the pipeline can maintain a Pareto frontier that sequential RL alone cannot sustain.

Comparison to prior Cascade RL. Nemotron-Cascade 1 (Wang et al., 2025) already showed that careful sequential ordering substantially reduces catastrophic forgetting compared to arbitrary-order or blended approaches. But Cascade 1 had no active mechanism for recovering already-lost capability: if a math benchmark regressed during later RL stages, that regression was permanent. MOPD is the first mechanism to actively reverse regressions within the Cascade framework, converting what was previously a one-way ratchet (each stage potentially losing prior gains) into a self-correcting loop.

Is this fundamental or incremental? The mechanism of MOPD — reverse-KL distillation with truncated importance weighting — is a modest technical extension of established on-policy distillation techniques. The conceptual framing — multi-teacher checkpoint preservation as a proactive Cascade RL stage — is a fundamental shift in how post-training pipelines should be architected. I call it fundamental because it changes the design principle from "order stages to minimize forgetting" to "periodically consolidate capabilities from internal peak checkpoints," which is a higher-level architectural insight that applies to any multi-stage RL pipeline, not just Cascade RL specifically.

Innovation 2: Domain-Ordering as a Dynamic Function of Model Behavior, Not a Static Recipe

The paper's explicit discussion of Cascade RL ordering (Section 4.1.1) elevates what could have been a report of "here's the order we used" into a diagnostic principle with transferable implications. The central insight is that the optimal ordering of RL stages is "a dynamic function of the model's underlying behaviors and learning trajectories" that changes as SFT data quality improves and RL environments become harder — and that Cascade 2's ordering differs from Cascade 1's precisely because these factors have fundamentally shifted.

This matters because the dominant assumption in multi-stage post-training (to the extent such work exists) has been that the ordering is either arbitrary, guided by heuristics, or determined by a fixed priority ranking (train highest-priority capability first or last). Cascade 2 provides specific, falsifiable criteria for ordering decisions:

IF-RL is placed first because "IF-RL can negatively impact human alignment capabilities" and "our subsequent GenRM-based RLHF has a negligible impact on instruction following scores." The logic is: place the stage that damages other capabilities earliest, then use later stages to recover what was damaged — but do not place a damaging stage late where there is no subsequent recovery opportunity.
RLHF is placed after MOPD because the GenRM optimization "can trade off against instruction-following behavior." MOPD consolidates IF-RL gains before RLHF can dilute them, so RLHF's negative side effects are bounded by the quality of the MOPD-consolidated model.
Long-context RL is placed late because "incorporating other domains during long-context RL negatively affects performance on unrelated benchmarks," suggesting that long-context training is particularly prone to causing interference, and isolating it prevents collateral damage.
Code RL and SWE RL are placed last because they require extreme context lengths (118K and 256K tokens, respectively) and complex environment interactions that could destabilize earlier training if introduced too soon.

Significance beyond this paper. The Cascade 2 ordering is presented not as the optimal ordering, but as the optimal ordering given their specific SFT data quality and RL environment complexity. The paper explicitly notes that "these advancements have fundamentally altered the model's behavioral dynamics, which require us to adopt a different order." This reframes the ordering problem as an empirical diagnostic question: given the current state of the model, which domain conflicts with which? This encourages future work to measure inter-domain interference explicitly rather than relying on intuition or fixed templates — a more scientific approach to pipeline design.

Is this fundamental or incremental? The observation that ordering matters is not new — Cascade 1 already made this case. What is new is the explicit articulation of the interference dynamics that determine ordering: which stages damage which other capabilities, and how later recovery stages can compensate. This transforms ordering from a trial-and-error heuristic into a diagnostic framework. I consider this a significant refinement of an existing principle, not a wholly new paradigm, but it is the kind of refinement that enables practitioners to adapt the Cascade framework to new models and domains without blind search.

Innovation 3: Agentless-to-Agentic Skill Transfer as an Empirical Principle for SWE Training

Table 4 contains a finding whose implications extend beyond the specific numbers: agentless code repair RL — training a model to fix bugs given the bug location, without interactive environment access — transfers positively to agentic SWE performance in OpenHands. This is a non-obvious result that challenges the natural assumption that agentic capabilities must be trained in agentic environments.

Specifically, agentless RL improves OpenHands avg@4 from 49.8% to 50.8% and pass@4 from 64.2% to 65.0%, while simultaneously improving Agentless Mini avg@4 from 41.9% to 44.3%. This means that improving the core code repair skill in isolation produces gains that generalize across scaffolds, consistent with the observation from Yang et al. (2026). The mechanism is straightforward in hindsight: whether the model operates inside OpenHands or a simpler agentless scaffold, the fundamental challenge — read a problem description, identify what's wrong with buggy code, produce a correct fix — is the same. Agentic scaffolding provides tools for navigation and testing, but if the underlying repair reasoning is weak, no amount of tool use can compensate. By strengthening the repair capability first (through agentless RL, which is simpler and more computationally efficient), the model becomes more effective when later deployed in the full agentic environment.

This finding has practical significance for the design of SWE training pipelines. Agentic RL (256K context, 200 turns, 16 prompts × 64 rollouts per step) is enormously expensive. Agentless RL (98K context, single-turn repair) is substantially cheaper. If agentless RL transfers positively to agentic evaluation, then the optimal pipeline should allocate more compute to agentless training and use agentic RL primarily for learning tool use and environment navigation — skills that cannot be learned agentlessly — rather than for learning code repair fundamentals.

Is this fundamental or incremental? The finding is a specific empirical result, not a new theoretical framework. But it has practical implications comparable to the well-known finding that SFT on reasoning traces transfers to improved RL training — it identifies a previously underappreciated transfer pathway that can reduce compute requirements for building SWE agents. I consider it an important empirical contribution within the specialized SWE subfield, but not a cross-domain innovation of the same magnitude as MOPD.

5. Experimental Analysis

Evaluation Methodology

Dataset. The primary evaluation suite is not a single dataset but a collection of benchmarks spanning reasoning, knowledge, alignment, agentic, long-context, and multilingual capabilities, described in Appendix A. Key reasoning benchmarks include AIME 2025 (30 problems), AIME 2026 (30 problems), HMMT February 2025 (30 problems), IMO-AnswerBench (400 problems), IMO-ProofBench (60 problems), IMO 2025 (6 problems), LiveCodeBench v6 (454 problems), LiveCodeBench Pro 25Q1 and 25Q2 (166 and 167 problems respectively), and SciCode (338 subproblems). For competitive programming, 40 Div.1/Div.2 Codeforces rounds (2501–2507) are used for ELO estimation. Alignment benchmarks include ArenaHard v2 (750 prompts), IFBench (294 prompts), and Scale AI Multi-Challenge (273 conversations). Agentic benchmarks include SWE-bench Verified (500 samples), τ2-Bench (278 total examples across airline/retail/telecom), Terminal Bench 2.0 (89 tasks), and BFCL v4. Long-context benchmarks include AA-LCR (100 questions), LongBench v2 (503 questions), and NIAH@1M (400 instances). Knowledge benchmarks include MMLU-Redux (3,000 questions), MMLU-Pro (12,000+ questions), GPQA-Diamond (198 questions), and HLE (2,158 text-only examples). The IMO 2025 problems are evaluated by a human expert (IMO 2015 Gold medalist) except Problem 2, which uses an LLM grader with reference solution and marking schema from ProofBench due to the extensive analytic geometry approach.
Base model(s). All Cascade RL post-training starts from Nemotron-3-Nano-30B-A3B-Base, a 30B total parameter MoE model with 3B activated parameters. The authors state this model was chosen because it is "representative of the capabilities of many contemporary LLMs" while being compact enough to demonstrate that strong post-training can compensate for moderate pretraining scale. The SFT checkpoint is trained from this base model, and all subsequent RL stages build on the SFT model. For teacher models in MOPD, the SFT checkpoint itself serves as the math teacher, a separately RLHF-optimized checkpoint (from the SFT model, not from the Cascade pipeline) serves as the RLHF teacher, and a checkpoint after IF-RL + Multi-domain RL serves as the multi-domain teacher.
Metrics. The primary metric varies by benchmark: accuracy (% correct) for math, code, knowledge, alignment, and agentic tasks; ELO rating for Codeforces performance; avg@k (expected pass@1 averaged over k generations) for AIME, HMMT, LiveCodeBench, GPQA-Diamond, ArenaHard, IFBench, and SWE-bench; exact match for MMLU-Redux and MMLU-Pro; XCOMET-XXL score for WMT24++ translation quality. For IMO 2025, the metric is points out of 42 (7 per problem) assigned by a human expert. For IOI 2025, the metric is points out of 600 using official OnlineJudge evaluation. For IMO-ProofBench, a custom aggregation rule is used: if any judge assigns a score of 0, the final score is 0; otherwise, return the mean score across 64 grading attempts — this rule was calibrated to reduce discrepancy with human ratings from 11.9 points to 4.2 points on DeepSeek-Math-V2.
Baselines. The paper compares against three categories of models. (1) Internal predecessors: Nemotron-3-Nano-30B-A3B (the same base model post-trained with the prior NVIDIA Nano-v3 pipeline, not Cascade RL) and Nemotron-3-Super-120B-A12B, both evaluated using the authors' own infrastructure. (2) Comparable open models: Qwen3.5-35B-A3B (2026-02-24 release), evaluated with official numbers when available or with recommended settings otherwise. (3) Frontier open and proprietary models for competitive benchmarks: DeepSeek-V3.2-Speciale-671B-A37B, Kimi-K2.5-1T-Thinking, Qwen3.5-397B-A17B, GPT-OSS-120B, Gemini Deep Think (IMO Gold), GPT-5.2-Thinking, Gemini 3 Pro, GPT-o4-mini, and others as listed in individual benchmark tables (Table 1, Table 5, Table 6). For Codeforces ELO, the paper simulates participation in 40 real contests and estimates rank and rating relative to human participants.
Generation budget / compute accounting. For most benchmarks, the generation budget is specified per benchmark: avg@64 for AIME/HMMT, avg@16 for IMO-AnswerBench, avg@8 for LiveCodeBench/GPQA-Diamond/IFBench, avg@4 for SWE-bench Verified and LongBench v2, avg@1 for MMLU-Redux/MMLU-Pro/NIAH@1M. The thinking budget (maximum response length) is 131K tokens for AIME/HMMT (extended to 256K for IMO-AnswerBench), 128K for LiveCodeBench v6 and Pro, 32K for IFBench and ArenaHard, 256K for SWE-bench Verified and agentic SWE RL. For test-time scaling on IMO, the paper uses a generate-verify-refine pipeline adapted from DeepSeek-Math-V2 with 128 proof generations, 64 verifications per proof, top 32 proofs selected for refinement, 8 verification analyses per proof, 4 refined proofs per round, up to 8 rounds, with reduced budgets for easier problems (32 generations, 16 verifications, top 8, 2 rounds). For IOI, up to 50 rounds × 40 candidate solutions = 2,000 model generations per subtask.
Cross-validation / statistical protocol. For IMO-ProofBench evaluation, the paper uses 64 grading attempts with the ProofAutoGrader prompt and DeepSeek-V3.2-Speciale as judge, applying the aggregation rule described above to reduce noise. The paper does not report confidence intervals, standard errors, or cross-validation splits for most benchmark results. The ELO rating analysis on Codeforces (Appendix D) uses 40 contests with expected penalty consideration, but no uncertainty quantification is provided.

Main Quantitative Results

This section is organized by evaluation domain, reflecting the paper's own structure: competitive reasoning (IMO, IOI, ICPC), math benchmarks, code benchmarks, alignment and instruction-following, and agentic tasks. Table 1 provides the comprehensive summary; detailed results appear in Tables 2, 5, 6, and the individual benchmark descriptions in Appendix A.

Competitive Reasoning: IMO, IOI, and ICPC World Finals

IMO 2025 (Table 2). Nemotron-Cascade-2-30B-A3B scores 35/42 points, solving 5 of 6 problems with perfect 7/7 scores (Problems 1, 3, 4, 5 scored 7/7 by human expert; Problem 2 scored 7/7 by LLM grader due to "extensive analytic geometry approach"). Problem 6 scored 0/7. The model's solutions and expert comments are provided in Appendix E. The paper emphasizes this as Gold Medal-level performance (the second open-weight model to achieve this, after DeepSeek-V3.2-Speciale-671B-A37B), noting that "these results are particularly encouraging, as they suggest that strong olympiad-level mathematical reasoning can emerge from a comparatively compact model when paired with effective inference-time scaling."

IMO-ProofBench (Table 5). Nemotron-Cascade-2-30B-A3B achieves 72.9 overall (92.5 Basic, 53.4 Advanced), compared to 80.2 for DeepSeek-Math-V2-671B-A37B (99.0 Basic, 61.9 Advanced), 76.7 for Gemini Deep Think (IMO Gold) (89.0 Basic, 65.7 Advanced), and 78.6 for the paper's reproduced DeepSeek-Math-V2 score under the same LLM-judge setup. The gap to DeepSeek-Math-V2 is ~8 points despite ~10× fewer active parameters. Figure 4 shows that on the Advanced split, the score improves from 40.7 at round 1 to 53.4 at round 5 of generate-verify-refine, while DeepSeek-Math-V2 reaches 57.7 (reproduced score) and 61.9 (reported human score). The Basic split uses a reduced compute budget (32 generations, 16 verifications, 2 rounds vs. 128/64/up to 8 rounds) for 11 of 30 Advanced problems.

IOI 2025 (Table 2). Nemotron-Cascade-2 achieves 439.28/600 (Gold medal threshold is approximately top 8.3% of participants), with detailed problem scores: Problem 1 (39/100), Problem 2 (88.53/100), Problem 3 (100/100), Problem 4 (100/100), Problem 5 (28.75/100), Problem 6 (83/100). The paper notes that "on Problem 2 which requires designing and optimizing a heuristic algorithm, our pipeline reached over 86 points in just 5 rounds (at most 200 model generations), demonstrating the effectiveness of self-refinement and cross-subtask insights." A score of 507.66 is achievable within 5,000 generations.

ICPC World Finals 2025 (Table 2). Nemotron-Cascade-2 solves 10/12 problems, achieving Gold medal placement (#4 position), with 8 problems (all except A and I) solved within only 100 submissions. The paper generates up to 1,000 solutions per problem and submits them "for official evaluation after initial filtering."

Math Benchmarks

AIME and HMMT (Table 1). Nemotron-Cascade-2-30B-A3B achieves AIME 2025: 92.4 (98.6 with Tool-Integrated Reasoning), AIME 2026: 90.9 (95.0 TIR), HMMT Feb25: 94.6. Compared to Qwen3.5-35B-A3B: 91.9 (AIME 2025), 91.1 (AIME 2026), 89.0 (HMMT); and Nemotron-3-Super-120B-A12B: 90.2 (AIME 2025), 89.8 (AIME 2026), 93.7 (HMMT). Cascade-2 outperforms or matches the 120B model on AIME despite having ~4× fewer total parameters, and substantially exceeds it on HMMT (94.6 vs. 93.7).

IMO-AnswerBench (Table 1). Cascade-2 scores 79.3, compared to Nemotron-3-Nano-30B-A3B (70.4), Nemotron-3-Super-120B-A12B (77.2), and Qwen3.5-35B-A3B (74.8). This 400-problem benchmark with expert-altered past Olympiad problems tests answer-level reasoning, and Cascade-2 shows a substantial 4-point edge over the 120B model.

Code Reasoning Benchmarks

LiveCodeBench and LiveCodeBench Pro (Table 6). This is the most extensively benchmarked code evaluation in the paper. Nemotron-Cascade-2-30B-A3B achieves LiveCodeBench v6: 87.2 (88.4 TIR), placing it above Qwen3.5-397B-A17B (83.6), Kimi-K2.5-1T-Thinking (85.0), and Qwen3.5-35B-A3B (74.6), and competitive with GPT-OSS-120B (87.0) and DeepSeek-V3.2-Speciale (88.7). On LiveCodeBench Pro 25Q2 Medium difficulty, Cascade-2 achieves 27.6 (36.8 TIR), substantially above Qwen3.5-35B-A3B (17.8) and Nemotron-3-Super-120B-A12B (23.2), and approaching GPT-OSS-120B (31.1) and DeepSeek-V3.2-Speciale (43.1). Notably, Cascade-2 achieves above 0% on the Hard split (0.7 without TIR, 2.2 with TIR), a threshold that many models fail entirely (GPT-5.2-high scores 5.9 without TIR on Hard, GPT-o4-mini scores 0.0).

Codeforces ELO (Table 6, Table 11, Table 12). Cascade-2 achieves ELO 2320 (percentile 99.6) without TIR and 2345 (99.7) with TIR, compared to DeepSeek-V3.2-Speciale (2353), Qwen3.5-397B-A17B (2350), and Kimi-K2.5-1T-Thinking (2333). The detailed round-by-round breakdown in Tables 11 and 12 shows the model solving problems across the full difficulty spectrum, from easy Div.2 A problems to challenging Div.1 D/E problems, with particularly strong showings on rounds 1015 (Div.1+2, Rank 4/11206) and 1021 (Div.1, Rank 21/651). The paper notes a remaining weakness: "dealing with problems that require constructive algorithms, interactive manner, and hypothesis-driven ideas."

SciCode (Table 1). Cascade-2 scores 36.4, compared to Nemotron-3-Nano-30B-A3B (33.3), Nemotron-3-Super-120B-A12B (42.1), and Qwen3.5-35B-A3B (38.0). This scientific coding benchmark shows weaker relative performance — Cascade-2 underperforms the 120B model by ~5.7 points and Qwen3.5 by 1.6 points — suggesting that scientific domain knowledge is more dependent on pretraining scale than competitive coding reasoning.

Alignment and Instruction-Following

ArenaHard v2 (Table 1, Table 3). Cascade-2 achieves ArenaHard v2 Average: 83.5, with Hard Prompt: 88.2 and Creative Writing: 78.7. The baseline Nemotron-3-Nano-30B-A3B (trained without Cascade RL) scores 67.7 Average, 72.1 Hard Prompt, 63.2 Creative Writing. Qwen3.5-35B-A3B scores 65.4 Average, 64.5 Hard Prompt, 66.3 Creative Writing. The MOPD vs. RLHF comparison in Table 3 shows that starting from an initial model with Hard Prompt 71.5 and Creative Writing 40.6, MOPD reaches 85.5 and 71.0 in 52 steps, while RLHF training requires 160 steps to reach only 80.7 and 71.2. This ~5-point advantage on Hard Prompt with 3× fewer steps is the paper's strongest evidence for MOPD's training efficiency.

IFBench (Table 1). Cascade-2 scores 82.9 (prompt-level), compared to Nemotron-3-Nano-30B-A3B (71.5), Nemotron-3-Super-120B-A12B (72.6), and Qwen3.5-35B-A3B (70.2). The ~10-point advantage over the 120B model demonstrates that IF-RL placed first in the Cascade pipeline successfully transfers to final model performance despite subsequent RLHF and other stages.

Scale AI Multi-Challenge (Table 1). Cascade-2 scores 45.3, compared to Nemotron-3-Nano-30B-A3B (38.5), Nemotron-3-Super-120B-A12B (55.2), and Qwen3.5-35B-A3B (60.0). This multi-turn conversation benchmark shows a relative weakness — Cascade-2 underperforms both Qwen3.5 and the 120B model, by ~14.7 and ~9.9 points respectively. The paper does not explicitly diagnose this regression, but it appears in the context of the authors' acknowledgment that Cascade-2 "underperforms Qwen3.5-35B-A3B on knowledge-intensive and agentic benchmarks" (Section 2, final paragraph).

Agentic Tasks

SWE-bench Verified (Table 1, Table 4). Cascade-2 scores 50.2 (OpenHands scaffold, avg@4), compared to Nemotron-3-Nano-30B-A3B (38.8), Nemotron-3-Super-120B-A12B (60.5), and Qwen3.5-35B-A3B (69.2). The paper's Table 4 shows that agentless SWE RL improves the model: starting from avg@4 49.8% and pass@4 64.2% on OpenHands, agentless RL raises these to 50.8% and 65.0%. Simultaneously, Agentless Mini improves from avg@4 41.9% to 44.3% and pass@4 55.2% to 57.4%. The ~8.4-point gap to Qwen3.5 on SWE-bench is one of the largest deficits in Table 1.

τ2-Bench (Table 1). Cascade-2 scores 58.9, compared to Nemotron-3-Nano-30B-A3B (49.0), Nemotron-3-Super-120B-A12B (61.2), and Qwen3.5-35B-A3B (81.2). The ~22.3-point gap to Qwen3.5 on this customer-service agent benchmark is the largest deficit reported in Table 1, and the paper notes that the evaluation setup uses "latest-turn thought retention" rather than the official "no thought carry-over policy," which "consistently reduces scores by 3–5 points." For the telecom subset, the system prompt is additionally modified to emphasize dual-control setting instructions.

Terminal Bench 2.0 (Table 1). Cascade-2 scores 21.1, compared to Nemotron-3-Nano-30B-A3B (8.5), Nemotron-3-Super-120B-A12B (31.0), and Qwen3.5-35B-A3B (40.5). This benchmark involves end-to-end terminal-based workflows including model training and system configuration, and Cascade-2's ~19.4-point deficit to Qwen3.5 suggests limitations in complex multi-step procedural execution.

BFCL v4 (Table 1). Cascade-2 scores 52.9, compared to Nemotron-3-Nano-30B-A3B (53.8) and Qwen3.5-35B-A3B (67.3). The near-identical score to the base Nano model (both trained from the same pretrained checkpoint) and the ~14.4-point gap to Qwen3.5 indicate that the Cascade RL pipeline does not substantially improve function-calling agentic capabilities, consistent with the paper's acknowledgment of agentic weaknesses.

Knowledge and STEM

MMLU-Redux, MMLU-Pro, GPQA-Diamond (Table 1). Cascade-2 scores MMLU-Redux: 86.3, MMLU-Pro: 79.8, GPQA-Diamond: 76.1. Compared to Qwen3.5-35B-A3B (93.3, 85.3, 84.2), Cascade-2 shows deficits of ~7, ~5.5, and ~8.1 points respectively. These knowledge-intensive benchmarks consistently favor Qwen3.5, confirming the paper's stated limitation: "it underperforms Qwen3.5-35B-A3B on knowledge-intensive [...] benchmarks, highlighting the importance of stronger knowledge-intensive pretraining."

HLE (Table 1). Cascade-2 scores 17.7 (no tool), compared to Nemotron-3-Nano-30B-A3B (10.6), Nemotron-3-Super-120B-A12B (18.3), and Qwen3.5-35B-A3B (22.4). The paper reports that using a boxed-answer prompt (Appendix C.2) rather than the official HLE response format "improves the accuracy by 6–7 points, primarily on the math subset," which should be considered when comparing to models evaluated with the official format.

Long Context

NIAH@1M, AA-LCR, LongBench v2, CL-Bench (Table 1). Cascade-2 scores NIAH@1M (RULER subset): 99.0, AA-LCR: 39.1, LongBench v2: 40.3, CL-Bench: 12.2. On NIAH@1M — the most synthetic long-context test — Cascade-2 achieves near-perfect performance (99.0), exceeding both Qwen3.5-35B-A3B (94.3) and the 120B model (98.3). On the more realistic and challenging AA-LCR (100K-token average documents requiring multi-document reasoning), Cascade-2 scores 39.1 vs. Qwen3.5's 58.5 and the 120B's 58.3 — a substantial deficit. LongBench v2 (40.3 vs. Qwen3.5's 59.0) shows a similar pattern. CL-Bench (12.2 vs. Qwen3.5's 15.5) shows a smaller gap. This pattern — strong on synthetic retrieval, weaker on realistic reasoning over long documents — mirrors the knowledge-intensive benchmark results and suggests that long-context RL (Section 4.6) effectively improves retrieval but does not fully compensate for differences in pretraining context exposure.

Multilingual

MMLU-ProX, WMT24++ (Table 1). Cascade-2 scores MMLU-ProX: 72.5, WMT24++ (en→xx): 84.1. Compared to Qwen3.5-35B-A3B (81.0, 87.6) and Nemotron-3-Super-120B-A12B (79.4, 86.7), Cascade-2 shows consistent but moderate deficits across both multilingual knowledge and translation quality.

Ablation Studies and Robustness Checks

The paper does not present a traditional ablation section with controlled experiments removing individual components. Instead, the primary "ablations" are implicit in the training dynamics analysis and comparative tables that demonstrate the contribution of specific design choices.

MOPD vs. GRPO for math recovery (Figure 3(c)): Starting from the same initial checkpoint (AIME25 = 89.9), GRPO reaches 91.0 after 25 steps, while MOPD reaches 92.0 within 30 steps and recovers teacher-level performance. This demonstrates that dense token-level distillation advantage converges to a higher ceiling than sparse sequence-level GRPO reward on the same math training data, even though GRPO is used for all other RLVR stages.

MOPD vs. RLHF for alignment (Table 3): Starting from an initial model with ArenaHard Hard Prompt 71.5 and Creative Writing 40.6, MOPD achieves 85.5 and 71.0 in 52 steps using a multi-teacher setup (math + RLHF + multi-domain teachers). Pure RLHF training reaches only 80.7 and 71.2 after 160 steps. The Hard Prompt gap (85.5 vs. 80.7) is substantial, while Creative Writing reaches equivalent performance faster. This is not a pure ablation (MOPD and RLHF use different training signals — MOPD uses token-level teacher advantages, RLHF uses GenRM pair-wise comparisons), but it demonstrates MOPD's training efficiency and its ability to improve alignment while recovering math performance that RLHF alone does not address.

Thinking mode vs. mixed mode in RLHF (Section 4.5.2): The paper reports that incorporating both thinking and non-thinking modes during RLHF "can improve training convergence and yield slight gains on evaluation benchmarks," but causes "significant degradation in instruction-following performance. The resulting drop is substantial enough that the gains obtained in the earlier RLVR stage cannot be fully recovered." This ablation (not quantified with specific benchmark numbers) directly motivates the exclusive thinking-mode RLHF design and supports the Cascade ordering principle of protecting earlier-stage gains.

Agentless RL transfer to agentic scaffolds (Table 4): After agentless RL, OpenHands avg@4 improves from 49.8% to 50.8% and pass@4 from 64.2% to 65.0%, while Agentless Mini avg@4 improves from 41.9% to 44.3% and pass@4 from 55.2% to 57.4%. This demonstrates positive skill transfer from the simpler agentless setting to the full agentic scaffold. The gains are modest (1–2.5 points) but consistent across both scaffolds and both metrics.

Extended Code RL generation budget (Section 4.7.2): The paper increases maximum response length to 118K tokens and rollouts to 16 per prompt, compared to Cascade 1's unspecified smaller budgets. The ablation value is that these extensions "enable the policy to better capture sparse reward signals on extremely difficult problems." The 22-step training duration with 3.5K prompts and the resulting LiveCodeBench Pro Medium score of 27.6 (vs. 17.8 for Qwen3.5-35B-A3B and 23.2 for the 120B model) demonstrate that this aggressive filtering + extended budget combination is effective.

Difficulty-based filtering in Code RL and Agentic SWE RL (Sections 4.7.1, 4.8.2): Code RL filters out prompts that GPT-OSS-120B solves in all 8/8 attempts, yielding 3.5K samples. Agentic SWE RL removes instances where all or none of 16 rollouts pass verification (with 90% random discarding of impossible cases). No ablation is provided showing what happens without this filtering, but the paper argues that this filtering is "critical for further boosting model performance" and that difficult prompts with zero-success rollouts "degrade the stability and effectiveness" of training.

LLM judge grading protocol for IMO-ProofBench (Section A.1.2, Table 5): The paper reports that using mean score directly gives DeepSeek-Math-V2 73.8 on Advanced (vs. human rating 61.9, an 11.9-point overestimate), while the adopted aggregation rule (0 if any judge gives 0, else mean) gives 57.7 (a 4.2-point underestimate). Re-evaluating DeepSeek-Math-V2 under this protocol yields 78.6 overall vs. the reported 80.2, confirming that the grading protocol does not substantially overestimate Cascade-2's performance.

Critical Assessment

Claim: Cascade RL + MOPD enables olympiad-level reasoning from a compact model. The evidence for this claim is strong but narrow. The IMO 2025 result (35/42, solving 5/6 problems) is verified by a human expert (with one problem verified by LLM judge due to analytic complexity). The IOI 2025 result (439.28/600, Gold) is verified by official OnlineJudge. The ICPC World Finals result (10/12, Gold) is verified by official evaluation. These are unambiguous, externally validated achievements that constitute genuine olympiad-level performance.

However, the paper does not isolate how much of this performance is attributable to Cascade RL + MOPD versus the test-time scaling strategies employed during evaluation. The IMO solutions use a generate-verify-refine pipeline with up to 8 rounds, 128 generations per round, and 64 verifications per proof — this is thousands of model generations per problem. The IOI pipeline uses up to 50 rounds × 40 solutions = 2,000 generations per subtask, with cross-subtask insight sharing. The ICPC results use up to 1,000 submissions per problem with initial filtering. The paper does not report what fraction of these scores would be achieved with single-pass greedy decoding or even modest best-of-N. This is a significant gap: the reader cannot distinguish between "Cascade RL produces a model with strong base reasoning" and "Cascade RL produces a model that is particularly amenable to test-time scaling strategies." Since the test-time scaling code is adapted from DeepSeek-Math-V2 and the IOI pipeline from Cascade 1, and since these strategies are applied equally to baselines where possible, the relative comparisons are valid, but the absolute claim of "gold-medal level reasoning" should be understood as "gold-medal level reasoning when combined with extensive test-time compute."

Claim: MOPD efficiently recovers benchmark regressions and sustains strong performance gains. The evidence in Figure 3(c) and Table 3 is persuasive but limited in scope. Figure 3(c) shows MOPD recovering AIME25 performance to teacher-level (92.0), but this is math-only training — it demonstrates that MOPD works for single-domain distillation, not that the multi-teacher setup simultaneously recovers math, instruction-following, and alignment. Table 3 shows MOPD improving ArenaHard Hard Prompt from 71.5 to 85.5, but this comparison is against RLHF-only training, not against a GRPO baseline with equivalent data. The paper does not provide a direct comparison showing the model's benchmark performance before vs. after MOPD across all tracked domains (e.g., a table showing math, code, IF, and ArenaHard scores immediately before and after the MOPD stage). Without this, the reader cannot assess how much capability MOPD actually recovered versus what the subsequent RL stages contributed.

A critical missing experiment is an ablation where MOPD is removed from the pipeline entirely and the remaining stages are run sequentially (IF-RL → Multi-domain → RLHF → Long-context → Code → SWE). If the final model without MOPD shows clear regressions on math and instruction-following relative to the model with MOPD, the case for MOPD's necessity would be stronger. As it stands, the paper provides evidence that MOPD is beneficial (it recovers performance faster than GRPO), but not that it is necessary (the model without MOPD might still eventually reach similar performance through the subsequent stages).

Claim: Cascade 2's ordering is optimized based on inter-domain interference dynamics. The paper provides qualitative reasoning for each ordering decision (IF-RL first because it damages alignment but later stages can recover; RLHF after MOPD because GenRM can dilute IF; Long-context isolated; Code/SWE last because of extreme context lengths). This reasoning is plausible but not experimentally validated. No experiment compares the chosen ordering to an alternative ordering (e.g., RLHF first, IF-RL later; or Code RL before MOPD). The paper's claim that the ordering "is not a universal constant; rather, it is a dynamic function of the model's underlying behaviors and learning trajectories" is a hypothesis, not a demonstrated result. The fact that Cascade 2 uses a different ordering from Cascade 1 is consistent with this hypothesis, but it could equally reflect different engineering choices by the authors rather than a principled response to behavioral dynamics.

General weaknesses in experimental design. Several patterns emerge across the evaluation:

Single base model family. All results use Nemotron-3-Nano-30B-A3B-Base as the starting point. The generalizability of Cascade RL + MOPD to other model architectures (dense transformers, other MoE configurations, different pretraining data mixtures) is completely untested. Given that the paper's central claim is about a post-training methodology, not a specific model, this is a significant limitation.
No confidence intervals or statistical testing. Across all benchmark results in Table 1, Table 5, and Table 6, no standard errors, confidence intervals, or significance tests are reported. For benchmarks with small test sets (e.g., HMMT Feb25 with 30 problems, GPQA-Diamond with 198 questions), the differences between models could be within sampling noise. For avg@k metrics, the paper reports expected pass@1 but does not quantify the variance of this estimator.
Knowledge-intensive benchmark regressions are acknowledged but not diagnosed. Cascade-2 underperforms Qwen3.5-35B-A3B by 7 points on MMLU-Redux, 5.5 points on MMLU-Pro, 8.1 points on GPQA-Diamond, 22.3 points on τ2-Bench, 19 points on Terminal Bench 2.0, and 19 points on AA-LCR. The paper attributes this to "stronger knowledge-intensive pretraining" in Qwen3.5 but provides no analysis of whether these regressions are caused by forgetting during Cascade RL (the SFT checkpoint might have been stronger) or by an inherent limitation of the base model's pretraining. The Nemotron-3-Nano-30B-A3B baseline scores in Table 1 are helpful but incomplete: they represent a different post-training pipeline (NVIDIA Nano-v3), not the SFT checkpoint's performance. Without SFT checkpoint baselines, the reader cannot distinguish between "Cascade RL caused these regressions" and "the base model never had these capabilities."
Test set contamination risk is not addressed. MATH benchmark problems (including AIME, IMO, HMMT) are known to appear in web-crawled pretraining data. The LiveCodeBench benchmarks are explicitly designed to be contamination-free by using recently released problems, but the math benchmarks are not. The paper does not discuss decontamination procedures for any benchmark. Given the use of DeepSeek-V3.2 and GPT-OSS-120B (both trained on web data) to generate SFT training data, there is a risk that the SFT stage inadvertently trains on problems similar to the evaluation set, inflating performance.
The human expert evaluation for IMO 2025 (Appendix E) reveals qualitative issues. The expert comments on individual problems note: "some proofs are longer than necessary, include superfluous intermediate steps or definitions, occasionally expose traces of intermediate reasoning, and sometimes contain minor typographical issues" (Section 5.1). For Problem 5, the expert notes that "some parts of the proof read like a thinking process. It feels like it realizes that c has not been proved, then decides that proving c is not actually necessary. The proof seems finished, but then it starts thinking again" (Appendix E, Problem 5 comment). Despite these issues, all five solved problems received 7/7. This suggests that the human expert graded leniently — rewarding mathematical correctness despite presentation flaws. Under stricter grading (typical of actual IMO judging), some of these proofs might have received partial credit.
The agentic benchmark deficits are substantial and unexplained. The paper acknowledges that Cascade-2 "underperforms Qwen3.5-35B-A3B on knowledge-intensive and agentic benchmarks" but provides no analysis of why. The SWE-bench gap (50.2 vs. 69.2) is ~19 points — larger than the gains Cascade RL achieves over the Nano baseline (50.2 vs. 38.8). The τ2-Bench gap (58.9 vs. 81.2) is ~22 points. These deficits are large enough that they undermine the claim of "best-in-class performance across a broad range of benchmarks" unless the definition of "broad range" excludes agentic tasks. The paper's acknowledgment that "agentic RL" is an area for "future work" (Section 2) is appropriate but insufficient: the reader needs to understand whether these deficits represent a failure of the Cascade RL methodology for agentic domains, insufficient agentic RL training data, or a fundamental limitation of the 3B activated parameter scale.
Missing SFT-only baselines for all benchmarks. The paper evaluates Nemotron-3-Nano-30B-A3B (trained with NVIDIA Nano-v3 pipeline, not Cascade RL) but does not report the performance of the SFT checkpoint that serves as the starting point for all Cascade RL stages. Without this baseline, it is impossible to determine how much of Cascade-2's final performance is attributable to the meticulously curated SFT data (which uses DeepSeek-V3.2, GPT-OSS-120B, and other strong teacher models to generate responses) versus the RL post-training stages. This is perhaps the most significant missing experiment: if the SFT model already achieves, say, 85 on AIME 2025, then the ~7-point gain from Cascade RL is meaningful but modest; if the SFT model achieves only 70, then the ~23-point gain is transformative. The paper's release of SFT data is commendable for reproducibility, but reporting SFT model performance on key benchmarks would have provided essential context.

6. Limitations and Trade-offs

6.1 The Cascade RL Pipeline's Performance Is Fundamentally Bounded by the Base Model's Pretraining Quality

Nemotron-Cascade 2 demonstrates that sophisticated post-training can dramatically amplify a base model's reasoning capabilities, but the paper is transparent that this amplification has hard boundaries set by what the base model learned during pretraining. Section 2 states directly:

"It underperforms Qwen3.5-35B-A3B on knowledge-intensive and agentic benchmarks, highlighting the importance of stronger knowledge-intensive pretraining and agentic RL in future work."

This is not a minor caveat — it reveals that Cascade RL + MOPD is an effective capability amplifier but not a capability creator. The base model, Nemotron-3-Nano-30B-A3B-Base, was pretrained with a specific data mixture and scale. If that pretraining did not sufficiently cover certain domains (e.g., graduate-level scientific knowledge, complex multi-turn customer service interactions, terminal-based system administration workflows), no amount of post-training RL can fully compensate.

The evidence is systematic across knowledge-intensive benchmarks in Table 1. Cascade-2 trails Qwen3.5-35B-A3B by 7 points on MMLU-Redux (86.3 vs. 93.3), 5.5 points on MMLU-Pro (79.8 vs. 85.3), and 8.1 points on GPQA-Diamond (76.1 vs. 84.2). On agentic tasks, the gaps are even starker: τ2-Bench shows a 22.3-point deficit (58.9 vs. 81.2), Terminal Bench 2.0 a 19.4-point deficit (21.1 vs. 40.5), and SWE-bench Verified a 19-point deficit (50.2 vs. 69.2). On long-context reasoning over real documents, AA-LCR shows a 19.4-point gap (39.1 vs. 58.5). These are not marginal differences — they represent categorical capability gaps where Cascade-2 simply cannot perform at the level of a model with stronger knowledge-intensive pretraining.

Critically, the paper does not isolate how much of these deficits originates from the base model versus from forgetting during Cascade RL. The Nemotron-3-Nano-30B-A3B baseline in Table 1 uses a different post-training pipeline (NVIDIA Nano-v3), so it is not a clean SFT baseline. Without SFT-only checkpoint performance on these benchmarks, the reader cannot determine whether knowledge-intensive regressions are a pretraining limitation (the base model never knew this content) or a Cascade RL limitation (the SFT model knew it but later stages caused forgetting). The paper releases the SFT data but not SFT model evaluation results, leaving this ambiguity unresolved.

The paper partially mitigates this by acknowledging the limitation and pointing to future work on pretraining improvements, but the mitigation is entirely aspirational — no experiments test whether a different base model would show different sensitivity, whether additional knowledge-intensive SFT data could close the gap, or whether specific Cascade RL stages are responsible for the regressions.

6.2 The IMO, IOI, and ICPC Results Inextricably Combine Post-Training Quality with Test-Time Compute, Making It Impossible to Attribute Gains to Either Factor

The paper's headline achievement — gold medals at IMO 2025, IOI 2025, and ICPC World Finals 2025 — relies on extensive test-time scaling strategies that are described but never ablated against simpler inference protocols. The IMO solutions use a generate-verify-refine pipeline with up to 8 rounds, 128 proof generations per round, 64 verifications per proof, top-32 refinement selection, and 8 verification analyses for each of 4 refined proofs per round (Section 5.1). This can consume thousands of model generations per problem. The IOI pipeline uses up to 50 rounds × 40 candidate solutions = 2,000 model generations per subtask, with cross-subtask insight sharing and submission history aggregation (Section 6.1). The ICPC results use up to 1,000 submissions per problem with initial filtering (Section 6.1).

The consequence is that the reader cannot answer a fundamental question: does Cascade-2 succeed because it has genuinely learned olympiad-level reasoning, or because it generates a large enough volume of candidates that one happens to be correct, with the test-time pipeline filtering effectively? These are qualitatively different capabilities. The former would mean the model can solve IMO problems with modest inference budgets (e.g., best-of-4 or single-pass with verification). The latter would mean the model produces a distribution where correct solutions exist at some non-trivial frequency, but the model itself cannot reliably distinguish correct from incorrect without extensive external verification. The paper provides no pass@1 results for IMO problems, no ablation showing how performance degrades as the test-time budget shrinks, and no comparison to a simpler best-of-N strategy to establish whether the multi-round refinement pipeline provides gains beyond what brute-force sampling would achieve.

The only partial evidence comes from Figure 4, which shows IMO-ProofBench Advanced scores improving from 40.7 at round 1 to 53.4 at round 5 of generate-verify-refine. This demonstrates that additional rounds help, but it does not establish whether round-1 performance (40.7) represents the model's single-pass reasoning capability or a partially scaled intermediate step. The gap between Cascade-2's round-5 score (53.4) and DeepSeek-Math-V2's score (61.9 human, 57.7 reproduced) narrows under test-time scaling but may reflect different scaling behavior rather than different base capability.

The paper does not attempt to mitigate this ambiguity — it treats the test-time scaling pipeline as part of the evaluation protocol without analyzing its contribution. Future work could establish a standard inference budget for IMO evaluation (e.g., best-of-64 with majority voting or a single round of verification) to enable cleaner comparisons between models' base reasoning capabilities independent of inference-time strategy.

6.3 Multi-Domain On-Policy Distillation Is Never Directly Ablated Against the Pipeline Without It

MOPD is the paper's central methodological contribution, described as "a critical stabilization point in this ordering" that "is effective at recovering benchmark performance that may have regressed during earlier, more specialized stages" (Section 4.1.1). The paper provides evidence that MOPD converges faster than GRPO on individual tasks (Figure 3(c) for AIME25, Table 3 for ArenaHard) and that MOPD recovers teacher-level performance. However, the paper never runs the full Cascade RL pipeline without MOPD and reports the final benchmark performance of that model.

This is the critical missing experiment. Without it, several questions are unanswerable:

Does MOPD contribute to the final model's performance, or do the subsequent RL stages (RLHF, long-context, code, SWE) naturally recover any regressions? The paper shows MOPD is faster than GRPO for math and alignment recovery, but speed is not the same as necessity — the full pipeline includes ~30 steps of RLHF, ~30 steps of long-context RL, 22 steps of code RL, and 40–50 steps of agentless SWE RL, all of which might incrementally recover regressed capabilities.
Are the domain teachers actually providing complementary signals, or would a single best teacher (e.g., the multi-domain checkpoint after IF-RL + Multi-domain RL) suffice? The paper selects three teachers (SFT for math, separate RLHF run for alignment, IF-RL + Multi-domain for instruction-following) based on the conceptual argument that no single checkpoint excels at all domains, but this is not tested.
Does MOPD's benefit come from the multi-teacher design, or simply from having a dense token-level training stage (which could be achieved with a single teacher or even with supervised fine-tuning on on-policy rollouts)?

The existing evidence is suggestive but insufficient. Figure 3(c) shows MOPD outperforming GRPO on math-only training, but this is single-domain, not multi-domain. Table 3 shows MOPD outperforming RLHF on ArenaHard, but this comparison starts from the same initial checkpoint, not from a model that has already undergone all Cascade stages — it demonstrates MOPD is a better alternative to RLHF for alignment, not that MOPD is necessary within the full pipeline. The paper's ablation of the MOPD warm-up and importance weighting (via gradient norm analysis in Figure 3(b)) focuses on optimization stability, not on the pipeline-level contribution.

The paper does not acknowledge this gap as a limitation. The MOPD section (4.4) argues for MOPD's benefits based on its training efficiency advantages and its ability to consolidate domain-specific teachers, but never states or implies that the full pipeline has been tested without it. This is a significant methodological gap because the paper's primary claim — that MOPD is a key technical advancement enabling Cascade RL to scale — rests on an implicit counterfactual that is never evaluated.

6.4 The Difficulty Estimation and Data Filtering Heuristics Are Critical but Not Characterized or Ablated

Throughout the Cascade RL pipeline, the paper applies aggressive filtering strategies that remove prompts deemed too easy or too hard for the model at its current capability level. Code RL retains only 3.5K prompts out of a much larger set, filtering out any prompt that GPT-OSS-120B solves in all 8/8 attempts (Section 4.7.1). Agentic SWE RL removes instances where all 16 rollouts pass (100% accuracy) or where none pass (0% accuracy), with 90% random discarding of the latter category (Section 4.8.2). Agentless SWE RL masks the loss for prompts where all rollouts score below a 0.5 reward threshold (Section 4.8.1). IF-RL applies dynamic filtering to remove prompts where all rollouts are entirely correct or entirely incorrect (Section 4.2.2).

The paper's rationale is consistent: prompts that are too easy provide no learning signal (the model already solves them), and prompts that are too hard provide pure noise (the group-normalized advantage is approximately zero). The consequence is that these filtering decisions fundamentally determine what the model learns and how its capabilities develop across stages, yet no experiment varies the filtering thresholds, the filtering criteria, or tests whether filtering is even necessary.

Several failure modes are plausible but unexplored:

Filtering-induced distribution shift. If 90% of the hardest agentic SWE instances are randomly discarded, the training distribution skews toward moderately difficult problems. The model may never encounter the most challenging debugging scenarios during RL, limiting its capability ceiling even if additional training steps are taken.
Teacher-dependent filtering creates misalignment. Code RL filters based on GPT-OSS-120B's success rate. If GPT-OSS-120B and the student model (Nemotron-3-Nano-30B-A3B) have different failure modes, the filtered set may be easy for GPT-OSS-120B but hard for the student, or vice versa. The retained 3.5K prompts represent problems that are "hard enough" relative to a 120B model — they may be impossibly hard for a 30B/3B model, creating the exact noise problem the filtering was designed to avoid.
Dynamic filtering creates a moving target. In IF-RL, prompts are filtered dynamically at each training step based on the current policy's behavior. As the policy improves, prompts transition from "all incorrect" to "mixed" to "all correct," and the filter removes them at both extremes. This means the training distribution is continuously changing in response to policy improvement, making it difficult to diagnose whether convergence plateaus are caused by the policy saturating on available prompts or by the filter removing too many prompts.

The paper provides no sensitivity analysis for any of these filtering decisions. The Code RL filter threshold (0/8 vs. 8/8) could be widened to, say, 1/8 and 7/8; the agentic SWE discarding rate (90%) could be 50% or 0%; the agentless RL reward threshold (0.5) could be 0.3 or 0.7. Without these ablations, the reader cannot assess whether the pipeline's performance is robust to these choices or whether it has been overfitted to specific filtering heuristics that happen to work for this model and these datasets.

The paper does not address this as a limitation. The filtering decisions are described as pragmatic engineering choices, and their impact is not discussed beyond the stated rationales. For practitioners attempting to replicate the pipeline with different models or datasets, the absence of filtering ablations means the recommended filtering thresholds are essentially unvalidated hyperparameters.

6.5 The Evaluation Protocol for Agentic Tasks Uses Non-Standard Inference Configurations That Inflate Scores Relative to Official Protocols

For the two agentic benchmarks where Cascade-2 shows competitive or strong results — τ2-Bench (58.9 vs. Qwen3.5's 81.2, but improved over the base Nano's 49.0) and SWE-bench Verified (50.2 vs. Qwen3.5's 69.2) — the paper deviates from official evaluation protocols in ways that the paper acknowledges but whose quantitative impact is stated rather than demonstrated through controlled comparison.

τ2-Bench evaluation (Section A.6). The official evaluation code uses a "no thought carry-over policy," which removes all prior reasoning content across turns. The paper instead uses a "latest-turn thought retention" policy, retaining the model's reasoning after the most recent user turn while discarding earlier reasoning. The paper states this "consistently reduces scores by 3–5 points relative to latest-turn thought retention" — but this means all scores in Table 1 for τ2-Bench are inflated by 3–5 points relative to what the official protocol would produce. Additionally, for the telecom subset, the paper modifies the system prompt to repeat dual-control instructions three times, and notes that a "full thought retention policy" gives similar accuracy but with longer contexts. The combination of these changes means Cascade-2's τ2-Bench score of 58.9 is not directly comparable to Qwen3.5's 81.2 if Qwen3.5 was evaluated under the official protocol. The paper does not report Cascade-2's score under the official no-thought-carryover policy, so the reader cannot compute the true gap.

SWE-bench Verified evaluation (Section A.6). The paper uses a "full interaction retention policy" in OpenHands, preserving the complete history of tool calls, observations, and model outputs across turns. The paper argues this is justified because it "closely mirrors our training environment," reducing train-test mismatch. While methodologically defensible, this means Cascade-2's SWE-bench score is not comparable to other models evaluated under the default OpenHands configuration (which may use different context retention policies). The paper does not report Cascade-2's SWE-bench score under the default configuration or specify what configuration was used for the baseline models (Nemotron-3-Super, Qwen3.5).

The consequence is that the paper's agentic benchmark results cannot be meaningfully compared to published numbers without knowing and matching the evaluation configuration. For τ2-Bench, the comparison is explicitly misleading if the baseline models use the official protocol while Cascade-2 uses the modified protocol. For SWE-bench, the comparison may be valid if the baselines were evaluated under similar configurations, but the paper does not provide this information.

The paper partially mitigates the τ2-Bench issue by disclosing the protocol change and estimating its quantitative impact, which is commendable transparency. However, the mitigation is incomplete: the reader needs Cascade-2's score under the official protocol, or all baseline models re-evaluated under the same modified protocol, to make valid comparisons. The SWE-bench configuration difference is not discussed as a potential source of incomparability, which is a more significant oversight.

6.6 The Entire Pipeline Is Demonstrated on a Single Base Model, Making It Unknown Whether Cascade RL + MOPD Generalizes to Other Architectures or Scales

All experiments use a single base model: Nemotron-3-Nano-30B-A3B-Base, a 30B total parameter MoE with 3B activated parameters. The Cascade RL pipeline has 7 stages, each with domain-specific hyperparameters (learning rates, batch sizes, rollout counts, sequence lengths, reward functions, filtering strategies). The MOPD stage involves 3 domain teachers, per-prompt teacher assignments, truncated importance weighting with thresholds [0.5, 2.0], and a specific reverse-KL distillation objective. The ordering of stages is described as "a dynamic function of the model's underlying behaviors and learning trajectories" (Section 4.1.1) that changed from Cascade 1.

The consequence is that every design choice in this pipeline — the ordering, the filtering thresholds, the teacher selection, the reward functions, the KL penalty coefficients, the stage durations — may be specific to this model, this pretraining data, and this SFT quality. The paper provides no evidence that the same pipeline would work for:

Dense transformer architectures rather than MoE. MoE models have different training dynamics (sparse activation, load balancing, expert specialization) that may interact with RL in ways that do not transfer to dense models.
Different pretraining data mixtures or scales. If the base model has different knowledge boundaries, the optimal ordering (e.g., whether to place knowledge-intensive stages before or after reasoning stages) may change. The Cascade 1 vs. Cascade 2 ordering change is cited as evidence that ordering adapts to model behavior, but this is a single data point (two orderings for related models in the same family), not a systematic study.
Models at different scales (e.g., 7B, 70B, 405B). The phenomena of entropy collapse, reward hacking, and catastrophic forgetting may manifest differently at different scales. The 3B activated parameter regime may be particularly amenable to the GRPO configuration used (16 rollouts, no KL penalty, learning rate 3e-6), but a 300B model might need different hyperparameters to converge stably.
Different SFT data quality. The paper's SFT uses responses generated by DeepSeek-V3.2, DeepSeek-V3.2-Speciale, and GPT-OSS-120B — frontier models that provide high-quality distillation targets. If a different base model were SFT'd with weaker teacher models, the starting point for Cascade RL would be lower, and the optimal ordering and filtering strategies might differ.

The paper does not address this limitation. The title describes "Cascade RL and Multi-Domain On-Policy Distillation" as general techniques, and Section 1 frames the advantages of Cascade RL in universal terms ("significantly simplifies the engineering complexity," "remarkably resistant to catastrophic forgetting"), but all evidence comes from a single instantiation. For practitioners considering adopting this pipeline, the lack of any cross-model validation means the methodology's claimed benefits are unproven outside the specific Nemotron-3-Nano context. The paper's release of model weights, training data, and methodological details enables reproduction, which could address this limitation over time, but the paper itself provides no reassurance that the pipeline transfers.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper doesn't propose a new training algorithm or a novel model architecture — it proposes a new design principle for post-training pipelines: domain-sequential RL stages, when instrumented with periodic multi-teacher on-policy distillation checkpoints drawn from the pipeline's own trajectory, can scale to cover seven diverse RL domains without the catastrophic trade-offs that have historically forced practitioners to choose between breadth and depth in post-training. The shift is from treating post-training as a single optimization problem (maximize expected reward across all prompts) to treating it as a capability management problem — one where the training process itself generates transient peaks of domain-specific performance, and the system architecture must actively preserve and re-integrate those peaks.

The magnitude of this shift should be calibrated carefully. This is not a paradigm-shattering breakthrough — Cascade RL itself was already introduced in Cascade 1, and on-policy distillation from teacher models is a well-established technique (Agarwal et al., 2024; Gu et al., 2024; Xiao et al., 2026; Zeng et al., 2026). What Cascade 2 contributes is the empirical demonstration that these two ideas compose into a self-stabilizing system when the teachers are internal pipeline checkpoints rather than external models. The core insight — that Cascade RL naturally produces a capability-diverse checkpoint collection, and that these checkpoints can serve as domain-specific distillation targets to reverse regressions — is a reframing that converts a bug (sequential RL causes forgetting) into a feature (sequential RL generates diverse teachers).

The most important landscape change is in how post-training pipelines should be architected for multi-domain capability. Before Cascade 2, the implicit design space for multi-domain RL post-training was:

Blend everything together and hope interference is manageable (common in early RLHF work).
Train domain-specific models and route prompts at inference time (fragments the model into specialized silos).
Sequential RL with careful ordering (Cascade 1), accepting that some regressions are inevitable.

Cascade 2 introduces a fourth option: sequential RL with check-pointed capability preservation, where mid-pipeline distillation stages actively recover performance on domains that earlier stages optimized and later stages might degrade. This means the design question shifts from "how do I order stages to minimize forgetting?" to "where do I insert consolidation stages to maximize multi-domain Pareto efficiency?" — a more structured optimization problem.

The work also reconciles a tension that has been visible but unarticulated in the RL post-training literature: the observation that RLVR training improves reasoning but often reduces response diversity and can degrade performance on benchmark categories that require different cognitive styles (creative writing, open-ended dialogue, knowledge recall). The paper names this tension explicitly (Section 4.4: "certain RLVR training often reduces model entropy and shortens reasoning traces, thus can negatively impact mathematical reasoning performance") and demonstrates that MOPD can reverse it. This provides a mechanistic explanation for why prior work sometimes found that RLHF improved alignment at the cost of reasoning benchmarks — those pipelines lacked a capability consolidation mechanism.

A more subtle landscape impact is on what constitutes a "strong" post-training result. Cascade 2 achieves IMO Gold, IOI Gold, and ICPC World Finals Gold with a 30B/3B MoE model — results that, before this paper, were associated with models 20× larger (DeepSeek-V3.2-Speciale-671B-A37B). This doesn't mean small models are categorically better — the paper is explicit about knowledge-intensive regressions — but it establishes that post-training quality can substitute for pretraining scale in the specific regime of high-difficulty reasoning tasks where the base model already possesses the necessary factual and procedural knowledge, and the challenge is orchestrating that knowledge into multi-step chains of deduction. This shifts the cost calculus for organizations deciding between training a larger model versus investing in post-training infrastructure: if the target task distribution skews toward reasoning over procedural and mathematical domains (competitive programming, theorem proving, formal verification), post-training may offer higher ROI per FLOP than pretraining scale.

The agentic performance deficits documented in Table 1 (SWE-bench: -19 points vs. Qwen3.5; τ2-Bench: -22 points; Terminal Bench: -19 points) create a contrary pressure: they demonstrate that Cascade RL + MOPD does not universally compensate for pretraining limitations. Tasks requiring extensive world knowledge, tool-use fluency, or long-horizon procedural execution in open-ended environments remain bottlenecked by pretraining quality and quantity. This establishes a boundary condition: Cascade RL is highly effective for closed-form reasoning tasks with verifiable rewards (math, code, formal proofs) but provides diminishing returns for open-ended agentic tasks where the reward signal is sparse, delayed, or learned. This boundary condition, while not the paper's primary focus, is an important empirical calibration for the field.

Follow-Up Research This Work Enables

Ablation of MOPD's necessity by running the full Cascade RL pipeline with and without the MOPD stage. The single most important follow-up experiment is one the paper does not report: starting from the same SFT checkpoint, run the full pipeline as described (IF-RL → Multi-domain → RLHF → Long-context → Code → SWE) with MOPD removed, and compare the final model's performance on math, code, IF, and alignment benchmarks to the Cascade-2 model. The paper provides evidence that MOPD is efficient (Figure 3(c) shows it recovers AIME25 performance faster than GRPO; Table 3 shows it achieves higher ArenaHard scores than RLHF at 1/3 the steps), but never demonstrates that MOPD is necessary for the final model's performance. It is plausible that the subsequent RL stages (RLHF, long-context, code, SWE) naturally recover any regressions, and that MOPD's contribution is primarily to training speed, not to the performance ceiling. A well-designed ablation would measure: (1) peak benchmark performance reached during the IF-RL stage, (2) performance after Multi-domain RL but before MOPD, (3) performance after a full pipeline without MOPD, and (4) performance after the full pipeline with MOPD. If (3) shows clear regressions on math and IF relative to (1) while (4) recovers them, MOPD's necessity is established. If (3) and (4) converge to similar final scores, MOPD is a speed-enhancer rather than a capability-enabler, which would reframe its contribution.

Benchmarking the SFT checkpoint to quantify how much of Cascade-2's performance comes from data curation versus RL. The paper releases the SFT dataset but does not evaluate the SFT model on the same benchmarks reported for Cascade-2. This makes it impossible to compute the "RL gain" — the delta between SFT-only performance and final Cascade-2 performance — for any benchmark. Given that the SFT data is generated by frontier models (DeepSeek-V3.2, DeepSeek-V3.2-Speciale, GPT-OSS-120B) and is meticulously curated across 10 domains, the SFT checkpoint likely achieves strong baseline performance that accounts for a substantial fraction of Cascade-2's final scores. A follow-up study should evaluate the SFT checkpoint on: AIME 2025/2026, HMMT Feb25, LiveCodeBench v6, ArenaHard v2, IFBench, SWE-bench Verified, and MMLU-Pro — the core benchmarks where Cascade-2 claims best-in-class performance. If the SFT model achieves, say, 85 on AIME 2025 (vs. Cascade-2's 92.4), then the RL pipeline contributes ~7 points. If the SFT model achieves only 60, the RL contribution is ~32 points. These numbers would fundamentally change how the field interprets Cascade-2's results: is this primarily a story about data curation and teacher distillation during SFT, or about RL post-training? The paper's current framing emphasizes Cascade RL, but the data to distinguish these hypotheses is not provided.

Testing whether the domain-teacher selection in MOPD requires three teachers, or whether a single best teacher suffices. The paper selects three teachers (SFT for math, separate RLHF run for alignment, IF-RL + Multi-domain checkpoint for instruction/multi-domain) based on the conceptual argument that no single checkpoint excels at all domains. This is untested. A straightforward ablation would train MOPD variants with: (a) only the math teacher, (b) only the RLHF teacher, (c) only the multi-domain teacher, (d) all three teachers (the reported configuration). If variant (d) outperforms all single-teacher variants on their respective domains (i.e., the math teacher alone produces better AIME scores than the RLHF teacher alone, but the three-teacher MOPD produces better AIME scores than the math-teacher-only MOPD), then the multi-teacher design is validated. If a single teacher (e.g., the multi-domain checkpoint after IF-RL) already captures most of the benefit across all domains, then the MOPD design can be simplified. This experiment would also reveal whether the teachers provide complementary signals or whether their contributions are redundant — critical information for practitioners implementing MOPD with their own checkpoint collections.

Characterizing the stability of Cascade RL ordering across different base models. The paper claims that the optimal stage ordering is "a dynamic function of the model's underlying behaviors and learning trajectories" (Section 4.1.1) and that Cascade 2's ordering differs from Cascade 1's. This is an empirical claim that should be tested. A follow-up study could take two architecturally different base models — for example, a dense 7B transformer and a 30B MoE — both SFT'd on the same data, and run Cascade RL with 2-3 different orderings (the Cascade 1 order, the Cascade 2 order, and a random or inverse order). If the optimal ordering differs between architectures in a way that aligns with the paper's diagnostic principles (stages that damage other capabilities should be placed early when recovery is possible), the claim is validated and the diagnostic framework is shown to transfer. If the optimal ordering is the same across architectures, the ordering may be more dependent on the specific RL datasets and reward functions than on model behavior, which would simplify adoption but reduce the claimed adaptivity. This experiment would also stress-test the paper's assumption that Cascade RL's resistance to forgetting is a general property, not specific to the Nemotron-3-Nano architecture.

Measuring whether agentless-to-agentic skill transfer in SWE generalizes across model scales and agent scaffolds. Table 4 shows that agentless RL (code repair only, no interactive environment) improves OpenHands performance by ~2 points (avg@4: 49.8 → 50.8). This is a small but positive transfer effect. A follow-up study should determine whether this effect scales: does agentless RL provide larger gains when the base model is weaker at code repair (suggesting that repair skill is the bottleneck)? Does it provide smaller or zero gains when the model is already strong at repair (suggesting that agentic performance is bottlenecked by other skills like exploration strategy or tool-use planning)? Does the transfer effect persist when switching from OpenHands to a different scaffold like SWE-Agent or Aider? A strong result would be: agentless RL consistently improves agentic performance by 2-5 points across scaffolds and scales, establishing it as a reliable pre-training step for SWE agents. A null or inverse result would indicate that the transfer is specific to the OpenHands scaffold or to the particular model's capability profile, narrowing the finding's applicability.

Practical Applications and Downstream Use Cases

Building competitive-programming and math-proving systems for competition-level deployment. The IMO, IOI, and ICPC results are not just benchmarks — they demonstrate that a 30B/3B model, when post-trained with Cascade RL and paired with test-time scaling, can operate at the level of the world's top human competitors in formal reasoning contests. This has immediate practical implications for organizations building AI systems to participate in olympiads, assist human competitors in training, or develop automated problem-solving tools for mathematics and algorithm design. The key numbers: 35/42 on IMO 2025 (Gold threshold ~28-32 depending on year), 439.28/600 on IOI 2025 (Gold threshold ~top 8.3%), 10/12 on ICPC World Finals 2025 (Gold medal, #4 placement). The deployment architecture is feasible: the model itself is 30B total / 3B activated parameters, meaning it can run on a single GPU at inference time, with the test-time scaling pipeline (generate-verify-refine, multi-round submission) running as an orchestration layer. The primary cost is inference compute (thousands of generations per problem), not model serving infrastructure. For organizations with access to GPU clusters, this is a deployable system today for contest-level reasoning tasks.

Cost-efficient data generation for self-improvement pipelines in math and code. The paper's finding that a compact model can achieve frontier-level reasoning performance when post-trained with Cascade RL has direct implications for synthetic data generation. Organizations training larger models often use frontier models (GPT-5, Gemini 3, DeepSeek-V3.2) to generate training data, incurring API costs or requiring access to proprietary systems. Cascade-2 demonstrates that a 30B/3B open model — which can be run on-premises with no API fees — can generate IMO gold-medal quality solutions and IOI gold-medal quality code. This means the model itself can serve as a data generator for bootstrapping further improvements, either through rejection sampling (generate many solutions, filter for correctness using verifiers) or through self-improvement loops where the model generates solutions, the best are selected, and the model is fine-tuned on them. The specific numbers: the SFT data that trained Cascade-2 was generated by DeepSeek-V3.2, DeepSeek-V3.2-Speciale, and GPT-OSS-120B — models with 120B-671B parameters. Cascade-2 achieves competitive performance with these teachers on reasoning benchmarks, suggesting it could replace them in subsequent data generation iterations, reducing dependence on external API models.

Post-training infrastructure design for multi-capability LLM products. For organizations deploying LLMs that must simultaneously excel at reasoning, instruction-following, creative writing, tool use, and agentic behavior — the standard requirement for general-purpose chat assistants and coding copilots — Cascade-2 provides a production-tested architecture for the post-training phase. The specific design decisions with practical implications: (1) place IF-RL early, before alignment training, to maximize instruction adherence without interference from RLHF; (2) insert a multi-teacher distillation stage (MOPD) at the midpoint to consolidate capabilities before specialized RL; (3) use a small KL penalty (0.03) only during RLHF, not during RLVR stages, to prevent reward hacking while allowing free exploration in verifiable domains; (4) filter training prompts by difficulty to remove noise from impossibly hard problems (Code RL retains only 3.5K prompts out of a much larger set) and trivial successes from easy problems; (5) train agentic capabilities last, with extended context budgets (256K tokens, 200 turns) and lower sampling temperature (0.8) to balance exploration with task completion. These are concrete, validated design rules that practitioners can adopt incrementally — a team could start with IF-RL + MOPD on their existing SFT model and measure whether instruction-following improves without alignment regressions, before committing to the full 7-stage pipeline.

When to Prefer This Method

The paper does not frame Cascade RL + MOPD in explicit opposition to named alternative post-training methodologies (e.g., "pure RLHF," "single-stage multi-domain RL," or "DPO-based alignment"). The introduction positions Cascade RL as a framework that "significantly simplifies the engineering complexity associated with multi-domain RL while achieving state-of-the-art performance" — a statement about its advantages, not about when alternatives are preferable. The body of the paper describes specific interference effects (IF-RL damages alignment, RLVR shortens reasoning traces, mixing thinking/non-thinking modes during RLHF degrades instruction-following) but does not systematically compare Cascade RL against a well-defined alternative pipeline under controlled conditions. The closest comparison is MOPD vs. RLHF in Table 3, which shows MOPD achieving higher ArenaHard scores in fewer steps, but this is a comparison of training stages within the pipeline, not a comparison of the full pipeline against an alternative.

Without a head-to-head evaluation against a specific competing methodology (e.g., "Cascade RL full pipeline vs. single-stage blended multi-domain RL at equivalent total FLOPs"), constructing a "prefer A when, prefer B when" decision rule would require the reviewer to extrapolate beyond the paper's evidence. The paper provides data to support Cascade RL's effectiveness for its chosen model and benchmarks, but provides no data on when a simpler approach would achieve equivalent or better results. The agentic benchmark deficits (Table 1) hint that Cascade RL may not be optimal for domains where the base model lacks strong pretraining priors, but this is an observation about the pipeline's limitations, not an empirical comparison against a named alternative that performs better on those domains. A fair decision framework would need controlled experiments comparing Cascade RL to, e.g., single-stage GRPO on the full multi-domain blend, or iterative DPO, at matched compute budgets — experiments the paper does not perform.

🎯 Pitch

1. Executive Summary

2. Context and Motivation

The Core Problem: Scaling RL Post-Training Across Many Diverse Domains Without Destabilizing the Model

The Catastrophic Forgetting Problem in Multi-Domain RL

Why Prior Approaches Fall Short

How This Paper Positions Itself

3.4.4 Multi-Domain RL — Stage 2

3.4.5 Multi-Domain On-Policy Distillation (MOPD) — The Stabilization Mechanism

3.4.6 Reinforcement Learning from Human Feedback (RLHF) — Stage 4

3.4.7 Long-Context RL — Stage 5

3.4.8 Code RL — Stage 6

3.4.9 Software Engineering Reinforcement Learning (SWE RL) — Stage 7

3.4.10 Summary of Design Choices and Their Justifications

4. Key Insights and Innovations

Innovation 1: Multi-Domain On-Policy Distillation as a Stabilization Mechanism Within Sequential RL

Innovation 2: Domain-Ordering as a Dynamic Function of Model Behavior, Not a Static Recipe

Innovation 3: Agentless-to-Agentic Skill Transfer as an Empirical Principle for SWE Training

5. Experimental Analysis

Evaluation Methodology

Main Quantitative Results

Competitive Reasoning: IMO, IOI, and ICPC World Finals

Math Benchmarks

Code Reasoning Benchmarks

Alignment and Instruction-Following

Agentic Tasks

Knowledge and STEM

Long Context

Multilingual

Ablation Studies and Robustness Checks

Critical Assessment

6. Limitations and Trade-offs

6.1 The Cascade RL Pipeline's Performance Is Fundamentally Bounded by the Base Model's Pretraining Quality

6.2 The IMO, IOI, and ICPC Results Inextricably Combine Post-Training Quality with Test-Time Compute, Making It Impossible to Attribute Gains to Either Factor

6.3 Multi-Domain On-Policy Distillation Is Never Directly Ablated Against the Pipeline Without It

6.4 The Difficulty Estimation and Data Filtering Heuristics Are Critical but Not Characterized or Ablated

6.5 The Evaluation Protocol for Agentic Tasks Uses Non-Standard Inference Configurations That Inflate Scores Relative to Official Protocols

6.6 The Entire Pipeline Is Demonstrated on a Single Base Model, Making It Unknown Whether Cascade RL + MOPD Generalizes to Other Architectures or Scales

7. Implications and Future Directions

How This Work Changes the Landscape

Follow-Up Research This Work Enables

Practical Applications and Downstream Use Cases

When to Prefer This Method

Adjacent in domain

Online Experiential Learning for Language Models

Beyond Single Tokens: Distilling Discrete Diffusion Models via Discrete MMD