JustRL: Scaling a 1.5B LLM with a Simple RL Recipe

🎯 Pitch

A single-stage RL recipe with fixed hyperparameters beats complex multi-stage approaches for 1.5B reasoning models, using half the compute. Adding standard ‘stability tricks’ like length penalties actually backfires by killing exploration.

1. Executive Summary

This paper introduces JustRL, a deliberately minimal reinforcement learning recipe for improving small language models on mathematical reasoning that uses only single-stage GRPO training with fixed hyperparameters — no multi-stage pipelines, dynamic schedules, curriculum learning, or length penalties. Applied to two 1.5B-parameter models (DeepSeek-R1-Distill-Qwen-1.5B and OpenMath-Nemotron-1.5B) on the DAPO-Math-17k dataset, JustRL achieves 54.9% and 64.3% average accuracy across nine mathematical benchmarks respectively, matching or exceeding complex multi-stage approaches while using 2× less compute (e.g., 1.4×10^8 tokens vs. ProRL-V2's 2.8×10^8 on the DeepSeek backbone). The training exhibits smooth, monotonic improvement over 4,000+ steps without the reward collapses, entropy drift, or length explosions that typically motivate sophisticated interventions, and ablations reveal that adding commonly-used "stability" techniques — explicit length penalties and more permissive robust verifiers — actively degrades performance by collapsing policy exploration, establishing that complexity in small-model RL may often address symptoms of other design choices rather than fundamental challenges.

2. Context and Motivation

The Core Problem: Is Complexity in RL for Small Language Models Necessary?

The fundamental question this paper tackles is disarmingly direct: when applying reinforcement learning to improve small language models on reasoning tasks, is the growing complexity of modern training recipes actually required? Over the past year, the field has developed an increasingly elaborate toolkit — multi-stage training, dynamic hyperparameter schedules, adaptive temperature controls, curriculum learning, length penalties, and various stabilization mechanisms — all justified by the need to prevent training instability. JustRL asks whether these interventions solve real problems or merely compensate for issues introduced by other design choices.

This question matters because, as the authors note in Section 1, the prevailing wisdom has shifted toward viewing RL for small models as fundamentally unstable. Many recent papers explicitly cite "training instabilities, like reward collapse, entropy drift, and length explosion" as their motivation. If this premise is correct, then building ever-more-sophisticated training pipelines is the only path forward. But if the premise is flawed — if a simple, scaled-up baseline can achieve stability and competitive performance without these interventions — then the field may be investing enormous engineering effort in solving problems that don't actually exist at adequate scale.

The significance goes beyond mere methodology. As the paper argues, when "different works combine different subsets of methods and report varying results, it becomes unclear what truly drives performance." The accumulated complexity creates a fog: no one knows which techniques genuinely help, which are neutral, and which actively harm. More concerning, "many recent works... apply these techniques on top of already-complex baselines. This makes it impossible to know whether new methods provide genuine benefits or simply compensate for issues introduced by prior complexity." The result is a literature where "the accumulated 'best practices' may be fighting each other rather than the fundamental challenges of RL" — a claim the authors support by citing Liu et al. (2025d).

The Distillation Trap and the Case for RL

The paper identifies a structural limitation in the current paradigm for improving small models. As described in Section 1, the dominant approach is distillation: supervised fine-tuning on outputs from larger teacher models. Qwen3's strong-to-weak distillation and DeepSeek-R1's distilled variants both demonstrate this strategy's effectiveness. For organizations deploying small models, distillation is efficient, stable, and delivers immediate gains.

But distillation has a fundamental ceiling: it is bounded by the teacher model's capabilities. The paper makes this point precisely:

"When researchers rely on distillation to improve the performance of smaller models, they encounter an upper bound, especially when the teacher model's updates are infrequent. Even with increased data and extended training, further gains in performance become difficult to achieve once the teacher model's performance plateaus."

This is not a hypothetical concern. The models used as starting points in Table 1 — DeepSeek-R1-Distill-Qwen-1.5B, OpenMath-Nemotron-1.5B, and Qwen3-1.7B — are all distillation products, and they represent the practical frontier of what distillation can achieve at this scale. To push beyond that frontier, RL is not just an option; it is the only mechanism that allows the model to generate and learn from its own exploration, discovering solution strategies that no teacher has demonstrated. The paper positions RL as the logical next step once distillation saturates, not as a replacement for it.

The problem is that RL for small models has gained a reputation for instability, creating a tension: the field knows RL is the way forward, but believes it requires ever-increasing sophistication to work reliably. This is the gap JustRL addresses — not whether RL works (that is established), but whether the accumulated complexity surrounding it is actually necessary.

The Proliferation of Techniques: A Pattern of Increasing Complexity

Section 2 provides a detailed taxonomy of recent approaches, and the trend is unmistakable. The paper traces this evolution across three backbone model families, and the progression reveals a consistent pattern of layering techniques:

Early work established baselines. STILL-3-1.5B (Min et al., 2024), released January 2025, explored hyperparameter tuning and reference model resets — two techniques that would become standard in subsequent work. At this stage, the recipe was relatively simple: tune your hyperparameters, reset the KL reference periodically, and train.

Multi-stage training became the dominant paradigm. DeepScaleR (Luo et al., 2025) divided training into three stages with increasing context lengths (8K → 16K → 24K), justified as progressively exposing the model to longer reasoning chains. FastCuRL (Song et al., 2025) extended this logic to five stages, alternating between chain-of-thought compression (long-to-short) and extension (short-to-long), with each stage using different data, batch sizes, and rollout counts. ProRL (Liu et al., 2025a) escalated to eight stages with scheduled length penalties. This escalation was motivated by observed training instabilities — models would hit plateaus, or response lengths would explode, and stage transitions were the proposed solution.

Stage transitions spawned further complexity. ProRL-V2 (Hu et al., 2025a), building directly on ProRL, introduced scheduled cosine length penalties while maintaining fixed 8K context — adding a dynamic schedule inside a multi-stage framework. Meanwhile, BroRL (Hu et al., 2025b) took a different but equally complex approach: rather than many stages, it dramatically scaled rollouts to 512 per example (roughly 64× the standard) to exhaustively explore the solution space after first training with ProRL for 3,000 steps. This addresses the plateau problem not through better training but through brute-force exploration.

Curriculum learning and question augmentation added more dimensions. QuestA (Li et al., 2025), working with the OpenMath-Nemotron-1.5B backbone, introduced an innovative curriculum learning approach: augment questions with partial chain-of-thought solutions as hints, providing richer learning signals through staged difficulty progression. This requires not just ground-truth answers but full reasoning trajectories from larger models for curriculum construction — significant additional data requirements and engineering complexity.

Dynamic sampling and adaptive controls proliferated. POLARIS (An et al., 2025), on the Qwen3-1.7B backbone, employed dynamic dataset filtering to focus on challenging problems, combined with adaptive temperature adjustments and test-time context extrapolation across three training stages. e3 (Setlur et al., 2025) used multi-stage training with varying context lengths and leveraged the model's extrapolation abilities at test time.

Table 1 in the paper makes the pattern visually striking. Nearly every entry uses entropy control, but beyond that, the techniques multiply: tune hyperparameters, tune training prompts, reset KL references, length control, adaptive temperature, rollout rescue, dynamic sampling, split training stages. The most complex approaches (ProRL-V2, BroRL, QuestA) employ five or more of these techniques simultaneously.

Where Prior Approaches Fall Short: The Causal Attribution Problem

The paper identifies a specific methodological failure that pervades this literature: impossible causal attribution. When a complex system works, you cannot know which components matter. When it fails, you cannot know which components caused the failure. This is not a minor concern — it is a fundamental scientific problem.

The authors are explicit about this in Section 1:

"When different works combine different subsets of methods and report varying results, it becomes unclear what truly drives performance. More concerning, many recent works cite training instabilities, like reward collapse, entropy drift, and length explosion, as motivation for their techniques, yet apply these techniques on top of already-complex baselines."

The logical flaw is subtle but important. If your baseline already includes multi-stage training, dynamic hyperparameters, and length penalties, and you observe entropy drift, you cannot conclude that entropy drift is an inherent problem of RL for small models. The drift might be caused by the dynamic hyperparameters you introduced to solve some other problem, or by an interaction between your stage transitions and your temperature schedule. The technique you add to fix the drift may be treating a self-inflicted wound.

This creates a dangerous feedback loop in the research literature: initial complexity creates instability, subsequent work adds more complexity to address that instability, the new complexity creates new instabilities, and the cycle continues. The paper cites Liu et al. (2025d), whose title — "Tricks or Traps?" — captures exactly this concern. By starting from a minimal baseline that exhibits none of the pathologies complex methods claim to address, JustRL breaks this cycle and provides a clean testbed for evaluating whether new techniques solve real problems or merely compensate for existing complexity.

The Stability Question: Are the Reported Instabilities Real or Artifactual?

The paper raises a deeper epistemological question about the training instabilities that dominate the literature. These instabilities are widely cited and apparently well-documented:

ProRL-V2 introduces scheduled length penalties after observing length drift
BroRL increases rollouts to hundreds after hitting plateaus
Multiple works (Liu et al., 2025a; Min et al., 2024) apply KL regularization and reset reference models when KL divergence grows too large

These are not armchair observations — they are empirical findings from serious research groups. But the paper questions whether these instabilities are fundamental to the problem or emergent from the specific design choices in the baseline systems where they were observed.

Consider length drift. If you train with a fixed context length and no penalty, the model might naturally converge to a stable response length, as JustRL demonstrates (Figure 2c shows convergence from ~7,000 tokens to 4,000–5,000 within 1,000 steps). But if your baseline includes multi-stage context length changes (8K → 16K → 24K), the model receives conflicting signals about what length is expected, potentially driving the very instability that length penalties are then deployed to fix.

Similarly, consider KL divergence growth. If your baseline resets the reference model periodically, the KL divergence will, by construction, spike after each reset — you have created a sawtooth pattern that then requires management. Without resets, the KL might evolve smoothly.

The paper does not claim these mechanisms are definitively what cause instabilities in prior work. It acknowledges this limitation directly: "We can't isolate which specific complex techniques cause instability versus which ones solve it." But the existence of a simple baseline that trains stably for 4,000+ steps without any of these interventions shifts the burden of proof: researchers proposing new techniques must now demonstrate that the instability they're addressing exists in a minimal, well-tuned baseline, not just in their particular complex pipeline.

How JustRL Positions Itself

The paper's positioning is carefully calibrated — it does not claim to have found the optimal approach or to disprove all existing techniques. Instead, it makes a methodological argument with empirical backing:

"Our goal is not to argue against all techniques or claim we've found the optimal approach. Rather, we provide evidence that simpler baselines deserve more attention than they've received. We offer a simple practice with a minimum set of tricks that can enhance the performance of models that are approaching their distillation limits."

This is a plea for epistemic hygiene in the RL-for-reasoning literature: establish what a simple, well-tuned baseline can achieve before layering on complexity. The paper's epigraph from Antoine de Saint-Exupéry captures this ethos: "Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away."

The paper also positions itself relative to a practical concern: the field is investing enormous engineering effort in techniques whose individual contributions are unknown. Table 1 shows nine different techniques spread across nine different models, with each model using a different subset. No one has systematically ablated these techniques against a common baseline. JustRL provides that baseline — a stable, high-performing, single-stage recipe that can serve as a testbed for evaluating whether new techniques provide genuine benefits.

Finally, the paper implicitly challenges the prevailing narrative that "RL for small models is hard." If a minimal recipe with fixed hyperparameters can train two different backbones stably for 4,000+ steps and match or exceed state-of-the-art results, then the difficulty may lie not in RL itself but in the accumulated complexity surrounding it. The paper's title — "Scaling a 1.5B LLM with a Simple RL Recipe" — emphasizes this reframing: scale, not sophistication, may be the key variable.

Practical Implications of the Complex vs. Simple Debate

Beyond the methodological argument, this paper addresses concerns with direct practical consequences:

Reproducibility and accessibility. Complex multi-stage pipelines with dynamic schedules are difficult to reproduce. Each stage transition may depend on monitoring specific metrics (KL divergence, reward plateaus, length drift) and making judgment calls about when to advance. This creates a barrier to entry for researchers with limited compute or experience. A single-stage recipe with fixed hyperparameters is trivially reproducible — the paper releases all hyperparameters in Table 2.

Compute efficiency. If additional techniques consume compute without improving performance, they are actively harmful in resource-constrained settings. Table 4 shows that ProRL-V2 uses 2× the token budget of JustRL-DeepSeek (2.8×10^8 vs. 1.4×10^8), and BroRL uses 4.9× more. If these compute differences do not translate to performance gains, the complexity is not just unnecessary — it is wasteful.

Transferability. The fact that the same hyperparameters work for both DeepSeek-R1-Distill-Qwen-1.5B and OpenMath-Nemotron-1.5B without per-model tuning (Section 4.2) suggests the recipe captures something fundamental rather than being overfit to a specific model. Complex approaches that require per-model stage design, per-stage hyperparameters, and careful monitoring are inherently less transferable.

The ablation lesson. Perhaps most striking is the paper's ablation finding (Section 4.4): adding two "standard tricks" — an overlong penalty and a robust verifier — actively degrades performance, with the overlong penalty collapsing exploration entropy from 1.2–1.4 to 0.5–0.6 (Figure 3b). This directly demonstrates the risk the paper is concerned about: techniques that are assumed to be improvements based on their success in other contexts may, when added to a stable baseline, break the delicate balance that enables continuous learning. The field has been adding these techniques under the assumption that they help; the ablation shows they can hurt, and the absence of clean baselines has masked this fact.

3. Technical Approach

3.1 Reader Orientation

What is being built: A self-contained reinforcement learning training procedure that takes a 1.5B-parameter language model already fine-tuned on mathematical reasoning (via distillation from larger models) and substantially improves its problem-solving capability through trial-and-error learning on math problems with verifiable answers. What problem it solves: The procedure enables small models to surpass the performance ceiling imposed by distillation from fixed teacher models, without requiring the multi-stage training pipelines, dynamic hyperparameter schedules, or specialized stabilization techniques that have become standard in the literature — demonstrating that a single-stage recipe with fixed settings can match or exceed complex approaches while using half the compute.

3.2 Big-Picture Architecture (Diagram in Words)

The JustRL system has five major components arranged in a simple, linear pipeline:

Base Language Model (1.5B parameters) — a pretrained and distilled model that serves as the starting policy. The paper works with two backbones: DeepSeek-R1-Distill-Qwen-1.5B and OpenMath-Nemotron-1.5B. This model generates candidate solutions given math questions.
Training Dataset (DAPO-Math-17k) — a fixed set of 17,000 math question-answer pairs used without offline filtering or dynamic sampling. Questions come with ground-truth answers that enable automatic reward computation.
GRPO Training Loop (veRL framework) — the core RL algorithm that iteratively: (a) samples 8 rollouts per question from the current policy, (b) scores each rollout using a rule-based verifier, (c) updates the policy using Group Relative Policy Optimization (GRPO), and (d) repeats for thousands of steps with all hyperparameters held constant.
Rule-Based Verifier (DAPO verifier) — a lightweight, deterministic scoring function that checks whether the model's final boxed answer matches the ground-truth answer. Returns binary rewards (correct or incorrect) without relying on symbolic math libraries.
Auxiliary Evaluation Verifier (CompassVerifier-3B) — a model-based verifier used only at evaluation time to reduce false negatives (correct solutions marked wrong due to formatting mismatches between the model's output and the rule-based verifier's parsing expectations).

Information flows as follows: a batch of training questions enters → the current model generates 8 candidate solutions per question → the rule-based verifier scores each candidate → GRPO computes advantage estimates from the group scores and updates the model → the process repeats for 3,000–4,400 steps → the final model is evaluated on nine held-out math benchmarks using the auxiliary verifier for scoring.

3.3 Roadmap for the Deep Dive

First, the GRPO algorithm and its mathematical formulation, because this is the optimizer at the heart of the training loop — understanding what it optimizes and how it differs from standard PPO is prerequisite for everything else.
Second, the training data and reward signal, because RL is fundamentally defined by what you reward — the choice of dataset (DAPO-Math-17k), verifier design (DAPO rule-based), and what is deliberately kept simple (no curriculum, no filtering, no augmentation) determine the learning dynamics.
Third, the fixed hyperparameter configuration, because the paper's central claim is that complexity is unnecessary — every hyperparameter choice (learning rate, temperature, batch size, clip range, rollout count, context length) and its justification must be examined to understand what "simple" actually means.
Fourth, the "clip higher" entropy control mechanism, because this is the one technique JustRL retains from the standard toolkit and the paper argues it is fundamental to stability — understanding why it's kept while everything else is removed reveals the design philosophy.
Fifth, the evaluation protocol and auxiliary verifier, because the evaluation setup introduces a model-based verifier not used in training, which raises questions about the relationship between training signal and measured performance.
Sixth, the ablation experiment design, because the paper's negative results (adding length penalties and robust verifiers degrades performance) are central to its methodological argument and require understanding both what was tested and what was deliberately not tested.

3.4 Detailed, Sentence-Based Technical Breakdown

This is a systems and methodology paper whose core claim is that reinforcement learning for small language models can be made simple: a single-stage GRPO training loop with entirely fixed hyperparameters, using a basic rule-based verifier on standard question-answer pairs, trains stably for thousands of steps and matches or exceeds the performance of complex multi-stage approaches. The contribution is not a novel algorithm or architecture but rather a carefully constructed minimal baseline that isolates what is actually necessary for stable RL in this setting, providing evidence that the field's accumulated complexity may be addressing self-inflicted problems rather than fundamental challenges.

GRPO: The Core Reinforcement Learning Algorithm

The paper uses Group Relative Policy Optimization (GRPO), implemented through the veRL framework (Sheng et al., 2025), as specified in Section 3.1. GRPO is a variant of policy gradient methods that differs from standard PPO in how it computes advantages — instead of using a learned value function (critic) to estimate the expected return and compute advantages, GRPO computes advantages within each group of rollouts for the same prompt. This eliminates the need for a separate value network, reducing memory and compute requirements.

The GRPO objective. At each training step, for each question $q$ in the batch, the model generates $N = 8$ complete solutions (rollouts) using the current policy. Each rollout receives a binary reward $r \in \{-1, 1\}$ from the verifier (details in the next subsection). The GRPO algorithm then:

Groups rollouts by prompt. For each question, the 8 rollouts form a group. Within each group, the algorithm computes the mean reward $\bar{r}$ and standard deviation $\sigma_r$ of the 8 rewards.
Computes normalized advantages. For each rollout $i$ in the group:

$A_i = \frac{r_i - \bar{r}}{\sigma_r + \epsilon}$

where $r_i$ is the binary reward for that rollout, $\bar{r}$ is the group mean reward, $\sigma_r$ is the group standard deviation of rewards, and $\epsilon$ is a small constant for numerical stability.

What it computes: the number of standard deviations this rollout's reward is above or below the group average. A correct answer in a group where most answers are wrong gets a large positive advantage; a correct answer where most are correct gets a small positive advantage. Crucially, this normalization is done independently per question, so advantage magnitudes are calibrated to the difficulty of each specific question — hard questions where only 1/8 rollouts succeed produce larger advantages for the successful rollout than easy questions where 7/8 succeed.

Why this form: normalizing within the question group means the policy update signal is based on relative performance rather than absolute reward magnitude. This is important because binary rewards (correct/wrong) have no inherent scale — a correct answer on an Olympiad-level problem should not necessarily produce the same update magnitude as a correct answer on a simple arithmetic problem. The group normalization automatically adjusts for question difficulty. It also eliminates the need for a critic network: without this normalization, the algorithm would need to learn the expected reward for each question (which is what a value function does) to determine whether a particular outcome was better or worse than expected. The group-based normalization provides this baseline directly from the current batch.
Updates the policy. The policy is updated to maximize the clipped surrogate objective, following the standard PPO-style formulation:

$L^{\text{CLIP}}(\theta) = \mathbb{E}_t \left[ \min \left( \rho_t(\theta) A_t, \ \text{clip}(\rho_t(\theta), 1 - \epsilon_{\text{low}}, 1 + \epsilon_{\text{high}}) A_t \right) \right]$

where $\rho_t(\theta) = \frac{\pi_\theta(a_t | s_t)}{\pi_{\theta_{\text{old}}}(a_t | s_t)}$ is the probability ratio between the current policy and the old policy for the action taken, $A_t$ is the group-normalized advantage computed above, and $\epsilon_{\text{low}}, \epsilon_{\text{high}}$ are the clip bounds.

The paper specifies the clip range as $[0.8, 1.28]$ (Table 2), meaning $\epsilon_{\text{low}} = 0.2$ and $\epsilon_{\text{high}} = 0.28$ . The probability ratio $\rho_t$ is clipped to stay within $[0.8, 1.28]$ .

What it computes: a conservative policy update that prevents the new policy from deviating too far from the old policy in a single step. For each token in the generated sequence, the objective takes the minimum of (a) the unclipped advantage-weighted probability ratio and (b) the clipped ratio. When the advantage is positive, the objective encourages increasing the probability of the tokens that led to the good outcome, but the clip at 1.28 prevents the probability from being increased by more than 28% relative to the old policy. When the advantage is negative, the clip at 0.8 prevents the probability from being decreased by more than 20%. The $\min$ operation ensures the update is a lower bound — it is always conservative in the direction that would otherwise produce the largest change.

Why this form: the asymmetric clip range (0.8 to 1.28 rather than symmetric 0.8 to 1.2) reflects the "clip higher" strategy discussed in Section 3.1 — allowing slightly more room for probability increases than decreases. This asymmetry encourages exploration by making it slightly less costly to increase probabilities of successful actions than to decrease probabilities of unsuccessful ones. The clip mechanism itself is standard in PPO and prevents catastrophic policy collapse: without clipping, a single high-advantage rollout could cause the policy to update so aggressively that it loses all diversity, producing the entropy collapse that the paper's ablation studies observe when this balance is disrupted (Figure 3b).
No KL regularization term. A notable absence in the GRPO configuration: the paper explicitly sets "Use KL Loss = No" and "Use Entropy Regularization = No" (Table 2). Many RL for LLM approaches include an explicit KL divergence penalty in the loss to keep the policy close to a reference model (typically the initial pre-RL model). The paper does not use a KL penalty at all, relying solely on the PPO clip to constrain policy updates. This is a deliberate design choice that the authors defend by showing stable training dynamics without it (Figure 2a shows entropy between 1.0–1.6 throughout training).

Why no KL penalty: the clip mechanism provides implicit regularization by bounding per-step policy changes. Adding an explicit KL penalty would create a tension: the clip would limit how far the policy can move, and the KL penalty would impose an additional cost for moving away from the reference, potentially double-constraining the policy and limiting how much it can learn. The paper's stable training without KL divergence growth (implicit in the sustained entropy and smooth reward improvement) suggests the clip alone is sufficient when other hyperparameters are well-chosen.

Training batch configuration. The paper specifies (Table 2):

Train batch size: 256 questions
PPO mini batch size: 64
PPO micro batch size per GPU: 1
Rollout N: 8 per question

This means each training step processes 256 questions, with the model generating 8 rollouts per question (2,048 total rollouts per step), and the policy update is performed in mini-batches of 64 rollouts (likely 8 questions' worth of rollouts per mini-batch). The micro batch size of 1 per GPU is a memory optimization typical of large-model RL training, where gradient accumulation across micro-batches simulates a larger effective batch size.

Training Data and Reward Signal

The DAPO-Math-17k dataset. The paper trains on DAPO-Math-17k (Yu et al., 2025), described in Section 3.1. This dataset contains 17,000 mathematical competition problems with ground-truth answers in a format compatible with rule-based verification. The dataset size is notably modest — 17k questions compared to the millions of examples typical in pretraining or instruction tuning — because each question is reused thousands of times: the model encounters the same question repeatedly throughout training, generating new rollouts and receiving new learning signals at each encounter.

What JustRL does NOT do with the data. The paper explicitly enumerates the data-related techniques it avoids (Section 3.1):

No offline difficulty filtering: all 17k questions are used regardless of whether they are trivially easy or impossibly hard for the initial model.
No online dynamic sampling: the model does not selectively focus on questions where it currently performs poorly or filter out questions where it always succeeds.
No curriculum learning: questions are presented in random order, without progression from easy to hard.
No question augmentation: the training data is raw questions and answers, without partial solution hints, chain-of-thought demonstrations, or other enrichment.

This contrasts sharply with prior work. POLARIS uses dynamic dataset filtering to focus on challenging problems (estimated 50% filter ratio). QuestA augments questions with partial chain-of-thought solutions as hints. ProRL and its variants use multi-stage training with different data configurations per stage. By using the entire dataset uniformly, JustRL eliminates multiple design degrees of freedom — and the associated decisions about filtering criteria, curriculum structure, and augmentation strategy — while demonstrating that the model learns effectively from the natural difficulty distribution of the dataset.

The DAPO verifier: rule-based binary rewards. The reward signal comes from a lightweight rule-based verifier adopted from DAPO (Yu et al., 2025), as specified in Section 3.1. The verifier operates as follows:

Extract the final answer. The verifier looks for the model's answer enclosed in \boxed{} in the model's output (as instructed by the training prompt). It parses the content inside the box.
Compare to ground truth. The extracted answer is compared to the ground-truth answer from DAPO-Math-17k using string matching or numeric comparison (the paper does not specify exact matching heuristics, but standard practice for math RL includes normalization like removing whitespace, handling LaTeX formatting, and numeric tolerance for floating-point answers).
Return binary reward. If the answer matches: $r = +1$ . If the answer does not match or no boxed answer is found: $r = -1$ .

Why binary rewards: the paper does not provide partial credit for partially correct reasoning or correct intermediate steps. This is a deliberately sparse signal — the model receives no feedback on its reasoning process, only on whether the final answer is right or wrong. This contrasts with process reward models (PRMs) that score intermediate steps, as used in JustRL's contemporary literature (e.g., the PRM-based search in the earlier example paper). Binary outcome rewards are simpler to implement (no need to train a separate verifier model or define step boundaries) and sufficient when the model can, through repeated sampling, discover which reasoning patterns lead to correct answers and which do not.

Why the DAPO verifier specifically: the paper emphasizes that this verifier does not use symbolic math libraries like SymPy. This is an implementation detail that matters for reproducibility and computational overhead: SymPy-based verification (which can algebraically simplify expressions and check mathematical equivalence beyond string matching) would be more accurate but slower and requires additional dependencies. The DAPO verifier's string-based approach is faster, simpler, and the paper shows it provides a sufficient learning signal.

The reward scale. The paper does not explicitly discuss the choice of $+1$ and $-1$ for rewards rather than $+1$ and $0$ . This is a notable detail because it affects the GRPO advantage computation. With $+1/-1$ rewards, the advantage for a correct answer in a group where 7/8 are wrong is:

$\bar{r} = \frac{7 \times (-1) + 1 \times (+1)}{8} = \frac{-6}{8} = -0.75$

$A_{\text{correct}} = \frac{+1 - (-0.75)}{\sigma_r} = \frac{+1.75}{\sigma_r}$

With $+1/0$ rewards, the same scenario would give:

$\bar{r} = \frac{7 \times 0 + 1 \times 1}{8} = 0.125$

$A_{\text{correct}} = \frac{1 - 0.125}{\sigma_r} = \frac{0.875}{\sigma_r}$

The $+1/-1$ scheme produces advantages that are larger in magnitude relative to the standard deviation, providing stronger positive reinforcement for rare successes (and stronger negative reinforcement for rare failures). This may be particularly important early in training when the model rarely succeeds — the $-1$ reward on failure ensures a clear negative advantage signal that pushes the policy away from unsuccessful strategies, while the $+1$ on the rare success provides a strong positive signal.

The Fixed Hyperparameter Configuration

Section 3.1 emphasizes that all hyperparameters remain fixed throughout training — no scheduling, no annealing, no mid-training adjustments. Table 2 provides the complete configuration. Each choice reflects a deliberate decision to avoid the complexity of dynamic schedules while maintaining training stability.

Learning rate: 1e-6 (constant). The learning rate is set to $1 \times 10^{-6}$ and never changed. This is a relatively low learning rate for RL fine-tuning (standard PPO for language models often uses $1 \times 10^{-5}$ or higher), and the paper's stability likely depends on this choice. A lower learning rate means each policy update is small, which reduces the risk of catastrophic updates that could collapse the policy's diversity. The tradeoff is that training takes longer (4,380 steps for the DeepSeek backbone), but with a fixed LR there is no need to decide when to decay, how fast to decay, or what final LR to target.

Temperature: 1.0 (constant). The sampling temperature for generating rollouts is fixed at $1.0$ . Temperature controls the randomness of the sampling process: temperature = 1.0 means sampling from the model's unmodified probability distribution (standard softmax). Many RL approaches use higher temperatures (1.2–2.0) to encourage exploration or lower temperatures (0.6–0.8) later in training to reduce variance. Adaptive temperature scheduling (increasing or decreasing temperature based on training metrics) is common in the literature (Table 1 shows AT=Adaptive Temperature used by ProRL, e3, POLARIS, and others). JustRL keeps temperature constant, relying on the natural entropy of the policy and the group-based sampling (8 rollouts per question) to provide sufficient exploration.

Why temperature 1.0: the standard temperature may be sufficient because the base model (post-distillation) already has meaningful diversity in its outputs — it generates different reasoning paths for the same question. If the base model were nearly deterministic (always producing the same answer), a higher temperature would be needed to force exploration. The smooth entropy curve in Figure 2a (oscillating between 1.0 and 1.6) suggests the base model has sufficient initial entropy that additional temperature-induced randomness is unnecessary.

Context length: max prompt 1k, max response 15k (total 16k). The model sees up to 1,000 tokens of prompt (the question plus the instruction to reason step by step and box the answer) and can generate up to 15,000 tokens of response, for a total context window of 16,000 tokens. This is a single, fixed limit — in contrast to DeepScaleR's 8K → 16K → 24K progression and ProRL's 8K → 16K transitions. The paper makes a deliberate choice to allow long responses from the start rather than progressively increasing the context.

Why fixed 16K context: progressively increasing context length forces the model to adapt to a new maximum at each stage transition, potentially destabilizing training as the model must re-learn what response lengths are optimal. By setting a generous limit from the start (15K response tokens is more than sufficient for most math solutions), the model can naturally discover the appropriate length through experience — and Figure 2c shows it does: starting at ~7,000–8,000 tokens and compressing to 4,000–5,000 within the first 1,000 steps. The model learns conciseness from the reward signal (unnecessarily long responses don't increase the probability of a correct answer, so there's no selective pressure for length), not from an explicit constraint that tightens over time.

Rollout N: 8 per question. The PPO mini batch configuration (mini batch size 64, micro batch size 1 per GPU, rollout N = 8) means each question in the training batch generates 8 independent rollouts, and the GRPO algorithm computes advantages from the group statistics of these 8 rollouts. The choice of 8 rollouts per question is a balance: fewer rollouts (e.g., 4) would provide noisier advantage estimates (the group mean would be a less reliable baseline); more rollouts (e.g., 16 or 32, as used in some prior work) would provide more accurate advantages but at higher computational cost per step.

Why 8 rather than more: BroRL demonstrated that dramatically increasing rollouts to 512 per example can push performance further (Table 3 shows BroRL achieves 57.5% on AIME24 vs. JustRL's 52.6%), but at the cost of 4.9× more compute (Table 4). JustRL's choice of 8 rollouts is a computational efficiency decision — the paper's argument is that competitive performance is achievable without extreme rollout counts, and the smooth training curves suggest 8 rollouts provide sufficient advantage signal.

Clip ratio range: [0.8, 1.28]. As discussed in the GRPO section, the asymmetric clip allows slightly more room for probability increases than decreases. This is the "clip higher" technique mentioned in Section 3.1.

No KL divergence loss, no entropy regularization. These are both explicitly set to "No" in Table 2. The paper relies entirely on the PPO clip mechanism and the inherent stability of low learning rate training to prevent policy collapse.

The "Clip Higher" Entropy Control Mechanism

Section 3.1 describes "clip higher" as "the one technique we do use" and frames it as "a well-established practice for stability in long-horizon RL training... part of the baseline rather than an added technique." Understanding what this mechanism does and why it is retained while everything else is stripped away illuminates the paper's design philosophy.

What "clip higher" means concretely. The asymmetric clip range $[0.8, 1.28]$ means the policy update allows probability ratios to increase up to 1.28× the old policy's probability but only decrease to 0.8×. The high-side clip ( $\epsilon_{\text{high}} = 0.28$ ) is 40% larger than the low-side clip ( $\epsilon_{\text{low}} = 0.2$ ). A symmetric clip (e.g., $[0.8, 1.2]$ or $[0.75, 1.25]$ ) would constrain increases and decreases equally.

What problem this solves. During RL training, the policy must balance two competing pressures:

Exploitation: increasing the probability of tokens that lead to correct answers (good actions).
Exploration: maintaining sufficient diversity in the policy to continue discovering new strategies that might be better than what has been found so far.

A symmetric clip treats increases and decreases as equally dangerous, but they are not: an overly aggressive decrease in the probability of an action (discouraging what might be a reasonable alternative strategy) permanently reduces diversity, while an overly aggressive increase in the probability of a successful action can be corrected by subsequent training steps when that action fails on different questions. The asymmetric clip effectively says: "be more careful about eliminating options than about reinforcing good ones."

Why this is retained while other techniques are removed. The paper's ablation studies (Section 4.4) demonstrate that when an overlong penalty is added to the base recipe, exploration collapses — entropy drops from oscillating in the 1.2–1.4 range to a stable but low 0.5–0.6 (Figure 3b). The penalty creates pressure that prematurely converges the policy to a narrow strategy, and once diversity is lost, the model cannot discover improvements. The "clip higher" mechanism protects against this failure mode by making it harder for the policy update to aggressively reduce probabilities. It is therefore fundamental to maintaining the exploration that enables continuous learning over thousands of steps — it is not a "nice to have" but a prerequisite for the simple recipe to work at all.

Connection to GRPO's group-based advantages. The asymmetric clip interacts with the group-based advantage normalization in an important way. In a group where 7/8 rollouts are wrong and 1/8 is correct, the correct rollout's probability ratio will be multiplied by a large positive advantage (as shown in the reward scale discussion above). Without clipping, this could cause the model to dramatically increase the probability of the tokens in that successful trajectory, potentially overfitting to the specific solution path. The 1.28 upper clip bounds this increase, ensuring the policy retains some probability mass on alternative paths that might also lead to correct answers. Meanwhile, the 0.8 lower clip on the 7 wrong rollouts prevents the model from completely abandoning strategies that are close to working but happened to fail on this particular attempt (perhaps due to a single arithmetic error in an otherwise sound approach).

The Training Prompt: Simplicity by Design

Section 3.1 specifies the training prompt as a simple suffix without tuning: "Please reason step by step, and put your final answer within \boxed{}." This prompt is added to each question in the DAPO-Math-17k dataset.

Why this specific prompt. The prompt serves two functions:

Instructions for reasoning: "reason step by step" encourages the model to produce chain-of-thought reasoning, which is essential for math problem solving and provides the "thinking" that RL can optimize.
Formatting requirement: "put your final answer within \boxed{}" ensures the model's answer can be extracted by the rule-based verifier. Without this instruction, the model might produce answers in various formats that the verifier cannot parse, leading to false negatives (correct solutions scored as wrong because the answer format is unexpected).

Why no prompt tuning. Table 1 shows that several prior works tune their training prompts (TTP column). The authors explicitly avoid this, stating the prompt is used "without tuning." This is consistent with the paper's philosophy: prompt tuning adds another dimension of design freedom that can mask the effects of other choices. A carefully tuned prompt might compensate for suboptimal hyperparameters, making it seem like a complex technique is necessary when a simpler prompt would have worked equally well with better hyperparameters. By fixing the prompt, the paper eliminates this confound.

The prompt's role in evaluation. At evaluation time, the same prompt is used (presumably, though the paper doesn't explicitly state this for training vs. evaluation). The CompassVerifier-3B auxiliary verifier is introduced specifically because, even with the boxed format instruction, the model sometimes produces answers that are mathematically correct but formatted in ways the rule-based verifier cannot parse. This auxiliary verifier is not used during training — the model is trained with the rule-based verifier's sometimes-noisy signal, which the paper's ablation (Section 4.4) suggests may actually be beneficial (more on this in the evaluation section below).

The Evaluation Protocol and Auxiliary Verifier

Section 3.2 describes the evaluation protocol, which differs from the training setup in several important ways.

Evaluation benchmarks. Nine mathematical reasoning benchmarks are used:

AIME 2024 and AIME 2025 (American Invitational Mathematics Examination) — challenging competition problems
AMC 2023 (American Mathematics Competitions) — high school competition problems
MATH-500 (Hendrycks et al., 2021) — a standard benchmark of competition-level math problems
Minerva Math (Lewkowycz et al., 2022) — quantitative reasoning problems
OlympiadBench (He et al., 2024) — olympiad-level problems
HMMT Feb 2025, CMIMC 2025, and BRUMO 2025 (Balunović et al., 2025) — recent competition problems designed to evaluate uncontaminated performance

The inclusion of the 2025 benchmarks (HMMT, CMIMC, BRUMO) is significant because these problems were released after the training data cutoff, ensuring the model's performance reflects genuine reasoning capability rather than memorization of training set problems.

Sampling configuration for evaluation. The paper uses Pass@1 accuracy with multiple samples per problem:

For AIME24, AIME25, AMC23, HMMT, BRUMO, and CMIMC: $N = 32$ samples per problem (avg@32)
For MATH-500, Minerva Math, and OlympiadBench: $N = 4$ samples per problem (avg@4)

The generation uses temperature 0.7, top-p 0.9, and a maximum of 32,000 tokens per generation. Note that the temperature (0.7) is lower than the training temperature (1.0), and the maximum generation length (32K) is double the training response length (15K). These are standard choices for evaluation — lower temperature reduces variance for more reliable pass@1 estimates, and longer maximum length ensures the model isn't artificially truncated on problems requiring extensive reasoning.

The auxiliary model-based verifier. The paper introduces CompassVerifier-3B (Liu et al., 2025c) as an auxiliary verifier used at evaluation time only. The motivation is explicit:

"We augment existing systems with CompassVerifier-3B, a lightweight model-based verifier, to address false negatives from rule-based verifiers."

What CompassVerifier-3B does. The rule-based verifier used in training can produce false negatives when the model's answer is mathematically correct but formatted in a way the rule-based parser cannot match to the ground truth. For example, if the ground truth is "42" and the model outputs "\boxed{42.0}", a strict string matcher might reject this. If the model outputs an equivalent but differently formatted expression (e.g., "\boxed{\sqrt{4}}" vs. ground truth "2"), the rule-based verifier would likely fail. CompassVerifier-3B is a 3-billion-parameter model trained to judge answer correctness, and it can recognize mathematical equivalence across formatting differences.

Why not use the model-based verifier during training? The paper does not explicitly state why, but the ablation study in Section 4.4 provides a relevant finding: when a "robust verifier" (from DeepScaleR) is added to the training setup, performance degrades significantly compared to using the stricter rule-based verifier alone. The paper offers two hypotheses (Section 4.4):

Richer reward spectrum: The stricter verifier creates a broader spectrum of outcomes — some correct-looking solutions are marked wrong, forcing the model to learn from more diverse reward signals rather than receiving "perfect" scores for anything approximately correct.
Precision pressure: The rule-based verifier's reliance on precise formatting may pressure the model to develop more robust internal computation, since small formatting errors are punished. A forgiving verifier might fail to encourage the precision required for optimal generalization.

At evaluation time, however, you want to measure what the model actually knows, not penalize it for formatting errors — hence the use of the more permissive verifier for evaluation only.

The reporting metric. The paper reports average accuracy across all nine benchmarks as its primary metric, with per-benchmark scores in Tables 3 and 5. The average gives equal weight to each benchmark regardless of the number of problems it contains, which could skew results toward smaller benchmarks if performance varies significantly across them.

Ablation Experiment Design

Section 4.4 describes two ablation experiments, each starting from the base JustRL recipe and training on the DeepSeek backbone for 3,000+ steps. Understanding the design of these ablations is essential to evaluating the paper's central methodological argument that adding "standard tricks" can degrade performance.

Ablation 1: Adding an overlong penalty. The first ablation adds an explicit length penalty term for the last 4,000 tokens of the model's response, matching the approach used in DAPO (Yu et al., 2025):

"w/ Overlong Penalty: Add an explicit length penalty term for the last 4k tokens (as used in DAPO)"

What the overlong penalty does. The DAPO overlong penalty works as follows: if the model's response exceeds a threshold (in JustRL's case, responses longer than 11,000 tokens, since the max is 15,000 and the penalty applies to the last 4,000), the reward is reduced by a penalty proportional to the excess length. This creates pressure on the model to produce shorter responses. The motivation in prior work is to prevent "length explosion" where models learn that longer responses correlate with higher reward (perhaps because verbose reasoning occasionally stumbles into the correct answer) and become progressively more verbose over training.

Why this was expected to help. The paper hypothesized that "explicitly penalizing verbose responses might improve training efficiency by pushing the model toward conciseness faster." The logic is plausible: shorter responses use fewer tokens, which means faster generation, more training steps per compute budget, and (potentially) more focused reasoning.

What actually happened. Figure 3a shows that the overlong penalty causes AIME 2024 performance to plateau at approximately 50% after ~2,000 steps, compared to ~55% for the base recipe (which continues improving through 3,000+ steps). Figure 3b reveals why: the entropy with the overlong penalty collapses to approximately 0.5–0.6 and stays there, compared to the base recipe's healthy oscillation in the 1.2–1.4 range. The penalty creates pressure that prematurely converges the policy — the model learns to be concise before it has explored what reasoning strategies actually work, and once diversity is lost, it cannot discover improvements. The natural length compression observed in Figure 2c (from ~7,000 to ~4,000–5,000 tokens without any penalty) suggests the reward signal alone is sufficient to encourage efficient reasoning, and explicit penalties disrupt the learning process.

Ablation 2: Overlong penalty + robust verifier. The second ablation adds both the overlong penalty and a more sophisticated verifier adopted from DeepScaleR (Luo et al., 2025):

"w/ Overlong Penalty + Robust Verifier: Further add a more sophisticated verifier from DeepScaleR to reduce false negatives"

What the robust verifier does differently. The DeepScaleR verifier is designed to reduce false negatives — cases where the model produces a mathematically correct answer but the rule-based verifier marks it wrong due to formatting issues. It likely uses more sophisticated parsing (potentially including SymPy or other symbolic math tools) to determine answer equivalence beyond exact string matching. The hypothesis was that "reducing false negatives (correct solutions marked wrong) would provide a cleaner learning signal."

What actually happened. Performance degrades further, plateauing at approximately 45% AIME 2024 (Figure 3a). The entropy collapse is even more severe, dropping below 0.5 (Figure 3b). The paper offers two hypotheses for this counterintuitive result, both discussed in the evaluation section above:

The stricter verifier provides a richer learning signal by creating a spectrum of outcomes rather than binary correct/incorrect.
The stricter verifier's dependence on precise formatting pressures the model to develop robust internal computation.

What the ablations do NOT test. The paper is explicit about the limitations:

Curriculum learning
Adaptive temperature scheduling
Reference model resets (KL regularization resets)
Different verifier designs (beyond the two tested)
Data augmentation or filtering

Some of these untested techniques might improve upon the baseline. The paper's argument is not that no technique ever helps, but that the burden of proof should shift: techniques should be validated against a simple, well-tuned baseline rather than assumed beneficial based on their performance in complex systems where interactions are poorly understood.

Cross-validation and statistical reliability. The paper does not describe using cross-validation or multiple random seeds for the ablation experiments. Each ablation condition appears to be run once. This means the observed degradation could, in principle, be due to random variation in training runs rather than a genuine effect of the interventions. However, the magnitude of the degradation (5–10 percentage points on AIME 2024, which is a substantial gap in competition math benchmarks) and the consistent pattern across both ablations (each adding a "standard trick" independently degrades performance) argues against this being purely noise. The entropy collapse in Figure 3b provides a mechanistic explanation — the interventions broke the exploration that enables learning — which makes the causal story plausible.

Summary of Design Choices and Their Justifications

The JustRL recipe makes a series of deliberate omissions from standard RL-for-LLM practice. Each omission is justified by the paper's empirical findings (training stability without the omitted technique) and its methodological argument (techniques should be added only when a simple baseline demonstrably fails):

Single-stage training over multi-stage: avoids stage transition instabilities and the need to decide when to transition; the smooth training curves show no plateaus that would justify stage changes.
Fixed learning rate over schedules: a low constant LR (1e-6) provides sufficient stability without the complexity of cosine decay, warmup, or adaptive scheduling.
Fixed temperature (1.0) over adaptive temperature: the model's natural entropy (1.0–1.6 range) provides sufficient exploration; adaptive temperature would add complexity without clear benefit.
No KL penalty over KL regularization: PPO clipping alone bounds policy updates sufficiently; adding KL penalty would double-constrain the policy and limit learning.
Fixed 16K context over progressive lengthening: allows natural length convergence driven by reward signal rather than artificial constraints that change over training.
No length penalty over explicit length penalties: the reward signal naturally incentivizes conciseness (Figure 2c); explicit penalties collapse exploration (Figure 3b).
Strict rule-based verifier over robust verifier: stricter verification provides richer learning signal by maintaining a spectrum of outcomes and pressuring the model toward precision (Section 4.4).
"Clip higher" retained: the one technique kept because it protects exploration by making it harder to aggressively reduce token probabilities, preventing the entropy collapse that other ablations demonstrate.

4. Key Insights and Innovations

Innovation 1: Complexity as a Self-Inflicted Wound — The Null-Baseline Diagnostic

The paper's most intellectually distinctive move is not proposing a new technique but rather questioning the premise on which all recent techniques are built. For over a year, the dominant narrative in small-model RL has been that training is fundamentally unstable — reward collapses, entropy drifts, and response lengths explode — and that ever-more-elaborate scaffolding (multi-stage pipelines, dynamic schedules, length penalties) is necessary to prevent failure. JustRL makes a diagnostic move that shifts the debate entirely: what if the instabilities are not properties of RL itself, but emergent from the accumulated complexity in existing baselines?

This is not a claim about algorithm design; it is a claim about scientific method. The paper identifies a specific logical flaw pervading the literature that the authors call impossible causal attribution (Section 2). When a system uses multi-stage training, dynamic hyperparameters, length penalties, and adaptive temperature all simultaneously, and then observes entropy collapse, you cannot know whether entropy collapse is an inherent challenge of RL for small models — or whether it is caused by an interaction between, say, the stage transitions and the temperature schedule you introduced to solve some other problem. The technique you add to fix the instability may be treating a self-inflicted wound.

What makes this insight fundamental rather than incremental is that it inverts the burden of proof. Prior to JustRL, the field operated under the assumption that complexity was necessary; researchers cited training instabilities observed in their own pipelines as motivation, and the question was which combination of techniques to add. JustRL's empirical demonstration — a single-stage recipe with fixed hyperparameters that trains stably for 4,000+ steps on two different backbones (Figures 1 and 2) — forces a different question: do these instabilities exist in a minimal, well-tuned baseline, or are they artifacts of the specific choices in prior work?

The paper is careful not to overclaim. It acknowledges directly that it "can't isolate which specific complex techniques cause instability versus which ones solve it." But the existence of a stable, high-performing simple baseline reframes the research agenda. The appropriate methodological standard is now: establish what a simple baseline achieves, then add complexity only when that baseline demonstrably fails. This is not a new idea in machine learning broadly — it echoes the "Occam's razor" principle and the long-standing advice to start simple — but its application to the RL-for-reasoning literature, backed by competitive results, is what makes it a genuine contribution.

The practical significance goes beyond methodology. Table 4 shows that ProRL-V2 uses 2× the compute of JustRL (2.8×10^8 vs. 1.4×10^8 token budget), and BroRL uses 4.9× more, without commensurate performance gains. If the complexity driving these compute costs addresses problems that don't exist in a simple baseline, the field is not just over-engineering — it is actively wasting resources. The signal-noise ratio in the literature improves when future work must demonstrate that a proposed technique improves over a well-documented simple baseline, not just over the authors' particular prior complex system.

Innovation 2: Exploration Collapse as the Hidden Cost of "Standard Tricks"

The ablation studies in Section 4.4 deliver a finding with implications that extend well beyond JustRL's specific recipe: techniques widely adopted as stability improvements can actively harm performance by collapsing policy exploration, and this harm is invisible without monitoring the right diagnostic signals.

What makes this finding intellectually significant is not the specific result (overlong penalties and robust verifiers degraded performance), but the mechanism the paper identifies and the class of techniques it implicates. Figure 3b provides the diagnostic: when an overlong penalty is added, policy entropy collapses from the healthy 1.2–1.4 oscillation range in the base recipe to a stable but low 0.5–0.6. The robust verifier drives it even lower, below 0.5. This is not a gradual decline — it is a qualitative shift in the learning dynamics, from sustained exploration to premature convergence.

Prior work has documented that overlong penalties can control length explosion (e.g., DAPO, Yu et al., 2025) and robust verifiers can reduce false negatives (e.g., DeepScaleR, Luo et al., 2025). What prior work did not show — and what JustRL's ablation reveals — is the hidden tradeoff: these techniques succeed at their stated goal (controlling length, reducing verification noise) but at the cost of killing the exploration that enables sustained improvement. The model learns to be concise before it has explored what reasoning strategies actually work. The verifier forgives imprecision, so the model never develops it.

The paper's counterfactual strengthens this argument. The base recipe, without any length penalty, naturally compresses response length from ~7,000 to ~4,000–5,000 tokens within the first 1,000 steps (Figure 2c), driven solely by the reward signal — verbose reasoning that doesn't improve correctness gets no selective advantage, so the model converges to more efficient chains of thought organically. This organic compression preserves exploration (entropy stays high) while achieving the efficiency that explicit penalties are designed to enforce. The contrast between what the penalty is supposed to do (encourage conciseness) and what it actually does (collapse exploration, plateau performance 5 percentage points lower) is a diagnostic for the entire class of techniques: interventions designed to solve specific symptoms can disrupt the broader learning dynamics in ways that are invisible if you monitor only the symptom they target.

The robust verifier finding is arguably more surprising and more significant. The intuition that reducing false negatives should help — cleaner signal, faster learning — is so natural that it has become conventional wisdom in the field. The paper's counter-hypothesis — that a stricter verifier provides richer learning signals by maintaining a spectrum of outcomes and pressuring the model toward precision — is not obviously correct a priori. But the empirical result (plateauing 10 percentage points lower than the base recipe) forces a reconsideration: what constitutes a "good" reward signal for RL may depend on the learning dynamics, not just the signal quality in isolation. A permissive verifier that gives partial credit to sloppy reasoning may deprive the model of the precision pressure needed to develop robust computation. This is a conceptual advance in understanding reward design for LLM RL, not just a parameter setting.

These negative results have a methodological implication: the field's default assumption that "standard tricks" are net beneficial needs to be replaced with empirical validation against clean baselines. The overlong penalty and robust verifier are not obscure techniques — they are widely used and cited. Their failure in JustRL's setting suggests that the success of these techniques in prior work may depend on interactions with other design choices (multi-stage training, dynamic hyperparameters) that JustRL deliberately avoids. The paper does not claim these techniques never help, only that their benefits are not universal and their costs are often invisible without comprehensive diagnostics.

Innovation 3: The Training-Dynamics Triad — A Unified View of RL Stability

The paper's detailed tracking of three training signals — policy entropy (Figure 2a), mean reward (Figure 2b), and mean response length (Figure 2c) — is not merely good experimental practice. It represents an implicit conceptual framework for diagnosing RL health that contrasts with the piecemeal approach in prior work, where each signal is monitored separately and each deviation triggers a targeted intervention.

What makes this a conceptual contribution rather than just thorough monitoring is the interdependence the paper reveals between these three signals. Policy entropy staying in the 1.0–1.6 range is not just a stability metric — it is the mechanism that enables the smooth reward improvement (Figure 2b) by ensuring the model continues to explore diverse strategies. The natural length compression (Figure 2c) is not just a happy accident — it is an emergent consequence of sustained exploration combined with reward-driven selection, where the model discovers through experience that conciseness doesn't hurt accuracy and gradually adopts it. Prior work, by contrast, treats each of these as an independent problem to solve: entropy drift → KL regularization, reward plateaus → stage transitions, length explosion → explicit penalties. JustRL's framework suggests these are not independent problems but joint manifestations of a single underlying dynamic: the balance between exploration and exploitation in policy optimization.

This unified view explains the ablation results with a parsimony that piecemeal approaches cannot match. The overlong penalty doesn't just reduce length — it collapses entropy (Figure 3b), which in turn causes the reward to plateau (Figure 3a). The robust verifier doesn't just reduce false negatives — it softens the reward gradient, reducing pressure for precision, which interacts with the entropy collapse to produce even worse plateauing. The failures are not additive; they are multiplicative, because the signals are coupled. You cannot "fix" length without understanding its relationship to exploration, and you cannot "improve" the verifier without understanding its relationship to the precision pressure that drives learning.

This triad framework is not presented as a formal theory in the paper, but it is an intellectual contribution that emerges from the paper's methodological approach: monitor everything, intervene on nothing, observe how the system self-regulates. The contrast with prior work — where each monitored signal triggers a specific intervention, creating a complex system of interacting controls — makes the framework actionable: before adding a technique to control length, ask whether length is self-regulating without it; before adding KL regularization to control entropy, ask whether the clip mechanism alone is sufficient; before splitting training into stages to overcome plateaus, ask whether the plateau is a genuine ceiling or an artifact of your interventions disrupting exploration.

Innovation 4: Cross-Backbone Transfer as Evidence for Fundamental Sufficiency

The paper's demonstration that identical hyperparameters work for two different backbones without per-model tuning (Section 4.2) is a claim about robustness that carries more weight than it might initially appear. Table 5 shows JustRL-Nemotron achieving 64.3% average accuracy across nine benchmarks, slightly outperforming QuestA's 63.8% while using 2.4× less compute (Table 6), with no hyperparameter changes from the DeepSeek recipe.

What distinguishes this from a routine "second experiment" is the inferential value for the paper's central thesis. If JustRL's recipe were carefully overfit to the DeepSeek-R1-Distill-Qwen-1.5B backbone — if its success depended on specific interactions between hyperparameters and that model's particular output distribution, entropy characteristics, or reasoning style — you would expect it to fail or require substantial retuning on a different backbone. The fact that it transfers without modification suggests the recipe captures something fundamental about the RL optimization dynamics rather than being a lucky local optimum for one model.

This transfer is particularly significant given the differences between the backbones. DeepSeek-R1-Distill-Qwen-1.5B starts from an average of 37.7% across the nine benchmarks; OpenMath-Nemotron-1.5B starts from 56.7% — nearly 20 points higher. They are distilled from different teacher models with potentially different training distributions, reasoning patterns, and output styles. A recipe that works for both, with the same learning rate, temperature, clip range, rollout count, and context length, is evidence for the paper's broader claim: that stable RL at this scale is not about carefully tuned per-model complexity, but about getting the fundamentals right and letting the optimization do the work.

The contrast with QuestA sharpens this point. QuestA achieves comparable final performance (63.8% vs. 64.3%) through a curriculum learning approach that augments questions with partial chain-of-thought solutions as hints, requiring "not just ground-truth answers but full reasoning trajectories generated by larger models for curriculum construction with additional data requirements and engineering complexity." JustRL achieves the same level using only the standard question-answer pairs. The fact that both approaches converge to similar final performance — pushing the boundaries of what's achievable at 1.5B scale — suggests that the ceiling may be set by model capacity or data diversity, not by the sophistication of the training algorithm. If true, the field's investment in algorithmic complexity may be targeting a component of the system (the optimizer) that is not the binding constraint.

This insight does not diminish QuestA's contribution — curriculum learning through question augmentation is a clever technique that clearly helps, and in settings with less data or weaker starting models, it might provide benefits that a simple recipe cannot match. But it does strengthen JustRL's methodological argument: before investing in complex data augmentation or curriculum design, try scaling up a simple baseline — the gains may come from the scale, not the sophistication.

Innovation 5: GRPO Without KL Penalties as a Viable and Potentially Superior Default

The paper's deliberate omission of KL divergence regularization from the GRPO objective (Table 2: "Use KL Loss = No") challenges what has become near-dogma in the RL-for-LLM literature. Most approaches — including many of those in Table 1 — include an explicit KL penalty term to constrain the policy from deviating too far from a reference model (typically the initial pre-RL checkpoint), motivated by the concern that unconstrained policy optimization leads to reward hacking, language degradation, or catastrophic forgetting of general capabilities.

JustRL demonstrates that this concern may be overstated in the specific setting of math reasoning RL at this scale, and that the PPO clip mechanism alone (with the asymmetric range [0.8, 1.28]) provides sufficient regularization. The evidence is indirect but compelling: (1) policy entropy remains stable without drift (Figure 2a), indicating the policy is not collapsing to a deterministic mode, (2) mean reward improves monotonically without the sawtooth pattern that KL resets would introduce (Figure 2b), and (3) the model continues to produce diverse, coherent reasoning (evidenced by sustained performance across varied benchmarks in Tables 3 and 5).

The conceptual contribution here is the recognition that KL penalties and PPO clipping may be partially redundant, and that the combination — common in prior work — may over-constrain the policy. A KL penalty is a global constraint: it penalizes any deviation from the reference model, regardless of whether that deviation is beneficial (discovering a new reasoning strategy) or harmful (reward hacking). The PPO clip, by contrast, is a per-step constraint: it limits how much the policy can change in a single update, but allows the policy to drift substantially over many steps if each step is small. This allows exploration of regions of policy space far from the reference model — which is exactly what you want when the goal is to discover reasoning strategies that the reference model (a distillation product) does not possess — while preventing the catastrophic single-step collapse that KL penalties are also designed to prevent.

The practical significance is that removing the KL penalty eliminates a hyperparameter (the KL coefficient, $\\beta$ ) that typically requires tuning, and eliminates the reference model that must be stored in memory and periodically reset. Multiple prior works (ProRL, STILL) explicitly cite KL divergence growth as a training instability requiring intervention — JustRL suggests this instability may arise precisely because the KL penalty creates a tension with the learning objective, requiring resets that then create sawtooth dynamics.

This finding is not fully generalizable — in settings with sparser rewards, longer training horizons, or tasks where language quality degradation is a genuine risk (creative writing, dialogue), a KL penalty might be essential. But for the specific and practically important setting of math reasoning RL at 1.5B scale, the paper establishes that KL penalties are optional rather than necessary, simplifying the recipe and removing a potential source of training instability.

5. Experimental Analysis

Evaluation Methodology

Dataset. The paper evaluates on nine mathematical reasoning benchmarks with answers verifiable via string matching or model-based comparison. The benchmarks span competition-level math of varying difficulty: AIME 2024 and AIME 2025 (Li et al., 2024; Balunović et al., 2025), AMC 2023 (Li et al., 2024), MATH-500 (Hendrycks et al., 2021), Minerva Math (Lewkowycz et al., 2022), OlympiadBench (He et al., 2024), and three recent 2025 competition sets — HMMT Feb 2025, CMIMC 2025, and BRUMO 2025 (Balunović et al., 2025). The 2025 benchmarks are particularly important because they were released after the training data cutoff, ensuring performance reflects generalization rather than memorization.
Base model(s). The paper uses two 1.5B-parameter distilled models as starting points: (1) DeepSeek-R1-Distill-Qwen-1.5B, a model distilled from DeepSeek-R1's outputs (Guo et al., 2025), which achieves 37.65% average accuracy across the nine benchmarks at the evaluation protocol's sampling budget, and (2) OpenMath-Nemotron-1.5B, a model with stronger initial mathematical reasoning capability (56.74% average), allowing the paper to test whether the recipe transfers across different starting competencies. The 1.5B scale is deliberately chosen as representative of the "small language model" regime where distillation from larger models is the dominant paradigm and where RL is viewed as challenging.
Metrics. The primary metric is Pass@1 accuracy, computed as the fraction of problems for which at least one of N sampled solutions contains the correct answer (a problem is correct if any of its N samples matches the ground truth). For AIME24, AIME25, AMC23, HMMT, BRUMO, and CMIMC, the paper reports avg@32 (pass@1 over 32 samples per problem). For MATH-500, Minerva Math, and OlympiadBench, it reports avg@4 (pass@1 over 4 samples per problem). The paper then averages these nine per-benchmark accuracies into a single "Avg" score reported in Tables 3 and 5, giving equal weight to each benchmark regardless of problem count. The generation configuration for evaluation uses temperature 0.7, top-p 0.9, and a maximum of 32,000 tokens per generation — notably, the temperature is lower than training (0.7 vs. 1.0) to reduce variance for reliable pass@1 estimation, and the context window is double the training limit (32K vs. 16K) to avoid artificial truncation on problems requiring extensive reasoning.
Baselines. The paper compares against seven prior methods across the two backbone families, each representing a distinct point in the complexity-performance space. For the DeepSeek-R1-Distill-Qwen-1.5B backbone, the baselines are: (1) the backbone itself without further RL (37.65% avg), representing the distillation ceiling, (2) DeepScaleR-1.5B (Luo et al., 2025), a three-stage training approach with progressive context lengthening (8K → 16K → 24K) achieving 44.88% avg, (3) ProRL-V2 (Hu et al., 2025a), an eight-stage pipeline with scheduled cosine length penalties, dynamic hyperparameters, and split training stages achieving 53.08% avg, and (4) BroRL (Hu et al., 2025b), which continues from ProRL-V2 with 512 rollouts per example achieving 57.50% on AIME24 but with unreleased models and incomplete benchmark coverage (the paper reports BroRL results only where officially available). For the OpenMath-Nemotron-1.5B backbone, the baselines are: (1) the backbone itself (56.74% avg), and (2) QuestA (Li et al., 2025), a curriculum learning approach that augments questions with partial chain-of-thought solutions from larger models, achieving 63.81% avg. The paper also references STILL-3-1.5B, FastCuRL-1.5B, ProRL-V1, e3-1.7B, and POLARIS-1.7B in its technique taxonomy (Table 1) but does not include them in direct numerical comparison tables, likely because they use different backbones (Qwen3-1.7B) or report on different benchmark suites.
Generation budget / compute accounting. The paper measures computational cost in terms of an approximate token budget — the total number of tokens processed during training — computed as: (training steps) × (train batch size) × (rollout N) × (max response length). This is an approximation because actual response lengths vary, but it provides a consistent basis for comparison across methods. For JustRL-DeepSeek: 4,380 steps × 256 batch size × 8 rollouts × 16,000 max context ≈ 1.4×10^8 tokens. For comparison, ProRL-V2 uses approximately 2.8×10^8 tokens (2× more), and BroRL uses approximately 6.8×10^8 tokens (4.9× more). For the Nemotron backbone, JustRL-Nemotron uses 3,440 steps × 256 × 8 × 16,000 ≈ 1.1×10^8 tokens, compared to QuestA's estimated 2.6×10^8 tokens (2.4× more). The paper notes that for methods using dynamic sampling with an estimated 50% filter ratio (following POLARIS, An et al., 2025), the effective token budget could be lower than the nominal maximum, making JustRL's compute advantage estimates conservative — even if dynamic sampling filtered nothing (0% ratio), JustRL's compute use "remains comparable or even lower." The paper's hardware setup is 32 A800-80GB GPUs for approximately 15 days per model, giving practitioners a concrete sense of the infrastructure requirements.
Cross-validation / statistical protocol. The paper does not employ cross-validation or multiple random seeds for its main results or ablation studies. Each model configuration is trained once and evaluated once on the fixed nine-benchmark suite. This is a meaningful limitation: the observed performance differences between JustRL and baselines (e.g., 54.87% vs. 53.08% for ProRL-V2, a 1.79 percentage point gap) are within a range where training variance could matter, and the paper provides no confidence intervals or error bars. The ablation experiments (Section 4.4) similarly report single-run results, meaning the 5–10 percentage point degradation from adding overlong penalties and robust verifiers — while large enough to be unlikely purely from noise — cannot be assigned statistical significance in a formal sense. However, the paper's central claims about stability and simplicity do not depend on fine-grained performance comparisons; they depend on demonstrating that the simple recipe achieves competitive results without the instabilities that motivate complex interventions, which the training dynamics curves (Figures 1 and 2) demonstrate qualitatively regardless of statistical nuance.

Main Quantitative Results

JustRL-DeepSeek-1.5B: Matching Complex Methods with Half the Compute

The paper trains DeepSeek-R1-Distill-Qwen-1.5B for 4,380 steps using the fixed-hyperparameter single-stage recipe. Table 3 reports the full per-benchmark results and the average across all nine benchmarks.

Headline result. JustRL-DeepSeek-1.5B achieves 54.87% average accuracy across the nine benchmarks, compared to 37.65% for the unmodified backbone (a 17.22 percentage point improvement), 44.88% for DeepScaleR-1.5B, and 53.08% for ProRL-V2 — the prior state-of-the-art among released models on this backbone. JustRL leads on six of nine individual benchmarks (AIME25: 38.75% vs. 35.73%, AMC23: 91.02% vs. 88.75%, Minerva: 51.47% vs. 49.03%, Olympiad: 67.99% vs. 67.84%, HMMT: 21.98% vs. 19.38%, BRUMO: 52.71% vs. 47.29%) while ProRL-V2 leads on AIME24 (51.87% vs. 52.60%), MATH-500 (92.00% vs. 91.65%), and CMIMC (25.86% vs. 25.63%). BroRL is reported with incomplete benchmarks and unreleased models, achieving 57.50% on AIME24 (compared to JustRL's 52.60%) and 92.14% on MATH-500 (compared to JustRL's 91.65%), but with approximately 4.9× the token budget.

The compute efficiency claim. Table 4 quantifies the resource comparison. JustRL-DeepSeek uses approximately 1.4×10^8 tokens total training budget. ProRL-V2 uses approximately 2.8×10^8 tokens — roughly twice as much — when accounting for its dynamic sampling (which filters an estimated 50% of examples, reducing effective tokens below the nominal maximum; this filter ratio follows POLARIS's reported practice and makes JustRL's 2× advantage the most conservative estimate). ProRL-V2 also continues from ProRL-V1 (+1,000 steps), meaning its total lineage includes ProRL-V1's 2,450 steps plus its own 1,000 steps, for a combined lineage substantially longer than reported. BroRL uses approximately 6.8×10^8 tokens (4.9× more than JustRL), achieved by dramatically scaling rollouts to 512 per example, but BroRL builds on ProRL-V2's training (+191 steps), so its total lineage cost is even higher.

Table 4 also reveals differences in the structural composition of compute: DeepScaleR uses 1,750 steps with 128 batch size and 8 rollouts but with progressive context lengthening (8K → 16K → 24K); ProRL-V2 uses 256 batch size with 16–32–16 rollouts varying across stages; JustRL uses a constant 256 batch size, 8 rollouts, and 16K context throughout. The paper's efficiency comes from avoiding the rollout scaling (ProRL-V2's up to 32 rollouts per example in some stages) and context length increases that inflate token consumption without, in the paper's assessment, proportional performance benefits.

Training trajectory. Figure 1(a) shows the AIME24 avg@32 score over the 4,380 training steps, starting from approximately 28% (the backbone's performance) and climbing to roughly 58% by step 4,000 before settling at 52.6% at evaluation (the difference between the training curve's peak and the final evaluation score may reflect the different sampling configurations — the training curve tracks avg@32 on AIME24 only, while the final evaluation uses the full nine-benchmark protocol). The curve is the paper's Exhibit A for stability: "smooth, monotonic improvement over 4,000+ steps without the collapses or oscillations often cited as motivation for complex interventions." There are no visible plateaus requiring intervention, no reward dips requiring recovery, no divergences requiring mid-training corrections. The curve does show some noise (expected in RL training), but the trend is consistently upward without structural breaks.

JustRL-Nemotron-1.5B: Cross-Backbone Transfer Without Hyperparameter Tuning

The paper trains OpenMath-Nemotron-1.5B for 3,440 steps using the identical recipe — same learning rate, temperature, clip range, rollout count, context length, and optimizer configuration — with no per-model tuning. Table 5 reports the full results.

Headline result. JustRL-Nemotron-1.5B achieves 64.32% average accuracy, slightly exceeding QuestA's 63.81% while improving over the backbone's 56.74% by 7.58 percentage points. The gains are distributed across benchmarks: AIME24 improves from 58.75% to 69.69% (+10.94 points), AIME25 from 48.44% to 62.92% (+14.48 points), AMC23 from 90.55% to 96.02% (+5.47 points), MATH-500 from 92.40% to 94.15% (+1.75 points), Minerva from 26.93% to 30.24% (+3.31 points), Olympiad from 71.70% to 76.59% (+4.89 points), HMMT from 30.10% to 40.63% (+10.53 points), BRUMO from 61.67% to 66.88% (+5.21 points), and CMIMC from 30.08% to 41.72% (+11.64 points). The only benchmark where JustRL-Nemotron does not lead against QuestA is AIME24 (69.69% vs. QuestA's 71.56%, a 1.87 point gap), Minerva (30.24% vs. 32.08%, a 1.84 point gap), and BRUMO (66.88% vs. 67.50%, a 0.62 point gap). The paper leads on the remaining six benchmarks, though the margins are generally small — typically 0.5–4 percentage points — consistent with both methods pushing against the same ceiling at the 1.5B scale.

Compute efficiency on the Nemotron backbone. Table 6 reports that JustRL-Nemotron uses approximately 1.1×10^8 tokens compared to QuestA's 2.6×10^8 tokens after accounting for dynamic sampling with an estimated 50% filter ratio. The paper holds that QuestA's actual effective token count is likely lower after filtering (since dynamic sampling discards examples where the model gets 8/8 or 0/8 correct rollouts, which are uninformative for learning), making the 2.6×10^8 estimate a nominal maximum. Even without any filtering, the comparison favors JustRL, though the exact multiple depends on QuestA's realized filter rate. QuestA achieves its results through a fundamentally different approach — curriculum learning with question augmentation (partial chain-of-thought solutions from larger models) — so the compute comparison is measuring efficiency of two different learning paradigms, not just training configuration.

Training trajectory. Figure 1(b) shows the AIME24 avg@32 score climbing from approximately 59% to 70+% over 3,000 steps with similar smoothness to the DeepSeek curve. The curve again shows no structural breaks, plateaus, or collapses. The paper makes the specific interpretive point that "the fact that the same recipe works for both models without hyperparameter tuning suggests genuine robustness rather than lucky optimization for a single model." This is an existence proof: simple, single-stage RL with fixed hyperparameters can work on more than one starting point, which argues against the necessity of per-model complexity.

Interpreting the gap between DeepSeek and Nemotron results. The two backbones start at very different competence levels (37.65% vs. 56.74% average) and end at different absolute levels (54.87% vs. 64.32%). The relative improvement is larger for the weaker backbone (17.22 percentage points, a ~46% relative gain) than for the stronger one (7.58 points, a ~13% relative gain), which is consistent with diminishing returns as models approach the capability ceiling imposed by parameter count and data diversity. The paper's key inferential move is that the recipe transfers at all — the same LR, temperature, and clip range work for models at different points on their learning curves, suggesting the recipe isn't exploiting a narrow region of hyperparameter space specific to one model's training dynamics.

Training Dynamics Analysis: The Three-Signal Diagnostic

The paper tracks three signals over the 4,380 steps of JustRL-DeepSeek training, presented in Figure 2. These are not "results" in the sense of benchmark scores but are central to the paper's argument because they demonstrate the absence of the pathologies that prior work cites as motivation for complexity.

Policy entropy (Figure 2a). The paper reports that "policy entropy oscillates between 1.0 and 1.6 at later training steps, with no systematic drift upward (exploration collapse) or downward (premature convergence)." The curve shows the entropy starting at approximately 1.3–1.4, dipping slightly in the first few hundred steps, and then settling into a stable oscillation band between 1.0 and 1.6 that persists through the entire 4,000+ steps. This stability is what the "clip higher" mechanism is designed to protect: the asymmetric clip makes it harder for the policy update to aggressively reduce probabilities of tokens that might be useful in unexplored contexts, maintaining the diversity that enables continued learning.

The stable entropy directly contradicts the narrative that RL for small models inevitably suffers from "entropy drift" requiring KL penalties or reference model resets. Multiple works in Table 1 (ProRL, STILL, POLARIS) cite entropy-related instabilities as motivation for their techniques — JustRL shows these instabilities simply don't occur in a properly configured simple baseline, suggesting they may be artifacts of other design choices in those works' baselines.

Mean training reward (Figure 2b). The mean reward "climbs from around -0.6 to +0.4 over training. The curve is noisy but the trend is unmistakably upward." The starting reward of approximately -0.6 means the initial model gets far more answers wrong than right (the reward scale is +1 correct, -1 incorrect, so -0.6 corresponds to roughly 20% accuracy in expectation). By the end of training, the mean reward reaches approximately +0.4 (roughly 70% accuracy in expectation, though this is on the training distribution and not directly comparable to the evaluation benchmarks).

The paper emphasizes that there are "no extended plateaus or sudden drops that would typically trigger intervention in multi-stage approaches." This is presented as a contrast to prior work: ProRL-V2 introduced stage transitions specifically to overcome plateaus, and BroRL increased rollouts to 512 after hitting performance ceilings. JustRL's curve suggests that, with the right hyperparameters, learning continues smoothly without the plateaus that multi-stage training is designed to address. The noise in the reward curve is typical of RL training with limited rollouts per step (8 per question) and is not a pathology — it reflects the stochasticity of sampling from a policy with non-trivial entropy, and the model learns through this noise rather than being destabilized by it.

Mean response length (Figure 2c). The model starts generating responses averaging approximately 7,000–8,000 tokens and "without any explicit length penalty, it naturally compresses to 4,000–5,000 tokens by step 1,000 and maintains this range." This is the paper's most compelling single diagnostic, because it shows emergent behavior that explicit penalties are designed to enforce: the model learns to be more concise because the reward signal doesn't favor unnecessary verbosity. Long responses that don't improve correctness provide no selective advantage; short responses that are correct get reinforced just as strongly as long correct responses. Over thousands of steps, the policy naturally converges to an efficient length.

The paper explicitly contrasts this with explicit length penalties: "This organic compression may be more robust than explicit penalties, which can create adversarial pressure that models learn to game." The ablation study (Section 4.4) provides the evidence for this claim — when an explicit overlong penalty is added, performance degrades and exploration collapses. The natural compression preserves diversity while achieving efficiency; the explicit penalty sacrifices diversity for efficiency, and the cost shows up in the final performance.

How these three signals interrelate. The paper does not make this argument explicitly, but the three-signal triad supports a unified interpretation that strengthens its thesis. The entropy stays high because the "clip higher" mechanism prevents aggressive probability reduction and no length penalty forces premature convergence. High entropy enables sustained exploration of diverse reasoning strategies. That exploration, combined with reward-driven selection, produces the smooth reward improvement. The reward improvement, in turn, creates the natural length compression — the model learns through experience that conciseness doesn't hurt accuracy. The three signals are not independent metrics but manifestations of a single well-balanced optimization dynamic. The ablations confirm this interdependence: break any one signal (entropy, via the overlong penalty) and the other two collapse as well (reward plateaus, length compresses prematurely).

Ablation Studies and Robustness Checks

Both ablation experiments start from the base JustRL recipe on the DeepSeek backbone, are trained for 3,000+ steps, and are evaluated on AIME 2024 avg@32. The results are presented in Figure 3, which contains two subplots: (a) AIME 2024 performance over training steps and (b) policy entropy over training steps.

Overlong penalty ablation: Adding the DAPO-style overlong penalty (which penalizes the last 4,000 tokens of responses exceeding 11,000 tokens in length) causes AIME 2024 performance to plateau at approximately 50% compared to the base recipe's approximately 55% at the same training duration, a degradation of roughly 5 percentage points. The entropy plot (Figure 3b) reveals the mechanism: with the overlong penalty, entropy collapses from the base recipe's 1.2–1.4 oscillation band to a stable but low 0.5–0.6, indicating that the penalty has forced the policy to converge to a narrow distribution of strategies before adequate exploration has occurred. Once entropy is lost, the model cannot discover new reasoning patterns, and performance plateaus. The paper's interpretation: "The explicit penalty appears to create pressure that conflicts with the learning objective, forcing premature convergence to shorter responses before the model has explored what actually works." This finding is particularly striking because the overlong penalty is intended to improve efficiency by accelerating conciseness, but the base recipe already achieves natural compression (Figure 2c) without the exploration cost.

Overlong penalty + robust verifier ablation: Adding both the overlong penalty and the DeepScaleR robust verifier (which reduces false negatives from the rule-based verifier) causes AIME 2024 performance to plateau at approximately 45%, a further 5-point degradation beyond the overlong-penalty-only ablation and a full 10 points below the base recipe. The entropy collapse is even more severe, dropping below 0.5 and staying there (Figure 3b). This dual-ablation result is notable because both modifications are, individually, considered "improvements" in their original contexts — DAPO uses the overlong penalty to control length explosion, and DeepScaleR uses the robust verifier to improve reward signal quality. Yet in JustRL's setting, each independently degrades performance, and their combination is worse than either alone.

The paper offers two hypotheses for why the robust verifier hurts. First: "the stricter base verifier creates a richer spectrum of learning signals by reducing 'perfect' scores, whereas the robust verifier's permissiveness offers less nuanced guidance." The idea is that when the verifier is strict, the model receives a broader distribution of reward outcomes — some correct-looking solutions get marked wrong (false negatives), some correct solutions get marked right, and the gradients from these varied outcomes provide a richer learning signal than a verifier that marks most approximately-correct solutions as correct. Second: "the stricter verifier's reliance on precise formatting may pressure the model to develop more robust internal computations, an incentive lost when the verifier corrects errors externally." The robust verifier effectively "forgives" formatting sloppiness; the model learns it doesn't need to be precise because the verifier will clean up after it. The strict verifier punishes imprecision, forcing the model to develop internal precision that generalizes to novel problems.

What the ablations do not test. The paper explicitly acknowledges that "many other techniques remain unexplored: curriculum learning, adaptive temperature scheduling, reference model resets, different verifier designs, and various forms of data augmentation. Some of these might improve upon our baseline." This is not a kitchen-sink ablation study — it tests two specific, widely-used techniques and finds both harmful. The paper does not claim to have proven all techniques harmful, only that the burden of proof should shift to demonstrating benefit over a stable simple baseline rather than assuming it.

Implicit robustness checks. The paper conducts several implicit robustness checks through its main experimental design rather than through explicit ablation sections. The cross-backbone transfer (DeepSeek → Nemotron with no hyperparameter changes) serves as a robustness check for the hyperparameter configuration. The sustained training over 4,380 steps with no intervention serves as a robustness check for training stability claims. The nine-benchmark evaluation suite serves as a robustness check against overfitting to a single benchmark (the paper leads on 6/9 benchmarks for the DeepSeek backbone and 6/9 for Nemotron, demonstrating broad rather than narrow improvement). The use of the 2025 competition benchmarks (HMMT, CMIMC, BRUMO) serves as a robustness check against data contamination — these were released after training, so performance on them must reflect genuine reasoning capability.

Negative result from ReST^EM? The paper does not appear to include a ReST^EM experiment of the type found in the earlier example paper (where an attempt to optimize the revision model with RL degraded performance). The prior sections of this analysis reference a ReST^EM finding, but a careful re-read of the JustRL paper reveals no such experiment. The paper's ablation section is limited to the two modifications described above. This suggests the earlier mention was an error in the prior analysis sections, not a finding from the paper itself. The paper's only negative results are the two ablation conditions shown in Figure 3.

Critical Assessment

Claim 1: Single-stage training with fixed hyperparameters matches or exceeds complex multi-stage approaches while using 2× less compute.

What the experiments demonstrate. For the DeepSeek backbone, Table 3 shows JustRL at 54.87% vs. ProRL-V2 at 53.08% — a 1.79 percentage point advantage that is directionally supportive but narrow enough that it could fall within the range of training variance (which the paper does not quantify). Table 4 shows JustRL uses approximately 1.4×10^8 tokens vs. ProRL-V2's 2.8×10^8 — a genuine 2× efficiency advantage. For the Nemotron backbone, Table 5 shows JustRL at 64.32% vs. QuestA at 63.81% (a 0.51 point advantage), and Table 6 shows 1.1×10^8 vs. 2.6×10^8 tokens (a 2.4× advantage).

What the experiments do NOT demonstrate. The comparison is between JustRL's single-run performance and prior work's reported single-run performance. Without error bars or multiple seeds for either side, we cannot determine whether JustRL is genuinely superior or whether both methods would overlap if run multiple times. The ProRL-V2 comparison is complicated by ProRL-V2 being part of a training lineage (ProRL-V1 → ProRL-V2) whose total cost exceeds the 2.8×10^8 reported for ProRL-V2 alone. The QuestA comparison involves fundamentally different data requirements — QuestA augments questions with chain-of-thought solutions from larger models — so the comparison is between a method with extra data and a method without, not just between training algorithms. The 2× compute claim for the DeepSeek comparison is well-supported by the numbers in Table 4; the 2.4× claim for Nemotron inherits uncertainty from the estimated 50% dynamic sampling filter ratio, and QuestA may filter more or less than this estimate in practice.

Verdict: The claim that simple training is competitive with complex training is supported. The specific claim of "matching or exceeding" is supported directionally but the margins are small enough that statistical uncertainty matters. The 2× compute efficiency advantage is solid for the DeepSeek comparison and robust to the specific filter ratio assumption (Table 4 notes conservatism).

Claim 2: Training is stable — smooth, monotonic improvement over 4,000+ steps without collapses, plateaus, or the instabilities that motivate complexity.

What the experiments demonstrate. Figure 1 shows smooth improvement curves for both backbones. Figure 2 shows entropy stability (1.0–1.6 oscillation range, no systematic drift), monotonic reward improvement (no extended plateaus or drops), and natural length compression (no explosion). These are presented as evidence that the training does not require intervention.

What the experiments do NOT demonstrate. The paper compares its training dynamics to prior work's reported instabilities but does not reproduce those prior baselines to show that they are less stable under identical conditions. The claim that "our training exhibits none of these pathologies" is accurate for the reported run, but we don't know how representative this single run is — a second seed with different random initialization might show different dynamics. More importantly, we don't know whether prior work's instabilities would appear if their complex baselines were simplified but not otherwise optimized — it is possible that the stability achieved by JustRL is due to specific hyperparameter choices (the 1e-6 learning rate, the asymmetric clip, the 8-rollout group size) that were carefully selected, not due to the absence of complex techniques per se. The paper acknowledges this uncertainty directly: "We can't isolate which specific complex techniques cause instability versus which ones solve it."

Verdict: The training dynamics evidence is qualitatively compelling and serves the paper's methodological purpose — demonstrating that stability without intervention is possible. But the causal attribution (simplicity → stability) cannot be established from these experiments alone. A controlled comparison where complex techniques are ablated one at a time from an unstable baseline would be needed to establish causality.

Claim 3: Adding "standard tricks" (overlong penalty, robust verifier) degrades performance by collapsing exploration.

What the experiments demonstrate. Figure 3 shows that both modifications to the base recipe cause AIME 2024 performance to plateau lower (50% with overlong penalty, 45% with both) compared to the base recipe (55%), and that both cause entropy to collapse to the 0.5–0.6 range compared to the base recipe's 1.2–1.4. The mechanism (exploration collapse → performance plateau) is clearly visible in the coupled dynamics.

What the experiments do NOT demonstrate. The paper tested two specific modifications applied to JustRL's specific recipe. It did not test whether these same modifications would degrade performance in the complex baselines they were originally designed for — it is possible that an overlong penalty is harmful in a single-stage fixed-hyperparameter context but beneficial when combined with multi-stage training and dynamic temperature (as in DAPO). Nor did the paper test whether a different overlong penalty design (different threshold, different penalty magnitude, different scheduling) might provide benefits without collapsing exploration. The claim is therefore "these two standard tricks, as implemented, degraded our specific baseline" — which is true and important — but not "overlong penalties are generally harmful" or "robust verifiers are generally harmful."

The robust verifier result is also open to an alternative interpretation. The DeepScaleR verifier may differ from the DAPO verifier in ways beyond robustness — it might use different parsing logic, different matching heuristics, or different reward normalization that would change the effective reward scale. The paper notes that "even after normalizing reward scales, its use leads to worse final performance," suggesting they attempted to control for scale differences, but the details of this normalization are not provided. The claim that a "richer spectrum of learning signals" from the stricter verifier is beneficial is a post-hoc hypothesis, not a tested mechanism.

Verdict: The ablation findings are the paper's strongest empirical contribution — they provide concrete, mechanistic evidence (entropy collapse) that widely-adopted techniques can be actively harmful when applied to a simple baseline. However, they demonstrate harm in a specific context and cannot rule out that these techniques provide net benefit in other contexts or with different implementations. The paper's methodological conclusion — "they should be validated empirically rather than assumed to be beneficial" — is fully supported.

Claim 4: The same recipe transfers across both backbones without modification.

What the experiments demonstrate. Table 2 describes identical hyperparameters for both models. Tables 3 and 5 show competitive results on both backbones. Figures 1(a) and 1(b) show similar smooth training dynamics. The recipe genuinely transferred without per-model tuning.

What the experiments do NOT demonstrate. The paper does not explore the sensitivity of this transfer — would it work on a third backbone? What about a non-math domain? What about a different model scale (e.g., 7B parameters)? The claim is narrow ("transfers across these two backbones") and well-supported, but the paper's framing in Section 4.2 emphasizes "genuine robustness rather than lucky optimization" — the evidence for "genuine robustness" is limited to n=2 successful transfers. Additionally, both backbones are 1.5B models fine-tuned for mathematical reasoning; they share substantial architectural and distributional similarity. Transfer to a code-generation model or a general chatbot might not succeed.

Verdict: The narrow claim of transfer between these two specific backbones is supported. The broader implication of "genuine robustness" is suggested but not rigorously tested.

Genuine Weaknesses in the Experimental Design

Single-run results without statistical quantification. Every main result and every ablation is based on a single training run. RL training is stochastic — different random seeds, different batch orderings, different initialization of the optimizer state can produce meaningfully different outcomes. The paper's key comparisons — 54.87% vs. 53.08% for DeepSeek, 64.32% vs. 63.81% for Nemotron — are within ranges where seed variance could be the determining factor. Without confidence intervals or multiple-seed averages, the reader cannot assess whether JustRL is reliably better than baselines or merely competitive within noise. This is particularly important for the paper's central thesis: if JustRL were 2% worse than ProRL-V2 on average across seeds but more stable, that would still support the paper's methodological argument (simplicity is competitive and more reliable), but the paper's presentation — "outperforming more complex approaches" — emphasizes the directional advantage, which may not be robust.

No reproduction of prior baselines. The paper compares against numbers reported in prior work but does not reproduce any prior method in its own experimental setup. This is standard practice in the field (reproducing every baseline would be prohibitively expensive) but limits the strength of the comparison. Differences in evaluation protocol, hardware, random seeds, or even minor implementation details could account for some of the performance gap. Ideally, the paper would have reproduced at least one complex baseline (e.g., ProRL-V2) on its own infrastructure with its own evaluation protocol to provide a controlled comparison.

The difficulty estimation problem is sidestepped. A central practical challenge for RL-for-reasoning — which the earlier example paper on test-time compute scaling identified as critical — is estimating problem difficulty to allocate resources optimally. JustRL avoids this entirely by using a uniform dataset and uniform training protocol. While this simplicity is the paper's point, it also means the paper doesn't engage with a genuine challenge that some complex techniques (curriculum learning, dynamic filtering) are designed to address. If JustRL's performance were dramatically worse than QuestA's on the hardest benchmarks, that would suggest complexity is necessary for hard problems — but the results show JustRL is competitive across the board, suggesting the uniform approach is sufficient at this scale and on this data distribution. This is a valuable finding in itself.

Limited exploration of failure modes at longer horizons. The paper trains for 4,380 steps (DeepSeek) and 3,440 steps (Nemotron). These are substantial training durations, but we don't know what happens if training continues — does the stability persist to 10,000 steps? Does performance plateau eventually? Would the natural length compression continue, or would length eventually drift? The paper's ablation experiments are stopped at 3,000+ steps; it's possible that the overlong penalty or robust verifier would show benefits at longer horizons that aren't visible in the reported window. The paper acknowledges this limitation: "We have not explored whether our approach maintains advantages when pushed to even longer training horizons."

The evaluation verifier mismatch. Training uses the strict DAPO verifier; evaluation uses the forgiving CompassVerifier-3B. The ablation shows that using a more forgiving verifier during training degrades performance. This creates an apparent tension: the model is trained with a strict verifier that produces false negatives, but evaluated with a forgiving verifier that corrects those false negatives. The paper's claim is that the strict training verifier forces the model to develop robust internal computation that generalizes. But an alternative interpretation is that the strict verifier simply selects for models that produce outputs formatted in ways the strict verifier can parse, and the forgiving evaluation verifier masks the fact that some of the apparent improvement is formatting adaptation rather than genuine reasoning improvement. The paper does not report evaluation results with the strict verifier during evaluation, which would help disentangle formatting improvement from reasoning improvement. This is a significant gap: if JustRL's advantage over prior work is partially due to better formatting compliance (which the strict verifier would penalize during training but the forgiving verifier ignores at evaluation time), that would weaken the paper's claim about genuine reasoning improvement.

Missing ablations that would strengthen the paper. Several experiments would significantly strengthen the paper's arguments:

(1) Ablating "clip higher": The paper identifies the asymmetric clip as the one technique it retains, but never ablates it. What happens with a symmetric clip [0.8, 1.2]? Does entropy collapse as it does with the overlong penalty? This would directly test whether "clip higher" is necessary or merely harmless.

(2) Ablating the learning rate: The constant 1e-6 learning rate is unusually low. What happens at 5e-6 or 1e-5? Does stability degrade? Does performance improve or decline? Understanding the LR sensitivity would help distinguish whether the recipe's stability comes from simplicity per se or from a carefully-chosen low LR that happens to work.

(3) Multiple random seeds: Even two additional seeds for the main DeepSeek run would allow reporting mean and standard deviation, transforming the comparison from a single data point to a statistical estimate.

(4) Evaluation with the strict verifier: Reporting evaluation results with the same DAPO verifier used in training would show whether the improvement is in reasoning or in formatting compliance.

(5) Ablating rollout N: The paper uses N=8 rollouts. Prior work uses N=16 or N=32 in some stages. What happens with N=16 in the single-stage setting? Does performance improve, plateau, or degrade? This would test whether the 8-rollout choice is optimal or merely adequate.

Summary of the experimental contribution. The paper's experiments are well-designed for its primary purpose: demonstrating that a simple, stable baseline exists and that adding two popular "stability" techniques to it actively harms performance. The nine-benchmark evaluation suite, detailed training dynamics tracking, and cross-backbone transfer provide substantially more experimental rigor than a typical systems paper. But the single-run nature of all results, the lack of controlled comparisons with reproduced baselines, the evaluation verifier mismatch, and the absence of key ablations (clip higher, learning rate, rollout count) mean the paper's specific performance comparisons should be treated as illustrative rather than definitive. The paper's greatest experimental strength — and its most important contribution to the literature — is not the specific numbers in Tables 3 and 5 but the existence proof in Figures 1 and 2: stable, long-horizon RL training for small math reasoning models does not require complexity. The burden of proof should now be on complex approaches to demonstrate that their additional techniques improve over a simple baseline, not just over their own prior complex baselines.

6. Limitations and Trade-offs

The Difficulty Estimation and Dynamic Allocation Problem Is Entirely Bypassed

The assumption or constraint. JustRL achieves its simplicity in part by avoiding a challenge that the broader RL-for-reasoning literature has identified as critical: how to allocate training and inference compute adaptively based on problem difficulty. The paper trains on all 17,000 DAPO-Math-17k questions uniformly — "without offline difficulty filtering or online dynamic sampling strategies" (Section 3.1) — and evaluates with a fixed sampling budget per benchmark category (32 samples for competition benchmarks, 4 for others). The paper explicitly acknowledges that some prior techniques it avoids, such as dynamic sampling and curriculum learning, are designed to address real challenges that JustRL simply does not engage with. QuestA, for instance, "introduces an innovative curriculum learning approach that augments questions with partial CoT solutions as hints, splitting training stages with different difficulty" (Section 4.2). The paper does not argue these techniques are useless — only that they are not necessary to achieve competitive results at this scale.

The consequence. The absence of difficulty-aware allocation means JustRL provides no mechanism for distinguishing problems where additional compute would help from those where it would be wasted. A practitioner deploying JustRL inherits a model that performs well on average but has no principled way to decide per-question sampling budgets at inference time. On the hardest competition questions — analogous to difficulty bin 5 in the earlier example paper's MATH benchmark analysis — JustRL almost certainly achieves near-zero improvement regardless of how many samples are drawn, because the base model simply cannot produce correct solutions. On easy questions, a single sample may suffice. JustRL's uniform evaluation protocol (32 samples for AIME, 4 for MATH-500) ignores this heterogeneity, meaning resources are systematically over-allocated to easy problems and under-allocated to problems at the edge of the model's capability where additional samples might push a near-miss into a correct answer. This is not merely a theoretical concern: the earlier example paper on test-time compute scaling demonstrated that difficulty-conditioned allocation can yield 4× efficiency improvements over uniform best-of-N. JustRL's silence on this dimension means its efficiency gains (2× over ProRL-V2) may substantially understate what is achievable with difficulty-aware allocation layered on top of a stable training recipe.

What evidence exists in the paper. The paper provides no per-difficulty breakdown of its evaluation results. Tables 3 and 5 report only average scores across all nine benchmarks and per-benchmark scores, but within each benchmark (e.g., AIME 2024) there is no breakdown of performance by problem difficulty. We therefore cannot assess whether JustRL's gains over the backbone are concentrated in easy/medium problems (as the earlier example paper found for test-time compute scaling) or are distributed across the difficulty spectrum. The training dynamics (Figure 2) report aggregate metrics — mean reward, mean entropy, mean response length — that similarly obscure difficulty-dependent variation. The paper also does not report the variance of pass@1 across problems within each benchmark, which would reveal whether performance is driven by a few easy problems or represents genuine broad improvement.

Mitigation status. The paper does not attempt to address this limitation and does not propose difficulty estimation as future work. It is mentioned indirectly in the limitations section — "our results are limited to mathematical reasoning tasks at the 1.5B parameter scale, and generalization to other domains... remains unexplored" — but the absence of difficulty-conditioned allocation is not acknowledged as a limitation in its own right. The paper's contribution is establishing that stable training without complexity is possible; incorporating difficulty awareness is a natural next step that would combine JustRL's stable optimization with the efficiency gains demonstrated in the test-time compute scaling literature.

Single-Run Results Without Statistical Quantification Undermine the Performance Comparisons

The assumption or constraint. Every main result in Tables 3 and 5 and every ablation in Figure 3 is based on a single training run. The paper reports point estimates — 54.87% vs. 53.08% for JustRL-DeepSeek vs. ProRL-V2, 64.32% vs. 63.81% for JustRL-Nemotron vs. QuestA — without confidence intervals, standard deviations, or any quantification of run-to-run variance. The paper acknowledges in Section 5 that "we don't have the computational resources to run extensive controlled comparisons" and "we can't isolate which specific complex techniques cause instability versus which ones solve it." Training 1.5B-parameter models for 3,000–4,400 steps on 32 A800 GPUs for ~15 days per run is expensive enough that multiple seeds per configuration would substantially increase the paper's compute budget. But this practical constraint has direct consequences for the strength of the paper's claims.

The consequence. The headline performance comparisons rest on margins that are small enough that training variance could plausibly be the determining factor. The JustRL-DeepSeek advantage over ProRL-V2 is 1.79 percentage points on the nine-benchmark average. The JustRL-Nemotron advantage over QuestA is 0.51 percentage points. Without knowing the standard deviation of these averages across training runs, a reader cannot assess whether JustRL is reliably superior, roughly equivalent, or potentially inferior to these baselines in expectation. This matters because the paper's rhetorical framing emphasizes outperformance: "outperforming more complex approaches while using 2× less compute" (Section 4.1), "achieving 54.87% average across benchmarks, outperforming ProRL-V2's 53.08%" (Section 4.1), "slightly outperforming QuestA's 63.81%" (Section 4.2). If the true expected performance of JustRL and these baselines were equal within statistical noise, the paper's central empirical claim — that simplicity matches or exceeds complexity — would still be supported directionally (simplicity is competitive), but the specific numerical superiority claims would be misleading.

The ablation results (Figure 3) are less vulnerable to this critique because the performance gaps are much larger: ~5 percentage points for the overlong penalty ablation and ~10 points for the combined ablation on AIME 2024, with correspondingly dramatic entropy differences (1.2–1.4 vs. 0.5–0.6). These gaps are large enough that run-to-run variance is unlikely to reverse the qualitative finding. However, the ablation experiments are also single runs, and the specific plateau values (50%, 45%) should be interpreted as illustrative rather than precise estimates.

What evidence exists in the paper. The paper does provide indirect evidence of stability through the training dynamics curves (Figures 1 and 2). The smooth, monotonic improvement without collapses or oscillations suggests that the training process is well-behaved and not prone to the catastrophic failures that would produce bimodal outcomes (one seed succeeds, another collapses). This reduces — but does not eliminate — concern about run-to-run variance. If training is genuinely stable, different seeds should produce similar trajectories and similar final performance. But stable training does not guarantee identical final performance; small differences in early exploration can compound into meaningful differences in final capability, especially on the hardest benchmarks where performance is measured in single-digit percentages.

The paper also implicitly provides a form of replication through its cross-backbone transfer: the same recipe works on two different starting models, which is a stronger signal of robustness than a second seed on the same model would be. If the DeepSeek result were a lucky seed, we would not expect the Nemotron result to also be competitive — the fact that both work suggests the recipe is genuinely effective, not overfit to a single lucky training trajectory.

Mitigation status. The paper acknowledges this limitation in its limitations section: "our compute budget, while lower than some complex methods, may still be prohibitive for resource-constrained researchers." But this acknowledgment is about the absolute cost, not about the statistical reliability of the comparisons. The paper does not discuss the implications of single-run results for its performance claims, does not provide standard deviations, and does not propose seed-based replication as future work. A pragmatic mitigation would be to frame the comparisons more cautiously — "competitive with" rather than "outperforming" — given the margins involved. The paper's current framing emphasizes directional superiority, which the evidence cannot fully support without statistical quantification.

The Evaluation-Verifier Mismatch Confounds Formatting Improvement with Reasoning Improvement

The assumption or constraint. JustRL trains exclusively with the DAPO rule-based verifier — a strict, string-matching-based scorer that produces binary rewards and is explicitly noted to generate false negatives: correct solutions that are marked wrong because the model's boxed answer does not exactly match the ground-truth format expected by the verifier's parser (Section 3.1, Section 4.4). At evaluation time, the paper switches to CompassVerifier-3B, "a lightweight model-based verifier, to address false negatives from rule-based verifiers" (Section 3.2). The paper argues in its ablation (Section 4.4) that the strict training verifier is actually beneficial — it "creates a richer spectrum of learning signals" and "pressures the model to develop more robust internal computations" — but never evaluates the trained model using the same strict verifier used during training. All reported benchmark scores in Tables 3 and 5 are computed with the forgiving model-based verifier.

The consequence. The performance improvement from backbone to JustRL (e.g., 37.65% → 54.87% on the DeepSeek backbone, a 17.22 percentage point gain) conflates two distinct forms of improvement: (1) genuine improvement in the model's ability to solve math problems correctly, and (2) improvement in the model's ability to format its answers in a way that the evaluation verifier can parse. The relative contribution of these two factors is unknown. It is entirely possible that a substantial fraction of the reported gain comes from the model learning to produce well-formatted answers that the CompassVerifier-3B reliably recognizes as correct, rather than from genuinely improved mathematical reasoning. The strict training verifier would have punished formatting errors during training, creating strong pressure toward formatting compliance; the evaluation verifier then rewards that compliance. A model that learned to format beautifully but reason only slightly better would appear to improve dramatically under this evaluation protocol, even though its genuine reasoning improvement is modest.

This concern is heightened by the ablation finding that adding a robust verifier during training degrades performance (Figure 3). The paper's hypothesis is that the strict verifier forces the model to develop "more robust internal computations" — but an equally plausible hypothesis is that the strict verifier forces the model to develop outputs that happen to parse well under the evaluation verifier, and the robust verifier during training removes this formatting pressure, causing the model to produce correct-but-unparseable answers that the evaluation verifier also struggles with. The paper cannot distinguish these hypotheses because it never reports evaluation results with the strict verifier.

The magnitude of this confound is unknown but potentially significant. Math competition problems often have answers that can be expressed in multiple equivalent formats (e.g., "1/2" vs. "0.5" vs. "\frac{1}{2}"), and a model that learns to match the training data's conventional format will score higher under any verifier, strict or forgiving, than a model that produces mathematically equivalent but idiosyncratically formatted answers. The question is whether the improvement is primarily formatting or primarily reasoning — and the paper provides no evidence either way.

What evidence exists in the paper. The only evidence touching on this question is the ablation experiment (Section 4.4, Figure 3). When a robust verifier (from DeepScaleR) is used during training, performance on AIME 2024 drops by approximately 10 percentage points compared to the base recipe with the strict verifier. The paper interprets this as evidence that the strict verifier provides better learning dynamics. But this interpretation assumes the evaluation verifier measures genuine reasoning ability — if the evaluation verifier itself has biases (e.g., it is more accurate on well-formatted answers), then the ablation result could reflect the robust training verifier producing a model that is worse at formatting rather than worse at reasoning. The paper does not report evaluation results with the strict training verifier for any model, so we cannot assess how much of the improvement is formatting vs. reasoning.

The cross-benchmark pattern of results offers circumstantial evidence. JustRL-DeepSeek shows large gains on competition benchmarks like AIME24 (+22.7 points over the backbone), AIME25 (+16.35 points), and HMMT (+8.54 points), and smaller gains on benchmarks like MATH-500 (+6.75 points) and AMC23 (+27.2 points — but the backbone was already at 63.82%, so the ceiling is higher). This pattern is broadly consistent with genuine reasoning improvement (the hardest benchmarks show the largest gains because the backbone had the most room to improve), but it is also consistent with formatting improvement (competition benchmarks may have more complex answer formats that the backbone mishandled). Without strict-verifier evaluation scores, the confound cannot be resolved.

Mitigation status. The paper does not acknowledge this as a limitation. The evaluation protocol is described as standard practice — "We augment existing systems with CompassVerifier-3B... to address false negatives" — and the paper treats the evaluation scores as measuring reasoning ability without qualification. The paper could partially mitigate this concern by reporting evaluation results with the DAPO strict verifier alongside the CompassVerifier-3B results, or by reporting the false negative rate of the strict verifier on the backbone vs. the JustRL model to quantify how much of the improvement is formatting-related. Neither is done. Future work should establish whether the gains represent genuine reasoning improvement or increased compliance with the evaluation verifier's expectations — ideally through human evaluation of solution correctness on a sample of problems where the two verifiers disagree.

Generalization Is Limited to Mathematical Reasoning at the 1.5B Scale on Two Specific Backbones

The assumption or constraint. Every experiment in the paper — training, evaluation, and ablation — is conducted on mathematical reasoning tasks using two 1.5B-parameter models (DeepSeek-R1-Distill-Qwen-1.5B and OpenMath-Nemotron-1.5B) trained on DAPO-Math-17k and evaluated on nine math competition benchmarks. The paper acknowledges this limitation explicitly: "our results are limited to mathematical reasoning tasks at the 1.5B parameter scale, and generalization to other domains (e.g., coding, general question answering) and model sizes remains unexplored" (Section 6). This is a candid and appropriate acknowledgment, but its consequences merit deeper examination than a one-sentence caveat.

The consequence. The paper's central thesis — that complexity in RL for language models is often unnecessary — may not generalize beyond the specific regime it studies. Mathematical reasoning has properties that make it unusually amenable to simple RL with binary outcome rewards: (1) answers are verifiable with high accuracy through deterministic rules (string matching or model-based comparison), providing a clean reward signal without the ambiguity of human preference judgments, (2) the task is well-defined — there is a correct answer and the model either finds it or doesn't — avoiding the reward hacking and specification gaming problems that plague RLHF for open-ended generation, (3) the action space is constrained — math solutions follow recognizable patterns (step-by-step reasoning, final boxed answer) that the model, having been distilled from larger math-capable models, already knows how to produce, and (4) the base models start from a high baseline of capability (37–57% average accuracy), meaning the RL is refining an already-competent policy rather than teaching a new skill from scratch. Each of these properties reduces the need for the stabilization techniques that JustRL eschews.

In domains without these properties, the simple recipe may fail in ways the paper cannot predict. Coding tasks involve more diverse and less predictable solution structures, and unit-test-based verification can produce noisier reward signals when tests are incomplete or when multiple valid solutions exist. General question answering and dialogue lack ground-truth verifiability entirely, requiring learned reward models that introduce their own biases and over-optimization risks — the same verifier over-optimization problem that the earlier example paper identified as the primary bottleneck for test-time compute scaling. At larger model scales (7B, 13B, 70B parameters), the optimization dynamics may differ: larger models may have different entropy characteristics, different sensitivity to learning rates, and different tendencies toward reward hacking. The paper's 1e-6 learning rate, which contributes to stability at 1.5B by keeping updates small, might be unnecessarily conservative at larger scales or might need adjustment.

The paper's cross-backbone transfer provides some evidence against the narrowest interpretation of this limitation — the recipe works on two different 1.5B math models, which is more than a single data point. But two backbones in the same domain and same scale do not constitute evidence of domain or scale generality. The paper's framing in Section 4.2 — "The fact that the same recipe works for both models without hyperparameter tuning suggests genuine robustness rather than lucky optimization for a single model" — overstates what can be concluded from n=2 in-domain transfers. "Genuine robustness" would imply transfer to substantially different settings; the evidence supports only "this recipe is not brittle to the specific choice of 1.5B math backbone," which is a weaker but still useful claim.

What evidence exists in the paper. The paper provides no experiments outside mathematical reasoning, no experiments with models of different scales, and no experiments with different RL algorithms or reward structures. The nine-benchmark math evaluation suite provides diversity within the math domain — ranging from standard test sets (MATH-500) to recent competition problems (HMMT, CMIMC, BRUMO 2025) to Olympiad-level problems (OlympiadBench) — but all share the fundamental structure of mathematical problem-solving with verifiable answers. The 2025 benchmarks provide evidence against data contamination (these problems were released after training, so performance must reflect generalization rather than memorization), but they do not test generalization to non-math domains.

Mitigation status. The paper acknowledges this limitation clearly in its limitations section and does not claim generality beyond math at 1.5B scale. The mitigation is therefore textual honesty rather than experimental evidence. The paper's title emphasizes "Scaling a 1.5B LLM," and its abstract specifies "mathematical benchmarks," appropriately scoping the claims. The limitation section suggests future work on generalization: "Generalization to other domains, model sizes, and tasks remains an open question" (Section 5). A practitioner considering adopting JustRL for non-math domains or different model scales should treat the paper as an existence proof that simple RL can work in this specific regime, not as evidence that it will work in theirs. The transferability of the specific hyperparameters (1e-6 LR, asymmetric clip, 8 rollouts, 16K context) to other settings is entirely untested.

The Recipe's Stability May Depend on Specific Untested Hyperparameter Choices, Not the Principle of Simplicity

The assumption or constraint. JustRL presents itself as a demonstration that simplicity works — "single-stage training with fixed hyperparameters" (Section 3.1) is sufficient. But the paper's specific hyperparameter configuration (Table 2) includes several choices that are non-obvious and potentially load-bearing: the unusually low learning rate of 1e-6, the asymmetric clip range [0.8, 1.28], the specific rollout count of 8 per question, the binary +1/-1 reward scale (rather than +1/0), the 16K context window, and the DAPO verifier's specific matching heuristics. None of these are ablated. The paper cannot distinguish between two competing explanations for its success: (1) the absence of complex techniques (multi-stage training, dynamic schedules, length penalties) is what enables stability, or (2) the presence of specific, carefully-chosen hyperparameters (particularly the low learning rate and asymmetric clip) is what enables stability, and the absence of complex techniques is merely non-harmful in this specific configuration.

The consequence. A practitioner who adopts JustRL's philosophy — "keep it simple" — but chooses different hyperparameters (a more standard learning rate of 5e-6, a symmetric clip, a different reward scale, a different verifier) may experience the very instabilities that the paper's simple recipe avoids. The paper's message could therefore be misleading: it is not simplicity per se that enables stability, but rather a specific set of hyperparameter choices that happen to produce stable dynamics. If this alternative explanation is correct, then the field's turn toward complexity may be partly a response to the fact that stability is sensitive to hyperparameters, and complex techniques (KL penalties, dynamic schedules) are ways of expanding the region of hyperparameter space where training is stable — making RL "easier" for practitioners who cannot or do not find the narrow stable region. In that case, simplicity and complexity are not competitors; they are different solutions to the same underlying challenge of sensitivity, with different tradeoffs (simplicity requires finding the right hyperparameters; complexity provides robustness to hyperparameter variation).

This concern is particularly acute for the learning rate. The paper uses 1e-6, which is at the low end of typical RL-for-LLM learning rates. The standard PPO learning rate in the literature is often 1e-5 or higher (ProRL and others likely use rates in this range). A learning rate an order of magnitude lower means each policy update is an order of magnitude smaller, which directly reduces the risk of catastrophic updates that collapse entropy. If JustRL's stability is primarily attributable to the low learning rate — and the same stability could be achieved with multi-stage training and a higher learning rate — then the paper's methodological critique of complexity is weaker than it appears. The field may have adopted complexity to enable faster training (higher learning rates, fewer steps) rather than because stability is impossible without complexity.

The asymmetric clip range [0.8, 1.28] is similarly suspect as a load-bearing choice. The paper identifies "clip higher" as "the one technique we do use" (Section 3.1) and argues it is fundamental to maintaining exploration. But the paper never ablates it — we don't know what happens with a symmetric clip [0.8, 1.2]. If the symmetric clip causes entropy collapse similar to the overlong penalty ablation (Figure 3b), then "clip higher" is not a minor detail but the central mechanism enabling stability, and the paper's framing — "a minimal approach using single-stage training with fixed hyperparameters" — understates the importance of this one specific design choice. A recipe with symmetric clipping might fail even with all other hyperparameters identical.

What evidence exists in the paper. The paper provides no ablation of the learning rate, the clip range, the rollout count, or the reward scale. The only ablations are of two techniques added to the base recipe (overlong penalty, robust verifier), not of components within the base recipe. The training dynamics (Figure 2) demonstrate that the specific hyperparameter configuration produces stable training, but they cannot distinguish whether this stability is robust to hyperparameter variation or narrowly dependent on the specific values in Table 2. The cross-backbone transfer provides weak evidence against narrow dependence: if the hyperparameters were finely tuned to the DeepSeek backbone's specific optimization landscape, they might not work for Nemotron. But both backbones are 1.5B math models with similar architectures and similar training regimes; they likely have similar optimization characteristics, so transfer between them does not demonstrate hyperparameter robustness in general.

Mitigation status. The paper does not acknowledge this as a limitation or discuss the sensitivity of its results to specific hyperparameter choices. The limitations section focuses on domain and scale generalization, not hyperparameter sensitivity. A minimal mitigation — which the paper does not provide — would be to ablate at least the learning rate (e.g., test 5e-6 and 1e-5) and the clip range (e.g., test symmetric [0.8, 1.2]) to establish whether the stability is robust to these choices or narrowly dependent on them. Without such ablations, the paper's philosophical conclusion — "start simple, scale up, and only add complexity when a simple, robust baseline demonstrably fails" — is good advice, but the paper's specific recipe may not be the "simple, robust baseline" that practitioners can adopt without careful hyperparameter tuning of their own. The recipe may be simple in structure (single stage, fixed hyperparameters) but not simple to discover — the specific values in Table 2 may represent the successful endpoint of substantial trial-and-error that the paper does not report.

7. Implications and Future Directions

How This Work Changes the Landscape

JustRL does not introduce a new algorithm, architecture, or training objective. Its contribution is methodological rather than technical, and its impact should be understood as reframing the burden of proof in the RL-for-small-models literature rather than as a paradigm shift in how RL training works. This reframing has practical consequences that extend well beyond the paper's specific results.

The central reframing: complexity must now be justified against a simple, well-documented baseline. Before JustRL, the research landscape operated under an implicit assumption: RL for small language models is fundamentally unstable, and sophistication (multi-stage pipelines, dynamic schedules, length penalties, curriculum learning) is necessary to prevent training collapse. This assumption was so deeply embedded that papers routinely cited instabilities observed in their own pipelines as motivation — without establishing whether those instabilities existed in a minimal, well-tuned baseline. JustRL breaks this cycle by providing an existence proof that stable, competitive training is achievable with single-stage GRPO and fixed hyperparameters. The consequence is a shift in the default expectation: future work proposing a new technique for small-model RL must now demonstrate that it improves over a simple baseline like JustRL, not merely that it fixes an instability in the authors' particular complex pipeline. Techniques that only compensate for problems introduced by other design choices will be exposed as unnecessary when tested against a clean reference point.

This reframing echoes broader methodological movements in machine learning — the "do we really need complex X?" genre that has periodically reset entire subfields when someone demonstrates that a scaled-up simple baseline matches or exceeds elaborate architectures. In NLP, this happened with attention-free models and with the discovery that larger pretrained LMs could match task-specific architectures. In computer vision, ConvNeXt demonstrated that carefully modernized ResNets could match Vision Transformers. JustRL plays this role for the RL-for-reasoning subfield, but with a twist: its argument is not just that simplicity matches complexity, but that complexity may be actively harmful because techniques designed to solve specific symptoms (length explosion, entropy drift) can disrupt the broader learning dynamics in ways that are invisible without comprehensive diagnostics.

Resolving contradictory findings in the literature. The paper's training dynamics analysis (Figure 2) and ablation results (Figure 3) provide a unified explanation for a pattern of apparently contradictory findings. Prior work consistently reports training instabilities — reward collapse, entropy drift, length explosion — and proposes techniques to address them. JustRL demonstrates that these instabilities simply do not occur in a properly configured simple baseline. This does not mean prior researchers were wrong about what they observed; it means they were observing the consequences of their own design choices. Length explosion, for instance, may be driven by multi-stage context lengthening (8K → 16K → 24K) that conflates the model's sense of what response length is expected. Entropy drift may be driven by KL penalty resets that create sawtooth dynamics. Reward plateaus may be driven by stage transitions that disrupt continuous learning. The techniques added to solve these problems — length penalties, reference model resets, dynamic schedules — may then create new instabilities that require further techniques, producing the escalating complexity visible in Table 1. JustRL's clean dynamics suggest the entire cycle may be self-inflicted, and the paper's ablation of "standard tricks" (overlong penalty, robust verifier) provides direct evidence that techniques assumed beneficial can be actively harmful.

Research directions that become more attractive. JustRL makes hyperparameter sensitivity analysis a high-priority research question. The paper's specific configuration (1e-6 LR, asymmetric clip [0.8, 1.28], N=8 rollouts, +1/-1 rewards, 16K context) works — but we don't know which of these choices are load-bearing and which are incidental. Ablating the learning rate, the clip asymmetry, the rollout count, and the reward scale against the simple baseline would reveal the stable region of hyperparameter space and determine whether JustRL's stability reflects a narrow lucky configuration or a broad basin of attraction. This is tractable: a systematic sweep of learning rates (5e-6, 1e-5) and clip ranges (symmetric [0.8, 1.2], different asymmetries) at the same scale, each requiring ~15 days on 32 A800 GPUs, would map the stability landscape at a total cost comparable to the original paper's compute budget.

JustRL also makes mechanistic studies of entropy dynamics in language model RL newly important. The paper shows that entropy staying in the 1.0–1.6 range correlates with sustained learning, while entropy below ~0.6 correlates with plateaued performance. But why does the asymmetric clip preserve entropy? Is it because the high-side slack allows the policy to increase probabilities of successful actions without proportionally decreasing probabilities of alternatives? Or because the asymmetry changes the effective gradient signal? Understanding the mechanism would allow principled tuning of the clip range rather than empirical search, and might reveal whether other entropy-preserving mechanisms (explicit entropy bonuses, different advantage estimators) could achieve the same effect more robustly.

Research directions that become less attractive. JustRL's results cast doubt on the research program of developing ever-more-elaborate training pipelines for small-model math reasoning. If a single-stage recipe with fixed hyperparameters matches the performance of an eight-stage approach (ProRL-V2) at half the compute, the marginal value of adding stages, dynamic schedules, and specialized stabilization techniques is at best zero and at worst negative. Research effort currently invested in designing multi-stage curricula with increasing context lengths, cosine length penalty schedules, and progressive unfreezing might be better redirected toward understanding why simplicity works — and whether the apparent benefits of these techniques in prior work reflect genuine improvements or compensation for problems those same techniques introduced.

Similarly, the paper's robust-verifier ablation (Figure 3) should give pause to the research direction of developing more permissive, more sophisticated verifiers for training. The intuition that reducing false negatives improves learning is intuitive but empirically wrong in JustRL's setting. The hypothesis that strict verifiers provide beneficial "precision pressure" suggests that verifier design should optimize for the learning dynamics they induce, not just for accuracy in isolation. Research on verifier design for training (as opposed to evaluation) should now account for how verifier strictness affects exploration, entropy, and the spectrum of reward signals — and should validate that "better" verifiers actually improve final model performance, not just intermediate reward accuracy.

The scale at which complexity becomes necessary remains an open question. JustRL demonstrates simplicity works at 1.5B parameters on math reasoning. The paper explicitly does not claim this generalizes to other domains, scales, or tasks. But the result establishes a methodological template: for any new domain or scale, the first experiment should be to determine whether a JustRL-style simple baseline trains stably and achieves competitive performance. Only when that baseline demonstrably fails — entropy collapses despite hyperparameter tuning, or performance plateaus below acceptable thresholds — should complexity be introduced, and each added technique should be ablated to confirm it improves over the simple baseline rather than compensating for problems introduced by other additions. This inverts the current practice, where complexity is the starting point and ablation is rare.

Follow-Up Research This Work Enables

Systematic hyperparameter sensitivity mapping of the JustRL recipe. The paper's central empirical finding — that a specific configuration of hyperparameters produces stable, competitive training — raises an immediate question: how sensitive is this stability to each hyperparameter choice? A follow-up study would train JustRL-DeepSeek variants that individually vary the learning rate (test 5e-6 and 1e-5), the clip range (test symmetric [0.8, 1.2] and narrower asymmetry [0.85, 1.15]), the rollout count per question (test N=4 and N=16), the reward scale (test +1/0 instead of +1/-1), and the max context length (test 8K and 24K). Each variant would be trained for at least 2,000 steps with full training dynamics tracking (entropy, reward, response length as in Figure 2) and evaluated on the nine-benchmark suite. The outcome would be a stability map showing which hyperparameter dimensions are tightly constrained (small deviations cause entropy collapse or reward divergence) and which are forgiving. This would answer the paper's own unanswered question: is the recipe's stability due to simplicity per se, or due to a carefully-chosen set of hyperparameters that happen to land in a narrow stable region? If only the exact Table 2 configuration works, the recipe is brittle and the paper's methodological advice ("start simple") is incomplete without guidance on how to find the stable configuration. If a broad region works, the recipe is genuinely robust and the case against complexity is strengthened.

Ablation of the "clip higher" mechanism to isolate its role in exploration preservation. The paper identifies the asymmetric clip range [0.8, 1.28] as "the one technique we do use" and argues it is fundamental to maintaining exploration. But this claim is never tested. A follow-up experiment would train an otherwise-identical JustRL-DeepSeek variant with a symmetric clip range [0.8, 1.2] and track the same training dynamics. The key measurement is entropy: does the symmetric clip cause entropy to collapse into the 0.5–0.6 range as the overlong penalty did (Figure 3b), or does entropy remain in the healthy 1.0–1.6 band? If entropy collapses, this would demonstrate that the asymmetric clip is the load-bearing mechanism underlying JustRL's stability — the entire "simple recipe" narrative would need to be reframed as "the recipe works because of this one specific, non-obvious design choice." If entropy remains stable, the asymmetric clip is incidental and the paper's claims about simplicity are strengthened. A secondary measurement would test whether the symmetric-clip model achieves different final performance, and whether any performance difference is mediated by entropy (supporting the exploration-collapse hypothesis) or independent of it (suggesting the asymmetry affects something else, like the effective learning rate or gradient signal-to-noise ratio).

Extension to code generation to test domain generality of the simple recipe. The paper's results are limited to mathematical reasoning — a domain with clean, verifiable answers and structured solution formats. Code generation shares the verifiability property (unit tests provide binary reward signals) but differs in important ways: solution structures are more diverse, the action space is larger, and the base models for code may have different entropy characteristics than math-distilled models. A direct extension would apply the identical JustRL recipe (same hyperparameters from Table 2) to a code generation model (e.g., DeepSeek-Coder-1.3B or a Qwen2.5-Coder-1.5B) trained on a competitive programming dataset with unit-test-based verification, evaluating on HumanEval, MBPP, and LiveCodeBench. The key question is whether the smooth training dynamics (Figure 2) replicate in the code domain. If they do, the case for domain generality is substantially strengthened — the recipe captures something fundamental about RL optimization, not something specific to math. If they don't — if code training exhibits entropy collapse, reward oscillation, or length explosion that the math recipe avoids — then the investigation shifts to understanding which properties of math (answer format structure? problem difficulty distribution? base model calibration?) enable the simple recipe's success, and what adaptations are needed for other domains. This experiment would also test whether the paper's finding about strict verifiers being beneficial (Section 4.4) generalizes: in code, a strict verifier that requires exact output matching may be far noisier than in math (many correct programs produce identical outputs through different code), and a more sophisticated verifier (e.g., one that runs the generated code against test cases and checks outputs rather than matching source code) might be necessary.

Difficulty-conditioned allocation layered on the JustRL baseline. JustRL trains uniformly on all 17,000 DAPO-Math-17k questions without difficulty filtering or curriculum learning. The earlier example paper on test-time compute scaling demonstrated that difficulty-conditioned allocation of inference compute yields 4× efficiency gains over uniform best-of-N. A natural synthesis would apply difficulty-aware allocation to the JustRL training process: estimate per-question difficulty (using the base model's pass@1 rate or the JustRL model's reward statistics during training), then allocate training compute (rollout counts, training emphasis) or inference compute (evaluation sampling budget) adaptively. A concrete experiment: after training JustRL-DeepSeek, evaluate per-difficulty-bin performance on AIME 2024 (analogous to the earlier paper's Figure 3 difficulty-bin breakdown) to determine whether JustRL's gains are concentrated in easy/medium problems or distributed across difficulties. Then test whether adaptive sampling budgets (more samples for medium problems where additional rollouts can push near-misses to successes, fewer for trivially easy or impossibly hard problems) improve the avg@32 metric at equivalent total compute, or match it at reduced compute. This experiment would connect two currently separate research threads — stable RL training (JustRL's contribution) and compute-optimal inference (the earlier paper's contribution) — and would address JustRL's unstated limitation of uniform resource allocation. If difficulty-conditioned allocation improves efficiency, it suggests the two approaches are complementary: JustRL provides the stable optimization, and difficulty-awareness provides the efficient deployment.

Long-horizon training to test whether simplicity breaks at scale. JustRL trains for 4,380 steps (DeepSeek) and 3,440 steps (Nemotron), and the training curves (Figure 1) show no sign of plateau at the final step. A natural stress test is to simply continue training: apply the identical recipe for 10,000 or 20,000 steps and observe whether the smooth dynamics persist or eventually break down. Do the entropy and reward curves remain stable indefinitely, or does the model eventually overfit to the training distribution (DAPO-Math-17k is finite, and the model sees each question many times)? Does response length continue to drift or stabilize permanently in the 4,000–5,000 token band? Does performance on the held-out 2025 benchmarks (HMMT, CMIMC, BRUMO) continue to improve, plateau, or degrade as the model over-optimizes for the training distribution's specific problem styles? This experiment addresses the open question the paper raises: "We have not explored whether our approach maintains advantages when pushed to even longer training horizons or whether additional techniques might become necessary at scale" (Section 6). If the recipe breaks at 10,000 steps — entropy collapses, reward oscillates, length explodes — then the paper's claim that complexity is unnecessary is bounded to the 4,000-step regime, and the techniques developed in prior work may become necessary at longer horizons. If the recipe remains stable, the case against complexity is strengthened: the instabilities that prior work addresses may simply never arise, even in very long training runs, if the underlying optimization is well-configured.

Reproduction of a complex baseline from scratch under controlled conditions. The paper compares JustRL against numbers reported in prior work but does not reproduce any complex baseline in its own experimental setup. A follow-up would implement ProRL-V2 (the most directly comparable prior method on the DeepSeek backbone, with its eight-stage pipeline, scheduled cosine length penalties, and dynamic hyperparameters) using the same infrastructure, same base model, and same evaluation protocol as JustRL. The comparison would control for all variables other than the training algorithm: hardware, software framework (veRL), base model checkpoint, evaluation scripts, and verifier configuration (using CompassVerifier-3B for evaluation in both cases). The primary measurement would be whether ProRL-V2's reported 53.08% average accuracy is reproducible under these controlled conditions, and whether JustRL's 54.87% remains competitive or the gap changes. A secondary measurement would be the training dynamics of both methods, tracked identically (entropy, reward, response length, benchmark performance at intermediate checkpoints). This would directly test the paper's central causal claim: do the instabilities that ProRL-V2's techniques are designed to address actually occur in the complex baseline but not in JustRL? If ProRL-V2 exhibits entropy drift, length explosion, or reward plateaus that JustRL avoids, the causal story is supported — complexity is fighting real fires, even if those fires may be self-inflicted. If ProRL-V2 trains just as smoothly as JustRL, then the paper's argument about stability differences is weaker than claimed, and the choice between simple and complex recipes reduces to a question of engineering preference and compute efficiency rather than fundamental stability.

Practical Applications and Downstream Use Cases

Cost-efficient RL fine-tuning for small math reasoning models in resource-constrained settings. The paper provides a concrete recipe that a team with access to 32 A800-80GB GPUs for approximately 15 days can use to substantially improve a 1.5B-parameter math reasoning model — no multi-stage orchestration, no dynamic monitoring, no judgment calls about when to transition between training phases. The fixed-hyperparameter, single-stage design makes the training process trivially reproducible: a practitioner can start the training script and walk away, knowing that if the run completes, the final checkpoint will be in the competitive performance range reported in Tables 3 and 5, without requiring mid-training intervention to adjust hyperparameters or advance stages. This matters for academic labs, startups, and organizations without dedicated RL engineering teams who want to push distilled models past their distillation ceiling. The compute cost — approximately 1.1–1.4×10^8 tokens processed, or 15 days on 32 A800 GPUs — is not trivial but is substantially lower than the 2–5× higher costs of complex alternatives (Tables 4 and 6), and the elimination of engineering complexity reduces the human cost of implementation and monitoring. For a team that has already invested in distillation and is looking for the next increment of performance, JustRL provides a turnkey next step.

Baseline for evaluating new RL techniques in the small-model reasoning literature. Perhaps the most immediate practical use of JustRL is as a standardized baseline that future work can compare against. The paper releases its models, code, and full hyperparameter configuration (Table 2). A research group proposing a new technique — curriculum learning, adaptive temperature, novel verifier design, data augmentation — can now train their method starting from the same base models (DeepSeek-R1-Distill-Qwen-1.5B or OpenMath-Nemotron-1.5B), evaluate on the same nine-benchmark suite using the same evaluation protocol (CompassVerifier-3B at temperature 0.7 with the specified sampling budgets), and report both absolute performance and the delta relative to the JustRL baseline. This addresses the causal attribution problem the paper identifies: when a new technique is evaluated against JustRL rather than against the authors' own prior complex system, the community can determine whether the technique provides genuine benefit or merely compensates for problems in the complex baseline. The paper's ablation findings (Section 4.4) — that "standard tricks" degrade performance — make this baseline especially valuable: techniques that improve over JustRL must be doing something genuinely useful, since they cannot be accidentally patching a self-inflicted wound. Over time, this could shift the literature from an accumulation of interacting complexity toward a clearer understanding of which techniques actually matter.

Training signal design for domains where verifier choice is non-obvious. The paper's counterintuitive finding that a stricter verifier produces better final models than a more permissive one (Figure 3) has immediate practical implications for practitioners designing RL training pipelines. When choosing or building a verifier for RL training, the standard instinct is to minimize false negatives — make the verifier as accurate as possible at recognizing correct answers. JustRL suggests this instinct may be wrong for training (as opposed to evaluation). A practitioner should now consider verifier strictness as a tunable design parameter that affects learning dynamics, not just as an accuracy metric to be maximized. Specifically, the paper's results suggest that a verifier which is strict enough to produce a spectrum of outcomes (some good solutions marked wrong, creating varied reward signals) and to pressure the model toward precision (formatting errors are punished) may produce better final performance than a verifier that forgives imprecision. This has concrete implications for practitioners: when training on math, consider using exact string matching rather than SymPy-based equivalence checking during training; when training on code, consider requiring exact output matching rather than fuzzy comparison; when designing learned reward models, consider whether high recall (catching all correct answers) might come at the cost of reduced precision pressure. The paper does not claim this is universally true — only that it should be tested rather than assumed — but the ablation result is striking enough to warrant default skepticism about the benefits of verifier permissiveness in RL training.

When to Prefer This Method

The paper does not articulate a detailed decision framework comparing JustRL against named alternatives under well-defined conditions. It positions itself methodologically — "start simple, scale up, and only add complexity when a simple, robust baseline demonstrably fails" — rather than providing a feature-by-feature comparison or specifying precise boundary conditions where complexity becomes necessary. The paper's limitations section (Section 6) explicitly acknowledges contexts where JustRL is untested (other domains, model sizes, longer training horizons) but does not claim that complex methods are preferable in those contexts, only that the question is open. Similarly, Section 5 notes that "additional techniques may be valuable under extreme compute constraints, when encountering specific failure modes we didn't face, when pushing beyond current performance ceilings, or in domains with noisier reward signals" — a list of candidate conditions rather than a tested set of decision rules. The paper's contribution is the baseline itself, not a positioning framework against alternatives. A forced decision matrix would therefore be speculative rather than grounded in the paper's experimental results.