Golden Goose: A Simple Trick to Synthesize Unlimited RLVR Tasks from Unverifiable Internet Text

🎯 Pitch

Models training with RLVR hit an improvement ceiling because we simply run out of verifiable tasks—but what if you could turn any dense, reasoning-rich text like a science textbook into a multiple-choice quiz where the reward is automatic? That’s the core trick here, and it lets a saturated Qwen-4B model gain +3.48% in STEM and +2.71% in math by continuing RL on synthetic data, while training further on the original data yields near-zero improvement.

1. Executive Summary

This paper introduces Golden Goose, a data synthesis pipeline that converts unverifiable internet text—such as science textbooks, theorem-proving solutions, and code problems without test cases—into unlimited RLVR (Reinforcement Learning with Verifiable Rewards) tasks by constructing a multiple-choice fill-in-the-middle problem (an LLM identifies a crucial reasoning span in a source passage, masks it, and generates plausible but incorrect distractors, producing a verifiable task that simply checks whether the selected option matches the ground-truth). Using GPT-5 to synthesize from AoPS-Instruct, rStar-Coder, and MegaScience corpora, the authors construct GooseReason-0.7M, a dataset of over 0.7 million tasks spanning mathematics, programming, and general science, and demonstrate on models from the ProRL family that it revives saturated RL training—continuing RL with GooseReason yields robust gains of 2.71% in math, 2.12% in coding, and 3.48% in STEM over 1,100 H100 GPU hours, while the original ProRL data produces near-zero improvement. Further training a Qwen-4B-Instruct on GooseReason achieves new state-of-the-art results among 4B-Instruct models across 15 diverse benchmarks, and deploying the pipeline on cybersecurity web scrapes (FineWeb) produces GooseReason-Cyber, which after only 100 RL steps yields a 4.44% absolute gain and surpasses a 7B domain-specialized model, establishing that abundant unverifiable text can substitute for scarce handcrafted verifiable data—but only when the source text contains sufficient reasoning density to support plausible distractor generation.

2. Context and Motivation

The Core Problem: Data Saturation Is the Bottleneck for Scaling RLVR

The central problem this paper addresses is deceptively simple: RLVR training data is finite, and models saturate on it quickly, yet the LLM reasoning community lacks a scalable mechanism for generating fresh, diverse, verifiable training tasks. This is not merely a data volume problem—it is a data regime problem that manifests differently depending on model strength and training duration, and it represents the primary bottleneck for continued RL scaling beyond what current recipes can achieve.

To understand why this matters, we need to first understand the RLVR paradigm itself. Reinforcement Learning with Verifiable Rewards (RLVR) is a post-training technique where an LLM is trained to improve its reasoning by generating multiple solution attempts to a problem, with each attempt receiving a binary or scalar reward based on whether it reaches the correct answer (as verified automatically, e.g., via a math equality check or unit test execution). The ProRL family of work (Liu et al., 2025a; Hu et al., 2025b, 2025c) demonstrated that prolonged RLVR training—up to thousands of steps—can yield substantial, continuous reasoning improvements, and this approach has been a core ingredient behind frontier reasoning models like DeepSeek-R1 (Guo et al., 2025b), OpenAI-o3 (OpenAI, 2025b), and Gemini-3 (Google DeepMind, 2025).

The bottleneck emerges from a specific phenomenon that the paper documents quantitatively: data staleness at scale. When a model has been trained extensively on a fixed set of RLVR problems, its performance on those problems gradually polarizes. Some problems become consistently solvable (the model always gets them right across all rollouts), while others remain consistently unsolvable (the model always fails). In both cases, the problem provides no learning signal—the advantage estimates, which depend on the contrast between successful and failed rollouts within a batch, collapse to zero. The paper measures this directly in Figure 3: for ProRL-1.5B-v2, a model trained for over 20,000 H100 GPU hours on a 136K-problem RLVR data blend, only ~25% of those original problems remain "effective" (meaning they elicit both correct and incorrect rollouts from the model). The other 75% provide no gradient information for further training.

This saturation has two compounding consequences. First, it means continued RL on the original data blend yields diminishing returns—the paper shows near-zero aggregate improvement over an additional 1,100 H100 GPU hours when continuing with ProRL data only (Figure 5). Second, and more critically for the field, saturation occurs earlier and is more severe with stronger models. While the ProRL recipe sustains continuous gains for R1-Distill-Qwen-1.5B over thousands of steps, applying the same recipe to the more capable Qwen-4B-Instruct causes performance to plateau or even degrade after merely 300 steps (Figure 2). A stronger model exhausts the available learning signal from the fixed problem set faster because it either masters or fails the problems more rapidly, collapsing the advantage distribution. This means that as base models improve, the data problem intensifies rather than diminishing—the very success of better pretrained models makes the RLVR data bottleneck more acute.

Why This Problem Matters: The Verifiability Constraint Creates a Data Wall

The core tension in RLVR data construction is that the verifiability requirement that makes RLVR so effective simultaneously makes its data supply artificially scarce. To be usable for RLVR, a problem must satisfy a strict format constraint: its solution must be automatically verifiable by a simple, programmatic check—a math expression evaluated for equality, a code submission executed against unit tests, a single-choice answer compared to a ground-truth label. This constraint excludes the overwhelming majority of reasoning-rich content that exists in the world.

The paper identifies several categories of excluded content that are particularly painful losses for reasoning training:

Theorem-proving problems: The AoPS forum (Mahdavi et al., 2025b) contains approximately 600K question-answer pairs, but a large subset consists of full mathematical proofs. These require reasoning at the highest level—strategic lemma selection, proof-structure planning, precise formal manipulation—but cannot be verified by a math-equality checker because the solution is an entire argument, not a single final expression.
Open-ended scientific explanations: The MegaScience corpus (Fan et al., 2025) extracted 650K question-answer pairs from university-level science textbooks spanning physics, biology, chemistry, medicine, and economics. Many of these questions require multi-paragraph explanations, free-form diagnostic reasoning, or specialized formula manipulation (e.g., chemical structures) that are impossible to validate with simple string matching.
Code problems without test cases: The rStar-Coder dataset (Liu et al., 2025b) contains over 1.6M synthesized competitive programming problems, but only ~23% of them (380K) successfully obtained test cases through an automated pipeline. The remaining ~1.3M problems—with expert-written oracle solutions—sit unused for RLVR because there is no way to automatically verify whether a model's code submission is correct.

The status quo prior to this paper has been to simply discard all such content and rely exclusively on human-authored verifiable problems (math problems with numeric answers, code problems with unit tests) or handcrafted procedural generators that algorithmically produce problems within a fixed, narrow environment (e.g., logic puzzles with a fixed rule set, arithmetic with configurable difficulty). The paper identifies fundamental limitations with both approaches:

Human-authored data is expensive and domain-narrow. Collecting, curating, and verifying large-scale RLVR datasets requires specialized human expertise (mathematicians for Olympiad problems, competitive programmers for code problems) and is inherently slow to scale. The existing large-scale RLVR datasets—Big-Math (Albalak et al., 2025), Omni-Math (Gao et al., 2024), AceReason (Chen et al., 2025), SCP-116K (Lu et al., 2025), and the data underlying ProRL—are substantial engineering efforts, but they are bounded by the availability of human-created problems with the right verification properties. Furthermore, they cluster in math and code because those are the only domains where large corpora of human-authored problems with clean verification signals naturally exist. Domains like medicine, law, economics, and cybersecurity are almost entirely absent from the RLVR data landscape.

Procedural environments do not scale to knowledge-rich domains. Handcrafted verifiable environments such as Reasoning Gym (Stojanovski et al., 2025), Reasoning Core (Lacombe et al., 2025), and RLVE (Zeng et al., 2025a) can generate unlimited examples within their pre-specified rule systems—logic puzzles, arithmetic, simple games, symbolic manipulation. The paper's experimental results with RLVE are telling: it is "highly effective on math" but its impact on STEM reasoning is "limited to a 0.62% gain" (Section 3.1.1). The fundamental issue is that procedural generation excels at algorithmic tasks where correctness follows from a closed set of formal rules, but it breaks down for knowledge-intensive tasks where reasoning requires facts, concepts, and relationships from a rich domain—the difference between solving a Sudoku puzzle and diagnosing a disease from symptoms. The paper states this limitation explicitly: "it remains unclear how to adapt such procedural generation to knowledge-intensive STEM domains like medicine, economics and cybersecurity" (Section 3.1.1).

Prior Approaches and Where They Fall Short

The paper positions itself relative to three families of prior work, each of which addresses pieces of the RLVR scaling problem but leaves the fundamental data supply constraint unresolved.

Scaling RL training duration and rollout budget (algorithmic scaling). ProRL (Liu et al., 2025a) proposed a mixture of data containing several reasoning tasks alongside modifications to GRPO (Group Relative Policy Optimization) to stabilize training over prolonged steps. ProRL v2 (Hu et al., 2025b) further validated that RL scaling laws hold when training is extended. BroRL (Hu et al., 2025c) scaled by increasing the number of rollouts per example rather than extending step count. ScaleRL (Khatri et al., 2025) conducted a large-scale analysis comparing different recipes. All of these works focus on the algorithmic axis: given a fixed dataset, how do we design the training procedure to extract maximum value from it? The paper's contribution is complementary and orthogonal: it focuses on the data axis, asking how to generate fresh RLVR data so that the algorithmic recipes have something to continue learning from. This is not a replacement for algorithmic scaling but a prerequisite for it to function beyond the data saturation point. The paper's experiments make this relationship concrete: the very same ProRL recipe that stagnates when continuing on ProRL data yields robust gains when supplied with GooseReason data, using identical algorithmic settings (Figure 5).

Large-scale human annotation for RLVR. Significant community effort has gone into curating verifiable problems at scale, producing high-quality datasets like Big-Math (250K+ math problems), Omni-Math (Olympiad-level mathematics), AceReason-Nemotron (math and code), DeepScaleR (scaling-focused problems), and SCP-116K (higher-education science). These datasets are valuable but share a structural limitation: they source problems from human-authored exam banks, competition archives, and textbook exercises—sources that are finite and biased toward domains with built-in answer formats. The paper positions GooseReason as a complement, not a competitor: rather than collecting more problems from the same narrow sources, it converts fundamentally different source material (proofs, explanations, incomplete problems) into the verifiable format that RLVR requires.

Automated data synthesis via procedural environments. RLVE (Zeng et al., 2025a) and Reasoning Gym (Stojanovski et al., 2025) represent the state-of-the-art in procedural data generation for RLVR. These systems produce problems with tunable difficulty by parameterizing rule-based environments—for instance, varying the number of variables in a logic puzzle or the complexity of an arithmetic expression. RLVE adds adaptive difficulty: it adjusts problem parameters based on the current policy's performance to maintain an effective learning regime. The paper's comparison with RLVE in Figure 5 and the breakdown Figures 11–12 reveals the pattern: RLVE achieves strong math gains (even comparable to GooseReason on some math benchmarks), but its STEM impact is modest because procedural generation cannot encode the factual and conceptual knowledge required for graduate-level science reasoning. GooseReason sidesteps this limitation entirely by extracting problems from existing human-authored explanations rather than attempting to generate problems from hand-coded rules.

How This Paper Positions Itself

The paper's positioning can be understood through a single organizing insight: abundant, reasoning-rich, unverifiable internet text can be converted into verifiable RLVR tasks by constructing a multiple-choice fill-in-the-middle problem, where the "verification" reduces to checking whether the selected option matches the known ground-truth. The authors do not claim this is the only solution or that it works for all types of text. But they demonstrate that it works for a sufficiently broad range of source material—Olympiad proofs, textbook explanations, code solutions without tests, web scrapes—that it substantially expands the RLVR data frontier.

This positioning is deliberate in several respects:

It is data-centric, not algorithmic. The paper explicitly adopts the existing ProRLv2 RL recipe (a GRPO variant with clipped objective and decoupled advantage normalization from REINFORCE++) without modification. All experimental gains are attributable to data, not training innovations. This cleanly isolates the contribution and makes the results directly attributable.
It is complementary, not competitive. GooseReason-0.7M is designed to be "seamlessly pluggable into any RL recipe" and is used alongside existing ProRL data in experiments, not as a replacement. The paper emphasizes that GooseReason supplements existing datasets, particularly in domains where they are weakest (STEM), rather than trying to outperform them in their areas of strength (math, where human-authored problems are already abundant).
It targets the saturation regime. The experiments are designed around models that have already plateaued on existing data—the scenario where the method provides the most value. The paper does not claim GooseReason is necessary for initial training from scratch; rather, it demonstrates that GooseReason unlocks continued scaling once the original data is exhausted. However, the compute-efficient scaling experiment (Figure 6) shows that GooseReason also accelerates training before saturation, suggesting its benefits apply across the training lifecycle.
It validates with real-world deployment. The cybersecurity experiment is not a toy demonstration. The authors take raw FineWeb scrapes—the kind of noisy, unstructured text that real-world practitioners would have access to—and synthesize RLVR data for a domain where no prior RLVR data exists. Training a general-purpose Qwen-4B-Instruct on this domain-specific RLVR data for only 100 steps produces a model that surpasses a 7B domain-specialized model (Llama-Primus-Instruct) that underwent extensive domain-specific pre-training and post-training. This is the paper's strongest evidence that the method is not merely a benchmark hack but a genuine capability unlock for specialized domains.

The paper frames its contribution as part of a broader vision: "automatically re-utilizing reasoning-rich unverifiable internet text to enable RL scaling" (Section 5), with explicit mention of law and medicine as future domains where "verifiable data is scarce but professional literature is abundant." This positions Golden Goose not as a one-off dataset release but as a methodological template for converting domain-specific corpora into RLVR training data wherever expert-written explanations exist.

3. Technical Approach

3.1 Reader Orientation

Golden Goose is a data synthesis pipeline—not a new RL algorithm or model architecture—that consumes unverifiable internet text (scientific textbooks, math proofs, code solutions without test cases, raw web scrapes) and outputs multiple-choice fill-in-the-middle problems that are automatically verifiable during RL training. The problem it solves is the data saturation bottleneck in RLVR: when models are trained extensively on a fixed set of verifiable problems, most problems stop providing any learning signal (the model either always gets them right or always fails), causing training to plateau. The solution takes the form of a format-conversion trick: take text that was unusable for RLVR because it lacked simple answer-checking, restructure it into a multiple-choice question where the answer is the original masked text, and now verification reduces to checking whether the selected option index matches the known ground-truth index.

3.2 Big-Picture Architecture (Diagram in Words)

The Golden Goose system has five major stages, with two variants depending on source-text quality:

Stage 1 — Source Text Selection. A corpus of reasoning-rich text is identified. For the reasoning domain (GooseReason-0.7M), this corpus consists of existing curated datasets: AoPS-Instruct (Olympiad math QA), rStar-Coder (competitive programming problems without test cases), and MegaScience (university textbook QA). For the cybersecurity domain (GooseReason-Cyber), the corpus is raw FineWeb scrapes filtered for cybersecurity content.

Stage 2 — Passage Extraction (noisy sources only). If the source text is noisy (e.g., web scrapes), an LLM is prompted to extract or summarize the raw text into a coherent, educationally valuable passage. If no suitable passage exists, the LLM returns an empty string and the document is discarded.

Stage 3 — Masking and Distractor Generation. For each passage, an LLM (GPT-5) is prompted to: (a) identify a contiguous span of crucial reasoning steps, (b) replace it with a [MASK] token to create the masked context, (c) treat the removed span as the ground-truth answer, and (d) generate a set of plausible but incorrect distractor options that are stylistically similar to the ground-truth.

Stage 4 — MCQ Assembly. The final task is assembled as (masked_context, {ground_truth} ∪ distractors), with options randomly ordered. The student model sees the masked context and a numbered list of candidate completions, and must select which option best fills the [MASK].

Stage 5 — Difficulty Filtering (noisy sources only). For data synthesized from noisy sources, each task is rolled out 16 times with the target student model. Tasks where the model achieves consistent zero accuracy or consistent 100% accuracy are filtered out, retaining only the medium-difficulty tasks that provide a learning signal (both successful and failed rollouts).

During RL training, verification is a simple exact-match check: compare the student model's predicted option (extracted from its generated response) against the stored ground-truth option index. There is no LLM-as-judge, no math verifier, and no sandbox execution required.

3.3 Roadmap for the Deep Dive

First, the core mechanism—the fill-in-the-middle MCQ construction—because it is the "trick" that makes the entire pipeline work and understanding its specifics (why MCQ over open-ended, why 9 distractors, what counts as a "crucial reasoning step") is prerequisite for everything else.
Second, the source corpora and how they are transformed, since the choice of source text determines what kind of reasoning the resulting tasks require and explains the domain coverage of GooseReason-0.7M.
Third, the design choices (MCQ vs. open-ended, number of distractors, difficulty filtering), because these are the non-obvious decisions that make the difference between a pipeline that produces effective RLVR data and one that produces tasks the model either ignores or trivially solves.
Fourth, the RL training recipe, since the experimental results depend on how GooseReason data is integrated into the training loop and understanding the RL algorithm provides context for why "effective" tasks are defined as those that produce both successful and failed rollouts.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a data-centric methods paper whose core idea is that unverifiable reasoning-rich text can be converted into verifiable RLVR tasks by constructing a multiple-choice version of the fill-in-the-middle objective, where the verification signal reduces to checking whether the selected option matches the known ground-truth. No new RL algorithm is introduced; the contribution is entirely in the data synthesis pipeline, the design choices that make the resulting tasks effective for RL training, and the empirical demonstration that this data unlocks continued scaling beyond the saturation point of existing RLVR datasets.

The Core Mechanism: Fill-in-the-Middle MCQ Construction

The fundamental operation in Golden Goose is converting a passage of reasoning-rich text into a multiple-choice question where the model must identify which candidate span correctly fills a masked region. This is not a new task format—fill-in-the-middle (FIM) is a standard pretraining objective where a model is trained to predict a missing span given surrounding context. What is novel is the repurposing of FIM as an RLVR task by constructing it as a multiple-choice problem rather than as open-ended generation.

The construction proceeds as follows. Given a source text S (which could be a paragraph from a textbook, a step in a proof, a segment of a code solution, or an extracted passage from a web scrape), an LLM is prompted to:

Identify a contiguous span t of crucial reasoning steps. The prompt instructs the LLM to select text that represents an important inferential step—not merely a fact statement that could be looked up, and not a trivial algebraic simplification, but a segment where non-obvious reasoning occurs. The paper does not provide the exact prompt text in the main body (it is in Appendix A, which is not included in the provided content), but the criterion is described as targeting "crucial reasoning steps."
Construct the masked context S_mask by replacing t in S with a special [MASK] token. The resulting S_mask is the problem statement—it contains the surrounding context that the student model must use to infer what was removed.
Treat t as the ground-truth answer. This is the key move that enables verification: because the removed text is known, the correct option in the multiple-choice set is predetermined. The synthesis LLM does not need to solve the problem; it already knows the answer because it chose which span to remove.
Generate a set of distractors D = {d_1, d_2, ..., d_k}. The LLM is prompted to produce options that are (a) plausible—they could reasonably fill the [MASK] given the context—and (b) similar in style and length to the ground-truth t—so that the task cannot be solved by surface-level heuristics like option length—and (c) incorrect in the context of S_mask. The incorrectness constraint is critical: each distractor must be a statement that, if inserted into the masked position, would make the overall passage false, inconsistent, or logically flawed.

The final task Q is assembled as:

$Q = (S_{\text{mask}}, \{t\} \cup D)$

where S_mask is the masked source text, t is the ground-truth answer, and D = {d_1, d_2, ..., d_k} is the set of k distractors.

What it computes: the task is a multiple-choice question where the student model receives the masked context S_mask and a randomized list of k+1 candidate completions (one correct, k incorrect), and must select the option that correctly fills the [MASK]. Verification during RL checks whether the predicted option index matches the stored index of the ground-truth t.

Why this form: the multiple-choice constraint is what enables verification without an LLM judge, a math parser, or a code sandbox. An open-ended fill-in-the-middle task (where the model freely generates the masked content) would require a judge model to assess whether the generated text is semantically equivalent to the ground-truth, introducing both computational overhead during RL training and a potential source of reward noise. By restricting the output space to k+1 discrete options, verification becomes a trivial string match: extract the option label from the model's response, compare to the ground-truth label, return a binary reward. This is the "simple trick" referenced in the paper's title—the multiple-choice format converts the unverifiable open-ended generation problem into a verifiable discrete-choice problem without losing the reasoning requirement (the model still needs to understand the masked content and evaluate which option is consistent with the surrounding context).

The paper explicitly considers and rejects an alternative open-ended formulation. As shown in Figure 4, when GooseReason-Math tasks are presented as open-ended fill-in-the-mask problems (where the model generates the missing content and an LLM-as-judge evaluates correctness), over 83% of examples result in consistent zero accuracy for ProRL-1.5B-v2—the model ignores the instruction to generate only the infill and instead attempts to solve the entire problem from scratch, producing long-form solutions that do not match the expected infill format. This instruction-following failure is particularly acute for models that have been heavily RL-tuned for reasoning, because their training reinforces the behavior of generating complete solution chains. The multiple-choice format bypasses this problem by constraining the output format: the model must select and output an option label (or the text of the chosen option), which is a much simpler instruction-following task than generating precisely the right infill span.

Distractor Generation and the Number of Options

The quality of the distractors determines whether the resulting MCQ requires genuine reasoning or can be solved through superficial elimination strategies. The paper's design of the distractor generation prompt targets three properties:

Plausibility. Each distractor must be a candidate that could reasonably fill the [MASK]—meaning it is grammatically consistent with the surrounding text and superficially plausible given the context. If distractors are obviously wrong (e.g., statements that contradict basic facts stated in S_mask), the task becomes trivial and provides no reasoning challenge.

Stylistic similarity. Distractors must match the style, tone, and approximate length of the ground-truth t. This prevents the model from using surface cues—such as option length (the correct option being systematically longer or shorter), formatting, or vocabulary mismatch—to identify the correct answer without reasoning about content.

Incorrectness. Each distractor must be incorrect when inserted into S_mask. This is the substantive constraint: the distractor must create a logical, factual, or mathematical inconsistency with the surrounding passage. The paper's formulation of "plausible yet incorrect" is the standard multiple-choice design principle—the distractors should represent common misconceptions, nearby-but-wrong inferences, or alternative reasoning paths that might seem correct to someone who does not fully understand the passage.

The number of distractors k is a critical hyperparameter that the paper ablates in Figure 4. Using the GooseReason-Math subset (synthesized from AoPS-Instruct), the authors measure the accuracy distribution of ProRL-1.5B-v2 across 16 rollouts per task for different values of k. The key finding is that the proportion of tasks falling into the medium-difficulty regime (both successful and failed rollouts) increases with the number of distractors:

With 3 options (2 distractors), the majority of problems are "overly easy"—the model achieves high accuracy across rollouts. The paper attributes this to an elimination strategy: with only two distractors, the model can identify flaws in the provided options rather than performing the intended reasoning to infer the masked content. The model effectively solves the MCQ by criticizing each option individually rather than reasoning forward from the context.
With 9 options (8 distractors), over 70% of problems fall into the medium-difficulty regime. The elimination strategy breaks down because there are too many options to individually falsify within a fixed generation budget (the model has limited output length and cannot exhaustively reason about 8 distractors). The model is forced to engage with the masked content itself—it must understand what the [MASK] should contain and identify which option matches that understanding.

The paper does not report results for intermediate values between 3 and 9, so the exact scaling behavior is unclear, but the choice of k = 8 distractors (for a total of 9 options) is justified as the configuration that maximizes the fraction of tasks in the effective difficulty zone for ProRL-1.5B-v2. This is a model-specific hyperparameter—the optimal number of distractors depends on the student model's capability level and its tendency to use elimination strategies. A stronger model might require more distractors; a weaker model might be overwhelmed by too many options.

The paper also provides an implicit justification for why distractors can be generated reliably by GPT-5: the synthesis LLM has access to the full passage S (both the context and the ground-truth), so it can assess whether a candidate distractor would create an inconsistency. The prompt can instruct the LLM to verify each distractor by mentally inserting it into S_mask and checking for logical coherence—a task that is much easier for an LLM than solving the problem from scratch, because it involves evaluating consistency rather than generating correct reasoning.

Handling Noisy Source Text: The Passage Extraction and Filtering Stages

The pipeline described above assumes the source text is coherent and educationally structured—a textbook paragraph, a proof step, or a solution explanation. For raw web scrapes (as in the cybersecurity domain experiment), the source text is noisy, unstructured, and often not directly usable as a reasoning passage. The Golden Goose pipeline adds two additional stages for such sources.

Passage extraction/summarization. Before masking, the LLM is prompted to extract or summarize the raw text S into a coherent, educationally valuable passage S'. The prompt instructs the LLM to identify content that contains reasoning—explanations, analyses, step-by-step procedures—rather than mere factual enumeration or noise. If the raw text contains no suitable passage, the LLM returns an empty string, and the document is discarded. The masking and distractor generation then operate on S' rather than S.

This extraction step serves as a quality filter that discards non-reasoning content before any synthesis occurs. The paper does not report what fraction of FineWeb cybersecurity scrapes pass this filter, but the final dataset size (180K tasks) implies that a substantial amount of raw text is discarded or that the extraction step is effective at condensing noisy text into concise reasoning passages.

Difficulty-based filtering. After MCQ construction, each task is evaluated with the target student model (the model that will be RL-trained) by generating 16 independent rollouts on the task. A task is retained only if the model achieves neither consistent 0% accuracy nor consistent 100% accuracy across these rollouts. In practice, this means the model must get the task correct on some rollouts and incorrect on others—the scenario that produces a meaningful advantage signal for policy gradient RL.

This filtering criterion is identical to the "effective example" metric introduced in Figure 3. The paper reports that for GooseReason-0.7M (synthesized from clean source corpora), approximately 70% of tasks are effective relative to ProRL-1.5B-v2, compared to only 25% for the original ProRL data blend after extensive training. This high effectiveness ratio is not an accident—it reflects the design choice of 9-option MCQs, which pushes more tasks into the medium-difficulty regime, combined with the reasoning difficulty of the source material (Olympiad proofs, university textbook explanations, competitive programming solutions). The paper notes that for reasoning-dense source text (AoPS-Instruct, rStar-Coder, MegaScience), the questions constructed by GPT-5 were "of sufficient quality and difficulty to require no further post-processing"—meaning difficulty filtering was only necessary for the noisy FineWeb source.

The difficulty filtering step introduces an exploration-exploitation tradeoff that the paper does not fully analyze: the filtering requires generating 16 rollouts per candidate task, which is a non-trivial computational cost. For the cybersecurity domain with 180K final tasks, if the initial synthesis produced a much larger number of candidates, the filtering cost could be substantial. The paper does not report the ratio of synthesized to retained tasks or the compute spent on filtering. However, because the filtering is a one-time cost (performed once before RL training begins) rather than a recurring cost (unlike adaptive difficulty estimation, which would need to be updated as the model evolves), it may be acceptable in practice.

Source Corpora and the Construction of GooseReason-0.7M

The GooseReason-0.7M dataset is synthesized from three existing corpora, each chosen because it contains reasoning-rich content that was excluded from prior RLVR datasets due to the lack of automatic verifiability. Understanding what is in these corpora—and why they were previously unusable—is essential for understanding what Golden Goose contributes.

AoPS-Instruct (mathematics). Mahdavi et al. (2025b) extracted approximately 600K question-answer pairs from the Art of Problem Solving (AoPS) forum, an online community where users post Olympiad-level math problems and community members contribute solutions. The forum is unstructured and noisy—solutions vary in format, completeness, and quality. The critical exclusion factor for RLVR is that AoPS contains a large number of theorem-proving problems whose solutions are entire mathematical proofs. A math verifier (which checks whether a final numeric answer matches a ground-truth) cannot validate a proof, because correctness depends on the logical structure of the entire argument, not a single output token. These proof problems represent some of the highest-quality reasoning data available (Olympiad-level mathematics requires strategic lemma selection, proof-structure planning, and precise formal manipulation), but they have been sitting unused in the RLVR context because no verification mechanism existed.

Golden Goose converts these proofs into RLVR tasks by extracting a crucial reasoning step from the solution—for instance, a key lemma application or a non-trivial algebraic manipulation—and constructing the MCQ around that step. The resulting task tests whether the model can identify the correct reasoning step given the surrounding proof context, which is a different skill from generating an entire proof but still requires meaningful mathematical reasoning.

rStar-Coder (coding). Liu et al. (2025b) curated 37.7K expert-written competitive programming problems with oracle solutions and used them as seeds to synthesize 1,656K new problems. However, their test case synthesis pipeline—which consists of a three-step input generation method and a mutual verification mechanism for output labeling—only succeeded for 380K of these synthesized problems. The remaining ~1,276K problems in the synthetic_sft split contain only problem statements and teacher model solutions, with no test cases. These are unusable for standard RLVR because there is no way to execute and verify model-generated code submissions.

Golden Goose converts these code problems by extracting a crucial segment of the solution—for instance, a key algorithmic insight, a non-trivial data structure manipulation, or a complex conditional logic block—and constructing the MCQ around that segment. The masked context would be the surrounding code or problem description. The task tests whether the model can identify the correct implementation fragment given the algorithmic context, which requires understanding the solution approach without needing executable code.

MegaScience (general science). Fan et al. (2025) extracted 650K question-answer pairs from nearly 12K university-level scientific textbooks spanning physics, biology, chemistry, medicine, computer science, mathematics, and economics. The exclusion factors for RLVR are domain-specific: chemistry problems involve specialized scientific formulas that are difficult to automatically verify; medicine and economics problems are free-form or open-ended, requiring multi-paragraph discussions or explanations. A simple string match against a ground-truth answer is not feasible because the same correct reasoning can be expressed in many different ways.

Golden Goose converts these by masking a crucial reasoning step within the textbook explanation and constructing distractors that represent common misconceptions or plausible-but-incorrect inferences. Because the original source is a textbook (not a problem set), the "question" is effectively the surrounding explanation with a gap, and the "answer" is the step that completes the reasoning chain. This format is particularly well-suited to science domains because scientific explanations naturally contain sequential reasoning steps where one step follows from the previous one.

From these three sources, the paper synthesizes over 0.7M RLVR tasks. The resulting dataset, GooseReason-0.7M, spans mathematics (from AoPS-Instruct), programming (from rStar-Coder), and general science (from MegaScience). Figure 3 shows the breakdown in terms of total examples and effective examples (relative to ProRL-1.5B-v2). The paper reports that GooseReason-0.7M provides over 450K effective examples in math, code, and STEM combined, which is a 13× increase over the total effective examples in the ProRL data blend (which contains only about 34K effective examples out of 136K total). This 13× figure is the paper's headline statistic for the magnitude of the data contribution, and it quantifies the degree to which existing RLVR datasets have been exhausted by current models versus the freshness of the synthesized data.

RL Training Recipe: ProRLv2 with GRPO Variant

The paper does not introduce a new RL algorithm but adopts the existing ProRLv2 recipe (Hu et al., 2025b), which is itself a variant of GRPO (Group Relative Policy Optimization; Shao et al., 2024). Understanding the RL recipe is important because it defines what constitutes an "effective" training example and how the advantage signal is computed.

GRPO is a policy-gradient algorithm designed for RLVR where the reward is a binary verifiable signal. For each training step, the algorithm:

Samples a batch of prompts (MCQ tasks from GooseReason or the ProRL data blend).
Generates N rollouts per prompt (the paper uses 16 rollouts in its effectiveness analysis; the training-time rollout count may differ). Each rollout is the model's full generated response—usually including a chain-of-thought reasoning trace followed by the selected option.
Computes a reward for each rollout by extracting the predicted option and comparing it to the ground-truth option index. The reward is binary: 1 if correct, 0 if incorrect.
Computes advantages within each group (all rollouts for the same prompt). The ProRLv2 recipe uses a clipped GRPO objective with decoupled advantage normalization from REINFORCE++ (Hu et al., 2025a): group-wise mean subtraction (subtract the mean reward of rollouts for the same prompt) followed by batch-level standardization (divide by the standard deviation across all rollouts in the training batch). This two-stage normalization is designed to stabilize training over prolonged steps by preventing advantage estimates from collapsing when a prompt becomes too easy or too hard.
Updates the policy using the clipped GRPO loss, which penalizes large policy updates.

The key connection to GooseReason's design is in step 4. The GRPO advantage for a rollout is approximately:

$A_i \approx \frac{r_i - \bar{r}_{\text{group}}}{\sigma_{\text{batch}}}$

where r_i is the binary reward for rollout i, r̄_group is the mean reward of all rollouts for the same prompt, and σ_batch is the standard deviation of rewards across the entire batch.

What it computes: the advantage A_i is a normalized measure of how much better (positive) or worse (negative) rollout i is than the average rollout for the same prompt. If the model gets all rollouts correct (r̄_group = 1), every rollout has advantage zero—there is no signal to learn from. If the model gets all rollouts wrong (r̄_group = 0), the same collapse occurs. The advantage is only non-zero when the model produces a mix of correct and incorrect answers for the same prompt.

Why this matters for data design: this advantage structure is the entire motivation for GooseReason's focus on medium-difficulty tasks. The filtering criterion—retaining tasks where the model produces both correct and incorrect rollouts—is a direct consequence of the GRPO advantage formula. Tasks that are too easy or too hard provide zero learning signal regardless of their reasoning quality. The 9-option MCQ format and the difficulty filtering stage are both designed to maximize the proportion of synthesized tasks that fall into this effective zone for the target student model.

The paper uses the ProRLv2 recipe without modification for all experiments, which cleanly isolates the data contribution: any performance differences between training with ProRL data alone and training with GooseReason data arise from the data, not from algorithmic improvements.

Cybersecurity Domain Deployment: GooseReason-Cyber

The cybersecurity experiment serves as a real-world validation of the pipeline's generality. Unlike the reasoning domain—where existing curated corpora (AoPS-Instruct, rStar-Coder, MegaScience) provided clean source text—the cybersecurity domain starts from raw web scrapes where no RLVR data previously existed.

Source data. Primus (Yu et al., 2025) released two components for cybersecurity LLM pretraining: Primus-Seed (data crawled from reputable sources such as MITRE, Wikipedia, and cybersecurity company websites, plus human-collected cyber threat intelligence) and Primus-FineWeb (text from FineWeb filtered for cybersecurity relevance, using Primus-Seed as positive examples for the filter). These are primarily web scrapes and are "extremely noisy" per the paper.

Pipeline adaptation. The full pipeline with passage extraction and difficulty filtering is applied. For each FineWeb scrape, GPT-5 is first prompted to extract or summarize the text into a coherent, educationally valuable passage—filtering out non-reasoning content such as news headlines, product listings, or unstructured logs. The masking and distractor generation then proceeds as in the reasoning domain, with one key modification: after MCQ construction, each task undergoes difficulty-based filtering against the target model (Qwen-4B-Instruct) using 16 rollouts.

Scale. The final GooseReason-Cyber dataset contains approximately 180K RLVR tasks. The paper does not report the initial number of raw documents, the extraction yield, or the filtering yield, so the overall efficiency of the pipeline on noisy web data is unknown.

Training. Qwen-4B-Instruct is trained on GooseReason-Cyber for only 100 RL steps (compared to 200+ steps in the reasoning domain experiments). This short training duration is notable—it suggests that the cybersecurity tasks provide a concentrated learning signal, or that the model reaches the data saturation point quickly due to the narrower domain.

Verification. The verification mechanism is identical to the reasoning domain: extract the option label from the model's response and compare to the ground-truth. There is no cybersecurity-specific verifier, sandbox, or expert evaluation during RL training.

The cybersecurity experiment demonstrates a key property of the Golden Goose pipeline: domain transferability. The pipeline does not require domain-specific engineering, handcrafted verifiers, or expert annotation. It only requires: (a) a source of domain-relevant text that contains reasoning, (b) a sufficiently capable synthesis LLM (GPT-5), and (c) the target student model for difficulty filtering. This generality is the paper's core argument for why Golden Goose can extend to law, medicine, and other domains where "verifiable data is scarce but professional literature is abundant" (Section 5).

Summary of Design Choices and Their Justifications

Multiple-choice over open-ended FIM: open-ended fill-in-the-middle fails because heavy RL-tuned models ignore the infill instruction and generate full solutions (83% zero-accuracy rate for ProRL-1.5B-v2; Figure 4). MCQ constrains the output format to a discrete choice, making verification a simple option-index match.
Nine options (8 distractors) over fewer options: with 3 options, the model uses elimination strategies (spotting flaws in distractors) rather than reasoning forward from the masked context. Nine options force engagement with the masked content because exhaustively falsifying 8 distractors is infeasible within a limited generation budget (Figure 4).
GPT-5 as synthesis model: the strongest LLM available at the time of the experiment, chosen because distractor plausibility and correctness require high-quality assessment of what constitutes a reasonable-but-incorrect completion in context.
Difficulty filtering by rollout consistency: tasks are retained only if the student model produces both correct and incorrect answers across 16 rollouts, ensuring the task provides non-zero advantage in GRPO (where advantage collapses when all rollouts have the same outcome). Applied only to noisy sources; clean reasoning-dense sources (AoPS, rStar-Coder, MegaScience) produce effective tasks without filtering.
Passage extraction for noisy sources: raw web text is preprocessed by prompting the LLM to extract educationally valuable passages before masking, filtering out non-reasoning content at the source level.
No modification to RL algorithm: the ProRLv2 GRPO recipe is used as-is, isolating the contribution to data rather than algorithmic innovation. This means GooseReason is "pluggable" into any RLVR pipeline.
Ground-truth is the removed span t: this is the crux of the verification trick. By defining the masked span as the answer, verification requires no judge model, no math parser, and no code execution—just a string comparison between the model's selected option and the stored ground-truth index. The synthesis LLM does not solve the problem; it selects which part of the existing solution to mask.

4. Key Insights and Innovations

Innovation 1: The Format-Conversion Insight — Unverifiable Text Becomes Verifiable by Changing the Output Constraint, Not the Task Content

The paper's most fundamental intellectual move is deceptively simple, but its implications are far-reaching: the reason certain reasoning-rich text is unusable for RLVR is not that it lacks correct answers, but that verification cannot handle the variety of valid expressions for those answers. Prior work operated under an implicit assumption that if content could not be verified by simple string matching, math parsing, or unit test execution, it was categorically excluded from RLVR. This forced the field into two narrow strategies: curate problems where the answer format is constrained by design (math problems with numeric answers, coding problems with test cases) or build handcrafted verifiable environments that procedurally generate problems with constrained answers (logic puzzles, arithmetic, symbolic manipulation).

Golden Goose challenges this assumption at its root. The insight is that the verification bottleneck is a property of the output format, not the reasoning content. If you can convert the task from an open-ended generation problem (where many different strings can express the same correct answer) to a discrete-choice problem (where the answer is one of k+1 known candidates), you eliminate the verification problem entirely without diluting the reasoning requirement. The model still needs to understand the masked content, reason about the surrounding context, and evaluate which option is consistent — it just expresses its answer by selecting an option rather than generating free text.

This is fundamentally different from prior approaches to synthetic data generation. RLVE (Zeng et al., 2025a) and Reasoning Gym (Stojanovski et al., 2025) generate new tasks by parameterizing difficulty within a fixed rule system — they create new reasoning content by combining logical rules or math operators in new ways. Golden Goose takes existing reasoning content (proof steps, textbook explanations, code fragments) and changes the interface through which the model demonstrates understanding. The reasoning the model must perform is the same as it would be for an open-ended task — understanding what fills the gap — but the response format is constrained. This is a fundamental reframing of the data synthesis problem: don't try to generate reasoning from scratch (which requires domain-specific procedural knowledge), but instead extract it from existing sources and change the verification mechanism.

The significance of this reframing is best illustrated by what it unlocks. Proof problems from AoPS-Instruct, which require strategic lemma selection and formal manipulation, were previously inaccessible because verifying an entire proof is an open research problem. Golden Goose doesn't solve proof verification — it sidesteps it entirely by extracting individual reasoning steps from proofs and making those the target of the MCQ. A step of a proof — "apply the Cauchy-Schwarz inequality to bound this expression" — is a reasoning nugget that can be verified as a discrete choice (is this the right inequality to apply here?) even though the full proof is unverifiable. The paper's results on STEM (3.48% gain vs. 0.13% with original ProRL data, Figure 5) provide concrete evidence that these extracted reasoning steps transfer to downstream reasoning capability — the model isn't just learning to take MCQs, it's learning the reasoning patterns embedded in the expert solutions.

The format-conversion insight also explains why Golden Goose generalizes across domains without domain-specific engineering. Whether the source text is a math proof, a biology explanation, a code solution, or a cybersecurity analysis report, the pipeline's logic is identical: find a reasoning step, mask it, generate plausible distractors, check the option index. Verification never needs to understand the domain. This is in stark contrast to procedural generators, which require domain experts to encode the rules, constraints, and answer formats for each new environment. The cybersecurity experiment (GooseReason-Cyber, 180K tasks, 4.44% absolute gain in 100 RL steps) is the paper's strongest evidence for this generality claim — no one handcrafted cybersecurity verification environments, yet the pipeline produced effective RLVR data from raw web scrapes.

A subtle but important aspect of this insight is that it decouples reasoning difficulty from verification difficulty. In prior RLVR data, harder reasoning tasks naturally required more complex verification (harder math requires more sophisticated equality checking, harder code requires more comprehensive test suites). Golden Goose breaks this coupling: the reasoning can be arbitrarily complex (Olympiad-level proof steps), but the verification remains trivial (option index matching). This means the data scaling problem is no longer bounded by the difficulty of building verifiers for increasingly complex answer formats — the ceiling is now the availability of reasoning-rich source text and the quality of LLM-based distractor generation.

Innovation 2: Diagnosing "Data Saturation" as a Concrete, Quantifiable Phenomenon — Not Just Plateauing Performance

Prior work on scaling RLVR (ProRL, BroRL, ScaleRL) observed that training eventually plateaus, but treated this as a generic optimization phenomenon — the model runs out of "room to improve" or the learning rate schedule isn't tuned for extended training. The key diagnostic move in this paper is to operationalize data saturation as a measurable property of the training data itself: a problem is "effective" (provides learning signal) if and only if the model produces both correct and incorrect rollouts on it, because GRPO-style advantage estimation requires within-group variance to produce non-zero updates.

This sounds like a minor technical observation, but it represents a fundamental shift in how to think about RLVR scaling bottlenecks. Before this paper, the dominant framework for understanding RLVR limits was algorithmic: the model overfits, the advantage estimates become noisy, the KL divergence from the reference policy grows too large, or the reward signal is too sparse. Solutions focused on algorithmic modifications — decoupled advantage normalization (REINFORCE++), clipped objectives (GRPO), increased rollout budgets (BroRL). The paper does not dispute that these algorithmic improvements matter — it uses them — but it identifies a data-centric ceiling that algorithmic improvements alone cannot surpass: if 75% of your training examples produce identical outcomes across all rollouts (Figure 3), no amount of advantage normalization can extract learning signal from them, because the signal literally does not exist.

The quantification in Figure 3 is the paper's strongest empirical evidence for this diagnostic. Of the 136K problems in the ProRL data blend, only ~25% (~34K) remain effective for ProRL-1.5B-v2. This is not a gradual decline — it is a collapse. The remaining 75% are "stale": the model either always gets them right (no room to improve) or always gets them wrong (no successful trajectories to reinforce). The paper's 13× figure — GooseReason provides 13× more effective examples than the ProRL blend — reframes the data scaling problem from "we need more problems" to "we need more problems that the current model finds appropriately challenging." This is analogous to the concept of "zone of proximal development" in educational psychology: learning happens not from tasks the student has already mastered or cannot begin, but from tasks at the edge of their current capability.

The observation that saturation occurs earlier and is more severe with stronger models is a critical extension of this diagnostic with significant implications for the field. Qwen-4B-Instruct, a stronger base model than R1-Distill-Qwen-1.5B, saturates on the same ProRL data blend after only 300 steps — and actually degrades in math (-1.29%) and STEM (-1.52%) with further training (Table 3, Figure 2). The mechanism is straightforward: a stronger model has higher pass@1 on the training problems, so it achieves consistent correctness on more of them faster, and the ones it still gets wrong are harder, so it may never achieve correct rollouts on them. The effective problem set shrinks from both ends. This means the RLVR data bottleneck is not a fixed ceiling but one that lowers as models improve — the better the base model, the more acute the need for fresh, appropriately-difficult data. This is a sobering finding for anyone hoping that better pretraining will make RLVR scaling easier; it suggests the opposite.

This diagnostic reframes the field's approach to RLVR scaling from "how do we train longer without collapsing?" to "how do we continually supply new problems at the model's current frontier of capability?" The answer Golden Goose provides — mine unverifiable text for reasoning steps and convert them to MCQs — is one solution, but the diagnostic itself is likely the paper's most enduring contribution, as it provides a measurable criterion (effective examples) for evaluating any future data synthesis method.

Innovation 3: The "Effective Example" Metric as a Principled, Algorithm-Agnostic Measure of Data Quality for RLVR

The paper introduces "effective examples" — the number or fraction of training tasks on which a given model produces both successful and failed rollouts — as a quantitative metric for assessing RLVR data quality. This is a conceptual innovation with practical implications that extend beyond Golden Goose.

Before this paper, RLVR data quality was assessed primarily through provenance (is this problem from a trusted human-authored source?), diversity (does the dataset cover multiple domains?), or downstream performance (does training on this data improve benchmark scores?). These are all valid signals, but they are indirect and model-agnostic — they don't capture whether a specific problem is actually providing learning signal for a specific model at a specific point in training. Two problems of identical provenance and difficulty can have fundamentally different effectiveness: one may be at the model's learning frontier, while the other may be already mastered or hopelessly out of reach.

The effective example metric fixes this by defining data quality relationally — as a function of the model-data interaction, not as an intrinsic property of the data. This is significant because it creates a feedback loop between model capability and data curation: as the model improves during training, the set of effective examples shifts, and data that was previously effective becomes stale. The paper doesn't implement dynamic data refreshing (it pre-computes effectiveness relative to a specific checkpoint), but the metric enables it — one could imagine periodically re-evaluating the training set and dropping or replacing stale examples during training.

The 13× comparison in Figure 3 is the paper's headline use of this metric, but the metric's broader significance is that it provides a unified framework for comparing data synthesis methods. The paper uses it to compare GooseReason-0.7M (70% effective) against the ProRL data blend (25% effective) relative to ProRL-1.5B-v2, but the same metric could compare RLVE-generated data, human-curated data, or any other RLVR data source against any model checkpoint. It captures the key variable — how much learning signal per example — without being tied to any specific algorithm or training recipe.

A limitation worth noting: the effective example metric depends on the rollout count (16 in the paper) and the specific RL algorithm (GRPO-style advantage normalization). A problem that is "stale" under 16-rollout GRPO might still provide signal with more rollouts (some variance might emerge at higher sample counts) or under different advantage estimation (e.g., outcome-based baselines rather than group normalization). The paper does not explore these interactions, but the metric's dependence on experimental parameters does not invalidate it — it just means the metric should be reported with its parameters clearly specified, which the paper does.

Innovation 4: Eliminating the Verified-Domain Constraint — RLVR Can Now Be Deployed in Knowledge-Intensive Specialized Domains Without Handcrafted Verifiers

The cybersecurity experiment (Section 3.2, Table 4) is more than a domain-transfer demonstration. It establishes that RLVR can be applied to specialized professional domains that have no existing RLVR data, no handcrafted verifiers, and no simple answer formats, using only publicly available text. This changes what RLVR is capable of as a training paradigm.

Prior to Golden Goose, the set of domains where RLVR could be applied was essentially the set of domains where automated verification was straightforward: math (numeric answer checking), code (unit test execution), and formal logic (rule-based answer validation). Efforts to extend RLVR to new domains required building domain-specific verifiers — a labor-intensive process that itself required domain expertise. The cybersecurity domain is a perfect case study: cybersecurity questions often involve open-ended analysis ("analyze this network log for intrusion patterns"), threat assessment ("evaluate the severity of this vulnerability"), or multi-step procedural reasoning ("design a defense against this attack vector"). None of these have clean answer formats. Building a verifier — perhaps an LLM judge trained on expert evaluations — would be a substantial research project in its own right.

Golden Goose sidesteps this entirely. The 180K GooseReason-Cyber tasks are synthesized from raw web scrapes with no cybersecurity-specific engineering. The 4.44% absolute gain across 3 cybersecurity benchmarks after only 100 RL steps (Table 4) is striking not just for its magnitude — it surpasses Llama-Primus-Instruct, a 7B model with extensive domain-specific pretraining and post-training — but for what it implies about the minimum viable infrastructure for RLVR in a new domain. You need: (1) text from the domain that contains reasoning (web scrapes suffice), (2) a capable synthesis LLM (GPT-5), and (3) a base model to train. No domain experts, no verification environments, no curated problem sets.

This has direct implications for the paper's envisioned extension to law and medicine (Section 5). Legal reasoning (case analysis, statutory interpretation) and medical reasoning (differential diagnosis, treatment planning) share the same structural properties that made cybersecurity amenable to Golden Goose: abundant professional literature containing sequential reasoning chains, scarce verifiable problem sets, and answer formats that resist simple automated validation. The pipeline would operate identically: extract reasoning passages from legal opinions or medical case reports, mask key inferential steps, generate distractors, and train. The primary risk — which the paper does not discuss — is that distractor generation quality may degrade in domains where the synthesis LLM has weaker capabilities. GPT-5 may generate plausible-but-incorrect legal distractors less reliably than mathematical ones, and bad distractors (those that are trivially identifiable as wrong) would produce tasks that are too easy to provide effective learning signal. The paper's difficulty filtering provides a partial safeguard (trivially easy tasks are filtered out), but it cannot create effective tasks where the synthesis LLM fails to produce adequate distractors in the first place.

An important nuance: the cybersecurity gain (4.44%) appears to be computed as an absolute improvement over the base Qwen-4B-Instruct model, but the paper's Table 4 reporting is somewhat ambiguous about the baseline. If the comparison is against the same model without GooseReason-Cyber training, the 4.44% absolute gain in 100 steps is indeed substantial relative to the previous SOTA's 1.44% gain over its base model (Llama-3.1-8B-Instruct). However, the paper does not report whether Qwen-4B-Instruct's base performance on cybersecurity benchmarks matches or differs from Llama-3.1-8B-Instruct's, which complicates the comparison. Even with this ambiguity, the key contribution is not the specific performance number but the existence proof: RLVR can work in a domain with zero prior RLVR infrastructure, using only publicly available text.

The cybersecurity result also implies a hierarchy of domain accessibility for RLVR. Domains with abundant, structured, reasoning-dense text (textbooks, academic papers, professional guidelines) are more amenable to the Golden Goose pipeline than domains where reasoning is tacit, embedded in non-text modalities, or expressed in formats that LLMs struggle to parse (legal contracts with complex cross-references, medical images with radiology reports). The paper does not analyze this hierarchy, but it is a natural extension of the work.

Innovation 5: The MCQ-as-Training-Proxy Finding — Task-Format Restriction Doesn't Prevent Generalization to Open-Ended Evaluation

A persistent concern with synthetic MCQ training data is that the model will learn to solve MCQs rather than learning the underlying reasoning — it will become a test-taking specialist that exploits elimination strategies, option-length heuristics, or other surface patterns, and this skill will not transfer to open-ended evaluation. The paper provides systematic evidence against this concern, and the structure of that evidence reveals something important about how RLVR operates.

The paper's evaluation benchmarks are overwhelmingly non-MCQ (Section 3.1). Math evaluation uses AIME (15-question, 3-hour, integer-answer exam), AMC (multiple-choice but with a different format and difficulty profile), MATH (open-ended problem-solving), Minerva (quantitative reasoning), and Olympiad Bench (Olympiad-level problems, many open-ended). Coding evaluation uses real execution-based benchmarks (APPS, CodeContests, HumanEvalPlus, LiveCodeBench). STEM evaluation uses GPQA Diamond (multiple-choice graduate-level science, but with questions designed to be "Google-proof" — not answerable by simple retrieval). None of these are the 9-option fill-in-the-middle MCQ format used in training.

Yet training on GooseReason data produces consistent gains across all these benchmarks (Tables 1–3, Figures 8–14). The gains are largest in STEM (3.48% absolute for ProRL-1.5B-v2 continued training, Figure 5), where the training data (derived from MegaScience textbook explanations) is furthest in format from the evaluation (GPQA Diamond multiple-choice). This is the paper's strongest evidence that the model is learning the reasoning content embedded in the source passages, not the surface patterns of the MCQ format. If the model were learning MCQ-specific heuristics, the gains would be largest on MCQ evaluation benchmarks and diminish or vanish on open-ended ones. The opposite pattern — largest gains in STEM, where reasoning transfer is most demanding — supports the interpretation that the MCQ format is an effective delivery mechanism for reasoning content, not a substitute for it.

This finding has methodological significance beyond this paper. It suggests that the RLVR training signal does not require the training task format to match the evaluation task format, as long as the reasoning content is transferable. This is consistent with prior work showing that models trained on code can improve at math (and vice versa), but extends it to a more extreme case: training exclusively on fill-in-the-middle MCQ tasks transfers to open-ended generation, execution-based code evaluation, and knowledge-intensive science questions. The mechanism is likely that the policy-gradient update reinforces the reasoning process that leads to the correct option selection — the chain-of-thought that evaluates each candidate against the masked context — rather than reinforcing option-selection heuristics. The model generates reasoning traces during training (the GRPO rollout includes the full chain-of-thought), and the reward signal shapes those traces toward patterns that are correct in general, not merely correct for MCQ selection.

A nuance: the transfer is not perfect. The paper does not compare GooseReason-trained models against models trained on an equivalent amount of open-ended RLVR data (because such data doesn't exist for these domains — that's the whole point). It's possible that open-ended RLVR training on the same reasoning content would produce even larger gains, and the MCQ format imposes some transfer penalty. But the paper's claim is not that MCQ training is optimal — it is that MCQ training works at all, and works well enough to unlock domains where open-ended RLVR is infeasible. This is a pragmatic, not a theoretical, contribution: it lowers the barrier for applying RLVR to new domains from "build a domain-specific verifier" to "find domain-specific text and run the Golden Goose pipeline."

5. Experimental Analysis

Evaluation Methodology

Dataset. The primary training data is the ProRL data blend (136K RLVR tasks spanning mathematics, coding, logical reasoning, STEM, and instruction-following), supplemented by GooseReason-0.7M (over 0.7M synthetic MCQ tasks from AoPS-Instruct, rStar-Coder, and MegaScience) and GooseReason-Cyber (180K tasks from FineWeb cybersecurity scrapes). The paper does not explicitly name or size a held-out validation set used for model selection during RL training; effectiveness filtering uses 16 rollouts per task against the student model checkpoint.
Base model(s). Two model families are used: ProRL-1.5B-v2, a heavily RL-trained model derived from R1-Distill-Qwen-1.5B (Guo et al., 2025a) with over 20,000 H100 GPU hours of prior ProRL training that has reached performance saturation (Hu et al., 2025b, 2025c); and Qwen-4B-Instruct (Team, 2025), a stronger base instruct model used to demonstrate that saturation occurs earlier and is more severe with increased capability. For the cybersecurity domain, the base model is also Qwen-4B-Instruct. The 1.5B model tests whether GooseReason revives an already-saturated checkpoint; the 4B model tests whether GooseReason prevents early saturation and enables compute-efficient scaling from scratch.
Metrics. All results report pass@1 accuracy (%) — the fraction of evaluation problems for which a single model generation (at the standard evaluation temperature and sampling parameters) produces the correct answer. For math benchmarks (AIME 2024/2025, AMC, MATH, Minerva, Olympiad Bench), correctness is determined by math answer matching. For coding benchmarks (APPS, CodeContests, CodeForces, TACO within the PRIME validation set; HumanEvalPlus; LiveCodeBench), correctness is determined by test case execution. For STEM (GPQA Diamond), correctness is determined by multiple-choice answer matching. For instruction following (IFEval), correctness is determined by programmatic constraint checking. For logical reasoning (Reasoning Gym), correctness is determined by the individual gym environments' verifiers. The paper uses non-MCQ evaluation benchmarks almost exclusively, which is a deliberate design choice to assess whether MCQ-format training transfers to open-ended reasoning.
Baselines. The paper compares against several baselines: (1) Continued RL with ProRL data only — training the saturated model further using the original 136K ProRL data blend under the identical ProRLv2 recipe, which represents the status quo for RLVR scaling; (2) RLVE (Zeng et al., 2025a) — a procedural data generation method that produces adaptive-difficulty RLVR tasks from handcrafted verifiable environments, using their publicly released checkpoint trained under an equivalent computational budget; (3) Llama-Primus-Instruct (Yu et al., 2025) — an 8B domain-specialized cybersecurity model built on Llama-3.1-8B-Instruct with extensive domain-specific pre-training and post-training, serving as the previous state-of-the-art for cybersecurity; (4) Qwen3-30B-Instruct — a 7.5× larger model included as a reference point for the 4B-scale results, though not a direct baseline since it differs in model family, scale, and training procedure.
Generation budget / compute accounting. Compute is measured in H100 GPU hours for the continued-training experiments (1,100 additional hours for ProRL-1.5B-v2 experiments) and RL training steps for the compute-efficient scaling experiments (200 steps from scratch, 100 steps for cybersecurity). The paper does not report the per-step computational cost or the total FLOPs for any experiment, making it impossible to compare the computational efficiency of different training runs on a standard FLOPs basis. The 1,100 H100 GPU hour budget applies to all three conditions in Figure 5 (ProRL data only, ProRL + GooseReason, RLVE), enabling a fair comparison across data strategies. For the GooseReason-0.7M synthesis itself, the paper does not report the computational cost of running GPT-5 over the source corpora to generate 0.7M tasks, which could be substantial given the size of the source corpora (600K AoPS QA pairs, 1.3M rStar-Coder code problems, 650K MegaScience QA pairs). This is a significant omission in the compute accounting.
Cross-validation / statistical protocol. The paper does not describe a cross-validation or statistical significance testing protocol. Results appear to be reported as single-run pass@1 scores on held-out benchmark test sets. The effectiveness filtering (16 rollouts per task) is performed once against a specific model checkpoint (ProRL-1.5B-v2 for the 1.5B experiments, Qwen-4B-Instruct for the 4B and cybersecurity experiments). There is no mention of multiple random seeds, confidence intervals, or significance tests for any of the reported gains. For the continued-training experiments, it is unclear whether the training curves represent a single training run or an average over multiple runs, which matters because RL training can exhibit high variance across seeds. The paper also does not report whether the benchmark evaluations use a single generation per problem or multiple generations with majority voting — the pass@1 metric implies single-generation evaluation, but the specific sampling parameters (temperature, top-p) are not provided in the main text.

Main Quantitative Results

The experimental section investigates two primary axes: scaling beyond data saturation (reviving models that have plateaued on existing RLVR data) and compute-efficient scaling from scratch (whether GooseReason accelerates training when budget is limited). A third axis, the cybersecurity domain deployment, validates the method's generality in a real-world setting with no prior RLVR infrastructure.

4.1 Scaling Beyond Data Saturation on ProRL-1.5B-v2

Headline result. Continuing RL training on the saturated ProRL-1.5B-v2 model for an additional 1,100 H100 GPU hours with ProRL data alone yields marginal aggregate improvements of 0.63% in math, 0.95% in coding, and 0.13% in STEM. Adding GooseReason-0.7M to the training data produces gains of 2.71%, 2.12%, and 3.48% in the same domains (Figure 5). The STEM gain is the most dramatic: GooseReason provides a 27× larger improvement over the ProRL-data-only baseline in STEM (3.48% vs. 0.13%). These numbers are reported as aggregate domain-level averages across the constituent benchmarks.

Domain-level breakdown. Figure 5 plots aggregate performance over training steps for math, coding, and STEM domains. The ProRL-only curve (blue) is essentially flat across all three domains, confirming that the model has exhausted the learning signal from the original data. The GooseReason curve (green) shows continuous upward trends in math and coding, and an especially steep improvement in STEM. The RLVE curve (orange, representing Zeng et al., 2025a's publicly released checkpoint at equivalent budget) is highly effective on math (tracking close to GooseReason), but its STEM impact is limited to a 0.62% gain — consistent with the paper's claim that procedural generation struggles to encode knowledge-intensive reasoning.

Per-benchmark decomposition. Appendix B provides per-benchmark breakdowns. Figures 11 (math benchmarks) and 12 (coding benchmarks) show that on individual math benchmarks, RLVE and GooseReason are competitive on several (AIME, MATH, Minerva), but GooseReason pulls ahead on Olympiad Bench. On coding benchmarks, GooseReason consistently outperforms RLVE across APPS, CodeContests, LiveCodeBench, and HumanEvalPlus. Figure 12 shows that on HumanEvalPlus, RLVE actually underperforms the ProRL-only baseline, while GooseReason provides a clear positive gain. The paper does not report exact per-benchmark numbers in the main text, so precise figures must be read from the appendix plots.

Evidence for data freshness, not just data volume. The paper's core argument is that GooseReason-0.7M works because it provides fresh examples — tasks the model hasn't seen and finds appropriately challenging — not merely because it provides more examples. Figure 3 supports this: GooseReason-0.7M retains a ~70% effectiveness ratio against ProRL-1.5B-v2, compared to ~25% for the original ProRL data. The 13× increase in effective examples (over 450K vs. ~34K) quantifies the freshness premium. This measurement is a snapshot taken against the ProRL-1.5B-v2 checkpoint before the continued training begins, so it captures the state at the start of the additional 1,100 GPU hours. The paper does not re-measure effectiveness during or after the continued training, so it's unclear whether GooseReason tasks also become stale as training progresses or whether the 70% ratio is sustained.

4.2 Scaling Beyond Data Saturation on Qwen-4B-Instruct

Headline result. The Qwen-4B-Instruct experiments reveal a critical pattern: data saturation is not just a problem for models that have been trained for thousands of steps — it occurs much earlier for stronger base models, and its consequences are more severe (performance degradation rather than mere plateau). Training Qwen-4B-Instruct with ProRL data alone for 300 steps shows performance plateau or degradation: -1.29% in math, +0.43% in coding, and -1.52% in STEM (Figure 2, Table 3). Adding GooseReason-0.7M reverses this trend: +2.18%, +2.24%, and +2.40% in the same domains. The net swing is substantial — roughly 3.5 percentage points in math (from -1.29% to +2.18%), 2 percentage points in coding, and 4 percentage points in STEM. The aggregate absolute improvement across all benchmarks from adding GooseReason is 2.27% (compared to a 0.79% degradation with ProRL data alone).

Training trajectory. Figure 2 shows the per-step accuracy trajectory over 500+ RL steps. The ProRL-only curve rises initially but plateaus around step 300 and begins to decline, particularly in math and STEM. The GooseReason curve continues to trend upward through at least step 500, with no sign of plateauing. This is the paper's strongest evidence that GooseReason enables continuous scaling rather than a one-time boost — the gains are sustained over training steps, not front-loaded from adding new data.

Per-benchmark results (Tables 1–3, Appendix Figures 8–10). Tables 1, 2, and 3 provide the main structured comparison. Table 1 (math benchmarks) shows that GooseReason-4B-Instruct achieves improvements over the base Qwen-4B-Instruct across all six math benchmarks: AIME 2024, AIME 2025, AMC, MATH, Minerva, and Olympiad Bench. The gains are largest on AIME (the hardest benchmark), consistent with the mechanism that the AoPS-derived tasks (Olympiad-level math) transfer to competition-level evaluation. Table 2 (coding benchmarks) shows similar broad improvements across APPS, CodeContests, LiveCodeBench, HumanEvalPlus, and the PRIME validation subsets. Table 3 (STEM, instruction-following, logic) shows the cross-domain transfer: GPQA Diamond (STEM) improves, IFEval (instruction following) improves, and Reasoning Gym (logical puzzles, games, algorithmic tasks, induction) improves across all four sub-categories despite GooseReason containing no explicit logic puzzle training data. The paper highlights this as evidence for "reasoning generalization" — skills acquired from the MCQ fill-in-the-middle tasks transfer to structurally different reasoning formats.

Comparison to 30B model. Table 3 includes Qwen3-30B-Instruct results as a gray reference. GooseReason-4B-Instruct achieves "comparable or even better performance across the board" against a model with 7.5× more parameters, according to the paper. Exact per-benchmark comparisons require reading the table values, which are not transcribed in the provided content. This claim should be interpreted cautiously: the 30B model is from a different family (Qwen3 vs. Qwen-4B-Instruct), may have different pretraining data, and is evaluated at a different scale — the comparison establishes that GooseReason-trained 4B models are competitive, not that they strictly dominate.

Task-level scaling behavior (Figure 7). Figure 7 categorizes the effect of GooseReason on Qwen-4B-Instruct continued training across individual tasks: "diverge" (ProRL-only regresses while GooseReason improves), "outpace" (both improve but GooseReason improves faster), and "align" (similar trends). The paper reports that STEM and most math tasks fall into the diverge category — GooseReason is not merely accelerating learning but preventing the degradation that occurs with ProRL-only training. Coding tasks primarily outpace — both data sources produce gains, but GooseReason produces faster ones. A few tasks align. This categorization provides nuance beyond aggregate numbers: GooseReason's benefit is largest where the original data is actively harmful (diverge tasks), substantial where it's merely insufficient (outpace tasks), and minimal where the original data is already adequate (align tasks). The paper does not provide a formal definition of the thresholds for these categories, but the qualitative pattern is visible in the per-task plots.

4.3 Compute-Efficient Scaling from Scratch

Headline result. When training Qwen-4B-Instruct from scratch for only 200 RL steps, joint training with GooseReason-0.7M consistently achieves higher performance at the same number of steps compared to training with ProRL data alone (Figure 6). This demonstrates that GooseReason provides benefits even before saturation occurs — it accelerates the learning rate in the early training regime.

Step-by-step efficiency. Figure 6 plots accuracy against training steps for math and coding aggregates. The GooseReason curves are consistently above the ProRL-only curves from the earliest steps, and the gap widens over the 200-step training horizon. The paper does not report the exact accuracy values at step 200, so precise numbers must be read from the figure. Appendix Figures 13 (math benchmarks) and 14 (coding benchmarks) provide the per-benchmark breakdown, showing that the GooseReason advantage holds across individual benchmarks.

Interpretation. This result is important because it counters a potential criticism: that GooseReason only helps when models have already saturated on existing data, making it useful only for niche prolonged-training scenarios. The compute-efficient scaling experiment shows that even when starting from a fresh instruct model with no prior RL, incorporating GooseReason tasks into the training mix produces better results per unit of compute. This expands the applicability of the method beyond the saturation-revival use case to general RLVR training.

Caveat. The paper does not ablate the ratio of GooseReason to ProRL data in this experiment. The joint training condition uses both datasets, and it's unclear whether the gain comes from the GooseReason data itself or merely from the increased total data volume (ProRL 136K + GooseReason 0.7M vs. ProRL 136K alone). A volume-matched baseline — training with ProRL data only but at a larger effective batch size or with data repetition to match the total number of unique tasks — would isolate whether the benefit is freshness or volume. The paper does not run this control.

4.4 Cybersecurity Domain Deployment

Headline result. Training Qwen-4B-Instruct on GooseReason-Cyber (180K synthetic MCQ tasks from FineWeb cybersecurity scrapes) for only 100 RL steps yields a 4.44% absolute gain averaged across three cybersecurity benchmarks: CTI-Bench (threat intelligence), CyberMetric (domain knowledge), and SecEval (software/network security) — Table 4. This surpasses Llama-Primus-Instruct, an 8B domain-specialized model built on Llama-3.1-8B-Instruct with extensive domain-specific pre-training and post-training, which achieved an average gain of only 1.44% over its base model.

Benchmark composition. CTI-Bench (Alam et al., 2024) assesses threat-intelligence reasoning and vulnerability analysis. CyberMetric (Tihanyi et al., 2024) tests knowledge in domains like compliance and penetration testing via retrieval-augmented generation. SecEval (Busch et al., 2014) evaluates proficiency across foundational areas such as software and network security. The paper reports the aggregate 4.44% gain across these three benchmarks but does not provide per-benchmark breakdowns, making it impossible to assess whether the gains are uniform or concentrated in specific sub-domains.

Comparison to Primus. The Primus model (Yu et al., 2025) used domain-specific pretraining on Primus-Seed (curated cybersecurity sources) and Primus-FineWeb (filtered web data), followed by domain-specific post-training. This is a substantially more resource-intensive pipeline than 100 RL steps on synthetic data. The 4.44% vs. 1.44% comparison is not perfectly controlled — different base models (Qwen-4B-Instruct vs. Llama-3.1-8B-Instruct), different model sizes (4B vs. 8B), different training paradigms (RLVR on synthetic MCQs vs. domain-adaptive pretraining + SFT). The paper's claim of "establishing a new state-of-the-art for cybersecurity LLMs" should be interpreted as outperforming the previous best published result, not as a controlled ablation showing RLVR superiority over pretraining.

Gain magnitude interpretation. A 4.44% absolute gain in 100 RL steps is substantial for a specialized domain where no prior RLVR data existed, but the paper does not provide information about the baseline accuracy of Qwen-4B-Instruct on these benchmarks before GooseReason-Cyber training. If the baseline is low (e.g., 20%), a 4.44% gain represents a large relative improvement. If the baseline is high (e.g., 80%), the gain represents ceiling effects being approached. The per-benchmark breakdown and baseline accuracies are missing from Table 4 as presented in the provided content.

Significance for domain transfer. The cybersecurity result is the paper's strongest evidence that Golden Goose works on genuinely noisy, real-world source text (not curated educational corpora) and in a domain with zero prior RLVR infrastructure. However, the result is from a single domain with a single base model and a single training duration (100 steps). Generalization to law, medicine, or other domains requires assuming that the FineWeb cybersecurity scrapes are representative of noisy domain text in general — an assumption the paper does not empirically validate.

Ablation Studies and Robustness Checks

The paper's ablation studies are concentrated in Section 2.3 ("Design Choice") and the cybersecurity difficulty filtering discussion in Section 2.1. The primary ablations address the multiple-choice format, the number of distractors, and the necessity of difficulty filtering.

Multiple-choice vs. open-ended FIM (Figure 4). The paper compares the standard Golden Goose MCQ formulation against an open-ended fill-in-the-middle variant where the model freely generates the masked content and an LLM-as-judge verifies correctness. Using GooseReason-Math (AoPS-derived tasks) evaluated on ProRL-1.5B-v2 with 16 rollouts per task: over 83% of open-ended tasks result in consistent zero accuracy — the model ignores the infill instruction and generates full solutions from scratch. In contrast, the 9-option MCQ format distributes accuracy broadly, with over 70% of tasks falling into the medium-difficulty regime (both correct and incorrect rollouts). This ablation is the paper's primary justification for the MCQ design choice: open-ended FIM fails not because the reasoning is too hard, but because RL-tuned models have been trained to generate complete solutions and cannot reliably follow the constrained output format. The Figure 4 plots show accuracy distributions as histograms for 3-option, 5-option, 7-option, 9-option MCQ, and open-ended formats. The open-ended distribution is a sharp spike at 0% accuracy. As the number of options increases from 3 to 9, the distribution shifts from right-skewed (mostly high accuracy, easy tasks) to centered (medium accuracy, effective tasks). The paper does not test whether few-shot prompting or format-specific fine-tuning could salvage the open-ended variant — it's possible that the instruction-following failure is fixable, but the paper's position is that the MCQ format solves it without additional training.
Number of distractors (Figure 4). The number of options (k + 1, where k is the number of distractors) is swept from 3 to 9 on the GooseReason-Math subset, with accuracy distributions computed over 16 rollouts on ProRL-1.5B-v2. With 3 options, the majority of problems fall into a high-accuracy regime — the model solves them easily, likely through elimination strategies (identifying which of the two distractors is wrong) rather than forward reasoning about the masked content. With 9 options, the accuracy distribution centers around intermediate values, with over 70% of problems in the medium-difficulty zone. The paper argues that with 9 options, the elimination strategy becomes infeasible under a fixed generation budget (the model cannot exhaustively analyze 8 distractors), forcing engagement with the masked content. The optimal number of distractors is likely model-dependent; stronger models might require even more distractors to prevent elimination, while weaker models might be overwhelmed by too many options. The paper only tests up to 9 options and only on GooseReason-Math with ProRL-1.5B-v2, so the generalizability of this hyperparameter is uncertain.
Difficulty-based filtering for noisy sources (Section 2.1, implicitly in Figure 3). For reasoning-dense source corpora (AoPS-Instruct, rStar-Coder, MegaScience), the paper states that GPT-5-synthesized questions were "of sufficient quality and difficulty to require no further post-processing." For noisy sources (FineWeb cybersecurity scrapes), difficulty filtering is applied: each task is rolled out 16 times with the target student model, and tasks with consistent 0% or 100% accuracy are discarded. The paper does not report the filtering yield — what fraction of initially synthesized cybersecurity tasks are retained or discarded — which makes it impossible to assess the efficiency of the pipeline on noisy data. Figure 3 shows that GooseReason-0.7M (without filtering) achieves ~70% effectiveness against ProRL-1.5B-v2; the cybersecurity filtering would presumably achieve a similar or higher effectiveness ratio for Qwen-4B-Instruct, but this is not measured.
RLVE comparison as an implicit ablation of data source type (Figure 5, Appendix Figures 11–12). The RLVE (Zeng et al., 2025a) comparison is not presented as an ablation but effectively serves as one: RLVE represents procedurally generated data from handcrafted rule-based environments, while GooseReason represents extracted data from human-authored reasoning text. The differential performance — RLVE is strong on math but weak on STEM (0.62% gain vs. GooseReason's 3.48%) — is the paper's evidence that the type of source data matters. Procedural generation can produce unlimited algorithmic reasoning tasks but cannot encode the factual and conceptual knowledge required for science reasoning. This is a qualitative claim rather than a controlled ablation (RLVE and GooseReason differ in many dimensions beyond data source type), but the performance pattern is consistent with the paper's argument.
No ablation of synthesis model strength. The paper uses GPT-5, "the strongest LLM available at the time of the experiment," for all synthesis. It does not test whether weaker models (e.g., GPT-4, Claude, open-source models) could produce distractors of sufficient quality. This is a significant gap — the Golden Goose pipeline's scalability depends on the assumption that a sufficiently capable synthesis LLM is available, and the paper provides no evidence about the minimum required capability. If only frontier models can produce challenging distractors, the pipeline is gated by API access and cost.
No ablation of source corpus quality within a domain. The reasoning domain experiments use three carefully curated corpora (AoPS-Instruct, rStar-Coder, MegaScience) while the cybersecurity experiment uses raw web scrapes with explicit extraction and filtering. The paper does not test what happens when the same domain is targeted with both clean and noisy sources — for example, whether math RLVR tasks synthesized from raw math-related web scrapes (without curated QA pairs) would be comparably effective to those from AoPS-Instruct. This makes it difficult to assess how dependent the 0.7M dataset is on the quality of the specific source corpora chosen.
No ablation of the passage extraction step for noisy sources. The cybersecurity pipeline includes an explicit extraction/summarization step before masking, but the paper does not ablate whether this step is necessary — i.e., whether masking raw FineWeb passages (without summarization) would produce useful tasks. This step introduces a potential quality bottleneck (if the extraction LLM fails to identify good reasoning passages) and a computational cost.
No ablation of the RL algorithm. All experiments use the ProRLv2 GRPO recipe without modification. The paper does not test whether GooseReason data works with other RLVR algorithms (e.g., standard GRPO, PPO, REINFORCE), which would establish whether the data contribution is algorithm-agnostic or specific to the ProRLv2 recipe. The paper claims GooseReason is "compatible with any RL algorithm applicable to RLVR," but provides no evidence for this claim beyond the ProRLv2 results.

Critical Assessment

Does GooseReason-0.7M "revive saturated models" as claimed? The evidence for this claim is strong but circumscribed. Figure 5 clearly shows that continuing RL on ProRL-1.5B-v2 with GooseReason-0.7M produces robust gains (2.71% math, 2.12% coding, 3.48% STEM) while continuing with ProRL data alone produces near-zero improvement. Figure 2 shows the same pattern for Qwen-4B-Instruct, with the additional finding that ProRL-only training causes degradation (-0.79% aggregate) while GooseReason reverses this (+2.27% aggregate). The "revival" claim is directly supported. However, the paper does not demonstrate that the revival continues indefinitely — the training curves in Figures 2 and 5 extend to 500+ and 1,100+ GPU hours respectively and show ongoing improvement, but it is unknown whether GooseReason data itself would eventually saturate. The 70% effectiveness ratio in Figure 3 is measured at the start of continued training; if GooseReason tasks also become stale after additional training (as the model masters them), the revival would be temporary. The paper does not measure effectiveness at the end of the continued training runs.

Does GooseReason "yield robust, sustained gains under continuous RL"? The "robust" claim is supported across two model scales (1.5B, 4B) and two training regimes (continued training from saturation, training from scratch). The "sustained" claim is supported by the upward-trending training curves, but the maximum training duration tested is 500+ steps (for Qwen-4B) and 1,100 H100 GPU hours (for ProRL-1.5B-v2). Whether gains would sustain beyond these horizons is untested. The degradation observed with ProRL-only training on Qwen-4B-Instruct (Figure 2) suggests that GooseReason's ability to prevent degradation may be as important as its ability to produce gains — but this is a claim about relative benefit rather than absolute scaling.

Does GooseReason achieve "new state-of-the-art results for 1.5B and 4B-Instruct models across 15 diverse benchmarks"? The claim is stated in the abstract and supported by Tables 1–3 for the 4B model. However, the "state-of-the-art" claim requires careful qualification. The paper compares against a specific set of baselines: the base Qwen-4B-Instruct, the same model trained with ProRL data only, and (for reference) Qwen3-30B-Instruct. It does not compare against other 4B-scale models that may have been trained with different RLVR recipes, different data blends, or different base models. The claim of "state-of-the-art among 4B-Instruct models" is plausible given the strong results, but the paper does not systematically survey all published 4B-scale results to establish this. The 1.5B results (ProRL-1.5B-v2 + GooseReason) are not presented in a comparable table format and are primarily shown as aggregate domain-level trends in Figure 5, making it harder to verify the "state-of-the-art" claim at that scale.

Does the cybersecurity result "establish a new state-of-the-art" and demonstrate that Golden Goose "can substitute for scarce handcrafted verifiable data"? Table 4 supports the SOTA claim relative to Llama-Primus-Instruct, but the comparison has significant confounds (different base models, different model sizes, different training paradigms). The substitution claim is supported in the specific sense that GooseReason-Cyber produced effective RLVR data for a domain where none previously existed, and the resulting model outperforms the previous best domain-specialized model. However, the paper does not demonstrate that GooseReason-Cyber is as effective as hypothetical handcrafted cybersecurity RLVR data — that comparison is impossible because such data does not exist. The claim is therefore about existence (Golden Goose enables RLVR in this domain) rather than optimality (Golden Goose data is as good as human-curated data would be).

Genuine weaknesses in the experimental design.

No statistical significance reporting. All results are presented as point estimates without confidence intervals, standard deviations, or significance tests. Given the known variance in RL training (different random seeds can produce substantially different outcomes), and the relatively small evaluation sets (e.g., AIME has 15–30 questions, making per-benchmark accuracy estimates noisy), the absence of uncertainty quantification is a significant limitation. The 4.44% cybersecurity gain, for instance, might represent a small number of additional correct answers on small test sets — a difference that could arise from sampling noise rather than genuine improvement.
No volume-matched baseline for the compute-efficient scaling experiment. The training-from-scratch comparison (Figure 6) pits ProRL data alone against ProRL + GooseReason, which differs in both data freshness and data volume. A volume-matched condition (ProRL data with repetition to match the total number of unique tasks in the joint training set) would isolate the freshness effect from the volume effect.
No cost accounting for data synthesis. The paper reports training compute (H100 GPU hours) but does not report the cost of synthesizing GooseReason-0.7M (GPT-5 API calls over 600K AoPS pairs, 1.3M code problems, and 650K textbook QAs) or GooseReason-Cyber (GPT-5 API calls over an unknown number of FineWeb scrapes, plus the extraction and filtering steps). This makes it impossible to assess the total cost of the approach relative to alternatives like human data curation or procedural generation. If the synthesis cost is comparable to or exceeds the RL training cost, the overall efficiency picture changes substantially.
Single model family for synthesis (GPT-5 only). The pipeline's effectiveness depends on the synthesis LLM's ability to identify crucial reasoning steps and generate plausible-but-incorrect distractors. The paper provides no evidence about whether other LLMs (including open-source models) could serve this role. This limits the reproducibility of the method and makes its scalability contingent on access to frontier proprietary models.
No effectiveness measurement during training. The "effective example" metric is computed once against a specific checkpoint (ProRL-1.5B-v2). As training progresses and the model improves, tasks that were previously effective may become stale. The paper does not track how the effectiveness of GooseReason tasks evolves during the continued training runs, which would reveal whether the 70% initial effectiveness is maintained or whether the tasks saturate as the model improves on them.
Limited cybersecurity benchmark detail. Table 4 reports an aggregate 4.44% gain across three benchmarks but does not provide per-benchmark breakdowns, baseline accuracies, or the number of test examples per benchmark. This makes it difficult to assess the reliability and practical significance of the gain.
No domain-transfer experiments beyond cybersecurity. The paper envisions extensions to law and medicine (Section 5) but provides no experimental evidence that Golden Goose works in these domains. The cybersecurity result is a single data point. It is unclear whether the pipeline's effectiveness generalizes across domains with different text characteristics (e.g., legal opinions with heavy citation structures, medical texts with specialized terminology and imaging references).

Missing experiments that would strengthen the paper.

Ablation of synthesis model capability (GPT-5 vs. GPT-4 vs. open-source models) to characterize the minimum required synthesis strength.
A volume-matched ProRL-only baseline for the compute-efficient scaling experiment to isolate the freshness vs. volume effect.
Dynamic effectiveness tracking during continued training to assess whether GooseReason tasks saturate.
Per-benchmark cybersecurity results and baseline accuracies to contextualize the 4.44% gain.
Multiple random seeds for key training runs to quantify variance.
A "GooseReason only" condition (without ProRL data) in the continued training experiments to isolate the contribution of the new data from the combination of old and new data.
Scale analysis: does the benefit of GooseReason increase, decrease, or stay constant as model size increases from 1.5B to 4B to (hypothetically) larger scales? The paper provides two data points (1.5B and 4B) but draws no systematic conclusion about scaling trends.

6. Limitations and Trade-offs

6.1 The Cost of Data Synthesis Is Not Accounted for in Any "Efficiency" Claim

The assumption or constraint. The paper's headline results—4× more effective examples (Figure 3), 4.44% absolute gain in cybersecurity (Table 4), sustained RL scaling beyond saturation (Figures 2, 5)—are computed from the moment training begins, excluding all costs incurred to synthesize the data in the first place. The Golden Goose pipeline requires running a frontier LLM (GPT-5) over millions of source passages: ~600K AoPS-Instruct QA pairs, ~1.3M rStar-Coder code problems, ~650K MegaScience QA pairs, and an unknown number of FineWeb cybersecurity scrapes. Each document must be processed through a multi-step prompt (Appendix A) that asks GPT-5 to identify a crucial reasoning span, generate the masked context, and produce 8 plausible-but-incorrect distractors. The paper makes no mention of the total API cost, token count, wall-clock time, or dollar amount for this synthesis. It reports only training compute (H100 GPU hours for RL) in every budget statement.

The consequence. The reported claims about "scaling up RLVR data" and "unlimited RLVR tasks" implicitly frame data synthesis as cheap relative to training. If the synthesis cost is substantial—for instance, if generating 0.7M tasks cost as much as several thousand H100 GPU hours of training compute—then the total cost of the approach (synthesis + training) may not be favorable compared to alternatives like hiring domain experts to author verifiable problems or investing in more extensive procedural environment development. The paper's central value proposition—"convert abundant unverifiable text into verifiable RLVR tasks"—assumes the conversion step is cheap enough to be practically scalable. Without cost numbers, a practitioner cannot evaluate whether Golden Goose is an economic improvement over the status quo or merely shifts the expense from human labor to API bills. This is especially acute for the cybersecurity domain: the pipeline includes an additional passage-extraction step before masking, and difficulty filtering requires 16 rollouts per candidate task against the student model, adding further cost that the paper does not quantify or discuss.

What evidence exists in the paper. None. The paper does not report synthesis cost anywhere—not in the main text, not in Appendix A (which covers data synthesis prompts but not compute), and not in Appendix B (which covers experiment details). The only cost metric reported anywhere is training H100 GPU hours, which appears in the 1,100-hour figure for the ProRL-1.5B-v2 continued training experiments (Section 3.1.1). The paper states that it uses "the strongest LLM available at the time of the experiment, GPT-5" (Section 2.1), which implies a non-trivial per-token cost, but no estimate is provided.

Mitigation status. Not addressed. The paper does not acknowledge the omitted cost, does not provide even an order-of-magnitude estimate, and does not discuss the tradeoff between synthesis compute and training compute. The "unlimited" framing of the title and abstract is therefore misleading in the economic sense—the data is unlimited in principle (there is always more internet text), but the cost of converting it may not be negligible.

6.2 The Synthesis Pipeline Depends on Access to a Frontier Proprietary Model

The assumption or constraint. All GooseReason datasets are synthesized using GPT-5, described as "the strongest LLM available at the time of the experiment" (Section 2.1). The pipeline's core operations—identifying crucial reasoning steps, generating plausible-but-incorrect distractors that are stylistically similar and contextually inconsistent—require a model that can (a) comprehend the source passage well enough to locate the reasoning-dense spans, (b) assess what would constitute an incorrect but plausible completion, and (c) generate distractors that are neither obviously wrong nor accidentally correct. Nowhere does the paper test whether weaker or open-source models can perform these operations with sufficient quality.

The consequence. If the pipeline only works with frontier proprietary models, access to Golden Goose is gated by API availability, pricing, and terms of service. This undermines the paper's vision of "automatically re-utilizing reasoning-rich unverifiable internet text to enable RL scaling" (Section 5) as a broadly accessible paradigm. A research group without GPT-5 access—or a practitioner operating under data privacy constraints that prohibit sending domain-specific text (e.g., proprietary legal documents, patient records) to external APIs—cannot replicate the method. Furthermore, if distractor quality degrades with weaker synthesis models, the resulting MCQs may be too easy (distractors are obviously wrong, enabling elimination strategies that the 9-option design was specifically intended to prevent) or too noisy (distractors are accidentally correct, corrupting the binary reward signal). The paper's choice of 9 options was calibrated using GPT-5-generated distractors (Figure 4); a different synthesis model might require a different number of distractors to achieve the same medium-difficulty distribution, and no guidance is provided for recalibration.

What evidence exists in the paper. Only the statement that GPT-5 was used, with no ablation of synthesis model capability. The paper does not test GPT-4, Claude, open-source models, or any weaker alternative. The per-benchmark results provide no breakdown by source corpus quality, so it is impossible to assess whether tasks synthesized from AoPS-Instruct (clean, structured math QA) require a less capable synthesis model than tasks from MegaScience (free-form textbook explanations) or FineWeb (raw web scrapes). The cybersecurity filtering yield (how many raw scrapes produce usable tasks after extraction and difficulty filtering) is unreported, so the end-to-end efficiency of the pipeline on noisy text—even with GPT-5—is unknown.

Mitigation status. Not addressed. The paper does not discuss the synthesis model as a dependency, does not suggest that weaker models might suffice, and does not propose open-source alternatives. The "simple trick" framing implies broad accessibility, but the reliance on GPT-5 creates a practical barrier that contradicts this framing.

6.3 The "Effective Example" Metric Depends on Fixed Hyperparameters and a Specific Model Checkpoint

The assumption or constraint. The paper's core diagnostic—that GooseReason-0.7M provides 13× more effective examples than the ProRL data blend (Figure 3)—is measured using a specific procedure: 16 rollouts per task against the ProRL-1.5B-v2 checkpoint, with a task classified as "effective" if it yields both correct and incorrect rollouts. This metric is used to argue that GooseReason data provides fresh learning signal where existing data has gone stale. However, the metric is a function of (a) the rollout count, (b) the specific model checkpoint, (c) the RL algorithm's advantage estimation procedure, and (d) the sampling parameters used during rollout evaluation. The paper does not test sensitivity to any of these.

The consequence. The 13× figure—the paper's most prominently displayed quantification of its contribution—may not generalize to different experimental settings. With more than 16 rollouts per task, some tasks currently classified as "stale" (consistent 0% or 100% accuracy) might exhibit variance and become effective, narrowing the gap between GooseReason and the original ProRL data. Conversely, with fewer rollouts, the effectiveness estimate becomes noisier. More importantly, the effectiveness measurement is a snapshot taken before continued training begins. As the model improves during the additional 1,100 H100 GPU hours of RL, tasks that were initially effective may become stale (the model masters them) or remain effective (if the model's accuracy stays in the medium range). The paper does not re-measure effectiveness during or after continued training, so it is unknown whether the GooseReason advantage is sustained or whether GooseReason tasks also saturate as training progresses. If GooseReason data saturates at a similar rate to the original ProRL data, the "revival" would be temporary, not a permanent solution to the data bottleneck.

What evidence exists in the paper. Figure 3 presents the effectiveness measurement as a single bar chart with no error bars, no sensitivity analysis, and no temporal tracking. The training curves in Figures 2 and 5 show continued improvement over the training horizon tested (500+ steps for Qwen-4B, 1,100 GPU hours for ProRL-1.5B-v2), suggesting that at least some GooseReason tasks remain effective throughout this period, but this is indirect evidence—it could also reflect the model learning from a shrinking subset of tasks while others become stale. The paper acknowledges the temporal dimension of staleness in its framing ("data saturation occurs... over prolonged training," Section 1), but does not measure how GooseReason tasks evolve along this dimension.

Mitigation status. Not addressed. The paper does not discuss the sensitivity of the effective example metric to its parameters, does not track effectiveness over training time, and does not acknowledge that the 13× figure is a point estimate at a single checkpoint. The metric is presented as an objective property of the data when it is fundamentally relational—a function of the model-data interaction that changes as the model changes.

6.4 The Cybersecurity Validation Is a Single Domain with Limited Baseline Detail

The assumption or constraint. The cybersecurity experiment (Section 3.2, Table 4) is the paper's only validation of Golden Goose on genuinely noisy, real-world source text (FineWeb scrapes) and its only demonstration of domain transfer to a specialized field with no prior RLVR infrastructure. The paper uses this single data point to support broad claims about "automatically re-utilizing reasoning-rich unverifiable internet text" and to envision extensions to "law and medicine, where verifiable data is scarce but professional literature is abundant" (Section 5).

The consequence. A single domain demonstration, however successful, does not establish generalizability. The characteristics that made cybersecurity amenable to Golden Goose—abundant web text, reasoning that can be expressed in natural-language passages, distractors that GPT-5 can generate plausibly—may not transfer to other domains. Legal reasoning, for instance, often depends on precise citation structures, multi-document cross-referencing, and jurisdiction-specific precedents that might be difficult to capture in a fill-in-the-middle MCQ. Medical reasoning may require integration of lab values, imaging findings, and patient history in formats that do not appear in clean extracted passages. The paper provides no evidence about which structural properties of a domain predict pipeline success. Additionally, the cybersecurity result itself is reported with insufficient detail to assess its reliability: Table 4 provides only an aggregate 4.44% gain across three benchmarks, with no per-benchmark breakdown, no baseline accuracies (how well did Qwen-4B-Instruct perform on these benchmarks before GooseReason-Cyber training?), and no indication of the number of test examples per benchmark. A 4.44% absolute gain on small test sets could represent only a handful of additional correct answers, and could fall within sampling variance without any statistical test.

What evidence exists in the paper. Table 4 reports the cybersecurity results as a single row showing the aggregate gain for GooseReason-Cyber vs. the previous SOTA (Llama-Primus-Instruct). The paper states the benchmarks (CTI-Bench, CyberMetric, SecEval) but provides no per-benchmark numbers. The 100-RL-step training duration is reported. No statistical significance test, confidence interval, or standard deviation is provided for any cybersecurity result. The Primus baseline uses a different base model (Llama-3.1-8B-Instruct vs. Qwen-4B-Instruct), different model size (8B vs. 4B), and a different training paradigm (domain-adaptive pretraining + SFT vs. RLVR on synthetic MCQs), introducing multiple confounds that the paper does not discuss.

Mitigation status. The paper acknowledges none of these limitations. It presents the cybersecurity result as a conclusive validation of domain transfer and a "new state-of-the-art" without qualifying the comparison, the sample sizes, or the single-domain nature of the evidence. The extensions to law and medicine are presented as a natural next step without discussion of what might make those domains harder.

6.5 No Evidence That GooseReason Data Enables Indefinite Scaling—Only That It Extends the Scaling Horizon

The assumption or constraint. The paper's title emphasizes "unlimited RLVR tasks," and the abstract claims the method "effectively revives models saturated on existing RLVR data, yielding robust, sustained gains under continuous RL." Both statements imply that GooseReason solves the data saturation problem, not merely postpones it. The experiments, however, test a fixed training horizon: 1,100 additional H100 GPU hours for ProRL-1.5B-v2 (Figure 5), 500+ steps for Qwen-4B-Instruct (Figure 2), 200 steps for compute-efficient scaling (Figure 6), and 100 steps for cybersecurity (Table 4). In all cases, the training curves show ongoing improvement at the end of the tested horizon, but no experiment runs to the point where GooseReason data itself saturates.

The consequence. A practitioner cannot know from these results how much additional training GooseReason-0.7M can sustain before it too becomes stale. The 0.7M tasks represent a finite dataset, and the paper's own diagnostic framework predicts that as the model improves on these tasks, an increasing fraction will become either trivially solvable or persistently unsolvable, collapsing the advantage signal. The initial 70% effectiveness ratio (Figure 3) is high relative to the ProRL data's 25%, but without tracking how this ratio evolves during training, there is no evidence that 0.7M tasks can sustain, say, 10,000 additional H100 GPU hours rather than 1,100. The "unlimited" claim in the title refers to the pipeline's ability to synthesize more data from new text (there is always more internet text), not to the sufficiency of any single synthesized dataset. But the paper provides no experiment showing multiple rounds of synthesis and training—the natural test of the "unlimited" claim would be to train to saturation on GooseReason-0.7M, synthesize a new batch from additional sources, train further, and demonstrate that the cycle can repeat. Without such an experiment, the claim is about potential rather than demonstrated capability.

What evidence exists in the paper. The training curves (Figures 2, 5, 6) show no sign of plateauing at the tested cutoff points, which is positive but inconclusive for longer horizons. The 70% effectiveness measurement (Figure 3) is a snapshot at the start of continued training. The paper does not re-measure effectiveness at intermediate or final checkpoints, does not report whether the model's accuracy distribution on GooseReason tasks shifts during training, and does not discuss how many tasks remain in the effective regime at the end of the training runs. The cybersecurity experiment at only 100 steps is far too short to assess saturation risk.

Mitigation status. Partially addressed through implicit acknowledgment in the paper's framing: the emphasis on the pipeline rather than the dataset suggests that the authors view GooseReason-0.7M as an instance of a repeatable process, not a one-time solution. Section 5 explicitly states the vision of "extending this paradigm to other high-value disciplines," implying iterative application. However, the paper never explicitly acknowledges that the current experiments do not demonstrate indefinite scaling, does not discuss what saturation of synthesized data might look like, and does not propose a protocol for when and how to synthesize additional rounds of data. The "unlimited" claim in the title therefore overstates the empirical evidence while being conceptually defensible.

6.6 Evaluation Is on a Narrow Slice of Benchmarks with No Statistical Uncertainty Quantification

The assumption or constraint. All evaluation uses 15 benchmarks spanning math, coding, STEM, instruction following, and logical reasoning (Section 3.1). Within each domain, the benchmarks are drawn from standard academic evaluation suites (AIME, AMC, MATH, Minerva, Olympiad Bench for math; APPS, CodeContests, HumanEvalPlus, LiveCodeBench for coding; GPQA Diamond for STEM; IFEval for instruction following; Reasoning Gym for logic). No results include confidence intervals, standard deviations, or statistical significance tests. The number of test examples per benchmark varies widely—AIME has 15–30 questions, while larger benchmarks like MATH have more—but this variance is not accounted for in aggregate domain-level reporting.

The consequence. The paper's claims about "new state-of-the-art results" (abstract) and "robust, continuous performance gains" (Section 1) are reported as point estimates that could be consistent with sampling noise, particularly on small benchmarks. RL training is known to exhibit high variance across random seeds—different initializations, data orderings, and rollout sampling can produce substantially different final performance—yet the paper does not report whether results are averaged over multiple runs or represent single training trajectories. This is especially concerning for the cybersecurity result (4.44% aggregate gain across three benchmarks), where the number of test examples is not even reported. A practitioner deciding whether to invest in the Golden Goose pipeline for their domain cannot assess the reliability of the reported gains: is a 2.71% math improvement (Figure 5) a robust effect that would replicate, or could it vary by ±1% across seeds? The absence of uncertainty quantification also makes it impossible to compare GooseReason's performance to prior work that does report variance, or to assess whether differences between conditions (e.g., GooseReason vs. RLVE in Figure 5) are statistically meaningful.

What evidence exists in the paper. All results are presented as single-number pass@1 accuracies in tables (Tables 1–4) or as single-line training curves (Figures 2, 5, 6, 8–14) with no error bars or shaded regions. The evaluation methodology section describes benchmarks and metrics but makes no mention of multiple seeds, statistical testing, or uncertainty quantification. The per-benchmark breakdowns in Appendix B (Figures 8–14) plot individual benchmark curves, again without error bars.

Mitigation status. Not addressed at all. This is the most standard methodological omission in the paper and is particularly problematic given the paper's ambition to establish new state-of-the-art results and provide guidance for practitioners. The ProRL papers that this work builds on (Liu et al., 2025a; Hu et al., 2025b) may report variance in their experiments, but the present paper does not reference or adopt any such practices. The consistent absence suggests an oversight rather than a deliberate choice, but it substantially weakens the strength of the paper's quantitative claims.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper introduces a data-centric reframing of the RLVR scaling bottleneck that shifts the field's attention from algorithmic improvements (better advantage normalization, longer training schedules, increased rollout budgets) to the data supply problem that makes those algorithms starve. Before Golden Goose, the dominant narrative was that RLVR scaling was an algorithmic challenge—models plateau because training becomes unstable, advantage estimates collapse, or the KL penalty constrains exploration. The ProRL family of work (Liu et al., 2025a; Hu et al., 2025b, 2025c) embodied this framing, proposing recipe modifications to sustain training over thousands of steps. This paper does not dispute that these algorithmic improvements are valuable—it uses them—but it demonstrates that they are necessary but not sufficient: when 75% of your training examples provide zero gradient signal (Figure 3), no amount of advantage normalization can extract learning from them because the signal literally does not exist.

The magnitude of this shift is best understood by analogy to the pretraining scaling laws literature. Before Hoffmann et al. (2022) introduced the Chinchilla scaling laws, the dominant approach to scaling pretraining was to increase model size while holding data constant—the assumption being that bigger models would automatically learn more from the same data. Chinchilla reframed the problem as a joint optimization over model size and data quantity, showing that for a fixed compute budget, data was systematically undersupplied. Golden Goose performs an analogous reframing for RLVR post-training: the dominant approach of scaling training duration on a fixed dataset (ProRL, BroRL, ScaleRL) is analogous to scaling model size on fixed data—the returns diminish not because the algorithm fails, but because the data cannot sustain further optimization. The paper's 13× effective-example gap (Figure 3) is the RLVR equivalent of demonstrating that models are data-starved by an order of magnitude.

This reframing has three concrete consequences for how the field will approach RLVR:

First, data synthesis becomes a first-class research direction rather than an auxiliary concern. Prior work treated RLVR data as something you collected once (curating human-authored problems, building procedural generators) and then focused algorithmic effort on extracting maximum value from it. Golden Goose demonstrates that the data supply can and should be scaled alongside training duration, and that doing so can revive saturated models and sustain continuous gains (Figures 2, 5). This means future RLVR scaling papers must report not just training compute and algorithmic details, but also data sourcing strategy, data freshness metrics, and the fraction of training examples that remain effective over the training horizon. The "effective example" concept introduced in Figure 3 is likely to become a standard reporting metric, analogous to how Chinchilla made data-to-parameters ratios a standard consideration in pretraining.

Second, the set of domains where RLVR is applicable expands dramatically. Before Golden Goose, RLVR was essentially restricted to math and code—the two domains where large corpora of human-authored problems with clean verification signals exist. The paper's cybersecurity experiment (Table 4) is an existence proof that RLVR can work in specialized professional domains with zero prior RLVR infrastructure, using only publicly available text. This is not an incremental extension—it changes the answer to "can we apply RLVR to domain X?" from "only if domain X has existing verifiable problem sets" to "only if domain X has abundant reasoning-rich text." The latter condition is satisfied by law (legal opinions, case analyses), medicine (clinical case reports, diagnostic reasoning textbooks), finance (investment theses, risk analyses), and virtually any profession where experts write down their reasoning. The paper explicitly names law and medicine as next targets (Section 5), but the logic extends further.

Third, it reconciles a tension in the RLVR scaling literature. The ProRL family demonstrated that prolonged RL training works and that scaling laws for RL compute exist (Hu et al., 2025b), while simultaneously showing that training saturates on finite data (Hu et al., 2025c). These findings created a puzzle: if RL scaling laws hold, why does training saturate? Golden Goose's answer is that the scaling laws are real, but they are conditional on a sufficient supply of fresh, appropriately-difficult training examples. Training longer on stale data violates the condition, producing the plateau or degradation observed in Figures 2 and 5. Training longer on fresh data (GooseReason) satisfies the condition, producing the sustained gains the scaling laws predict. This resolution is not merely theoretical—it provides practical guidance: when your RL training plateaus, the first thing to check is not your learning rate or KL coefficient, but whether your training examples still produce mixed-accuracy rollouts.

The work also redirects research attention away from procedural environment complexity and toward source text quality. The paper's comparison with RLVE (Zeng et al., 2025a) in Figure 5 is revealing: RLVE, which represents the state of the art in handcrafted procedural generation, matches GooseReason on math but underperforms dramatically on STEM (0.62% gain vs. 3.48% gain). The fundamental limitation is not algorithmic—RLVE uses sophisticated adaptive difficulty mechanisms—but structural: procedural generators can produce unlimited algorithmic reasoning tasks but cannot encode the factual and conceptual knowledge required for science reasoning, because that knowledge exists in textbooks and professional literature, not in hand-coded rule systems. For research groups deciding where to invest effort, this suggests that improving source text processing (better extraction, better masking strategies, better distractor generation) has higher marginal returns than building more complex procedural environments, at least for knowledge-intensive domains. The cybersecurity result reinforces this: no one built a cybersecurity procedural environment, yet Golden Goose produced effective RLVR data from raw web scrapes with zero domain-specific engineering.

A more subtle landscape shift concerns the relationship between base model capability and RLVR data requirements. The paper's finding that data saturation "occurs earlier and is more severe with stronger LLMs" (Section 3.1.1, Figure 2) has counterintuitive implications. The field's natural assumption has been that as base models improve, RLVR becomes easier—a stronger model should learn more from the same data. The paper shows the opposite: Qwen-4B-Instruct saturates on the ProRL data blend in 300 steps, while the weaker R1-Distill-Qwen-1.5B sustained gains for over 2,000 steps on the same data. The mechanism is that stronger models polarize faster on fixed problems—they quickly master the ones they can solve and remain stuck on the ones they can't, collapsing the effective example set from both ends. This means that the data bottleneck intensifies as models improve, which is the opposite of what you'd expect if the bottleneck were primarily algorithmic. For the field, this implies that investment in data synthesis infrastructure must grow with model capability, not shrink. The "unlimited" framing in the paper's title is therefore not just aspirational—it reflects a genuine structural requirement for continued RLVR scaling as base models get stronger.

Follow-Up Research This Work Enables

Dynamic effectiveness tracking and iterative data refreshing. The paper measures effectiveness once, at a single checkpoint before continued training (Figure 3). The natural extension is to track how the effectiveness of GooseReason tasks evolves during training. A straightforward experiment: take the ProRL-1.5B-v2 checkpoint, begin continued RL with GooseReason-0.7M, and at regular intervals (every 200 GPU hours, say) re-evaluate all tasks with 16 rollouts to compute the fraction that remain effective. This would answer the question the paper leaves open: does GooseReason data saturate at the same rate as the original ProRL data, or does its higher initial effectiveness translate to slower saturation? If GooseReason tasks saturate at a similar rate (70% → 25% effective over the same training horizon), then Golden Goose is a one-time boost rather than a permanent solution, and the pipeline must be run repeatedly—synthesize, train to saturation, synthesize again—to achieve indefinite scaling. If GooseReason tasks saturate more slowly, the higher initial effectiveness may be due to some structural property of the MCQ format (more options = harder to master) rather than just data freshness. A follow-up could also test when to trigger resynthesis: is it when overall effectiveness drops below some threshold (e.g., 30%), when performance on a held-out validation set plateaus, or when the training loss curve flattens? This would transform Golden Goose from a one-shot dataset release into an active data management strategy.

Source text quality ablation: clean corpora vs. raw web text within the same reasoning domain. The paper's reasoning-domain experiments use three carefully curated corpora (AoPS-Instruct, rStar-Coder, MegaScience), while the cybersecurity experiment uses raw FineWeb scrapes with explicit extraction and filtering. The paper never compares these two source types for the same target domain. A strong follow-up would do exactly this: take, for instance, the math reasoning domain, and synthesize GooseReason-Math from two parallel pipelines—one using clean AoPS-Instruct QA pairs (as in the paper) and one using raw math-related web scrapes from FineWeb or Common Crawl (with the passage extraction and difficulty filtering steps used for cybersecurity). Train identical models on the two resulting datasets and compare both the initial effectiveness ratios and the downstream benchmark gains. This would quantify how much the data quality depends on curated source corpora versus the pipeline's extraction and filtering capabilities. If raw web text produces comparable results, the "unlimited" claim is empirically validated—any domain with web-scale text becomes amenable to RLVR. If raw web text produces substantially worse results, the pipeline's applicability is constrained to domains with existing curated text collections (textbooks, forums, academic papers), and the paper's cybersecurity result may not generalize to other domains with similar text quality.

Minimum viable synthesis model capability study. The paper's exclusive use of GPT-5 creates a practical barrier and leaves the method's reproducibility untested. A follow-up should systematically ablate the synthesis model: take a fixed set of source passages (e.g., a 1,000-document sample from MegaScience) and run the Golden Goose pipeline with GPT-5, GPT-4, Claude-3.5, and the strongest available open-source model (e.g., Llama-3-70B or DeepSeek-V3). For each synthesis model, measure three outcomes: (1) the fraction of generated tasks that pass a quality check (are the distractors genuinely plausible-but-incorrect, as judged by a held-out strong model or human evaluation?), (2) the effectiveness ratio of the resulting tasks against a fixed student model checkpoint (ProRL-1.5B-v2), and (3) the downstream performance after RL training on each dataset. This would establish the capability threshold below which synthesis quality degrades to the point that the resulting data is not useful for RL. If open-source models can produce effective distractors, Golden Goose becomes a democratized tool. If only frontier proprietary models work, the method remains gated, and research into distillation-based synthesis (training a smaller model to mimic GPT-5's masking and distractor generation) becomes a priority.

Cross-domain robustness stress test. The paper's domain-transfer evidence consists of exactly one domain (cybersecurity) with one set of benchmarks. A systematic stress test would apply Golden Goose to 3–5 additional domains that vary along theoretically relevant dimensions: text structure (highly structured legal opinions with formal citations vs. narrative medical case reports), reasoning type (deductive legal reasoning vs. abductive medical diagnosis vs. causal economic analysis), and text quality (peer-reviewed academic literature vs. professional blog posts vs. regulatory filings). For each domain, the pipeline would synthesize RLVR data from publicly available text, train a base model, and evaluate on existing benchmarks (e.g., the LSAT or BAR exam for law, USMLE or MedQA for medicine, CFA exam questions for finance). The goal is not just to show that Golden Goose "works" in these domains, but to characterize when it works and when it fails: do domains with heavy citation dependencies produce poor distractors because context from the cited source is missing? Do domains where reasoning is implicit rather than explicit in the text (e.g., radiology reports that don't explain the diagnostic reasoning) produce tasks that test memorization rather than inference? A negative result in some domains would be as informative as positive results in others, because it would bound the method's applicability and guide users toward source text characteristics that predict success.

Combination with model-based difficulty estimation for dynamic data allocation. The "compute-optimal test-time scaling" framework from the other paper analyzed in this session (Snell et al., 2024) introduces the concept of using verifier score distributions to estimate problem difficulty before allocating inference compute. A natural synthesis with Golden Goose would be to use the effectiveness ratio not just for filtering binary decisions (keep/discard), but for dynamic sampling weights during training. Tasks that are in the medium-difficulty zone (accuracy ~0.3–0.7 across rollouts) would be upweighted; tasks near 0 or 1 would be downweighted or temporarily removed and re-evaluated later. This would convert the static difficulty filtering into an online curriculum that adapts as the model improves—tasks that become too easy over training would naturally drop out of the effective set and stop consuming training budget, while tasks that were initially too hard might become accessible as the model improves and would be re-introduced. A concrete experiment: train Qwen-4B-Instruct with ProRL data + GooseReason-0.7M, but instead of uniform sampling, use a weighted sampling distribution where the weight for each task is proportional to the variance of the 16-rollout accuracy (which peaks at 0.5 accuracy and goes to zero at 0 and 1). Compare the sample efficiency (benchmark accuracy per training step) against the uniform sampling baseline from Figure 6. This would test whether the "effective example" concept can be operationalized as a continuous curriculum rather than a binary filter.

Multi-turn synthesis and iterative RL scaling. The paper's title promises "unlimited" tasks, but the experiments demonstrate only a single round of synthesis and training. The natural stress test of the "unlimited" claim is to run multiple cycles: synthesize GooseReason-0.7M, train to saturation on it, use the saturated model as the new student model for difficulty filtering, synthesize 0.7M new tasks from fresh source text (e.g., additional textbooks, forums, or domains), train further, and repeat for 3–5 cycles. At each cycle, measure the effectiveness ratio of the newly synthesized data against the current checkpoint, the downstream benchmark gains, and whether the per-cycle gain diminishes, stays constant, or increases. This experiment directly answers whether the pipeline can sustain indefinite scaling or whether some fundamental limit (e.g., the synthesis LLM's maximum distractor quality, or the student model's ability to benefit from additional fill-in-the-middle training) imposes a ceiling. A negative result—gains diminish after 2–3 cycles despite fresh source text—would suggest that the MCQ fill-in-the-middle format itself saturates as a training signal, and that the method provides a substantial but finite boost. A positive result—gains continue for 5+ cycles—would validate the "unlimited" claim empirically and establish Golden Goose as a template for indefinite RLVR scaling.

Practical Applications and Downstream Use Cases

Self-improving domain-specialized LLMs with no human annotation. The cybersecurity result (Table 4) demonstrates the most immediately actionable application: an organization with access to domain-specific text (internal documentation, incident reports, threat analyses, compliance guidelines) can run the Golden Goose pipeline to synthesize RLVR data, train a general-purpose instruct model on it for 100–200 RL steps, and obtain a domain-specialized model that outperforms models trained with far more expensive domain-adaptive pretraining. The 4.44% absolute gain on cybersecurity benchmarks in only 100 steps, compared to Llama-Primus-Instruct's 1.44% gain from extensive domain-specific pretraining and post-training, suggests that the cost-to-performance ratio strongly favors the Golden Goose approach when the alternative is pretraining on domain data from scratch. For a corporate security team, the pipeline requires only: (1) their internal text corpus, (2) API access to GPT-5 for synthesis (or a comparable model), and (3) modest RL training infrastructure (100–200 steps on a 4B model is feasible on a small GPU cluster). No domain experts need to write problems, no verification environments need to be built, and no pretraining data pipeline needs to be constructed. The same template applies to any organization with proprietary reasoning-rich text: law firms with internal case analyses, hospitals with clinical case reports, financial institutions with investment memos, engineering firms with design review documents. The key practical constraint is that the source text must actually contain reasoning—incident reports that merely state facts without explaining the analytical process would produce poor distractors and ineffective tasks.

Data-efficient RLVR from scratch for new model releases. The compute-efficient scaling experiment (Figure 6) shows that even when starting from a fresh instruct model with no prior RL training, adding GooseReason-0.7M to the training mix accelerates learning—the GooseReason curves are above the ProRL-only curves from the earliest training steps. For model development teams preparing a new base model release, this suggests a default data strategy: rather than spending weeks curating domain-specific RLVR datasets or building procedural generators, run the Golden Goose pipeline over publicly available reasoning corpora (AoPS, MegaScience, rStar-Coder, plus domain-specific web text for target applications), synthesize a large pool of MCQ tasks, difficulty-filter against the new base model, and use the resulting data from the first RL step. The paper's result that Qwen-4B-Instruct requires only 200 steps to reach strong performance when trained jointly with GooseReason-0.7M (Figure 6) means the data synthesis cost can be amortized over a short training run, and the per-step efficiency gain means less total GPU time for a given target accuracy. For a team releasing a model at a new scale (e.g., a 7B or 13B instruct model), synthesizing GooseReason-style data against that specific model would produce a difficulty-calibrated dataset without any assumptions about transfer from smaller models—a significant advantage over using a fixed dataset like the ProRL blend, which the paper shows saturates faster on stronger models.

Resurrecting stale RLVR training runs in production. The paper's primary experimental setup—taking a saturated model (ProRL-1.5B-v2, 20,000 H100 GPU hours of prior training) and reviving it with GooseReason data for an additional 1,100 GPU hours (Figure 5)—maps directly onto a common production scenario. A team invests substantial compute in RLVR training on a fixed dataset, observes diminishing returns, and faces the decision of whether to continue training (wasting compute on stale data) or to stop and release the current checkpoint (leaving potential performance on the table). Golden Goose provides a third option: synthesize fresh data from unverifiable sources and resume training. The key practical decision is when to trigger resynthesis. Based on the paper's diagnostic, the signal would be when the effective example ratio on the training set drops below a threshold (the paper shows 25% is too low to sustain gains for ProRL-1.5B-v2; whether 40% or 50% is sufficient is unknown). A production system could periodically evaluate effectiveness on a sample of training tasks, and when the ratio drops, automatically trigger a new round of Golden Goose synthesis from additional source text. This converts RLVR training from a one-shot process into a continual learning loop where data freshness is actively managed.

Benchmark contamination defense via synthesis from novel sources. The paper mentions but does not deeply explore a side benefit of Golden Goose: because the training data is synthesized from source text chosen by the practitioner (not from standard RLVR datasets), the resulting model is less likely to have inadvertently memorized evaluation benchmark problems that leaked into public training sets. This is particularly relevant for math and code benchmarks, where test problems sometimes appear in community forums (AoPS, Codeforces) that feed into training corpora. By masking reasoning steps from those forums rather than training on the full problems with answers, Golden Goose reduces (though does not eliminate) the risk that the model learns to pattern-match specific benchmark solutions. For benchmark developers and model evaluators, this suggests a methodological improvement: when assessing whether a model's performance reflects genuine reasoning or data contamination, check whether the model was trained on Golden Goose-synthesized data (which uses forum text as source material but in a transformed format) or on the original forum QA pairs (which include complete solutions). The synthesis process acts as a form of data augmentation that preserves reasoning content while scrambling the surface format that contamination detectors look for. This is not a primary contribution of the paper, but it is a practical consideration for teams concerned about benchmark integrity in their training pipelines.