Phi-4-reasoning Technical Report

ArXiv: 2504.21318

🎯 Pitch

A 14B model fine-tuned from Phi-4 hits 78% on the contamination-free AIME 2025 math benchmark, surpassing the 70B DeepSeek‑R1‑Distill‑Llama and closing in on the full 671B DeepSeek‑R1. The key is a “teachable prompts” data‑curation strategy—filtering for problems just beyond the base model’s ability and solving them via o3‑mini chains—combined with a short RL phase that pushes trace length to over 8K tokens. Reasoning transfers as a meta‑skill, yielding 30–60 point jumps on out‑of‑domain algorithmic tasks even though the training mix was limited to STEM and code.

1. Executive Summary

This technical report introduces Phi-4-reasoning, a 14-billion parameter reasoning model obtained by supervised fine-tuning Phi-4 on a carefully curated dataset of over 1.4M "teachable" prompts with long chain-of-thought traces generated by o3-mini, and Phi-4-reasoning-plus, a variant further enhanced through a short phase of outcome-based reinforcement learning using GRPO on ~6K math problems. On the AIME 2025 benchmark—released after training data finalization and thus contamination-free—Phi-4-reasoning achieves 63.1% pass@1 while Phi-4-reasoning-plus reaches 78.0%, outperforming the significantly larger DeepSeek-R1-Distill-Llama-70B (51.5%) and approaching the full 671B-parameter DeepSeek-R1 (70.4%). The models demonstrate that meticulous data curation for SFT—filtering seeds to the boundary of the base model's capabilities and constructing synthetic variants for verifiability—combined with targeted RL yields reasoning as a transferable meta-skill, with gains of 30–60 percentage points on out-of-domain algorithmic tasks (TSP, 3SAT, calendar planning) and non-trivial improvements on general-purpose benchmarks, establishing that reasoning-focused post-training generalizes broadly even when the training data is limited to STEM, coding, and safety domains.

2. Context and Motivation

The Gap: Reasoning as a Learned Skill, Not Just an Emergent Property

The fundamental question this paper addresses is deceptively practical: can a small (14B) language model be taught to reason at a level competitive with models 10–50× its size, and if so, what training recipe makes this possible? This is not merely an academic exercise in model compression. It strikes at a core tension in the current LLM landscape: frontier reasoning capabilities are increasingly concentrated in massive, proprietary models (OpenAI's o1/o3, DeepSeek-R1 at 671B parameters) that are expensive to serve, impossible to inspect, and impractical for many deployment scenarios. A 14B model that approaches this performance tier would fundamentally change the economics of reasoning workloads.

The paper's framing is that reasoning is not solely an emergent property of scale but a skill that can be systematically taught through data curation and training methodology. This is a direct extension of the "data-centric" philosophy that characterized the earlier Phi model family (Phi-1 through Phi-4), which demonstrated that high-quality synthetic training data could produce small models with outsized capabilities. However, those models were not explicitly trained to produce extended chain-of-thought reasoning traces that leverage inference-time compute — they could solve problems, but not through the kind of structured, self-reflective multi-step reasoning that characterizes the o1/R1 generation of models. The gap this paper addresses is: how do you take a strong but "direct-answer" base model and teach it to think step-by-step in a way that generalizes across domains?

Why This Gap Matters: The Deployment Economics of Reasoning

The practical importance of this problem is multi-dimensional:

Serving costs dominate at scale. Frontier reasoning models like DeepSeek-R1 (671B MoE) and o1/o3-mini are computationally expensive at inference time. A 14B dense model that achieves comparable reasoning accuracy would reduce per-query costs by roughly an order of magnitude in FLOPs, making reasoning capabilities economically viable for high-throughput applications (customer support, educational tools, automated code review).

Latency constraints in interactive settings. Smaller models generate tokens faster. A 14B model producing reasoning traces at (say) 100 tokens/second supports interactive use cases — tutoring, pair programming, real-time decision support — where a 671B model generating at 15–20 tokens/second creates unacceptable user experience friction. The paper's exploration of the token-length vs. accuracy tradeoff between Phi-4-reasoning and Phi-4-reasoning-plus (Figure 11) is specifically motivated by this practical concern: different deployment contexts require different points on the efficiency frontier.

Inference-time compute scaling interacts with model size. As the paper shows in Figure 2, parallel test-time compute (majority voting over many samples) can push a 14B model's AIME 2025 accuracy from ~78% (pass@1) to ~95% (best-of-64). This suggests that small models + inference-time compute can reach accuracy regimes that were previously believed to require frontier-scale models. Understanding this interaction — and building models that are explicitly designed to exploit it through structured chain-of-thought generation — is a research direction with immediate deployment implications.

The transfer of reasoning as a meta-skill. Perhaps the most striking motivation for this work is the observation that reasoning capabilities acquired during domain-limited training (STEM, coding, safety) transfer to entirely novel task types. The paper reports 30–60 percentage point improvements on algorithmic problem solving (TSP, 3SAT) and calendar planning — tasks that "were not intentionally targeted" during SFT or RL (Section 5.1.3). This suggests reasoning is not being learned as a narrow pattern-matching ability tied to training distribution but as a generalizable cognitive strategy. If true, this has profound implications for how we think about AI training efficiency: targeted reasoning training may be far more sample-efficient than training on broad task coverage.

Prior Approaches and Where They Fall Short

The paper positions itself at the intersection of several active research threads, each with documented limitations:

1. Frontier reasoning models via large-scale RL (DeepSeek-R1, Kimi K1.5). DeepSeek-R1 [21] demonstrated that pure reinforcement learning on top of a strong base model could induce sophisticated reasoning behaviors including self-verification, backtracking, and multi-strategy exploration. However, the recipe was developed for a 671B MoE model, and the question of whether these techniques work at smaller scales remained open. DeepSeek attempted to address this through distillation — producing R1-Distill models (Llama-70B, Qwen-32B, etc.) by supervised fine-tuning on R1's outputs. These distilled models showed strong performance but the distillation process was relatively opaque, and the specific curriculum design, data filtering, and mixture decisions that matter most for small-model reasoning were not elucidated.

2. Data-centric Phi models (Phi-1 through Phi-4). The Phi family established that small models trained on carefully curated synthetic data could punch far above their weight class. Phi-4 achieved performance competitive with GPT-4o on certain reasoning benchmarks despite being orders of magnitude smaller. However, Phi-4 was not trained to produce extended chain-of-thought reasoning. It solved problems efficiently but through relatively direct answer generation, not through the kind of explicit step-by-step decomposition and self-reflection that enables inference-time compute scaling. The paper explicitly notes: "despite its solid reasoning foundations, Phi-4 was not explicitly trained to produce extended chain-of-thought (CoT) reasoning traces to utilize additional inference time compute" (Section 2). This is the specific capability gap that Phi-4-reasoning is designed to bridge.

3. Distillation from strong teachers without systematic seed curation. Several open-weight reasoning models (OpenThinker2, QwQ-32B, EXAONE-Deep-32B) use distillation from large teacher models. The implicit assumption in much of this work is that teacher quality is the dominant factor — use a good enough teacher to generate reasoning traces, train on enough data, and the student will learn. This paper challenges that assumption. The authors find that seed selection — choosing prompts at the "teachable" boundary of the student's current capabilities — is equally or more important than teacher quality for the resulting model's generalization. The paper goes so far as to show that training on the wrong set of seeds can degrade performance, and that data mixtures can be optimized independently per domain and then combined additively (Section 3.1). This level of methodological detail about what makes distillation work — beyond "use a good teacher" — is largely absent from prior work.

4. RL for reasoning in small models (DeepScaleR, LIMR, DAPO). Several concurrent efforts have explored applying reinforcement learning to small reasoning models. These works generally start from a distilled SFT model and apply RL (often GRPO or DPO variants) to further improve math reasoning. However, the interaction between SFT data quality and downstream RL effectiveness remains poorly understood. The paper provides a concrete case study: GRPO on top of Phi-4-reasoning (which was already carefully SFT-trained) yields ~10% improvement on AIME in only 90 steps (~6K examples), but further training does not help (Figure 7a). This "early saturation" behavior is a specific empirical finding that illuminates the diminishing returns of RL when starting from a high-quality SFT model — an insight with direct implications for how to allocate training compute between SFT and RL phases.

5. Benchmarking methodology for reasoning models. The paper explicitly engages with a growing critique of reasoning model evaluation: that single-run, single-average accuracy reporting on small benchmarks (especially AIME, with only 30 questions for the 2025 edition) is statistically unreliable due to the high nondeterminism of reasoning model outputs. The authors show (Figure 9) that two independent sets of 5 runs can differ by 5–10 percentage points on AIME 2025, making side-by-side model comparisons based on single reported numbers essentially meaningless. This echoes recent critiques from Balachandran et al. [10] and Hochlehnert et al. [25], but the paper goes further by implementing the recommended remedy: reporting standard deviations across 50–64 runs for small benchmarks, using larger unsatured benchmarks for primary evaluation, and visualizing accuracy distributions rather than point estimates. The motivation here is not just methodological purity — it's that without addressing these evaluation issues, the entire enterprise of reasoning model development lacks reliable progress indicators.

How This Paper Positions Itself

The paper does not claim to invent new architectures, novel RL algorithms, or fundamentally different training paradigms. Its contribution is more subtle and, in some ways, more practically valuable: a detailed, reproducible recipe for turning a strong base model into a reasoning model through systematic data curation, with careful attention to what makes that recipe work and where its boundaries are.

Specifically, the paper positions itself as:

A data-centric approach to reasoning, not a scaling-centric one. The core argument is that which seeds you train on matters more than how many seeds you train on. The filtering pipeline described in Section 2.1 — using LLM-based difficulty estimation, plurality-voting agreement gaps between strong and weak models as a "teachability" signal, and rubric-based reasoning-step complexity assessment — is the paper's true intellectual contribution, not the SFT training loop itself. This aligns with the Phi family tradition but extends it to the specific challenges of reasoning data (where difficulty is harder to define, teacher quality interacts with seed selection, and domain transfer depends on seed diversity).

A bridge between SFT-based distillation and RL-based refinement. Prior work often treats these as competing approaches. The paper shows they are complementary: SFT establishes the ability to generate structured reasoning traces (thinking/response separation, systematic decomposition), while RL refines the quality and length of those traces to maximize reward on verifiable problems. The observation that SFT response lengths decrease during training (Figure 4b) while RL response lengths increase (Figure 7d) is a concrete illustration of how these phases serve different functions. The paper does not claim to have found the optimal SFT+RL recipe — it explicitly notes the RL was limited to math and only ~6K examples — but it provides a clear template for what each phase contributes.

An explicit contribution to evaluation methodology. The paper devotes substantial space (Section 5.1.2, Figure 9, Figure 10) to arguing for a new standard in reasoning model evaluation: reporting accuracy distributions rather than point estimates, using larger benchmarks, showing per-year and per-topic breakdowns, and being transparent about variance. This is not an afterthought — it is presented as a necessary precondition for the field to make reliable progress. The paper's own model comparisons are explicitly qualified by variance estimates, and the authors acknowledge that some reported baseline numbers may not be directly comparable due to undocumented evaluation procedures.

A contributor to the "small models can reason" evidence base. The paper joins a growing body of work (rStar-Math, DeepScaleR, LIMR, QwQ-32B) demonstrating that reasoning capabilities are not the exclusive province of massive models. However, it distinguishes itself through the methodological detail of its data pipeline and the breadth of its generalization analysis. The finding that reasoning training improves performance on calendar planning, maze solving, and spatial reasoning — tasks with no explicit representation in the training data — is the paper's strongest evidence that it has taught the model a transferable reasoning strategy rather than a collection of domain-specific heuristics. This generalizability claim, if it holds up under replication, is of greater scientific interest than the specific benchmark numbers.

The Opening the Paper Exploits

A key enabling factor that the paper implicitly relies on is the availability of strong teacher models (specifically o3-mini, used in both medium and high reasoning-effort modes) that can generate high-quality reasoning traces at scale. This creates a historically specific opportunity: for the first time, a team without frontier-scale training infrastructure can produce a reasoning model by curating distillation data from a third-party teacher. The paper is transparent about this dependency — o3-mini is credited as the teacher throughout — but also demonstrates that teacher choice involves subtle tradeoffs (Section 3.2): o3-mini with medium effort is more token-efficient and produces comparable results to DeepSeek-R1 as a teacher, while high-effort o3-mini is stronger but produces longer responses. These practical calibration insights are exactly the kind of detail that prior distillation work has not surfaced.

3. Technical Approach

3.1 Reader Orientation

This paper describes a training pipeline—not a single model architecture—for transforming a 14-billion parameter base language model (Phi-4) into a structured reasoning model. The system solves the problem of teaching a relatively small model to produce extended, self-reflective chain-of-thought reasoning traces that generalize across domains, using a two-stage recipe: supervised fine-tuning on carefully curated "teachable" prompts with long reasoning demonstrations from a strong teacher, followed by targeted reinforcement learning that incentivizes longer, more accurate reasoning on verifiable math problems.

3.2 Big-Picture Architecture (Diagram in Words)

The system has five major components, deployed across two training stages:

1. Seed Database — a broad collection of prompts from web sources, existing datasets, and synthetic generation, spanning STEM, coding, and general-purpose domains. Seeds are the raw prompts that will be answered by the teacher model.

2. Seed Filtering Pipeline — a multi-stage LLM-based curation process that selects only those seeds at the "teachable" boundary of the base model's capabilities, using difficulty estimation via agreement gaps and reasoning-step complexity assessment.

3. SFT Data Generation — o3-mini (the teacher) generates long chain-of-thought reasoning traces for each filtered seed, structured into thinking and response blocks. The resulting 1.4M prompt-response pairs (8.3B tokens) form the supervised fine-tuning dataset.

4. Phi-4-reasoning (SFT Stage) — Phi-4 is fine-tuned on this dataset with architectural modifications (new special tokens for thinking/response demarcation, doubled RoPE base frequency, 32K context length). The model learns to produce structured reasoning traces.

5. Phi-4-reasoning-plus (RL Stage) — The SFT model undergoes outcome-based reinforcement learning using GRPO on ~6K math problems with verifiable answers. A length-aware reward function incentivizes concise correct answers and longer thinking on incorrect answers.

Information flows as follows: raw seeds enter the filtering pipeline → "teachable" seeds are sent to o3-mini for reasoning trace generation → prompt-response pairs are assembled into the SFT dataset → Phi-4 is fine-tuned to produce structured reasoning → the resulting model is further optimized via RL with a rule-based reward on math problems → Phi-4-reasoning-plus emerges with longer, more accurate reasoning traces, particularly on math.

3.3 Roadmap for the Deep Dive

• First, the seed database construction and filtering pipeline (Section 3.4.1), since seed quality is the paper's central claim and determines everything downstream. Understanding what makes a seed "teachable" is prerequisite to understanding why the SFT works.

• Second, the synthetic data generation process (Section 3.4.2), including the teacher model choice, the structured format (thinking/response blocks), and the rewriting of seeds into verifiable variants. This is where raw prompts become training examples.

• Third, the SFT training recipe (Section 3.4.3), including architectural modifications (special tokens, RoPE frequency doubling, context length extension), hyperparameter selection, the additive data mixture property, and the observed dynamics (decreasing response lengths, generalization beyond training domains).

• Fourth, the RL training stage (Section 3.4.4), including the GRPO algorithm details, the length-aware reward function (the paper's most technically novel component), and the training dynamics (growing response lengths, plateauing reward, maintained entropy).

• Fifth, the evaluation methodology (Section 3.4.5), since the paper makes a methodological contribution to how reasoning models should be evaluated. This includes the argument for multi-run statistical rigor, the benchmarks chosen, and the per-year/per-topic breakdown approach.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a training methodology paper whose core idea is that reasoning in small language models is a teachable skill that emerges from (1) careful seed selection at the boundary of the base model's capabilities, (2) distillation of structured chain-of-thought traces from a strong teacher, and (3) targeted reinforcement learning that shapes the length and quality of reasoning traces, with the crucial practical finding that these phases serve complementary functions—SFT establishes the reasoning format and broad generalization, while RL refines domain-specific accuracy and inference-time compute allocation.

3.4.1 Seed Database Construction and the "Teachable" Filtering Pipeline

The foundational component of the entire training pipeline is the seed database—a collection of raw prompts or problems that will eventually be answered by the teacher model to create training data. The paper's central methodological claim is that which seeds are selected matters more than how many seeds are selected, and that filtering seeds to lie at the "boundary" of the base model's current capabilities is the key to achieving generalization.

Seed collection. Seeds are sourced from three channels (Section 2.1): publicly available websites, existing datasets, and licensed collections. These are supplemented with "synthetic questions generated to be grounded in high-quality, filtered web content." At this initial collection stage, the focus is on maximizing diversity and coverage across reasoning-heavy domains, particularly STEM disciplines and coding, while also incorporating general-purpose question-answer style prompts and alignment-focused data for safety. The raw seed database spans a broad range of difficulty levels—many seeds are trivially easy for Phi-4, some are impossibly hard, and only a subset lies in the productive "teachable" zone.

The concept of "teachable" seeds. A seed is teachable if the base model (Phi-4) cannot currently solve it reliably but has the foundational knowledge to learn from a demonstration. Formally, this corresponds to seeds where Phi-4's pass@1 is neither near 100% (already mastered—nothing to learn) nor near 0% (completely out of reach—no foundation to build on). The paper operationalizes this through two heuristic filtering mechanisms (Section 2.1):

Heuristic 1: Agreement-gap difficulty estimation. When verifiable ground-truth solutions are available (as in math), difficulty can be estimated directly by measuring Phi-4's accuracy. However, the paper explicitly notes that "verifiable ground-truth solutions or objective notions of difficulty may not be available across all domains." For domains without ground truth, the authors use a two-model agreement protocol:

1. Generate plurality responses from a strong reference model (the paper does not specify which, but it is presumably o3-mini or a similar frontier model) to serve as a proxy for ground truth.
2. Generate responses from a weaker model (e.g., Phi-4 or GPT-4o) for the same seeds.
3. Compute the agreement rate between the weaker model's generations and the proxy ground-truth solution.
4. Retain seeds that show a "meaningful gap"—the weaker model disagrees with the proxy ground truth at a non-trivial rate, indicating room for improvement, but not at a rate approaching random chance, which would indicate a complete lack of capability.

This protocol is an operationalization of Vygotsky's "zone of proximal development" applied to language models: the productive training examples are those the model cannot solve independently but can learn from with demonstration. Seeds where the model already agrees with the reference (easy) contribute nothing because the model already knows the answer. Seeds where the model disagrees at near-random rates (hard) may be beyond the model's fundamental capacity to learn from a single demonstration.

Heuristic 2: Rubric-based reasoning complexity assessment. In addition to the difficulty estimate, the authors use "rubric-based LLM evaluators" to assess the number and complexity of reasoning steps required to solve a prompt. This provides a further filtering and prioritization signal: prompts that demand complex multi-step reasoning are prioritized over those primarily testing factual recall. The paper is not explicit about the rubric design, but the intent is clear—the goal is to select seeds that teach reasoning as a process (decomposition, reflection, multi-strategy exploration), not seeds that teach domain-specific facts.

The iterative feedback loop with SFT experiments. The seed filtering process was not a one-time preprocessing step. The paper describes an iterative cycle where "signals from early SFT runs were used to iteratively expand and improve the training data mixture" (Section 3.1). This means: run an SFT experiment with a candidate seed set, evaluate on held-out reasoning benchmarks (AIME 2024, GPQA diamond), identify which data sources or seed characteristics correlate with improvements, and refine the filtering criteria accordingly. This iterative coupling between data curation and training experiments is a methodological practice that the paper implicitly advocates but does not formalize as an algorithm.

Synthetic seed transformation. A subset of filtered seeds is rewritten and transformed into new synthetic datasets (Section 2.1). The paper provides a concrete example in Figure 3:

• Original (from web): "On the sides AB and BC of triangle ABC, points M and N are taken, respectively. It turns out that the perimeter of △AMC is equal to the perimeter of △CNA, and the perimeter of △ANB is equal to the perimeter of △CMB. Prove that △ABC is isosceles."
• Transformed (synthetic): "ABC is a triangle with AB = 13, and BC = 10. On the sides AB and BC of triangle ABC, points M and N are taken, respectively. It turns out that the perimeter of △AMC is equal to the perimeter of △CNA, and the perimeter of △ANB is equal to the perimeter of △CMB. What is AC?"

The transformation serves two purposes. First, it converts a proof problem (which requires open-ended verification) into a computational problem with a specific numeric answer (amenable to rule-based verification in downstream RL). Second, it produces "short, precise final answers" that are "conducive to reinforcement learning using verifiable math problems" (Section 3.1). This transformation is an explicit design decision to make the seed data dual-use: the same seeds can serve SFT (where long reasoning traces are demonstrated) and RL (where a verifiable final answer enables reward computation).

Decontamination. The full training data is passed through the same rigorous decontamination process used for Phi-4, checking against "popular reasoning as well as general-purpose benchmarks including many not discussed in this report" (Section 2.2). The explicit list includes AIME-2024, MATH, GPQA, LiveCodeBench, Codeforces, OmniMATH, SWE-Bench Verified, SimpleQA, DROP, AGIEval, ARC-Challenge, ARC-Easy, CommonsenseQA, GSM8k, HellaSwag, HumanEval, MBPP, OpenBookQA, PIQA, WinoGrande, ArenaHard, MT-Bench, and PhiBench. Notably, AIME-2025 was released after the data for Phi-4-reasoning was finalized, so that benchmark remains contamination-free—a fact the paper uses to argue for the validity of its AIME 2025 results as a genuine out-of-sample test.

3.4.2 SFT Data Generation: Teacher Model, Format, and Safety

Once the seed database is filtered and decontaminated, the next stage is generating the actual training examples—the prompt-response pairs that will be used for supervised fine-tuning.

Teacher model selection. The paper uses o3-mini as the teacher model for generating reasoning traces. The choice is not arbitrary: the authors conducted explicit comparisons between o3-mini (medium reasoning effort), o3-mini (high reasoning effort), and DeepSeek-R1 as teachers (Section 3.2). They found that "o3-mini with medium 'reasoning effort' to have similar effect to DeepSeek-R1 when used as teachers, but o3-mini medium was more token efficient." Furthermore, "o3-mini with high-effort to be a stronger teacher than medium-effort consistently across tasks" but "it also resulted in longer responses, increasing inference-time compute." This teacher selection is not merely a practical convenience—it is an explicit tradeoff between demonstration quality and inference-time compute cost for the student model. A stronger teacher produces better reasoning traces but teaches the student to generate longer responses, which increases serving costs. The paper does not resolve this tradeoff but surfaces it as a design dimension.

Structured response format. Every generated response follows a specific structure (Section 2.2, Section 3):

<thinking>
... detailed reasoning trace ...
</thinking>
<response>
... final answer ...
</response>

This format is enforced by a system message that instructs the model to "structure your response into two main sections: Thought and Solution using the specified format." The thinking block contains the step-by-step reasoning process, including "analysing questions, summarizing relevant findings, brainstorming new ideas, verifying the accuracy of the current steps, refining any errors, and revisiting previous steps." The response block contains "the final solution that you deem correct" presented in a "logical, accurate, and concise" manner.

This structured format serves multiple purposes simultaneously. First, it enables the model to learn the distinction between exploration (in the thinking block) and presentation (in the response block)—the thinking block can contain dead ends, revisions, and uncertainty, while the response block should be polished and authoritative. Second, it enables downstream applications to selectively hide or show the reasoning trace: the paper notes that "teaching the model to not reveal the guidelines or the chain-of-thoughts in the 'answer' block... may be a desirable behavior for applications that want to choose to only show the final output to users to reduce cognitive load and overreliance on chain-of-thought traces" (Section 2.2). Third, it makes the final answer extractable for verification in the RL stage.

Safety data generation. A "diverse dataset of alignment and safety prompts" is sourced from Phi-4 and the AgentInstruct pipeline. These prompts are augmented with "detailed safety guidelines" that cover topics including "User Understanding and Clarity, Security and Ethical Guidelines, Limitations, Disclaimers and Knowledge Scope, Handling Complex and Sensitive Topics, Safety and Respectful Engagement, Confidentiality of Guidelines and Confidentiality of Chain-of-Thoughts" (Section 2.2). Responses are generated using the same teacher model and pipeline as the math/coding data. Critically, during training, "we remove the safety guidelines from the prompt to incentivize the model to implicitly learn the expected behavior." This is a deliberate design choice: the model should internalize the safety norms as learned behavior rather than externally prompted constraints.

The paper notes an interesting side effect: "the model tends to regurgitate variations of these guidelines in the 'thinking' block." This is framed as an active research question around whether revealing safety reasoning in the thinking block is desirable, particularly for open-source models where complete generations are visible to users and downstream developers.

Scale of the SFT dataset. The final SFT dataset comprises "over 1.4 million prompt-response pairs, totaling 8.3 billion unique tokens of reasoning domains such as math and coding, and alignment data for safety and Responsible AI" (Section 3). At an average of approximately 5,900 tokens per example, this represents a substantial but not enormous SFT corpus—roughly equivalent to the token count of a single pass through a medium-sized web crawl, but vastly more curated.

3.4.3 SFT Training: Architecture Modifications, Hyperparameters, and Training Dynamics

The supervised fine-tuning stage takes the filtered seed dataset with o3-mini-generated reasoning traces and trains Phi-4 to produce structured chain-of-thought reasoning.

Architectural modifications. Two changes are made to the base Phi-4 architecture (Section 3):

Modification 1: Special tokens for reasoning demarcation. "Two placeholder tokens from the base model were repurposed as <|thinking|> and <|response|> tokens to mark the beginning and end of a reasoning ('thinking') block, respectively." These are not newly initialized embeddings—they repurpose existing unused tokens from the tokenizer, meaning no vocabulary expansion is needed. The model learns to use these tokens to segment its output into the structured thinking/response format.

Modification 2: Context length extension from 16K to 32K tokens. The base Phi-4 model originally supported a maximum token length of 16K tokens. To accommodate the long chain-of-thought reasoning traces (o3-mini's high-effort responses in particular), the model's context length is extended to 32K. The mechanism is "doubling the RoPE base frequency" (Section 3), a technique from the rotary position embedding literature where the base frequency parameter of the sinusoidal position encoding is scaled to extend the effective context window without architectural changes. The specific RoPE base frequency values before and after doubling are not provided, but the paper confirms that the model was "trained for a maximum length of 32K tokens."

Training hyperparameters. The SFT training uses (Section 3):

• Optimizer: AdamW
• Learning rate: $1 \times 10^{-5}$ (selected via grid search over $[1 \times 10^{-6}, 2 \times 10^{-5}]$)
• Global batch size: 32
• Context length: 32K tokens
• Training steps: "roughly 16K steps"
• Weight decay: $1 \times 10^{-4}$
• Learning rate schedule: linear warmup over 450 steps

These hyperparameters were not simply inherited from Phi-4's instruction tuning. The paper explicitly states that "SFT recipes used for instruction finetuning of Phi-4 do not transfer directly to reasoning-focused training" and that "the optimal hyperparameters for reasoning data differed significantly from those used for alignment-focused tuning in Phi-4" (Section 3). The learning rate selection in particular involved a grid search from $1 \times 10^{-6}$ (Phi-4's SFT learning rate) to $3 \times 10^{-5}$ (Phi-4's mid-training learning rate), with $1 \times 10^{-5}$ providing "the best balance in terms of reasoning performance." Higher learning rates produced lower training loss but "saturation and/or degradation across various downstream evaluations"—a classic overfitting signal.

The additive data mixture property. One of the paper's most practically useful findings is described as the "additive property" of data mixtures across domains (Section 3.1). The authors clustered data sources by (1) domain (math, code) and (2) quality, assigning the same training weight (number of epochs) to all members of a cluster. They then optimized weights independently per domain—finding the optimal math data mixture in isolation, then the optimal code data mixture in isolation—and discovered that "mixtures could be optimized independently for each domain—such as math and code—and then combined by simply concatenating their respective weights." The resulting composite mixture "preserved the domain-specific gains achieved during isolated tuning."

This additive property is not theoretically guaranteed—in general, multi-task training can exhibit destructive interference where optimizing for one task degrades performance on another. The fact that addition works in this setting is an empirical finding that substantially simplifies the data mixture optimization problem: instead of searching a combinatorially large joint space, one can decompose the search into independent per-domain optimizations and then concatenate.

Training dynamics: decreasing response length during SFT. Figure 4b shows a counterintuitive pattern: as SFT training progresses, the average response length on AIME 2024 and GPQA diamond decreases. The mean tokens on AIME 2024 drop from roughly 20,000 at step 2,500 to roughly 13,000–14,000 by step 15,000 (reading from Figure 4b). The 75th percentile drops from approximately 28,000 to approximately 18,000–20,000 over the same period. Simultaneously, accuracy improves (Figure 4a).

The paper interprets this as evidence that "the model is learning to use its token budget more efficiently as training progresses" (Section 3). This is in direct contrast to the RL stage, where response lengths increase with training. The two stages serve fundamentally different functions: SFT teaches the model how to reason efficiently (getting to the right answer in fewer tokens), while RL teaches the model when to allocate more tokens (thinking longer on harder problems). The paper does not explicitly frame this as a design principle, but the empirical pattern suggests that SFT-first-then-RL is a natural decomposition: learn the efficient reasoning skill, then learn when to override efficiency in favor of thoroughness.

The system message and format robustness. The paper used a fixed reasoning-focused system message during SFT training (reproduced in full in Section 3.1). The message explicitly instructs the model about the thinking/response structure and describes the expected contents of each section. Ablation experiments tested partially removing or replacing this system message with generic variants during training, finding that this "increased robustness under random system messages at inference time" but "when evaluated under the original reasoning message, we observed greater variability in benchmarks scores and a slight decrease in average benchmark performance." The final training configuration used the fixed message, trading robustness to system message variation for peak performance under the intended prompt format.

Base model selection: Phi-4 vs. Phi-4-base. The authors experimented with two starting checkpoints: Phi-4 (the fully post-trained model) and Phi-4-base (the mid-trained checkpoint before vanilla post-training). Both "performed similarly on reasoning benchmarks, while Phi-4 performed slightly better in terms of safety and alignment, as measured by the automated measurement of Responsible AI metrics for LLMs framework." The final choice was Phi-4, to "preserve the benefits of prior non-reasoning post-training." This choice matters because it means the reasoning SFT is not training from a blank slate—the model already has instruction-following and safety behaviors that the reasoning training must not catastrophically overwrite. The paper's observation that "we do not see any catastrophic forgetting compared to the base Phi-4 model on more general capabilities" is a specific empirical validation that the SFT recipe successfully augments reasoning without destroying previously learned behaviors.

Training progression and manual verification. The paper includes a qualitative observation about the learning dynamics: "the model begins to use explicit 'thinking' tokens very early in training, indicating the superficial structured format itself is learned quickly. However, the efficacy of the chain-of-thought block and the ability of the model to reason improves throughout training." This suggests a two-phase learning process within SFT itself: rapid acquisition of the surface format (which is a simple token-level pattern), followed by slower acquisition of the substantive reasoning skill (which involves learning when and how to decompose problems, verify intermediate steps, and explore alternatives). The paper does not provide quantitative evidence for this phase distinction but presents it as a qualitative observation from manual inspection of intermediate checkpoints.

3.4.4 Reinforcement Learning Stage: GRPO with Length-Aware Reward

The second training stage applies outcome-based reinforcement learning to the SFT-trained Phi-4-reasoning model to produce Phi-4-reasoning-plus. The RL training is focused exclusively on mathematical reasoning and uses the Group Relative Policy Optimization (GRPO) algorithm.

Why RL on top of SFT? The paper frames the RL stage as addressing a specific limitation of SFT: while SFT teaches the model to produce structured reasoning traces, it does not optimize for the quality-vs-length tradeoff. SFT demonstrations are generated by a fixed teacher at a fixed reasoning effort level—the student learns to mimic that level but has no signal about when more or fewer tokens would be beneficial. RL provides a mechanism to shape the model's token allocation based on outcomes: correct answers are incentivized to be concise, incorrect answers are incentivized to be more thorough. This creates a learned policy where the model dynamically adjusts its reasoning depth based on problem difficulty—a form of implicit inference-time compute scaling that the SFT stage alone does not teach.

Seed data for RL. The seed dataset for GRPO consists of "72,401 mathematical problems (prompts without solutions), from which we subsample 64 problem seeds per RL iteration" (Section 4). The seed set is "curated from the larger training corpus described in Section 2," meaning it draws from the same filtered seed database used for SFT but only the math subset. Over the 90-step RL training run, "only ∼6k examples" are seen, with 8 trajectory responses generated per example.

The reward function: length-aware accuracy reward. The RL stage's most technically novel component is the reward function, which is explicitly designed to shape both answer correctness and response length. The reward is rule-based—there is no learned reward model—to "avoid complexities and potential reward hacking associated with neural reward models."

The primary reward component is the length-aware accuracy reward, denoted $R_{\text{acc\_scaled}}$. This is computed from two inputs:

• $R_{\text{acc\_raw}} \in \{0, 1\}$: the binary correctness of the extracted final answer, determined by regex-matching \boxed{} tags and comparing against ground truth (with fallback to "external LLM verifiers if simple answer extraction fails")
• $L$: the generation length (number of tokens in the response)

The reward function defines four hyperparameters that control the length-sensitivity:

• $L_{\text{max}} = 31,744$: maximum response length (1,024 tokens reserved for the prompt out of 32,768 total)
• $L_{\text{pos\_control}} = 25,600$: maximum length that does not incur a length penalty for correct answers
• $L_{\text{neg\_control}} = 3,702$: minimum length that does not incur a length penalty for incorrect answers

The computation diverges based on whether the answer is correct or incorrect:

Case 1: Correct answer ($R_{\text{acc\_raw}} = 1$). Define a progress variable:

$\rho^+ = \min\left(1, \frac{\max(L - L_{\text{pos\_control}}, 0)}{L_{\text{max}} - L_{\text{pos\_control}}}\right)$

where $\rho^+$ is a scalar in $[0, 1]$ representing how far beyond the "acceptable" length the response has gone. When $L \leq L_{\text{pos\_control}}$, no penalty applies ($\rho^+ = 0$). When $L = L_{\text{max}}$, the penalty is maximal ($\rho^+ = 1$).

The length-aware accuracy reward is then:

$R_{\text{acc\_scaled}} = R^+_{\text{min}} + 0.5 \cdot (R^+_{\text{max}} - R^+_{\text{min}}) \cdot (1 + \cos(\pi \rho^+))$

where $R^+_{\text{min}} = 0.5$ and $R^+_{\text{max}} = 1.0$.

What this computes: a cosine-scheduled interpolation between 1.0 (short correct answers) and 0.5 (correct answers at or beyond maximum length). The cosine schedule means the penalty is gentle near $\rho^+ = 0$ (short answers are all close to 1.0) and steepens near $\rho^+ = 1$ (the penalty accelerates as length approaches the maximum). The minimum reward for a correct answer is 0.5, meaning a correct answer always receives positive reward regardless of length—the model is never penalized into negative territory for being verbose if it gets the answer right.

Why this form: a linear penalty would make all extra tokens equally costly, which would incentivize the model to cut off reasoning prematurely. The cosine schedule is flat near zero, meaning moderate-length correct answers are not significantly penalized—the model can use a reasonable number of reasoning tokens without cost. Only when answers become excessively long (approaching the 31,744-token limit) does the penalty become substantial. The floor of 0.5 ensures correctness is always rewarded, preventing the model from learning that some correct answers are "not worth it."

Case 2: Incorrect answer ($R_{\text{acc\_raw}} = 0$). Define a progress variable:

$\rho^- = \min\left(1, \frac{L}{L_{\text{neg\_control}}}\right)$

where $\rho^-$ is a scalar in $[0, 1]$ representing what fraction of the "minimum acceptable length for incorrect answers" the response has reached. When $L = 0$, no penalty applies yet (but this is impossible in practice). When $L \geq L_{\text{neg\_control}}$, the penalty is fully realized ($\rho^- = 1$).

The length-aware accuracy reward is then:

$R_{\text{acc\_scaled}} = R^-_{\text{max}} + 0.5 \cdot (R^-_{\text{min}} - R^-_{\text{max}}) \cdot (1 + \cos(\pi \rho^-))$

where $R^-_{\text{min}} = -1.0$ and $R^-_{\text{max}} = -0.5$.

What this computes: a cosine-scheduled interpolation between -0.5 (incorrect answers at or above the minimum acceptable length) and -1.0 (incorrect answers that are very short). The cosine schedule means the penalty for short incorrect answers is severe—if the model gives up quickly and gets the wrong answer, it receives near-maximum negative reward. As the incorrect answer's length approaches $L_{\text{neg\_control}} = 3,702$ tokens, the reward rises to -0.5.

Why this form: the asymmetry with Case 1 is the key design insight. For correct answers, length is discouraged (reward decreases with length). For incorrect answers, length is encouraged (reward increases with length, i.e., becomes less negative). This creates a training signal where the model learns: "if you know the answer, say it concisely; if you don't know the answer, think longer." The threshold $L_{\text{neg\_control}} = 3,702$ is specifically chosen to be relatively short—3,702 tokens is enough for a brief reasoning attempt but not an extended exploration. Incorrect answers that fail to reach this threshold are heavily penalized, pushing the model to invest more tokens before giving an incorrect answer.

Format violation overrides. Three additional rules override the length-aware accuracy reward (Section 4.1):

1. Incompleteness penalty: missing the end-of-sequence token <|im_end|> incurs $R_{\text{acc\_scaled}} = -0.5$.
2. Invalid thinking block penalty: incorrect or missing use of <|thinking|> tag incurs $R_{\text{acc\_scaled}} = -1.0$.

These overrides are applied manually—they bypass the length-aware computation entirely. The rationale is that format violations represent failures to follow the basic output structure, and the severity of such failures should not depend on answer correctness or length.

Repetition penalty. A secondary reward component penalizes repetitive outputs:

$R_{\text{rep}} = -\max\left(\frac{\#\{5\text{-grams with freq.} > 5\}}{\#\{5\text{-grams}\}}, \frac{\max \text{ freq. of } 5\text{-grams with freq.} > 5}{\#\{\text{words}\}/5}\right)$

What this computes: two measures of 5-gram repetition, taking the maximum of (1) the fraction of all 5-grams that appear more than 5 times and (2) the maximum repetition frequency among 5-grams that repeat more than 5 times, normalized by the total number of 5-grams in the response. The result is a negative value (the penalty) whose magnitude increases with repetition severity.

Why this form: repetition is a common failure mode in reasoning model training, where the model can get stuck in loops (repeatedly checking the same condition, restating the problem, or cycling through solution attempts). The 5-gram granularity is chosen to detect phrase-level repetition rather than common-word repetition. The max-of-two-metrics design ensures that both widespread moderate repetition (many different 5-grams repeating a few times) and concentrated severe repetition (a few 5-grams repeating many times) are penalized.

Final reward composition. The final reward is a weighted sum:

$R_{\text{final}} = w_{\text{acc}} R_{\text{acc\_scaled}} + w_{\text{rep}} R_{\text{rep}}$

where $w_{\text{acc}} = \frac{8}{13}$ and $w_{\text{rep}} = \frac{1}{13}$.

What this computes: the accuracy reward contributes roughly 61.5% of the total weight, the repetition penalty contributes roughly 7.7%, and the remaining ~30.8% of the total reward range is effectively unused (the weights sum to $9/13 \approx 0.69$, not 1.0). This means the maximum possible final reward is $(8/13) \times 1.0 \approx 0.615$ for a perfectly correct, non-repetitive, short answer.

Why these weights: the paper does not provide explicit justification for the $8/13$ and $1/13$ choices. The asymmetry ensures that accuracy dominates the reward signal while repetition serves only as a tiebreaker or mild regularizer. The unnormalized weights (summing to less than 1) effectively introduce a constant negative bias—even a perfect answer receives only 0.615 reward rather than 1.0—which may help with training stability by keeping advantage estimates centered.

GRPO training configuration. The RL training uses (Section 4.2):

• Algorithm: Group Relative Policy Optimization (GRPO) [48, 21]
• Framework: verl [49]
• Global batch size: 64 across 32 Nvidia H100 GPUs
• Optimizer: Adam with learning rate $5 \times 10^{-8}$
• Learning rate schedule: cosine warm-up over first 10 steps
• GRPO group size: $G = 8$ (8 trajectory responses per prompt)
• KL regularization coefficient: $\beta = 0.001$
• Entropy coefficient: $\gamma = 0.001$
• Maximum generation length: 31,744 tokens (1,024 reserved for prompt)
• Training duration: 125 steps, with checkpoint at step 90 selected as final

The GRPO objective. The paper reproduces the GRPO objective:

$\frac{1}{G} \sum_{i=1}^{G} \frac{1}{|o_i|} \sum_{t=1}^{|o_i|} \left\{ \min\left[ \frac{\pi_\theta(o_{i,t} | q, o_{i,<t})}{\pi_{\theta_{\text{old}}}(o_{i,t} | q, o_{i,<t})} \hat{A}_{i,t}, \text{clip}\left(\frac{\pi_\theta(o_{i,t} | q, o_{i,<t})}{\pi_{\theta_{\text{old}}}(o_{i,t} | q, o_{i,<t})}, 1 - \epsilon, 1 + \epsilon\right) \hat{A}_{i,t} \right] - \beta D_{\text{KL}}(\pi_\theta || \pi_{\theta_{\text{old}}}) + \gamma \text{Entropy}(\pi_\theta) \right\}$

where $G$ is the number of response trajectories per prompt (8), $o_i$ is the $i$-th trajectory with $|o_i|$ tokens, $\pi_\theta$ is the current policy, $\pi_{\theta_{\text{old}}}$ is the frozen reference policy (the SFT checkpoint), and $\hat{A}_{i,t}$ is the group-relative advantage estimate:

$\hat{A}_{i,t} = \frac{R_{\text{final}}(q, o_i) - \text{mean}(\{R_{\text{final}}(q, o_1), ..., R_{\text{final}}(q, o_G)\})}{\text{std}(\{R_{\text{final}}(q, o_1), ..., R_{\text{final}}(q, o_G)\})}$

What this computes: for each prompt $q$, 8 complete response trajectories are generated. Each trajectory receives a single scalar reward $R_{\text{final}}$ based on the reward function described above. The advantage for each trajectory is its reward minus the mean reward of all 8 trajectories, divided by the standard deviation—this is z-score normalization within the group, meaning the 8 trajectories are compared to each other rather than to an absolute threshold. Trajectories with above-average reward receive positive advantage (policy is reinforced); trajectories with below-average reward receive negative advantage (policy is discouraged). The advantage is broadcast to all tokens in the trajectory equally—every token in a good trajectory receives the same positive signal.

The PPO-style clipped objective ($\min[\text{ratio} \cdot \hat{A}, \text{clip}(\text{ratio}, 1-\epsilon, 1+\epsilon) \cdot \hat{A}]$) prevents the policy from changing too much in a single update. The KL penalty ($-\beta D_{\text{KL}}$) keeps the policy close to the SFT checkpoint. The entropy bonus ($+\gamma \text{Entropy}$) encourages the policy to maintain diversity in its output distribution, preventing mode collapse.

Why GRPO over standard PPO: GRPO eliminates the need for a separate value function (critic) by using the group mean as a baseline. For language model RL, this is practically significant because training a value function that accurately predicts expected future reward from partial sequences is challenging—the value function must understand the relationship between partial reasoning traces and eventual answer correctness. By comparing complete trajectories against each other, GRPO sidesteps this problem entirely. The tradeoff is that the baseline is higher-variance (based on only 8 trajectories rather than a learned estimator), but the PPO clipping and KL regularization compensate.

Training dynamics: length growth and reward plateau. Figure 7 provides a detailed view of the RL training dynamics over 125 steps:

• Accuracy vs. RL steps (Figure 7a): AIME 2024 and AIME 2025 accuracy both improve rapidly in the first 90 steps, then plateau. AIME 2024 rises from approximately 68% to approximately 75%; AIME 2025 rises from approximately 62% to approximately 78%. After step 90, further training "does not translate to additional gains." This early saturation is attributed in part to the 31K-token response clipping—"outputs that are longer than 31k are clipped to their first 31k tokens... which limits the extent to which GRPO can help."

• Accuracy vs. reward (Figure 7b): AIME scores are "weakly correlated with reward." The reward function is a noisy proxy for true performance because it incorporates length penalties and format checks that may not align perfectly with answer correctness.

• Accuracy vs. response length (Figure 7c): There is a strong positive correlation between average response length and AIME accuracy. Longer responses tend to be more accurate—a pattern consistent with the idea that the model is learning to allocate more inference-time compute to harder problems.

• Length growth vs. RL steps (Figure 7d): Response lengths grow throughout training, with incorrect answers growing faster than correct ones. The mean length of incorrect answers rises from approximately 6,000 to approximately 14,000 tokens; the mean length of correct answers rises from approximately 4,000 to approximately 8,000 tokens. The 75th percentile of all responses (which tracks incorrect answers) grows from approximately 8,000 to approximately 22,000 tokens. This divergence is a direct consequence of the asymmetric reward function: incorrect answers are incentivized to be longer (to reduce penalty), while correct answers are incentivized to be shorter (to maximize reward). The model learns to "think more before answering when its current answer is incorrect."

• Reward and clip ratio vs. RL steps (Figure 7e): The final reward plateaus around step 25–50, while the fraction of responses that hit the 31K-token limit and are clipped rises steadily from near 0% to approximately 12% by step 125. This suggests that as training progresses, the model increasingly wants to generate responses longer than the maximum allowed, and clipping becomes a binding constraint.

• Entropy vs. RL steps (Figure 7f): Despite the length clipping and reward plateau, output entropy remains "healthy" and stable (around 0.8–1.0 nats), "suggesting sustained exploration within its solution space." The model is not collapsing to a deterministic policy—it continues to generate diverse outputs, which is important for the approach to continue benefiting from parallel test-time compute (majority voting, best-of-N) at inference time.

Checkpoint selection. The final Phi-4-reasoning-plus model is the checkpoint at step 90, selected based on "the best observed AIME 2024 score." This is an important practical detail: the model is selected based on a validation metric, not the final training step. The paper does not report whether AIME 2024 was explicitly decontaminated from the RL seed data, but given that the RL seeds were drawn from the SFT seed database and the SFT data was decontaminated against AIME 2024, it is likely contamination-free.

3.4.5 Evaluation Methodology: Statistical Rigor and Multi-Run Reporting

The paper does not only contribute a training recipe—it also makes an explicit methodological argument about how reasoning models should be evaluated. This is integrated into the technical approach because the evaluation choices directly affect the interpretation of results and the paper's claims about model comparisons.

The variance problem in small benchmarks. The central methodological concern is that small benchmarks—particularly AIME 2025 (30 problems)—exhibit "large generation nondeterminism" when evaluated with reasoning models at high temperatures (0.6–1.0). The paper demonstrates that "average-of-5 results from two independent runs can differ significantly by up to 5-10 percentage points on AIME 2025, thus making side-by-side comparison of models unreliable" (Section 5.1). This is not a hypothetical concern—it means that a single reported accuracy number like "Phi-4-reasoning-plus: 78.0%" can fluctuate by ±5 percentage points purely due to sampling noise, which is larger than the difference between many models being compared.

The solution: distribution reporting. To address this, the paper adopts several practices:

1. Large number of independent runs for small benchmarks. For AIME 2025, pass@1 accuracy is averaged over 50 independent runs. For HMMT February 2025, 64 runs are used. For GPQA diamond, 64 runs are used. These numbers are substantially larger than the typical 5-run or single-run evaluation.

2. Standard deviation reporting. All results that the authors produce themselves include standard deviations in parentheses (Table 1, Figure 8). For AIME 2025, Phi-4-reasoning is reported as $63.1 (6.3)$, meaning the mean accuracy is 63.1% with a standard deviation of 6.3 percentage points across 50 runs. This explicitly quantifies the reliability of the estimate.

3. Kernel density estimation visualization. Figure 9 shows the full accuracy distribution for several models on AIME 2025, approximated by kernel density estimation from 50 runs. This visual representation makes the variance immediately apparent: DeepSeek-R1-Distill-Llama-70B's accuracy spans from 30% to 70%, while o3-mini's spans from 70% to 100%. The overlap (or lack thereof) between distributions is more informative than point estimates.

4. Per-year breakdowns. Figure 10 shows AIME accuracy broken down by competition year (1983–2025). This reveals systematic difficulty trends that single-number averages obscure: most models perform substantially worse in certain years (e.g., 1994 and 2025), and the difficulty of problems has generally increased over time. This breakdown also serves as a robustness check—if a model's advantage comes primarily from easier earlier years, that is a different claim than if the advantage is consistent across years.

5. Per-topic breakdowns. Figures 15 and 16 show accuracy broken down by mathematical topic (Algebra, Calculus, Discrete Mathematics, Geometry, etc.) and scientific domain (Biology, Chemistry, Physics). These reveal systematic weaknesses: "all models display lower performance on problems related to discrete mathematics and geometry" (Figure 15), and "improvements for all models on biology and chemistry are lower than on physics" (Figure 16). These patterns are consistent across model families, suggesting they reflect fundamental challenges in the benchmarks or in current training approaches rather than model-specific failures.

6. Larger, less saturated benchmarks. The paper emphasizes Omni-MATH (4,428 problems) as a primary evaluation target because it provides more statistical power and "a more diverse benchmark" that reveals weaknesses (like discrete math and geometry performance) that are invisible on smaller benchmarks.

The best-of-N vs. average analysis. Figure 12 (and Figure 17 in the appendix) compares four aggregation strategies for the same five runs: worst-of-5, average pass@1, majority vote, and best-of-5. The gap between average and best-of-5 is substantial for all models—on AIME 2025, Phi-4-reasoning-plus averages 78.0% but achieves 96.7% best-of-5. The paper interprets this gap as evidence that "there may exist even better trajectories in our models' generations revealing potential for further progress" and that better verifiers at training time could extract capabilities that are currently only accessible through expensive parallel sampling. The gap also highlights the practical tradeoff: extracting near-100% accuracy requires running the model 5+ times and selecting the best answer, which multiplies inference cost.

Baseline model evaluation consistency. The paper evaluates all models under "the same evaluation methodology" (Section 5.1) using the Eureka ML Insights framework. For models with undocumented evaluation procedures in their respective reports (e.g., Codeforces for DeepSeek-R1), the authors explicitly note that their numbers may not be directly comparable. Temperature settings follow model card recommendations (0.8 for Phi models, 0.6 for DeepSeek family, 1.0 or API defaults for others). Maximum token lengths are set "as many tokens as possible to all models, within limits that do not present other experimental hurdles such as timeouts from the API" (Section 5.1.1). For AIME, HMMT, GPQA, and Codeforces evaluations, the Phi models use 65,536 maximum tokens—double their training length—"without changing any RoPE parameters," meaning the extended context is an out-of-distribution extrapolation that the paper acknowledges the model "has not properly trained on."

General-purpose benchmark evaluation. In addition to reasoning benchmarks, the paper evaluates on standard general-purpose benchmarks (MMLU, MMLU-Pro, HumanEvalPlus, ArenaHard, IFEval, FlenQA, Kitab, Toxigen, PhiBench) to verify that reasoning training does not degrade general capabilities. Temperature for general-purpose evaluations is 0.8 for the reasoning models, compared to 0.0 for the base Phi-4 (Table 2), meaning the general-purpose numbers are not directly comparable to Phi-4's original reported numbers. The FlenQA benchmark (Section 5.2) is particularly informative: it tests the interaction between reasoning ability and long-context processing by presenting logical reasoning tasks embedded in varying amounts of padding text. Results (Figure 13) show that "reasoning models are more robust to longer inputs compared to conventional models" and "are not affected by the dispersion of key information in the context"—the reasoning process appears to help the model locate and integrate information even when it is distributed across a long document.

Safety evaluation. The safety evaluation uses two frameworks: the automated RAI measurement framework (which simulates adversarial conversations and measures jailbreak success, harmful content generation, and IP leakage) and Toxigen (which measures toxic vs. neutral content detection across 13 demographic groups). The paper notes that "appropriate evaluation of output generated by reasoning models remains a challenge mainly due to the fact that current LLM judges and tools have not yet been optimized for long, step-by-step traces" (Section 5.3). This is presented as a limitation of current evaluation methodology rather than a limitation of the models—the field lacks robust tools for assessing the safety of long, non-linear reasoning traces.

Summary of Design Choices and Their Justifications

• Seed filtering to the "teachable" boundary over training on all available seeds: ensures the model learns from examples where it has foundational knowledge but hasn't yet mastered the reasoning process, maximizing the learning signal per example.

• o3-mini as teacher over other strong teachers: empirically comparable to DeepSeek-R1 for teaching quality but more token-efficient in medium-effort mode, with high-effort mode available for stronger (but longer) demonstrations when needed.

• Structured thinking/response format over free-form reasoning: enables downstream selective display of reasoning, facilitates answer extraction for RL verification, and teaches the model to separate exploration (thinking) from presentation (response).

• Independent per-domain data mixture optimization over joint optimization: the empirically observed additive property means domain mixtures can be tuned independently and concatenated, dramatically reducing the search space.

• SFT before RL rather than RL-only: SFT establishes the reasoning format and broad generalization; RL refines the length-quality tradeoff. The decreasing response length during SFT and increasing length during RL confirm these phases serve different functions.

• Length-aware asymmetric reward for RL over uniform correctness reward: incentivizes concise correct answers and thorough incorrect answers, teaching the model to dynamically allocate inference compute based on problem difficulty.

• Cosine-scheduled reward scaling over linear scaling: provides flat penalty near the "acceptable" length threshold (avoiding premature truncation) and accelerating penalty near the maximum (preventing runaway length growth).

• GRPO with group-relative advantages over PPO with learned value function: eliminates the need for training a critic that can evaluate partial reasoning traces, at the cost of higher-variance advantage estimates compensated by PPO clipping and KL regularization.

• Multi-run statistical rigor in evaluation over single-run reporting: addresses the documented nondeterminism of reasoning model outputs, particularly on small benchmarks where sampling variance can exceed the performance gap between competing models.

• RoPE base frequency doubling over interpolation or architectural change: a minimal modification to extend context length from 16K to 32K without changing model architecture or requiring extensive continued pretraining.

4. Key Insights and Innovations

Innovation 1: "Teachable" Seed Selection as the Central Mechanism for Reasoning Transfer

The paper's most intellectually distinctive contribution is not any single algorithmic choice but rather the concept of "teachable" seeds as the discriminating factor between reasoning models that genuinely generalize and those that merely pattern-match on training distributions. The field has largely assumed that distillation quality is a function of teacher quality and data quantity—use a strong enough teacher, generate enough examples, and the student will learn. This paper challenges that framing at its root by demonstrating that the filtering boundary matters as much or more than the teacher's absolute strength.

What makes this a conceptual innovation rather than an implementation detail is the operationalization of "teachability." Prior work on data filtering for reasoning has typically relied on difficulty proxies (correctness rate, problem source prestige, benchmark labels) or heuristic quality filters (length, format, diversity). These are input-side criteria: they ask "is this a good problem?" The paper flips this to an interaction-side criterion: "does this problem sit at the boundary between what the student model can and cannot do?" This shifts the filtering problem from absolute quality assessment to relative capability gap estimation—a fundamentally different optimization target.

The practical manifestation is the agreement-gap protocol (Section 2.1), where seeds are retained only if a weaker model (the student) disagrees with a stronger model (the reference) at a non-trivial but non-random rate. This is Vygotsky's "zone of proximal development" operationalized for language model training: the productive examples are those the model cannot solve independently but has the foundational knowledge to learn from when demonstrated. Seeds where agreement is near-perfect are already mastered—training on them wastes compute without improving capability. Seeds where agreement approaches random chance are beyond the model's conceptual reach—the gap is too large for a single demonstration to bridge. Only the intermediate region produces learning.

The paper's evidence for this claim is not a controlled ablation (the authors did not train identical models on unfiltered vs. filtered seeds), but the pattern of generalization tells the story. The models show 30–60 percentage point improvements on tasks like TSP, 3SAT, and calendar planning (Figure 8)—domains that "were not intentionally targeted" during SFT or RL. This level of out-of-domain transfer is unusual for distillation approaches and strongly suggests the model learned reasoning as a strategy rather than a collection of domain-specific templates. The "teachable" filter plausibly selects for seeds where the reasoning process itself is the learning objective—not the specific answer, not the domain facts, but the decomposition-verification-reflection loop.

This finding also explains a puzzle in the distillation literature: why some models distilled from strong teachers generalize poorly while others generalize well, even when trained on similar quantities of data. If the seed filtering is the critical variable, then two teams training on R1-distilled data could get dramatically different results depending on whether their seed curation pipeline selects for the boundary region. The paper's claim that seed selection matters more than teacher selection (within a quality band) is a concrete, falsifiable hypothesis that future work can test.

Significance: This is a fundamental shift in how to think about reasoning data curation—from "more and better" to "precisely at the frontier." If the finding replicates, it changes the practical recipe for distillation from "collect good problems and generate teacher traces" to "measure your student model's capability distribution and select problems at the boundary."

Innovation 2: SFT and RL as Complementary Phases with Opposite Length Dynamics

The paper's second conceptual contribution is the empirical demonstration that supervised fine-tuning and reinforcement learning serve complementary, non-interchangeable functions in reasoning model training, with their distinct roles revealed through the counterintuitive behavior of response length.

The standard narrative in reasoning model development—particularly following DeepSeek-R1—is that RL is the primary driver of reasoning capability, with SFT serving as a preliminary warmup or format-instruction phase. The paper provides concrete evidence that this narrative misses a key dynamic: SFT teaches the model to reason efficiently, while RL teaches the model when to override efficiency in favor of thoroughness. This is not a subtle distinction—it has direct implications for how training compute should be allocated between phases and what each phase can realistically achieve.

The evidence is Figure 4b versus Figure 7d. During SFT, response lengths decrease—the model learns to produce correct answers in fewer tokens as training progresses. The mean AIME 2024 response drops from ~20,000 tokens at step 2,500 to ~13,000–14,000 by step 15,000. The paper interprets this as "learning to use its token budget more efficiently." During RL, response lengths increase—incorrect answers grow from ~6,000 to ~14,000 tokens, correct answers grow from ~4,000 to ~8,000 tokens. This is a direct consequence of the asymmetric reward function, which penalizes short incorrect answers more heavily than long incorrect answers, creating a gradient toward more thorough reasoning.

What makes this insight novel is that it resolves a tension in the literature about whether reasoning models should be trained to be concise or verbose. The answer, per this paper, is both, in sequence, for different reasons. SFT establishes the baseline: "here is what good reasoning looks like, and it can be done concisely." RL adds the meta-cognitive layer: "but when you're uncertain, invest more tokens before committing to an answer." The two phases are not redundant—removing either would produce a qualitatively different model. An SFT-only model would be efficient but lack the adaptive thoroughness that RL teaches. An RL-only model (starting from a base without structured reasoning format training) would lack the efficient reasoning baseline to build on—it would learn to generate long sequences but might not learn to structure them productively.

The paper's RL design—specifically the asymmetric length-aware reward—is the mechanism that makes this complementarity legible. The cosine-scheduled penalties for correct answers (gently discouraging excessive length) and for incorrect answers (sharply penalizing premature termination) create a training signal where the model learns a conditional compute policy: allocate more tokens to harder problems. This is fundamentally different from DeepSeek-R1's approach, which relied on emergent "aha moments" and self-verification behaviors that appeared spontaneously during RL without explicit length shaping. The paper's approach makes the length tradeoff an explicit optimization target rather than an emergent side effect.

The saturation behavior (Figure 7a) adds an important boundary condition: RL on top of high-quality SFT plateaus quickly—~90 steps, ~6K examples—after which further training does not improve and may degrade due to response clipping. This suggests that the marginal value of RL depends on the quality of the SFT starting point. A weaker SFT model might benefit from more RL; a stronger SFT model saturates early. This is a concrete, testable claim about the interaction between training phases that prior work has not quantified.

Significance: This finding reframes the SFT-vs-RL debate from "which is better" to "what does each contribute and in what order should they be applied?" It also provides practical guidance: if you have a limited RL budget, invest it after SFT rather than before or instead of SFT, and expect early saturation if your SFT model is already strong.

Innovation 3: Agreement-Gap Difficulty Estimation as a Domain-Agnostic Proxy for Ground Truth

The paper's third conceptual contribution is the agreement-gap protocol—a method for estimating problem difficulty without requiring ground-truth labels—and its demonstrated sufficiency for guiding data curation decisions.

This matters because the "teachable seed" concept (Innovation 1) requires knowing which problems the student model can and cannot solve. In domains with verifiable answers (math, coding with test cases), this is straightforward: measure pass@1 against ground truth. But for the majority of reasoning domains—scientific explanation, logical argumentation, planning, safety reasoning—ground truth is either unavailable or inherently ambiguous. Without a difficulty estimation method that works across domains, the teachable seed approach would be restricted to verifiable domains, severely limiting its applicability.

The agreement-gap protocol solves this by using model consensus as a proxy for correctness. A strong reference model's plurality response serves as a pseudo-ground-truth, and a weaker model's agreement rate with that pseudo-ground-truth serves as a difficulty proxy. This is not a new idea in NLP broadly—ensemble agreement has been used for uncertainty estimation, data filtering, and label quality assessment. What is novel is its application to reasoning curriculum design for language model training, where the goal is not to estimate uncertainty at inference time but to select training examples that maximize learning.

The protocol's validity rests on an assumption: that the strong reference model is correct (or at least consistent) on problems where the student model is uncertain. If the reference model makes the same errors as the student, the agreement rate would be high for the wrong reasons, and truly teachable seeds would be filtered out as "already mastered." The paper does not directly validate this assumption, but the generalization results (30–60 point gains on out-of-domain algorithmic tasks) provide indirect support: if the filtering were systematically misclassifying seeds, the generalization pattern would be different.

A subtle but important feature of the protocol is that it implicitly captures problem difficulty relative to the student model's specific capability profile, not absolute difficulty. A problem that is easy for Phi-4 but hard for a weaker model would have a large agreement gap and be classified as "teachable"—but it wouldn't actually teach Phi-4 anything new. The protocol is only valid when the reference model is substantially stronger than the student, which the paper ensures by using frontier models (likely o3-mini) as references for Phi-4.

Significance: This is a practical enabling technology for the teachable seed approach. Without it, difficulty-based filtering is restricted to domains with ground truth. With it, the entire pipeline can be applied to open-ended reasoning domains where correctness is ambiguous—exactly the domains where reasoning models are most needed but hardest to train.

Innovation 4: Variance-Aware Evaluation as a Prerequisite for Reliable Reasoning Model Comparison

The paper's fourth contribution is methodological rather than modeling-focused: the argument that single-run, single-average accuracy reporting on small benchmarks is scientifically unreliable for reasoning models, and the demonstration of what rigorous evaluation should look like instead.

This is not merely a critique of sloppy benchmarking. The paper provides concrete evidence that the current standard practice—reporting one accuracy number from one evaluation run—can produce ordinal ranking errors. Two independent sets of 5 runs on AIME 2025 can differ by 5–10 percentage points (Section 5.1.2), which exceeds the performance gap between many models being compared. A model reported at 75% might genuinely be 70% or 80%, and a model reported at 72% might genuinely be 67% or 77%—the confidence intervals overlap heavily, making the reported ranking uninformative.

The paper's solution is multi-dimensional: large numbers of independent runs (50–64 for small benchmarks), standard deviation reporting, kernel density visualization of accuracy distributions (Figure 9), per-year and per-topic disaggregation (Figures 10, 15, 16), and explicit comparison of best-of-N vs. average performance (Figure 12). Each of these practices addresses a different failure mode of single-number reporting: variance, distribution shape, systematic difficulty trends, and the gap between typical and optimal performance.

The kernel density visualization (Figure 9) is particularly illuminating. It shows that models with similar mean accuracy can have qualitatively different reliability profiles. DeepSeek-R1-Distill-Llama-70B has a broad, almost uniform distribution from 30% to 70%—it sometimes performs excellently and sometimes performs poorly, with high variance. Phi-4-reasoning-plus has a narrower distribution around 78%—it is more consistent but has a lower ceiling than o3-mini (which spans 70–100%). This distribution shape information is lost in mean-only reporting but has direct practical implications: for applications requiring reliability (you need a guaranteed minimum performance), narrower distributions are preferable even if the mean is slightly lower.

The best-of-N vs. average gap analysis (Figure 12) reveals a different insight: all models have substantial "untapped potential"—correct answers exist in their generation distributions that simple pass@1 sampling misses. Phi-4-reasoning-plus averages 78% on AIME 2025 but achieves 96.7% best-of-5. This gap is both encouraging (better verifiers or decoding could unlock near-perfect performance) and sobering (extracting that performance requires 5× inference compute). By reporting both numbers, the paper surfaces the inference-compute tradeoff that pass@1-only reporting hides.

The paper's contribution is not the individual evaluation techniques—multi-run evaluation, standard deviation reporting, and disaggregated analysis all predate this work. The contribution is the integrated argument that these practices are not optional quality-of-life improvements but necessary conditions for valid inference when comparing reasoning models. The paper demonstrates this by showing that its own conclusions would be different (or at least less reliable) under standard single-run reporting. This is a meta-scientific contribution: it changes what counts as adequate evidence in reasoning model research.

Significance: If adopted by the field, these evaluation practices would eliminate a major source of irreproducibility and enable more reliable identification of genuine progress. The paper's willingness to implement these practices for its own models—reporting standard deviations, showing distributions, acknowledging variance—establishes credibility for the specific performance claims while also raising the bar for future work.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The primary reasoning benchmarks are AIME 2025 (30 problems, released after training data finalization and thus contamination-free), AIME 1983–2024 (949 problems), HMMT February 2025 (30 problems), Omni-MATH (4,428 olympiad-level problems with human annotations), GPQA Diamond (198 graduate-level science questions), LiveCodeBench (2024-08 through 2025-01 problems), and Codeforces (143 problems from contest IDs 1505–1536). The paper also uses a comprehensive suite of additional benchmarks from the Eureka ML Insights repository: BA-Calendar (2,000 calendar planning problems), TSP-Opt and 3SAT-Search (new benchmarks for NP-hard problem solving with 960 and 800 samples respectively), Maze (1,500 spatial reasoning questions on 10×10 mazes), and SpatialMap (1,500 relational reasoning questions). For general-purpose evaluation, the paper uses MMLU, MGSM, MMLU-Pro, HumanEvalPlus, ArenaHard, FlenQA (12K questions, 250–3000 tokens each), Kitab (information retrieval with constraint filters), IFEval, Toxigen, and PhiBench 2.21. Decontamination was performed against all benchmarks explicitly listed in Section 2.2, with AIME-2024 explicitly included; AIME-2025 was released after the training data was finalized and is contamination-free.

• Base model. The paper evaluates three model checkpoints from the same lineage: Phi-4 (the base 14B-parameter model, used as a pre-reasoning-training baseline), Phi-4-reasoning (SFT-trained on the curated 1.4M-example dataset), and Phi-4-reasoning-plus (further trained with 90 steps of GRPO on ~6K math problems). All three share the same architecture and parameter count (14B). The base model is evaluated at temperature 0.8 for reasoning benchmarks and 0.0 for general-purpose benchmarks; the reasoning models are evaluated at temperature 0.8 for all benchmarks. For AIME, HMMT, GPQA, and Codeforces, the reasoning models are evaluated with a maximum token limit of 65,536 (despite being trained for 32K), a deliberate extrapolation to test robustness; all other evaluations use 32,768 max tokens.

• Metrics. The primary metric throughout is pass@1 accuracy — the fraction of test problems for which a single sampled generation produces the correct answer. For benchmarks with small problem counts (AIME 2025, HMMT, GPQA), accuracy is averaged over 50 or 64 independent runs rather than the conventional 5, and standard deviations are reported. For math reasoning, correctness is determined by extracting the final answer (typically from \boxed{} tags) and checking against ground truth via the grading functions from MathArena or Eureka ML Insights. For LiveCodeBench and Codeforces, standard automated evaluation protocols are used. For general-purpose benchmarks, the paper reports standard accuracy as defined by each benchmark's evaluation script. The paper also reports token usage (mean generation length in tokens, with standard deviations computed per-instance and then averaged) as a secondary metric for the accuracy-vs-efficiency tradeoff analysis (Figure 11). For the best-of-N analysis, the paper reports worst-of-5, average pass@1, majority vote, and best-of-5 on the same five independent runs (Figures 12 and 17).

• Baselines. The paper compares against an extensive set of models spanning open-weight and proprietary families. Open-weight baselines include: DeepSeek-R1 (671B Mixture-of-Experts, temperature 0.6, max 65,536 tokens), DeepSeek-R1-Distill-Llama-70B (temperature 0.6, max 65,536 tokens), OpenThinker2-32B, QwQ-32B, EXAONE-Deep-32B, and DeepSeek-R1-Distill-Qwen-14B (only for Codeforces validation). Proprietary baselines include: o1-mini, o1, o3-mini-high (all via API, with o1 using a plain non-CoT prompt to avoid policy violation refusals), Claude 3.7 Sonnet (with thinking enabled, temperature 1.0, max 32,768 tokens), Gemini 2 Flash Thinking (temperature 1.0, max 32,768 tokens), Gemini 2.5-Pro, and GPT-4o (temperature 1.0, max 4,096 tokens). Baseline numbers for LiveCodeBench and Codeforces are reproduced from the DeepSeek-R1 report or LiveCodeBench leaderboard where available; HMMT February 2025 numbers are from MathArena leaderboards; OpenThinker2, QwQ, EXAONE-Deep, and Gemini 2.5-Pro numbers are reproduced from their respective technical reports. All models are evaluated using the same prompts and evaluation pipelines from the Eureka ML Insights framework, with the exception that o1 uses a non-CoT prompt template.

• Generation budget / compute accounting. The paper does not use a formal FLOPs-based compute accounting framework (unlike the inference-time scaling work in the reference example). Instead, it measures computational cost along two axes: token usage (mean generation length, used for accuracy-vs-efficiency analysis in Figure 11) and inference-time compute via parallel sampling (number of independent generations used for majority voting or best-of-N, analyzed in Figure 2 and Figures 12/17). There is no direct comparison of training compute (SFT + RL) to inference compute, and no FLOPs-matched analysis between model sizes. The efficiency analysis is therefore primarily about the tradeoff between accuracy and tokens consumed at inference time, with the models compared at fixed generation counts (pass@1) rather than fixed FLOPs.

• Cross-validation / statistical protocol. No formal cross-validation is described in the evaluation section (the two-fold cross-validation described in Section 3 was part of the training recipe validation, not the final evaluation). The primary statistical protocol is: for small benchmarks (AIME 2025, HMMT, GPQA), accuracy is computed over 50 or 64 independent runs rather than the conventional 5, with standard deviations reported; for larger benchmarks (Omni-MATH, BA-Calendar, TSP, 3SAT, Maze, SpatialMap), 5 independent runs are used; for general-purpose benchmarks, 5 runs are used. All results produced by the authors include standard deviations in parentheses (Table 1, Figures 8 and 11). The paper explicitly argues that single-run or average-of-5 reporting on small benchmarks is statistically unreliable due to the high nondeterminism of reasoning model outputs (illustrated in Figure 9 via kernel density estimation from 50 runs), and that comparisons between models should be based on accuracy distributions rather than point estimates.

Main Quantitative Results

Reasoning Benchmark Performance: Aggregate Results

The headline result is that Phi-4-reasoning and Phi-4-reasoning-plus achieve performance competitive with or exceeding models ~5–50× their size across a broad range of reasoning tasks. On AIME 2025, the key contamination-free benchmark, Phi-4-reasoning achieves 63.1% (standard deviation 6.3% across 50 runs) and Phi-4-reasoning-plus achieves 78.0% (4.6%), compared to 12.9% for the base Phi-4, 51.5% (5.8%) for DeepSeek-R1-Distill-Llama-70B, 70.4% (4.3%) for the full 671B-parameter DeepSeek-R1, 82.5% (4.9%) for o3-mini-high, and 71.4% (5.7%) for o1 (Table 1, Figure 1, Figure 8). This represents a gain of ~50.2 percentage points over the base model for Phi-4-reasoning and ~65.1 points for Phi-4-reasoning-plus.

On AIME 1983–2024, a larger benchmark with 949 problems, Phi-4-reasoning achieves 74.6% (5.1%) and Phi-4-reasoning-plus achieves 81.3% (1.8%), compared to 79.5% for QwQ-32B, 69.3% (2.7%) for DeepSeek-R1-Distill-Llama-70B, 78.7% (3.8%) for DeepSeek-R1, and 88.0% (5.5%) for o3-mini-high (Table 1). Note the lower standard deviation for Phi-4-reasoning-plus (1.8% vs. 5.1% for Phi-4-reasoning), indicating more consistent performance.

On Omni-MATH (4,428 problems, a substantially larger and more diverse math benchmark designed to reveal model weaknesses beyond what AIME captures), Phi-4-reasoning achieves 76.6% (0.5%) and Phi-4-reasoning-plus achieves 81.9% (0.1%), compared to 31.9% for Phi-4, 63.4% (0.4%) for DeepSeek-R1-Distill-Llama-70B, 85.0% (0.6%) for DeepSeek-R1, 67.5% (0.9%) for o1, and 74.6% (5.1%) for o3-mini-high (Table 1, Figure 8). The near-zero standard deviations on this larger benchmark confirm that the high variance observed on AIME is primarily a small-sample phenomenon. Notably, Phi-4-reasoning-plus outperforms o3-mini-high on this benchmark (81.9% vs. 74.6%) while being a significantly smaller model, and approaches the performance of the full DeepSeek-R1 (81.9% vs. 85.0%). This is a stronger claim than the AIME results because Omni-MATH is both larger and more diverse, covering algebra, applied mathematics, calculus, discrete mathematics, geometry, number theory, and precalculus.

On GPQA Diamond, Phi-4-reasoning achieves 67.1% (2.7%) and Phi-4-reasoning-plus achieves 69.3% (2.1%), compared to 54.7% for Phi-4, 66.2% (2.4%) for DeepSeek-R1-Distill-Llama-70B, 73.0% (1.7%) for DeepSeek-R1, 76.7% (1.8%) for o1, and 77.7% (0.6%) for o3-mini-high (Table 1, Figure 8). The GPQA results also show the breakdown by scientific domain (Figure 16): all models perform significantly better on physics (e.g., Phi-4-reasoning-plus at 88.9%) than on biology (54.1%) or chemistry (50.5%), a pattern that is consistent across model families and suggests a shared limitation in biology and chemistry reasoning that no current model overcomes.

On coding benchmarks, Phi-4-reasoning achieves 53.8% on LiveCodeBench (2024-08 through 2025-01) and a Codeforces Elo of 1,736, while Phi-4-reasoning-plus achieves 53.1% and 1,723 respectively (Table 1). These are comparable to o1-mini (53.8%, Elo 1,650) and slightly below DeepSeek-R1-Distill-Llama-70B (57.5%, Elo 1,633), but substantially below the full DeepSeek-R1 (65.9%, Elo 2,029) and o3-mini-high (68.8%, Elo 2,130). The fact that Phi-4-reasoning-plus performs slightly worse than Phi-4-reasoning on LiveCodeBench is notable and consistent with the RL stage being focused exclusively on math—the paper acknowledges this explicitly: "the seed data contained no coding exercises, as perhaps evident by the LiveCodeBench scores of our model" (Section 4).

Out-of-Domain Generalization: Algorithmic Reasoning, Planning, and Spatial Tasks

The paper's most striking results are on tasks that "were not intentionally targeted" during SFT or RL—algorithmic problem solving (TSP-Opt, 3SAT-Search), calendar planning (BA-Calendar), and spatial reasoning (Maze, SpatialMap). These tasks are presented as evidence that the models have acquired reasoning as a transferable meta-skill.

On BA-Calendar (calendar planning with 2,000 problems involving time zone constraints, buffer times, participant availability, and priority scheduling), Phi-4-reasoning achieves 67.7% and Phi-4-reasoning-plus achieves 65.6%, compared to 16.0% for Phi-4, 66.8% for DeepSeek-R1-Distill-Llama-70B, 79.2% for DeepSeek-R1, 86.1% for o1, and 62.0% for o3-mini-high (Figure 8). This is a gain of ~51.7 percentage points over the base model, despite the task having no explicit representation in the training data. The finding that Phi-4-reasoning (SFT-only) slightly outperforms Phi-4-reasoning-plus (67.7% vs. 65.6%) on this task is consistent with the RL stage being math-focused and potentially specializing the model in ways that slightly reduce its versatility on non-math reasoning.

On TSP-Opt (Traveling Salesman Problem instances requiring optimal route finding, 960 problems), Phi-4-reasoning achieves 37.5% and Phi-4-reasoning-plus achieves 42.6%, compared to 8.2% for Phi-4, 32.4% for DeepSeek-R1-Distill-Llama-70B, 46.7% for DeepSeek-R1, 45.8% for o1, and 56.4% for o3-mini-high (Figure 8). This is a gain of ~29.3–34.4 points over the base model. On 3SAT-Search (800 problems for 3-literal Satisfiability), a representative NP-hard problem, Phi-4-reasoning achieves 78.0% and Phi-4-reasoning-plus achieves 70.9%, compared to 17.1% for Phi-4, 72.2% for DeepSeek-R1-Distill-Llama-70B, 80.7% for DeepSeek-R1, 89.0% for o1, and 96.1% for o3-mini-high (Figure 8, labeled "SAT"). The pattern of Phi-4-reasoning outperforming Phi-4-reasoning-plus on this task (78.0% vs. 70.9%) echoes the BA-Calendar result and reinforces the specialization tradeoff.

On spatial reasoning tasks, Phi-4-reasoning achieves 55.1% on Maze (1,500 problems on 10×10 mazes requiring counting turns, determining spatial relationships) and 73.7% on SpatialMap (1,500 problems requiring relational reasoning about object positions), while Phi-4-reasoning-plus achieves 53.4% and 73.3% respectively (Figure 8). The base Phi-4 achieves 51.4% and 61.3%—the gains on Maze are modest (3–4 points) while gains on SpatialMap are substantial (12 points). This asymmetry suggests that the types of spatial reasoning required by these tasks differ in how well they benefit from chain-of-thought training. Curiously, DeepSeek-R1-Distill-Llama-70B performs worse than the base Phi-4 on Maze (45.2% vs. 51.4%), indicating that reasoning training does not universally improve spatial reasoning.

Difficulty-Level Analysis

Figure 14 provides accuracy and token usage breakdowns by difficulty level for TSP, BA-Calendar, and Omni-MATH. The key finding is that Phi-4-reasoning-plus offers improvements primarily at easy-to-mid difficulty levels (L1–L5), with accuracy degrading at higher difficulty levels for all models. On TSP, both Phi-4-reasoning models show strong accuracy on easy instances (L1–L3, >60%) but decline to near-zero on the hardest instances (L7–L8), a pattern shared with all evaluated models. On BA-Calendar, accuracy is strongly correlated with problem constrainedness (a measure of how many calendar constraints must be satisfied), with all models declining at high constrainedness levels. On Omni-MATH, the difficulty tags are acknowledged as potentially "noisy and imperfect" with "imbalanced" data per level, but the general pattern of decline at higher difficulty is consistent.

The Omni-MATH topic-level analysis (Figure 15) reveals that "all models display lower performance on problems related to discrete mathematics and geometry." For Phi-4-reasoning-plus, accuracy on algebra is approximately 88%, on number theory approximately 86%, on calculus approximately 82%, but on discrete mathematics approximately 58% and on geometry approximately 62%. This pattern is consistent across all models—even o3-mini-high shows a drop from ~94% on algebra to ~72% on discrete mathematics—suggesting these are genuinely harder reasoning categories rather than model-specific weaknesses.

The GPQA domain-level analysis (Figure 16) shows a similar pattern: all models are substantially stronger on physics (Phi-4-reasoning-plus: 88.9%) than on chemistry (50.5%) or biology (54.1%). The token usage analysis in Figure 16 also shows that models consume more tokens on physics questions (Phi-4-reasoning-plus: 14,155 mean tokens) than on biology (7,600) or chemistry (6,528), suggesting that model confidence (as reflected in reasoning length) correlates with domain difficulty.

AIME 2025 Accuracy Distributions and Variance Analysis

Figure 9 presents kernel density estimates of the accuracy distribution for six models on AIME 2025 from 50 independent runs. The key observations:

• Phi-4-reasoning-plus shows a relatively concentrated distribution centered around 78%, with a range spanning from approximately 67% to 90%. The distribution is narrower than DeepSeek-R1-Distill-Llama-70B (which spans 30–70%) but broader than Phi-4 (which is concentrated near 13%).
• DeepSeek-R1-Distill-Llama-70B has an extremely broad, almost uniform distribution from 30% to 70%, indicating very high run-to-run variability. Two independent sets of 5 runs could easily produce means differing by 10–15 percentage points.
• o3-mini-high has a distribution spanning 70–100%, largely overlapping with Phi-4-reasoning-plus in the 75–90% range but extending to higher maximum values.
• DeepSeek-R1 (671B) has a distribution centered around 70% that is mostly disjoint from Phi-4-reasoning-plus's distribution (which has higher mean) but overlaps with o3-mini-high's lower tail.
• The paper notes that "models on the two extremes of average accuracy demonstrate more robust accuracy"—Phi-4 and Phi-4-reasoning-plus have narrower distributions than the intermediate models.

Figure 10 shows AIME performance broken down by competition year from 1983 to 2025. All models show substantial performance variation across years, with "most models perform substantially worse in 1994 and 2025." There is a general downward trend in accuracy over time, consistent with problems becoming harder in more recent competitions. The performance ranking between models is largely consistent across years, though the gaps vary.

Token Usage vs. Accuracy Tradeoffs

Figure 11 presents accuracy vs. mean token count for all models across nine benchmarks. The key findings:

• Phi-4-reasoning-plus consistently generates longer responses than Phi-4-reasoning. On average across reasoning benchmarks, Phi-4-reasoning-plus's generations are ~1.5× longer. The gap is largest on AIME (both 2025 and older), GPQA, and Omni-MATH, and smallest on Maze, SpatialMap, and 3SAT.
• For math benchmarks, longer responses correlate with higher accuracy across models. On AIME 2025, o3-mini-high achieves the highest accuracy (~82.5%) with the longest mean token count (~20,000–25,000), while Phi-4 achieves the lowest accuracy with the fewest tokens. Phi-4-reasoning-plus sits at an intermediate point: higher accuracy than Phi-4-reasoning but with more tokens.
• For non-math reasoning tasks (Maze, SpatialMap, BA-Calendar), the correlation breaks down. On Maze, all models generate similar token counts (5,000–15,000) despite varying accuracy (45–80%). On SpatialMap, Claude 3.7 Sonnet and Gemini 2 Flash Thinking generate fewer tokens than the reasoning models but achieve competitive accuracy. On BA-Calendar, o1 achieves 86.1% with fewer tokens than o3-mini-high (62.0%), which has the longest generations.
• The horizontal error bars (standard deviation of token length per instance) are large for reasoning models, indicating that token usage varies substantially across prompts within the same benchmark—the models allocate tokens dynamically based on perceived difficulty.

General-Purpose Benchmark Results

Table 2 presents results on general-purpose benchmarks, with the key finding that reasoning training produces "non-trivial and often large benefits" on tasks not explicitly targeted during SFT or RL:

• FlenQA (3K-token subset, testing long-context logical reasoning): Phi-4-reasoning achieves 97.7% and Phi-4-reasoning-plus achieves 97.9%, compared to 82.0% for Phi-4 and 90.8% for GPT-4o. Both reasoning models are competitive with o3-mini (96.8%). Figure 13 shows that reasoning models maintain accuracy at longer context lengths (2,500–3,000 tokens) where conventional models degrade, and are unaffected by the location of key information in the context (first, last, middle, or random dispersion), while conventional models struggle with randomly dispersed information.

• IFEval Strict (instruction following): Phi-4-reasoning achieves 83.4% and Phi-4-reasoning-plus achieves 84.9%, compared to 62.3% for Phi-4, 81.8% for GPT-4o, and 91.5% for o3-mini. This is a gain of ~21–22 points over the base model on a task that measures compliance with formatting and content constraints.

• ArenaHard: Phi-4-reasoning achieves 73.3% and Phi-4-reasoning-plus achieves 79.0%, compared to 68.1% for Phi-4 and 69.0% for GPT-4o (Table 2). The 10-point gain from Phi-4 to Phi-4-reasoning-plus is attributed to improved reasoning in chat-like interaction scenarios.

• HumanEvalPlus (code generation): Phi-4-reasoning achieves 92.9% and Phi-4-reasoning-plus achieves 92.3%, compared to 83.5% for Phi-4 and 84.9% for GPT-4o.

• MMLUPro: Phi-4-reasoning achieves 74.3% and Phi-4-reasoning-plus achieves 76.0%, compared to 71.5% for Phi-4 and 73.5% for GPT-4o. The 3–5 point gain is modest but consistent.

• Kitab (information retrieval with constraint filters): With retrieval context provided, Phi-4-reasoning achieves precision 93.8% / recall 74.8%, and Phi-4-reasoning-plus achieves 93.6% / 75.4%, compared to Phi-4's 88.5% / 68.1% and o3-mini's 94.0% / 76.1%. Without context (parametric knowledge only), both reasoning models show improved precision (23.2% and 27.6% vs. 19.3% for Phi-4) but degraded recall (4.9% and 6.3% vs. 8.2% for Phi-4), a pattern the paper notes as "challenging for our models, potentially due to their significantly smaller scale."

• Toxigen Discriminative (toxic language detection): Phi-4-reasoning achieves 86.7% on toxic category detection and 84.7% on neutral category detection—a more balanced profile than Phi-4 (72.6% toxic, 90.0% neutral) and Phi-4-reasoning-plus (77.3% toxic, 90.5% neutral). The paper notes that "lower imbalance is also a preferable behavior for applications such as content filtering and moderation." The group-level breakdown (Figure 18b) shows that Phi-4-reasoning has "narrowed some of the group-based discrepancies, previously observed in Phi-4."

• PhiBench 2.21: Phi-4-reasoning achieves 70.6% and Phi-4-reasoning-plus achieves 74.2%, compared to 58.2% for Phi-4 and 73.1% for GPT-4o.

Parallel Test-Time Compute Scaling

Figure 2 shows the effect of parallel test-time compute on AIME 2025 via majority voting (Maj@N). Phi-4-reasoning improves from approximately 63% at N=1 to approximately 87% at N=64, while Phi-4-reasoning-plus improves from approximately 78% to approximately 95% (best-of-64). Notably, the Maj@64 performance of Phi-4-reasoning-plus (approximately 90–92%, reading from Figure 2) surpasses o3-mini's pass@1 (~82.5%), and the Best-of-64 performance (~95%) approaches the ceiling of what the 30-problem benchmark can distinguish. The figure also shows that at N=1, Phi-4-reasoning-plus is at approximately 78% while o3-mini is at approximately 82.5%, but at higher N, the gap narrows and eventually the Phi models exceed o3-mini's single-generation performance.

Figures 12 and 17 show the gap between worst-of-5, average pass@1, majority vote, and best-of-5 across all reasoning benchmarks. For AIME 2025, Phi-4-reasoning-plus averages 78.0% but achieves 96.7% best-of-5—a gap of 18.7 percentage points. Similar gaps exist across models and benchmarks: DeepSeek-R1 averages 70.4% but achieves 93.3% best-of-5 on AIME 2025; o3-mini-high averages 82.5% but achieves 96.7% best-of-5. The gap between average and best-of-5 is interpreted as "potential for further progress" that could be unlocked through improved verifiers or decoding strategies.

Per-Year AIME Performance Trends

Figure 10 reveals performance variation across AIME competition years (1983–2025) from 5 independent runs. The key observations:

• All models show substantial year-to-year variance, with a general downward trend in accuracy for more recent years. The paper specifically notes: "most models perform substantially worse in 1994 and 2025."
• The relative ranking between models is mostly preserved across years, though with some reversals. For example, o3-mini-high consistently outperforms Phi-4-reasoning-plus across most years, but there are years (e.g., 1990, 2008, 2022) where the gap narrows substantially.
• The "hardest" years—where all models perform poorly—are consistent across model families, suggesting genuine differences in problem difficulty rather than model-specific weaknesses.

Ablation Studies and Robustness Checks

The paper does not report formal ablation studies in the standard sense (controlling for one variable while holding others constant in a systematic comparison). However, several implicit ablations and robustness analyses can be extracted from the paper's experimental narrative:

Teacher model effort level (Section 3.1, implicit ablation): The paper reports that "o3-mini with medium 'reasoning effort' to have similar effect to DeepSeek-R1 when used as teachers, but o3-mini medium was more token efficient. We also found o3-mini with high-effort to be a stronger teacher than medium-effort consistently across tasks, it also resulted in longer responses, increasing inference-time compute." This is not a controlled ablation (the models were not trained separately with each teacher), but is presented as an empirical observation from the exploration phase. The finding that teacher quality and token efficiency trade off in predictable ways is practically significant.

Synthetic seed data effect (Section 3.1, Figure 5, experiments 4–5): "An example small-scale study on the effect of this targeted synthetic augmentation is shown in Figure 5 experiments 4 and 5." The addition of synthetic math data (transformed from proofs to verifiable computation problems) "led to consistent and significant gains in complex tasks—improving performance by 3–10% on AIME 2022–2024." No further quantitative detail is provided, making this more of a directional finding than a precise characterization.

Learning rate selection (Section 3.1): The paper reports a grid search over $[1 \times 10^{-6}, 2 \times 10^{-5}]$ for the SFT learning rate, finding that $1 \times 10^{-5}$ "provided the best balance in terms of reasoning performance." Higher learning rates "result in lower training loss, but saturation and/or degradation across various downstream evaluations." This is a standard hyperparameter sweep, not a scientific ablation, but it confirms that reasoning SFT benefits from a learning rate approximately 10× higher than Phi-4's instruction tuning rate.

Weight decay effect (Section 3.1): "We also tested the effect of zero versus small weight decay (1e-4) and found the differences to be within benchmark variance." This is reported as a negative result—weight decay, within this range, does not significantly affect reasoning performance.

System message robustness (Section 3.1): Training with partially removed or replaced system messages "increased robustness under random system messages at inference time" but "when evaluated under the original reasoning message, we observed greater variability in benchmarks scores and a slight decrease in average benchmark performance." The final model was trained with the fixed reasoning system message, trading robustness for peak performance.

Base model selection (Section 3.1): Phi-4 versus Phi-4-base: "Both variations performed similarly on reasoning benchmarks, while Phi-4 performed slightly better in terms of safety and alignment... we ultimately selected it as the base for Phi-4-reasoning to preserve the benefits of prior non-reasoning post-training." This is an important practical finding: starting from an instruction-tuned base does not degrade reasoning performance and preserves safety behaviors.

Data mixture additivity (Section 3.1, Figure 5, experiments 8–12): The paper demonstrates that math and code data mixtures can be optimized independently and then combined without loss: "experiment 8 consists of the optimized data mixture for math alone (no code data), while experiment 9 is a simple addition of code data with uniform weights which shows an improved score on LiveCodeBench. We then independently tuned the data mixture for code data, and combined it with the math recipe in experiments 10–12." The additive property is empirically observed but not theorized about or formally tested.

RL checkpoint selection (Section 4.2): The paper reports that "further training for more steps does not translate to additional gains, hinting the potential of an already strong SFT model is near its performance ceiling." The model at step 90 was selected over later checkpoints (up to step 125) based on AIME 2024 validation accuracy. The specific accuracy values for later checkpoints are not provided.

RL reward function components: The effect of the length-aware accuracy reward (vs. binary accuracy reward) and the repetition penalty (vs. no penalty) are not ablated. The specific coefficients (w_acc = 8/13, w_rep = 1/13) are presented without alternatives. The thresholds (L_pos_control = 25,600, L_neg_control = 3,702) are not justified through sweep experiments. This means the reward design, while thoughtfully motivated, is not empirically validated through component-wise ablation.

Context length extrapolation (Section 5.1): For AIME, HMMT, GPQA, and Codeforces evaluations, the Phi reasoning models use 65,536 maximum tokens, double their training length of 32K, "without changing any RoPE parameters." The paper acknowledges this is out-of-distribution and that "neither model has properly trained on this length." This serves as an implicit robustness check: the model can generalize to longer generation lengths without catastrophic degradation, but the paper does not quantify the performance difference between 32K and 65K settings.

Temperature sensitivity: The paper uses temperature 0.8 for the Phi reasoning models on all benchmarks, consistent with DeepSeek-family recommendations (0.6–1.0). No temperature sweeps are reported, so the sensitivity of results to this parameter is unknown.

Data decontamination: The paper's claim that AIME 2025 is contamination-free rests on the fact that it "was released after the training data for Phi-4-reasoning was finalized." The decontamination process for other benchmarks is described as "rigorous" and the same as Phi-4's, but no details are provided beyond the list of benchmarks checked. There is no quantitative decontamination analysis (n-gram overlap, embedding similarity, or other metrics) reported.

Critical Assessment

The experiments presented in this paper are extensive in breadth—covering reasoning, coding, planning, spatial understanding, general-purpose capabilities, and safety—and the evaluation methodology represents a genuine advance in rigor for reasoning model assessment. However, several aspects of the experimental design warrant careful scrutiny when evaluating whether the paper's central claims are fully supported.

Claim 1: "Meticulous data curation allows smaller models to compete with larger counterparts." The evidence for this claim is strong but comes with an important caveat: the curation pipeline itself is dependent on a frontier-scale teacher model (o3-mini). The paper demonstrates that a 14B student can approach the teacher's performance when trained on the teacher's outputs, but it does not disentangle the contribution of curation from the contribution of teacher quality. A critical missing experiment would be: train Phi-4-reasoning with the same curation pipeline but using a weaker teacher (e.g., DeepSeek-R1-Distill-Llama-70B, or Phi-4 itself). If the curated-seed model still substantially outperforms an uncurated-seed model trained on the same teacher, that would isolate the curation effect. Without this, the claim that "meticulous data curation" (rather than "training on o3-mini's outputs") is the key driver remains partially unsubstantiated.

Furthermore, the paper reports that o3-mini with medium effort produced results comparable to DeepSeek-R1 as a teacher (Section 3.2), but does not report the actual numbers or a controlled comparison. This is an important missing quantitative result—without it, the reader cannot assess whether teacher quality differences are small (meaning curation dominates) or large (meaning curation primarily serves to select which teacher outputs to include).

Claim 2: "Reasoning acquired as a transferable meta-skill (30–60 point gains on out-of-domain tasks)." The evidence for transfer is the most compelling part of the paper, but the interpretation requires care. The models show large gains on BA-Calendar (~51 points), TSP (~29–34 points), and 3SAT (~53–61 points) relative to Phi-4. However, it is not demonstrated that these gains are due to transfer of reasoning strategies rather than simpler explanations:

• Benchmark difficulty alignment: If Phi-4's poor performance on these tasks is due to insufficient instruction-following or format compliance rather than reasoning ability, the gains could be largely attributable to the improved instruction-following and structured output format learned during SFT, not to genuine reasoning transfer. The IFEval results (83.4% for Phi-4-reasoning vs. 62.3% for Phi-4) suggest that instruction-following does improve substantially with reasoning training.

• O3-mini's reasoning traces may contain transferable patterns. The teacher model's solutions to math and coding problems may demonstrate general problem-solving strategies (decomposition, constraint propagation, search) that apply to algorithmic tasks. The transfer could be from exposure to these strategies in the training data rather than from the model learning a domain-independent reasoning capability.

A stronger test of meta-skill transfer would be: train on math-only SFT data (no code, no puzzles, no planning) and test on algorithmic tasks. If the transfer persists, that would be stronger evidence for genuine meta-skill acquisition. The current training mixture includes "logical puzzles" alongside math and code (Section 3.2), which may overlap with the algorithmic tasks.

Claim 3: "RL provides additional gains, particularly on math where it was applied." The RL results are well-documented and the training dynamics (Figures 7a–7f) are informative, but the claim that RL "further amplifies" SFT is demonstrated only for math—specifically AIME. The paper is explicit about this limitation ("the seed data contained no coding exercises"), and the LiveCodeBench and several out-of-domain results actually show slight regressions for Phi-4-reasoning-plus compared to Phi-4-reasoning. This suggests that the RL stage produces specialization, not amplification—it improves the targeted domain (math) at a small cost to generalization. The paper does not explore whether this tradeoff is inherent to RL or specific to their implementation (length-aware reward, math-only seeds, short training).

The RL also suffers from a context-length bottleneck that the paper acknowledges: "outputs that are longer than 31k are clipped to their first 31k tokens... which limits the extent to which GRPO can help" (Section 4.2). The clip ratio rises to ~12% by step 125 (Figure 7e), meaning a non-trivial fraction of training trajectories are truncated. This is a significant confound—the model may be learning not just the reward signal but also to avoid generating beyond the clip boundary, creating an artificial length ceiling.

Claim 4: "Thinking effort vs. accuracy tradeoff offers two different points on the efficiency frontier." The token usage analysis (Figure 11) supports this claim for math benchmarks, where Phi-4-reasoning-plus's additional tokens correlate with higher accuracy. However, for non-math tasks, the additional tokens do not translate to higher accuracy (BA-Calendar: 65.6% vs. 67.7% for SFT-only; 3SAT: 70.9% vs. 78.0%). The paper's characterization of this as "avenues for improving RL exploration and verification in broader task sets" is fair but also acknowledges that the current RL recipe does not generalize its benefits beyond math. The efficiency frontier analysis is therefore task-dependent: Phi-4-reasoning-plus is on a better frontier for math but a worse one for algorithmic reasoning.

Claim 5: "Comprehensive evaluations demonstrate the need for moving beyond single-score accuracy reporting." This is the paper's most robustly supported claim. Figure 9 definitively shows that single-run AIME 2025 comparisons are unreliable, and the per-year breakdown (Figure 10) reveals systematic difficulty variation that single-number averages obscure. The paper's implementation of its own recommendations—50–64 runs, standard deviations, distribution visualizations, disaggregated analysis—sets a concrete standard that future work can adopt or argue against. The limitation is that this rigor is applied primarily to the authors' own models; baseline numbers are drawn from third-party reports where available and may not meet the same standard. The paper acknowledges this for Codeforces ("they may not be directly comparable due to the lack of public information on their exact evaluation procedure") but does not extend the same caveat to all baseline numbers.

Missing experiments that would strengthen the paper:

• A controlled study of seed filtering: Train two models—one on filtered "teachable" seeds and one on uniformly sampled seeds from the same raw database—using the same teacher and training recipe. This is the single most important ablation for the paper's central thesis about seed curation, and it is absent.

• Domain-specific SFT ablations: Train a model on math-only SFT data and test on coding, planning, and spatial reasoning. This would cleanly measure transfer from math reasoning to other domains, separating it from the contributions of code, puzzle, and general-purpose data in the full mixture.

• RL reward function component ablations: Test the binary accuracy reward vs. the length-aware reward, with and without the repetition penalty, to quantify the contribution of each component. The current reward design is intellectually motivated but empirically unvalidated.

• Context length extrapolation quantification: Compare model accuracy at 32K vs. 65K max tokens on the same benchmarks to measure the cost (in accuracy degradation) of out-of-distribution length extrapolation.

• Teacher quality vs. curation quality factorial experiment: Train models with two teachers (e.g., o3-mini and a weaker teacher like GPT-4o) crossed with two curation strategies (filtered teachable seeds vs. unfiltered), creating a 2×2 design that would disentangle the contributions.

Benchmark size and statistical power concerns: While the paper commendably uses 50–64 runs for small benchmarks, AIME 2025 still contains only 30 problems. The kernel density estimate (Figure 9) for Phi-4-reasoning-plus shows a range of roughly 67–90% across runs, meaning the difference between a "good" run (~86%) and a "bad" run (~72%) is 14 percentage points—larger than the gap between many models being compared. Reporting a single mean (78.0%) with standard deviation (4.6%) is responsible, but it does not change the fact that the benchmark itself, at 30 problems, has limited capacity to reliably rank models whose true performance differs by less than 5–10 percentage points. The paper's own recommendation to use "larger, challenging and non-saturated benchmarks" should be weighed more heavily than the AIME 2025 numbers in assessing the model's capabilities. Omni-MATH at 4,428 problems is statistically far more reliable, and the finding that Phi-4-reasoning-plus (81.9%) outperforms o3-mini-high (74.6%) on this benchmark is both more surprising and more robust than the AIME 2025 comparison.

The "contamination-free" claim for AIME 2025: The paper states that AIME 2025 "was released after the training data for Phi-4-reasoning was finalized, so that benchmark remains contamination free." This is true in a narrow sense—the specific problems did not exist when training data was collected. However, AIME problems from different years share structural similarities (same competition format, similar problem types, common mathematical techniques). Models trained on AIME 1983–2024 problems (which were in the training data and were decontaminated against, but the decontamination is never perfect) may benefit from format familiarity and technique overlap even on unseen 2025 problems. The paper does not discuss this form of "soft contamination" or benchmark leakage through structural similarity, which is more difficult to detect than literal overlap.

Safety evaluation limitations: The paper acknowledges that "appropriate evaluation of output generated by reasoning models remains a challenge mainly due to the fact that current LLM judges and tools have not yet been optimized for long, step-by-step traces" (Section 5.3). The RAI measurement framework results are described only qualitatively ("Phi-4-reasoning shows minor regression compared to the base model Phi-4"), and the Toxigen results show a mixed picture (improved toxic detection balance but modest absolute gains). The safety evaluation is therefore preliminary and does not support strong claims about safety properties. The paper is appropriately cautious about this.

In summary, the experiments provide strong support for the claim that Phi-4-reasoning and Phi-4-reasoning-plus are capable reasoning models that punch above their weight class (14B parameters) on math reasoning specifically and reasoning broadly. The generalization results are genuinely impressive but the mechanism of transfer is not experimentally isolated. The evaluation methodology is a significant contribution in itself. The paper's claims about data curation's importance, while plausible and consistent with the results, are not directly tested through controlled ablation, leaving open the question of whether teacher quality, data quantity, or curation quality is the dominant factor. The RL results demonstrate a clear math specialization with associated generalization tradeoffs, establishing an important boundary condition that the paper is transparent about but does not resolve.

6. Limitations and Trade-offs

6.1 The Hardest Reasoning Problems Remain Unsolved — Test-Time Compute Cannot Create Capability from Nothing

The assumption or constraint. The entire training pipeline — SFT distillation from o3-mini, RL with verifiable math rewards — operates under an implicit assumption that the base model has sufficient foundational knowledge to learn from demonstrations of reasoning. The paper acknowledges this boundary explicitly when discussing difficulty-level analysis: "Phi-4-reasoning-plus offers improvements primarily at easy-to-mid difficulty levels" (Section 5.1.3, Figure 14), and the hardest difficulty bins show near-zero accuracy for all models regardless of training approach.

The consequence. On difficulty level 7–8 TSP instances, on highly constrained BA-Calendar problems, and on the hardest AIME competition years (e.g., 1994, 2025), neither Phi-4-reasoning nor Phi-4-reasoning-plus makes meaningful progress over the base model. The RL stage compounds rather than solves this: Figure 7a shows accuracy plateauing at ~78% on AIME 2025 after 90 steps, with "further training for more steps does not translate to additional gains." The paper attributes this partly to the 31K-token response clipping constraint — "outputs that are longer than 31k are clipped to their first 31k tokens... which limits the extent to which GRPO can help" (Section 4.2) — but the deeper issue is that the base model simply lacks the capability to navigate the solution space for the hardest problems. No amount of SFT curation or RL optimization can teach a model to solve problems for which it has no viable solution path in its generation distribution. This is the fundamental boundary condition identified in prior inference-time scaling work: test-time compute amplifies existing capability but does not create it.

What evidence exists in the paper. The evidence is distributed across multiple figures. Figure 14 (difficulty-level breakdowns for TSP, BA-Calendar, Omni-MATH) shows accuracy declining to near-zero at the highest difficulty levels for all models, with Phi-4-reasoning and Phi-4-reasoning-plus tracking the same trajectory as the base model. Figure 10 (per-year AIME performance) shows all models performing substantially worse on certain years (1994, 2025), with the ranking between models preserved but absolute scores low. The Omni-MATH topic-level analysis (Figure 15) reveals that discrete mathematics and geometry remain challenging for all models: Phi-4-reasoning-plus achieves approximately 58% on discrete math and 62% on geometry, compared to 85–88% on algebra and number theory. The GPQA domain breakdown (Figure 16) shows a similar pattern: biology (54.1%) and chemistry (50.5%) lag far behind physics (88.9%) for Phi-4-reasoning-plus. The paper does not report whether RL training on math transfers any benefit to these non-math hard domains; the evidence from BA-Calendar and 3SAT (where Phi-4-reasoning-plus slightly underperforms Phi-4-reasoning) suggests it does not.

Mitigation status. The paper is transparent about this limitation. It frames the difficulty-sensitivity as an inherent characteristic of the approach rather than a bug: reasoning training helps on problems within the model's "zone of proximal development" but cannot bridge fundamental capability gaps. The authors explicitly identify the 31K-token clipping as a bottleneck during RL and suggest that "enabling the model to support even longer context windows — such as 64k tokens, potentially through interpolation techniques... could yield additional benefits in GRPO training" (Section 4.2). They also note that the hard-problem limitation is shared by all evaluated models, including frontier-scale ones: "accuracy drops towards higher difficulty levels are common for all models, even the most capable ones" (Section 5.1.3). No solution is proposed beyond longer context windows and future work on extending RL to broader task sets. A practitioner deploying these models for hard mathematical reasoning or tasks that consistently fall in the highest difficulty bins should not expect meaningful improvement over the base model and may be better served by a larger pretrained model.

6.2 The SFT and RL Data Mixture Is Domain-Limited, and RL Induces Specialization at the Cost of Generalization

The assumption or constraint. The SFT training data is limited to STEM (science, technology, engineering, mathematics), coding, and safety domains. The RL training data is further limited to mathematics only — "the seed data contained no coding exercises" (Section 4). The paper's central generalization claim rests on the assumption that reasoning strategies learned from these domains transfer broadly to novel task types.

The consequence. The transfer is real but incomplete and asymmetric. Phi-4-reasoning-plus, the RL-enhanced model, shows systematic regressions compared to the SFT-only Phi-4-reasoning on tasks that were not targeted during RL: LiveCodeBench (53.1% vs. 53.8%, Table 1), BA-Calendar (65.6% vs. 67.7%, Figure 8), 3SAT (70.9% vs. 78.0%, Figure 8), Maze (53.4% vs. 55.1%, Figure 8), and SpatialMap (73.3% vs. 73.7%, Figure 8). While these regressions are individually small (1–7 percentage points), they are consistent across non-math tasks. The paper interprets this as evidence that the RL stage produces specialization, not amplification — improving the targeted domain at a small cost to out-of-domain generalization. The SFT-only model is in this sense a more "versatile" reasoner; the SFT+RL model is a stronger math reasoner but a slightly weaker general reasoner.

A practitioner choosing between the two models faces a genuine tradeoff: Phi-4-reasoning-plus offers 15 percentage points higher AIME 2025 accuracy (78.0% vs. 63.1%) but may be slightly worse on coding, planning, and spatial reasoning. The paper does not explore whether this tradeoff is inherent to RL-based refinement (the reward signal inevitably narrows the model's output distribution toward reward-maximizing behaviors that may not align with non-rewarded task requirements) or specific to this implementation (math-only seeds, relatively short training, length-aware reward that incentivizes longer math reasoning at the expense of other reasoning styles).

What evidence exists in the paper. The evidence for the specialization tradeoff is reported directly in the benchmark tables and figures. Table 1 shows LiveCodeBench and Codeforces numbers for both models; Figure 8 shows BA-Calendar, TSP, 3SAT, Maze, and SpatialMap comparisons. The paper is explicit about the specialization: "Phi-4-reasoning-plus marks important advantages over Phi-4-reasoning on math (also targeted during the RL stage)... This is less pronounced on algorithmic reasoning, planning, coding, and spatial understanding tasks, encouraging further work that enables exploration and verification for a broader set of tasks" (Section 5.1.3). The paper also notes that even on tasks where Phi-4-reasoning-plus shows higher mean accuracy (TSP: 42.6% vs. 37.5%), the gain is smaller than on math and may be attributable to TSP's partial overlap with mathematical optimization rather than genuine reasoning transfer.

Mitigation status. The paper frames this as an opportunity for future work rather than a solved problem: "we observe that even though the generations of Phi-4-reasoning-plus are on average across tasks 1.5x longer, on tasks like Calendar Planning, 3SAT, Maze, and SpatialMap, the lengthened traces do not lead to higher accuracy" (Section 5.1.3). The implication is that the RL reward function and seed distribution would need to be broadened — perhaps including coding, planning, or structured reasoning tasks with verifiable solutions — to extend RL's benefits beyond math. The paper does not attempt this. The observation that "RL exploration and verification for a broader set of tasks" is needed is left as a direction for future investigation. No concrete approach is proposed. For practitioners, this means that Phi-4-reasoning is the safer default for multi-domain reasoning workloads, while Phi-4-reasoning-plus should be preferred only when math is the dominant or exclusive task.

6.3 Difficulty Estimation Cost Is Unaccounted for, and the Curriculum Relies on a Frontier Teacher Model

The assumption or constraint. The seed filtering pipeline that identifies "teachable" prompts depends on two resources that are not available in a self-contained deployment scenario: (1) a strong frontier model (o3-mini, described as the teacher throughout Sections 2–3) to generate high-quality reasoning traces, and (2) computationally expensive difficulty estimation procedures — generating responses from both a reference model and the student model, computing agreement gaps, and running rubric-based LLM evaluators to assess reasoning-step complexity (Section 2.1). The paper reports the final SFT dataset statistics (1.4M prompt-response pairs, 8.3B tokens) but does not report the computational cost of producing this dataset: how many teacher model queries were required, what the token cost of the difficulty estimation pipeline was, or how many seeds were discarded during filtering.

The consequence. The paper's central methodological claim — that "meticulous data curation" enables small models to compete with larger ones — conflates curation quality with teacher quality. The student model (Phi-4-reasoning, 14B parameters) is trained on outputs from o3-mini, a frontier proprietary model. The paper provides evidence that o3-mini medium-effort is comparable to DeepSeek-R1 as a teacher (Section 3.2), but this comparison only establishes that one frontier model can substitute for another. It does not establish whether a non-frontier teacher (e.g., a larger open-weight model, or even an earlier generation of Phi) would produce comparable results with the same curation pipeline. This matters because the practical value proposition of this work — a 14B model approaching frontier reasoning performance — depends on whether the recipe is replicable without access to a frontier teacher. If the primary driver of performance is teacher quality, the methodology reduces to distillation from o3-mini, which is not a generalizable recipe for organizations that cannot access o3-mini. If the primary driver is seed curation, the methodology should work with any sufficiently capable teacher. The paper does not disentangle these factors because it lacks a controlled experiment varying teacher quality while holding curation constant (or vice versa).

Furthermore, the computational cost of the seed filtering pipeline — generating responses from both reference and student models for thousands of seeds, computing agreement rates, and running rubric-based assessments — is not quantified or amortized into the training cost. In a production setting where a team wants to replicate this approach for a new base model on a new domain, the difficulty estimation cost could substantially exceed the SFT training cost itself. The paper's suggestion that the pipeline is cost-effective relative to pretraining is plausible but unquantified.

What evidence exists in the paper. The paper acknowledges the dependence on o3-mini transparently — the teacher is credited in the abstract and throughout Sections 2–3 — but does not present an ablation with a different teacher. The exploration of "o3-mini medium vs. high effort" effects is a within-teacher comparison, not a between-teacher comparison. The statement that o3-mini medium "had similar effect to DeepSeek-R1 when used as teachers, but o3-mini medium was more token efficient" (Section 3.2) is the closest the paper comes to addressing teacher dependence, but no quantitative results are provided to support this claim. There is no corresponding comparison between o3-mini and, for example, GPT-4o, Claude, or an open-weight model, which would establish the robustness of the pipeline to teacher quality.

The cost of the seed filtering pipeline is not discussed anywhere in the paper. Section 2.1 describes the methodology in qualitative terms — "we rely heavily on LLM-based evaluation and filtering pipelines" — but provides no information about scale (how many seeds were evaluated), cost (how many API calls or GPU-hours were consumed), or yield (what fraction of evaluated seeds were retained). Without this information, the claim that the pipeline is practically replicable is difficult to evaluate.

Mitigation status. Not addressed. The paper does not propose methods for reducing dependence on frontier teachers (e.g., using the student model itself as the teacher for iterative self-improvement, or training multiple student generations with progressively weaker teachers). The difficulty estimation cost is explicitly noted as a practical concern in related work but not addressed here. Future work on "pretraining or finetuning models to directly predict difficulty of a question" is mentioned in the broader literature but not proposed or evaluated in this paper. A practitioner seeking to replicate this methodology without access to a frontier teacher has no guidance from this paper on whether the results would hold.

6.4 Small Benchmarks Dominate Headline Comparisons, and Variance Makes Model Rankings Unreliable

The assumption or constraint. The paper's headline comparison — Phi-4-reasoning-plus outperforming DeepSeek-R1-Distill-Llama-70B and approaching DeepSeek-R1 on reasoning — relies heavily on AIME 2025 (30 problems) and AIME 1983–2024 (949 problems). Both benchmarks are small by machine learning standards, and AIME 2025 is tiny. The paper acknowledges this limitation explicitly and makes a methodological contribution around it: "average-of-5 results from two independent runs can differ significantly by up to 5-10 percentage points on AIME 2025, thus making side-by-side comparison of models unreliable" (Section 5.1). The paper's solution is to run 50 independent evaluations and report standard deviations.

The consequence. Despite the paper's methodological rigor, the fundamental problem remains: AIME 2025's 30 problems cannot reliably rank models whose true performance differs by less than 10–15 percentage points, regardless of how many runs are performed. The standard deviation of Phi-4-reasoning-plus across 50 runs is 4.6 percentage points; the standard deviation of DeepSeek-R1 is 4.3 percentage points (Table 1). Their means are 78.0% and 70.4% respectively — a gap of 7.6 points. With standard deviations of ~4.5 points, the 95% confidence interval for the difference is approximately 7.6 ± 12.5 points, meaning the true difference could plausibly be zero or negative. The kernel density estimates in Figure 9 visually confirm this: the distributions for Phi-4-reasoning-plus and DeepSeek-R1 overlap substantially in the 70–80% range.

The paper's own recommendation is to "scale up our quantitative analysis across a diversified set of benchmarks that have a larger number of samples between 800-4500" (Section 5.1.2). When this recommendation is followed, the results are more reliable but less dramatic for the Phi models. On Omni-MATH (4,428 problems), Phi-4-reasoning-plus (81.9%) does outperform DeepSeek-R1-Distill-Llama-70B (63.4%) by a wide margin, but trails DeepSeek-R1 (85.0%) and notably outperforms o3-mini-high (74.6%). On GPQA Diamond (198 problems), Phi-4-reasoning-plus (69.3%) is comparable to DeepSeek-R1-Distill-Llama-70B (66.2%) and below DeepSeek-R1 (73.0%) — a much less striking result than the AIME 2025 comparison. On LiveCodeBench, Phi-4-reasoning-plus (53.1%) is below DeepSeek-R1-Distill-Llama-70B (57.5%) and far below DeepSeek-R1 (65.9%). The paper's central claim that the models "achieve performance competitive with or exceeding much larger models across several benchmarks" (Section 1) is true but highly benchmark-dependent, with the strongest evidence on math-specific benchmarks and much weaker evidence on coding and scientific reasoning.

The Codeforces comparison illustrates the practical difficulty of reliable benchmarking. The paper reports a Codeforces Elo of 1,723 for Phi-4-reasoning-plus and 1,633 for DeepSeek-R1-Distill-Llama-70B, claiming a substantial advantage. However, the paper also notes that "they may not be directly comparable due to the lack of public information on their exact evaluation procedure" (Section 5.1.1) — the DeepSeek-R1 numbers are reproduced from the DeepSeek-R1 report and may have been computed under different protocols. This caveat undermines the comparison but is the best the authors can do given incomplete baseline documentation.

What evidence exists in the paper. Figure 9 and the standard deviation reporting in Table 1 and Figure 8 are the primary evidence for variance concerns. Figure 10 (per-year AIME performance) shows that all models exhibit substantial year-to-year variation, and that the relative ranking between models is not perfectly consistent across years — years where the gap narrows or reverses exist. The paper's comparison of best-of-5 vs. average accuracy (Figure 12) further demonstrates that small changes in evaluation protocol (how many runs, how aggregation is performed) can shift reported numbers by 5–15 percentage points. The broad benchmark suite in Figure 8 reveals that Phi-4-reasoning-plus's advantage over DeepSeek-R1-Distill-Llama-70B varies dramatically by task: large on Omni-MATH, small-to-nonexistent on GPQA, reversed on Maze. The paper itself provides the evidence that the headline comparison is fragile.

Mitigation status. The paper partially addresses this by reporting standard deviations, using large numbers of runs for small benchmarks, and presenting results on larger benchmarks (Omni-MATH, BA-Calendar, etc.). It advocates for "moving beyond single-score accuracy reporting on small-scale datasets" and calls for "larger, challenging and non-saturated benchmarks" as the primary evaluation targets (Section 5.1). However, the paper's own narrative — the abstract's claim of competitive performance, Figure 1's headline comparison — still centers AIME 2025 and small benchmarks. The tension between the paper's methodological recommendations and its own presentation choices is unresolved. A practitioner reading only the abstract and Figure 1 would take away an impression of Phi-4-reasoning-plus as roughly matching DeepSeek-R1 and o3-mini across the board; a careful reader of the evaluation section would understand that this impression is largely driven by math benchmarks and that performance on coding, science, and spatial reasoning is more mixed. The paper's methodological contribution to evaluation is genuine, but it does not fully inoculate the reader against overinterpreting the small-benchmark results.

6.5 The RL Stage Is Constrained by a 31K-Token Response Limit That Truncates the Longest Reasoning Traces

The assumption or constraint. The GRPO training for Phi-4-reasoning-plus is conducted with a maximum response length of 31,744 tokens (1,024 reserved for the prompt out of the 32,768-total context window). Responses exceeding this limit are "clipped to their first 31k tokens" before reward computation (Section 4.2). The model is rewarded based on the truncated output — any reasoning, verification, or final answer that would have appeared beyond the 31K-token boundary is invisible to the reward signal.

The consequence. This creates a binding constraint that grows more severe as training progresses. Figure 7e shows that the fraction of responses hitting the clip boundary rises from near 0% at step 0 to approximately 12% by step 125. At the checkpoint selected for release (step 90), the clip ratio is approximately 8–10%. This means that for roughly 1 in 10 to 1 in 12 training trajectories, the model's output is truncated before the reward is computed. The model never receives feedback on the truncated portion — it cannot learn whether the truncated material would have led to a correct answer, nor can it learn to avoid generating output that will be truncated.

The length-aware reward function compounds this problem. Incorrect answers are explicitly incentivized to be longer — the reward increases (becomes less negative) as length approaches $L_{\text{neg\_control}} = 3,702$ tokens and beyond, up to the maximum context limit. This creates a gradient pushing incorrect answers toward the clip boundary. Figure 7d confirms this: the mean length of incorrect answers grows from ~6,000 to ~14,000 tokens, and the 75th percentile of all responses grows from ~8,000 to ~22,000 tokens. The model is being rewarded for generating longer incorrect answers, but the longest of these answers are being clipped, creating a perverse incentive: the model may learn to generate responses that are long enough to receive higher reward but not so long that they are clipped and lose the opportunity for the final answer box. This could produce a "sweet spot" strategy that is an artifact of the clip boundary rather than genuine reasoning depth optimization.

The paper acknowledges this: "due to the maximum sequence length constraint, incorrect answers might not always get corrected once they use up all 31k of maximum allowed generation tokens before the model gets a chance to produce a final answer in the end with the \boxed{} tag, thus reward plateaus as clipping of excessively long generations goes up" (Section 4.2). The plateau in final reward (Figure 7e) and the plateau in accuracy (Figure 7a) after step 90 are partially attributed to this constraint: "further training for more steps does not translate to additional gains, hinting the potential of an already strong SFT model is near its performance ceiling. A caveat to this observation is the fact that we clip responses beyond 31k output tokens during GRPO, which limits the extent to which GRPO can help."

What evidence exists in the paper. Figure 7d (length growth vs. RL steps) shows the increasing response lengths. Figure 7e (reward and clip ratio vs. RL steps) directly quantifies the clip ratio. Figure 7a (accuracy vs. RL steps) shows the plateau. The paper's text in Section 4.2 explicitly discusses the clip constraint as a limitation.

Mitigation status. The paper proposes but does not implement a solution: "enabling the model to support even longer context windows — such as 64k tokens, potentially through interpolation techniques similar to those used to extend context length from 16k to 32k during SFT or alternative RoPE interpolation methods — could yield additional benefits in GRPO training" (Section 4.2). This is left as future work. The paper does not discuss alternative mitigation strategies such as: applying length penalties more aggressively to prevent the model from reaching the clip boundary, using a dynamic context window that expands as training progresses, or employing rejection sampling that filters out truncated responses from training. For now, the 31K-token constraint represents a hard ceiling on how much the model can improve through RL, and the paper's own evidence suggests it is binding at the released checkpoint.

A practitioner considering using Phi-4-reasoning-plus for problems that require very long reasoning chains should be aware that the model was trained with a length ceiling that may have shaped its strategy — it may have learned to avoid generating extremely long traces (since those would have been clipped during RL) even when such traces would be beneficial for problem-solving. Conversely, the model may not have learned effective strategies for reasoning within a 64K-token budget, since it was never trained with that budget. The model's performance when evaluated at 65,536 tokens (as the paper does for AIME, HMMT, GPQA, and Codeforces) may therefore reflect out-of-distribution behavior that is not optimized. The paper acknowledges that "neither model has properly trained on this length" (Table 3 footnote), but does not quantify the performance difference between 32K and 65K evaluation settings, leaving this open as an unmeasured risk.

6.6 The Safety and Alignment Properties of Long Chain-of-Thought Traces Are Poorly Understood and Insufficiently Evaluated

The assumption or constraint. The paper adds safety and alignment data to the SFT training mixture and evaluates safety using two frameworks: the automated RAI measurement framework (adversarial conversation simulation measuring jailbreak success, harmful content generation, and IP leakage) and Toxigen (discriminative toxic language detection). However, the safety evaluation does not specifically target the unique risks introduced by long chain-of-thought reasoning traces — the "thinking" block that the model generates as an intermediate step before producing the final answer.

The consequence. The thinking block is generated by the model and, in open-weight deployments, fully visible to users and downstream developers. The paper notes that "the model tends to regurgitate variations of [safety] guidelines in the 'thinking' block" (Section 2.2) and that "the effect of this on the model safety is an active topic of research in particular for open-source models, for which users and developers have open access to complete generations." Long, unstructured reasoning traces present several safety risks that the paper's evaluation does not assess:

• Harmful content in reasoning traces: A model might produce a safe final answer (the "response" block) while the "thinking" block contains harmful reasoning — explicit consideration of dangerous methods, biased assumptions, or unethical tradeoffs that are ultimately rejected but exposed to the user. The paper's safety evaluation does not separately assess the thinking block.

• Reasoning trace as attack surface: Adversarial users could prompt the model to reveal sensitive information or produce harmful content within the thinking block, even if the final answer is sanitized. The SFT training teaches the model to separate thinking from response, but does this separation create a "protected" space where the model feels less constrained by safety guidelines?

• Overreliance on chain-of-thought: The paper explicitly notes that teaching the model to "not reveal the guidelines or the chain-of-thoughts in the 'answer' block... may be a desirable behavior for applications that want to choose to only show the final output to users to reduce cognitive load and overreliance on chain-of-thought traces" (Section 2.2). However, the thinking block may still be visible in the raw model output, and applications that choose to display it may inadvertently encourage users to trust the reasoning process as authoritative even when it contains errors or biases.

• Long-trace evaluation challenges: The paper acknowledges that "current LLM judges and tools have not yet been optimized for long, step-by-step traces and may struggle with the extended length and the non-linear generations" (Section 5.3). This means that the automated safety evaluation frameworks used in the paper may be missing safety violations that occur in long reasoning traces, particularly if the violations are embedded in complex chains of thought that are difficult for classifier models to parse.

What evidence exists in the paper. The safety evaluation results are reported qualitatively and quantitatively in Section 5.3 and Table 2. The RAI measurement framework results are described only as "Phi-4-reasoning shows minor regression compared to the base model Phi-4" — no specific numbers are provided. Toxigen results show a mixed picture: Phi-4-reasoning achieves 86.7% accuracy on toxic content detection and 84.7% on neutral content detection, which the paper characterizes as "a more balanced accuracy on detecting neutral vs. toxic content" compared to Phi-4-reasoning-plus (77.3% toxic, 90.5% neutral). The paper notes that "as we add reasoning capabilities to models, we are able to improve performance on only one of the toxic or neutral categories at a time" and that "detecting toxicity without causing erasure is still a challenging problem for all models." This suggests that reasoning training reshapes the model's safety behavior in complex ways that are not fully understood.

The paper explicitly flags the gap in safety evaluation for reasoning models: "Future research is needed to mature the practices in these areas by potentially improving safety-oriented verification methods and by decomposing and simplifying the evaluation of long traces" (Section 5.3). It also notes that the evaluation "focuses on measuring engagement and refusal in jailbreak scenarios and on classification-style toxic language detection" and that "it is also important to evaluate models, including ours, in more open-form and benign scenarios, beyond jailbreaks and adversarial scenarios." These are acknowledgments that the safety evaluation is incomplete, not just for this model but for the field.

Mitigation status. The paper partially mitigates the risk of harmful content in final answers through the SFT training on safety data with explicit guidelines, and through the structured format that separates thinking from response. The instruction to "not reveal the guidelines or the chain-of-thoughts in the 'answer' block" provides a layer of protection for applications that display only the response block. However, for applications that display the full generation (including thinking), or for users who directly interact with the raw model output, no specific mitigation is provided. The paper does not evaluate the thinking block separately for safety violations, does not propose filtering or moderation strategies for long reasoning traces, and does not test whether safety guidelines are equally effective in the thinking and response contexts. This is left as an open challenge that "will benefit from more research aimed at improving safe deployment of reasoning models" (Section 5.3). For practitioners deploying these models in user-facing applications, this means that the current safety evaluations provide an incomplete picture, and additional testing on the specific deployment configuration (whether the thinking block is shown or hidden) is necessary before production use.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper shifts the reasoning-model development conversation from "scale or distill" to "curate, then distill, then optionally specialize." The prevailing narrative in early 2025—shaped primarily by DeepSeek-R1—is that reasoning capabilities emerge from large-scale reinforcement learning on top of strong base models, with distillation as a secondary mechanism for compressing those capabilities into smaller models. Phi-4-reasoning challenges this framing at a foundational level by demonstrating that SFT on carefully curated data can produce a 14B model that outperforms a 50× larger distilled model (DeepSeek-R1-Distill-Llama-70B) on math reasoning benchmarks, and that subsequent RL provides a targeted but narrow improvement rather than being the primary source of reasoning ability. This is not merely an incremental efficiency gain; it reorders the causal story about where reasoning comes from.

The operational implication is a reframing of the field's resource allocation. Prior to this work, a team wanting to build a reasoning model faced an implicit choice: train a massive model with RL (the DeepSeek-R1 path, requiring frontier-scale infrastructure) or distill from someone else's massive model (the R1-Distill path, requiring access to frontier model outputs). This paper introduces a third path that is qualitatively different from both: invest heavily in seed curation and data mixture optimization, use a strong but accessible teacher for trace generation, and achieve frontier-competitive performance at 14B parameters without any RL. The RL step is then optional—a targeted specialization for math that provides meaningful gains (78.0% vs. 63.1% on AIME 2025) but is not the primary engine of reasoning capability. This changes the economics: data curation expertise becomes as valuable as raw compute access.

The paper also provides an organizing principle for understanding the complementary roles of SFT and RL that resolves tensions in prior work. DeepSeek-R1 showed that pure RL could induce reasoning; this paper shows that pure SFT can too, but the resulting models have qualitatively different properties. The decreasing response length during SFT and increasing response length during RL (Figures 4b and 7d) are the paper's clearest contributions to this understanding. SFT teaches efficient reasoning—the model learns to get the right answer in fewer tokens over the course of training. RL teaches adaptive thoroughness—the model learns to allocate more tokens when uncertain. These are complementary capabilities, not redundant ones, and the paper's explicit demonstration of their opposite length dynamics provides a diagnostic framework for future reasoning model developers: if your SFT response lengths are increasing, you may be overfitting to long teacher traces; if your RL response lengths are not increasing, your reward function may not be incentivizing deeper reasoning on hard problems.

A subtler but important shift is the paper's methodological contribution to evaluation as a first-class concern. The field has known for years that small benchmarks produce noisy estimates, but this paper operationalizes that concern into a specific, actionable critique: single-run average-of-5 reporting on AIME 2025 can produce ordinal ranking errors because the standard deviation across runs (4–6 percentage points) exceeds the performance gap between many compared models. The kernel density estimates in Figure 9 make this critique visually undeniable in a way that previous methodological papers' arguments did not. By implementing the recommended practices—50-run evaluations, standard deviation reporting, distribution visualization, per-year and per-topic disaggregation—the paper establishes a de facto standard that future reasoning model papers will need to either meet or explicitly argue against. In a field where benchmark gaming is a persistent concern, this is a contribution to scientific infrastructure, not just to model performance.

The paper's findings also redirect research attention toward what makes distillation data effective. Prior work has largely treated distillation as a function of teacher quality × data quantity—use a good teacher, generate many examples, train the student. This paper's emphasis on seed filtering to the "teachable" boundary (Section 2.1) suggests that which examples the teacher solves matters as much as how well the teacher solves them. If this finding replicates, it implies that the current practice of generating massive distillation datasets by querying a teacher on every available problem is wasteful. A more efficient strategy is: (1) characterize the student model's capability distribution, (2) select problems at the capability frontier, (3) generate teacher traces only for those problems. The agreement-gap protocol provides a domain-agnostic mechanism for step (1). This is a data-centric innovation that could substantially reduce the cost of producing high-quality reasoning models.

The paper also clarifies which research directions are becoming less attractive. The finding that lookahead-style search and sophisticated RL exploration do not dramatically outperform simpler approaches—Phi-4-reasoning-plus plateaus after 90 GRPO steps on only ~6K examples—suggests that algorithmic innovation in RL for reasoning may be hitting diminishing returns when applied to strong SFT starting points. The bottleneck is not the RL algorithm but the data distribution (limited to math), the context window (31K-token clip), and the verifier quality (rule-based answer checking). Research investment should shift from "more complex RL" to "broader RL task coverage," "longer context windows for RL training," and "better verifiers for non-math domains." The paper's own data supports this: the ReST^EM experiment in the broader literature showed degradation, and the paper's RL gains are narrow (math-only) despite the model's SFT reasoning skills being broad. The path forward is not better RL algorithms but RL-compatible task design—creating verifiable problems in coding, planning, spatial reasoning, and scientific domains that can serve as RL rewards.

Follow-Up Research This Work Enables

1. Quantifying the causal contribution of seed curation vs. teacher quality through a factorial experiment. The paper's central claim—that seed curation matters independently of teacher quality—is plausible and consistent with the results but is not experimentally isolated. A strong follow-up would train a 2×2 grid of models crossing two curation strategies (filtered "teachable" seeds vs. uniformly sampled seeds from the same raw pool) with two teacher quality levels (o3-mini vs. GPT-4o or DeepSeek-R1-Distill-Llama-70B). If the filtered-seed model trained on the weaker teacher outperforms or matches the unfiltered-seed model trained on the stronger teacher, this would establish curation as the dominant factor. If teacher quality dominates regardless of curation, the paper's narrative would need to be revised to acknowledge that curation primarily amplifies teacher quality rather than substituting for it. The experiment should measure both in-domain (math) and out-of-domain (planning, spatial reasoning) transfer to determine whether curation affects generalization breadth or only peak performance.

2. Extending RL to non-math domains through synthetic verifier construction. The paper's RL stage is math-only, and the specialization tradeoff (Phi-4-reasoning-plus underperforming Phi-4-reasoning on BA-Calendar, 3SAT, Maze) demonstrates that math-focused RL narrows rather than broadens reasoning capability. The next step is to extend RL to new domains, and the key challenge is verifier construction: math has ground-truth answers, but calendar planning, spatial reasoning, and algorithmic problem-solving require more sophisticated correctness checks. A concrete experiment would: (1) curate a seed set of BA-Calendar problems that Phi-4-reasoning can partially but not perfectly solve (analogous to the RL math seed selection), (2) implement a verifier that checks constraint satisfaction (all participants available, buffer times respected, priority honored), (3) run GRPO with a similar length-aware reward on this seed set, and (4) measure whether RL transfers back to math or further specializes. A negative result—RL on calendar planning degrades math while only modestly improving planning—would establish an important boundary condition: that RL always induces specialization, and the SFT model is the generalist. A positive result—mutual improvement—would suggest the specialization tradeoff is an artifact of seed choice rather than RL per se.

3. Training a difficulty predictor to eliminate the expensive seed filtering pipeline. The paper's teachable-seed methodology requires generating responses from both reference and student models for thousands of candidate seeds—a cost that is not amortized into any of the reported training efficiencies. A practical follow-up would train a lightweight classifier that predicts, from the question text alone, whether a given seed lies in Phi-4's "teachable" boundary. The training data would be the already-computed agreement gaps from the paper's filtering pipeline: each seed is labeled as "teachable" or "not teachable" based on the gap between reference and student agreement rates. A small model (e.g., a fine-tuned BERT or a few-shot prompted LLM) could be trained to predict this label from the question text. If the classifier achieves, say, 85% precision/recall on held-out seeds, the cost of filtering new seed databases drops from thousands of expensive model queries to a single classifier inference per seed. The experiment should also test whether seeds classified as "teachable" by the predictor produce the same downstream SFT performance as seeds selected by the full agreement-gap pipeline.

4. Measuring the effect of context length on RL training through dynamic or staged expansion. The paper identifies the 31K-token clip as a binding constraint during RL (Figure 7e: clip ratio rises to ~12%), hypothesizes that longer context would yield additional GRPO gains, but does not test this. A direct follow-up would run multiple RL training runs starting from the same Phi-4-reasoning checkpoint, varying only the maximum generation length: 32K (current), 48K (via RoPE interpolation), and 64K (via further interpolation or architectural modification, such as additional RoPE base frequency doubling). The key metrics would be: (1) whether the accuracy plateau in Figure 7a shifts upward and to the right with longer context (does the model continue improving beyond step 90?), (2) whether the clip ratio at equivalent accuracy levels decreases, and (3) whether the model learns qualitatively different reasoning strategies (e.g., deeper verification, more alternative-solution exploration) when not constrained by the clip boundary. A negative result—longer context does not change the plateau, only delays it—would suggest the plateau is due to seed diversity or reward function limitations rather than context length.

5. Characterizing the "thinking" block as an independent artifact with distinct safety, faithfulness, and usability properties. The paper introduces a structured thinking/response format but evaluates the model only on final answer correctness—the thinking block itself is treated as a means to an end. Given the paper's own observation that "the model tends to regurgitate variations of safety guidelines in the 'thinking' block" (Section 2.2), a systematic study of the thinking block's properties is needed. A comprehensive evaluation would measure: (1) faithfulness: on problems where the model gets the correct answer, does the thinking block's reasoning actually support that answer, or does it contain unrelated or contradictory reasoning? This can be assessed by asking a separate LLM judge to check whether the thinking block's final conclusion matches the response block's answer. (2) Safety: are there prompts where the thinking block contains harmful, biased, or dangerous content that the response block sanitizes? This requires running standard safety benchmarks (e.g., Toxigen, HarmBench) and separately evaluating the thinking and response blocks. (3) Usefulness for human-AI collaboration: when users are shown the thinking block vs. only the response block, does accuracy, trust calibration, or task completion time change? This requires a human subjects study. (4) Compressibility: can the thinking block be significantly shortened without degrading answer accuracy, perhaps by training a separate "summarizer" model? This would test whether the thinking block's length is necessary for reasoning or partially decorative.

6. Stress-testing generalization by training on a deliberately narrow domain and measuring far-transfer. The paper claims that reasoning training produces "reasoning as a transferable meta-skill" evidenced by gains on out-of-domain tasks (TSP, 3SAT, BA-Calendar). However, the SFT training mixture includes math, coding, logical puzzles, and general-purpose data—any of which could plausibly contribute to algorithmic reasoning ability. A stringent test of meta-skill transfer would be: train Phi-4-reasoning on math-only SFT data (no coding, no puzzles, no planning examples, no general-purpose data beyond formatting), then evaluate on TSP, 3SAT, BA-Calendar, Maze, and SpatialMap. If the gains persist (even at reduced magnitude), this is strong evidence that mathematical reasoning transfers to algorithmic and spatial domains—the model has learned general decomposition, constraint satisfaction, and systematic exploration strategies that apply beyond the training distribution. If the gains vanish, the paper's generalization claims are better explained by the breadth of the training mixture than by the transferability of reasoning itself. This experiment also has direct practical implications: if math-only training transfers, then the data curation burden for new reasoning domains is dramatically reduced.

Practical Applications and Downstream Use Cases

1. Cost-efficient batch evaluation of competition math problems. For organizations that need to solve large volumes of competition-level math problems—mathematics competition platforms, automated grading services, educational technology companies generating solution explanations—Phi-4-reasoning offers a specific cost advantage. The model achieves 74.6% on AIME 1983–2024 (949 problems) compared to DeepSeek-R1's 78.7% (Table 1), but at roughly 1/50th the parameter count (14B vs. 671B). For batch processing where per-query latency is less critical than total throughput, a cluster of Phi-4-reasoning instances can process many more queries per GPU-hour than a DeepSeek-R1 deployment. The majority-voting scaling in Figure 2 further allows trading additional inference compute for higher accuracy: Maj@16 on Phi-4-reasoning-plus achieves approximately 87% on AIME 2025 (reading from Figure 2), exceeding o3-mini's pass@1 (82.5%). Organizations that need near-perfect accuracy on math benchmarks can deploy Phi-4-reasoning-plus with parallel sampling rather than paying for frontier API access.

2. On-device or edge-deployed reasoning for educational tools. At 14B parameters and with 32K context, Phi-4-reasoning is in the size range where quantization (4-bit or 8-bit) enables deployment on consumer GPUs or high-end edge devices. A mathematics tutoring application could run Phi-4-reasoning locally, providing step-by-step reasoning traces for student problems without sending data to cloud APIs. The structured thinking/response format enables the application to selectively display the reasoning (for learning) or hide it (for assessment). The model's performance on Omni-MATH (76.6% for Phi-4-reasoning, 81.9% for Phi-4-reasoning-plus) covers a broad range of mathematics topics (algebra, calculus, number theory, precalculus), and the topic-level analysis (Figure 15) identifies specific weak areas (discrete mathematics: ~58%, geometry: ~62%) where the application should fall back to a larger model or human tutor. The finding that response lengths decrease during SFT (Figure 4b) means the SFT-only Phi-4-reasoning is relatively token-efficient (~13,000–14,000 tokens per AIME problem), keeping latency within acceptable bounds for interactive tutoring.

3. Safety-oriented content moderation with balanced toxic/neutral detection. The Toxigen results (Table 2, Figure 18) show that Phi-4-reasoning achieves a more balanced accuracy profile than other models: 86.7% on toxic content detection and 84.7% on neutral content detection. This balance is practically valuable for content moderation applications where false positives (flagging neutral content as toxic) cause user friction and false negatives (missing toxic content) cause harm. GPT-4o, by comparison, achieves 87.6% toxic detection but only 85.1% neutral detection—a slightly larger imbalance. Phi-4-reasoning-plus achieves 77.3% toxic and 90.5% neutral—a more extreme imbalance in the opposite direction. For a moderation pipeline that needs to minimize both error types equally, Phi-4-reasoning is the preferred choice among the evaluated models. Additionally, the group-level breakdown (Figure 18b) shows that Phi-4-reasoning has "narrowed some of the group-based discrepancies," suggesting improved fairness across demographic categories—a critical requirement for production moderation systems.

4. Data generation for self-improvement loops in reasoning model training. The paper's finding that the SFT data mixture can be optimized independently per domain and then combined additively (Section 3.1) has direct implications for iterative self-improvement pipelines. In a ReST-style or STaR-style loop, a model generates its own training data, filters for correctness, and retrains. The additive mixture property means that new data from different domains (e.g., a newly curated set of chemistry problems, or model-generated solutions to novel planning tasks) can be incorporated by simply concatenating their optimized weights onto the existing mixture, without re-optimizing the full joint distribution. This dramatically reduces the cost of expanding a reasoning model's domain coverage over time. For a team maintaining a reasoning model that must evolve to cover new domains as they become important, the recipe is: (1) curate domain-specific seeds using the agreement-gap protocol, (2) generate teacher traces, (3) independently find the optimal training weight for this new data source via small-scale SFT runs, (4) add it to the existing mixture at that weight. The paper's results suggest this will preserve gains in all other domains while adding the new domain's capability, without destructive interference.