Why Does Self-Distillation (Sometimes) Degrade the Reasoning Capability of LLMs?

ArXiv: 2603.24472

🎯 Pitch

Self-distillation can slash math reasoning accuracy by up to 40%—not due to reward hacking, but because richer teacher contexts systematically suppress the model's own expressions of uncertainty like “Wait” or “Hmm,” stripping out the adaptive self-correction that unseen problems demand.

---

1. Executive Summary

This paper analyzes why self-distillation—a post-training paradigm where a model acts as both student and teacher by conditioning the teacher on richer information—degrades reasoning performance on mathematical benchmarks despite improving it in domains like chemistry and code generation. Through controlled experiments varying conditioning contexts across DeepSeek-R1-Distill-Qwen-7B, Qwen3-8B, and Olmo-3-7B-Instruct, the authors identify suppression of epistemic verbalization (the model's expression of uncertainty via tokens like "Wait," "Hmm," "perhaps") as the core mechanism driving the degradation, finding that richer teacher conditioning produces reasoning trajectories with dramatically fewer uncertainty markers—from 182.5 epistemic tokens in unguided generation down to 8.8 under full solution guidance. On-policy self-distillation with full solution conditioning causes performance drops of up to 40% on AIME24 for DeepSeek-Distill-7B and progressively worsening OOD scores for Qwen3-8B, establishing that self-distillation's conciseness gains come at the cost of removing adaptive uncertainty signals critical for robust generalization—but only when task coverage is broad and evaluation problems lie outside the training distribution, while narrow task coverage enables rapid in-domain optimization with no OOD penalty.

2. Context and Motivation

The Core Problem: Self-Distillation Is an Empirical Paradox

The paper confronts a genuine puzzle in the post-training literature: self-distillation sometimes dramatically improves LLM performance, and sometimes catastrophically degrades it, even when all training data consists of correct reasoning traces. Figure 1 captures this tension starkly. In chemistry domains, self-distillation (via SDPO) reduces response length by roughly half while rapidly outperforming GRPO—the standard reinforcement-learning-from-verifiable-rewards algorithm. Yet on DAPO-Math-17k, the same procedure causes training scores to drop by approximately 20 percentage points relative to GRPO, even as response length shrinks monotonically. The central question driving the paper, quoted verbatim, is:

"Why does performance sometimes degrade despite the model being trained to move toward the correct answer?"

This specificity is crucial: the degradation is not a failure to optimize the training objective, nor is it a consequence of training on incorrect outputs. The model is genuinely learning to generate correct final answers on the training set—or at least, the reward signal indicates it is—yet generalization to held-out math benchmarks collapses. This distinguishes the problem from standard overfitting or reward hacking narratives; it is instead a case where the training signal itself reshapes the model's reasoning style in ways that are orthogonal to final-answer correctness but fatal for generalization.

Why This Problem Matters: Beyond a Single Training Algorithm

The significance of this phenomenon extends well beyond self-distillation as a specific method. Three broader stakes make the paper's investigation consequential:

1. Self-distillation is increasingly popular and empirically successful in some domains. Several recent works—including zhu2025token, understanding, SDPO, shenfeld2026self, song2026expanding, zhao2026self, and opcd—have demonstrated that combining self-distillation with reinforcement learning from verifiable rewards (RLVR) yields highly efficient performance gains, particularly in agentic environments, scientific reasoning, and code generation. The approach is attractive for practical reasons: it requires no external teacher model, no human preference data, and no reward model training, making it computationally cheaper than alternatives like RLHF or conventional knowledge distillation. If self-distillation is broadly adopted, understanding its failure modes becomes urgent.

2. The suppression of epistemic verbalization is a general risk in reasoning models. The paper's central finding—that richer conditioning suppresses expressions of uncertainty—is not specific to self-distillation. Any training paradigm that rewards concise, confident outputs (which includes most RL-based fine-tuning, rejection sampling, and distillation approaches) risks removing the uncertainty markers that strong reasoning models like DeepSeek-R1 explicitly produce. The paper shows that models trained to reason well already use epistemic tokens like "Wait" and "Hmm" to backtrack, reconsider, and explore alternative hypotheses. If post-training methods inadvertently remove these signals, the consequence is reasoning that appears fluent but is brittle to novel or harder problems. This has implications for model evaluation: a model might improve on benchmarks that mirror its training distribution while degrading on genuinely OOD problems, creating a false impression of progress.

3. The finding challenges assumptions about length reduction as uniformly desirable. A persistent narrative in the LLM post-training literature is that shorter reasoning traces are better—more efficient, more user-friendly, cheaper at inference time. The self-distillation work that the paper builds on (e.g., SDPO) explicitly celebrates conciseness gains. This paper provides a counterexample where shorter traces come at the cost of generalization, complicating the simple "shorter = better" story. The practical implication is that post-training objectives may need to explicitly preserve or encourage uncertainty expression, rather than treating it as inefficiency to be optimized away.

Prior Approaches and Where They Fall Short

The paper positions itself against several lines of prior work, each of which has limitations that motivate the current investigation:

Self-distillation with solution conditioning (SDPO, OPSD, et al.). The dominant paradigm in prior self-distillation work is to condition the teacher model on the ground-truth solution and then minimize the KL divergence between the student's and teacher's next-token distributions. This has been shown to work well in chemistry (sciknoweval), tool use (toolalpaca), and code generation (livecodebench). The critical limitation, from the paper's perspective, is that these prior evaluations are almost entirely in-domain—the training and evaluation distributions are either identical or cover the same narrow set of problem types. The paper argues (and demonstrates in Section 6) that when task coverage is limited, the conciseness bias of self-distillation is beneficial because it strips away unnecessary reasoning steps for problem types the model sees repeatedly. The failure mode only emerges when task coverage broadens and evaluation shifts OOD, which is precisely the regime that math benchmarks represent. Prior work did not systematically vary task coverage, leaving the boundary conditions of self-distillation's effectiveness unexplored.

epistemic verbalization analysis (understanding). The paper explicitly builds on understanding, which first identified that suppressing epistemic markers like "Wait" or "Hmm" significantly degrades reasoning performance. understanding established the correlation between epistemic token removal and performance drops. The current paper extends this work by identifying self-distillation as a specific mechanism that causes this suppression, and by characterizing when the suppression matters (broad task coverage, OOD evaluation) and when it doesn't (narrow coverage, in-domain evaluation). This turns a static observation into a dynamic, training-regime-dependent phenomenon.

GRPO and reinforcement learning from verifiable rewards. GRPO (Group Relative Policy Optimization) is the standard RLVR baseline the paper compares against. It optimizes a policy to maximize a correctness reward by generating multiple rollouts per prompt, computing advantages relative to the group mean, and clipping updates—familiar from the PPO lineage. The paper's findings are not that GRPO is universally better; rather, GRPO serves as a foil to reveal self-distillation's specific failure mode. GRPO tends to increase response length and epistemic token usage as it learns, particularly when task coverage is broad (Figure 8, Figure 11), because it rewards correct answers without penalizing verbosity. Self-distillation, by contrast, compresses the teacher's confident reasoning style into the student, suppressing epistemic verbalization. The comparison makes clear that the degradation is not due to RL per se but to the information asymmetry inherent in self-distillation's teacher-student setup.

Teacher-student distillation with separate models. Conventional knowledge distillation uses a larger, better teacher to guide a smaller student. Self-distillation uses the same model as both teacher and student, differing only in that the teacher receives richer conditioning (the ground-truth solution). This creates a unique dynamic: the teacher produces reasoning that is valid only given information the student will not have at inference time. The teacher's traces assume access to the correct answer, so they skip the exploration, backtracking, and uncertainty-driven hypothesis testing that the student needs when solving a problem from scratch. Training the student to imitate these traces is therefore training it to reason as if it already knows the answer—a style that collapses on problems where the answer is not obvious. This dynamic does not exist in conventional distillation because the teacher there genuinely has more capability, not more information about the specific problem instance. The paper's insight is that richer conditioning creates a subtle form of information leakage that reshapes reasoning style, and that this reshaping is the proximal cause of OOD degradation.

How This Paper Positions Itself

The paper frames its contribution not as proposing a new algorithm but as providing an information-theoretic diagnosis of why a popular algorithm fails in specific regimes. The theoretical framework in Section 2 formalizes the intuition: self-distillation's teacher receives a context $c$ (e.g., the ground-truth solution) that provides mutual information $I(y; c \mid x)$ about the target response $y$. Higher $I(y; c \mid x)$ means the teacher faces less uncertainty and therefore produces responses with fewer epistemic markers and shorter length. The student, trained to match this distribution, learns to reason confidently—but when it encounters problems where the answer is not implied by surface patterns (i.e., genuinely novel, hard, or OOD problems), the absence of uncertainty-driven exploration causes failure.

The paper's empirical strategy is explicitly comparative: it systematically varies the informativeness of the conditioning context (Section 3), examines both off-policy SFT (Section 4) and on-policy RL (Section 5), and sweeps task coverage (Section 6) to map out the boundary conditions of self-distillation's success and failure. This is a diagnostic agenda, distinct from prior work that primarily sought to demonstrate self-distillation's benefits in favorable regimes. The paper positions its findings as cautionary—not that self-distillation should be abandoned, but that practitioners and researchers should account for its effects on reasoning style, particularly epistemic verbalization, when evaluating its suitability for a given domain and task distribution.

3. Technical Approach

3.1 Reader Orientation

The paper constructs a controlled experimental framework—not a deployable system—for diagnosing why self-distillation degrades reasoning, particularly on mathematical benchmarks, even when training exclusively on correct solutions. The investigation isolates one specific mechanism—suppression of epistemic verbalization—and traces how two structural factors (information richness and task coverage) modulate this suppression to produce domain-dependent outcomes that range from beneficial conciseness gains to catastrophic OOD performance collapse.

3.2 Big-Picture Architecture (Diagram in Words)

The experimental framework has four investigative layers, each progressively illuminating the degradation mechanism:

1. Information Richness Analysis (Section 3): A static, inference-time study comparing how four levels of conditioning information (no context → full solution) change response length, epistemic token frequency, and answer correctness, establishing the monotonic relationship between richer context and suppressed uncertainty expression.

2. Off-Policy Self-Distillation (Section 4): A SFT study training on two datasets that differ only in epistemic density—unguided responses (high uncertainty) versus solution-guided responses (low uncertainty)—to isolate whether epistemic suppression, even with correct answers, degrades standalone reasoning.

3. On-Policy Self-Distillation (Section 5): A reinforcement learning study (GRPO vs. SDPO) across three model families and two thinking-mode configurations, tracking training dynamics, response length trajectories, epistemic token evolution, and OOD benchmark performance to link algorithmic choices directly to generalization failures.

4. Task Coverage Manipulation (Section 6): A controlled sweep of training set size (|D| ∈ {1, 8, 64, 128, 512}) under both GRPO and SDPO, revealing that self-distillation's conciseness bias is beneficial at low coverage but harmful at high coverage, with OOD performance monotonically worsening as training diversity increases.

Information flows linearly through these layers: the static analysis (Section 3) establishes the core mechanism (richer information → less uncertainty expression); the SFT experiments (Section 4) demonstrate that imitating confident reasoning without access to the teacher's privileged information causes standalone degradation; the on-policy experiments (Section 5) show this degradation generalizes across models, thinking modes, and teacher configurations; and the task coverage sweep (Section 6) explains the apparent contradiction with prior work by delineating when the degradation occurs (broad, diverse training sets) and when it does not (narrow, in-domain coverage).

3.3 Roadmap for the Deep Dive

• First, the information-theoretic framing and the role of epistemic verbalization, since the paper's entire analytical apparatus depends on defining what "richer information" means quantitatively and why uncertainty expression matters for reasoning. This includes the conditional mutual information formalization and the epistemic token measurement methodology.

• Second, the four-level conditioning experiment (Section 3), because this is the foundational empirical result that establishes the monotonic suppression effect—all subsequent experiments test whether this suppression translates into training-induced generalization failures.

• Third, the off-policy SFT experiment (Section 4), which is the cleanest causal test: two training datasets, identical except for epistemic density, producing divergent reasoning outcomes. This isolates the effect of imitating solution-guided reasoning style from confounds introduced by on-policy dynamics or RL optimization.

• Fourth, the on-policy self-distillation experiments (Section 5), covering SDPO vs. GRPO across DeepSeek-Distill-7B, Qwen3-8B (thinking on/off), and Olmo-3-7B-Instruct, plus the fixed vs. moving teacher ablation. These connect the static suppression mechanism to dynamic training trajectories and OOD evaluation.

• Fifth, the task coverage sweep (Section 6), which resolves the paper's central paradox by showing that the same mechanism (epistemic suppression) produces opposite outcomes depending on whether training and evaluation distributions are narrow (beneficial conciseness) or broad (harmful loss of adaptive uncertainty). This section also includes the cross-domain comparison with chemistry and code generation.

3.4 Detailed, Sentence-Based Technical Breakdown

This is an empirical diagnostic paper whose core idea is that self-distillation's teacher—conditioned on rich solution information—generates reasoning traces with suppressed epistemic verbalization, and that training the student to imitate this confident style degrades OOD reasoning when task coverage is broad because the student loses the uncertainty-driven exploration needed for genuinely novel problems.

---

Information-Theoretic Framing and Epistemic Verbalization

The paper grounds its analysis in an information-theoretic view of self-distillation, formalizing the teacher's privileged access through conditional mutual information, and operationalizing epistemic verbalization through a defined set of uncertainty markers.

Formalizing information richness. Let $x \in \mathcal{X}$ be an input prompt and $y = (y_1, \dots, y_T)$ be a generated reasoning sequence. The autoregressive model defines:

$\pi_\theta(y \mid x) = \prod_{t=1}^T \pi_\theta(y_t \mid x, y_{<t})$

where $\pi_\theta$ is the base language model, $x$ is the prompt, $y_t$ is the token at position $t$, and $y_{<t}$ denotes all previously generated tokens.

In self-distillation, the teacher receives an additional conditioning context $c$ (e.g., the ground-truth solution) that the student does not have:

$\pi_\theta^T(\cdot \mid x, c) = \pi_\theta(\cdot \mid x, c)$

The student generates without this context: $y \sim \pi_\theta(\cdot \mid x)$. Training minimizes the KL divergence:

$\mathcal{L}_{\text{SD}}(\theta) = \sum_t \text{KL}\left(\pi_\theta(\cdot \mid x, y_{<t}) \;\|\; \text{stopgrad}(\pi_\theta(\cdot \mid x, c, y_{<t}))\right)$

where $\mathcal{L}_{\text{SD}}$ is the self-distillation loss, $t$ indexes token positions, KL is Kullback-Leibler divergence, and stopgrad prevents gradients from flowing through the teacher's predictions (the teacher is treated as a fixed target distribution at each training step). The sum runs over all token positions in the generated sequence.

What this loss computes: at each token position $t$, the student's predicted next-token distribution (conditioned only on the prompt and previous tokens it generated) is compared against the teacher's predicted next-token distribution (conditioned additionally on the ground-truth solution $c$). The KL divergence penalizes the student for placing probability mass differently from the teacher. The stopgrad operator ensures the teacher is not updated—only the student policy changes to better match the teacher's more-informed predictions. The result is a scalar training loss that, when minimized, causes the student to generate sequences whose token-by-token probabilities resemble what a model with access to the solution would produce.

Why this form: the KL divergence is the natural objective when distilling a target distribution because it is the expected log-likelihood ratio under the student, meaning it heavily penalizes the student for assigning low probability to tokens the teacher considers likely. Stopgrad prevents the degenerate outcome where both models drift together; without it, the teacher would simply adapt to match the student, defeating the purpose of knowledge transfer. However, the paper's critical insight is that this objective only cares about matching token-level distributions—it is indifferent to whether the student retains the epistemic markers ("Wait," "Hmm") that the teacher suppresses when it has access to the solution. The loss does not distinguish between removing unnecessary verbosity and removing adaptive uncertainty signals.

Information-theoretic interpretation. The paper defines the additional information the teacher receives as the conditional mutual information:

$I(y; c \mid x) = H(y \mid x) - H(y \mid x, c)$

where $I(y; c \mid x)$ is the mutual information between the target response $y$ and the conditioning context $c$ given the prompt $x$, $H(y \mid x)$ is the conditional entropy of $y$ given $x$ alone (the student's uncertainty), and $H(y \mid x, c)$ is the conditional entropy of $y$ given both $x$ and $c$ (the teacher's reduced uncertainty after seeing the solution).

What it computes: the reduction in uncertainty about the response $y$ when the teacher receives the additional context $c$. If $c$ contains the full solution, $H(y \mid x, c)$ is very low—the teacher knows exactly what the correct reasoning should look like—so $I(y; c \mid x)$ is high. If $c$ is empty, $I(y; c \mid x) = 0$ by definition (no reduction because no information was provided). The magnitude of $I(y; c \mid x)$ corresponds to how much the teacher's behavior differs from the student's solely due to having access to privileged information.

Why this form: mutual information is the canonical information-theoretic quantity for measuring dependence between variables, and the conditioning on $x$ ensures we are measuring the additional information from $c$ beyond what the prompt already provides. The decomposition into entropy differences makes the mechanism transparent: higher $I(y; c \mid x)$ means the teacher effectively has a much easier problem than the student, and its reasoning style will reflect that reduced difficulty through shorter, more confident traces. The paper's central claim is that training the student to imitate this confident style—despite not having access to $c$ at inference time—is what causes OOD degradation.

Defining epistemic verbalization. Following prior work (understanding), the paper operationalizes "epistemic verbalization" through a specific set of 10 tokens:

$\mathcal{T} = \{\text{wait}, \text{hmm}, \text{perhaps}, \text{maybe}, \text{actually}, \text{alternatively}, \text{seems}, \text{might}, \text{likely}, \text{check}\}$

The epistemic token count for a response $y$ is computed as:

$E(y) = \sum_{t \in \mathcal{T}} \text{count}(t, y)$

where $E(y)$ is the total number of epistemic markers in the response, $\mathcal{T}$ is the fixed set of 10 tokens defined above, and $\text{count}(t, y)$ is the occurrence count of token $t$ in response $y$. The sum ranges over all 10 tokens in the defined set.

What this computes: a simple aggregate count of how many times the model produced explicit uncertainty markers during a single reasoning trace. A response with many "Wait" and "Hmm" tokens would score high; a response that proceeds confidently from start to finish would score low. The paper reports $\mathbb{E}[E(y)]$, the average over all generated responses in a given experimental condition.

Why this form: the 10-token set is a practical, computable proxy for the model's uncertainty expression, not a theoretically complete inventory of epistemic language. A model could express uncertainty through other tokens or through sentence-level constructions not captured by this count. The paper acknowledges this limitation implicitly by noting the set is a "practical indicator" from prior work. The proxy's value is that it is automatically computable, correlates strongly with response length and conditioning richness (Table 1), and enables quantitative tracking of how training algorithms reshape reasoning style. The alternative—human annotation of uncertainty expression—would be prohibitively expensive at scale and introduce annotator noise.

The self-Bayesian reasoning connection. The paper frames math reasoning as "self-Bayesian reasoning" where the model iteratively updates beliefs over intermediate hypotheses, generating each step conditioned only on the problem and previous tokens. Within this framework, epistemic verbalization serves a functional role: tokens like "Wait" signal that a previous step may be flawed and trigger hypothesis revision, while "Alternatively" signals exploration of a different path. Removing these markers is not merely stylistic cleanup—it removes the model's mechanism for detecting and recovering from its own errors. This framing explains why the degradation is domain-specific: math requires multi-step logical deduction where early errors compound if not caught, while chemistry problems with narrow coverage may rely more on pattern matching where confident tracing is sufficient.

---

The Four-Level Conditioning Experiment (Section 3)

This experiment is the foundational empirical contribution: it directly measures how response characteristics change as the teacher's conditioning context becomes richer, establishing the monotonic suppression effect before any training occurs.

Experimental design and problem selection. Using the DAPO-Math-17k dataset and DeepSeek-R1-Distill-Qwen-7B as the base model, the authors select 100 problems on which the base model achieves accuracy between 0.125 and 0.5 over 8 rollouts. This selection criterion is deliberate: problems that are too easy (accuracy near 1.0) would show no room for degradation; problems too hard (accuracy near 0.0) would not benefit from any conditioning; the intermediate difficulty range ensures the model's reasoning can meaningfully change under different information conditions. Let $s$ denote the full solution (including chain-of-thought content within \think tags), $s_{\setminus\text{think}}$ denote the solution with \think content removed, and $\tilde{y}$ denote a response previously generated under full solution guidance. The four generation settings are:

Setting (1): Unguided generation ($c = \emptyset$). The model receives only the problem prompt with no additional information, so $I(y; c \mid x) = 0$ by definition. The prompt used is: "{question}\n\nPlease reason step by step, and put your final answer within \boxed{}." This is the baseline representing the student's natural reasoning behavior when solving problems from scratch.

Setting (2): Solution-guided generation ($c = s$). The model receives the full solution, including the chain-of-thought reasoning within \think tags. This provides maximal guidance, yielding the largest $I(y; c \mid x)$ among all four settings. The teacher effectively sees the correct answer and the reasoning steps, so it can produce confident, concise responses that implicitly assume the solution is known.

Setting (3): Solution-guided generation without think contents ($c = s_{\setminus\text{think}}$). The model receives the solution but with the \think portion removed (the paper's Table 1 reveals this removes 640 out of 13,054 response tokens in the unguided case, a substantial information reduction). Since $s_{\setminus\text{think}}$ is a strict informational subset of $s$, we have $I(y; s_{\setminus\text{think}} \mid x) \leq I(y; s \mid x)$. This setting tests whether the chain-of-thought content specifically drives the suppression effect or whether the final answer alone is sufficient.

Setting (4): Regeneration-conditioned generation ($c = \tilde{y}$). The model receives $\tilde{y}$, a response it previously generated under Setting (2)'s full solution guidance. The prompt follows the SDPO regeneration format: "{question}\n\nPlease reason step by step, and put your final answer within \boxed{}.\n\nCorrect solution: {previously correct solution}\n\nCorrectly solve the original question." By the data processing inequality, $I(y; \tilde{y} \mid x) \leq I(y; s \mid x)$. This setting simulates the self-distillation scenario where the teacher's output from a previous iteration serves as the conditioning for the current generation.

Information ordering. These settings induce a clear ordering over the conditional mutual information:

$I(y; c \mid x) = 0 \text{ at (1)} < I(y; s_{\setminus\text{think}} \mid x) \text{ at (3)} \leq I(y; \tilde{y} \mid x) \text{ at (4)} \leq I(y; s \mid x) \text{ at (2)}$

This ordering enables a direct test: if richer information causes epistemic suppression, then we should observe monotonically decreasing response length and epistemic token count as we move from setting (1) to (2), with settings (3) and (4) falling in between.

Results of the four-level comparison (Table 1). The paper reports three metrics per setting: average score (accuracy on the 100 selected problems), average response length $\mathbb{E}[L(y)]$ in tokens, and average epistemic token count $\mathbb{E}[E(y)]$. The results are:

• Setting (1), unguided: score 0.30, length 13,054, epistemic tokens 182.5
• Setting (2), full solution $c = s$: score 0.98, length 1,873, epistemic tokens 8.8
• Setting (3), solution without think $c = s_{\setminus\text{think}}$: score 0.78, length 12,036, epistemic tokens 159.8
• Setting (4), regeneration-conditioned: score 0.95, length 2,808, epistemic tokens 24.1

Pattern 1: Monotonic length reduction. Both response length and epistemic token count follow the predicted monotonic ordering:

• For length: $\mathbb{E}[L(y)]|_{(1)} = 13{,}054 > \mathbb{E}[L(y)]|_{(3)} = 12{,}036 > \mathbb{E}[L(y)]|_{(4)} = 2{,}808 > \mathbb{E}[L(y)]|_{(2)} = 1{,}873$
• For epistemic tokens: $\mathbb{E}[E(y)]|_{(1)} = 182.5 > \mathbb{E}[E(y)]|_{(3)} = 159.8 > \mathbb{E}[E(y)]|_{(4)} = 24.1 > \mathbb{E}[E(y)]|_{(2)} = 8.8$

This confirms the central mechanism: richer conditioning information produces more concise and confident reasoning traces. The jump from unguided (13,054 tokens) to full-solution-guided (1,873 tokens) represents a roughly 7× reduction in length and a roughly 21× reduction in epistemic markers.

Pattern 2: The think content is informationally dense. The difference between settings (1) and (3) is relatively small (13,054 vs. 12,036 length; 182.5 vs. 159.8 epistemic tokens), suggesting that removing the chain-of-thought content leaves the remaining solution still quite sparse in useful guidance. However, the difference between settings (2) and (3) is enormous—removing think content causes the length to explode from 1,873 to 12,036 and epistemic tokens from 8.8 to 159.8. This indicates that the chain-of-thought reasoning within \think tags contains the bulk of the uncertainty-reducing information.

Pattern 3: Regeneration preserves much of the full solution's structure. Setting (4), where the context is the model's own regeneration from full-solution guidance, achieves length (2,808) and epistemic tokens (24.1) much closer to full solution guidance (1,873 and 8.8) than to unguided generation (13,054 and 182.5). This means $\tilde{y}$ preserves most of the informative structure of $s$, making it an effective compression of the solution guidance. This has direct implications for self-distillation: the teacher's regenerated responses under solution conditioning will themselves be concise and confident, so training the student on them will propagate the same epistemic suppression.

Pattern 4: Per-token breakdown consistency (Appendix A.1.1, Figure 9). All ten epistemic tokens follow the same ordering across settings, with "wait," "maybe," and "perhaps" showing the largest variation. For example, "wait" drops from prominent usage in unguided generation to near-zero in full solution guidance. This confirms that the suppression is not limited to a subset of markers—it is a broad restructuring of the model's uncertainty expression.

What this experiment establishes but also what it does not show. The four-level comparison demonstrates that richer conditioning context suppresses epistemic verbalization at inference time—a static, one-shot effect. It does not yet show that training on these suppressed traces causes the student to permanently lose uncertainty expression. That causal link requires the SFT and on-policy experiments that follow. However, the static experiment provides the mechanistic foundation: it establishes the monotonic relationship between $I(y; c \mid x)$ and epistemic suppression that makes self-distillation's degradation mechanism plausible.

---

Off-Policy Supervised Fine-Tuning with Self-Distillation (Section 4)

This experiment provides the cleanest causal test: it trains the model on two datasets that differ only in their epistemic density, holding all other factors (correctness, model architecture, optimization) constant.

Dataset construction. Using DeepSeek-R1-Distill-Qwen-7B, the authors construct two datasets, each containing 800 correct responses:

• $\mathcal{D}_{\text{ug}}$: unguided responses generated under $c = \emptyset$ (Setting 1 from Section 3). These have high epistemic token count ($\mathbb{E}[E(y)]$ high) and long response length ($\mathbb{E}[L(y)] \approx 12\text{k}$ tokens).
• $\mathcal{D}_{\text{sg}}$: solution-guided responses generated under $c = s$ (Setting 2 from Section 3). These have low epistemic token count ($\mathbb{E}[E(y)]$ low) and short response length ($\mathbb{E}[L(y)] \approx 2\text{k}$ tokens).

The crucial design choice: both datasets consist entirely of correct trajectories. The only difference between them is the style of reasoning—one expresses uncertainty and explores alternatives extensively; the other proceeds confidently with minimal backtracking. If training on $\mathcal{D}_{\text{sg}}$ degrades performance while training on $\mathcal{D}_{\text{ug}}$ does not, the degradation cannot be attributed to training on incorrect answers; it must be attributed to the suppression of epistemic verbalization.

Training procedure. The base model DeepSeek-R1-Distill-Qwen-7B is fine-tuned using standard supervised fine-tuning (SFT) on each dataset separately. The training objective is the standard next-token prediction loss (cross-entropy) on the response tokens. No reinforcement learning, no KL divergence to a teacher, no verifier—just maximum likelihood estimation on the 800 examples. This simplicity isolates the effect of the training data's reasoning style from algorithmic confounds.

Evaluation benchmarks and baseline performance (Table 2). The trained checkpoints are evaluated on four math benchmarks: AIME24, AIME25, AMC23, and MATH500. The base model's performance serves as the reference:

• Base DeepSeek-R1-Distill-Qwen-7B: AIME24 54.79, AIME25 37.92, AMC23 89.06, MATH500 92.19
• SFT on $\mathcal{D}_{\text{ug}}$ (unguided, high epistemic density): AIME24 51.04, AIME25 40.00, AMC23 87.66, MATH500 90.93
• SFT on $\mathcal{D}_{\text{sg}}$ (solution-guided, low epistemic density): AIME24 20.21, AIME25 12.71, AMC23 57.03, MATH500 65.52

Interpretation of the results. Training on the unguided dataset ($\mathcal{D}_{\text{ug}}$) produces no significant performance change—the small fluctuations (AIME24 drops from 54.79 to 51.04; AIME25 rises from 37.92 to 40.00) are within the variance expected from fine-tuning on 800 examples, and the model retains its core reasoning capability. In contrast, training on the solution-guided dataset ($\mathcal{D}_{\text{sg}}$) causes catastrophic degradation: AIME24 drops by roughly 34.6 percentage points, AIME25 by roughly 25.2 points, AMC23 by roughly 32 points, and MATH500 by roughly 27 points.

The causal mechanism. Since $\mathcal{D}_{\text{sg}}$ contains only correct answers, the degradation cannot be blamed on the model learning wrong final answers. Instead, the paper argues the model learns a reasoning style that is inappropriate for standalone problem-solving. The solution-guided responses are concise and confident precisely because the model that generated them had access to the ground-truth solution $s$. When the student is trained to imitate these responses without having access to $s$ at inference time, it learns to reason as if it already knows the answer—skipping the uncertainty checks, backtracking, and alternative exploration that the unguided model uses to detect and correct errors. The student effectively overfits to a reasoning pattern that is only valid given privileged information it does not possess.

Connection to prior work. The paper notes these results are "consistent with understanding, which shows that suppressing epistemic verbalization significantly degrades reasoning performance." The key extension is that the suppression is not achieved through explicit removal of tokens but through imitation of a teacher's reasoning style—a subtler mechanism that standard training pipelines would not detect because the training loss and training accuracy would appear normal (the model is learning to match the teacher's high-confidence traces).

Limitations of this experiment. The SFT experiment uses off-policy data (pre-generated responses), while real self-distillation is typically on-policy (the student generates its own responses, then learns from the teacher's evaluation of those responses). The paper addresses this with the on-policy experiments in Section 5. Additionally, the datasets contain only 800 examples, which is small relative to typical fine-tuning datasets; this may amplify the style-overfitting effect. The task coverage experiments in Section 6 directly address this concern.

---

On-Policy Self-Distillation: SDPO vs. GRPO (Section 5)

This section provides the paper's most extensive empirical investigation, comparing two on-policy training algorithms—GRPO (Group Relative Policy Optimization) and SDPO (Self-Distillation via Preference Optimization)—across three model families, two thinking-mode configurations, and multiple conditioning contexts. The goal is to establish that the epistemic suppression observed statically in Section 3 translates into training-induced degradation under realistic on-policy conditions.

Algorithm descriptions. GRPO is a reinforcement learning from verifiable rewards (RLVR) algorithm that works as follows: for each prompt, the policy generates multiple rollouts (8 in the paper's experiments). Each rollout receives a binary reward based on whether its final answer matches the ground-truth answer. The policy is updated to increase the probability of rollouts with above-average rewards and decrease the probability of rollouts with below-average rewards, using a clipped surrogate objective similar to PPO but with the group mean as the baseline rather than a learned value function.

SDPO combines self-distillation with preference optimization: the student generates a response (without solution access), and the teacher generates a response to the same prompt but with access to the ground-truth solution. The training objective is the JS divergence (Jensen-Shannon) between the student's and teacher's next-token distributions, applied only to the top-K tokens (K = 100 by default) to avoid forcing the student to match the teacher's distribution over unlikely tokens. The teacher policy is fixed to the initial checkpoint (EMA rate 0.0) because the paper finds this yields better performance than a moving target (Section 5.4).

Common training configuration. Unless otherwise noted, all experiments share the hyperparameters in Tables 4-6: AdamW optimizer, maximum prompt length 2048, maximum response length 20480, question batch size 256 (reduced to 64 for the task coverage experiments in Section 6 to accommodate smaller training sets), mini-batch size 64 or 128, 8 rollouts per prompt, inference engine vLLM, temperature 1.0, 10 warmup steps, weight decay 0.01, gradient clip norm 1.0. GRPO-specific: $\epsilon$-high 0.28, IS clip 2, KL coefficient ($\lambda$) 0.0, learning rate $1 \times 10^{-6}$. SDPO-specific: distillation divergence Jensen-Shannon, top-K distillation 100, EMA update rate 0.0, learning rate $1 \times 10^{-5}$. The 10× higher learning rate for SDPO reflects the different loss landscape of KL- vs. policy-gradient-based optimization.

Training data. All experiments use DAPO-Math-17k, which the paper describes as containing 14,000 distinct problems with train/eval splits that do not overlap with the evaluation benchmarks (AIME 2024/2025, AMC 2023, MATH 500). The prompt format is simplified from the original DAPO format to "{question}\nPlease reason step by step, and put your final answer within \boxed{}." because the authors observed this format "consistently yielded higher evaluation performance."

DeepSeek-R1-Distill-Qwen-7B results (Figure 3, Section 5.1). This model is a "representative high-reasoning model" known for extensive epistemic verbalization within \think tags and long responses.

Training dynamics (Figure 3a): GRPO slightly increases response length $\mathbb{E}[L(y)]$ with modest score improvement. SDPO with $c = s$ (full solution conditioning) causes a sharp initial drop in both length and score; performance gradually recovers but remains below GRPO throughout training. When conditioning is reduced to $c = s_{\setminus\text{think}}$ (solution without think content), the length drop is attenuated and the score trajectory approaches that of GRPO. This directly mirrors the static findings from Section 3: richer conditioning → more aggressive epistemic suppression → larger performance degradation.

OOD evaluation (Figures 3b, 3c): GRPO yields modest gains on both benchmarks (AIME24: 54.79 → ~56.0; AMC23: 89.06 → ~91.1) with slight length increase. SDPO with $c = s$ degrades performance substantially—approximately 40% drop on AIME24 (from ~55 to ~15) and approximately 15% drop on AMC23 (from ~89 to ~74). Reducing conditioning to $c = s_{\setminus\text{think}}$ mitigates the degradation but performance still falls below the base model.

Reasoning pattern (Figure 3d): The paper tracks epistemic token counts on AIME24 across training. GRPO increases $\mathbb{E}[E(y)]$ relative to the base model (the model expresses more uncertainty as it learns). SDPO suppresses $\mathbb{E}[E(y)]$ aggressively, with $c = s$ showing stronger suppression than $c = s_{\setminus\text{think}}$. The correlation is clear: algorithms that preserve or increase epistemic expression maintain or improve OOD performance; algorithms that suppress it degrade OOD performance.

Qwen3-8B with thinking mode ON (Figure 4, Section 5.2). This model initially generates "very long responses, even longer than those of DeepSeek-R1-Distill-Qwen-7B, along with a high number of epistemic tokens" (Appendix A.1.2).

Training dynamics (Figure 4a): Unlike DeepSeek-Distill-7B, Qwen3-8B with thinking mode shows decreasing response length under both GRPO and SDPO, but SDPO's reduction is larger and accompanied by a correspondingly larger performance drop. A notable pattern: the response length "first drops sharply then increases slightly." The paper interprets this as follows: the teacher policy is fixed to the initial reference policy, so as the student shortens its responses by approximately 900 tokens, the informational content of the teacher's conditioning context decreases (less response → less $I(y; c \mid x)$). The student then compensates by increasing epistemic verbalization, causing length to partially recover. This is a dynamic feedback effect not present in the static analysis.

OOD evaluation (Figures 4b, 4c): GRPO maintains largely stable performance with gradually decreasing length. SDPO falls below the base model, particularly with $c = s$. A subtle finding: although GRPO and SDPO with $c = s_{\setminus\text{think}}$ achieve comparable training performance, their OOD results diverge—especially on the more challenging AIME24, where SDPO with $c = s_{\setminus\text{think}}$ shows progressive performance degradation as training proceeds. This suggests that training-set performance can be a misleading indicator when epistemic suppression is at play; the model appears to be learning equally well by in-domain metrics while its OOD generalization quietly erodes.

Reasoning pattern (Figure 4d): Both methods reduce $\mathbb{E}[E(y)]$ relative to the base model, with SDPO more aggressively so. The paper interprets this as Qwen3-8B originally generating "more epistemic verbalization than necessary," so some reduction is beneficial (explaining GRPO's stable performance despite decreasing uncertainty expression), but SDPO's aggressive suppression removes epistemically useful signals, causing OOD degradation.

Qwen3-8B with thinking mode OFF (Figure 5, Section 5.3). Without the \think tag, this configuration tests the model's default (non-reasoning-enhanced) behavior.

Training dynamics (Figure 5a): The model initially produces much shorter responses and exhibits significantly lower performance than with thinking mode on. GRPO rapidly increases $\mathbb{E}[L(y)]$ by promoting epistemic verbalization (the paper notes this in Appendix D.1), quickly achieving a high training score. In contrast, SDPO reduces $\mathbb{E}[L(y)]$ and improves much more slowly. This is the starkest contrast: GRPO learns to express more uncertainty and improves dramatically; SDPO suppresses uncertainty and struggles.

OOD evaluation (Figure 5b): Even when the SDPO training score slightly increases, AIME24 performance slightly declines (from 0.25 to 0.23), while at the same training step, GRPO's AIME24 performance has improved substantially. This illustrates "the cost of epistemic suppression under self-distillation" in its purest form: the model is being rewarded for correct answers on the training set, but the reasoning style it adopts to achieve those rewards is brittle to novel problems.

Olmo-3-7B-Instruct results (Appendix D.2, Figure 14). This model from a different family confirms the findings are "not model-dependent but reflect robust characteristics of reasoning behavior across diverse model families." SDPO degrades performance with OOD evaluation scores falling below the base model, consistent with the DeepSeek and Qwen results.

Fixed vs. Moving Target Teacher (Section 5.4, Figure 6). The paper investigates an important implementation detail: should the teacher policy be updated during training (EMA-smoothed to the current student) or fixed to the initial checkpoint? The default EMA rate in SDPO is 0.05, but the authors find that setting it to 0.0 (fixed teacher) yields better performance.

Results (Figure 6a): Even slow EMA updates (rate 0.05) lead to a sharper reduction in response length and larger performance degradation compared to a fixed teacher. Figure 6b shows the corresponding AIME24 evaluation: the moving-target teacher amplifies the degradation.

Mechanism: The paper identifies a "feedback loop in self-distillation: the model is trained to produce increasingly confident outputs, and when a checkpoint of the same model is used as the teacher, it generates even more confident responses, amplifying the effect over iterations." This is because the updated student, when serving as teacher for the next iteration, has already suppressed some epistemic verbalization, so its solution-conditioned responses are even more confident and concise. Training the next iteration's student on these even-more-confident traces further suppresses uncertainty, creating a vicious cycle. The fixed teacher breaks this cycle because the teacher's confidence level remains constant.

Pass@16 results (Appendix D.3, Figure 15). The paper also reports pass@16 scores, which measure whether the correct answer appears in any of 16 sampled responses. The gap between GRPO and SDPO in pass@16 is larger for DeepSeek-Distill-7B than for Qwen3-8B, and more pronounced on the harder benchmark (AIME24) compared to AMC23. This metric is less susceptible to selection bias (since pass@16 does not require selecting which of 16 responses is best) and confirms that the degradation is in the model's ability to generate correct reasoning paths, not merely in the verifier's ability to identify them.

Ablation on hyperparameters (Appendix E, Figure 16). The paper tests whether the degradation can be mitigated by hyperparameter choices. Increasing top-K distillation from 100 to 256 yields "no significant difference in training dynamics or final performance" (Figure 16a). Reducing the learning rate from $1 \times 10^{-5}$ to $1 \times 10^{-6}$ "merely slows the rate of degradation; the model ultimately converges to the same reasoning behavior" (Figure 16b). These negative results strengthen the paper's claim that the degradation is not a tuning artifact but a structural consequence of the self-distillation objective when applied to broad-coverage math reasoning.

---

Task Coverage and Generalization (Section 6)

This section resolves the paper's central paradox: why does self-distillation improve performance in chemistry and code generation (as reported in prior work) while degrading it in math? The answer, developed through a controlled sweep of training set size and a cross-domain comparison, is that task coverage determines whether epistemic suppression is beneficial or harmful.

The cross-domain puzzle. The paper compiles evidence from prior SDPO work showing that on Science Q&A (Chemistry, Physics, Biology, Materials Science), tool use, and LiveCodeBench v6, SDPO achieves higher accuracy than GRPO while producing substantially shorter outputs with fewer epistemic markers. In these domains, self-distillation simultaneously suppresses epistemic verbalization and improves performance—the exact opposite of the math results.

Table 3: Dataset composition analysis. The paper contrasts three domains to explain this discrepancy:

• Science Q&A (Chemistry): 2,400 questions total, but drawn from only six main problem types (Reaction Balancing, Molecular Descriptor Counting, Molecular Weight Calculation, Property Prediction, Precursor/Reactant Selection, Product Prediction). The train/eval split is 90/10, meaning evaluation problems come from the same narrow distribution as training. Problems differ primarily in "surface details rather than underlying structure."

• LiveCodeBench v6: 131 total problems, with all 131 used for both training and evaluation. Only 50% of public test cases are used during training; full test cases (including hidden) are used for evaluation. The small total size means the model sees repeated exposure to the same problem types.

• DAPO-Math-17k: 14,000 distinct problems spanning a "broad, non-overlapping range of problem types." Training and evaluation use completely disjoint problem sets. The paper notes that 78% of the 25,600 samples drawn over 100 steps are distinct due to repeated sampling, emphasizing the diversity.

The hypothesis: when task coverage is narrow (chemistry, code), epistemic suppression is beneficial because it removes unnecessary uncertainty expression for problem types the model sees repeatedly. The model can learn a compact pattern-matching strategy that works on the limited set of problem structures. When task coverage is broad (math with thousands of distinct problem types and OOD evaluation), epistemic suppression is harmful because the model encounters genuinely novel problems that require uncertainty-driven exploration and error recovery.

The task coverage sweep (Section 6.2). To test this hypothesis directly, the paper varies the number of training questions $|\mathcal{D}| \in \{1, 8, 64, 128, 512\}$ from DAPO-Math-17k and trains with both GRPO and SDPO using Qwen3-8B with thinking mode OFF.

Training logs (Figure 7): When $|\mathcal{D}| \leq 128$, SDPO "quickly achieves high scores while reducing $\mathbb{E}[L(y)]$ by up to 8×," showing higher training efficiency than GRPO on small task sets. However, at $|\mathcal{D}| = 512$, further length reductions begin to hurt the training score relative to GRPO, whose response length gradually increases with $|\mathcal{D}|$. The interpretation: as the model must accommodate more reasoning patterns (more problems), GRPO addresses this by increasing epistemic expression (longer responses, more uncertainty markers), allowing adaptation to diverse problem structures. SDPO's confident, concise style works well when the problem set is small enough to be memorized or pattern-matched, but becomes limiting when the problem set demands genuine adaptation.

OOD evaluation (Figure 8): The distinction becomes pronounced on OOD benchmarks (AIME24, MATH500). Under GRPO, performance scales consistently with $|\mathcal{D}|$: $|\mathcal{D}| = 1$ converges quickly but soon stops improving; larger $|\mathcal{D}|$ yields progressively higher final scores accompanied by increasing response length. Under SDPO, the pattern reverses: smaller $|\mathcal{D}|$ leads to more severe OOD degradation. Even at the largest $|\mathcal{D}|$ (512, which is still a subset of DAPO-Math-17k and much smaller than the full 14,000 problems), SDPO still underperforms the base model. The implication is striking: self-distillation's conciseness gains are most beneficial precisely when they are least needed (narrow, in-domain evaluation), and most harmful precisely when they are most needed (broad, OOD evaluation).

Epistemic token analysis (Figure 11, Appendix A.2): Across GRPO and SDPO with $|\mathcal{D}| \in \{1, 64, 512\}$, GRPO tends to increase epistemic token usage relative to the base model as $|\mathcal{D}|$ grows, whereas SDPO reduces epistemic token usage, with smaller $|\mathcal{D}|$ leading to greater reduction. The $|\mathcal{D}| = 512$, SDPO condition shows the smallest reduction in epistemic markers (the model is forced to retain some uncertainty expression to handle the diverse problem set), but performance still degrades, suggesting that any net suppression of epistemic verbalization is harmful when the evaluation distribution is OOD and diverse.

Synthesis: When does epistemic suppression help vs. hurt? The paper's results support a clear boundary: epistemic suppression is beneficial when the model needs to compress repeated, structurally similar reasoning patterns into efficient traces (narrow coverage, in-domain evaluation)—it removes genuinely redundant uncertainty expression. Epistemic suppression is harmful when the model encounters novel problem structures that require exploration, hypothesis testing, and error recovery (broad coverage, OOD evaluation)—it removes adaptive uncertainty signals that the model needs for robust reasoning. Self-distillation's information asymmetry (teacher has solution, student does not) creates a consistent pressure toward the former regime, which explains why the algorithm succeeds in chemistry and code generation but fails in math with broad coverage.

Comparison with OPSD (Appendix C). The paper briefly contrasts its setup with OPSD (zhao2026self), which demonstrated performance gains in mathematical reasoning through self-distillation on the Qwen3 series. OPSD uses a hybrid configuration where the student has thinking mode disabled while the teacher has it enabled—effectively making this "more akin to conventional teacher–student distillation, despite using the same underlying model" because the thinking-enabled teacher produces longer, more epistemic-rich traces. The paper's experiments with this hybrid setup on Qwen3-1.7B (Figure 12a) show an initial performance improvement followed by degradation as training progresses, suggesting the hybrid approach delays but does not eliminate the epistemic suppression problem. The paper explicitly notes that this configuration "is inherently limited to model families such as Qwen3 that support toggling thinking mode on and off."

---

Summary of Key Design Choices and Their Justifications

• Four-level conditioning experiment with $I(y; c \mid x)$ ordering rather than a binary rich-vs-poor comparison: enables the paper to establish a monotonic relationship between information richness and epistemic suppression, which is stronger evidence than a simple two-point comparison and supports the claim that the degree of suppression can be modulated by controlling information content.

• Selection of problems with accuracy 0.125–0.5 for the static analysis: avoids ceiling effects (too-easy problems show no variation) and floor effects (too-hard problems cannot benefit from any conditioning), ensuring the experiment can detect meaningful changes in reasoning behavior.

• Off-policy SFT with identical-correctness datasets that differ only in epistemic density: the cleanest possible causal test—hold correctness, model, optimization, and dataset size constant; vary only the epistemic style of the training traces. Any performance difference must be attributed to style transfer, not to learning wrong answers.

• Fixed teacher (EMA rate 0.0) rather than moving target (EMA rate 0.05): breaks the feedback loop where increasingly confident students produce increasingly confident teachers, which would amplify epistemic suppression over iterations and conflate the algorithm's steady-state behavior with its transient dynamics.

• Task coverage sweep ($|\mathcal{D}| \in \{1, 8, 64, 128, 512\}$) rather than a single full-dataset comparison: reveals the non-monotonic relationship between coverage and the harmfulness of epistemic suppression, explaining why prior work on narrow-domain tasks reached opposite conclusions. Without this sweep, the paper could only report that self-distillation degrades math reasoning; with it, the paper can explain when and why it degrades vs. improves.

• Cross-model validation (DeepSeek, Qwen, OLMo) rather than a single model family: rules out model-specific artifacts and establishes epistemic suppression as a general mechanism rather than a quirk of a particular training pipeline or architecture.

• Comparison against GRPO as baseline rather than against a no-training baseline: ensures the degradation is attributed to self-distillation specifically, not to RL-based fine-tuning in general. GRPO serves as a control that undergoes similar training (same data, same number of steps, same reward signal) but does not impose the information asymmetry that drives epistemic suppression.

4. Key Insights and Innovations

Innovation 1: Epistemic Verbalization as a Functionally Necessary Reasoning Component, Not Stylistic Redundancy

The paper's most fundamental intellectual contribution is recasting expressions of uncertainty—tokens like "Wait," "Hmm," "perhaps"—from stylistic tics or inefficiencies to be optimized away into functional mechanisms for error detection and recovery during reasoning. Prior work on post-training, particularly self-distillation and RLVR, treated response length reduction as an unqualified good: SDPO explicitly celebrates that it "learns to reason concisely," and the dominant narrative in the field equates shorter traces with better models. The assumption—rarely stated but broadly operational—was that verbose reasoning contains waste, and that compressing it while maintaining correctness produces a more capable model.

This paper refutes that assumption not by arguing length reduction is always bad, but by differentiating between two types of response content that length-based metrics conflate: genuinely redundant verbiage and epistemically functional uncertainty signals. The evidence in Table 1 makes this distinction concrete: unguided generation produces 182.5 epistemic tokens and 13,054 total tokens; full solution guidance collapses these to 8.8 and 1,873 respectively—but the resulting confident traces are only valid given access to the solution, not as a general reasoning strategy. When the student is trained to imitate these traces without the teacher's privileged information (Section 4), performance on AIME24 drops from 54.79 to 20.21, despite training exclusively on correct answers.

This is a fundamental conceptual shift, not an incremental refinement. It means that evaluating post-training methods solely by training accuracy and response length—the standard metrics in the self-distillation literature—is systematically blind to a failure mode that degrades generalization without leaving traces in in-domain performance. The paper effectively argues that reasoning style is a first-class optimization target, orthogonal to answer correctness, and that standard objectives provide no signal about whether useful epistemic behaviors are being preserved or destroyed. This insight generalizes beyond self-distillation: any training method that rewards confident, correct outputs (RL with verifiable rewards, rejection sampling, standard distillation) risks the same suppression if it does not explicitly account for uncertainty expression.

The contrast with prior work is sharp. understanding established the correlation between epistemic marker removal and performance drops, but treated it as a static property of reasoning traces. This paper shows it is a training-induced, algorithm-dependent phenomenon—the suppression is not a pre-existing feature of certain responses but is actively caused by the self-distillation objective's pressure toward the teacher's confident distribution. This transforms epistemic verbalization from a descriptive observation into a diagnostic concept for evaluating post-training algorithms.

Innovation 2: Information Asymmetry as the Root-Cause Mechanism, Not Reward Hacking or Optimization Failure

When self-distillation degrades performance, the natural diagnostic assumption would be some form of reward hacking—the model exploiting the verifier, overfitting to spurious patterns, or the RL optimization diverging. Prior work on failure modes in RL fine-tuning (e.g., reward over-optimization in RLHF, policy collapse in PPO) has established a vocabulary of optimization pathologies that would be the default explanation.

The paper explicitly rejects these explanations and provides a different causal account grounded in information theory. The degradation is not due to the model learning incorrect answers (both SFT datasets in Section 4 contain only correct responses) nor to RL instability (GRPO with the same reward signal improves performance). Instead, the mechanism is structural: the self-distillation teacher receives conditioning context $c$ that provides mutual information $I(y; c \mid x)$ about the target response, and this privileged information reshapes the teacher's reasoning style to presuppose knowledge the student will not have at inference time. The student, trained via KL divergence to match this style, learns to reason as if it has access to the solution—a strategy that collapses when the answer is not obvious.

The evidence for this account is unusually clean. The four-level conditioning experiment (Section 3, Table 1) establishes a monotonic dose-response relationship: as $I(y; c \mid x)$ increases from 0 (unguided) to maximum (full solution), epistemic tokens drop from 182.5 to 8.8, and response length from 13,054 to 1,873. The off-policy SFT experiment (Section 4) then isolates causality: two datasets differing only in the epistemic density produced by this information asymmetry, with training on the confident traces causing catastrophic degradation while training on the uncertain traces preserves performance. The fixed-vs-moving teacher ablation (Section 5.4) further shows that the degradation is not an optimization artifact but an equilibrium property—a fixed teacher still causes degradation, while a moving teacher amplifies it through a feedback loop.

This is a fundamental diagnostic contribution that changes how the field should analyze post-training failures. It identifies a failure mode that is invisible to training metrics (the model achieves high training scores by producing correct answers using a confident style that overfits to the teacher's privileged information) and that operates through a mechanism (imitation of information-conditioned reasoning) distinct from the reward hacking and distribution shift narratives that dominate RL fine-tuning analysis. The implication is that self-distillation's teacher-student setup contains an inherent tension—the teacher is solving an easier problem than the student, so imitating the teacher's process is harmful even when imitating its outputs would be beneficial—that no amount of hyperparameter tuning can eliminate (as the negative ablation results in Appendix E confirm).

Innovation 3: Task Coverage as the Modulator That Determines Whether Epistemic Suppression Helps or Hurts

The paper's most practically significant insight is resolving the apparent contradiction between self-distillation's success in chemistry and code generation (prior work) and its failure in math (this paper) through a single variable: task coverage. This is not a domain-specific claim ("self-distillation works for science but not for math") but a structural principle: epistemic suppression is beneficial when training and evaluation distributions are narrow and overlapping, and harmful when they are broad and disjoint.

The evidence architecture supporting this claim is the paper's most sophisticated contribution. Table 3 provides a qualitative comparison: Chemistry has 2,400 questions but only six problem types with shared structure; LiveCodeBench has 131 problems with identical train/eval splits; DAPO-Math-17k has 14,000 distinct problems with evaluation on completely disjoint benchmarks. The task coverage sweep in Figure 8 provides the quantitative causal test: as $|\mathcal{D}|$ increases from 1 to 512 under SDPO, OOD performance on AIME24 and MATH500 worsens—the opposite of GRPO, where more data monotonically improves OOD performance. At $|\mathcal{D}| = 1$, SDPO actually outperforms GRPO (the conciseness bias is beneficial when the model can memorize a single problem), but by $|\mathcal{D}| = 512$, SDPO underperforms the base model while GRPO shows substantial gains.

This finding fundamentally reframes the evaluation of post-training methods. Prior work assessed self-distillation by measuring in-domain accuracy and length reduction, concluding it was beneficial when both metrics improved—which is exactly what happens under narrow coverage. The paper shows this evaluation protocol is circular: the conditions that make self-distillation look good (narrow coverage, in-domain evaluation) are precisely the conditions where epistemic suppression is harmless, masking the degradation that would appear under realistic deployment conditions where the model encounters novel problems. This is not merely an incremental caution about overfitting—it is a diagnostic framework that explains when and why a popular algorithm will succeed or fail, with testable boundary conditions.

The cross-domain comparison also reveals a subtlety that distinguishes this from standard generalization failure. The degradation under broad coverage is not because the model "forgets" how to solve training problems (training scores remain reasonable under SDPO at $|\mathcal{D}| = 512$ in Figure 7). Rather, the model learns a reasoning style that is locally optimal for the training distribution but structurally inappropriate for novel problems—a form of stylistic overfitting that is invisible to loss-based or accuracy-based monitoring. This concept has no direct precedent in the post-training literature and suggests that evaluation on OOD benchmarks is not merely best practice but is necessary to detect this failure mode, since in-domain metrics will show improvement or stability.

Innovation 4: Self-Distillation as an Amplifier of Reasoning Style, Not a Neutral Knowledge Transfer Mechanism

The paper reveals a dynamic that is absent from how self-distillation is typically understood: the algorithm does not merely transfer the teacher's knowledge about correct answers to the student; it actively reshapes the student's reasoning style through the interaction of the KL divergence objective and the teacher's information-conditioned confidence. This reshaping has a directionality—toward less uncertainty expression—that is consistent across all model families, thinking modes, and conditioning contexts tested.

This is significant because prior formulations of self-distillation frame it as knowledge transfer: the teacher knows more (because it sees the solution), and distillation moves that knowledge into the student. Under this framing, the only risk is imperfect transfer—the student might not fully learn what the teacher knows. The paper shows the actual dynamic is more complex: the teacher not only knows more but also reasons differently because of what it knows, and training the student on the teacher's reasoning process transfers a style that is maladaptive when the knowledge that produced it is absent. The fixed-vs-moving teacher ablation (Figure 6) demonstrates this is not a static transfer but an amplification dynamic: even a fixed teacher causes degradation, but a moving teacher (where the increasingly confident student becomes the next iteration's teacher) creates a feedback loop that accelerates epistemic suppression.

This insight has implications beyond self-distillation. Any training paradigm that exposes a model to its own outputs conditioned on privileged information—self-play with access to environment rewards, iterative refinement with oracle feedback, data augmentation using model-generated solutions—risks the same dynamic: the model learns to imitate a reasoning style that presupposes information it will not have at deployment. The paper effectively identifies a new class of training pathologies: information-conditioned style collapse, where the model's reasoning process adapts to the information available during training rather than the information available during inference. This is conceptually distinct from distribution shift (where training and deployment data differ), reward hacking (where the model exploits the reward function), and catastrophic forgetting (where the model loses previously learned capabilities). It is a structural property of algorithms that create asymmetric information between training and inference—a category that includes self-distillation but potentially extends to other self-supervised and self-play paradigms.

The finding that this dynamic operates even with a fixed teacher (EMA rate 0.0) and cannot be eliminated by learning rate reduction or top-K adjustment (Appendix E, Figure 16) underscores that this is not a tuning issue but an inherent property of the self-distillation objective when applied to broad-coverage reasoning tasks. The objective pushes the student toward the teacher's distribution, the teacher's distribution is shaped by privileged information, and the student's loss of uncertainty expression is not penalized because the KL divergence only cares about token-level distribution matching, not about the functional role those tokens play in reasoning. This is a fundamental limitation that suggests mitigating epistemic suppression will require architectural or objective-level changes, not hyperparameter adjustments.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The primary training dataset is DAPO-Math-17k (from dapo), containing approximately 14,000 distinct math problems spanning a broad, non-overlapping range of problem types. For the static analysis in Section 3, the authors select a subset of 100 problems on which the base model achieves accuracy between 0.125 and 0.5 over 8 rollouts. For the off-policy SFT experiments in Section 4, they construct two 800-example datasets from model-generated responses. For the task coverage sweep in Section 6, they subsample training sets of size |D| ∈ {1, 8, 64, 128, 512} from DAPO-Math-17k. The cross-domain comparison in Section 6 draws on prior SDPO results using Science Q&A (Chemistry, specifically 2,400 questions covering six problem types with a 90/10 train/eval split) and LiveCodeBench v6 (131 questions with identical train/eval splits). Evaluation benchmarks for OOD testing are AIME 2024, AIME 2025, AMC 2023, and MATH 500—all of which use questions not present in DAPO-Math-17k, making them strictly out-of-distribution relative to training.

• Base model(s). Three model families are tested to establish cross-architecture robustness: DeepSeek-R1-Distill-Qwen-7B (a distilled reasoning model known for extensive epistemic verbalization within \think tags and long response lengths, achieving 54.79 on AIME24, 37.92 on AIME25, 89.06 on AMC23, and 92.19 on MATH500 as the base model in Table 2), Qwen3-8B in both thinking mode ON and OFF configurations (the thinking-ON variant initially generates responses even longer than DeepSeek-Distill-7B with high epistemic token counts; the thinking-OFF variant produces much shorter responses and exhibits substantially lower base performance), and OLMo-3-7B-Instruct (tested in Appendix D.2 to confirm findings are not model-family-dependent). These models span different pretraining recipes, reasoning architectures, and base capabilities, making consistent degradation patterns across them strong evidence for a general mechanism rather than a model-specific artifact.

• Metrics. Three categories of metrics are tracked. Performance metrics: training score (the fraction of rollouts achieving correct final answers via exact match followed by mathematical equivalence checking using math-verify), acc@16 (the fraction of problems where the correct answer appears in any of 16 sampled responses), and pass@16 (a related metric reported in Appendix D.3). For OOD evaluation, the same correctness verification is applied to the selected final answer from each benchmark. Response characteristics: average response length $\mathbb{E}[L(y)]$ measured in tokens, and average epistemic token count $\mathbb{E}[E(y)]$ computed as $\sum_{t \in \mathcal{T}} \text{count}(t, y)$ where $\mathcal{T} = \{\text{wait}, \text{hmm}, \text{perhaps}, \text{maybe}, \text{actually}, \text{alternatively}, \text{seems}, \text{might}, \text{likely}, \text{check}\}$ (following the epistemic marker set from understanding). These are reported per-response and then averaged across all generations in a given experimental condition.

• Baselines. The primary comparative baseline is GRPO (Group Relative Policy Optimization), a reinforcement learning from verifiable rewards (RLVR) algorithm that generates multiple rollouts per prompt (8 in these experiments), computes advantages relative to the group mean, and updates the policy using a clipped surrogate objective. GRPO serves as the control because it operates on the same training data with the same reward signal but does not introduce the information asymmetry (teacher conditioned on solutions) that defines self-distillation. For the static analysis in Section 3, the baseline is unguided generation ($c = \emptyset$), representing the model's natural reasoning behavior without any conditioning information. For the OPSD comparison in Appendix C, the baseline is hybrid teacher-student distillation where the student has thinking mode disabled while the teacher has it enabled, following the configuration from zhao2026self.

• Generation budget / compute accounting. The paper does not report FLOPs or wall-clock time as the primary compute metric. Instead, training steps serve as the implicit compute axis for on-policy experiments—Figures 3, 4, 5, 6, 7, 8, and 14 all plot metrics against training steps, with consistent hyperparameters (batch size, rollout count) ensuring comparability between GRPO and SDPO at the same step count. For inference-time analysis (Section 3, Table 1), the comparison is across four conditioning settings at a fixed generation budget (single rollouts per problem). For evaluation, all methods use 16 rollouts per problem (acc@16) with the best response selected by the verifier, ensuring the test-time compute budget is identical across compared methods.

• Cross-validation / statistical protocol. The paper does not employ cross-validation in the traditional sense of repeated train/test splits. Instead, evaluation is conducted on fixed, held-out benchmarks (AIME 2024/2025, AMC 2023, MATH 500) that are completely disjoint from the DAPO-Math-17k training data, making the evaluation strictly OOD. For the static analysis in Section 3, problem selection uses a filtering criterion (base model accuracy between 0.125 and 0.5 over 8 rollouts) rather than random sampling, which controls for floor and ceiling effects but may introduce selection bias relative to the full problem distribution. For the SFT experiments in Section 4, each training run uses a fixed 800-example dataset, and evaluation is a single pass over the full benchmark test sets—there is no averaging over multiple training seeds or data orderings, which means the reported scores are point estimates without confidence intervals. The paper acknowledges (implicitly) the robustness limitation this creates by validating across three model families rather than relying on statistical replication within a single model.

Main Quantitative Results

Static Information Richness Analysis (Section 3, Table 1)

The foundational empirical result establishes the monotonic relationship between conditioning information and epistemic suppression. On 100 DAPO-Math-17k problems selected for intermediate difficulty (base model accuracy 0.125–0.5), DeepSeek-R1-Distill-Qwen-7B is evaluated under four conditioning settings with strictly increasing mutual information $I(y; c \mid x)$:

Setting (1), unguided ($c = \emptyset$, $I(y; c \mid x) = 0$): Average score 0.30, average response length $\mathbb{E}[L(y)] = 13{,}054$ tokens, average epistemic token count $\mathbb{E}[E(y)] = 182.5$. This is the baseline representing the model's natural reasoning behavior when solving problems from scratch—long, exploratory, and rich in uncertainty markers.

Setting (2), full solution guidance ($c = s$, maximum $I(y; c \mid x)$): Average score 0.98, $\mathbb{E}[L(y)] = 1{,}873$, $\mathbb{E}[E(y)] = 8.8$. Providing the full solution (including chain-of-thought within \think tags) collapses response length by approximately 7× (from 13,054 to 1,873 tokens) and epistemic markers by approximately 21× (from 182.5 to 8.8) while driving accuracy to near-ceiling.

Setting (3), solution guidance without think contents ($c = s_{\setminus\text{think}}$): Average score 0.78, $\mathbb{E}[L(y)] = 12{,}036$, $\mathbb{E}[E(y)] = 159.8$. Removing only the \think content (which the paper notes accounts for 640 out of 13,054 tokens in the unguided case) causes a dramatic reversion toward unguided behavior—length recovers to 12,036 tokens and epistemic tokens to 159.8—indicating that the chain-of-thought reasoning carries the bulk of the uncertainty-reducing information.

Setting (4), regeneration-conditioned ($c = \tilde{y}$): Average score 0.95, $\mathbb{E}[L(y)] = 2{,}808$, $\mathbb{E}[E(y)] = 24.1$. Conditioning on the model's own response previously generated under full solution guidance produces results much closer to Setting (2) than to Setting (1)—length is only approximately 1.5× longer than full solution guidance and approximately 4.6× shorter than unguided—demonstrating that $\tilde{y}$ preserves most of the informative structure of the full solution.

The ordering holds exactly as predicted by the information-theoretic framework: $\mathbb{E}[L(y)]|_{(1)} > \mathbb{E}[L(y)]|_{(3)} > \mathbb{E}[L(y)]|_{(4)} > \mathbb{E}[L(y)]|_{(2)}$, and identically for $\mathbb{E}[E(y)]$. The per-token breakdown in Appendix A.1.1 (Figure 9) confirms that all ten epistemic tokens follow this ordering, with "wait," "maybe," and "perhaps" showing the largest variation across settings.

What this demonstrates: Richer conditioning information causes the model to produce more concise, confident reasoning traces—a static, inference-time effect. It does not yet show that training on such traces causes permanent degradation. That causal link requires the training experiments that follow.

---

Off-Policy SFT: Epistemic Density as the Causal Variable (Section 4, Table 2)

This experiment provides the cleanest causal isolation in the paper: two SFT datasets, each containing 800 correct responses from DeepSeek-R1-Distill-Qwen-7B, differing only in whether the responses were generated under unguided conditions ($\mathcal{D}_{\text{ug}}$, high epistemic density, $\mathbb{E}[L(y)] \approx 12\text{k}$ tokens) or under full solution guidance ($\mathcal{D}_{\text{sg}}$, low epistemic density, $\mathbb{E}[L(y)] \approx 2\text{k}$ tokens). Both datasets consist entirely of trajectories with correct final answers. The base model is fine-tuned separately on each dataset using standard next-token prediction loss, then evaluated on four math benchmarks.

SFT on $\mathcal{D}_{\text{ug}}$ (unguided, high epistemic density): Performance is essentially unchanged from the base model. AIME24 drops slightly from 54.79 to 51.04; AIME25 edges up from 37.92 to 40.00; AMC23 declines from 89.06 to 87.66; MATH500 drops from 92.19 to 90.93. These fluctuations are within the expected variance from fine-tuning on 800 examples, and the model retains its core reasoning capability.

SFT on $\mathcal{D}_{\text{sg}}$ (solution-guided, low epistemic density): Catastrophic degradation across all benchmarks. AIME24 collapses from 54.79 to 20.21 (a drop of approximately 34.6 percentage points, or approximately 63% relative reduction). AIME25 falls from 37.92 to 12.71 (approximately 25.2 points, 66% relative). AMC23 drops from 89.06 to 57.03 (approximately 32.0 points, 36% relative). MATH500 declines from 92.19 to 65.52 (approximately 26.7 points, 29% relative).

What this demonstrates: Since $\mathcal{D}_{\text{sg}}$ contains only correct answers, the degradation cannot be attributed to the model learning wrong final answers. The causal variable is the reasoning style of the training data: solution-guided responses are concise and confident precisely because the model that generated them had access to the solution, and training a student to imitate this style without that privileged access causes it to adopt a reasoning pattern that presupposes knowledge unavailable at inference time. The unguided dataset, despite containing the same correct answers, preserves the uncertainty expression and exploration patterns that enable the model to solve problems from scratch, and training on it produces no degradation.

---

On-Policy Self-Distillation: SDPO vs. GRPO (Section 5)

The on-policy experiments test whether the static suppression observed in Section 3 translates into training-induced degradation under realistic reinforcement learning conditions, tracking training dynamics, response characteristics, and OOD benchmark performance across multiple model configurations.

DeepSeek-R1-Distill-Qwen-7B (Figure 3, Section 5.1):

Training score and length (Figure 3a): GRPO training produces a slight increase in $\mathbb{E}[L(y)]$ with modest score improvement, consistent with the model learning to explore more extensively while improving correctness. SDPO with $c = s$ (full solution conditioning) causes a sharp initial drop in both $\mathbb{E}[L(y)]$ and score; performance gradually recovers but remains below GRPO throughout training. SDPO with $c = s_{\setminus\text{think}}$ (solution without think content) attenuates both the length reduction and the performance gap relative to GRPO, directly mirroring the information richness gradient from Table 1.

OOD evaluation (Figures 3b, 3c): The paper reports that GRPO yields "modest gains on both OOD benchmarks (AIME24: 54.7 → 56.0; AMC23: 89.3 → 91.1, Figures 3b and 3c) with a slight increase in $\mathbb{E}[L(y)]$." SDPO with $c = s$ causes severe degradation: approximately 40% drop on AIME24 and approximately 15% drop on AMC23, as shown in the same figures. Reducing conditioning to $c = s_{\setminus\text{think}}$ mitigates the degradation but performance still remains below the base model.

Epistemic token trajectory (Figure 3d): On AIME24, GRPO increases $\mathbb{E}[E(y)]$ during training—the model expresses more uncertainty as it learns—while SDPO suppresses $\mathbb{E}[E(y)]$, with $c = s$ showing stronger suppression than $c = s_{\setminus\text{think}}$. The figure shows this as a monotonic divergence: GRPO's epistemic token count trends upward, SDPO's trends downward, and the gap widens with training steps.

Qwen3-8B with thinking mode ON (Figure 4, Section 5.2):

Training dynamics (Figure 4a): Unlike DeepSeek-Distill-7B where GRPO increased length, Qwen3-8B shows response length decreasing under both GRPO and SDPO. However, SDPO's reduction is larger and accompanied by a larger performance drop. Notably, the paper observes a non-monotonic pattern: length "first drops sharply then increases slightly." The interpretation is that as the student's responses shorten by approximately 900 tokens, the fixed teacher's conditioning context becomes less informative (less response content → lower $I(y; c \mid x)$), causing the student to partially compensate by increasing epistemic verbalization, creating a partial length recovery.

OOD evaluation (Figures 4b, 4c): GRPO maintains "largely stable performance with gradually decreasing $\mathbb{E}[L(y)]$." SDPO falls below the base model, particularly with $c = s$. A critical finding: "although GRPO and SDPO with $c = s_{\setminus\text{think}}$ achieve comparable training performance, their OOD results diverge—especially on the more challenging AIME24, where SDPO with $c = s_{\setminus\text{think}}$ shows progressive performance degradation as training proceeds." This means training performance can be a misleading indicator—two algorithms achieving similar in-domain accuracy have qualitatively different OOD generalization trajectories, with SDPO's progressively worsening.

Epistemic token evolution (Figure 4d): Both methods reduce $\mathbb{E}[E(y)]$ relative to the base model, with SDPO more aggressive. The paper interprets this as Qwen3-8B originally generating more epistemic verbalization than necessary, so some reduction is beneficial (hence GRPO's stable performance), but SDPO's aggressive suppression removes useful uncertainty signals, causing OOD degradation.

Qwen3-8B with thinking mode OFF (Figure 5, Section 5.3):

Training dynamics (Figure 5a): This configuration produces the starkest contrast between algorithms. The base model initially produces much shorter responses and achieves substantially lower performance than with thinking mode enabled. GRPO "rapidly increases $\mathbb{E}[L(y)]$ by promoting epistemic verbalization (as shown in Appendix D.1), quickly achieving a high training score." SDPO "reduces $\mathbb{E}[L(y)]$ and improves much more slowly." The training score trajectories diverge dramatically: GRPO achieves high scores early in training while expanding response length; SDPO struggles to improve while compressing responses.

OOD evaluation (Figure 5b): The paper reports a specific numeric comparison: on AIME24, "even when the training score slightly increases, as shown in Figure 5b, performance on AIME24 slightly declines (0.25 → 0.23), further illustrating the cost of epistemic suppression under self-distillation." The extended results in Appendix D.1 (Figure 13) add nuance: on AMC23, SDPO improves acc@16 from 0.67 to 0.73 while reducing response length by roughly half—a genuine efficiency gain on the less-challenging benchmark—while on AIME24, acc@16 decreases from 0.25 to 0.23 and pass@16 drops "more substantially." The comparison with GRPO on AMC23 is instructive: SDPO achieves approximately a 6-point improvement with shorter responses, while GRPO achieves approximately a 36-point improvement at the cost of considerably longer responses—illustrating the fundamental tension between conciseness gains and performance gains that the paper argues cannot be simultaneously optimized through current self-distillation approaches.

OLMo-3-7B-Instruct (Appendix D.2, Figure 14):

The paper confirms consistent degradation patterns on this third model family. Figure 14a shows SDPO reducing training score and response length. Figure 14b shows OOD evaluation on AIME24 falling below the base model. The authors state this "confirms that our findings are not model-dependent but reflect robust characteristics of reasoning behavior across diverse model families."

Pass@16 results (Appendix D.3, Figure 15):

The pass@16 metric (whether the correct answer appears in any of 16 samples) is reported for DeepSeek-Distill-7B and Qwen3-8B (thinking enabled). Figure 15 shows that "the gap between GRPO and SDPO in pass@16 is larger for DeepSeek-Distill-7B than for Qwen3-8B, and more pronounced on the harder benchmark (AIME24) compared to AMC23." This metric confirms the degradation is in the model's ability to generate correct reasoning paths—not merely in the verifier's ability to select among candidates—since pass@16 does not depend on answer selection quality.

---

Task Coverage Sweep: When Epistemic Suppression Helps vs. Hurts (Section 6)

This experiment resolves the paper's central paradox by showing that the same mechanism (epistemic suppression) produces opposite outcomes depending on training set diversity. Using Qwen3-8B with thinking mode OFF, the authors train with both GRPO and SDPO on subsets of DAPO-Math-17k of sizes |D| ∈ {1, 8, 64, 128, 512}, tracking training dynamics and OOD evaluation on AIME24 and MATH500.

Training dynamics (Figure 7):

When |D| ≤ 128, SDPO "quickly achieves high scores while reducing $\mathbb{E}[L(y)]$ by up to 8×," demonstrating higher training efficiency than GRPO on small task sets. The extreme case is |D| = 1, where SDPO can essentially memorize the single problem and produce confident, correct traces with minimal length. However, at |D| = 512, "further reductions in $\mathbb{E}[L(y)]$ begin to hurt the training score relative to GRPO, whose $\mathbb{E}[L(y)]$ gradually increases with |D|." GRPO's response length grows monotonically with training set size—the model learns to express more uncertainty to handle more diverse problems—while SDPO's conciseness bias becomes increasingly misaligned with the demands of broader coverage.

OOD evaluation (Figure 8):

Under GRPO, OOD performance on both AIME24 and MATH500 "scales consistently with |D|." At |D| = 1, GRPO converges quickly but soon stops improving; at larger |D|, it achieves "progressively higher final scores accompanied by increasing $\mathbb{E}[L(y)]$." The pattern is monotonic: more diverse training data → longer, more uncertain responses → better OOD generalization.

Under SDPO, the pattern reverses: "smaller |D| leads to more severe OOD degradation." At |D| = 1, SDPO's OOD performance is worst (the model has maximally suppressed uncertainty to handle a single, memorizable problem). At larger |D|, the degradation is attenuated because the model is forced to retain some uncertainty to handle diverse problems, but critically, "even at the largest |D| (DAPO setting), SDPO still underperforms the base model." The paper does not report the exact accuracy values for all |D| settings on AIME24 and MATH500 in the main text, but shows them graphically in Figure 8.

Epistemic token analysis by |D| (Figure 11, Appendix A.2):

Comparing the change in epistemic token counts relative to the base model on AIME24 across GRPO and SDPO at |D| ∈ {1, 64, 512}: "GRPO tends to increase epistemic token usage relative to the base model as |D| grows, whereas SDPO reduces epistemic token usage, with smaller |D| leading to a greater reduction." The |D| = 512 SDPO condition shows the smallest net reduction in epistemic markers among SDPO runs, yet performance still degrades below the base model—suggesting that any net suppression of epistemic verbalization is harmful when evaluation is OOD, even if the suppression is partially mitigated by training diversity.

Cross-domain comparison (Table 3):

The paper contextualizes these results by comparing dataset characteristics across domains where SDPO succeeded versus failed. In Science Q&A (Chemistry), the 2,400 questions draw from only six problem types (Reaction Balancing, Molecular Descriptor Counting, Molecular Weight Calculation, Property Prediction, Precursor/Reactant Selection, Product Prediction) that "differ primarily in surface details rather than underlying structure." The 90/10 train/eval split means evaluation problems come from the same narrow distribution. In LiveCodeBench v6, the 131 total questions with identical train/eval splits lead to "repeated exposure during training." In contrast, DAPO-Math-17k exposes the model to 14,000 distinct problems spanning a broad range of non-overlapping types, with evaluation on completely disjoint benchmarks.

The paper explicitly states the implication: in domains with narrow coverage, "self-distillation suppresses epistemic verbalization and improves performance simultaneously" because the model can learn compact, pattern-matched reasoning that works for the limited problem types. In domains with broad, OOD coverage, "the same mechanism leads to performance degradation" because the model encounters genuinely novel problems requiring uncertainty-driven exploration and error recovery.

---

Fixed vs. Moving Target Teacher (Section 5.4, Figure 6)

The standard SDPO implementation uses an EMA-smoothed teacher with rate 0.05, meaning the teacher is a slowly-updated moving average of the student. The paper finds that setting the EMA rate to 0.0 (fixing the teacher to the initial checkpoint) yields better performance.

Training comparison (Figure 6a): With DeepSeek-R1-Distill-Qwen-7B, "even slow updates (e.g., rate 0.05) lead to a sharper reduction in response length, resulting in larger performance degradation" compared to the fixed teacher. The moving-target teacher produces a steeper length decline and lower training score.

OOD evaluation (Figure 6b): On AIME24, the fixed teacher configuration substantially outperforms the EMA teacher, with the gap widening over training steps.

Mechanism: The paper identifies a feedback loop: the model trained under self-distillation produces increasingly confident outputs; when a checkpoint of this model serves as the teacher for the next iteration, it generates even more confident (shorter, fewer epistemic markers) solution-conditioned responses; these are then used as targets for the next student iteration, further amplifying confidence, creating a vicious cycle. The fixed teacher breaks this loop because the teacher's confidence level remains constant at the initial model's baseline.

---

Comparison with OPSD Hybrid Distillation (Appendix C, Figure 12)

The paper contrasts its homogeneous setup (both student and teacher have the same thinking mode configuration) with OPSD's hybrid approach (student has thinking mode disabled, teacher has it enabled). On Qwen3-1.7B:

Hybrid setup (Figure 12a, orange line): In early training, "the thinking-enabled teacher drives the student toward longer responses with improved performance." However, as training progresses, "response length gradually decreases, accompanied by a corresponding degradation in performance." This creates a non-monotonic trajectory: initial improvement followed by decline.

Homogeneous setup (Figure 12b): With both student and teacher having thinking mode enabled, "performance declines consistently, while response length and epistemic token count also decrease steadily," consistent with the paper's main results.

The paper notes this hybrid configuration "is inherently limited to model families such as Qwen3 that support toggling thinking mode on and off" and that understanding the initial-improvement-then-decline dynamic "is beyond the scope of this work."

---

Hyperparameter Ablations (Appendix E, Figure 16)

Two hyperparameter sweeps test whether the degradation can be mitigated through tuning:

Top-K distillation (Figure 16a): Increasing K from 100 to 256 yields "no significant difference in training dynamics or final performance." The length suppression and score degradation trajectories are essentially identical.

Learning rate (Figure 16b): Reducing the learning rate from $1 \times 10^{-5}$ to $1 \times 10^{-6}$ "merely slows the rate of degradation; the model ultimately converges to the same reasoning behavior." The trajectories diverge in early training (lower LR delays the decline) but converge to the same degraded endpoint.

These negative results support the paper's claim that the degradation is not a tuning artifact but a structural consequence of the self-distillation objective.

---

Ablation Studies and Robustness Checks

• Information richness gradient (four conditioning levels): Table 1 and Figure 9 establish that response length, epistemic token count, and accuracy all vary monotonically with the mutual information between the conditioning context and the target response. The deliberate inclusion of two intermediate conditions ($s_{\setminus\text{think}}$ and $\tilde{y}$) eliminates the alternative explanation that the effect is binary (any solution access causes the shift) and instead demonstrates a graded dose-response relationship, strengthening the causal interpretation that information quantity—not merely information presence—drives the behavioral change.

• Per-token epistemic marker analysis (Appendix A.1.1, Figure 9): All ten tokens in the epistemic set follow the same ordering across conditioning levels, with "wait," "maybe," and "perhaps" showing the largest variation. This rules out the possibility that the aggregate trend is driven by a single token (e.g., a model-specific verbal tic) and confirms the suppression effect is broad-based across the uncertainty-expression vocabulary.

• Cross-model epistemic token comparison (Appendix A.1.2, Figure 10): Comparing DeepSeek-Distill-7B, Qwen3-8B (thinking ON), and Qwen3-8B (thinking OFF) reveals different preferred epistemic tokens and different baseline levels: Qwen3-8B thinking-ON produces the most epistemic tokens, followed by DeepSeek-Distill-7B, then Qwen3-8B thinking-OFF. However, all three show the same qualitative pattern—solution-guided generation dramatically reduces epistemic token counts—confirming the suppression mechanism is not tied to a specific model's epistemic vocabulary.

• Dataset correctness control (Section 4): Both off-policy SFT datasets contain only correct responses. The 34.6-point AIME24 drop on $\mathcal{D}_{\text{sg}}$ versus no significant change on $\mathcal{D}_{\text{ug}}$ isolates epistemic suppression as the causal variable, ruling out the alternative explanation that the model simply learned incorrect answers from noisy training data.

• Fixed vs. moving target teacher (Section 5.4, Figure 6): The fixed teacher (EMA rate 0.0) consistently outperforms the EMA-smoothed teacher (rate 0.05). This is a non-obvious finding: the standard SDPO implementation uses an EMA teacher, and the expectation would be that a slowly-updated teacher provides a more stable target. The paper's counter-intuitive result (fixed teacher is better) is explained by the feedback-loop mechanism—any teacher updating, even slow, amplifies epistemic suppression—and is validated by the monotonic relationship between EMA rate and degradation severity.

• Top-K distillation variation (Appendix E, Figure 16a): K ∈ {100, 256} produces no meaningful difference. This negative result eliminates the possibility that the degradation is caused by the KL divergence forcing the student to match the teacher's distribution over too narrow a set of tokens. Increasing K (matching more of the teacher's distribution) does not mitigate the suppression, suggesting the problem is not about which tokens are matched but about the direction of the distribution shift (toward confidence).

• Learning rate variation (Appendix E, Figure 16b): LR ∈ {$1 \times 10^{-5}$, $1 \times 10^{-6}$} only changes the rate of convergence to the same degraded behavior, not the endpoint. This negative result eliminates the possibility that the degradation is an optimization artifact caused by overly aggressive updates. Even with 10× slower learning, the model eventually converges to the same suppressed reasoning style.

• Thinking mode ON vs. OFF (Sections 5.2, 5.3): The degradation occurs in both Qwen3-8B configurations, with qualitative differences (thinking-ON starts with longer responses and the degradation is a compression of existing epistemic behavior; thinking-OFF starts with shorter responses and GRPO must first build up epistemic expression before the SDPO-vs-GRPO gap becomes visible). That the degradation manifests in both regimes—despite very different starting behaviors—suggests the phenomenon is robust to the initial reasoning style and operates through the same underlying mechanism (KL divergence toward the teacher's confident distribution).

• Three model families (DeepSeek, Qwen, OLMo): Consistent degradation across models with different architectures, pretraining recipes, and base capabilities (Appendix D.2, Figure 14; main Figures 3, 4, 5) provides the strongest available evidence that the finding is not model-specific. The OLMo results are particularly valuable because OLMo is a fully open model with documented training data and procedures, reducing the concern that the effect is driven by unknown pretraining artifacts in the distilled Qwen and DeepSeek models.

---

Critical Assessment

The paper's central claims, as distilled from the executive summary and prior sections, are: (1) self-distillation suppresses epistemic verbalization through the teacher's access to privileged solution information; (2) this suppression degrades reasoning performance, particularly OOD; (3) the degradation is modulated by task coverage—harmful when coverage is broad and evaluation is OOD, but potentially beneficial when coverage is narrow and evaluation is in-domain. How well do the reported experiments support each claim?

Claim 1 (self-distillation suppresses epistemic verbalization): Very strongly supported, but with a measurement caveat.

The evidence for this claim is the paper's strongest empirical contribution. The four-level conditioning experiment (Table 1) provides a clean dose-response curve: increasing $I(y; c \mid x)$ → decreasing $\mathbb{E}[E(y)]$ and $\mathbb{E}[L(y)]$, with a strictly monotonic relationship across all four settings and all ten epistemic tokens individually (Figure 9). The on-policy experiments (Figures 3d, 4d, 11) show that training under self-distillation actively reduces epistemic token usage over time, with the effect growing stronger as the teacher's conditioning becomes richer ($c = s$ vs. $c = s_{\setminus\text{think}}$). The fixed-vs-moving teacher ablation (Figure 6) adds mechanistic depth by demonstrating a feedback loop that amplifies suppression.

The measurement caveat concerns the epistemic token set $\mathcal{T}$. The paper uses 10 tokens from prior work (understanding), and all analyses are based on counting these specific tokens. However, the paper does not validate that this 10-token set captures all or most uncertainty expression. A model could express uncertainty through paraphrases ("I'm not sure," "Let me reconsider"), through sentence-level constructions not captured by token counts, or through structural features (backtracking, alternative branches) that do not contain any of the 10 tokens. The paper acknowledges this limitation implicitly by calling the set a "practical indicator"—not a complete inventory—but the analyses treat $\mathbb{E}[E(y)]$ as the primary metric for epistemic verbalization. If the 10-token set captures only a fraction of uncertainty expression, the reported suppression magnitudes may understate or overstate the true effect. Conversely, some of these tokens (e.g., "actually") could have non-epistemic uses, inflating counts. The paper would be stronger with a validation study—e.g., human annotation of a sample of responses to confirm that the token-based count correlates with perceived uncertainty expression—but such validation is absent.

Additionally, the epistemic token analysis is entirely correlational for the training experiments: Figures 3d, 4d, and 11 show that SDPO reduces both epistemic tokens and OOD performance, but they do not establish that the token reduction causes the performance drop. The causal link relies on the off-policy SFT experiment (Section 4), where the only difference between $\mathcal{D}_{\text{ug}}$ and $\mathcal{D}_{\text{sg}}$ is the epistemic density of the training responses, and training on $\mathcal{D}_{\text{sg}}$ causes catastrophic degradation. This is strong causal evidence, but it operates at the dataset level (high vs. low epistemic density) rather than the token level (removing specific epistemic markers causes specific failures). The paper does not conduct the stronger experiment of taking $\mathcal{D}_{\text{ug}}$ responses, removing only the 10 epistemic tokens, and showing that this token removal alone causes degradation—which would cleanly isolate those specific tokens as causally necessary. Without this experiment, the possibility remains that other stylistic features covarying with epistemic density (sentence complexity, step count, verification behaviors) are the true causal factors, and the 10 tokens are merely correlated markers.

Claim 2 (suppression degrades reasoning performance, particularly OOD): Supported, but the degradation magnitude varies enormously across configurations and the "OOD" characterization deserves scrutiny.

The evidence for performance degradation is extensive: off-policy SFT on $\mathcal{D}_{\text{sg}}$ drops AIME24 from 54.79 to 20.21 (Table 2); SDPO with $c = s$ causes approximately 40% drop on AIME24 for DeepSeek-Distill-7B (Figure 3b); Qwen3-8B thinking-ON shows progressive AIME24 degradation under SDPO even when training performance matches GRPO (Figure 4c); and OLMo-3-7B-Instruct shows OOD scores falling below the base model (Figure 14b). The cross-model replication is compelling.

However, the degradation magnitude is highly variable, and the paper does not systematically characterize this variability. On AIME24, the drop ranges from approximately 40 percentage points (DeepSeek-Distill-7B, SDPO with $c = s$) to approximately 2 points (Qwen3-8B thinking-OFF, AIME24 dropping from 0.25 to 0.23 in Figure 5b). On AMC23, SDPO with thinking-OFF actually improves from 0.67 to 0.73 (Figure 13b) while AIME24 degrades. This variability is not fully explained by the paper's framework. The task coverage analysis in Section 6 provides one axis of explanation (broad coverage → degradation), but all SDPO experiments in Section 5 use the full DAPO-Math-17k (broad coverage) and yet show different degradation magnitudes across models and thinking modes. The paper attributes some of this to baseline epistemic density (Qwen3-8B thinking-ON starts with "more epistemic verbalization than necessary," so some suppression is benign), but this is a post-hoc interpretation rather than a predicted or manipulated variable. The paper would be stronger with a systematic model-capability × epistemic-baseline × degradation-magnitude analysis.

The "OOD" characterization also warrants scrutiny. The paper treats AIME24, AIME25, AMC23, and MATH500 as OOD relative to DAPO-Math-17k because they use disjoint problem sets. This is valid in the strict sense of no problem overlap. However, these are all competition-level math benchmarks testing similar mathematical reasoning skills. They are OOD with respect to specific problems but potentially in-distribution with respect to problem type, difficulty, and required skills. The paper's own cross-domain analysis (Table 3) argues that Chemistry and CodeGen have narrow coverage because they involve few problem types, while DAPO-Math-17k has broad coverage. But the evaluation benchmarks (AIME, AMC, MATH) may also represent a narrower slice of math reasoning than the full DAPO-Math-17k distribution—competition math problems have characteristic structures that may not fully represent the diversity of the training set. If the evaluation benchmarks are genuinely OOD, the degradation is a generalization failure; if they are near-OOD or within the broader distribution, the degradation might instead reflect the model losing capabilities that were present at initialization. The paper does not analyze how the difficulty distribution of the evaluation benchmarks relates to the training distribution, which would strengthen the OOD claim.

A related concern: the paper does not report training set accuracy in a way that enables direct comparison with OOD accuracy. In Figures 3a, 4a, 5a, training scores are plotted as trajectories, but the final training accuracy at the checkpoint where OOD evaluation is performed is not always clearly stated. For the claim that "training performance and OOD performance diverge" (made for Qwen3-8B thinking-ON in Section 5.2), the evidence is that GRPO and SDPO with $c = s_{\setminus\text{think}}$ have comparable training curves (Figure 4a) but diverging OOD curves (Figure 4c). However, without explicit numeric comparison at matched training steps, the magnitude of this divergence is difficult to assess. The paper would be stronger with a scatter plot or table showing training accuracy vs. OOD accuracy at the final checkpoint for each configuration.

Claim 3 (task coverage modulates whether suppression helps or hurts): Strongly supported by the within-experiment sweep, but the cross-domain comparison relies on qualitative argumentation rather than controlled manipulation.

The task coverage sweep (Figures 7, 8, 11) is the paper's most sophisticated experimental design and provides compelling within-experiment evidence: using the same model, same algorithm, same data source, varying only the number of training problems, SDPO's OOD performance worsens as coverage increases while GRPO's improves. The reversal at small |D| (SDPO outperforms GRPO at |D| = 1 because conciseness is beneficial for memorization) strengthens the claim by showing the effect is not monotonic. The epistemic token analysis in Figure 11 adds mechanistic detail: SDPO's suppression magnitude decreases with |D| (the model is forced to retain some uncertainty), but any suppression still harms OOD performance.

The cross-domain comparison (Table 3) is weaker methodologically. The paper compares dataset characteristics across Chemistry, LiveCodeBench, and DAPO-Math-17k, arguing that narrow vs. broad coverage explains the success vs. failure pattern—but this is an observational comparison across different studies with different models, different hyperparameters, different evaluation protocols, and different domains (not just different coverage). The paper does not conduct the controlled experiment that would directly test the cross-domain claim: take a single model, create narrow-coverage and broad-coverage math datasets with matched difficulty, and show that SDPO outperforms GRPO on the former while underperforming on the latter. The within-experiment |D| sweep is consistent with the coverage hypothesis but does not establish it for the Chemistry/CodeGen domains specifically, because math at |D| = 1 is not the same as Chemistry at |D| = 2,400 (the problem types, required skills, and reasoning patterns differ). The paper's language appropriately hedges here ("We hypothesize that the answer lies in differences in task coverage"), but a reader should be aware that the cross-domain claim is an interpretation supported by analogy and correlation, not by direct experimental manipulation of coverage while holding domain constant.

Missing experiments that would strengthen the paper:

1. A direct intervention on epistemic tokens: Train on $\mathcal{D}_{\text{ug}}$ with the 10 epistemic tokens programmatically removed (or replaced) to test whether those specific tokens are causally necessary for the performance preservation. If removing just those tokens from the unguided dataset causes degradation, the causal chain is closed. If not, other features covarying with epistemic density are the true causal factors.

2. A controlled coverage experiment holding domain constant: Take a single domain (e.g., math), create datasets with systematically varied problem-type diversity (e.g., only algebra problems vs. mixed algebra/geometry/arithmetic vs. full diversity), and test SDPO vs. GRPO across these conditions. This would directly test whether coverage—not domain—is the modulating variable.

3. Training accuracy at evaluation checkpoints: For every on-policy experiment, report the training set accuracy at the specific training step where OOD evaluation is performed, enabling quantitative comparison of in-domain vs. OOD performance. The current plots show training trajectories and OOD trajectories on separate axes or figures, making it difficult to directly quantify the divergence.

4. More granular difficulty analysis: The paper's central mechanism (epistemic suppression removes error-correction behaviors) predicts that degradation should be most severe on problems requiring multi-step reasoning with non-obvious intermediate steps—exactly the problems where "Wait" and backtracking are functionally necessary. Analyzing performance by problem difficulty or required reasoning depth would strengthen the mechanistic account. The existing difficulty binning is limited to the 100-problem static analysis (accuracy 0.125–0.5), which is a coarse filter.

5. Human evaluation of reasoning quality: The paper relies entirely on final-answer correctness as the evaluation metric. This captures whether the model got the right answer but not whether its reasoning process became genuinely less robust or merely less likely to stumble upon the correct final answer through exploration. Human evaluation of reasoning quality (coherence, logical validity, appropriate uncertainty) on a sample of generated traces would substantiate the claim that epistemic suppression degrades reasoning, not just answer accuracy.

6. Verifier quality analysis: The on-policy experiments (SDPO, GRPO) use the ground-truth answer for reward computation. The paper does not analyze whether the verifier (answer extraction + math-verify) introduces systematic biases—e.g., whether certain reasoning styles produce answers that are harder for the verifier to parse correctly. If SDPO's confident traces produce answers in a format that the verifier handles poorly, some apparent performance degradation could be a measurement artifact rather than genuine capability loss.

Overall assessment:

The paper's experiments genuinely support its core mechanistic claim—that self-distillation suppresses epistemic verbalization through the teacher's privileged information, and that this suppression can degrade OOD reasoning—with unusually clean causal evidence from the off-policy SFT experiment (Section 4) and convergent validity from the on-policy experiments across three model families. The task coverage analysis (Section 6) is the paper's most innovative empirical contribution, providing a within-experiment demonstration that the same algorithm's effects reverse depending on training set diversity.

The primary limitations are: (1) the epistemic token operationalization is an unvalidated proxy for uncertainty expression, and the causal role of those specific tokens (versus covarying stylistic features) is not directly tested; (2) the "OOD" characterization of the evaluation benchmarks is plausible but not systematically characterized—we do not know how problem difficulty and type distribution in AIME/AMC/MATH relate to DAPO-Math-17k; (3) the cross-domain coverage claim is supported by observational comparison rather than controlled manipulation; (4) the paper reports point estimates without confidence intervals or error bars, making the statistical reliability of the reported degradation magnitudes difficult to assess—this is particularly relevant for the smaller-scale experiments (100-problem static analysis, 800-example SFT datasets, |D| = 1, 8, 64 settings) where variance could be substantial; (5) several natural follow-up experiments (direct epistemic token removal, controlled coverage manipulation within a single domain, difficulty-stratified analysis) are absent, leaving some mechanistic claims at the level of strong correlation rather than demonstrated causation.

These limitations are consistent with the paper's self-positioning as a diagnostic investigation opening a research direction rather than a definitive solution. The paper explicitly flags several open questions (the OPSD hybrid dynamics in Appendix C, the need for better difficulty estimation, the extension to other domains) and does not claim to have solved the epistemic suppression problem—only to have identified it as the mechanism behind self-distillation's math degradation and characterized its boundary conditions. Within that scope, the experimental evidence is substantial and largely compelling, with the caveats above representing opportunities for strengthening rather than fatal flaws.

6. Limitations and Trade-offs

Epistemic Token Operationalization Remains an Unvalidated Proxy for Uncertainty Expression

The paper's entire mechanistic account rests on measuring epistemic verbalization through a fixed set of 10 tokens ($\mathcal{T} = \{\text{wait}, \text{hmm}, \text{perhaps}, \text{maybe}, \text{actually}, \text{alternatively}, \text{seems}, \text{might}, \text{likely}, \text{check}\}$). The paper explicitly borrows this set from prior work (understanding) and acknowledges its status as a "practical indicator" rather than a complete inventory. Crucially, no validation study—human annotation of a sample of reasoning traces, correlation with independent uncertainty metrics, or comparison against broader lexical inventories—is conducted to establish that counting these 10 tokens captures what the paper claims it captures: the model's functional expression of uncertainty during reasoning.

The consequence is a measurement validity problem with two distinct failure modes. First, the 10-token set may undercount uncertainty expression, since models can express doubt through paraphrases ("I'm not sure about this step"), sentence-level constructions ("Let me reconsider whether..."), or structural features (backtracking branches, explicit hypothesis comparison) that do not contain any of the ten tokens. If SDPO suppresses epistemic reasoning through these unmeasured channels rather than through the measured tokens, the paper's quantitative metrics ($\mathbb{E}[E(y)]$ dropping from 182.5 to 8.8 in Table 1, for example) would misrepresent the true magnitude and mechanism of suppression. Second, some of the 10 tokens may have non-epistemic uses—"actually" often serves as a discourse marker emphasizing a point rather than expressing revised uncertainty, and "check" could be a procedural instruction rather than an uncertainty signal. If non-epistemic uses are suppressed at different rates than genuinely epistemic uses, the aggregate count conflates style shifts with functional degradation.

The paper provides substantial evidence of a correlation between token counts and performance outcomes—Table 2 shows training on low-epistemic-density data degrades accuracy, Figures 3d and 4d show SDPO reduces both epistemic tokens and OOD performance—but correlation with a proxy does not establish that the proxy captures the construct of interest. The paper does not include an experiment that would close this gap: taking the high-epistemic-density dataset $\mathcal{D}_{\text{ug}}$ from Section 4, programmatically removing or replacing only the 10 epistemic tokens, and measuring whether this token-level manipulation alone causes the performance degradation observed when training on $\mathcal{D}_{\text{sg}}$. Without such an experiment, the causal claim that suppression of these specific 10 tokens drives the degradation remains an interpretation, not a demonstrated fact. The degradation might instead be caused by other stylistic features covarying with epistemic density—response length, sentence complexity, step count, explicit verification behaviors—that correlate with the 10-token count but are not captured by it.

The qualitative examples on the authors' blog (referenced in Section 4) likely provide illustrative traces, but no systematic content analysis appears in the paper to validate the proxy against human judgment of reasoning quality or uncertainty expression. For a practitioner, this limitation means the paper identifies epistemic verbalization as a correlate of robust reasoning but does not provide a validated operational definition for monitoring or preserving it during training. A practitioner wanting to prevent epistemic suppression in their own training pipeline cannot simply track these 10 tokens and assume that stable counts imply preserved reasoning capability—the proxy may be insufficiently sensitive or specific.

---

Difficulty Estimation Cost and Dynamic Adaptation Are Not Addressed

The experiments in Sections 3–5 use the full DAPO-Math-17k dataset or controlled subsets thereof, and evaluation is conducted on fixed held-out benchmarks. The task coverage analysis (Section 6) establishes that self-distillation's harmfulness depends on the breadth and diversity of the training distribution relative to the evaluation distribution. However, the paper provides no mechanism for identifying at deployment time whether a given prompt distribution is narrow enough for self-distillation to be safe or broad enough for it to be dangerous. The cross-domain comparison (Table 3) characterizes Chemistry as having six problem types and LiveCodeBench as having 131 problems, but this is a post-hoc qualitative analysis, not a deployable difficulty or coverage estimator.

The consequence for practitioners is a fundamental chicken-and-egg problem: the paper's central prescriptive insight—use self-distillation when coverage is narrow, avoid it when coverage is broad—requires knowing the coverage characteristics of the deployment distribution, but those characteristics are typically unknown before deployment. A practitioner training a math reasoning model for an application where users ask diverse, open-ended problems faces exactly the regime where the paper shows degradation (broad coverage, OOD evaluation), yet the paper provides no tool for quantifying how broad is "too broad." The 14,000-problem diversity of DAPO-Math-17k causes degradation, while the 6-problem-type Chemistry domain does not, but where the threshold lies—at 50 problem types? 500? 5,000?—is uncharacterized. Without this characterization, the paper offers a diagnosis of when self-distillation fails but not a decision procedure for practitioners to use when deciding whether to deploy it.

Furthermore, the paper does not explore dynamic difficulty or coverage estimation, where the model assesses during training or inference whether it is encountering problems requiring uncertainty-driven reasoning, and modulates its behavior accordingly. The static, one-size-fits-all conditioning in self-distillation—the teacher always receives the full solution, and the student always imitates the resulting confident traces—is what the paper identifies as the root cause of degradation, but no mechanism for adaptive conditioning (e.g., providing the teacher with only partial solutions on hard problems, or adjusting the KL divergence weight based on estimated problem difficulty) is proposed or tested. The OPSD hybrid setup (Appendix C) represents one attempt at softening the information asymmetry—the thinking-enabled teacher provides richer traces than a thinking-disabled teacher—but this is limited to specific model families and shows only a delay of degradation, not elimination.

The paper does not attempt to address this limitation. Section 7 acknowledges the need for "developing more robust training strategies" but does not propose specific mechanisms for coverage estimation or adaptive conditioning. This means the paper's practical contribution is currently cautionary (don't naïvely apply self-distillation to broad-coverage math reasoning) rather than prescriptive (here's how to safely apply self-distillation when coverage characteristics are unknown).

---

The "OOD" Characterization of Evaluation Benchmarks Is Not Systematically Verified

The paper treats AIME 2024, AIME 2025, AMC 2023, and MATH 500 as out-of-distribution relative to DAPO-Math-17k because they use completely disjoint problem sets. This is the basis for the paper's central degradation narrative: self-distillation suppresses epistemic verbalization needed for OOD generalization. The paper does report this setup explicitly in Table 3: "evaluation is conducted on standard benchmarks (AIME 2024/2025, AMC 2023, MATH 500) using questions not part of the training data."

However, "no problem overlap" is not synonymous with "out of distribution" in the sense relevant to the paper's claims. The evaluation benchmarks and DAPO-Math-17k are all competition-level math problems testing similar mathematical reasoning skills—algebra, geometry, number theory, combinatorics, and so on. They may be drawn from similar distributions of problem types, difficulty levels, and required reasoning patterns. The paper's own cross-domain argument (Table 3) distinguishes Chemistry's six problem types from the "broad, non-overlapping range of problem types" in DAPO-Math-17k, but the evaluation benchmarks may represent a narrower, more structured subset of that broad range—competition problems have characteristic formats, difficulty calibration, and solution structures that differ from the broader math reasoning distribution.

The consequence is that what the paper interprets as an OOD generalization failure may instead reflect a more complex dynamic: the model might be losing capabilities that were present at initialization but not specifically OOD generalization. If AIME24 problems are within the broader DAPO-Math-17k distribution (just not the exact training examples), the degradation represents the model forgetting how to solve problems it could previously solve, rather than failing to generalize to novel problem types. The paper's epistemic suppression mechanism would still be relevant, but the interpretation would shift: it would not be that uncertainty expression is needed for OOD problems specifically, but that uncertainty expression is needed for the model to maintain capabilities acquired during pretraining, and self-distillation's confident style overrides those capabilities.

The paper does not analyze the difficulty distribution, problem-type composition, or required reasoning depth of the evaluation benchmarks relative to DAPO-Math-17k. No experiment varies the OOD "distance"—for example, by creating held-out sets at varying levels of similarity to the training distribution and measuring whether degradation magnitude correlates with OOD distance. The task coverage sweep (Figure 8) shows that degradation worsens as training set diversity increases, which is consistent with the OOD interpretation (more diverse training → harder to memorize → more reliance on general reasoning strategies that require epistemic verbalization), but does not directly test whether the evaluation set's relationship to the training distribution modulates degradation magnitude. A practitioner would want to know: if my deployment distribution is somewhat similar to training but not identical, will self-distillation degrade performance? The paper provides no quantitative basis for answering this question beyond the binary observation that DAPO-Math-17k → AIME/AMC/MATH causes degradation.

---

Point Estimates Without Statistical Characterization Limit Reliability Assessment

Throughout the paper, quantitative results are reported as point estimates without confidence intervals, error bars, or statistical tests. Table 2 reports base model performance to two decimal places (AIME24: 54.79 → 20.21 after SFT on $\mathcal{D}_{\text{sg}}$). Figure 3 reports training trajectories and OOD evaluation curves without shaded regions indicating variance across runs. The task coverage sweep (Figures 7, 8) plots performance across five training set sizes but does not indicate whether the observed patterns are robust to data sampling (different random draws of |D| problems from DAPO-Math-17k) or training seed variation.

The consequence is that the reliability of the reported degradation magnitudes—and, more importantly, the robustness of the comparative claims between algorithms—is difficult to assess. This is particularly acute for the smaller-scale experiments:

• The static analysis (Section 3, Table 1) uses 100 problems selected for intermediate difficulty. With 100 problems, a ±5 percentage point confidence interval around the reported accuracy of 0.30 for unguided generation would be reasonable, meaning the apparent ordering between settings (1) and (3) (scores of 0.30 vs. 0.78) is robust, but finer comparisons within similar-performing settings might not be.

• The off-policy SFT experiment (Section 4) uses 800 examples for training and evaluates on the full benchmark test sets. The training dataset construction involves sampling from model-generated responses, which introduces randomness. If the 800 $\mathcal{D}_{\text{sg}}$ examples happened to contain particularly misleading confident traces, the catastrophic degradation (AIME24: 54.79 → 20.21) could partially reflect sampling variance. The paper reports no replication with different random seeds or different subsets of generated responses.

• The task coverage sweep at |D| = 1, 8, 64, 128, 512 uses specific subsamples of DAPO-Math-17k. At |D| = 1, the result depends entirely on which single problem is selected. At |D| = 8, the eight problems could be unusually easy or unusually diverse depending on the random draw. The paper reports each |D| as a single data point, not an average over multiple random draws, so the observed pattern—SDPO outperforms GRPO at small |D|, underperforms at large |D|—could be influenced by which specific problems were included at each size.

The paper partially mitigates this concern through convergent validity—the same qualitative patterns appear across three model families (DeepSeek, Qwen, OLMo), across two thinking modes, and across multiple evaluation benchmarks—but convergent validity across models does not substitute for statistical characterization within models. If training a Qwen3-8B model twice with different random seeds produces AIME24 degradation of 5% in one run and 15% in another, the practical implications change substantially.

The paper does not report statistical testing, does not mention multiple training runs per configuration, and does not include confidence intervals or error bars in any figure. This is a standard practice in the empirical LLM literature (driven by computational cost), but it limits the paper's prescriptive force. A practitioner deciding whether the degradation risk is worth the conciseness gains needs to know not just that SDPO degrades AIME24 by ~40% for DeepSeek-Distill-7B in one run, but whether that 40% figure is stable across reasonable variations in training data ordering and random seed.

---

The Cross-Domain Coverage Claim Lacks Controlled Manipulation

The paper's resolution of its central paradox—why self-distillation improves performance in chemistry and code generation while degrading it in math—relies on comparing dataset characteristics across domains (Table 3) and sweeping training set size within a single domain (Figures 7, 8). The within-domain sweep provides strong evidence that task coverage modulates self-distillation's effects when domain is held constant (DAPO-Math-17k). The cross-domain comparison, however, involves changing multiple variables simultaneously: domain (chemistry vs. math vs. code), dataset size (2,400 vs. 14,000 vs. 131), problem-type diversity (6 types vs. broad range vs. coding problems), model families (unspecified for the Chemistry/LiveCodeBench results from prior work vs. DeepSeek/Qwen/OLMo in this paper), hyperparameters, and evaluation protocols (in-domain vs. OOD, with the Chemistry and LiveCodeBench evaluations being in-domain or nearly so).

The consequence is that attribution of the performance difference—chemistry succeeds while math fails—to task coverage specifically is an interpretation supported by analogy rather than by direct experimental manipulation. The paper does not create a narrow-coverage math dataset (e.g., only linear algebra problems with surface variations) and show that self-distillation succeeds on it, nor does it create a broad-coverage chemistry dataset and show that self-distillation fails on it. The within-domain coverage sweep shows that coverage matters within math, but does not establish that coverage explains the cross-domain difference—the cross-domain difference could also be driven by domain-inherent properties (math requires more multi-step error correction than chemistry regardless of coverage; code generation benefits from concise reasoning because the compiler catches errors that epistemic verbalization would catch in math).

The paper is appropriately hedged about this limitation. Section 6 states: "We hypothesize that the answer lies in differences in task coverage between the training and evaluation distributions." The word "hypothesize" signals that the cross-domain claim is interpretive, not experimentally demonstrated. However, the paper's narrative—particularly the abstract's framing that suppression "enabl[es] rapid in-domain optimization with limited task coverage but harm[s] OOD performance"—treats the coverage account as the primary explanation for why self-distillation sometimes works and sometimes doesn't, and a practitioner reading the paper might reasonably conclude that narrow coverage guarantees safety. The controlled experiment that would test this—varying coverage while holding domain constant, and showing the effect reverses—is the one piece of evidence that would transform the coverage hypothesis from plausible interpretation to demonstrated mechanism. Its absence means a practitioner considering self-distillation for a new domain (planning, dialogue, scientific reasoning beyond chemistry) cannot simply assess coverage breadth and be confident in the outcome; domain-specific factors may interact with coverage in unknown ways.

---

Hardest Problems Show No Improvement and the Boundary of "Unsolvable" Is Not Characterized

Across multiple experiments, the paper shows that test-time strategies (self-distillation, SDPO, GRPO) provide minimal or zero benefit on the hardest problems, but the characterization of what makes a problem "hard" is limited to a single coarse filter. In the static analysis (Section 3), problems are selected for having base model accuracy between 0.125 and 0.5 over 8 rollouts—problems above this range are excluded as too easy (ceiling effects), and problems below this range are excluded as too hard (floor effects). This selection criterion serves the experimental design well (it isolates problems where conditioning can change behavior), but it means the paper's findings about epistemic suppression do not directly address the hardest problems—those where base model accuracy is below 0.125—which are also the problems where the practical stakes of degradation are highest.

The consequence for practitioners is ambiguity about the interaction between problem difficulty and epistemic suppression. The paper's mechanism suggests that epistemic verbalization is most valuable on problems requiring exploration, hypothesis testing, and error recovery—precisely the characteristics of hard problems where the model's initial intuitions are unreliable. If self-distillation's confident style causes the model to skip this exploration, the degradation should be most severe on the hardest problems. But the paper's experiments, by design, exclude the very hardest problems from the causal analysis. The 100-problem static analysis uses mid-difficulty problems; the SFT experiment evaluates on full benchmarks but does not break down results by problem difficulty; the task coverage sweep reports aggregate OOD performance without difficulty stratification. A deployment scenario where the model encounters a mix of easy, medium, and hard problems—the realistic case—cannot be assessed from the current data because we do not know whether the 40% degradation on AIME24 for DeepSeek-Distill-7B is concentrated on the hardest AIME problems (where epistemic expression would be most needed) or distributed across difficulty levels.

The paper's finding that "even at the largest |D|... SDPO still underperforms the base model" (Section 6.2) and that hard problems in the prior work's analysis (the example paper in the prior sections) show near-zero improvement regardless of test-time compute budget, suggests a general boundary: test-time strategies amplify existing capability but cannot create it. For self-distillation specifically, if the base model cannot solve certain problems even with extensive epistemic exploration, suppressing that exploration cannot help—but the paper does not quantify where that boundary lies in terms of base model pass@1, problem difficulty metrics, or required reasoning depth. A practitioner needs to know: for models with a given base capability level, on problem distributions with a given difficulty profile, does self-distillation's degradation affect all problems equally, or does it disproportionately harm the subset of problems where uncertainty expression is most functionally necessary? The paper provides no evidence to answer this question.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper introduces a diagnostic framework rather than a new algorithm, and its primary impact lies in reorienting how the field evaluates post-training methods—particularly self-distillation and related paradigms that create information asymmetry between training and inference. The shift is not a paradigm revolution but a substantive reframing with practical bite: the paper demonstrates that a widely-used training objective (KL divergence toward a solution-conditioned teacher) contains a structural failure mode that is invisible to standard evaluation metrics (training accuracy, response length) yet causes catastrophic OOD degradation on math benchmarks. This changes the burden of proof for self-distillation methods: prior work could claim success by showing improved in-domain accuracy with reduced length; this paper shows that this exact combination of outcomes can mask a model that has learned to reason as if it knows the answer—a style that collapses on genuinely novel problems.

The reframing has two specific consequences for how the field thinks about reasoning in LLMs. First, it elevates reasoning style from an implementation detail to a first-class optimization target. The paper's central finding is not that self-distillation fails to optimize the training objective—it often optimizes it efficiently (Figure 7, small |D| regimes)—but that the objective is indifferent to whether the model preserves uncertainty expression, backtracking, and hypothesis exploration. Standard loss functions (cross-entropy, KL divergence) treat "Wait, let me reconsider this step" and "The answer is 42" as sequences of tokens to be matched to a target distribution, with no signal distinguishing epistemically functional tokens from stylistic filler. This means that any loss function operating purely at the token level is blind to the functional role of epistemic verbalization, and practitioners evaluating such methods using only final-answer accuracy and response length will systematically miss this degradation mode until deployment reveals it. The paper's evidence that this blindness is not hypothetical—SDPO and GRPO achieve comparable training performance while diverging sharply on OOD benchmarks (Figure 4c)—makes the case that evaluation protocols in the post-training literature need to include explicitly OOD benchmarks, not merely held-out splits from the training distribution.

Second, the paper resolves a genuine contradiction in the post-training literature: why self-distillation and related RLVR methods produce strong gains in chemistry, code generation, and agentic environments (zhu2025token, SDPO, shenfeld2026self, song2026expanding, zhao2026self, opcd) while failing in math reasoning. The explanation—that task coverage modulates whether epistemic suppression is beneficial or harmful—is not domain-specific but structural. Narrow coverage means the model can safely compress reasoning into confident, pattern-matched traces because it sees the same problem types repeatedly; broad coverage with OOD evaluation means the model encounters genuinely novel problems where uncertainty-driven exploration is functionally necessary. This resolution shifts the research agenda: rather than asking "does self-distillation work?" (a question prior work answered inconsistently by testing on different implicit coverage regimes), the field should ask "under what coverage conditions does self-distillation's conciseness bias become harmful, and how can we detect those conditions at deployment time?"

The paper also redirects research attention toward verifier and training objective design rather than toward more sophisticated RL algorithms. The ablation results in Appendix E are revealing: changing top-K distillation from 100 to 256 changes nothing (Figure 16a); reducing the learning rate by 10× merely delays the same degradation (Figure 16b). This suggests the degradation is not an optimization artifact that can be tuned away within the current objective family. Mitigating it will require changes to the objective itself—explicitly rewarding or preserving uncertainty expression, conditioning the teacher more carefully, or designing verifiers that penalize overconfidence on problems where uncertainty is appropriate. The finding that a fixed teacher (EMA rate 0.0) outperforms the standard EMA-smoothed teacher (Section 5.4) provides a concrete, immediately actionable improvement (practitioners should not use EMA teachers for self-distillation on broad-coverage reasoning), but the deeper implication is that the feedback loop between student confidence and teacher confidence represents a general risk in any iterated self-training pipeline.

Finally, the paper implicitly challenges the narrative that "shorter reasoning traces are better"—a narrative that has driven significant investment in length-reduction techniques across the post-training literature. The paper does not argue that longer traces are always better (Qwen3-8B thinking-ON initially generates more epistemic verbalization than necessary, and GRPO's length reduction in that setting is benign). Rather, it argues that length reduction is not a monotonic good: compressing reasoning traces removes both waste and signal, and current training objectives provide no mechanism for distinguishing between them. This has implications beyond self-distillation for any method that rewards concise outputs—RL with length penalties, rejection sampling favoring shorter correct traces, or distillation from more capable models that naturally produce shorter reasoning.

Follow-Up Research This Work Enables

Direct causal test of epistemic token necessity via targeted removal. The paper establishes a strong correlation between epistemic token suppression and performance degradation, but the causal chain—that removing those specific tokens causes the degradation, rather than merely covarying with other stylistic features—remains untested. A direct follow-up would take the high-epistemic-density dataset $\mathcal{D}_{\text{ug}}$ from Section 4, programmatically remove or replace only the 10 epistemic tokens ($\mathcal{T}$) from the training responses (e.g., stripping "Wait, Hmm, perhaps" and their surrounding context sentences), then fine-tune the base model on this sanitized dataset. If performance degrades comparably to SFT on $\mathcal{D}_{\text{sg}}$ (AIME24 dropping from 54.79 toward 20.21), the causal role of those specific tokens is confirmed. If performance is preserved, the true causal factors are other features covarying with epistemic density—response length, verification behaviors, step count—and the 10-token set is merely a correlated marker. The experiment is computationally cheap (SFT on 800 examples) and would substantially clarify the mechanism.

Within-domain coverage manipulation to isolate coverage from domain. The paper's cross-domain claim—that task coverage explains why chemistry succeeds while math fails—is supported by analogy (Table 3) and within-math coverage sweep (Figures 7, 8), but not by controlled cross-domain manipulation. A strong follow-up would create narrow-coverage and broad-coverage versions of the same domain and test whether self-distillation's effects reverse. Concretely: take DAPO-Math-17k and construct two subsets—a "narrow" set of 500 algebra problems with surface-level variations (different coefficients, different wording, same underlying solution structure) and a "broad" set of 500 problems spanning algebra, geometry, number theory, combinatorics, and word problems with non-overlapping structure. Train SDPO vs. GRPO on both, evaluate on held-out benchmarks of each type. The prediction from the paper's framework: SDPO should match or outperform GRPO on the narrow set (conciseness is beneficial when problem structure is shared) but underperform on the broad set (uncertainty suppression harms OOD generalization). A null result—SDPO underperforms on both—would suggest domain-inherent factors (math requires epistemic reasoning regardless of coverage) rather than coverage per se is the modulating variable.

Explicit epistemic preservation objectives. The paper identifies a failure mode (suppression of epistemic verbalization) but proposes no mitigation beyond avoiding self-distillation in broad-coverage regimes. A natural algorithmic follow-up is to augment the self-distillation objective with an explicit term that rewards preservation of epistemic markers. One concrete approach: add an auxiliary loss that penalizes the KL divergence between the student's distribution over the 10 epistemic tokens and the unguided teacher's distribution over those tokens (i.e., require the student to match the base model's uncertainty expression even as it learns to produce correct answers). The experiment would compare standard SDPO against this augmented version on DAPO-Math-17k with OOD evaluation on AIME24/AMC23, measuring whether the augmented loss preserves epistemic token counts and prevents OOD degradation while still achieving training score improvements. A positive result would establish that epistemic expression can be explicitly optimized; a negative result (augmented loss hurts training score without preventing OOD degradation) would suggest that the necessary uncertainty signals are not captured by the 10-token set, motivating a broader operationalization.

Difficulty-stratified degradation analysis. The paper reports aggregate OOD degradation but does not characterize how the degradation distributes across problem difficulty. The mechanism (epistemic suppression removes error-recovery behaviors) predicts that degradation should be most severe on problems requiring multi-step reasoning with non-obvious intermediate steps—exactly where "Wait" and backtracking are functionally necessary. A follow-up would take the trained checkpoints from the Section 5 on-policy experiments (SDPO vs. GRPO on DeepSeek-Distill-7B and Qwen3-8B) and evaluate them on AIME24 problems stratified by difficulty, using either human difficulty ratings, average number of reasoning steps in correct solutions, or the base model's pass@1 rate as a difficulty proxy (following the example paper's quintile binning approach from the style reference, adapted to this context). The prediction: SDPO's degradation relative to GRPO should be concentrated in the hardest difficulty bins, while both methods perform comparably on easy problems. A flat degradation across difficulty would challenge the mechanistic account and suggest a more general capability loss rather than specific suppression of error recovery.

Interaction with test-time compute scaling. The paper studies self-distillation's effect on the model's single-trajectory reasoning quality (acc@1, pass@16). An open question is whether test-time compute strategies—best-of-N sampling, majority voting, or verifier-guided search—can compensate for the epistemic suppression induced by self-distillation. The hypothesis: if SDPO-trained models produce confident but brittle reasoning traces, they might achieve high accuracy when the correct answer appears in any of N samples (pass@N remains high) but suffer when a verifier must select among samples (acc@N degrades) because the traces lack the uncertainty signals that help verifiers distinguish correct from plausible-but-wrong reasoning. A follow-up would compare SDPO and GRPO checkpoints at matched training steps on AIME24/AMC23 under increasing test-time compute budgets (N ∈ {1, 4, 16, 64}), measuring both pass@N and acc@N. If the gap between SDPO and GRPO narrows with increasing N for pass@N but persists for acc@N, that would suggest self-distillation's degradation operates through reduced identifiability of correct traces (the verifier cannot tell which confident trace is right) rather than reduced generation of correct traces.

Cross-task generalization of the epistemic suppression mechanism. The paper's experiments are confined to mathematical reasoning. The mechanism—richer teacher conditioning suppresses uncertainty expression, harming performance when evaluation is OOD and diverse—should generalize to any domain where uncertainty-driven exploration is functionally necessary for robust reasoning. Strong candidate domains include multi-step logical reasoning (e.g., FOLIO, PrOntoQA), complex code generation with novel algorithmic requirements (e.g., LiveCodeBench with strictly held-out problem types), and scientific reasoning with compositional generalization (e.g., ScienceQA with novel combinations of known concepts). A follow-up would replicate the Section 4 off-policy SFT experiment (high vs. low epistemic density training data, matched correctness, evaluation on OOD benchmarks) in one of these domains, using a model known to produce epistemic verbalization in that domain. A positive replication would establish epistemic suppression as a general mechanism; a null result would identify boundary conditions (perhaps code generation benefits from confident traces even for OOD problems because the execution environment provides error signals that substitute for epistemic verbalization).

Practical Applications and Downstream Use Cases

Post-training pipeline evaluation for math reasoning models. Organizations fine-tuning LLMs for mathematical reasoning (educational technology, automated tutoring, competition math solvers, quantitative research assistants) should incorporate an explicit OOD evaluation checkpoint into their post-training pipelines when using self-distillation or related methods. The paper's core finding—that training accuracy and OOD accuracy can diverge under self-distillation (Figure 4c shows GRPO and SDPO with comparable training curves but diverging AIME24 trajectories)—means that monitoring only in-domain metrics creates a false sense of safety. The concrete prescription: when training on a dataset like DAPO-Math-17k or any broad-coverage math corpus, evaluate every N checkpoints on a held-out benchmark (AIME24, AMC23, or a custom OOD set) and track epistemic token counts on that benchmark. A simultaneous decline in OOD accuracy and epistemic token count relative to the base model is an early warning signal of the degradation pattern identified in this paper, even if training accuracy is stable or improving. The paper's fixed-teacher finding (EMA rate 0.0 in Section 5.4) provides an immediately actionable mitigation that reduces the severity of epistemic suppression without requiring objective-level changes.

Model selection for deployment under unknown query distributions. When deploying a reasoning model where the incoming query distribution is not fully characterized—a common scenario for API-based services, customer-facing math assistants, or internal tools where users ask diverse, unpredictable questions—the paper provides evidence that self-distillation-trained models carry hidden brittleness risk even when they appear performant on standard benchmarks. A practitioner choosing between a GRPO-trained checkpoint and an SDPO-trained checkpoint with similar benchmark scores should prefer the GRPO variant if the deployment distribution is expected to be broad and include novel problem types, because SDPO's confident reasoning style degrades specifically on problems that differ from the training distribution. The quantitative basis: on Qwen3-8B thinking-ON (Figure 4c), SDPO with full solution conditioning shows progressive AIME24 degradation over training steps while maintaining competitive training performance, meaning a checkpoint selected for high training score would be the one with worst OOD generalization. The decision rule is not "never use self-distillation" but "validate self-distillation-trained models on OOD benchmarks that exceed the training distribution's diversity before deployment, and prefer alternative methods if OOD performance declines during training."

Curriculum design for self-improvement and iterative training pipelines. Systems that use model-generated data for iterative self-improvement (STaR, ReST$^{EM}$, rejection sampling fine-tuning) face the specific feedback-loop risk identified in Section 5.4: as the model becomes more confident, its self-generated training data becomes less epistemically rich, and training on this data further suppresses uncertainty expression. The paper's fixed-vs-moving teacher ablation demonstrates that even slow EMA updates amplify this effect. The practical implication is that self-improvement pipelines should periodically regenerate training data from a fixed reference checkpoint rather than using the continuously-updated model, to prevent the feedback loop where increasingly confident models produce increasingly confidence-biased training traces. The crucial detail: the fixed reference checkpoint should be the base model or an early-training checkpoint before significant epistemic suppression has occurred—not a midpoint checkpoint that may have already partially suppressed uncertainty expression. The paper's specific finding that even EMA rate 0.05 causes measurable amplification (Figure 6) suggests the reference model should be truly frozen, not slowly updated, for this purpose.