ArXiv: 2509.24726

🎯 Pitch

A training framework starting from just 100 seed questions enables an 8B model to outperform prior data synthesis methods by 20 percentage points on math benchmarks. By having a Teacher agent probe a Solver's specific failures and a Generator distill the strategy into a difficulty-calibrated curriculum, the synthetic data from a 32B model surprisingly surpasses the training utility of data from GPT-5 and Claude-4.1-Opus.

1. Executive Summary

This paper introduces Socratic-Zero, a fully autonomous framework that bootstraps mathematical reasoning from only 100 seed problems through the co-evolution of three agents: a Solver, a Teacher, and a Generator. The Solver improves by learning from preference feedback on its own correct and incorrect solution trajectories via Direct Preference Optimization (DPO), while the Teacher adaptively crafts increasingly challenging problems based on the Solver's specific failures, and the Generator distills the Teacher's question-design strategy through value-weighted supervised fine-tuning (WSFT) (weighting training examples by a Gaussian utility function centered at a 50% Solver success rate) to enable scalable curriculum generation without perpetual reliance on the expensive Teacher model. Running on Qwen3-8B, the Socratic-Solver achieves a "+20.2 points" average improvement over prior data synthesis baselines across seven mathematical reasoning benchmarks, while synthetic data from the Socratic-Generator-32B enables a student model to reach 37.72% downstream utility—surpassing data from several state-of-the-art commercial models including GPT-5, Gemini-2.5-Pro, and Claude-4.1-Opus—establishing that strategic, difficulty-calibrated specialization through co-evolution can match or exceed the data-generation quality of models over 20× larger, but only when the initial supervised fine-tuning phase provides foundational reasoning patterns that subsequent preference optimization can build upon.

2. Context and Motivation

The Core Problem: The Data Bottleneck in Mathematical Reasoning

The paper addresses a fundamental structural problem in how we build LLMs capable of advanced mathematical reasoning. The current dominant paradigm — exemplified by models like DeepSeek-R1, Qwen3, and others that achieve strong performance on competition-level math — depends on access to massive, high-quality, human-annotated datasets of problem-solution pairs. As the authors state in Section 1:

"Current state-of-the-art models depend on millions of human-annotated problem-solution pairs and hand-designed curricula (Yu et al., 2024), creating a fundamental bottleneck that limits both accessibility and the potential for models to evolve beyond human-curated knowledge boundaries."

This bottleneck is not merely about cost. It creates a systemic ceiling: the model can only learn to solve problems that humans have already solved and curated. If you want the model to tackle novel problem types or to push beyond the frontier of human-curated difficulty, the standard pipeline provides no mechanism for doing so. You are bounded by what annotators can produce.

The specific gap is therefore: How can we bootstrap mathematical reasoning capability from a minimal seed (100 problems) with no ongoing human annotation, such that the system autonomously generates its own training curriculum and continuously improves? This is a generalization and autonomy problem — moving from a supervised, static data paradigm to a self-supervised, dynamic one.

Why This Problem Matters

The significance spans practical, theoretical, and economic dimensions:

Practical: Democratizing reasoning capability. If advanced mathematical reasoning can be bootstrapped from minimal human effort, then the barrier to building capable models drops dramatically. Organizations without access to massive annotation budgets or pre-existing large-scale reasoning datasets can still develop strong reasoning systems. This is particularly relevant for specialized domains (scientific research, engineering, finance) where curated reasoning datasets may not exist at all.

Theoretical: Understanding self-improvement dynamics. The paper investigates whether a system can be designed such that a model's own failures become productive training signals — whether an LLM can, in effect, teach itself to reason better by having its mistakes systematically identified and addressed. This touches on fundamental questions in machine learning about the conditions under which self-play and self-generated curricula lead to genuine capability improvement (as opposed to dataset memorization or distributional collapse).

Economic: Reducing dependence on proprietary teacher models. The framework's Generator component is designed to eventually replace the expensive Teacher model. If a 32B Generator can be trained to produce curriculum as effective as that produced by a 235B Teacher (which the results in Table 5 suggest — 37.72% vs. 37.13% downstream utility), then the ongoing inference cost of curriculum generation drops by roughly 7×, making continuous self-improvement economically viable at scale.

Structural: The static curriculum problem. The authors identify a deeper issue with existing approaches that goes beyond data quantity:

"Current methodologies remain entrenched in a static paradigm: datasets are frozen upon collection, curricula are handcrafted in advance, and models are trained on fixed problem distributions. This approach suffers from critical weaknesses: it cannot adapt to evolving model capabilities during training, fails to exploit rich feedback signals for targeting specific weaknesses, and requires extensive human expertise for curriculum design."

This is the curriculum misalignment problem: as the model improves, a fixed dataset becomes increasingly suboptimal. Problems the model has already mastered provide no learning signal (wasted computation), while problems far beyond the model's current capability produce noise rather than useful gradients. An ideal curriculum should track the model's evolving competence to maintain the learning signal at its maximum — what the authors formalize through their Gaussian utility function with target success rate μ=0.5\mu = 0.5 (Equation 6), which incentivizes problems at exactly the frontier of the Solver's capabilities.

Prior Approaches and Their Limitations

The paper situates itself against three categories of prior work, identifying specific shortcomings in each.

Data Synthesis Approaches

Early methods like MetaMath (Yu et al., 2024) and WizardMath (Luo et al., 2023) used prompt engineering to guide LLMs in generating question-answer pairs, but these produced static datasets — generated once, frozen, then used for training with no adaptation to model progress. This means the curriculum cannot respond to which specific concepts the model has mastered or where it continues to struggle.

More recent work — LLM2LLM (Lee et al., 2024) and WarriorMath (Chen et al., 2025b) — introduced deficiency-aware mechanisms where a teacher model identifies knowledge gaps and generates targeted data. This is closer to the Socratic-Zero vision, but the paper identifies a critical weakness:

"While these advances achieve data autonomy, they lack effective quality control mechanisms, resulting in repeated use of low-value samples that severely impact effectiveness."

The problem is that not all failures are equally informative. A failure caused by the problem being genuinely too hard for the current model provides no useful gradient — the model simply cannot reach the correct answer. A failure caused by a specific, addressable reasoning error on a problem within reach is highly valuable. Without a mechanism to distinguish between these cases (which Socratic-Zero provides via its utility-weighted approach), the curriculum fills with low-value problems that waste compute and can degrade training.

Most recently, Absolute Zero (Zhao et al., 2025) and R-Zero (Huang et al., 2025b) explored fully autonomous self-play paradigms where models generate their own tasks and learn from them. These represent the closest prior work to Socratic-Zero's ambition, but the authors argue they suffer from the same quality control failure mode — generating problems without a principled mechanism for assessing which problems will provide optimal learning signal.

Data Distillation Approaches

Knowledge distillation transfers capabilities from powerful teacher models to lighter student models. Orca (Mukherjee et al., 2023) used imitation learning on teacher reasoning chains. GKD (Agarwal et al., 2024) allowed students to learn from their own sequences using teacher feedback for policy correction. The paper identifies two limitations in this line of work:

First, students passively accept teacher feedback without evaluating its reliability:

"However, students passively accept teacher feedback without evaluating reliability, degrading learning quality when guidance is suboptimal."

If the Teacher makes an error in its evaluation or generates a flawed problem, the student has no mechanism to detect or reject this — errors propagate into the training signal.

Second, and more fundamentally, these methods still operate on static datasets. They do not dynamically adjust content based on students' evolving capabilities. The curriculum is fixed; only the feedback mechanism adapts. This means they inherit the curriculum misalignment problem described above.

Preference Learning Approaches

Preference learning provides the optimization backbone for Socratic-Zero's Solver training, and the paper builds directly on DPO (Rafailov et al., 2023), which allows direct optimization of a policy from preference pairs without training a separate reward model. Methods like Self-Refine (Madaan et al., 2023) and Self-Play Fine-Tuning (Chen et al., 2024b) introduced preliminary closed-loop capabilities where models generate their own correction signals.

However, the paper argues these approaches lack a unified, co-evolving framework for feedback generation and validation:

"However, these methods lack unified, co-evolving frameworks for feedback generation and validation."

In isolation, preference learning optimizes the model's policy given a set of preference pairs, but it does not address where those pairs come from or how their quality is maintained. Self-play approaches can generate feedback autonomously, but without a co-evolutionary curriculum mechanism, the feedback may become stale or misaligned as the model improves. What is missing is a system where the source of training data and the training process itself co-evolve, each adapting to changes in the other.

How Socratic-Zero Positions Itself

Socratic-Zero's positioning can be understood along three dimensions that collectively distinguish it from prior work.

1. From static to co-evolutionary curriculum. The paper's central conceptual contribution is replacing the static data paradigm with a co-evolutionary loop (Section 3.1, Figure 3). Rather than a fixed dataset → training pipeline, the system operates as:

  • The Solver attempts problems and generates failures.
  • The Teacher converts failures into targeted new problems (adapting difficulty upward based on specific error patterns).
  • The Generator learns to replicate this conversion process, eventually replacing the Teacher for scalable curriculum generation.
  • The Solver improves on the expanded curriculum, generating new (and different) failures.
  • The cycle repeats.

The key difference from prior iterative approaches is the Generator agent. In LLM2LLM or WarriorMath, the teacher model must be invoked at every iteration to produce new problems, creating a computational bottleneck (the Teacher, Qwen3-235B, runs on 16 AMD MI308X GPUs per the paper's Appendix D). In Socratic-Zero, the Generator learns to internalize the Teacher's strategy, enabling curriculum expansion without perpetual reliance on the expensive Teacher. This is what the authors mean by "strategic distillation" — the Generator doesn't just copy the Teacher's outputs; it learns the policy that produces those outputs, weighted by their utility for the current Solver.

2. Quality-controlled, utility-weighted curriculum generation. The paper's second major positioning move is introducing a principled quality control mechanism through the Gaussian utility function (Equation 6):

U(q′∣πθS)=exp⁡(−(sq′−μ)22σ2)U(q'|\pi_{\theta_S}) = \exp\left(-\frac{(s_{q'} - \mu)^2}{2\sigma^2}\right)

where sq′s_{q'} is the Solver's success rate on problem q′q', μ=0.5\mu = 0.5 targets problems at the frontier of capability, and σ=0.2\sigma = 0.2 controls tolerance. This function assigns high utility to problems the Solver solves approximately 50% of the time, low utility to problems it consistently solves (too easy, no learning signal) or consistently fails (too hard, noise rather than signal).

This directly addresses the "lack of effective quality control mechanisms" critique leveled at prior work. Rather than blindly generating problems from all failures, the Teacher (and eventually the Generator) focuses on failures that are maximally informative — those arising from problems in the Solver's zone of proximal development.

The paper also implements a zone-adaptive generation strategy (Appendix C.2) that formalizes this further. Problems are dynamically categorized into three zones based on Solver success rate:

  • Mastered Zone (sq=1s_q = 1): Problems the Solver consistently solves. The Teacher generates harder variants.
  • Learning Zone (0<sq<10 < s_q < 1): Problems the Solver solves intermittently. The Teacher generates variants targeting specific errors.
  • Too Difficult Zone (sq=0s_q = 0): Problems the Solver consistently fails. These are excluded from generation to prevent counterproductive difficulty escalation.

This triage mechanism ensures the curriculum remains within the Solver's reachable frontier. Prior iterative approaches that generate from all failures risk overloading the curriculum with impossible problems, which the paper shows (in the ablation of reward functions, Table 6b) degrades performance.

3. Socratic philosophy as an architectural metaphor. The paper explicitly frames its design around the Socratic method (Section 1, Figure 1(a)), where learning occurs through guided questioning rather than direct instruction. This is not merely decorative — it maps directly to the system's operation:

  • Socrates (Teacher): Does not give answers but produces questions designed to reveal ignorance. In the computational system, the Teacher does not directly train the Solver; it generates problems that force the Solver to confront its reasoning errors.
  • Aristotle (Solver): Learns by being led through a path of inquiry, not by receiving answers. The Solver improves through its own attempts and the preference feedback that contrasts correct and incorrect trajectories — it learns how to reason, not what the answer is.
  • Plato (Generator): Learns to teach by observing the master's method. The Generator studies the Teacher's problem-generation behavior and learns to produce equally effective questions independently.

This metaphor provides conceptual coherence to the architecture and distinguishes it from approaches that frame the problem purely in optimization terms. It emphasizes that the learning signal comes from well-designed questions, not from correct answers directly — a subtle but important shift in perspective.

Summary of Positioning

Socratic-Zero positions itself at the intersection of three trends — data synthesis, iterative self-training, and preference learning — but claims to solve the quality control and scalability problems that limit each individually. The key differentiators are: (1) co-evolutionary dynamics where curriculum and model adapt to each other, not a static dataset, (2) utility-weighted generation that actively filters for maximally informative problems rather than generating from all failures, and (3) strategic distillation through the Generator that enables the system to eventually operate without the expensive Teacher, making continuous self-improvement computationally viable.

3. Technical Approach

3.1 Reader Orientation

Socratic-Zero is a training system that takes a base language model with no special mathematical reasoning ability except what it absorbed during pretraining, plus 100 example math problems, and autonomously produces both a model that is substantially better at solving math problems and a second model that can generate high-quality new math problems at scale — all without any human providing additional annotations, labels, or problem designs. The system solves the autonomous curriculum generation and self-improvement problem: given only a minimal seed, how do you create a training loop where a model's own failures become the raw material for generating increasingly targeted and appropriately difficult training data, and where the data-generation process itself improves over time? The shape of the solution is a closed-loop co-evolutionary system with three interacting agents — one that tries to solve problems (the Solver), one that evaluates attempts and crafts harder problems from failures (the Teacher, a fixed powerful LLM), and one that learns to replicate the Teacher's problem-crafting strategy so the system can scale without perpetual reliance on an expensive oracle model (the Generator).

3.2 Big-Picture Architecture (Diagram in Words)

The system has three agents and a curriculum that evolves through time:

  1. Solver (\(\pi_{\theta_S}\), parameterized by \(\theta_S\)): A language model being trained to solve mathematical reasoning problems. It takes a problem text \(q\) as input and outputs a full solution trajectory \(y\), which includes reasoning steps and a final answer. It learns through online preference optimization — at each training iteration, it generates multiple attempts per problem, the Teacher verifies which are correct, and the Solver is updated via DPO to prefer correct trajectories over incorrect ones.

  2. Teacher (\(T\), a fixed model — Qwen3-235B-A22B-Instruct-2507): The oracle component, frozen throughout training, providing two functions. The verification function \(V(q, y) \rightarrow \{0,1\}\) judges whether a solution \(y\) is correct for problem \(q\). The problem refinement function \(G(q, y_{\text{fail}}) \rightarrow (q', y'_{\text{ref}})\) takes a problem \(q\) that the Solver failed on (specifically, a failed solution \(y_{\text{fail}}\)) and generates a new problem \(q') (with its reference solution \(y'_{\text{ref}}\)) designed to target the specific error that caused the failure.

  3. Generator (\(\pi_{\theta_G}\), parameterized by \(\theta_G\), initialized from Qwen3-32B): A model trained to eventually replace the Teacher for curriculum expansion. It takes a problem and a failed solution \((q, y_{\text{fail}})\) and outputs a new problem \(q'\). It learns through value-weighted supervised fine-tuning (WSFT) — it is trained to mimic the Teacher's generation outputs, but training examples are weighted by a Gaussian utility function that assigns higher weight to problems at the frontier of the Solver's capability (problems the Solver solves roughly 50% of the time).

  4. Curriculum (\(D_t\)): The evolving set of problems and their reference solutions at iteration \(t\). It starts as \(D_0\), a seed set of 100 problems from the MATH dataset. At each iteration, the curriculum expands by adding \(D_{\text{new}}\) — the set of new problems the Teacher generates from the Solver's current failures.

How information flows through one iteration \(t\):

  1. Solver generates: The current Solver policy \(\pi_{\theta_S}^{(t)}\) generates \(k = 8\) solution attempts for each problem in \(D_t\).
  2. Teacher verifies: The Teacher's verification function \(V\) labels each attempt as correct (1) or incorrect (0), producing winning sets \(Y_w(q)\) and losing sets \(Y_l(q)\) for each problem, and a collection of failures \(\mathcal{F}_t\).
  3. Solver updates: The Solver's parameters are updated via the DPO loss (Equation 5), which uses the winning and losing pairs as preference data.
  4. Teacher generates: For each failure \((q, y_{\text{fail}})\) in \(\mathcal{F}_t\), the Teacher's refinement function \(G\) produces a new problem-solution pair \((q', y'_{\text{ref}})\).
  5. Generator training data is constructed: The newly generated problems \(q'\) are evaluated by having the updated Solver \(\pi_{\theta_S}^{(t+1)}\) attempt them, producing a success rate \(s_{q'}\). The utility \(U(q'|\pi_{\theta_S}^{(t+1)})\) is computed via Equation 6. Training triples \((q, y_{\text{fail}}, q')\) are collected with their utility weights.
  6. Generator updates: The Generator's parameters are updated via the WSFT loss (Equation 7), which encourages producing high-utility problems.
  7. Curriculum expands: \(D_{t+1} = D_t \cup D_{\text{new}}\) (plus historical replay — 25% of problems from previous iterations are kept in the batch).

3.3 Roadmap for the Deep Dive

I will explain the technical approach in the following order:

  • First, the formal co-evolutionary loop (the algorithm that ties everything together), because understanding the overall iteration structure is a prerequisite for understanding each component's role within it.
  • Second, the Solver training mechanism — how preference pairs are constructed from the Teacher's verification and how the DPO loss is applied — because the Solver's failure generation is the engine that drives the rest of the system.
  • Third, the Teacher's role and functions — the verification and problem refinement operations that provide both the training signal for the Solver and the raw material for curriculum expansion.
  • Fourth, the Generator training mechanism — the utility function that defines "good" problems and the WSFT objective that distills the Teacher's strategy — because this is the novel mechanism that distinguishes Socratic-Zero from prior iterative self-training work.
  • Fifth, the curriculum dynamics and quality control — how problems are categorized into difficulty zones, how the adaptive generation strategy works, and how problem quality is maintained across iterations, because these operational details are essential for understanding why the system doesn't collapse or diverge.

3.4 Detailed, Sentence-Based Technical Breakdown

This paper is primarily a systems and training methodology paper whose core idea is that a curriculum of mathematical problems and a model's reasoning capability can co-evolve — each adapting to changes in the other — such that the system bootstraps from minimal seed data to strong performance without external supervision. The architecture instantiates this idea through three agents with well-defined roles, a utility function that operationalizes "desirable difficulty," and a distillation mechanism that makes the system computationally sustainable.

The Co-Evolutionary Loop (Algorithm 1)

The full training procedure is formalized in Algorithm 1 (Appendix H) and iterates for \(T\) total rounds. The loop has three phases per iteration.

Phase 1: Online Solver Evolution. This phase generates training data and updates the Solver. For each problem \(q\) in the current curriculum \(D_t\), the Solver produces \(k = 8\) independent solution attempts \(\{y^{(i)}_S\}_{i=1}^k\). The Teacher's verification function \(V\) labels each attempt, producing a winning set \(Y_w(q)\) (correct solutions) and a losing set \(Y_l(q)\) (incorrect solutions). Preference pairs are formed by pairing winning and losing trajectories, and the Solver is updated via DPO using these pairs. The set of all failures \(\mathcal{F}_t\) — problem-failed-solution pairs — is collected for the next phase.

Phase 2: Offline Generator Evolution and Curriculum Expansion. The Teacher's refinement function \(G\) is applied to each failure in \(\mathcal{F}_t\) to produce new problem-solution pairs \(D_{\text{new}}\). For each new problem \(q'\), its utility \(U(q'|\pi_{\theta_S}^{(t+1)})\) is estimated by having the updated Solver make multiple attempts on it and computing the success rate. The utility is used as a weight in the Generator's training loss, which updates \(\theta_G\) to encourage producing problems that receive high utility scores.

Phase 3: Curriculum Update. The curriculum for the next iteration is formed by appending the newly generated pairs: \(D_{t+1} = D_t \cup D_{\text{new}}\). The training batches for the next iteration combine 100% of the new problems with 25% of historical curriculum problems for replay, ensuring the Solver does not forget earlier patterns while adapting to new challenges.

Why this three-phase structure: Separating Solver updating (Phase 1) from Generator updating (Phase 2) is necessary because the Generator's training depends on evaluating new problems with the updated Solver. If both were updated simultaneously, the utility estimates would be stale — computed with an obsolete Solver policy. The sequential structure ensures that curriculum generation is always evaluated against the most recent Solver capability.

Solver Training via Online Preference Optimization (Phase 1, Section 3.2)

The Solver's improvement mechanism operates by converting the Teacher's binary correctness judgments into preference pairs and applying the DPO loss. This is "online" because the data is generated by the current policy at each iteration, not drawn from a fixed dataset.

Step 1: Generation and verification. For each problem \(q \in D_t\), the Solver \(\pi_{\theta_S}^{(t)}\) generates \(k = 8\) independent solution attempts. Each attempt \(y^{(i)}_S\) is a full text sequence containing reasoning steps and a final answer. The Teacher's verification function \(V(q, y^{(i)}_S)\) returns 1 if the final answer matches the ground truth (using the dual-verification mechanism of MathRule extraction plus LLM judge, described in Appendix G) and 0 otherwise.

Step 2: Constructing preference pairs. The attempts are partitioned into two sets:

Yw(q)={yS(i)∣V(q,yS(i))=1}∪{yref∣if ∀i,V(q,yS(i))=0}Y_w(q) = \{y^{(i)}_S \mid V(q, y^{(i)}_S) = 1\} \cup \{y_{\text{ref}} \mid \text{if } \forall i, V(q, y^{(i)}_S) = 0\}

Yl(q)={yS(i)∣V(q,yS(i))=0}Y_l(q) = \{y^{(i)}_S \mid V(q, y^{(i)}_S) = 0\}

where \(Y_w(q)\) is the set of winning (correct) solutions and \(Y_l(q)\) is the set of losing (incorrect) solutions. The critical design choice here is the fallback: if the Solver fails to produce any correct solution for a problem, the reference solution \(y_{\text{ref}}\) from the curriculum is added to \(Y_w(q)\) as the sole winning example. This guarantees that a valid preference pair \((y_w, y_l)\) with \(y_w \in Y_w(q), y_l \in Y_l(q)\) can always be constructed, even when the Solver performs poorly. Without this fallback, problems where the Solver generates all-incorrect responses would produce no training signal for that problem in that iteration, wasting the computational cost of generation and verification.

Step 3: DPO update. The Solver's parameters are updated using the Direct Preference Optimization loss:

LDPO(θS;θref)=−Eq∼Dt,yw∼Yw(q),yl∼Yl(q)[log⁡σ(βlog⁡πθS(yw∣q)πθref(yw∣q)−βlog⁡πθS(yl∣q)πθref(yl∣q))]L_{\text{DPO}}(\theta_S; \theta_{\text{ref}}) = -\mathbb{E}_{q \sim D_t, y_w \sim Y_w(q), y_l \sim Y_l(q)}\left[\log \sigma\left(\beta \log \frac{\pi_{\theta_S}(y_w|q)}{\pi_{\theta_{\text{ref}}}(y_w|q)} - \beta \log \frac{\pi_{\theta_S}(y_l|q)}{\pi_{\theta_{\text{ref}}}(y_l|q)}\right)\right]

where \(\pi_{\theta_{\text{ref}}}\) is a frozen reference policy (the Solver's policy at the start of the current iteration, \(\pi_{\theta_S}^{(t)}\)), \(\beta\) is a temperature hyperparameter controlling how strongly the optimization is regularized toward the reference policy (values swept in \([0.05, 0.2]\) per Table 7), and \(\sigma\) is the sigmoid function.

What this loss computes: For each preference pair, the term inside the sigmoid is the difference between the log-ratio of the current policy to the reference policy for the winning response and the same log-ratio for the losing response. If the current policy assigns higher relative probability to the winning response compared to the reference policy (a positive difference), the sigmoid approaches 1 and the negative log approaches 0 (low loss). If the current policy favors the losing response (a negative difference), the sigmoid approaches 0 and the negative log is large (high loss). The expectation averages this over all constructed preference pairs in the curriculum. The output is a scalar loss that drives \(\theta_S\) to increase the relative likelihood of correct solutions over incorrect ones.

Why DPO over alternatives: The standard RLHF pipeline requires training a separate reward model on preference data, then using reinforcement learning (typically PPO) to optimize the policy against that reward model. This involves three models (policy, reference, reward) and is notoriously unstable. DPO eliminates the reward model entirely by reparameterizing the preference probability in terms of the policy itself — the implicit reward is \(\beta \log \frac{\pi_{\theta_S}(y|q)}{\pi_{\theta_{\text{ref}}}(y|q)}\). This simplifies the training pipeline to a single loss function, reduces memory requirements, and avoids the instability of online RL. For a self-evolving system where training must run reliably across many iterations without human intervention, this simplicity and stability are essential.

Training configuration details (Table 7): The Solver DPO training uses AdaFactor optimizer with learning rates swept in \([1\times 10^{-6}, 5\times 10^{-6}]\), per-device batch size 2, gradient accumulation steps 4–16, maximum sequence length 2048 tokens, maximum training steps 10–200 (depending on curriculum size at each iteration), DPO regularization \(\beta \in [0.05, 0.2]\), warmup steps 2–20, weight decay 0.01, and maximum gradient norm 1.0. The Solver first undergoes a LoRA-based SFT phase (LoRA rank 64, alpha 128, dropout 0.1, learning rate \(5\times 10^{-5}\), 1 epoch) on 1,500 Level-5 MATH problems before entering the co-evolutionary loop, to establish basic reasoning patterns.

The Teacher's Role: Verification and Problem Refinement (Section 3.1, Appendices A, C)

The Teacher is a fixed, high-capacity model (Qwen3-235B-A22B-Instruct-2507) that is never updated during the co-evolutionary process. It provides two deterministic functions — critical because determinism ensures reproducibility and consistency of the training signal across iterations. If the Teacher's outputs varied with sampling, the same Solver failure might produce different new problems at different times, making curriculum evolution unstable.

Function 1: Verification \(V(q, y) \rightarrow \{0, 1\}\). This judges whether a solution is correct by evaluating the final answer against the reference. The implementation (detailed in Appendices A.2 and G) uses a two-stage process. First, a rule-based extractor (MathRule) parses the solution text to identify the final answer using pattern matching on indicators like "\boxed{}" and standardized formatting. If MathRule successfully extracts a clear answer, it is compared against the reference using standardized representations (e.g., converting fractions to decimals when appropriate). If extraction fails or the result is ambiguous, the Teacher model itself acts as an LLM judge: it receives the problem, the reference solution, and the student's answer in a structured prompt (Appendix A.2), and returns a JSON with correctness judgments and brief error analysis. The judge runs at temperature 0.1 to maximize consistency. This dual-verification approach achieves 94.2% agreement with human experts per Appendix K.

Why a fixed Teacher rather than a learned verifier: The paper uses the strongest available model as an oracle rather than training a separate verifier (contrast this with process reward models in the reference example paper). This choice prioritizes verification accuracy over computational efficiency — a 235B model evaluating 8B model outputs has a large capability gap that makes verification reliable. The tradeoff is cost (the Teacher runs on 16 AMD MI308X GPUs), which is why the Generator exists to eventually reduce dependence on the Teacher.

Function 2: Problem Refinement \(G(q, y_{\text{fail}}) \rightarrow (q', y'_{\text{ref}})\). This is where the Socratic method becomes operational. Given a problem \(q\) that the Solver failed on and the specific failed solution \(y_{\text{fail}}\), the Teacher generates a new problem \(q'\) (with its reference solution \(y'_{\text{ref}}\)) designed to target the error exhibited in \(y_{\text{fail}}\).

The Teacher's prompt (Appendix A.3) instructs it to: (1) receive the original problem and the error analysis from the verification step, (2) generate an enhanced problem that targets the specific error points identified, (3) maintain the core mathematical essence of the original problem while modifying its structure to help the Solver avoid similar errors, (4) produce a detailed solution and final answer for the new problem, and (5) output everything in a structured JSON format.

The zone-adaptive generation strategy (Appendix C.2): Which problems get refined depends on how the Solver performed on them. The curriculum is dynamically partitioned into three zones:

  • Mastered Zone \(D_{\text{mastered}} = \{q \mid s_q = 1\}\): Problems the Solver consistently solves correctly (success rate exactly 1.0 over \(k\) attempts). For these, the Teacher receives a successful solution and generates a more complex variant — pushing the boundaries outward. The goal is to increase difficulty from a foundation of competence.
  • Learning Zone \(D_{\text{learning}} = \{q \mid 0 < s_q < 1\}\): Problems the Solver solves intermittently (success rate between 0 and 1, exclusive). For these, the Teacher receives a failed solution and generates a variant targeting the specific error in that failure. The goal is to address identifiable weaknesses.
  • Too Difficult Zone \(D_{\text{difficult}} = \{q \mid s_q = 0\}\): Problems the Solver consistently fails. These are explicitly excluded from the generation process. The reasoning (Section C.2): generating even harder variants from problems the Solver already cannot solve would produce problems far beyond the zone of proximal development, generating noise rather than learning signal.

Why this zone-based triage: This is one of the key quality control mechanisms that the paper argues prior work lacks. Approaches like LLM2LLM that generate from all failures risk compounding difficulty — a failure on an already-too-hard problem generates an even harder variant, which the model also fails, generating yet harder variants, creating a runaway difficulty escalation that fills the curriculum with unsolvable problems. The triage mechanism prevents this by ensuring generation only happens from problems within or at the boundary of the Solver's capability.

Dynamic recategorization (Appendix C.3): As the Solver improves, problems migrate between zones. A problem in the Too Difficult zone at iteration \(t\) may move to the Learning zone at \(t+1\) as capability grows, re-entering the active generation pool exactly when the Solver is ready. Similarly, Learning zone problems migrate to the Mastered zone as skills consolidate. This dynamic recategorization is what makes the curriculum adaptive — it continuously recalibrates to the moving frontier of the Solver's capabilities.

Generator Training via Offline Value-Weighted Distillation (Phase 2, Section 3.3)

This is the most architecturally novel component of Socratic-Zero. The Generator (\(\pi_{\theta_G}\), initialized from Qwen3-32B) learns to produce problems that are optimally challenging for the current Solver, with the goal of eventually replacing the Teacher for curriculum expansion. The training has three conceptual steps: defining what makes a problem "good" (the utility function), constructing training data from the Teacher's behavior, and optimizing the Generator with utility-weighted supervision.

Step 1: The Gaussian utility function. The quality of a problem \(q'\) is defined relative to the Solver's performance on it. The Solver's success rate on \(q'\) is computed as:

sq′=1k∑i=1kV(q′,yS(i))s_{q'} = \frac{1}{k} \sum_{i=1}^k V(q', y^{(i)}_S)

where \(k\) is the number of attempts (8) and \(V\) is the Teacher's verification function. This is the empirical fraction of attempts that produce the correct answer.

The utility of problem \(q'\) given the Solver policy \(\pi_{\theta_S}\) is then defined by an unnormalized Gaussian centered at \(\mu = 0.5\):

U(q′∣πθS)=exp⁡(−(sq′−μ)22σ2)U(q'|\pi_{\theta_S}) = \exp\left(-\frac{(s_{q'} - \mu)^2}{2\sigma^2}\right)

where \(\mu = 0.5\) is the target success rate, and \(\sigma = 0.2\) controls how quickly utility decays as the success rate deviates from the target.

What this function computes: It maps a Solver success rate \(s_{q'}\) (a number between 0 and 1) to a utility score between 0 and 1. The function reaches its maximum of 1.0 when \(s_{q'} = 0.5\) — the Solver solves the problem exactly half the time. It decays symmetrically as the success rate moves toward 0 (too hard) or 1 (too easy), with the decay rate controlled by \(\sigma\). At \(s_{q'} = 0\) or \(s_{q'} = 1\), the utility is \(\exp(-0.5^2 / (2 \cdot 0.2^2)) = \exp(-3.125) \approx 0.044\), meaning problems at the extremes receive roughly 4.4% of the weight of problems at the frontier.

Why a Gaussian centered at 0.5: Problems the Solver always solves (\(s_{q'} = 1\)) provide no learning signal — every attempt is correct, so there are no failures to learn from. Problems the Solver never solves (\(s_{q'} = 0\)) also provide no useful learning signal — the Solver cannot produce a correct trajectory, so preference pairs can only include the reference solution as the winner, and the gap between the reference and the Solver's attempts may be too large for the DPO gradient to provide meaningful direction. Problems at \(s_{q'} = 0.5\) maximize the information content: approximately half the attempts are correct (providing winning trajectories the model actually generated), and half are incorrect (providing losing trajectories tied to specific, addressable errors). This is the zone of proximal development operationalized as a continuous function.

Why this specific form over alternatives: Table 6b provides the empirical justification. The authors compared the Gaussian \(\mathcal{N}(\mu=0.5, \sigma=0.2)\) against linear functions \(\Psi_\rho(a, b) = a\rho + b\) with various parameterizations, and against Gaussians with different means \(\mu \in \{0.3, 0.4, 0.6, 0.7\}\). The chosen Gaussian achieved the highest benchmark average (35.72%) while maintaining high validity (89.9%). Lower means (\(\mu = 0.3, 0.4\)) encouraged too-easy problems that provided insufficient challenge; higher means (\(\mu = 0.6, 0.7\)) encouraged too-hard problems that generated noise. The Gaussian shape (as opposed to linear weighting) provides a principled way to sharply penalize problems far from the target while being relatively permissive near the center — the curvature of the exponential creates a "sweet spot" that linear functions cannot replicate as cleanly.

Step 2: Constructing training data. The training data for the Generator, \(D_G\), is constructed from the Solver failures \(\mathcal{F}_t\) and the Teacher's corresponding refinements:

DG={(q,yfail,q′)∣(q,yfail)∈Ft,(q′,yref′)=G(q,yfail)}D_G = \{(q, y_{\text{fail}}, q') \mid (q, y_{\text{fail}}) \in \mathcal{F}_t, (q', y'_{\text{ref}}) = G(q, y_{\text{fail}})\}

Each data point is a triple: the original problem \(q\), the Solver's failed solution \(y_{\text{fail}}\), and the Teacher's generated problem \(q'\). The utility \(U(q'|\pi_{\theta_S}^{(t+1)})\) is then estimated for each \(q'\) by having the updated Solver generate \(k\) attempts on \(q'\) and computing the success rate. This utility becomes the weight for that training example.

Why estimate utility with the updated Solver: This is a crucial design detail. The Teacher generated \(q'\) based on the old Solver's failures, but by the time the Generator trains, the Solver has already been updated (Phase 1 completed). Using the old Solver's performance to weight Generator training would produce stale utility estimates — a problem that was optimally difficult before the update might be too easy after. By re-evaluating with the updated Solver, the utility weights reflect the current state of the co-evolutionary system, ensuring the Generator learns to produce problems that are challenging for the Solver as it exists now, not as it existed previously.

Step 3: Weighted supervised fine-tuning. The Generator is trained to maximize the utility-weighted log-likelihood of producing the Teacher's refined problems:

LWSFT(θG)=−E(q,yfail,q′)∼DG[U(q′∣πθS)⋅log⁡πθG(q′∣q,yfail)]L_{\text{WSFT}}(\theta_G) = -\mathbb{E}_{(q, y_{\text{fail}}, q') \sim D_G} \left[ U(q'|\pi_{\theta_S}) \cdot \log \pi_{\theta_G}(q' \mid q, y_{\text{fail}}) \right]

where \(\pi_{\theta_G}(q' \mid q, y_{\text{fail}})\) is the Generator's probability of producing problem \(q'\) given the original problem and the failed solution.

What this loss computes: For each training triple, the Generator produces a probability distribution over possible output problems given the input \((q, y_{\text{fail}})\). The loss penalizes the negative log-probability assigned to the Teacher's actual output \(q'\), but scaled by the utility weight \(U(q'|\pi_{\theta_S})\). If \(q'\) is a high-utility problem (Solver success rate near 0.5), the weight is near 1.0 and the Generator is strongly encouraged to reproduce it. If \(q'\) is a low-utility problem (too easy or too hard), the weight is near 0.044 and the Generator receives a much weaker signal to reproduce it. Over many examples, this steers the Generator toward producing problems in the optimal difficulty zone.

Why weighted SFT rather than filtering: An alternative approach would be to filter the training data — only train on examples where \(q'\) has utility above a threshold, discarding the rest. The weighted approach is more data-efficient: low-utility examples still provide some signal (the Generator still sees what the Teacher produced, just with reduced emphasis), which may be valuable for learning the general form of problem refinement even when the specific difficulty level was suboptimal. Filtering would discard these examples entirely, potentially losing information about the Teacher's refinement strategy even when the specific output wasn't ideally calibrated.

Training configuration (Table 7): Generator training uses learning rate \(1\times 10^{-5}\), per-device batch size 1, gradient accumulation steps 8, maximum sequence length 2048 tokens, LoRA with rank 64 and alpha 128, 2 epochs. The Generator is initialized from Qwen3-32B and is trained offline (its training examples come from the Teacher, not from its own generations, avoiding the distribution shift that complicates online Generator training).

The strategic distillation rationale: The Generator is not merely imitating the Teacher — it is learning a policy (what problem to produce given a specific failure context) that is weighted by outcomes (how useful that problem turned out to be for the Solver). This is closer to reinforcement learning than to standard supervised distillation: the utility weight acts as a reward signal that shapes which Teacher behaviors the Generator prioritizes learning. The paper's results (Table 5) showing that the Socratic-Generator-32B produces data achieving 37.72% downstream utility, slightly surpassing its own Teacher (37.13%) despite being over 20× smaller, suggest that the utility weighting is effectively filtering the Teacher's outputs — the Generator learns to produce only the high-utility subset of the Teacher's behavior, discarding the occasional poorly-calibrated generations.

Curriculum Dynamics and Quality Control

The curriculum is not a passive data structure — it is an actively managed system with mechanisms for expansion, pruning, diversity maintenance, and quality assurance.

Curriculum expansion (Equation 2):

Dt+1=Dt∪{G(q,yS)∣(q,yS)∈Ft}⏟DnewD_{t+1} = D_t \cup \underbrace{\{G(q, y_S) \mid (q, y_S) \in \mathcal{F}_t\}}_{D_{\text{new}}}

At each iteration, all newly generated problem-solution pairs are appended to the existing curriculum. The training batches for the next iteration are constructed by combining "100% new problems with 25% historical curriculum for replay" (Section 4.1, curriculum settings). This mixing ratio balances two objectives: the Solver must learn to handle the new, targeted problems (the 100% new component), but it must also retain capability on older problems to prevent catastrophic forgetting (the 25% replay component). The 25% figure is a design choice — if it were too low, the Solver might overfit to the specific error patterns in the newest problems; if too high, the Solver would spend most of its training on problems it has already seen, slowing adaptation.

Problem quality control (Appendix I): When the Teacher evaluates Solver attempts and finds that all \(k = 8\) solutions for a given problem are incorrect (\(s_q = 0\)), this triggers a self-verification protocol. The Teacher re-examines both the problem statement and its originally provided reference solution by: re-solving the problem independently at temperature 0.1, cross-validating the original reference against the new solution, and checking for mathematical consistency, ambiguous statements, or computational errors. Problems that fail this self-verification (the Teacher cannot reproduce its own reference solution, or multiple valid interpretations exist, or computational errors are detected) are immediately flagged and excluded from further curriculum evolution.

Why this matters: In a self-evolving system where problems are generated automatically, errors can propagate and compound. If a flawed problem (with an incorrect reference solution) enters the curriculum, the Solver will be trained to match the incorrect answer — learning the wrong thing. If that flawed problem is then used as the basis for generating new problems, the errors amplify. The self-verification protocol acts as a circuit breaker: problems that the Teacher itself cannot consistently solve are removed before they can contaminate the training signal.

Diversity maintenance (Appendix J): Three mechanisms prevent the curriculum from collapsing to a narrow distribution. First, multi-domain initialization: the 100 seed problems span all seven MATH subject areas (Algebra, Number Theory, Geometry, Combinatorics, Precalculus, Intermediate Algebra, Prealgebra) across difficulty levels 2–4, as detailed in Table 8, providing diverse starting points. Second, high-temperature sampling: temperature 0.8–0.9 is used at three stages — Solver trajectory generation (ensuring diverse failure modes), Teacher error analysis (ensuring varied interpretation of failures), and Teacher problem generation (ensuring diverse enhancement strategies rather than templated rewrites). Third, compounding diversity: diverse seeds produce diverse failures, which (with high-temperature generation) produce diverse new problems, even when originating from similar error patterns.

Problem categorization and adaptive generation (Appendix C): As described in the Teacher section above, the dynamic categorization into Mastered, Learning, and Too Difficult zones, combined with the zone-adaptive generation strategy (variants from Mastered problems increase difficulty; variants from Learning problems target specific errors), ensures that the curriculum remains calibrated to the Solver's current frontier. Problems flow between zones as capability evolves, creating a continuous feed of appropriately challenging material without human intervention.

Seed selection protocol (Appendix F): The initial curriculum \(D_0\) of 100 problems is selected through a systematic process. Problems are drawn from the MATH dataset Levels 2–4, with Level 1 excluded as too easy (trivial curriculum generation) and Level 5 excluded as too difficult (universal failure producing poor learning signals). Candidates are pre-filtered by having the base model attempt each problem 8 times; problems with success rates between 10% and 70% are retained — ensuring neither complete failure nor trivial success as starting points. The 100 problems are distributed across subject areas (Table 8: 15 each from Algebra, Number Theory, Geometry, Combinatorics, Precalculus, and Intermediate Algebra; 10 from Prealgebra), and within each area, problems are selected to maximize methodological diversity by clustering problem embeddings and sampling from different clusters. All selected problems undergo a quality check: clarity (unambiguous statements), answer verification (Teacher validates reference solutions with multiple attempts), educational value (clear learning objectives, no trick questions), and contamination avoidance (exclusion of any problems appearing in evaluation benchmarks).

Why this elaborate seed selection: The 100 seeds are the only human-curated data in the entire system. Everything else is generated autonomously. The quality and diversity of these seeds therefore has an outsized influence on the entire training trajectory — if the seeds are too narrow, the curriculum evolution will remain narrow; if the seeds contain errors, those errors will propagate. The systematic selection protocol ensures that the one point of human input is maximally leveraged.

Summary of Design Choices and Their Justifications

  • Fixed Teacher as oracle rather than learned verifier: Prioritizes verification accuracy over computational efficiency. The 235B-to-8B capability gap makes verification reliable; the Generator exists to reduce long-term Teacher dependence.
  • DPO over RLHF for Solver training: Eliminates the separate reward model, reducing pipeline complexity and avoiding RL instability — critical for a system that must run autonomously across many iterations.
  • Gaussian utility at \(\mu=0.5, \sigma=0.2\) over linear or alternative-mean weighting: Empirically optimal (Table 6b); the exponential curvature sharply penalizes problems far from the frontier while being permissive near the center, creating a well-defined "sweet spot" for curriculum difficulty.
  • Zone-adaptive generation with Too-Difficult-zone exclusion: Prevents the runaway difficulty escalation that plagues prior iterative approaches by ensuring generation only from problems within or at the boundary of capability.
  • Utility estimation with the updated Solver: Ensures Generator training weights reflect current co-evolutionary state, not stale performance estimates.
  • WSFT over filtering for Generator training: Data-efficient; low-utility examples still provide information about the Teacher's refinement strategy even when the specific output wasn't ideally calibrated.
  • Dual-verification (MathRule + LLM judge) for answer correctness: Rule-based extraction provides speed and objectivity; LLM judge handles ambiguous cases. Together they achieve 94.2% human-expert agreement.
  • 100% new + 25% historical replay in training batches: Balances adaptation to new targeted problems with retention of previously learned patterns.
  • Teacher self-verification protocol on \(s_q=0\) problems: Circuit breaker preventing error propagation from flawed auto-generated problems.
  • Initial SFT on 1,500 Level-5 problems: The ablation in Table 6a shows this is essential — without SFT providing foundational reasoning patterns, DPO training alone yields only marginal gains (2.34 points vs. 18.38 points with SFT). The SFT phase establishes basic solution structures that subsequent preference optimization can refine.

4. Key Insights and Innovations

Innovation 1: Co-Evolution as a First-Class Design Principle for Self-Improving Systems

The paper's deepest conceptual contribution is not any individual component — the DPO training, the Gaussian utility function, or the Generator distillation — but rather the elevation of co-evolution from an emergent property to a designed architectural primitive. Prior work in iterative self-training (LLM2LLM, WarriorMath, Absolute Zero, R-Zero) treated curriculum generation as a service provided to a learner: a teacher model produces data, and a student model consumes it. The relationship is unidirectional and asymmetric. Even when the teacher adapts to student failures (as in deficiency-aware mechanisms), the adaptation is reactive — the teacher responds to the student's current state, but the student does not, in turn, reshape how the teacher operates. The teacher's generation strategy is fixed (determined by its prompt and base capabilities), and the student merely provides inputs to that fixed strategy.

Socratic-Zero introduces a bidirectional, mutually adaptive relationship between three agents where each agent's evolution changes the operating conditions for the others, creating a dynamical system rather than a pipeline. The Solver's improving capability changes which problems are in the Learning Zone versus the Mastered Zone, which changes what kinds of failures the Teacher sees, which changes what problems the Teacher generates, which changes the curriculum the Solver trains on — and around the loop goes. The Generator adds a second co-evolutionary axis: as the Teacher's outputs evolve (because the failures it receives evolve), the Generator's distillation target shifts, and as the Generator improves, it produces higher-quality curriculum that further accelerates Solver improvement.

What makes this genuinely novel — rather than an obvious "add more feedback loops" design — is that the paper formalizes the co-evolution through two mechanisms that actively stabilize the dynamics rather than letting them run uncontrolled. The Gaussian utility function centered at μ = 0.5 acts as an attractor: regardless of how the Solver's capability drifts, the Generator is trained to produce problems that pull the difficulty back toward the 50% success-rate frontier. The zone-adaptive generation with explicit exclusion of the Too Difficult zone acts as a safety constraint: it prevents the positive feedback loop (harder problems → more failures → even harder problems) that would otherwise drive the system toward divergence. These are not merely implementation details — they are necessary conditions for co-evolution to be stable rather than pathological, and identifying them as such represents a conceptual advance in understanding what self-improving training systems require.

The significance of this reframing extends beyond the paper's specific implementation. It suggests that the field's default architecture for self-improvement — a static teacher generating data for an improving student — is fundamentally limited by the fact that the teacher's generation strategy does not co-adapt with the student. A static teacher will eventually produce problems that are either too easy (wasted compute) or too hard (noise) relative to the student's evolved capability, degrading the efficiency of the training signal. Co-evolution solves this by coupling the data-generation process to the learner's trajectory, creating a system that remains calibrated across the entire training run. This is a fundamental architectural insight, not an incremental tuning improvement.

Evidence: The oscillatory convergence patterns in Table 9 (Solver performance drops from 52.1% in Stage 1 to 48.7% in Stage 2, then recovers to 50.1% in Stage 3) are not artifacts of noisy training — they are the dynamical signature of co-evolution. The Teacher increases difficulty faster than the Solver adapts (Stage 1 → 2 drop), then the Solver catches up (Stage 2 → 3 recovery). The Generator's stability (Table 10: high-reward problem percentage fluctuating only 1.3% across stages) confirms that the utility-weighted objective successfully anchors the system near the target frontier despite the Solver's drift. The cross-architecture replication (Table 2: GLM4-9B and Qwen3-14B both show the same staged improvement pattern) provides evidence that the co-evolutionary dynamics are a property of the system design, not an artifact of a specific model.

Innovation 2: The Problem Utility Function as a Curriculum Quality Metric

Prior to this work, the dominant approach to filtering or weighting synthetic training data was correctness-based: keep problems the model gets wrong (deficiency-aware), discard problems that are malformed (validity filtering), or weight by some notion of difficulty derived from the problem text itself rather than from the model's relationship to it. The field lacked a principled, continuous metric for answering the question: how useful is this specific problem for this specific model at this specific point in training?

Socratic-Zero's Gaussian utility function, U(q′|π_{θ_S}) = exp(−(s_{q′} − μ)²/(2σ²)), provides exactly such a metric. It reframes curriculum quality from an absolute property of the problem ("is this problem hard?") to a relational property between the problem and the learner ("is this problem at the frontier of this model's current capability?"). This is a fundamentally different ontological commitment: problem difficulty is not an intrinsic attribute but a function of the solver. A problem that is trivially easy for Qwen3-235B may be at the optimal frontier for Qwen3-8B, and a problem that was at the frontier in iteration t may be mastered by iteration t+1. The utility function captures this relational, dynamic quality in a single scalar that can be computed automatically.

The specific choice of a Gaussian centered at μ = 0.5 is justified empirically (Table 6b: μ = 0.3, 0.4, 0.6, 0.7 all underperformed), but the deeper conceptual contribution is the operationalization of the zone of proximal development — a concept from educational psychology (Vygotsky) that describes the learning frontier where tasks are challenging enough to require effort but achievable with support. Translating this from a qualitative educational principle to a computable objective function (with parameters μ, σ that can be tuned empirically) bridges a gap between learning theory and ML system design. The fact that the empirically optimal μ = 0.5 corresponds exactly to maximum entropy in the binary outcome (correct/incorrect) is theoretically elegant — it maximizes the information content of each training example by ensuring neither outcome is predictable a priori.

The utility function also enables a form of automatic curriculum calibration that was previously manual. Prior systems required human curriculum designers to sequence problems from easy to hard, relying on intuition or coarse difficulty labels (e.g., MATH dataset Levels 1-5). The utility function replaces this with a continuous, data-driven signal: as long as the Teacher generates a distribution of problems and the utility function filters/weights them, the curriculum automatically tracks the Solver's frontier. This is a fundamentally more scalable approach to curriculum design — it removes the human from the loop not just for problem generation but for difficulty sequencing.

Evidence: Table 6b provides the direct empirical justification — the Gaussian with μ = 0.5 and σ = 0.2 achieves 35.72% average benchmark performance, with all alternatives (linear functions, different means) underperforming by 0.20–0.40 points. The Generator's downstream effectiveness (Table 5: 37.72% for Socratic-Generator-32B, surpassing the Teacher's 37.13%) demonstrates that the utility weighting successfully filters the Teacher's outputs to retain only the most valuable problems — the Generator learns to produce the high-utility subset of the Teacher's behavior. The validity rate (Table 4: 95.6% for the Socratic-Generator) confirms that the utility function does not simply select for easy-to-generate problems — it maintains high problem quality while optimizing for difficulty.

Innovation 3: Strategic Distillation — Learning to Generate, Not Just to Solve

The field has extensively studied knowledge distillation where a student model learns to replicate a teacher's solutions — Orca (Mukherjee et al., 2023) had students imitate teacher reasoning chains; GKD (Agarwal et al., 2024) had students learn from teacher feedback on their own sequences. These approaches distill the teacher's reasoning capability into the student. Socratic-Zero introduces a complementary but distinct form of distillation: the Generator learns to replicate the teacher's curriculum design capability — not how to solve problems, but how to create problems that are optimally challenging.

This is a genuinely different skill. Solving a math problem requires finding a valid reasoning path from premises to conclusion. Creating a good math problem requires understanding what makes a problem difficult in specific ways, anticipating where solvers will make errors, and designing problem structures that target those error patterns while remaining solvable. The Teacher (Qwen3-235B) possesses both capabilities implicitly through its massive pretraining, but solving and creating are not the same capability, and the Generator's training objective (WSFT weighted by Solver utility) specifically targets the creation skill.

The strategic distillation is made possible by the utility function: it provides a reward signal that distinguishes good problems (those at the Solver's frontier) from any problems (whatever the Teacher happens to generate). Without this signal, the Generator would simply imitate the Teacher's full output distribution — including poorly calibrated problems that are too easy or too hard. With the signal, the Generator learns a refined policy that produces the high-utility subset. The paper's result that the Socratic-Generator-32B surpasses its own Teacher in downstream data quality (37.72% vs. 37.13% in Table 5) — despite the Teacher being over 20× larger — is the empirical validation of this claim. The Generator has not just copied the Teacher; it has improved upon the Teacher's output distribution by learning to focus on the useful subset.

This has significant practical implications for the economics of self-improving systems. The Teacher (Qwen3-235B) requires 16 AMD MI308X GPUs for inference (Appendix D); the Generator (Qwen3-32B) is over 7× smaller and correspondingly cheaper. If the Generator can produce curriculum of equal or better quality, the system can scale to many more iterations without being bottlenecked by Teacher inference cost. This is what makes continuous self-improvement economically viable — without the Generator, the cost of running the Teacher at every iteration would make long training runs prohibitively expensive.

Evidence: Table 5 is the central result. The downstream utility of data from the Socratic-Generator-32B (37.72%) not only surpasses its base model Qwen3-32B (34.97%, a +2.75 point gain) but also beats: the Teacher itself Qwen3-235B (37.13%, +0.59 points), GPT-5 (36.62%, +1.10), Gemini-2.5-Pro (37.20%, +0.52), Grok-4 (37.01%, +0.71), Claude-4.1-Opus (37.63%, +0.09), and DeepSeek-V3.1-671B (36.62%, +1.10). The validity rate (Table 4: 95.6%) provides convergent evidence — the Generator's problems are not just well-calibrated for difficulty but also well-formed and solvable. The comparison to the Qwen3-32B base model (+6.5 points in validity rate) isolates the gain attributable to the strategic distillation process from the gain attributable to the base model's inherent capability.

Innovation 4: The Essential Role of Initial SFT as a Capability Foundation

One of the paper's most practically important findings is a negative result that reframes how self-improvement systems should be initialized. The ablation in Table 6a shows that Socratic-Zero without initial SFT yields only a +2.34 percentage point improvement on AIME-24 (from 9.64% to 11.98%) after three stages of co-evolutionary training. With initial SFT on 1,500 Level-5 MATH problems, the same three stages yield an +18.38 point improvement (to 28.02%) — a 7.9× larger gain from the identical co-evolutionary process.

This result demonstrates that preference optimization (DPO) amplifies existing capability but does not create it from nothing. The Solver needs a foundation of basic reasoning patterns — the ability to structure a solution, to apply standard mathematical techniques, to produce syntactically valid chains of reasoning — before the preference signal from correct/incorrect trajectory pairs can drive meaningful improvement. Without SFT, the Solver's attempts are so far from correct that the DPO gradient provides minimal direction: the winning trajectory (the reference solution) is too distant from the losing trajectories (the Solver's attempts) for the preference comparison to yield informative gradients. The SFT phase compresses this gap, bringing the Solver's output distribution close enough to the space of correct solutions that DPO can effectively steer it.

This finding directly contradicts a naive interpretation of self-play and self-improvement — that a model can bootstrap from near-zero capability solely through iterative feedback. The Socratic-Zero framework provides rich feedback (targeted problems, preference pairs, utility-weighted curriculum), but that feedback is only productive when the model already possesses the basic building blocks of the task. The SFT phase is the bridge from pretrained-but-untuned capability to the regime where co-evolutionary refinement works.

This has significant implications for the design of self-improving systems in new domains. It suggests that the initial capability threshold is a first-order concern — you cannot simply take a base model, wrap it in a co-evolutionary loop, and expect improvement. You need to verify that the model's initial performance is above some minimum level (the paper's seed selection protocol used 10–70% success rate as the viable range, per Appendix F) before the self-improvement dynamics can engage. This is a boundary condition on self-improvement that prior work did not identify clearly, and it connects to the broader finding in the test-time compute scaling literature (see the reference example paper in the prompt) that inference-time strategies only help when the base model has non-trivial capability on the problem class.

Evidence: Table 6a provides the direct comparison: Socratic-Zero without SFT goes from 9.64% → 11.15% → 11.98% across three stages; with SFT, from 9.64% → 13.44% → 14.48% → 28.02%. The Qwen3-8B-Base zero-shot performance of 9.64% represents the untuned starting point. The SFT-only performance is not reported in Table 6a, but Table 1 shows Qwen3-8B-Base + SFT achieves 35.9% average across all benchmarks (vs. 29.9% zero-shot), providing context for how much capability SFT alone contributes before co-evolution begins. The cross-model results (Table 2: GLM4-9B goes from 35.2% zero-shot to 42.8% with SFT to 52.3% after Stage 3; Qwen3-14B from 43.0% to 51.8% to 60.3%) all follow the same pattern — SFT provides a substantial initial boost, and co-evolution builds on that foundation.

Innovation 5: Zone-Adaptive Curriculum as Quality Control Through Exclusion

The paper's systematic approach to curriculum quality control — specifically the explicit exclusion of the Too Difficult zone from problem generation — represents a conceptual shift from how prior iterative self-training systems handled failure. The dominant assumption in prior work (LLM2LLM, WarriorMath, R-Zero) was that all failures are informative and should be used to generate more training data. The more failures, the more data, the better. Socratic-Zero challenges this by distinguishing between productive failures (from the Learning Zone, where the model has partial competence and the errors are addressable) and unproductive failures (from the Too Difficult Zone, where the model has no competence and the errors are symptoms of a fundamental capability gap rather than specific, correctable mistakes).

This is not merely a filtering heuristic — it is a recognition that feedback-driven data generation has a non-monotonic relationship with problem difficulty. Generating harder problems from failures in the Learning Zone is productive because the model has a foothold — it can sometimes solve the original problem, so a variant targeting its specific error can plausibly be solved with improved reasoning. Generating harder problems from failures in the Too Difficult Zone is counterproductive because the model has no foothold — it cannot solve the original problem at all, so a harder variant will certainly be unsolvable, producing no correct trajectories and therefore no useful preference signal. The system actively wastes compute generating and attempting problems that provide zero learning signal.

The self-verification protocol (Appendix I) adds a second layer to this quality control: when the Teacher finds that all Solver attempts on a problem fail, it re-verifies the problem itself. This catches a different failure mode — not that the problem is too hard for the Solver, but that the problem itself is flawed (incorrect reference solution, ambiguous statement, computational error). In a system where problems are generated automatically, this kind of error propagation is a real risk: a single flawed problem can spawn a cascade of flawed variants that contaminate the curriculum. The self-verification protocol is a circuit breaker that prevents this cascade.

The dynamic recategorization (Appendix C.3) — where problems migrate from Too Difficult to Learning to Mastered as the Solver improves — completes the quality control system. It ensures that exclusion is temporary, not permanent. Problems that were genuinely beyond the Solver's capability at iteration t are not discarded forever; they are held in reserve and reintroduced when the Solver's capability grows to meet them. This prevents the system from permanently shrinking its curriculum to only-easy problems while still protecting against premature exposure to impossible ones.

Evidence: The system's stability across three stages of training (Tables 1, 2, 9, 10) provides indirect evidence for the quality control mechanisms working. Without zone-adaptive exclusion, one would expect either: (a) runaway difficulty escalation (curriculum fills with impossible problems, Solver performance collapses) or (b) curriculum stagnation (only easy problems survive, Solver performance plateaus early). Neither occurs. Instead, Solver performance shows the characteristic oscillate-then-recover pattern (Table 9), Generator stability remains high (Table 10), and cross-architecture results (Table 2) show consistent improvement without collapse — all consistent with a quality control system that successfully maintains the curriculum at the frontier of capability. The comparison with LLM2LLM (Table 1, bottom rows for LLM2LLM vs. Qwen3-8B-base with Socratic-Zero) provides a suggestive contrast: LLM2LLM's Stage 1 performance actually drops relative to Static Augmentation on most benchmarks (AMC: −4.2, MATH-500: −9.6, Olympiad: −3.5), which is consistent with uncontrolled curriculum expansion degrading training signal quality — the very failure mode Socratic-Zero's quality control is designed to prevent.

5. Experimental Analysis

Evaluation Methodology

  • Dataset. The Solver is evaluated on seven mathematical reasoning benchmarks: AMC (Cao et al., 2025), Minerva (Nagrani et al., 2025), MATH-500 (Hendrycks et al., 2021), GSM8K (Cobbe et al., 2021), Olympiad-Bench (He et al., 2024), AIME-2024, and AIME-2025. For general reasoning transfer, three additional benchmarks are used: BBEH (Kazemi et al., 2025), MMLU-Pro (Wang et al., 2024), and SuperGPQA (Team et al., 2025). The Generator evaluation uses 1,000 seed problems from SAND-Math (Zhang et al., 2025). The initial seed curriculum of 100 problems (for bootstrapping the co-evolutionary loop, not for final evaluation) is drawn from the MATH training set across difficulty Levels 2–4, with balanced domain coverage (Table 8).

  • Base model(s). The primary Solver model is Qwen3-8B-base (Yang et al., 2025). Cross-architecture generalization is tested on Qwen3-14B-base and GLM4-9B-base (GLM et al., 2024). The Teacher is Qwen3-235B-A22B-Instruct-2507, a frozen oracle model providing verification and problem refinement. The Generator is initialized from Qwen3-32B. For downstream evaluation of generated data quality, DeepSeek-R1-Distill-Llama-8B (DeepSeek-AI et al., 2025a) is fine-tuned as the student model. The 8B parameter scale for the primary Solver is chosen to demonstrate that substantial reasoning gains are achievable from a modestly sized base model when augmented with the co-evolutionary framework, with the 14B and 9B variants testing generalization of the approach.

  • Metrics. For the Solver, the primary metric is Mean@32 accuracy: for each test question, 32 independent solutions are generated (temperature 0.7, top-p 0.9), and the fraction of correct solutions is computed, with correctness determined by a dual-verification mechanism combining rule-based answer extraction (MathRule) and semantic validation via an LLM judge (Qwen3-235B). The per-benchmark accuracy is the average of these fractions across all questions in that benchmark. For the Generator, two metrics are reported: Validity Rate — the percentage of generated problems successfully solved by Qwen3-235B under a 4,096-token, 600-second timeout constraint — and Downstream Utility — the Mean@16 accuracy of a DeepSeek-R1-Distill-Llama-8B student model fine-tuned on the generated question-answer pairs, evaluated across the seven math benchmarks.

  • Baselines. Four baselines are compared against the Socratic-Solver. Zero-shot: the base model (e.g., Qwen3-8B-base) prompted with the Solver reasoning prompt (Appendix A.1) without any fine-tuning. SFT: the base model fine-tuned via supervised learning on a static dataset of 1,500 Level-5 MATH problems, using the same LoRA configuration as Socratic-Zero's initial SFT phase. Static Augmentation (SA): follows the approaches of MetaMath (Yu et al., 2024) and WizardMath (Luo et al., 2023) — the SFT model is further trained on a fixed set of synthetic questions generated offline from the initial seed problems using the Teacher's problem refinement prompts, with no adaptive curriculum evolution. LLM2LLM (Lee et al., 2024): an iterative self-training baseline where the model (starting from the same SFT initialization as Socratic-Zero) generates new questions based on its current failures at each iteration and retrains on the augmented dataset, using the same Teacher model for verification and generation but without the Generator distillation component or the utility-weighted curriculum quality control. For the Generator evaluation, baselines include the base model Qwen3-32B, the Teacher model Qwen3-235B-A22B, and five commercial SOTA models: DeepSeek-V3.1-671B, Gemini-2.5-Pro, GPT-5, Grok-4, and Claude-4.1-Opus. All generators are prompted with the same 1,000 SAND-Math seeds and evaluated under the same protocol.

  • Generation budget / compute accounting. The paper does not use a unified FLOPs-based or generation-count budget for comparing methods. Instead, fairness is approached through matched initialization: all Solver training methods (Socratic-Zero, LLM2LLM, Static Augmentation) begin from the identical SFT checkpoint and use the same base model, the same Teacher model for verification and generation (where applicable), and the same evaluation protocol (Mean@32). The number of training stages (iterations) is the primary axis of comparison: all iterative methods are evaluated after Stage 1, Stage 2, and Stage 3, with each stage representing one complete pass through the co-evolutionary loop (Solver DPO update + curriculum expansion + Generator update, in the case of Socratic-Zero). The Socratic-Zero framework does incur additional computational cost from the Teacher and Generator components that the Static Augmentation baseline does not, but since the claim is about data efficiency and final performance rather than training FLOPs efficiency, this cost is treated as part of the method rather than normalized away. The Teacher runs on 16×AMD MI308X GPUs, and Solver training on 8×NVIDIA H20 GPUs (Appendix D), but no end-to-end FLOPs comparison across methods is provided.

  • Cross-validation / statistical protocol. The paper does not employ k-fold cross-validation for the Solver evaluation. Results are reported as single-number accuracies on the fixed test sets of each benchmark. The Mean@32 metric provides some statistical stability by averaging over 32 independent decoding runs per problem, but no confidence intervals, standard deviations, or significance tests are reported for any of the main results. For the Generator evaluation, each generator produces 3,000 total problems (1,000 seeds × 3 variants each); the downstream utility is measured by fine-tuning a student model on the full set of valid QA pairs and evaluating on the seven benchmarks. The paper does not report variance across different fine-tuning runs or different random seeds for the generation process. The absence of uncertainty quantification is a notable gap — with test sets like AIME-2024 and AIME-2025 containing only 30 questions each, the point estimates in Tables 1, 2, and 5 may have substantial sampling variance that the reader cannot assess.

Main Quantitative Results

Solver Performance Across Training Methods

The headline result appears in Table 1. After three stages of co-evolutionary training, the Socratic-Solver-8B achieves 56.1% average accuracy across the seven mathematical reasoning benchmarks. This represents:

  • A +20.2 percentage point improvement over the Static Augmentation baseline (40.7% → 56.1%)
  • A +15.2 point improvement over LLM2LLM at Stage 3 (40.9% → 56.1%)
  • A +20.2 point improvement over the SFT-only model (35.9% → 56.1%)
  • A +26.2 point improvement over the zero-shot base model (29.9% → 56.1%)

The gains are not uniform across benchmarks. The largest absolute improvements over Static Augmentation appear on the competition-level benchmarks: Olympiad: +19.2 points (35.9% → 55.1%), AMC: +17.9 points (45.8% → 63.7%), MATH-500: +18.5 points (62.7% → 81.2%), AIME-24: +16.1 points (12.3% → 28.4%), and AIME-25: +13.2 points (11.4% → 24.6%). GSM8K, the least challenging benchmark, shows a smaller but still substantial gain of +12.7 points (74.6% → 87.3%), reflecting a ceiling effect as performance approaches saturation.

The stage-wise progression reveals the co-evolutionary dynamics. The Socratic-Solver's Stage 1 performance (38.7%) is actually below the Static Augmentation baseline (40.7%) by 2.0 points, and below LLM2LLM Stage 1 (37.5%) by only 1.2 points. This is not a failure — it reflects the fact that Stage 1 is the initial co-evolutionary iteration where the curriculum has just begun adapting and the Generator has not yet learned to produce well-calibrated problems. The performance at Stage 1 is roughly comparable to non-adaptive baselines. The critical divergence occurs at Stages 2 and 3. By Stage 2, Socratic-Zero (41.7%) surpasses Static Augmentation (40.7%) by 1.0 point and LLM2LLM Stage 2 (38.3%) by 3.4 points. By Stage 3, the gap widens dramatically: Socratic-Zero (56.1%) leads Static Augmentation by 15.4 points and LLM2LLM Stage 3 (40.9%) by 15.2 points.

LLM2LLM shows minimal improvement over stages. The LLM2LLM baseline, which uses the same Teacher for verification and generation but lacks the Generator distillation and utility-weighted quality control, shows remarkably flat scaling. From Stage 1 (37.5%) to Stage 2 (38.3%) to Stage 3 (40.9%), the total improvement is only +3.4 points across all three stages. On several benchmarks, LLM2LLM performance actually declines at Stage 1 relative to Static Augmentation: AMC drops by −4.2 points, MATH-500 by −9.6 points, Olympiad by −3.5 points, AIME-25 by −4.7 points, and AIME-24 by −3.4 points. This regression is consistent with the paper's argument that generating from all failures without utility-based quality control introduces low-value or counterproductive problems into the curriculum — the model is training on problems that are either too hard (providing no useful learning signal) or flawed (teaching incorrect patterns). Socratic-Zero avoids this through the zone-adaptive generation that excludes the Too Difficult zone, the utility-weighted Generator training that prioritizes frontier problems, and the self-verification protocol that filters flawed problems.

The SFT-only baseline contextualizes the contribution of supervised pre-training. SFT on 1,500 Level-5 MATH problems lifts the base model from 29.9% to 35.9% average accuracy (+6.0 points). This is a meaningful but modest gain, concentrated on benchmarks closest to the SFT distribution: MATH-500 (+8.1 points), AMC (+6.6 points), Minerva (+6.5 points). The co-evolutionary process adds a further +20.2 points beyond SFT — more than 3× the SFT gain — indicating that the dynamic curriculum and preference optimization are the primary drivers, not simply exposure to more training data. This is consistent with the ablation in Table 6a, which shows that DPO without initial SFT yields minimal gains (+2.34 points on AIME-24), confirming that SFT provides a necessary foundation but is not sufficient for the full performance.

Cross-Architecture Generalization

Table 2 reports results for two different model families trained with the Socratic-Zero framework, using the same Teacher (Qwen3-235B) and the same training protocol. On GLM4-9B-base, Socratic-Zero Stage 3 achieves 52.3% average accuracy, representing a +17.1 point improvement over the zero-shot baseline (35.2%) and a +9.5 point improvement over SFT-only (42.8%). On Qwen3-14B-base, Stage 3 achieves 60.3% average accuracy, a +17.3 point improvement over zero-shot (43.0%) and a +8.5 point improvement over SFT-only (51.8%).

The cross-architecture replication is important for establishing that the co-evolutionary dynamics are not specific to the Qwen3-8B architecture or training regime. The consistent stage-wise improvement pattern — each stage adds meaningful gains for both architectures — suggests the framework is capturing something fundamental about curriculum-driven self-improvement rather than exploiting model-specific artifacts. The Qwen3-14B's higher absolute performance (60.3% vs. 56.1% for 8B) is expected given the larger parameter count, but the relative gain over SFT is similar (+8.5 points for 14B vs. +20.2 points for 8B), suggesting diminishing returns to co-evolution as base capability increases — the 8B model has more room for improvement.

Transfer to General Reasoning Benchmarks

Table 3 evaluates whether the mathematical reasoning improvements transfer to three general reasoning benchmarks: BBEH, MMLU-Pro, and SuperGPQA. Across three stages of Socratic-Zero training on Qwen3-8B-base, the model gains an average of +6.02 points (27.47% → 33.49%). The improvement is most pronounced on MMLU-Pro (+10.89 points, from 50.00% to 60.89%) and SuperGPQA (+5.32 points, from 24.73% to 30.05%), with BBEH showing a smaller +1.86 point gain (7.68% → 9.54%).

The transfer gains are modest relative to the in-domain gains (+20.2 points on math benchmarks). This is expected: the training curriculum consists entirely of mathematical reasoning problems, so any transfer to general reasoning must come from the Solver developing meta-cognitive skills — structured problem-solving, systematic verification, multi-step logical planning — that partially generalize. The particularly strong gain on MMLU-Pro, which includes substantial STEM content where mathematical thinking is directly applicable, is consistent with this interpretation. The near-flat BBEH result suggests the acquired skills do not transfer to tasks requiring fundamentally different reasoning modalities.

Generator Performance: Problem Quality and Downstream Effectiveness

The Generator evaluation addresses whether a 32B model trained through strategic distillation can produce training data comparable to or better than that from much larger models, including commercial SOTA systems.

Problem quality (Table 4): The Socratic-Generator-32B achieves a 95.6% validity rate on generated problems, as measured by Qwen3-235B's ability to solve them under constrained decoding. This is a +6.5 point improvement over its base model Qwen3-32B (89.1%), and it approaches the validity rates of the Teacher itself (95.1%) and commercial models: GPT-5 (95.8%), Gemini-2.5-Pro (94.2%), Grok-4 (95.7%), DeepSeek-V3.1 (96.5%), and Claude-4.1-Opus (96.9%). The Socratic-Generator's validity rate is lower than Claude-4.1-Opus (−1.3 points) and DeepSeek-V3.1 (−0.9 points), but the gap is small relative to the parameter difference (32B vs. 671B+).

Downstream effectiveness (Table 5): When the valid question-answer pairs from each generator are used to fine-tune DeepSeek-R1-Distill-Llama-8B, the Socratic-Generator-32B's data yields a 37.72% average accuracy across the seven benchmarks. This is the highest value in the table, surpassing:

  • The base Qwen3-32B (34.97%, +2.75 points)
  • The Teacher Qwen3-235B-A22B (37.13%, +0.59 points)
  • GPT-5 (36.62%, +1.10 points)
  • Gemini-2.5-Pro (37.20%, +0.52 points)
  • Grok-4 (37.01%, +0.71 points)
  • Claude-4.1-Opus (37.63%, +0.09 points)
  • DeepSeek-V3.1-671B (36.62%, +1.10 points)

The margins over the commercial models are small — 0.09 points over Claude-4.1-Opus, 0.52 points over Gemini-2.5-Pro — and without confidence intervals, we cannot assess whether these differences are statistically significant. However, the directional result that a 32B specialized generator can match or slightly exceed 235B–671B general-purpose models in the specific task of producing training data for mathematical reasoning is the paper's central claim for the Generator component. The Socratic-Generator's advantage is most visible on Minerva (+2.1–2.9 points over the commercial models, 18.4% vs. 15.5–16.7% range) and AMC-23 (48.1% vs. 45.0–47.5% range), while on Olympiad it slightly underperforms several models (24.6% vs. 25.4% for Gemini-2.5-Pro and 25.9% for GPT-5).

Curriculum Dynamics Analysis

Tables 9 and 10 (Appendix J) provide descriptive statistics on the co-evolutionary dynamics that underlie the performance improvements.

Solver mean reward evolution (Table 9): The Solver's mean reward (average correctness rate across all curriculum problems) follows a characteristic pattern: Stage 1 = 52.1%, Stage 2 = 48.7% (a decline of −3.4 points), Stage 3 = 50.1% (a recovery of +1.4 points). This oscillatory trajectory is the signature of adaptive curriculum: the Teacher increases problem difficulty faster than the Solver can initially adapt (causing the Stage 1 → 2 decline), then the Solver's capability catches up to the expanded curriculum (causing the Stage 2 → 3 recovery). A static curriculum would show monotonically increasing reward as the model overfits; the oscillation indicates the curriculum is actively tracking the Solver's frontier.

Generator reward stability (Table 10): The percentage of high-reward problems (those near the target success rate μ = 0.5) remains remarkably stable across stages: Stage 1 = 50.7%, Stage 2 = 49.4%, Stage 3 = 50.2%. The total fluctuation range is only 1.3 percentage points. This stability indicates the Generator has successfully learned to produce problems calibrated to the optimal difficulty zone, and this calibration is maintained even as the Solver's capability evolves and the curriculum expands. The "Target Range (45–55%)" is noted as "Maintained" for all stages, confirming that the curriculum remains within the zone of proximal development throughout training.

Ablation Studies and Robustness Checks

Initial SFT necessity (Table 6a): Without the initial SFT phase on 1,500 Level-5 MATH problems, the Qwen3-8B-Base model's Socratic-Zero training yields minimal improvement. Starting from 9.64% (zero-shot) on AIME-24, the no-SFT variant reaches only +2.03 points at Stage 1 (11.67%), +1.51 at Stage 2 (11.15%), and +2.34 at Stage 3 (11.98%). In contrast, the SFT-initialized variant reaches +3.80 at Stage 1 (13.44%), +4.84 at Stage 2 (14.48%), and +18.38 at Stage 3 (28.02%). The Stage 3 gain ratio is approximately 7.9× in favor of SFT initialization. This confirms the paper's claim that SFT provides essential foundational reasoning patterns without which DPO-based preference optimization cannot effectively steer the model — the model's initial attempts are too far from correct solutions for the preference gradient to provide meaningful direction. The specific AIME-24 benchmark is chosen for this ablation (rather than the full average) but is representative of the competition-level problems where the gains are largest.

Reward function formulation (Table 6b): Seven alternative formulations are compared against the chosen Gaussian N(μ = 0.5, σ = 0.2) using downstream Generator training effectiveness (benchmark average) and problem validity. The chosen Gaussian achieves 35.72% average with 89.9% validity. Three linear functions Ψρ(a,b) = aρ + b with different parameterizations all underperform: Ψρ(0,1) achieves 35.52% (−0.20, validity 89.4%), Ψρ(1,0) achieves 35.47% (−0.25, validity 89.8%), and Ψρ(−1,1) achieves 35.42% (−0.30, validity 88.9%). Four alternative Gaussian means all underperform: μ = 0.3 achieves 35.32% (−0.40, validity 89.5%), μ = 0.4 achieves 35.37% (−0.35, validity 89.7%), μ = 0.6 achieves 35.50% (−0.22, validity 89.7%), and μ = 0.7 achieves 35.43% (−0.29, validity 89.8%). The degradation is asymmetric: low means (μ = 0.3, 0.4) hurt more than high means (μ = 0.6, 0.7), suggesting that under-challenging the Solver with too-easy problems is worse than slightly over-challenging it. The validity rates are relatively stable across all formulations (88.9–89.9%), indicating that the reward function primarily affects difficulty calibration rather than problem well-formedness. The 0.40-point gap between the best and worst formulations is modest in absolute terms but large relative to the ≈2.75-point gap between the Socratic-Generator and its base model (Table 5).

Cross-architecture replication (Table 2): While not presented as an ablation, the consistent improvement patterns on GLM4-9B and Qwen3-14B serve as a robustness check against model-family-specific effects. The Qwen3-14B's lower relative gain (+8.5 points over SFT vs. Qwen3-8B's +20.2 points over SFT) is expected — the larger base model has higher initial capability (51.8% after SFT vs. 35.9% for 8B), leaving less room for co-evolutionary improvement. The GLM4-9B's gain pattern (+9.5 points over SFT) falls between the two Qwen variants, consistent with its intermediate base capability.

LLM2LLM as a process-of-elimination ablation: The LLM2LLM baseline (Table 1) shares the Teacher model (same verification and generation functions) and the iterative structure with Socratic-Zero but lacks the Generator distillation, the utility-weighted curriculum selection, the zone-adaptive triage, and the self-verification protocol. Its near-flat scaling (+3.4 points across three stages) and initial regression on several benchmarks serve as evidence that the Teacher alone, without Socratic-Zero's quality control and co-evolutionary mechanisms, is insufficient for effective self-improvement. This is a critical point: the Teacher's raw generation capability (producing new problems from failures) does not automatically produce a useful curriculum — it must be filtered, weighted, and directed by the mechanisms Socratic-Zero adds.

Missing ablations: Several ablations that would strengthen the paper are not reported. There is no ablation on the number of seed problems (100 vs. 50 vs. 200) to assess whether the framework's performance is sensitive to seed quantity. There is no ablation on the historical replay ratio (25% fixed) to see if curriculum stability depends on this parameter. There is no ablation on the number of Solver attempts per problem (k = 8), which affects both the accuracy of success-rate estimates and the computational cost. There is no ablation on the Generator training frequency (every iteration vs. every N iterations) to assess whether more gradual distillation improves stability. There is no direct comparison of the Socratic-Generator to a version trained without utility weighting (i.e., standard SFT on Teacher outputs) — this would isolate the contribution of the Gaussian utility function to the Generator's downstream effectiveness.

Critical Assessment

Does the Solver evidence support the +20.2 point improvement claim?

The claim as stated: "Our Socratic-Solver-8B achieves an average gain of +20.2 percentage points over prior data synthesis methods across seven mathematical reasoning benchmarks" (Abstract, Figure 2(b)).

What was actually tested: Table 1 reports the +20.2 point gain as the difference between Socratic-Zero Stage 3 (56.1%) and the Static Augmentation baseline (40.7%). The Static Augmentation baseline is described as following "traditional approaches via MetaMath and WizardMath" — it augments the SFT training data with fixed synthetic questions generated offline from the initial seed problems, with no adaptive curriculum evolution. The +20.2 figure is therefore the gain over a specific, non-adaptive data synthesis method, not over "prior data synthesis methods" in general. The comparison to LLM2LLM (+15.2 points at Stage 3) is a more relevant comparison to a prior adaptive method, and the gain is smaller but still substantial.

Strengths of the evidence: The gain is consistent across all seven benchmarks (no benchmark shows degradation), is replicated across two additional model architectures (Table 2), and scales monotonically with training stages (Stage 1 < Stage 2 < Stage 3 for Socratic-Zero, unlike LLM2LLM which shows initial regression). The SFT baseline (35.9%) isolates the contribution of the initial supervised phase from the co-evolutionary gains, confirming that the majority of the improvement (+20.2 points beyond SFT) comes from the co-evolutionary process rather than simply from more training data.

Weaknesses and open questions: The Static Augmentation baseline may not represent the strongest possible non-adaptive data synthesis approach. MetaMath and WizardMath are specific methods with their own design choices; a more extensive sweep of data synthesis strategies (varying generation prompts, filtering criteria, difficulty distributions) might produce a stronger baseline. The comparison to LLM2LLM is more informative, but LLM2LLM was originally designed for a different setting and the paper's reimplementation may not be optimally tuned. The ablation in Table 6a was conducted on AIME-24 only, not on the full benchmark suite — it is possible that the no-SFT variant shows different relative performance on other benchmarks. Finally, the paper does not report the performance of a model trained on all Teacher-generated problems without utility weighting (i.e., an unfiltered version of the Socratic-Zero curriculum) — this would directly quantify the contribution of the quality control mechanisms to the final performance.

Does the Generator evidence support the claim of surpassing commercial SOTA models?

The claim as stated: "Synthetic data from our Socratic-Generator-32B enables student LLMs to achieve superior performance compared to other state-of-the-art commercial LLMs" (Abstract, Figure 2(a)).

What was actually tested: Table 5 reports the downstream utility (Mean@16 accuracy of a fine-tuned DeepSeek-R1-Distill-Llama-8B student) when trained on data from various generators. The Socratic-Generator-32B achieves 37.72%, compared to 37.63% for Claude-4.1-Opus, 37.20% for Gemini-2.5-Pro, 37.01% for Grok-4, and 36.62% each for GPT-5 and DeepSeek-V3.1. The margin over Claude-4.1-Opus is 0.09 points; over the Teacher Qwen3-235B, 0.59 points.

Strengths of the evidence: The evaluation protocol is well-standardized: all generators receive the same 1,000 SAND-Math seeds, produce the same number of variants (3,000 total, though only valid ones are used for fine-tuning), and the student model architecture and fine-tuning procedure are held constant. The validity rate metric (Table 4) provides convergent evidence — the Socratic-Generator is not simply producing easier-to-solve problems that inflate downstream utility through reduced difficulty. The improvement over the base Qwen3-32B (+2.75 points) isolates the gain from strategic distillation over raw model capability.

Weaknesses and open questions: The critical issue is the absence of confidence intervals or significance testing. With margins of 0.09–0.59 points over commercial models, and test sets like AIME-2024/2025 containing only 30 questions, the sampling variance in the Mean@16 metric could easily exceed these margins. Without standard deviations, the reader cannot determine whether "37.72% vs. 37.63%" represents a genuine superiority or a statistical tie. The paper claims the Generator "surpasses" these models, but the evidence supports a weaker claim: the Generator achieves roughly equivalent data quality to models 20× larger, which is impressive in its own right but does not require the stronger "superior" framing.

An additional concern: the student model used for evaluation (DeepSeek-R1-Distill-Llama-8B) is a specific architecture trained with a specific distillation procedure. It is possible that the Socratic-Generator's data is particularly well-suited to this student (e.g., the problem distribution matches the student's pretraining distribution better than the commercial models' data does), inflating the relative performance. Evaluating on multiple student architectures would address this. The validity rate comparison (Table 4) shows the Socratic-Generator (95.6%) slightly underperforming Claude-4.1-Opus (96.9%) and DeepSeek-V3.1 (96.5%), suggesting a possible quality-quantity tradeoff where the Socratic-Generator sacrifices some problem well-formedness for better difficulty calibration, or vice versa.

Does the co-evolutionary dynamics evidence support the framework's theoretical claims?

The claim as stated: The framework implements "a self-improving loop among three agents" where "the curriculum dynamically evolves to maintain optimal challenge levels for the Solver's current capabilities" (Section 3.1, Figure 3 caption).

What was actually tested: Tables 9 and 10 provide descriptive statistics on Solver reward and Generator reward distributions across stages. The oscillatory pattern in Table 9 (52.1% → 48.7% → 50.1%) and the stability in Table 10 (50.7% → 49.4% → 50.2%) are consistent with the claimed dynamics.

Strengths of the evidence: The data show the predicted qualitative pattern — initial difficulty escalation followed by Solver adaptation, with the Generator maintaining calibration near the target μ = 0.5. The Stage 1 → 2 decline in Solver mean reward is a nontrivial prediction of the co-evolutionary model: if the curriculum were static or adapting too slowly, Solver reward would monotonically increase (as the model overfits and masters the existing problems). The observed decline therefore provides evidence that the curriculum is genuinely evolving in difficulty.

Weaknesses and open questions: The evidence is descriptive and aggregate — mean reward across all problems in the curriculum — and does not directly show that the zone-adaptive mechanism (Mastered/Learning/Too-Difficult triage) is operating as designed. The paper does not report, for instance, how many problems are in each zone at each stage, how many problems migrate between zones, or what fraction of generated problems are excluded by the self-verification protocol. The analysis is also limited to three data points (Stages 1, 2, 3), which is insufficient to characterize whether the oscillations are converging to a stable equilibrium, diverging, or exhibiting limit-cycle behavior — any of which would have different implications for the framework's scalability to more stages. The paper's own discussion (Appendix M) acknowledges this as an open theoretical question: "The oscillatory convergence patterns documented in Table 9... suggest the system reaches dynamic equilibria rather than static optima." This is an honest characterization, but it also means the evidence for long-term stability beyond three stages is absent.

Missing experiments that would strengthen the paper

Scaling the number of seed problems. The framework starts from 100 seed problems. How does performance change with 50 seeds? With 200? If the framework genuinely bootstraps from minimal data, there should be a saturating curve where additional seeds provide diminishing returns beyond some threshold. If the gains require the full 100 seeds, the "minimal seed data" claim is weaker than stated.

Scaling the number of co-evolutionary stages. All results stop at Stage 3. Do the Solver gains continue, plateau, or reverse at Stage 4, 5, 10? The LLM2LLM baseline shows stagnation, and the Socratic-Zero's own cross-stage gains are uneven (Stage 1: −2.0 points vs. SA; Stage 2: +1.0 points; Stage 3: +15.4 points) — the large Stage 3 jump raises the question of whether this represents a genuine acceleration or a one-time threshold effect that would not be sustained.

Direct comparison to a "Teacher-only" variant. The most critical missing ablation is a version of Socratic-Zero where the Teacher generates curriculum in every iteration (i.e., no Generator, no distillation), using the same utility-weighted filtering and zone-adaptive triage as Socratic-Zero. This would directly quantify how much the Generator contributes to the final Solver performance versus simply providing computational efficiency. If the Teacher-only variant (which is more expensive per iteration) achieves similar Solver accuracy, the Generator's contribution is primarily economic (reducing Teacher inference cost). If the Teacher-only variant underperforms, the Generator's distillation provides a genuine quality improvement over the Teacher's raw outputs.

Generator evaluation on multiple student architectures. The downstream utility evaluation uses a single student model (DeepSeek-R1-Distill-Llama-8B). Evaluating on additional student architectures (e.g., Qwen3-8B, GLM4-9B, Llama-3-8B) would test whether the Generator's advantage generalizes or is specific to the DeepSeek distillation lineage.

Computational cost comparison. The paper does not report total GPU-hours for each method. While the Abstract claims Socratic-Zero operates "from minimal seed data," the computational cost is substantial: the Teacher runs on 16× AMD MI308X GPUs, Solver training on 8× NVIDIA H20 GPUs, and Generator training adds further cost. A comparison of total FLOPs or GPU-hours to achieve the reported accuracies versus simply training on a larger static dataset would contextualize the practical value of the co-evolutionary approach.

Summary of evidential support

The Solver performance gains over non-adaptive baselines (+15–20 points) are robust across benchmarks and model architectures, with the LLM2LLM comparison providing evidence that the specific co-evolutionary mechanisms (not just iterative retraining with a Teacher) drive the improvement. The Generator results are directionally impressive — a 32B model roughly matching 235B–671B models — but the claimed "superiority" over commercial models is not statistically supported given the small margins and absent confidence intervals. The co-evolutionary dynamics analysis is descriptive and preliminary, establishing that the curriculum adapts but not that the specific zone-adaptive and utility-weighting mechanisms are necessary for that adaptation. The complete reliance on the Qwen3 model family for the Teacher (235B), Generator initialization (32B), and primary Solver (8B, 14B) leaves open the question of whether the co-evolutionary dynamics depend on the Teacher and Generator sharing a pretraining distribution — would a GPT-based Teacher with a Qwen-based Solver produce similar gains? The absence of a Teacher-only ablation, scaling curves beyond three stages, and computational cost comparisons limits the reader's ability to assess whether the framework's complexity (three agents, utility functions, zone triage, self-verification) is justified relative to simpler alternatives.

6. Limitations and Trade-offs

Limitation 1: The Generator's Claimed Superiority Over Commercial Models Is Not Statistically Substantiated

The assumption or constraint. The paper's most prominent Generator claim — that Socratic-Generator-32B "achieves 37.72% downstream training effectiveness, outperforming leading commercial models including... Claude-4.1-Opus at 37.63%, Gemini-2.5-Pro at 37.20%... [and] GPT-5 at 36.62%" (Abstract) — is presented as a ranking of point estimates without any measure of uncertainty. The paper never reports standard deviations, confidence intervals, or significance tests for any of the downstream utility numbers in Table 5.

The consequence. The margins separating the Socratic-Generator from the commercial models in Table 5 are extremely narrow: +0.09 points over Claude-4.1-Opus, +0.52 points over Gemini-2.5-Pro, +0.59 points over the Teacher Qwen3-235B, and +1.10 points over GPT-5. The test sets used to compute these numbers — particularly AIME-2024 and AIME-2025, each containing only 30 questions — introduce substantial sampling variance. A difference of a single correct answer on AIME-2024 (out of 30 questions, with 16 decoding runs each) can shift the average by roughly (1/30)/7 ≈ 0.48 points across the seven-benchmark average, depending on the weighting. With margins of this magnitude being comparable to plausible sampling error, the claim of "superior performance" (Figure 2(a)) overstates the evidence. A more appropriate interpretation is that the Socratic-Generator achieves roughly equivalent data-generation quality to models over 20× larger — which is itself a striking result that does not require the statistically unsupported superiority framing.

What evidence exists in the paper. Table 5 reports the raw point estimates. Table 4 provides a convergent validity-rate comparison, where the Socratic-Generator (95.6%) slightly underperforms Claude-4.1-Opus (96.9%) and DeepSeek-V3.1 (96.5%) — suggesting a possible quality-quantity tradeoff. The Solver results in Tables 1 and 2 also lack uncertainty quantification (the Mean@32 metric averages over 32 decoding runs but reports only a single scalar per benchmark), meaning none of the paper's main claims about relative performance carry statistical confidence bounds. The paper does not acknowledge this as a limitation.

Mitigation status. Not addressed. The paper does not mention the absence of confidence intervals, does not report variance across decoding runs or fine-tuning seeds, and does not qualify its superiority claims with appropriate hedging. A straightforward fix — computing bootstrap confidence intervals on the benchmark averages, or reporting per-benchmark standard deviations across the 32 decoding runs — would substantially strengthen the evidential basis for the Generator comparison.

Limitation 2: All Co-Evolutionary Results Are From a Single Teacher-Generator-Solver Model Family Triad

The assumption or constraint. The entire Socratic-Zero framework is built and evaluated within a single model ecosystem: the Teacher is Qwen3-235B-A22B-Instruct-2507, the Generator is initialized from Qwen3-32B, and the primary Solver is Qwen3-8B-base (with cross-architecture tests on Qwen3-14B-base and GLM4-9B-base, but crucially with the same Qwen3-235B Teacher throughout). The paper shows cross-architecture generalization for the Solver (different base model families can serve as the learner), but never tests whether the framework works with a Teacher from a different model family (e.g., a GPT-based or Gemini-based Teacher evaluating a Qwen-based Solver).

The consequence. The Teacher's ability to accurately verify Solver solutions and to generate appropriately targeted problem refinements depends on the Teacher's understanding of the Solver's output distribution. If the Teacher and Solver share a pretraining lineage (as they do — Qwen3 family throughout), the Teacher may benefit from distributional alignment: it "understands" the kinds of errors the Solver makes because those errors occur in a linguistic and structural format the Teacher has been trained on. A GPT-based Teacher evaluating a Qwen-based Solver might misclassify correct-but-differently-formatted solutions as incorrect, or fail to generate problem variants that target the Solver's specific error patterns because the error patterns look unfamiliar. The paper provides no evidence on whether the co-evolutionary dynamics are robust to Teacher-Solver distribution shift, which is the realistic deployment scenario for most practitioners (who would likely use the strongest available API model as Teacher regardless of its pretraining lineage).

What evidence exists in the paper. The cross-architecture Solver results in Table 2 (GLM4-9B, Qwen3-14B) demonstrate that different model families can serve as the Solver, but in all cases the Teacher remains Qwen3-235B. The GLM4-9B results (+17.1 points over zero-shot after Stage 3) provide some indirect evidence that a Qwen-based Teacher can effectively guide a non-Qwen Solver, since GLM and Qwen are different model families. However, the Generator remains Qwen3-32B-initialized, and the paper does not test a non-Qwen Generator. The paper does not acknowledge Teacher-Solver distribution shift as a potential limitation.

Mitigation status. Not addressed. The paper does not discuss the possibility that the co-evolutionary dynamics depend on Teacher-Solver pretraining alignment, nor does it suggest experiments with cross-family Teachers (e.g., using GPT-5 or Claude-4.1-Opus as the Teacher for a Qwen3-8B Solver). Practitioners adopting Socratic-Zero would need to either: (a) use a Qwen3-235B Teacher (assuming access to the specific model and hardware), or (b) empirically verify that their chosen Teacher-Solver pair exhibits stable co-evolutionary dynamics, which the paper provides no principled way to predict.

Limitation 3: Mathematical Reasoning Only — No Evidence of Transfer to Qualitatively Different Reasoning Domains

The assumption or constraint. All training and primary evaluation in Socratic-Zero occurs within the domain of mathematical reasoning — problems with well-defined correctness criteria, answers expressible in standardized formats (numbers, algebraic expressions), and a limited set of reasoning patterns (algebraic manipulation, number theory, combinatorics, geometry). The Teacher's verification function V(q, y) → {0, 1} depends on the ability to objectively determine correctness via the MathRule extractor and LLM judge operating on the final answer (Appendix G). The Teacher's problem refinement function G(q, y_fail) → (q', y'_ref) depends on the Teacher's ability to analyze mathematical errors and produce valid mathematical variants.

The consequence. The framework's architecture makes strong domain assumptions that may not hold outside mathematical reasoning. In domains where correctness is ambiguous or multidimensional — code generation (where multiple functionally equivalent solutions exist), scientific reasoning (where answers may be probabilistic or open-ended), creative writing, or multi-step planning — the binary verification function V would need fundamental redesign. The Gaussian utility function (Equation 6) assumes a scalar success rate s_q' computable from binary outcomes; in domains where solution quality is graded rather than binary, the utility function would need a different form. The zone-adaptive triage (Mastered/Learning/Too Difficult) similarly assumes a clean success-rate threshold. The paper's transfer-to-general-reasoning experiment (Table 3) shows only modest gains (+6.02 points average across BBEH, MMLU-Pro, and SuperGPQA), and these benchmarks still contain substantial STEM content where mathematical thinking transfers naturally. Tasks requiring fundamentally different reasoning modalities — legal reasoning, ethical analysis, negotiation — have no analogue in the paper's evaluation.

What evidence exists in the paper. Table 3 provides the only evidence for domain transfer, and the gains are substantially smaller than the in-domain gains (+6.02 vs. +20.2 points). Within Table 3, the strongest transfer occurs on MMLU-Pro (+10.89 points), which includes substantial STEM content, while BBEH shows near-zero transfer (+1.86 points). Appendix K acknowledges the domain limitation optimistically: "The value function and curriculum evolution mechanisms developed in Socratic-Zero are domain-agnostic in principle, suggesting potential for broader applicability." However, this is presented as a direction for future work rather than an established property. The paper provides no experimental evidence in non-mathematical domains.

Mitigation status. Partially acknowledged but not addressed experimentally. Appendix K discusses domain transfer as future work: "Future work should investigate whether a Generator trained on mathematical problems can effectively create challenging problems in adjacent domains like physics or computer science, potentially through few-shot adaptation or domain-specific fine-tuning." The paper does not propose concrete modifications to the framework that would enable operation in domains without clear binary correctness signals, and the current architecture's dependence on V and the success rate s_q' is not analyzed for domain-specificity.

Limitation 4: The Computational Cost of the Teacher Is Not Amortized Into the Headline Gains

The assumption or constraint. The paper frames Socratic-Zero as operating "from minimal seed data" (Abstract) and emphasizes data efficiency — bootstrapping from only 100 problems. However, the computational cost of running the Teacher model (Qwen3-235B-A22B-Instruct-2507) across the co-evolutionary loop is substantial and is never quantified, compared to baseline methods, or amortized into the efficiency claims. The Teacher runs on "16×AMD MI308X GPUs" (Appendix D), handles verification for k = 8 attempts per problem across an expanding curriculum, and generates new problems for every failure in F_t at each iteration. The Solver training itself uses "8×NVIDIA H20 GPUs" (Appendix D), and Generator training adds further cost.

The consequence. The +20.2 point Solver improvement and the Generator's downstream utility of 37.72% are achieved with significant computational investment that is not accounted for in any efficiency metric. A practitioner evaluating whether to adopt Socratic-Zero needs to answer: would the same GPU-hours spent on a larger static dataset (e.g., simply generating more problems from the Teacher upfront without the co-evolutionary loop) achieve comparable results? The paper's own Static Augmentation baseline represents a lower-bound estimate — it uses "fixed synthetic questions generated offline" but doesn't specify how many questions or how much Teacher compute went into generating them. If Static Augmentation used far less Teacher compute than Socratic-Zero's three stages, the comparison is not FLOPs-matched, and the +20.2 point gap conflates the algorithmic benefit of co-evolution with the benefit of simply using more Teacher-generated data. Conversely, if the Teacher is the dominant cost and Socratic-Zero uses the Teacher more efficiently (by targeting generation to high-utility failures), the paper would benefit from showing this explicitly.

What evidence exists in the paper. The paper describes infrastructure (Appendices B and D) but never reports total GPU-hours, Teacher inference calls per stage, or any cost-normalized comparison. The LLM2LLM baseline (Table 1) also uses the Teacher for verification and generation but without the Generator or utility weighting — its poor scaling suggests that simply using the Teacher more times (LLM2LLM does so at each iteration) is not sufficient for improvement, which indirectly supports Socratic-Zero's efficiency. However, without knowing the relative Teacher-inference budgets, this comparison remains qualitative. The Generator is explicitly motivated by reducing Teacher dependence ("without perpetual reliance on the expensive Teacher," Section 3.1), which implicitly acknowledges the Teacher as a cost bottleneck, but the tradeoff is never quantified in FLOPs or GPU-hours.

Mitigation status. Partially addressed by the Generator's existence — the Generator is designed to eventually replace the Teacher for curriculum generation, reducing long-term cost. However, the paper's main Solver results (Tables 1 and 2) come from a system where the Teacher is used at every stage for both verification and generation, meaning the cost savings from the Generator are not realized during the same training run that produces the headline accuracy numbers. The paper does not report a version of the Solver trained using only the Generator for curriculum expansion (i.e., Teacher used for verification only, Generator for problem generation) — this would directly demonstrate cost savings without sacrificing accuracy. The suggestion of future work on "amortizing difficulty estimation cost" is analogous but not identical; here the issue is the Teacher's generation and verification cost, not difficulty estimation per se.

Limitation 5: Performance Plateaus at Three Stages and the Hardest Problems Remain Near-Unsolvable

The assumption or constraint. The paper stops co-evolutionary training after three stages for all experiments. The stage-wise Solver gains in Table 1 are uneven: Stage 1 yields −2.0 points relative to Static Augmentation (the Solver's average accuracy is actually below the non-adaptive baseline after one co-evolutionary iteration), Stage 2 yields +1.0 points, and Stage 3 yields a dramatic +15.4 points. This trajectory — lag, catch-up, then a large jump — raises questions about what happens at Stage 4 and beyond. Does the gain accelerate further? Plateau? Reverse due to curriculum collapse or verifier over-optimization (as observed in the test-time compute scaling literature with PRM over-optimization at high budgets)? The paper provides no evidence.

Simultaneously, the Solver's performance on the hardest problems remains extremely low. On AIME-2024 (Table 1), even after Stage 3 the Socratic-Solver-8B achieves only 28.4% accuracy — the model fails to solve approximately 72% of these competition-level problems despite three rounds of targeted co-evolutionary training. On AIME-2025, the figure is 24.6%. On Olympiad, 55.1% — still leaving nearly half the problems incorrect.

The consequence. The combination of an accelerating Stage 3 gain and absent Stage 4+ data creates uncertainty about the framework's scaling trajectory. If the Stage 3 jump represents a threshold effect — the Solver crossing some capability boundary where accumulated skills suddenly compound — then further stages might yield continued large gains. If it represents an anomaly (e.g., the Stage 3 curriculum happened to align unusually well with the evaluation benchmarks), then Stage 4 might show plateau or regression. The LLM2LLM baseline's flat scaling (+3.4 points across all three stages) demonstrates that iterative retraining alone does not guarantee continued improvement, and the field has documented cases of self-improvement collapse (the paper's own Appendix K notes ReST^EM-trained models degrading). Without Stage 4+ data, the reader cannot assess whether Socratic-Zero has found a stable improvement trajectory or is merely delayed in reaching the same plateau that LLM2LLM hit early.

The near-zero performance on the hardest competition problems mirrors a finding from prior work on test-time compute scaling (see the reference example paper in the prompt, Section 5): that learning-based approaches amplify existing capability but do not create it from nothing. If the base Solver's pass@1 on AIME-level problems is near zero, the co-evolutionary process may be fundamentally unable to bridge the gap because there are too few correct trajectories to form informative preference pairs — the reference solution becomes the only winning example (enforced by Equation 4's fallback), and the preference comparison between the reference and the Solver's incorrect attempts may provide insufficient gradient signal. This is the same boundary condition observed in the test-time compute scaling literature for difficulty bin 5 problems.

What evidence exists in the paper. Table 1 shows the AIME-24 and AIME-25 scores (28.4% and 24.6% at Stage 3) as well as Olympiad (55.1%). The zero-shot base model scores are 5.1% and 4.2% on AIME-24 and AIME-25 respectively — while Socratic-Zero improves these substantially (roughly 5× on AIME-24), the absolute performance remains low. The cross-architecture results (Table 2) show the same pattern: GLM4-9B reaches 31.1% on AIME-24 at Stage 3; Qwen3-14B reaches 30.1%. All three architectures converge to roughly 25–31% on AIME-24 after three stages, suggesting a ceiling that may be inherent to the 8–14B parameter scale or to the co-evolutionary approach itself — distinguishing between these requires data from further stages, which the paper does not provide. The paper's discussion of convergence (Appendix M) acknowledges this as an open question: "Future theoretical work should investigate conditions under which the system exhibits stable convergence versus chaotic dynamics."

Mitigation status. Not addressed experimentally. The paper acknowledges the theoretical uncertainty about long-term convergence in Appendix M but does not run additional stages to provide empirical evidence. The Appendix M discussion is framed as a call for "theoretical development" rather than as a limitation of the current empirical results. The paper does not characterize which specific problem types or difficulty levels remain unsolved, nor does it analyze whether the failures are "near-misses" (incorrect but with partially correct reasoning) or "complete failures" (no coherent approach) — this analysis would help distinguish between a ceiling that further co-evolution might breach and one that is fundamental to the base model's capability envelope.

Limitation 6: The Generator Evaluation Protocol Relies on a Single Student Model and May Overfit to the Distillation Lineage

The assumption or constraint. The downstream utility metric in Table 5 is measured by fine-tuning exactly one student model: DeepSeek-R1-Distill-Llama-8B (DeepSeek-AI et al., 2025a). The Generator evaluation protocol (Section 4.3.1) states: "We used all valid question-answer (QA) pairs to fine-tune the student model, DeepSeek-R1-Distill-Llama-8B." The choice of this specific student architecture is justified by the model being a standard distillation of a strong reasoning model (DeepSeek-R1), but no alternative student architectures are tested.

The consequence. The Generator's training process (WSFT) produces a problem-generation policy that is optimized relative to a specific Solver (Qwen3-8B with Socratic-Zero training) as evaluated by a specific Teacher (Qwen3-235B). The downstream evaluation then tests these problems on an entirely different model (DeepSeek-R1-Distill-Llama-8B) with a different architecture, different pretraining data, and different reasoning style. If the Socratic-Generator's problems happen to be well-aligned with the DeepSeek distillation lineage — for instance, if the problem formats, difficulty distributions, or mathematical conventions match what the student model encountered during its R1 distillation — the downstream utility metric may overstate the Generator's generalizability. A Generator that performs well for DeepSeek-R1-distilled students might perform poorly for Qwen-based students, or for base models fine-tuned from scratch, or for models using different reasoning paradigms (e.g., chain-of-thought vs. structured proof formats). The paper's claim that the Generator "enables student LLMs to achieve superior performance" (Abstract, emphasis added) implies generalizability across student architectures that the single-student evaluation does not support.

This concern is amplified by the fact that the Socratic-Generator was trained using a Gaussian utility function with μ = 0.5 — problems at exactly the 50% success-rate frontier for Qwen3-8B at a specific training stage. These problems are not necessarily the optimal training distribution for a different model at a different capability level. The transfer of "optimal difficulty calibration" from one model to another is not guaranteed.

What evidence exists in the paper. The evaluation protocol in Section 4.3.1 specifies DeepSeek-R1-Distill-Llama-8B as the sole student model. Table 5 reports the downstream utility for this specific student across all generators (Socratic-Generator-32B, Teacher, commercial models). The paper reports cross-architecture Solver results in Table 2 (using Qwen3-8B, Qwen3-14B, and GLM4-9B as Solvers within the Socratic-Zero framework), but this tests the Solver's ability to learn from the co-evolutionary process, not the Generator's ability to produce broadly useful training data. The paper does not fine-tune any non-DeepSeek student on the Socratic-Generator's data and does not report student-architecture robustness for the Generator evaluation.

Mitigation status. Not addressed. The paper does not discuss the choice of student model as a potential confound, does not report evaluation on multiple student architectures, and does not qualify the Generator superiority claims with respect to student-model specificity. The most straightforward mitigation — fine-tuning 2–3 additional student architectures (e.g., Qwen3-8B, GLM4-9B, Llama-3-8B) on the Socratic-Generator's data and confirming that the relative advantage over other generators persists — would directly address this concern and is not computationally prohibitive given that the Generator evaluation already involves fine-tuning runs.

7. Implications and Future Directions

How This Work Changes the Landscape

Socratic-Zero represents a methodological reframing rather than a paradigm shift. It does not introduce a fundamentally new learning algorithm (DPO and supervised fine-tuning are established techniques) or a new model architecture. Instead, it reorganizes existing components — a verifier, a generator, a preference learner — into a closed-loop architecture where curriculum and capability co-evolve, and it provides empirical evidence that this reorganization yields substantially better data efficiency and final performance than either static datasets or prior iterative approaches that lack the co-evolutionary quality-control mechanisms.

The shift is from thinking about data generation and model training as sequential, decoupled stages (generate data → freeze it → train on it → repeat) to thinking about them as a coupled dynamical system where the data-generation process adapts continuously to the learner's evolving frontier. This is conceptually distinct from even the most advanced prior iterative self-training methods like LLM2LLM and R-Zero. Those methods implemented feedback loops where the teacher adapts to student failures, but the adaptation was reactive and undirected — all failures spawned new problems, regardless of whether those failures were informative or pathological. Socratic-Zero adds a stabilizing attractor (the Gaussian utility function at μ = 0.5) and a safety constraint (zone-adaptive triage that excludes the Too Difficult zone) that together keep the curriculum anchored at the frontier of learnable difficulty. The LLM2LLM baseline's stagnation at +3.4 points across three stages (Table 1), versus Socratic-Zero's acceleration to +15.4 points by Stage 3, provides the headline empirical evidence that feedback alone is insufficient — the feedback must be directed by a quality metric and constrained by a difficulty ceiling.

This reframing resolves a latent contradiction in the self-improvement literature. On one side, methods like Absolute Zero and R-Zero demonstrated that fully autonomous self-play can improve reasoning, supporting the proposition that models can teach themselves. On the other side, LLM2LLM showed that iterative retraining on teacher-generated data from failures can sometimes degrade performance (as it does on several benchmarks at Stage 1 in Table 1), supporting the counter-proposition that self-generated curricula are noisy and potentially harmful. Socratic-Zero's reconciliation is that both observations are correct, but they describe different operating regimes of the same underlying system. When curriculum generation is undirected (all failures → new problems), the system eventually fills with low-quality, too-difficult, or flawed problems, and performance stagnates or regresses. When curriculum generation is quality-controlled (utility-weighted, zone-adaptive, self-verified), the system maintains a productive learning signal, and performance improves continuously. The Socratic-Zero framework thus provides the boundary conditions under which self-improvement works, converting a binary debate ("does self-play work?") into an engineering question ("under what quality-control constraints does self-play work?").

The work also redirects research attention in a specific way. The Generator component demonstrates that a relatively small model (32B) can be trained to produce curriculum that matches or slightly exceeds the data-generation quality of models over 20× larger (Table 5). This suggests that curriculum design is a learnable skill that can be distilled from a powerful oracle into a much cheaper specialized model, analogous to how reasoning capabilities can be distilled from large to small models. This opens the possibility that the field's focus on scaling up teacher models for data generation (using GPT-5, Gemini-2.5-Pro, or Claude-4.1-Opus as data factories) might be partially replaced by a phase of training a specialized curriculum generator that then operates at a fraction of the inference cost. If the Generator's advantage over its Teacher (37.72% vs. 37.13% downstream utility) is robust, it implies that strategic distillation can produce a generator that is not just cheaper but better than its teacher at the specific task of curriculum design — the distillation process filters out the teacher's poorly-calibrated outputs, producing a refined policy.

Which research directions become more attractive: The finding that co-evolutionary dynamics can be stabilized by a continuous utility function (the Gaussian) and a zone-based triage mechanism makes it attractive to explore other self-improving systems where the learner's relationship to training data can be quantified similarly — code generation (where unit test pass rates provide a natural success metric), formal theorem proving (where proof verification provides binary feedback), or any domain with automated correctness oracles. The Generator's success at matching much larger models makes specialized small-model distillation for data generation an attractive alternative to perpetual reliance on API-sized models for synthetic data pipelines.

Which become less attractive: The ablation on initial SFT (Table 6a: only +2.34 points gain without SFT vs. +18.38 with SFT) should temper enthusiasm for "zero-shot self-improvement" — the idea that a base model with no task-specific fine-tuning can bootstrap to strong performance purely through self-play. The evidence strongly suggests there is a minimum capability threshold below which self-improvement dynamics are weak or absent. Research that proposes to bootstrap reasoning from scratch (no supervised initialization) must contend with this finding. Similarly, the LLM2LLM baseline's stagnation makes unfiltered, generate-from-all-failures approaches less attractive — the paper's evidence suggests that quality control is not optional but essential for stable self-improvement.

Follow-Up Research This Work Enables

Characterize the minimum capability threshold for co-evolutionary self-improvement to engage. The ablation in Table 6a shows that without initial SFT, Socratic-Zero yields near-zero gains (+2.34 points on AIME-24). But the paper uses a single SFT initialization (1,500 Level-5 problems). A systematic study would sweep SFT dataset size and difficulty (e.g., 0, 100, 500, 1,500, 5,000 problems; Levels 1–5) and measure the resulting co-evolutionary gain after a fixed number of stages. The prediction: there exists a threshold SFT performance level below which DPO gradients are too weak to drive improvement (because the Solver produces too few correct trajectories) and above which co-evolution takes off. Identifying this threshold as a function of base model size and domain difficulty would provide an engineering guideline for when Socratic-Zero is applicable to a new domain without requiring the full ablation to be re-run. The experiment would use the same infrastructure as the paper (Qwen3-8B, same Teacher) and report co-evolutionary gain vs. SFT performance on a held-out benchmark.

Ablate the utility function's role by comparing Teacher-only filtered vs. unfiltered curriculum. The paper never trains a Solver on the full, unfiltered Teacher-generated curriculum (i.e., using all of D_new at each stage without utility weighting and without zone-based exclusion). This is the most critical missing ablation: it would isolate the contribution of the quality-control mechanisms (Gaussian utility + zone triage + self-verification) from the contribution of simply having more Teacher-generated data. The experiment would run Socratic-Zero identically to the paper but with the Generator replaced by the Teacher (no distillation), and with two variants: (a) all Teacher-generated problems included regardless of Solver success rate, and (b) only zone-filtered problems included (Learning + Mastered zones, no Too Difficult). Comparing both to the full Socratic-Zero pipeline would decompose the gain into: curriculum expansion alone, zone-based filtering, and utility-weighted distillation. The paper's LLM2LLM baseline approximates variant (a) but uses a different training protocol; a direct within-framework ablation would be cleaner.

Test the Generator's cross-architecture generalizability by evaluating on 3+ diverse student models. The paper's Generator downstream evaluation (Table 5) uses exactly one student architecture: DeepSeek-R1-Distill-Llama-8B. The claim that the Socratic-Generator "enables student LLMs to achieve superior performance" (Abstract, emphasis on the plural) requires testing on multiple student architectures that differ in pretraining distribution, scale, and reasoning style. A strong follow-up would fine-tune Qwen3-8B-base, GLM4-9B-base, Llama-3-8B-base, and DeepSeek-R1-Distill-Qwen-7B on the Socratic-Generator's data and on data from the same commercial generators used in Table 5 (GPT-5, Gemini-2.5-Pro, Claude-4.1-Opus). If the Socratic-Generator's relative advantage persists across all student architectures, the "superior data generation" claim is robust. If the advantage is specific to DeepSeek-distilled students, it reveals a distribution-matching confound. The experiment would use the identical evaluation protocol from Section 4.3.1 and report downstream utility with per-student confidence intervals (bootstrapped across fine-tuning runs).

Test co-evolution beyond three stages to distinguish threshold effects from sustained scaling. The paper's Stage 3 jump (+15.4 points over Static Augmentation, from a Stage 2 advantage of only +1.0 points) is the largest single-stage gain and is not well-explained by the co-evolutionary model — is it a threshold effect (the Solver crossing a capability boundary where accumulated skills compound) or an artifact of the specific Stage 3 curriculum aligning unusually well with the evaluation benchmarks? Running Stages 4, 5, and 6 would reveal whether the gain trajectory accelerates, plateaus, or oscillates. The prediction from the paper's own convergence discussion (Appendix M) is that bounded oscillations around a dynamic equilibrium will emerge, but this has not been demonstrated. If Stage 4 shows a decline (as Stage 1 → 2 did for the Solver mean reward in Table 9), it would support the oscillatory model and suggest that the optimal stopping point is not "as many stages as possible" but rather a specific point in the oscillation cycle. This experiment requires no architectural changes, only additional training stages and evaluation.

Extend the framework to a domain with non-binary feedback (e.g., code generation with partial credit). The paper's entire quality-control apparatus — the Gaussian utility function, zone-adaptive triage, and DPO preference pairs — assumes binary correctness feedback V(q,y) → {0,1}. Many important domains lack clean binary oracles: code generation (functional correctness is binary via unit tests, but code quality, efficiency, and style are graded), scientific reasoning (answers may be probabilistic or have multiple valid approaches), and open-ended generation. A natural extension would adapt the utility function to accept graded feedback: U(q'|π_θS) = f(score(q', π_θS)) where score is no longer an empirical success rate but a continuous quality metric (e.g., average unit test coverage, BLEU against a reference set, or a learned reward model score). The experiment would apply Socratic-Zero to a code generation benchmark (HumanEval or MBPP) using a teacher that provides both binary correctness (pass/fail on test cases) and continuous quality scores, comparing a Gaussian-on-binary utility to a Gaussian-on-continuous-score utility. The question is whether the zone of proximal development concept — targeting problems at the 50th percentile of performance — generalizes from binary to graded feedback, and what the analogue of μ = 0.5 is for continuous metrics.

Apply Socratic-Zero with a cross-family Teacher to test distribution-shift robustness. All experiments use a Qwen3-235B Teacher with Qwen3-family or GLM4 Solvers. A critical robustness test is whether the co-evolutionary dynamics survive when the Teacher does not share a pretraining lineage with the Solver — the realistic scenario where a practitioner uses the strongest available API model as Teacher regardless of the Solver's architecture. The experiment would replace Qwen3-235B with GPT-5, Gemini-2.5-Pro, or Claude-4.1-Opus as the Teacher (all available via API) while keeping the Solver as Qwen3-8B and the Generator initialized from Qwen3-32B. If the Solver shows comparable co-evolutionary gains with a cross-family Teacher, the framework's reliance on Qwen3-235B specifically is not a limitation. If gains degrade (e.g., due to the cross-family Teacher misclassifying differently-formatted correct answers or generating problem variants in a style the Qwen Solver cannot parse), it reveals a domain-adaptation requirement for the Teacher-Solver interface that practitioners would need to address. The experiment would report Solver accuracy across stages comparing the Qwen3-235B Teacher to at least two cross-family Teachers, with qualitative analysis of verification disagreements and problem-generation style differences.

Practical Applications and Downstream Use Cases

Cost-efficient training data generation for specialized reasoning models. The most direct application of Socratic-Zero is replacing expensive, recurring API calls to large commercial models with a trained Generator for synthetic data pipelines. The numbers from Table 5: using the Socratic-Generator-32B instead of GPT-5 for generating training data would save roughly (assuming GPT-5 is a 671B+ parameter model, and Qwen3-32B is 32B parameters) approximately 20× in inference FLOPs per generated problem, while achieving slightly higher downstream utility (37.72% vs. 36.62%). For an organization generating millions of training problems for mathematical reasoning fine-tuning, replacing a GPT-5-based pipeline with a Socratic-Generator-32B deployed on 8×H20 GPUs (the paper's Solver training hardware, Appendix D) could reduce inference costs by an order of magnitude while maintaining or slightly improving data quality. The Generator's 95.6% validity rate (Table 4) means only ~4.4% of generated problems are discarded, keeping the pipeline efficient. This use case does not require running the full co-evolutionary loop in deployment — only the trained Generator, which produces problems autoregressively from seed inputs without needing the Teacher at inference time.

Curriculum design for adaptive learning platforms. The Gaussian utility function U(q'|π_θS) = exp(−(s_q' − 0.5)²/(2 × 0.2²)) provides a computable metric for whether a problem is at the optimal difficulty frontier for a given learner. In educational technology, adaptive learning platforms currently rely on coarse difficulty labels (easy/medium/hard) or item response theory models calibrated on student populations. Socratic-Zero's utility function could be deployed as a real-time difficulty calibrator: as a student works through problems, the system estimates the student's current success rate on candidate next problems (by simulating the student model if available, or by querying a learned difficulty estimator), and selects problems that maximize the utility function — targeting the 50% success-rate frontier where learning is theorized to be most efficient. The zone-adaptive triage (Mastered/Learning/Too Difficult) provides the curriculum management logic. The paper's evidence that μ = 0.5 is empirically optimal for model training (Table 6b) provides initial validation, though transfer to human learners would require human-subject studies. The Generator could be fine-tuned on a corpus of student interaction data to produce personalized problem variants, analogous to how the Socratic-Generator learned to produce problems targeting the Solver's specific weaknesses.

Bootstrapping domain-specific reasoning in low-resource scientific or engineering domains. The framework's key selling point — bootstrapping from minimal seed data — is most valuable in specialized domains where large curated reasoning datasets do not exist. A research lab studying protein folding, semiconductor design, or climate modeling could apply Socratic-Zero by: (1) collecting 100 seed problems from the domain literature (with verified solutions), (2) using a strong general-purpose model (e.g., GPT-5 or Claude-4.1-Opus) as the Teacher to verify domain-specific solutions and generate novel problem variants, (3) training a domain-specialized Solver via the co-evolutionary loop. The paper's cross-architecture results (Table 2: consistent gains on GLM4 and Qwen3 families) suggest the framework is robust to the choice of base Solver architecture, meaning it could be applied to domain-adapted base models. The key requirement is that the Teacher be capable of verifying correctness in the domain — for domains with objective verification criteria (correct protein folding prediction, correct circuit behavior), this is feasible. For domains where even expert verification is ambiguous, the binary-feedback assumption would need the graded-feedback extension described above. The 100-seed starting point (Table 8) provides a template for seed selection: balance across subdomains, filter for problems with 10–70% initial success rate, and verify reference solution quality with the Teacher. A lab with access to a strong API model and modest GPU resources (8×H20, per the paper's Solver training setup) could deploy this within weeks for a new domain, assuming the Teacher can operate reliably as a verifier.

When to Prefer This Method

The paper does not explicitly articulate a decision rule comparing Socratic-Zero against named alternatives. It positions against Static Augmentation and LLM2LLM as baselines rather than as points in a tradeoff space where different conditions favor different methods. The ablation in Table 6a establishes one clear boundary condition — Socratic-Zero requires initial SFT to be effective — but this is a precondition for using the method, not a choice between this method and another. The Generator's advantage over commercial models (Table 5) is presented as a one-sided superiority claim rather than as a tradeoff with specific conditions. Therefore, a structured decision matrix comparing Socratic-Zero against alternatives across problem characteristics, compute budgets, or domain properties would be extrapolating beyond what the paper's experiments support, and is not included.